How to make my program DEP-compatible?

by

DEP runs in one of two modes:

  1. Hardware DEP is for CPUs that can mark memory pages as non-executable. This helps to prevent certain exploits such as buffer overflows.

  2. Software DEP is for CPUs that do not have hardware DEP support. It doesn’t prevent execution of code in data pages, but instead stops SEH overwrite (another type of attack).

On Windows XP with CPUs that support it, hardware DEP is enabled by default only for certain Windows system binaries, and also for programs that choose to “opt-in”.

On Vista with CPUs that support it, hardware DEP is enabled by default for nearly all processes. This can occasionally be problematic, usually for older programs and drivers, and for ISVs that haven’t done any Vista testing.

So I suspect that the first step is to discover whether you’re dealing with software or hardware DEP. Also, are you using C#/VB or Managed C++? And are you using any native code or components? If your application uses a native component or an ActiveX control that was built using the old ATL framework, then it’s quite possible that your application will fail with hardware DEP.

Since .NET Framework 2.0 SP1, I believe that the C# compiler emits managed code which is DEP-compatible. But if your application is generating DEP exceptions, then you can try clearing the IMAGE_DLLCHARACTERISTICS_NX_COMPAT flag for your executable. To do this you use EDITBIN.EXE from the VC toolset like so:

editbin.exe /NXCOMPAT:NO <your binary>

If you’re using Visual Studio, you can add a post-build step to your executable’s project. You’ll need to setup the environment so that EDITBIN’s dependencies can be resolved. When I’m using native code as part of my app, the post-build step looks like this:

call $(DevEnvDir)..\tools\vsvars32.bat
editbin.exe /NXCOMPAT:NO $(TargetPath)

More Related Contents:

  1. How to use DoEvents() without being “evil”?
  2. Variables are not behaving as expected
  3. How can I run another application within a panel of my C# program?
  4. Windows batch script to parse CSV file and output a text file
  5. Using parameters in batch files at Windows command line
  6. How to create an infinite loop in Windows batch file?
  7. How to create SSH tunnel using PuTTY in Windows?
  8. How to force consistent line endings in Git commits with cross-platform compatibility
  9. IIS Manager in Windows 10
  10. Tracking CPU and Memory usage per process
  11. How can I get a list of all open named pipes in Windows?
  12. How do I configure Git to trust certificates from the Windows Certificate Store?
  13. What’s the nohup on Windows?
  14. Git Bash and Pageant are not using keys
  15. ‘node’ is not recognized as an internal or external command
  16. How to view DLL functions?
  17. Get Folder Size from Windows Command Line
  18. Delphi XE and Trapping Arrow Key with OnKeyDown
  19. Difference between %variable% and !variable! in batch file
  20. As a programmer, what do I need to worry about when moving to 64-bit windows?
  21. How can I programmatically refresh Windows Explorer?
  22. Why does ReadDirectoryChangesW omit events?
  23. How do I create drag-and-drop Strawberry Perl programs?
  24. Batch file to upload .txt to FTP
  25. Is it possible to use fzf (command line fuzzy finder) with windows 10 git-bash?
  26. what is a loader lock?
  27. Escape percent signs in given variables
  28. OpenSSL and multi-threads
  29. Bypassing elevation when launching a new process
  30. How can I run a command with administrator rights with Visual Studio code terminal?

Leave a Comment