Ok, so I’ve solved it but I’m not too sure how much information you actually want so I guess I will just split it into two spoiler tags
- What does the binary want from me?
A quick look into the binary tells you that it reads in 33 characters (32 actual characters and the last one as a terminator) from standard input within the main function and continues with some sort of mini virtual machine to process it.
The mini virtual machine reads in “instructions” that have an operation code that determines what the operation does and up to two parameters that determine how the operation is done (add 1,2 or 3 etc.)
push binary.1382120 ;"The VunMachine needs a password, begin with V:"
call dword ptr ds:[<&printf>]
call dword ptr ds:[<&_iob_func>]
push binary.138339C ;a buffer for the input
call dword ptr ds:[<&fgets>]
call binary.1381110 ;function handling the main VM loop
The mini virtual machines then executes “instructions” following a hard coded scheme and exits if a compare “instruction” returns false.
- What does the VM do?
The following “instructions” are executed within the mini VM:
Set first memory value to 31
start of loop:
Read input string character at the position of the first memory value to the second memory position
Compare if first memory value is 0: exit if they equal and says flag was right
Decrease content at first memory position by 1
Read input string character at the position of the first memory value to the third memory position
Xor the second memory value with the third memory value and save the result to the second memory position
Compare the second memory value to the character within the compare string (the string you found) at the position of the first memory value: exit if they are not equal
The code then jumps back to the loop start
Building the flag/password out of that and the hint that it has to start with V is really easy now. I guess I will leave the rest to you as you don’t need any assembly from now on.
Was fun to look into it, I hope this helps some how.