TrustedUser
2,147
edits
Difference between revisions of "HOWTO-Reverse Engineering"
Jump to navigation
Jump to search
m
HOWTO-Reverse Engineering (view source)
Revision as of 15:14, 25 October 2018
, 15:14, 25 October 2018Text replacement - "</source>" to "</syntaxhighlight>"
Dreammaster (talk | contribs) m (Added a miscellaneous section) |
m (Text replacement - "</source>" to "</syntaxhighlight>") |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 3: | Line 3: | ||
== Resources == | == Resources == | ||
[ | [https://www.scummvm.org/frs/extras/IDA/idafree50.exe IDA Freeware Version 5.0] - IDA is the preferred tool for disassembling old games from scratch. The most recent freeware version no longer supports disassembling DOS games, but this earlier version still supports it. | ||
IDA is | |||
[http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | [http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | ||
| Line 26: | Line 25: | ||
[https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | [https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | ||
Nice tool for a easy writing and injecting hooks to game binaries. Useful for watching how code executes, when are internal functions called, for dumping structures from memory of target process and for changing data in memory on the fly. Works for newer binaries of games (32bit and windows xp) | Nice tool for a easy writing and injecting hooks to game binaries. Useful for watching how code executes, when are internal functions called, for dumping structures from memory of target process and for changing data in memory on the fly. Works for newer binaries of games (32bit and windows xp) | ||
[http://www.bttr-software.de/products/insight/ Insight - real-mode DOS debugger] | |||
May prove useful as an alternative to the DosBox debugger | |||
== Using the DosBox Debugger == | == Using the DosBox Debugger == | ||
| Line 62: | Line 64: | ||
=== Naming Methods === | === Naming Methods === | ||
Methods can be renamed using the general 'N' hotkey (as well as via the menus), and the 'Y' can be used to specify a C-like prototype for a method. This is particularly useful when some of the parameters for a method are passed using registers. By explicitly documenting what the method expects, it makes it easier to remember later on when you're reversing methods that call it. Standard methods where parameters are passed via the stack are easy, since IDA can automatically set up the function prototype for you. If a method does have parameters passed in registers, prototypes like the below can be used: | Methods can be renamed using the general 'N' hotkey (as well as via the menus), and the 'Y' can be used to specify a C-like prototype for a method. This is particularly useful when some of the parameters for a method are passed using registers. By explicitly documenting what the method expects, it makes it easier to remember later on when you're reversing methods that call it. Standard methods where parameters are passed via the stack are easy, since IDA can automatically set up the function prototype for you. If a method does have parameters passed in registers, prototypes like the below can be used: | ||
< | <syntaxhighlight lang="c"> | ||
int __usercall sub_100FB<ax>(__int8 param1<al>, int param2<bx>) | int __usercall sub_100FB<ax>(__int8 param1<al>, int param2<bx>) | ||
</ | </syntaxhighlight> | ||
In this case, the method takes an 8-bit parameter in the al register, and another 16-bit value in bx, then returns a result in ax | In this case, the method takes an 8-bit parameter in the al register, and another 16-bit value in bx, then returns a result in ax | ||
| Line 72: | Line 74: | ||
When dealing with data, you'll frequently see cases like | When dealing with data, you'll frequently see cases like | ||
< | <syntaxhighlight lang="asm"> | ||
mov bx, 30h | mov bx, 30h | ||
mul bx | mul bx | ||
mov ax, [bx+2D00h] | mov ax, [bx+2D00h] | ||
</ | </syntaxhighlight> | ||
In this case, an initial index in the ax register is multiplied by 30h (30 hexadecimal = 48 decimal). So from this we can determine that the given structure is 48 bytes in size, and can create a new structure accordingly. For smaller sized structures, you may want to create as many 2 byte word fields as needed to make up the correct size for the structure. For larger sizes, the easiest way is to simply declare an array of the needed structure size - 1, and follow it with a single byte field. You can then delete/undefine the array. The remaining byte will keep the structure at the correct size, and you can then later fill in the fields as you find references to them. | In this case, an initial index in the ax register is multiplied by 30h (30 hexadecimal = 48 decimal). So from this we can determine that the given structure is 48 bytes in size, and can create a new structure accordingly. For smaller sized structures, you may want to create as many 2 byte word fields as needed to make up the correct size for the structure. For larger sizes, the easiest way is to simply declare an array of the needed structure size - 1, and follow it with a single byte field. You can then delete/undefine the array. The remaining byte will keep the structure at the correct size, and you can then later fill in the fields as you find references to them. | ||