265
edits
Difference between revisions of "HOWTO-Reverse Engineering"
Jump to navigation
Jump to search
m
Mention IDA Free 5.0 must be run in Administrator mode.
Dreammaster (talk | contribs) m (Remap IDA link to locally hosted version that supports disassembling DOS executables) |
Dreammaster (talk | contribs) m (Mention IDA Free 5.0 must be run in Administrator mode.) |
||
| (8 intermediate revisions by 5 users not shown) | |||
| Line 3: | Line 3: | ||
== Resources == | == Resources == | ||
[https:// | [https://downloads.scummvm.org/frs/extras/IDA/idafree50.exe IDA Freeware Version 5.0] - IDA is the preferred tool for disassembling old games from scratch. The most recent freeware version no longer supports disassembling DOS games, but this earlier version still supports it. Note: In more recent Windows versions, you will need to run it in Administrator mode, or it will error out on startup. | ||
[https://ghidra-sre.org Ghidra] Ghidra is an open source alternative to IDA that can be used for disassembling old games. It is not as mature as IDA and is missing some features, but it has a nice decompiler. | |||
[http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | [http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | ||
The DosBox Debugger is an invaluable tool for running old DOS games, to monitor how the program executes, and what values are generated by the executing code. | The DosBox Debugger is an invaluable tool for running old DOS games, to monitor how the program executes, and what values are generated by the executing code. | ||
[https://imhex.werwolv.net/ ImHex Open Source Hex Editor] | |||
A powerful and flexible hex editor for all OSes and the Web. | |||
[http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm XVI32 Hex File Viewer] | [http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm XVI32 Hex File Viewer] | ||
A very primitive hex editor compared to ImHex, but may be useful if you just need a quick and simple way to view a file's contents and make changes. | |||
[http://www.ctyme.com/rbrown.htm Ralf Brown's Interrupt List] | [http://www.ctyme.com/rbrown.htm Ralf Brown's Interrupt List] | ||
| Line 18: | Line 23: | ||
[http://beginners.re/ "Reverse Engineering for Beginners" free book] | [http://beginners.re/ "Reverse Engineering for Beginners" free book] | ||
This site has a free | This site has a free eBook that may be useful as a gentle introduction to reverse engineering techniques in general. | ||
[http://godbolt.org/ Compiler Explorer] | [http://godbolt.org/ Compiler Explorer] | ||
A pretty cool online tool that lets you paste in C code and shows you the compiled assembly under various different compilers. Useful if you're familiar with C, and | A pretty cool online tool that lets you paste in C code and shows you the compiled assembly under various different compilers. Useful if you're familiar with C, and want to see what kinds of assembly are produced for various different code fragments. | ||
[https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | [https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | ||
Nice tool for | Nice tool for easy writing and injecting hooks to game binaries. Useful for watching how code executes, check when the internal functions are called, for dumping structures from memory of a target process and for changing data in memory on the fly. Works for newer binaries of games (32bit and Windows XP). | ||
[http://www.bttr-software.de/products/insight/ Insight - real-mode DOS debugger] | |||
May prove useful as an alternative to the DosBox debugger. | |||
== Using the DosBox Debugger == | == Using the DosBox Debugger == | ||
| Line 61: | Line 69: | ||
=== Naming Methods === | === Naming Methods === | ||
Methods can be renamed using the general 'N' hotkey (as well as via the menus), and the 'Y' can be used to specify a C-like prototype for a method. This is particularly useful when some of the parameters for a method are passed using registers. By explicitly documenting what the method expects, it makes it easier to remember later on when you're reversing methods that call it. Standard methods where parameters are passed via the stack are easy, since IDA can automatically set up the function prototype for you. If a method does have parameters passed in registers, prototypes like the below can be used: | Methods can be renamed using the general 'N' hotkey (as well as via the menus), and the 'Y' can be used to specify a C-like prototype for a method. This is particularly useful when some of the parameters for a method are passed using registers. By explicitly documenting what the method expects, it makes it easier to remember later on when you're reversing methods that call it. Standard methods where parameters are passed via the stack are easy, since IDA can automatically set up the function prototype for you. If a method does have parameters passed in registers, prototypes like the below can be used: | ||
< | <syntaxhighlight lang="c"> | ||
int __usercall sub_100FB<ax>(__int8 param1<al>, int param2<bx>) | int __usercall sub_100FB<ax>(__int8 param1<al>, int param2<bx>) | ||
</ | </syntaxhighlight> | ||
In this case, the method takes an 8-bit parameter in the al register, and another 16-bit value in bx, then returns a result in ax | In this case, the method takes an 8-bit parameter in the al register, and another 16-bit value in bx, then returns a result in ax | ||
| Line 71: | Line 79: | ||
When dealing with data, you'll frequently see cases like | When dealing with data, you'll frequently see cases like | ||
< | <syntaxhighlight lang="asm"> | ||
mov bx, 30h | mov bx, 30h | ||
mul bx | mul bx | ||
mov ax, [bx+2D00h] | mov ax, [bx+2D00h] | ||
</ | </syntaxhighlight> | ||
In this case, an initial index in the ax register is multiplied by 30h (30 hexadecimal = 48 decimal). So from this we can determine that the given structure is 48 bytes in size, and can create a new structure accordingly. For smaller sized structures, you may want to create as many 2 byte word fields as needed to make up the correct size for the structure. For larger sizes, the easiest way is to simply declare an array of the needed structure size - 1, and follow it with a single byte field. You can then delete/undefine the array. The remaining byte will keep the structure at the correct size, and you can then later fill in the fields as you find references to them. | In this case, an initial index in the ax register is multiplied by 30h (30 hexadecimal = 48 decimal). So from this we can determine that the given structure is 48 bytes in size, and can create a new structure accordingly. For smaller sized structures, you may want to create as many 2 byte word fields as needed to make up the correct size for the structure. For larger sizes, the easiest way is to simply declare an array of the needed structure size - 1, and follow it with a single byte field. You can then delete/undefine the array. The remaining byte will keep the structure at the correct size, and you can then later fill in the fields as you find references to them. | ||
| Line 170: | Line 178: | ||
==== The Hotspot list ==== | ==== The Hotspot list ==== | ||
In adventure games, hotspots are areas of the screen that has an interactable item. Moving the mouse over the area causes a description of the hotspot. Using a breakpoint in the write string method, I was able to see what the caller was. Then examining the caller to find out how the description came to be passed, I was able to figure out the structure of how the list of hotspots were stored in memory. This allowed me to create a Hotspot structure, and set up the array of hotspots in memory. From there I was able to go in two direcitons. Firstly, by identifying other methods that access the same hotspot list, then finding out which one set up the values, I was able to locate the hotspot | In adventure games, hotspots are areas of the screen that has an interactable item. Moving the mouse over the area causes a description of the hotspot. Using a breakpoint in the write string method, I was able to see what the caller was. Then examining the caller to find out how the description came to be passed, I was able to figure out the structure of how the list of hotspots were stored in memory. This allowed me to create a Hotspot structure, and set up the array of hotspots in memory. From there I was able to go in two direcitons. Firstly, by identifying other methods that access the same hotspot list, then finding out which one set up the values, I was able to locate the hotspot loading code, which was part of the overall scene loading code. With scene loading identified, I could look at the other files being accessed, and the structures being loaded, and get further ideas of what information each scene contains. | ||
The other direction I was able to go in was hotspot interaction. By setting breakpoints in other methods that accessed the hotspot list and then trying to interact with a hotspot, I was able to identify the general method that handles actually doing item interactions. In this case, it turned out to use other fields of the Hotspot record, and then load a script and call it to be executed. Since I'd already identified the script execution method, it made it easier to realize that script data was being loaded, since it was immediately passed on to be executed. | The other direction I was able to go in was hotspot interaction. By setting breakpoints in other methods that accessed the hotspot list and then trying to interact with a hotspot, I was able to identify the general method that handles actually doing item interactions. In this case, it turned out to use other fields of the Hotspot record, and then load a script and call it to be executed. Since I'd already identified the script execution method, it made it easier to realize that script data was being loaded, since it was immediately passed on to be executed. | ||