265
edits
Difference between revisions of "HOWTO-Reverse Engineering"
Jump to navigation
Jump to search
m
Remap IDA link to locally hosted version that supports disassembling DOS executables
Peterkohaut (talk | contribs) |
Dreammaster (talk | contribs) m (Remap IDA link to locally hosted version that supports disassembling DOS executables) |
||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 3: | Line 3: | ||
== Resources == | == Resources == | ||
[ | [https://www.scummvm.org/frs/extras/IDA/idafree50.exe IDA Freeware Version 5.0] - IDA is the preferred tool for disassembling old games from scratch. The most recent freeware version no longer supports disassembling DOS games, but this earlier version still supports it. | ||
IDA is | |||
[http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | [http://vogons.zetafleet.com/viewtopic.php?t=7323 DosBox Debugger] | ||
| Line 25: | Line 24: | ||
[https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | [https://www.frida.re/ FRIDA - Dynamic Instrumentation Framework] | ||
Nice tool for a easy writing and injecting hooks to game binaries. Useful for watching how code executes, when are internal functions called | Nice tool for a easy writing and injecting hooks to game binaries. Useful for watching how code executes, when are internal functions called, for dumping structures from memory of target process and for changing data in memory on the fly. Works for newer binaries of games (32bit and windows xp) | ||
== Using the DosBox Debugger == | == Using the DosBox Debugger == | ||
| Line 188: | Line 187: | ||
There's lot of areas you can proceed to delve in. Hopefully the suggestions above will help get you over the initial barrier of not being sure how to start, and assist you getting started in your efforts. | There's lot of areas you can proceed to delve in. Hopefully the suggestions above will help get you over the initial barrier of not being sure how to start, and assist you getting started in your efforts. | ||
== Extending/Logging DosBox == | |||
Sometimes you'll want to be able to compare the operation of the original in DosBox against your implementation in ScummVM. One of the easiest ways to do this is to set up debug(..) calls in ScummVM to output relevant information, and likewise hack the source code of DosBox to likewise output similar information in the equivalent method in the original. This way, you can compare the output of the two in any file comparison tool, like for example TortoiseGitMerge. To do this, the first step is for you to install and compile the DosBox source. You can find relevant guides on the DosBox website, and any help as needed in the Dosbox forums. | |||
Once you're able to compile the source, the next step is add in relevant debugging information when certain CS:IP locations are hit in the executable; if you've been reversing the game executable in IDA, you should be able to use it to figure out specific addresses for whatever method you're interested in. The appropriate place to add in logging information is in the CPU_Core_Normal_Run method's loop in core_normal.cpp. Here you can check for specific memory addresses having been reached, and output values from various registers. For example, considering the following example code fragment I once added: | |||
<pre> | |||
Bit32u segCS = SegValue(cs); | |||
if (segCS == 0x8a80) { | |||
Bit32u segIP = reg_eip; | |||
if (segIP == 0x2E9 || segIP == 0x307) | |||
fprintf(stderr, "-----------------------------\n"); | |||
else if (segIP == 0x3d0) { | |||
Bit16u ax = reg_eax; | |||
fprintf(stderr, "%.2x %.2x\n", ax & 0xff, ax >> 8); | |||
} | |||
} | |||
</pre> | |||
In this case, I was implementing the sound player from World of Xeen, and I needed to output the register/value pairs passed to a "write to adlib" method. So I added this fragment to print their values when the method was called, as well as an extra separator printed out whan the "playSound" method was called, to separate initialization writes that initialize the Adlib card from calls made after a sound effect call is made. | |||
When the changed Dosbox was compiled, I went to a command prompt. Presuming I'd changed the C drive to the Dosbox build directory, and the game in the current directory (on D drive), I used a command: | |||
<pre>c:dosbox 2> d:\temp\dosbox.txt</pre> | |||
The 2> causes a redirection of the stderr output to a file. Afterwards, I was able to compare the result with a similar output from ScummVM. With differences identified, I could set up a conditional breakpoint in ScummVM in the engine's adlib write method for the last call before wrong values started being written, and stepped through the code until I figured out what the engine was doing incorrectly. | |||
== Miscellaneous == | |||
This is a small section of miscellaneous tips and musings to help you as you reverse engineer games | |||
* Don't get too bogged down in the minutiae of what methods do.. it's sometimes best to just get a high level understanding and then implement your own code from scratch. For example, several games I've reversed had complicated logic as part of loading resources to allocate memory handles and reserve space in extended/expanded memory. Obviously this isn't something you need to worry about in your reimplementation anyway. It was enough to know the resource method returned a pointer to the data, and callers were responsible for calling a free method when they were done with it. So all the extra logic could be ignored in favor of simple C new byte[..] and delete[] calls. | |||
* In a similar vein, drawing code in EGA or earlier based games can be particularly labyrinthine. Since you'd be more likely to set up an 8-bit per pixel graphics surface when reimplementing such games, it may be simpler for you to experiment with your own implementation of drawing methods based on examining the raw data. EGA in particular had a nasty graphic layout, with single bytes represent parts of multiple pixels, and port usage to control what portion of the pixels' values were modified. This can make it complex to understand what original drawing code is. And frequently data will be passed to such drawing methods using one pixel per byte, or two pixels per byte in any case. So taking a stab at it yourself first may end up saving time. | |||
* Visual Studio (but alas not IDA) has a nice feature where if you hold down Alt whilst dragging the cursor, you can select a square area of text to copy. I've found this very helping when copying arrays of static data to create arrays. If you've copied a raw block of memory from DosBox, you can paste it into Visual Studio and then Alt-drag the cursor down along the start of all the lines. Then when you can start putting sequences of ', 0x' between the byte values and have it be applied on all the lines. This saves a lot of time converting the data to a properly formatted C array. | |||
== Final Words == | == Final Words == | ||