User:Buddha^/GSoC2007-ProjectDiary

2007-05-28 (Monday) - GSoC starts!

So the GSoC 2007's coding period started. I'm just getting started, I installed Ubuntu 7.04 on my Dell CPX laptop last saturday and got ScummVM to compile after some trying (Just playing with packages, knowing what to install etc. After all the needed packages were there, ./configure and make did the trick).

Right now I'm getting acquainted with ScummVM's internals & workings. And after knowing enough to do so I'll be working on AGI version fallback detection (sev did work on that previously).

Just installed Code::Blocks 2007-05-23 nightly build on my laptop.

So, I took some screenshots of most official AGI games. And submitted my first patch, woot! It was screenshots of Larry I (640x480, hq2x + aspect ratio correction turned on, opting + advpng -optimized).

2007-05-29 (Tuesday) - Taking screenshots of AGI games etc

Today I took and submitted screenshots for the following games (In addition to the Larry I screenshots I submitted yesterday):

Official Sierra AGI games:

• King's Quest I, II, III & IV
• Space Quest I & II
• Police Quest I
• Manhunter I & II
• Gold Rush!
• The Black Cauldron
• Mixed-Up Mother Goose

Fanmade AGI games:

• Voodoo Girl: Queen of the Darned
• Space Quest 0: Replicated
• Space Quest X: The Lost Chapter
• Serguei's Destiny 1

And I also made small thumbs for most of the games like "Larry's head thumb" for the Larry series and a "Sarien head thumb" for the Space Quest series etc.

2007-05-30 (Wednesday) - Starting work on WinAGI's wag-file loading

Installed MinGW and MSYS on my desktop machine (It's got Windows 2000 on it) and compiled ScummVM on it.

Worked on *.wag file loading.

2007-05-31 (Thursday) - Made a quick & ugly rudimentary parser for WinAGI's wag-file loading

Worked some more on *.wag file loading. I've already got a *.wag file parser but it's not pretty :). I'll have to convert it into something better suited for ScummVM.

2007-06-03 (Sunday) - Made class for WinAGI's properties

Made a Doxygen documented WagProperty class that represents a single property from a *.wag file (That can be the game's ID, the game's author's name, an about message about the game, the last edit date of the game or something else). Made it so that you can just read a property's header and choose whether to read the property's data also into memory or just skip it (Useful when you know beforehand what properties you are interested in and what you aren't).

Made WagFileParser class use WagProperty when parsing the *.wag file.

2007-06-04 (Monday) - More work on the WinAGI's wag-file parser

Modified WagProperty class and implemented more of WagFileParser's needed functionality. Documented WagFileParser's implemented parts.

2007-06-05 (Tuesday) - Fallback detection and WinAGI's wag-file parsing continues

Worked on the WagFileParser and Agi::fallbackDetector related code.

2007-06-06 (Wednesday) - Ran into trouble with fallback detection

Installed Visual Studio 2005 Express edition because MinGW+MSYS-combo jammed my Windows 2000 machine some of the time when using make. Hopefully this'll work better...

Ran into trouble with the AGI engine's fallbackDetector-routine needing dynamic ADGameDescription-structs. ADGameDescription has constant strings in it and we'd need to use dynamic strings so it looks like something will have to change. Sev said he'll talk with someone about this. Otherwise it looks like I might be getting to submit my first code patch real soon now. We'll see how soon that'll be...

And about Visual Studio 2005 Express... it does work quite nicely at the moment, thank you :) (Knocks wood).

2007-06-07 (Thursday) - Finished fallback detection for the AGI engine

After some talk with a friend of mine and much talk with my mentor Sev about the problems with the current fallback detection system I set out to implement Fingolfin's suggestion.

Got it done! It took some hours though ;) And it compiles too... let's see about testing and hopefully my first code patch tomorrow.

2007-06-08 (Friday) - Communication lessons and first code patch

Talked with my mentor Sev and got a little wiser about portability. Now I know that seeking in files can be very slow on some platforms, like e.g. GP32. I had ignored Sev's advice on some code choices because I didn't think seeking in files would be an issue at all. There was some discussion and I'll try to be more frank and communicative about my own decisions and ask about the reasons behind others' choices in the future.

I think it's good to know the real reasons behind choices... in this particular case the reason for a certain coding choice was "We shouldn't do much file seeking because file seeking is very slow on some platform (e.g. GP32) that we support".

So I changed my WAG-file parsing code to read in the whole file and handle parsing it in memory rather than go seeking in it on the disk. I separated the changes needed for the dynamic string content generation in the fallback detection from my WAG-file parsing code and submitted the former's changes as a patch. Woot! My first code patch to ScummVM :)

2007-06-11 (Monday) - Changes to the first patch and cleaning WAG-parsing

During the weekend Fingolfin had given feedback to my first code patch in the tracker. And his second alternative approach was clearly better so I took it, made some fixes to it and submitted that to the tracker.

I also took my WAG-file parsing code and simplified and cleaned it. I took some code out to make the code more maintainable (Things like allowing to read only the header of a property and afterwards also its data or skipping that data altogether if we know we don't need that property's data). At least in this case I think it was better to not have them there, rather have more maintainable code than some little added performance boost or less memory usage in this particular non-critical area.

Oh, and I also came to notice that I had somehow managed to get non-normalized line endings in some of the source code files in my local ScummVM copy. Probably something to do with using diff and patch, applying Fingolfin's version of the fallback detection patch to my own local copy etc (It had 0x0A (LF) as the line ending, my local source code files had 0x0D & 0x0A (CR & LF)). Oh, the joys of choice between CR, LF, CR & LF etc :P

2007-06-12 (Tuesday) - Got first patch accepted to the trunk and submitted second patch (WAG file parsing)

Okay, so today I got my first patch with additional changes by Fingolfin & Sev accepted to the trunk. Yay!

I also submitted my second code patch that deals with WAG file parsing. Here's its description:

"Here's a patch that makes use of WinAGI's (http://www.winagi.com/) *.wag file format in AGI games' fallback detection. WAG-files may include information like game ID, game description, used AGI interpreter version, game's last edit date etc. This patch makes the AGI fallback detector use those information, if present."

After this I started looking into AGI256 & AGI256-2 related stuff because that's what's next!

2007-06-13 (Wednesday) - Started looking into AGI256

Read some AGI specifications on the ScummVM wiki. Read some of the ScummVM's AGI engine code. Now I know that in the ScummVM's AGI engine the priorities ("Sort of a Z-buffer") are saved in the same buffer as the picture data (Each pixel's lower 4 bits give the picture data and upper 4 bits give the priority data). So that'll make it a bit more of an effort to support 256 color images in the AGI engine.

Also found that Ctrl-D brings up the debug console and that first 16 colors are for the EGA/Amigaish-palette and the next 16 colors are for the console's transparency (So you can still see the underlying game with the console window on top). We'll see what's to be done with those console transparency colors when we need to use the whole 256 colors for the images' palette. Maybe map them through a table lookup to the static AGI256 256 color palette?

Looked at the AGI 256 pictures' static palette with GIMP. There's a pattern in it. Here's my current theory on the subject:

• AGI256 pictures' palette's structure:
  • First 16 colors are the default EGA palette.
  • Next 16 colors are greyscale colors from black to white.
  • Next 72 colors are probably a HSL/HSI (Or something like that) color sweep with full saturation and brightness.
  • Then the same color sweep as before but with less saturation and/or brightness.
  • And again the same color sweep as before but with even less saturation and/or brightness.
  • Last 8 colors are greyscale colors from low intensity grey to almost black.

Also looked at differences in AGI.EXE between the original Sierra On-Line's AGI interpreter version 2.936 and the hacked AGI256 AGI.EXE. Used HT editor for that. Didn't learn much with that approach though. But then I took Interactive Disassembler Freeware v4.3 and started looking at AGI256's AGIGRAF.OVL file.

Using IDA I got some info out of the AGIGRAF.OVL. Commented some of the functions in it. But without understanding what the functions are that are called from outside I can't understand it fully (AGIGRAF.OVL is like a DOS age DLL so it references functions that aren't in AGIGRAF.OVL but should be in memory when it's loaded into memory).

• Some guesses for the function names from AGI256's AGIGRAF.OVL (*HIGHLY* preliminary and subject to change):
  • setVideoMode (320x200x256c)
  • setTextMode (40x25x16c)
  • showAgi256Pic?
  • calcHeightLUT
    • Calculates table of y*320 values in range 0 <= y < 200
  • setPalette?
  • readNextAgi256Pic?
  • readAgi256Pic?
  • setClrCurrPixel
  • setWhiteMenuBar

2007-06-14 (Thursday) - Taking a look at the AGI256 hacked interpreter

Started to convert ScummVM's AGI engine to a 256 color version so that the color screen and priority screen would be separated into totally different buffers (At the moment they are married/merged together so that each pixel in the 160x168 AGI screen buffer contains 4 bits of color data and 4 bits of priority data). Doing the conversion in this way would require changes in many places in the AGI engine... so Sev hinted that it would be good first to take a look at the hacked AGI256 AGI interpreter with IDA because whoever did that hack highly probably didn't just rewrite half of the engine :-).

So started to look into AGI256 hacked interpreter files. Also found Nick Sonneveld's commented disassemblies for various AGI versions at http://www.agidev.com/projects/nagi/dev.php and they've been helpful.

2007-06-15 (Friday) - Disassembling and reading specifications

Looked more into AGI256's hacked files and read AGI specifications etc.

Commented almost all the differences between the original AGI 2.936's AGI.EXE and the hacked AGI256's AGI.EXE.

2007-06-18 (Monday) - Reverse engineering AGI256

Got more into working with IDA. Using info from Sonneveld's IDBs for various AGI.EXE versions I could make out where in the memory the "DLL"-files are loaded (AGIDATA.OVL, AGIGRAF.OVL, AGIOBJS.OVL). So I made a flat AGI.EXE that has them included in the correct positions. It helps with IDA debugging because IDA can now see all the functions and give cross references between them etc.

So after making the flattened AGI.EXE for the AGI256 hack inspection I read more of the Sonneveld's IDBs, commented and tried to understand parts of the flattened AGI.EXE, made some structs in IDA, fiddled with IDA's segments etc.

Figured out some functions like the agi256ReadPic -function and also a weird looking function that did something with the palette... as a matter of fact it turned out to be a synchronized palette blinking effect that's called from the screen shaking routine (Command number 110: cmd_shake_screen).

Learned a few things today... like that call pushes the location of the next instruction after it (In retrospect that's actually very rational) rather than the location of the call instruction itself. There was some interesting "call $+3 followed by a pop to get the IP" -usage in the agi256ReadPic routine and that's how I came to ponder the call instruction's true workings. HelpPC helped.

It looks like some call is jumping in the middle of a "shr al, 1" instruction. I'll see about that tomorrow...

2007-06-19 (Tuesday) - Got prototypes for AGI256 and AGI256-2 working

The jump instruction that was pointing in the middle of a "shr al, 1" instruction (I talked about that in yesterday's post) was part of some obsolete Hercules graphics mode code. So it was never used in AGI256's or AGI256-2's code.

Reverse engineered some more of the AGI256. Tried a prototype in ScummVM based on a theory about how AGI256 works. Didn't work straight away... some things were off. So took another look at the blitting function in AGIOBJS.OVL as the blitting function was one of the points were I wanted/needed more information. Finally at some point it worked! AGI256 prototype support for ScummVM is here :-).

Encouraged or just on a plain roll I looked at the AGIOBJS.OVL differences in the blitting function between AGI256 and AGI256-2. There wasn't much difference in them. Just that the AGI256-2 didn't use mirroring or run length encoding for its views (That info I had already got from AGI256-2's readme, though) and little else (How it wrote the pixels into memory for one). Tried another prototype for supporting AGI256-2 and other some fiddling and coding, it now works too.

There's some discrepancy in AGI256-2's intro (The background area surrounding the AGI256-logo, the color bar and the authors text isn't all black, it's partially grey). But I get something very similar when I run the AGI256-2's hacked AGI.EXE from command line in full screen mode under Windows 2000 (Although as I tried it just now, it doesn't always do that). Running the AGI256-2 demo just by clicking on the AGI256-2's hacked AGI.EXE in Explorer works correctly. So I'm not sure what's causing it exactly. Gotta look into it...

2007-06-20 (Wednesday) - My first commits to ScummVM's trunk, yay!

Changed the AGI256 and AGI256-2 prototypes into something that I could honestly put into the trunk :-). Did my first commit to the ScummVM's trunk today! Yay! But the log message didn't get there... "svn ci" did spring up a window to me and I wrote the log message into it but it didn't go anywhere. Well, live and learn :), for the next commit I wrote the log message into a file before calling "svn ci" and it worked just fine.

This was the log message that was supposed go into my first commit:

• Add setting of 256 color palette for AGI256 and AGI256-2 games. Uses parts of patch #1728713.

Cut AGI256 and AGI256-2 support commits into smaller pieces and committed those. So now my first real feature additions to ScummVM's AGI engine are in the trunk. I'm thrilled that it's so :)

2007-06-21 (Thursday) - Thinking about what to do next and leaving for Midsummer festivities

Didn't do very much today, looked at the bug tracker to see if there were any AGI related bugs listed. Talked about the Gold Rush being incompletable bug on #scummvm, thinked about whether AGIPAL256 (Similar to AGIPAL but supporting 256 colors) support would be a nice addition to ScummVM etc. Updated my wiki front page a bit. Left for Midsummer festivities during the afternoon.

2007-06-25 (Monday) - Some bug fixing and removing obsolete code

Cleaned up obsolete console color code from AGI engine. Almost completely fixed priority screen showing in AGI256 and AGI256-2 mode (Only the ego's priority showing is still missing). Committed a game specific workaround for bug #1737343 ("GOLDRUSH: no intro after trivia").

Tested game saving and restoring with AGI256 and AGI256-2 games. I think the code probably is broken at the moment (But it should be broken only for AGI256 and AGI256-2 games). I tried AGI256-demo, AGI256-2-demo and Dashiki game's 256 color demo and they worked with saving and restoring. Although maybe they worked just because you can only save and restore in them in a single room... I'll look into this.

As a sidenote, pressing F12 breaks from ScummVM into the Visual C++ 2005 Express Edition's debugger. I didn't know that before :). So now I know. And the AGI engine in ScummVM uses F12 for switching between showing the priority screen and the color screen. So it felt very odd when I tried out the priority screen showing code and pressed F12 and all of a sudden it just breaked into some SDL_delay -routine. Weird, really... but not so weird anymore... so it's MSVC's feature, not a bug in ScummVM :).

2007-06-26 (Tuesday) - Some bug fixing and AGI256 hack documenting

Fixed saving & restoring with AGI256 and AGI256-2 games.

Went through the differences of AGI256's AGI.EXE and the Sierra's original v2.936 AGI.EXE and documented almost all the changes more thoroughly (Useful for making some public documentation of the hack later).

Debugged the fanmade AGI game Hobbits, looked at its logic in AGI studio. Played it with NAGI and tried to play it under Windows 2000's command prompt and DosBox 0.70 but AGI didn't like them somehow and gave me a prompt telling that there's not enough memory. With NAGI I got the whole intro and could also get to the starting room and play. With ScummVM some sprites and text were missing from the intro and I could get to the starting room by pressing F10 but the player character was missing and soon the intro restarted for some reason. There's really something bugging with this one :-). Didn't really make heads or tails of the why behind this yet.

Also tried the fanmade AGI game Space Trek (There's a bug listed for it in the ScummVM's bug tracker) and yes, the bug exists, I confirmed that, but that's about it.

2007-06-27 (Wednesday) - A bit more of AGI256 hack documenting

Didn't do much today... probably going to catch up a bit by doing some work on saturday. Looked at AGI256 hack's blitting function (One of the last functions I hadn't documented yet) and documented some AGI flags related functions that were used in the blitting function.

2007-06-28 (Thursday) - Reverse engineered AGI256's and AGI256-2's blitCel-function to a C subroutine

Okay... so agi256BlitCel-function was the last function in AGIOBJS.OVL that I hadn't already reverse engineered into a C subroutine. And I have to say... woah, reverse engineering agi256BlitCel-function took time. Like, many, hours. But it was worth it, now I can say that I've almost 100% reverse engineered AGI256 and AGI256-2 hacks... there's one little bit of data near AGI.EXE's start (Offsets 0x265 - 0x277 in the EXE-file) that are still a mystery to me. I tried putting zeroes in their place and it still ran so it may be that they aren't needed. Also I don't really know what's the usage (If any) of the relative call (db 0xE8 0x49 0x00) at 0x262 - 0x264 in AGI256's and AGI256-2's AGI.EXE.

2007-06-29 (Friday) - AGI monitor type variable (v26) stuff, CGA rendering info etc

Looked into AGI CGA rendering (NAGI and ScummVM). Tried out CGA modes with NAGI, DosBox and ScummVM. Looked into the toggle.monitor-command for changing between different CGA modes. Added support for changing AGI monitor type variable (v26) based on the rendering mode. Had a little talk on #scummvm about the mysterious bytes in the AGI256 AGI.EXE's start (The bytes I mentioned in yesterday's diary entry). Fiddled a bit with AGI256 AGI.EXE's disassembly in IDA, added a couple of segments etc.

Oh and I happened to commit an enum with an extra comma in its declarations end... jvprat told me that GCC threw an error with it (MSVC8 didn't), so I went and corrected it. First time to break and fix the trunk :P

Also loaded Larry 1's walkthrough on Amiga as a video from Youtube. I thought there might be some menu usage to see more examples of what the Amiga style menus are like, but at least with a little of jumping around the video, I didn't find any.

2007-06-30 (Saturday) - Relaxation and some looking into Amiga style menus' colors

During the week I thought I might do some work on ScummVM today but mostly didn't. Slept much, went and had some thai food with a friend and after that we went to the Turku's Medieval Market. At the restaurant Teini there was some folk music band playing, lots of delay, drums, a flute, a saxophone etc. Nice. Drank a few beers and then went home. Later I Googled and found out that the nice sounding band was Tervakello.

At night did some looking into what the Amiga style menus are like. At a glance it would seem that there are two types of Amiga AGI games in respect to menu colors.

My current theory on colors in Amiga style menus:

• A positive element (Usually left if there are two elements):
  • White text with a green (Unchosen) or a purple (Chosen, Type 1) or a light orange (Chosen, Type 2) background.
  • An example: Restart in a Restart/Cancel-dialog.
  • The chosen positive element's background color differs, examples:
    • In King's Quest I it's light orange (Type 1)
    • In King's Quest III it's purple (Type 2)
• A negative element (Usually right if there are two elements):
  • White text with a red (Unchosen) or cyan (Chosen) background.
  • An example: Cancel in a Restart/Cancel-dialog.

(Correction to the above (Added on 2007-07-05): All chosen elements have black text color, not white).

Some elements aren't in color. Like the elements in the top dropdown menu. Or elements in the inventory. They're just white text on a black background (If chosen, vice versa if not). Maybe we should talk about Amiga style dialogs rather than menus to distinguish them from the top dropdown menu, inventory etc.

Oh well... it seems there can be colored stuff in the inventory too like the Cancel-button.

2007-07-02 (Monday) - A change of scenery

Took a bus from Turku to Pori (To my parents' place at the countryside. There are cows in the field next to our house :-)). I'm going to be here in the countryside for the better part of July (My parents have ADSL and a WLAN router so I can get to the net from here). Took my IBM Thinkpad T23 with me. I haven't had it for long, just bought it some weeks ago so I installed Visual C++ 2005 Express on it, got it to compile ScummVM and checked out branch-0-10-0 so I might commit the Gold Rush intro workaround to it too in addition to the trunk. Tomorrow I'm going to do some work too ;-)

2007-07-03 (Tuesday) - First workaround backport and Amiga style menus prove a little bit problematic

Backported the Gold Rush intro skipping bug's workaround to the 0.10.0 branch. Investigated ScummVM's AGI engine's menu system a bit. It's not without its difficulties though, as in Amiga AGI games in a selection box there is no concept of a currently chosen element unless you go and click on them with your mouse. In ScummVM's AGI engine there always is a chosen element in a selection box by default. It wouldn't be very intuitive e.g. if you had an orange button (Currently chosen) on the left and a red button on the right and you'd press right to change the selected button and the left button would change color to green and the right button would change color to cyan. How would you know which of the buttons is the currently chosen one and which is not if you didn't know about the color coding beforehand?

So I've got two ways to make the Amiga style menus happen:

• Make it so that you have to click on things with your mouse (Or whatever emulates a mouse) because that's the way Amiga AGI games did it.
  • This would get around the problem of not knowing what button is selected based on the button's color as you'd have to go and click on the button you want to press.
  • This would also be more faithful to the Amiga AGI games as they did it this way (At least as far as I know).
  • This would change the usability of the ScummVM's AGI engine in Amiga mode because now you'd have to click on things to press them.
  • Probably a bit harder to implement than the latter approach.

• Make somehow apparent which of the buttons is currently selected (E.g. draw a black border around the chosen button).
  • This also would get around the problem of not knowing what button is selected as you'd see it by the chosen button's obvious border coloring (Or whatever method would be used).
  • This wouldn't be completely faithful to the Amiga AGI games as they didn't have a currently chosen button as far as I know.
  • This wouldn't change the usability of the ScummVM's AGI engine from its current state.
  • Probably a bit easier to implement than the former approach.

2007-07-04 (Wednesday) - Investigating EGA & Amiga-ish palette & doing groundwork for implementing Amiga style menus

Took a look at the Amiga-ish palette (newPalette-array in engines/agi/graphics.cpp) and the normal EGA palette (egaPalette-array in engines/agi/graphics.cpp). Compared them a bit. Here's how they look:

EGA (Top) vs. Amiga-ish (Bottom) palette

Here's only the EGA palette on its own as a 8x2 image (Scaled up by a factor of 40 for easier visibility), so one can more easily compare the different shades of similar colors:

EGA palette (Top: Colors 0-7. Bottom: Colors 8-15)

And here's the Amiga-ish palette on its own:

Amiga-ish palette (Top: Colors 0-7. Bottom: Colors 8-15)

Pondered about the way to pass information to the drawButton-function in the AGI engine. Probably going to pass some struct to it that has info on what colors to use in which situation (e.g. use this color when a positive button is in focus, use this color when a negative button is unchosen etc).

Consulted Sev about which way to go about the Amiga style menus and yes, I'm going to implement them by a little non-authenticity by using a border around the currently chosen button. This has the positive aspect of not altering the usability of ScummVM from its current state.

2007-07-05 (Thursday) - Working on Amiga style menus implementation

Took screenshots of a couple of Amiga AGI games using WinUAE 1.4.2a. Worked on the Amiga style menus implementation. Probably going to try a bit different way at the implementation tomorrow.

Screenshots from King's Quest I and King's Quest III on Amiga showing different button colors (Screenshots taken with WinUAE 1.4.2a):

King's Quest I (Amiga) screenshots

King's Quest I (Amiga): Restart-dialog. No buttons are being pressed with the mouse.

King's Quest I (Amiga): Restart-dialog. Restart-button is being pressed with the mouse.

King's Quest I (Amiga): Restart-dialog. Cancel-button is being pressed with the mouse.

King's Quest III (Amiga) screenshots

King's Quest III (Amiga): Save-dialog. No buttons are being pressed with the mouse.

King's Quest III (Amiga): Save-dialog. OK-button is being pressed with the mouse.

2007-07-06 (Friday) - Amiga cursors and palette

ScummVM's AGI engine has a SCI-style black & white mouse cursor. I added an Amiga cursor for the Amiga rendering mode (Color values taken from some Amiga AGI game - I don't remember which - running on WinUAE 1.4.2a). Also took the Amiga-style busy mouse cursor (An hourglass) from Space Quest II running on WinUAE 1.4.2a and added its data to ScummVM. The busy mouse cursor is not used at the moment though. But its there, if someone likes to implement using it.

Clone2727 on #scummvm had made a page on ScummVM's wiki for showcasing different palettes used in ScummVM's AGI engine. It turned out my fiddlings with GIMP for the visualization of ScummVM's AGI engine's palettes didn't go to waste as the images I made (See my 2007-07-04 diary entry) found their way to that page.

Talked about Amiga palettes etc with Raziel_AOne on #scummvm. He told how he had modified King's Quest's - it was probably I or II - executable back in the day on Amiga in order to change King Graham's skin color from yellow to pink. The hack wasn't without its difficulties though, as some of trees etc turned pink too :-). Raziel just wanted to know if there was a way to modify ScummVM so King Graham's skin color in King's Quest I & II would be pink. Of course its possible but off the top of my head it may require some sprite specific modifications. Came to the conclusion that an engine-specific GUI-option for turning the skin color modification on would be good after talking about it with Raziel and Sev.

It turned out that different Amiga AGI games use a different palette. As it dawned on me that Raziel had sometime hex-edited a Sierra AGI executable on Amiga in order to change the palette I thought that that's gotta be the place where the palette is defined. So I asked enthusiastically if he could show me the position in the executable. Later he sent me an email with the info and also some of the different palettes from various Amiga AGI games. Amiga AGI game palettes, here I come :-).

2007-07-07 (Saturday) - Comparing menu borders and decoding Amiga cursor format

Compared different menu border styles for the Amiga-style menus with GIMP. Probably going to simply use a thick black border around the currently chosen button as its easy to code and in my opinion it looks sufficiently intuitive and ok.

Also looked at the cursor files that come with some of the Amiga AGI games (The files are named Busy and Pointer). Finally figured them out almost completely.

Amiga AGI games' cursor format

My current theory on the format is as follows (Based on the Busy and Pointer -files from the Amiga version of King's Quest III):

Cursor's bitmap data

• Starts with four bytes of zero.
  • Don't know what these are for. Maybe flags?
  • Were zero in both Busy and Pointer -files.
• 16-bit big endian word: Bitplane 0 of row 0
  • The most significant bit is graphically the leftmost one
  • The least significant bit is graphically the rightmost one
• 16-bit big endian word: Bitplane 1 of row 0
• 16-bit big endian word: Bitplane 0 of row 1
• 16-bit big endian word: Bitplane 1 of row 1
• ... and so on for all the rows 0-15

Cursor's palette

• Again four bytes of zero.
  • Don't know what these are for. Maybe flags?
  • Were zero in Both Busy and Pointer -files.
• Then the palette for four colors:
• Palette entry for color 0:
  • 16-bit big endian word: ARGB-color with 4-bits per color component
    • Alpha is bits 12-15
      • Value of 0x0F means the color is totally transparent
    • Red is bits 8-11
    • Green is bits 4-7
    • Blue is bits 0-3
• Likewise palette entries for colors 1, 2 and 3

That's it. So the files are 80 bytes long, the bitmap is 16x16 in size and uses 4 colors. Some of the zero bytes are still such that I'm not completely sure what they're there for. And I'm not 100% sure about the first palette entry if its really just that and not some other data but it does fit the theory nicely.

Here are images of the Busy and Pointer -files:

Busy -mouse cursor

Pointer -mouse cursor

2007-07-09 (Monday) - Amiga AGI game palette extracting

Today I made a command line tool for detecting and extracting Amiga AGI game palettes. You can make it print out a C-array of the 16-color 12-bit RGB palette if its found. Removes duplicate palettes too (Found in some file I tried).

Here's the format of the Amiga AGI games' palettes:

• The palette consists of 16 colors.
• Each color consists of red, green and blue components (In this order).
• Each component is a 16-bit big endian word in range 0-15 (i.e. 4 bits).

The palette should be found in the game's executable file.

I use the following heuristic for detecting a possible palette from a file:

• First color must be black i.e. R = G = B = 0
• Last color must be white i.e. R = G = B = 0x0F
• All components of all colors must be in range 0-15
• No other black color must exist in the palette than the first color
  • Added to help weed out some false positives

2007-07-10 (Tuesday) - Added Amiga-style menus (i.e. colored options)

Added Amiga-style colored options (FR #657645) to the trunk.

Here are the differences (That I can think of at the moment) to the original Amiga AGI engine:

• Buttons are a bit thinner than in the original Amiga AGI engine
• Currently chosen button has a black border around it
  • There was no such thing in the original Amiga AGI engine
• Pressed buttons changed color in the original Amiga AGI engine
  • Currently ScummVM's AGI engine doesn't use the pressed state although there is some support for it in the code

2007-07-11 (Wednesday) - Added Amiga AGI game palette extractor tool

Went to town (Pori) to buy some new clothes. Didn't do very much strictly work today. Mostly worked on the Python version of the Amiga AGI game palette extractor. I had made a C++ version of it but felt that it might be good to learn a language that would be easier to do things fast in. And as Python had come up in discussions with a good friend of mine, I thought, why not try it.

I started dabbling with Python a day or two ago. Just downloaded the Python Windows installer and the HTML-version of the documentation from python.org. Seems like a nice language. Laughed out loud at some of the Monty Python jokes in the examples :-). Like the list of strings ['cat', 'window', 'defenestrate'] in this example. Like what? Defenestrate? What's that? Have a look what Meyer-Webster has to say about defenestrate :-).

So I made a tool for extracting palettes from Amiga AGI games' executable files and added it to ScummVM's trunk. You can see the source code here. My first Python program, yay!

2007-07-12 (Thursday) - Did more work on the Amiga AGI palettes

Worked on the Amiga AGI game palette extractor tool. Added wildcard support and a preliminary test that speeded the program up like a gazillion times or so :-).

2007-07-13 (Friday) - Left for a short vacation in eastern Finland

Went to eastern Finland to the city of Kouvola with my parents and my older brother. The occasion was my older brother's godparents' son's graduation party. Worked a bit on the ride with my laptop. Did some additional tool hacking with Python to parse the data Raziel_AOne sent me about different Amiga AGI palettes. Also made a program to compare different palettes so I can easily see which ones are unique.

2007-07-14 (Saturday) - Some festivities and a trip back from eastern Finland

Celebrated my older brother's godparents' son's graduation and left back for the city of Pori. Did a bit of work on sorting out the different Amiga AGI palettes on the way back.

2007-07-15 (Sunday) - Sorted out the Amiga AGI palettes

Talked with Raziel_AOne, got original palette data from him. Compared all the palette data I have and found that there are only three unique palettes.

Added the different Amiga AGI palettes to ScummVM and modified palette handling in the AGI engine so it'll be easier to enable the support for those palettes later (Now the palette data just is there but isn't used yet).

The three generations of Amiga AGI palettes

From all the Amiga AGI games I and Raziel_AOne gathered palette data from I found only three unique palettes. Here's my current theory on them:

• First generation Amiga AGI palette
  • Used with Amiga AGI versions 2.0xx - 2.1xx
  • Used with Amiga AGI games released in 1986-1987
    • Maybe also earlier game versions use this palette?
  • Examples of different AGI versions using this palette:
    • 2.082 (King's Quest I v1.0U 1986)
    • 2.082 (Space Quest I v1.2 1986)
    • 2.090 (King's Quest III v1.01 1986-11-08)
    • 2.107 (King's Quest II v2.0J 1987-01-29)
    • x.yyy (Black Cauldron v2.00 1987-06-14)
    • x.yyy (Larry I v1.05 1987-06-26)
• Second generation Amiga AGI palette
  • Used with Amiga AGI versions 2.2xx
  • Used with Amiga AGI games released in 1988
  • Examples of different AGI versions using this palette:
    • 2.202 (Space Quest II v2.0F. Intro says 1988. ScummVM 0.10.0 detects as 1986-12-09)
• Third generation Amiga AGI palette
  • Used with Amiga AGI versions 2.3xx
  • Used with Amiga AGI games released in 1989
  • Examples of different AGI versions using this palette:
    • 2.310 (Police Quest I v2.0B 1989-02-22)
    • 2.316 (Gold Rush! v2.05 1989-03-09)
    • x.yyy (Manhunter I v1.06 1989-03-18)
    • 2.333 (Manhunter II v3.06 1989-08-17)
    • 2.333 (King's Quest III v2.15 1989-11-15)

Palette data comparison (12-bit RGB data): (From color 0 to 15 from top to bottom, from first to third generation from left to right)

• 0x000, 0x000, 0x000 (No difference)
• 0x00F, 0x00F, 0x00B (Some difference)
• 0x080, 0x080, 0x0B0 (Some difference)
• 0x0DB, 0x0DB, 0x0BB (Some difference)
• 0xC00, 0xC00, 0xB00 (Some difference)
• 0xB7D, 0xB7D, 0xB0B (Major difference)
• 0x850, 0x850, 0xC70 (Some difference)
• 0xBBB, 0xBBB, 0xBBB (No difference)
• 0x777, 0x777, 0x777 (No difference)
• 0x0BF, 0x0BF, 0x00F (Major difference)
• 0x0E0, 0x0E0, 0x0F0 (Some difference)
• 0x0FD, 0x0FD, 0x0FF (Some difference)
• 0xF98, 0xF98, 0xF00 (Major difference)
• 0xF70, 0xD0F, 0xF0F (Major difference)
• 0xEE0, 0xEE0, 0xFF0 (Some difference)
• 0xFFF, 0xFFF, 0xFFF (No difference)

2007-07-16 (Monday) - Worked on the online AGI256 and AGI256-2 documentation

Added information about AGI256 and AGI256-2 to the AGI Specifications page. The info is in the Fan-Made Extensions section. I'm quite pleased with the general descriptions but not so much with the implementation details subpages for AGI256 and AGI256-2. They're a bit of a mess at the moment to say it politely ;).

2007-07-17 (Tuesday) - Banged my head to the wall with bug #1751483

Tried to fix bug #1751483 (Description: "AGI: The -x command-line option appears to be broken").

Was somehow unfocused today, did only about a half a day's work. Didn't get far with the bugfixing as I banged my head to the wall with it for some hours. It just seemed that yes, the save_slot parameter does get correctly parsed from the command line and set to the configuration manager's key save_slot but doesn't for some unknown reason get to the AGI engine at all.

Sooo... it seems that it was just the way I used the command line options that made it act that way. I did use the --save-slot i.e. -x command line option but used the launcher with it to choose a game. Oh well, I did realize that after some hours :) and now I can at least reproduce the bug on my machine (By also giving the game's name on the command line, not just the save slot number).

The problem with my approach was that if no game was given on the command line the save_slot's value was cleared when loading a game. It got cleared as a part of the configuration manager's transient domain (kTransientDomain i.e. __TRANSIENT) that got cleared at game loading. So no wonder the value never got to the AGI engine's side at all :-).

And yeah, it's broken at the moment, the bug is true. With Police Quest I's PC version I also got the menus to disappear when loading some save game for it from the command line. And when I tried accessing the menu the game died. So it's definitely broken. But I haven't come up with a reasonably easy way to fix it yet and starting work on the Apple IIGS sound support is around the corner so I'll probably put this bug back to the pool.

2007-07-18 (Wednesday) - Started looking into Apple IIGS sound support

Searched the net for information on Apple IIGS sound. Read some. Filled in the mid-term student survey with quite a bit of comments.