417
edits
(Started page about Cinematique engine's internals, file formats etc. Added preliminary info about 'vol.cnf' file in Operation Stealth.) |
(Added more detail to chapter title "Compression format" with "Compression format (Bytekiller 1.3)".) |
||
(34 intermediate revisions by 2 users not shown) | |||
Line 3: | Line 3: | ||
This is a place intended for information about the [[Cine|Cinématique]] engine's internals, file formats etc. | This is a place intended for information about the [[Cine|Cinématique]] engine's internals, file formats etc. | ||
Adding any additional information here is encouraged. | Adding any additional information here is encouraged. | ||
Information probably is inaccurate in places so if you have more accurate information, | Information probably is inaccurate in places (e.g. may be specific to Operation Stealth rather than apply to Future Wars and/or Cruise For A Corpse too) | ||
please add it here. | so if you have more accurate information, please add it here. | ||
== | ==File formats== | ||
===' | ===.ADL files=== | ||
AdLib music. | |||
===.AMI files=== | |||
NOTE: This file type seems to be specific to Amiga versions of Operation Stealth. Not sure if Future Wars uses this. | |||
It looks like *.AMI files are samples (Loaded a couple of them into [http://audacity.sourceforge.net/ Audacity] using raw 8-bit signed audio data import and played them and they sounded alright). | |||
===.ANI files=== | |||
Animation. | |||
===.H32 files=== | |||
Probably Roland MT-32 music. | |||
===.HP files=== | |||
Probably [http://en.wikipedia.org/wiki/PC_speaker PC speaker] music files. | |||
===.IST files=== | |||
NOTE: These seem to be specific to the PC versions of Operation Stealth. Don't know about Future Wars. | |||
All filenames of this type start with "BOND" and the files are 15 bytes in size. | |||
Here's a list of all the 12 different IST-files I've seen so far: | |||
* BOND1.IST, BOND4.IST, BOND10.IST, BOND11.IST, BOND14.IST, BOND16.IST, BOND20.IST, BOND22.IST, BOND23.IST, BOND24.IST, BOND31.IST, BOND32.IST | |||
These probably have something to do with Roland MT-32 music as there are | |||
corresponding files without the IST extension as well (BOND1, BOND4, ..., BOND32) | |||
and the corresponding files without the IST extension contain names of .H32 files | |||
(e.g. DEEPSNAR.H32, ELECGTR1.H32, SLAPBAS1.H32, STRSECT1.H32) in addition to other data. | |||
===.MIG files=== | |||
"INTRO.MIG" is the only file I've seen of this type (And that's with Operation Stealth, | |||
I'd guess Future Wars hasn't got any files with this extension but that's only a guess). | |||
I'd guess "INTRO.MIG" is Operation Stealth's introduction sequence's music file. | |||
Here are the [http://en.wikipedia.org/wiki/MD5 MD5 checksums] of the two versions of the INTRO.MIG file I've seen: | |||
* 65613c9f127ecf8bd7988aef7e49b698 (1624 bytes, Amiga versions of Operation Stealth) | |||
* efb187353d289d9b85e1e455734e38f9 (3424 bytes, PC and Atari ST versions of Operation Stealth) | |||
The Amiga version's file starts with the characters "SONG" and then a bit later | |||
there reads "A_COEUR.AMI" (AMI files seem to be samples). In the PC and Atari ST versions' | |||
file there reads "COEUR___.H32" and values "02", "03", ..., "15". BTW coeur is French and means heart. | |||
===.MSG files=== | |||
A formatted file containing a list of strings. | |||
===.MSK files=== | |||
An image mask (1 bit per pixel). | |||
===.OBJ files=== | |||
A formatted file containing descriptions of objects in a scene. | |||
Each object has info about X, Y, Mask, Frame, Costume, Name and Part. | |||
===.PI1 files=== | |||
Possibly some kind of an image mask or image data? | |||
===.PRC files=== | |||
Script files. PRC = Procedure? | |||
===.REL files=== | |||
Script files. REL = Relation? | |||
===.SEQ files=== | |||
Some type of animation data. | |||
===.SET files=== | |||
An image set. | |||
===.SPL files=== | |||
NOTE: This file type seems to be specific to Amiga and Atari ST versions of Operation Stealth. Not sure if Future Wars uses this. | |||
Some type of animation data. | |||
===Files without any extension=== | |||
Examples: | |||
* BOND1, BOND4, BOND10, BOND11, BOND14, BOND16, BOND20, BOND22, BOND23, BOND24, BOND31, BOND32 | |||
** These files probably have something to do with music. For more info take look at [[Cine/Specifications#.IST_files | .IST files]]. | |||
Examples from Operation Stealth's Amiga demo: | |||
* CELLO2, CELLOSEC, ETOMSIMO, FUZZGUIT, PRO1002, PRO11, PRO1120, PRO13, PRO3, PRO300, PRO69, PRO700, PRO800, PRO9874 | |||
** These seem to be sample files (Possibly identical in format to the [[Cine/Specifications#.AMI_files | .AMI files]]) | |||
===Savegames=== | |||
NOTE: This is specific to Operation Stealth's PC version's savegame format at the moment. | |||
* 0x0000: Current disk (Uint16BE) | |||
* A chunk of current string variables: | |||
** 0x0002: Current part name (String, 13 bytes) | |||
** 0x000F: Current procedure name (String, 13 bytes) | |||
** 0x001C: Current relation name (String, 13 bytes) | |||
** 0x0029: Current message name (String, 13 bytes) | |||
** 0x0036: Current background names (8 strings, 13 bytes each) | |||
** 0x009E: Current Ct name (String, 13 bytes) | |||
* Object structs: | |||
** 0x00AB: Entry count (Uint16BE), value discarded on load and assumed to be 255 in the PC version of Operation Stealth | |||
** 0x00AD: Entry size (Uint16BE), 32 in the PC version of Operation Stealth | |||
** 0x00AF: Object structs (Entry count structs, entry size bytes each): | |||
** struct objectStruct: | |||
*** x (Int16BE) | |||
*** y (Int16BE) | |||
*** mask (Uint16BE), seems to be a priority or a Z-buffer value this one | |||
*** frame (Int16BE), indexes the animDataTable | |||
*** costume (Int16BE), maybe this should be named room number instead? | |||
*** name (String, 20 bytes) | |||
*** part (Uint16BE) | |||
* Palettes: | |||
** 0x208F: Temporary color palette (256 colors, 3 bytes per color, 768 bytes total) | |||
*** TODO: Make sure whether this is the backup color palette or the active one | |||
** 0x238F: Color palette (256 colors, 3 bytes per color, 768 bytes total) | |||
*** TODO: Make sure whether this is the active color palette or the backup one | |||
* 0x268F: Global variables (255 Uint16BE values, 510 bytes total) | |||
* 0x288D: Zone data table (16 Uint16BE values, 32 bytes total) | |||
* 0x28AD: Command variables (4 Uint16BE values, 8 bytes total) | |||
* 0x28B5: Command buffer (String, 80 bytes) | |||
* 0x2905: Zone query table (16 Uint16BE values, 32 bytes total) | |||
* 0x2925: Current music name (String, 13 bytes) | |||
* A chunk of Uint16BE variables: | |||
** 0x2932: Is music loaded? (Uint16BE, Boolean) | |||
** 0x2934: Is music playing? (Uint16BE, Boolean) | |||
** 0x2936: Player's command string's vertical position on-screen (Uint16BE) | |||
** 0x2938: Some unknown zero value (Uint16BE) | |||
** 0x293A: Allow player input? (Uint16BE, Boolean) | |||
** 0x293C: Player command (Uint16BE) | |||
** 0x293E: commandVar1 (Uint16BE) | |||
** 0x2940: Is draw command enabled? (Uint16BE, Boolean) | |||
** 0x2942: var5 (Uint16BE) | |||
** 0x2944: var4 (Uint16BE) | |||
** 0x2946: var3 (Uint16BE) | |||
** 0x2948: var2 (Uint16BE) | |||
** 0x294A: commandVar2 (Uint16BE) | |||
** 0x294C: Default menu background color (Uint16BE) | |||
** 0x295E: adBgVar1 (Uint16BE) | |||
** 0x2950: currentAdditionalBgIdx (Uint16BE) | |||
** 0x2952: currentAdditionalBgIdx2 (Uint16BE) | |||
** 0x2954: additionalBgVScroll (Uint16BE) | |||
** 0x2956: adBgVar0 (Uint16BE) | |||
** 0x2958: Is system menu disabled? (Uint16BE, Boolean) | |||
* Animation data structs: | |||
** 0x295A: Entry count (Uint16BE), value discarded on load and assumed to be 255 in the PC version of Operation Stealth | |||
** 0x295C: Entry size (Uint16BE), 36 in the PC version of Operation Stealth | |||
** 0x295E: Animation data structs (Entry count structs, entry size bytes each): | |||
** struct animData: | |||
*** Width (Uint16BE) | |||
*** Var1 (Uint16BE) | |||
*** Bits per pixel (Uint16BE), or maybe type? | |||
*** Height (Uint16BE) | |||
*** Data pointer (Uint32LE, Real mode far pointer, only test for equality or inequality with zero!) | |||
*** File index (Int16BE) | |||
*** Frame index (Int16BE) | |||
*** Name (String, 20 bytes) | |||
* 0x4D3A: Unknown screen parameters (6 parameters, Uint16BE each, 12 bytes total) | |||
* Global scripts: | |||
** 0x4D46: Entry count (Uint16BE) | |||
** 0x4D48: Global script structs (Entry count structs, entry size undetermined) | |||
*** TODO: Determine the entry size used by a global script | |||
* Object scripts: | |||
** Entry count (Uint16BE) | |||
** Object script structs (Entry count structs, entry size undetermined) | |||
*** TODO: Determine the entry size used by an object script | |||
* seqList elements: | |||
** Entry count (Uint16BE) | |||
** seqList element structs (Entry count structs, 32 bytes each) | |||
* Overlays: | |||
** Entry count (Uint16BE) | |||
** Overlay structs (Entry count structs, 22 bytes each) | |||
* Background incrusts | |||
** Entry count (Uint16BE) | |||
** Background incrust structs (Entry count structs, 22 bytes each) | |||
And there the savegame file ends. | |||
===Part file format=== | |||
NOTE: This applies to both Future Wars and Operation Stealth. | |||
<pre> | |||
Part file's start: | |||
Byte Meaning | |||
----- ----------------------------------------------------------- | |||
0-1 Number of elements in this part file (Uint16BE) | |||
2-3 Entry size (Uint16BE). Normally 0x1E i.e. 30 | |||
----- ----------------------------------------------------------- | |||
</pre> | |||
Then comes info for each element (Entry size in length each): | |||
<pre> | |||
Byte Meaning | |||
------ ----------------------------------------------------------- | |||
0-13 Name (ASCIIZ string) | |||
14-17 Data's starting offset in this part file (Uint32BE) | |||
18-21 Packed size (Uint32BE) | |||
22-25 Unpacked size (Uint32BE) | |||
26-29 ??? | |||
------ ----------------------------------------------------------- | |||
</pre> | |||
After that it's the data for all the elements contained in this part file. | |||
===VOL.CNF=== | |||
NOTE: This file is specific to Operation Stealth. | |||
This file contains list of resource files (e.g. PROCS10, RSC04, SONS2, LABYBASE etc) | This file contains list of resource files (e.g. PROCS10, RSC04, SONS2, LABYBASE etc) | ||
Line 78: | Line 304: | ||
"GIRL SET" -> "GIRL.SET" | "GIRL SET" -> "GIRL.SET" | ||
</pre> | </pre> | ||
===Compression format (Bytekiller 1.3) === | |||
The compression algorithm used by all Delphine's adventure games uses | |||
sliding window compression (Quite like [http://en.wikipedia.org/wiki/LZ77 LZ77]) | |||
combined with a fixed non-adaptive [http://en.wikipedia.org/wiki/Entropy_coding entropy coding] | |||
scheme. It seems that the algorithm is [http://www.pouet.net/prod.php?which=47994 Bytekiller 1.3]. | |||
This is based on comparing the reverse engineered | |||
[https://github.com/scummvm/scummvm/blob/branch-2-0-0/engines/cine/unpack.cpp#L92 decompression routine] | |||
in ScummVM with the | |||
[https://github.com/aperture-software/colditz-escape/blob/v1.2/low-level.c#L150 uncompress]-function | |||
from [https://github.com/aperture-software/colditz-escape Colditz Escape]'s source code. | |||
The compressed data is in big endian 32-bit chunks, working backwards from the buffer's end. | |||
So we start from the data's end and work backwards. Also outputting the unpacked data is | |||
done backwards, starting from the destination buffer's end and working backwards byte by byte. | |||
<pre> | |||
Compression format: | |||
NOTE: As the whole data consists of unsigned big endian 32-bit integers, I use indexing | |||
in 32-bit addresses here. By -1 I mean the last 32-bits of the data | |||
(i.e. bytes src[srcLen-4], src[srcLen-3], src[srcLen-2] and src[srcLen-1]), | |||
by -2 the second to last 32-bits etc. | |||
Dword Meaning | |||
--------- ------------------------------------------------------------------------ | |||
-1 Unpacked length (Uint32BE). | |||
-2 Error code (Uint32BE). Xor of the whole packed data in Uint32BE chunks. | |||
0 - (-3) The packed data (In Uint32BE chunks). | |||
-------- ------------------------------------------------------------------------ | |||
</pre> | |||
====Bit sequences in the compressed stream==== | |||
The whole packed data consists of commands and their parameters. | |||
The source stream is read as a bit stream. Behind the scenes it is read in one | |||
unsigned big endian 32-bit integer value at a time and when such a chunk becomes | |||
depleted of bits then another chunk is read in etc. Each chunk's least significant | |||
bit is the first bit read from it, the most significant bit is the last bit read from it etc. | |||
=====First 32-bit chunk===== | |||
Because of the way the Delphine's decompressor routine handled the bit stream the first | |||
chunk acts in a sort of a peculiar way (Take a look at the functions rcr | |||
([http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/engines/cine/unpack.h?revision=32689&view=markup#l_62 Declaration], | |||
[http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/engines/cine/unpack.cpp?revision=32689&view=markup#l_43 Definition]) | |||
and nextBit | |||
([http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/engines/cine/unpack.h?revision=32689&view=markup#l_69 Declaration], | |||
[http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/engines/cine/unpack.cpp?revision=32689&view=markup#l_52 Definition]) | |||
in Cine's revision 32689). All other chunks in the source stream always contain | |||
full 32 bits but the first chunk contains only 0 to 31 bits. If the first chunk is zero, | |||
then we simply discard it and read another chunk. If the first chunk is not zero, then it | |||
contains as many source bits as its highest set bit's position is (e.g. 100b contains 2 bits | |||
because its highest set bit is at bit position 2, 110101b contains 5 bits, 1b contains 0 bits etc). | |||
So all the less significant bits than the first chunk's highest set bit are valid source stream | |||
bits (That means the highest set bit in the first chunk is always discarded). | |||
=====Command length===== | |||
First bit of a command always tells how many bits the whole command takes. | |||
If the first bit of a command is zero, then the whole command (Including the | |||
first bit) takes two bits total, otherwise it takes three bits (i.e. the first bit is one). | |||
=====Command types===== | |||
There are two possible types of commands: | |||
* unpackRawBytes(N) | |||
** Copies N bytes straight from the source stream and writes them to the destination | |||
* copyRelocatedBytes(OFFSET, N) | |||
** Copies N bytes from position OFFSET in the sliding window (i.e. from the unpacked buffer) | |||
Commands may have predefined values or a restricted value range for some of their parameters. | |||
If a command's parameter isn't a predefined value then it is read from the source stream. | |||
=====All possible commands===== | |||
Here are all the possible commands (Including their first bit that tells the command's length): | |||
<pre> | |||
Bits => Action: | |||
0 0 => unpackRawBytes(3 bits + 1) i.e. unpackRawBytes(1..8) | |||
1 1 1 => unpackRawBytes(8 bits + 9) i.e. unpackRawBytes(9..264) | |||
0 1 => copyRelocatedBytes(8 bits, 2) i.e. copyRelocatedBytes(0..255, 2) | |||
1 0 0 => copyRelocatedBytes(9 bits, 3) i.e. copyRelocatedBytes(0..511, 3) | |||
1 0 1 => copyRelocatedBytes(10 bits, 4) i.e. copyRelocatedBytes(0..1023, 4) | |||
1 1 0 => copyRelocatedBytes(12 bits, 8 bits + 1) i.e. copyRelocatedBytes(0..4095, 1..256) | |||
</pre> | |||
=====Examples of parsing commands===== | |||
Some examples of parsing the commands: | |||
* We read one bit from the stream, it's 0. Okay, so the command has length of two bits. We read the second bit, it's 0, so the command is unpackRawBytes(3 bits + 1). That means we read 3 bits from the source stream, let's call that number X. Then we call unpackRawBytes(X + 1). | |||
* We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 1 and 0 (i.e. 10b). So we've got command copyRelocatedBytes(12 bits, 8 bits + 1). Let's first read 12 bits from the source stream and call that OFFSET. Then let's read 8 bits from the source stream and call that X. Then we'll call copyRelocatedBytes(OFFSET, X + 1). | |||
* We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 1 and 1 (i.e. 11b). So we've got command unpackedRawBytes(8 bits + 9). Let's then read 8 bits from the source stream and call that X. Then we'll call unpackRawBytes(X + 9). | |||
* We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 0 and 0 (i.e. 00b). So we've got command copyRelocatedBytes(9 bits, 3). Let's read 9 bits from the source stream and call that OFFSET. Then we'll call copyRelocatedBytes(OFFSET, 3). | |||
====End of decompression==== | |||
The unpacking ends when we've written <i>unpacked length</i> bytes into the destination buffer. | |||
If at that point the [http://en.wikipedia.org/wiki/XOR exclusive-or] of all the read source stream | |||
chunks together equals the <i>error code</i> from the source stream's header (i.e. their exclusive-or is zero), | |||
that means the packed data is probably intact (Only probably because the exclusive-or of 32-bit chunks isn't a very good | |||
error detection method, it lets errors through relatively easily if compared to other more robust | |||
error detection methods like [http://en.wikipedia.org/wiki/SHA-1 SHA-1] or even [http://en.wikipedia.org/wiki/Cyclic_redundancy_check CRC]). | |||
===Fonts=== | |||
Fonts are loaded from file "TEXTE.DAT". | |||
====TEXTE.DAT file's format==== | |||
<pre> | |||
Byte Meaning | |||
------ ------------------------------------------------------------ | |||
0-1 Entry size (16-bit big endian integer) | |||
2-3 Entry count (16-bit big endian integer) | |||
4-end Font data (4-bit bitplaned data in 16-bit big endian chunks) | |||
------ ------------------------------------------------------------ | |||
</pre> | |||
* Characters are 16x8 in size | |||
* Entry size has been 8 in all observed data | |||
* Entry count divided by entry size gives the number of characters in the font | |||
* Take a look at [http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/engines/cine/gfx.cpp?revision=33976&view=markup#l_1669 gfxConvertSpriteToRaw] to see how the font data can be unpacked | |||
====Known different font versions==== | |||
There are 4 known different font versions: | |||
=====78 characters version===== | |||
* Used by most PC, Amiga and Atari ST versions of Future Wars, but also by Operation Stealth's Amiga demo | |||
* [[Image:Cinematique-78_characters_font.png]] | |||
=====85 characters version===== | |||
* Used by all observed versions of German Future Wars (Amiga and PC), possibly by Spanish Future Wars too | |||
* [[Image:Cinematique-85_characters_font.png]] | |||
=====90 characters version===== | |||
* Used by most PC, Amiga and Atari ST versions of Operation Stealth | |||
* [[Image:Cinematique-90_characters_font.png]] | |||
=====93 characters version===== | |||
* Used by all observed versions of German Operation Stealth (Amiga and PC) | |||
* [[Image:Cinematique-93_characters_font.png]] |
edits