Cine/Specifications

From ScummVM :: Wiki
< Cine
Revision as of 08:28, 18 June 2008 by Buddha^ (talk | contribs) (Rewording of the "Files without any extension" part.)
Jump to navigation Jump to search

Cine Specifications (version ε)

This is a place intended for information about the Cinématique engine's internals, file formats etc. Adding any additional information here is encouraged. Information probably is inaccurate in places (e.g. may be specific to Operation Stealth rather than apply to Future Wars and/or Cruise For A Corpse too) so if you have more accurate information, please add it here.

File formats

.ADL files

Adlib music.

.AMI files

NOTE: This file type seems to be specific to Amiga versions of Operation Stealth. Not sure if Future Wars uses this.

It looks like *.AMI files are samples (Loaded a couple of them into Audacity using raw 8-bit signed audio data import and played them and they sounded alright).

.ANI files

Animation.

.H32 files

Possibly Roland MT-32 music?

.HP files

Maybe something to do with Roland MT-32? Don't know really.

.IST files

NOTE: These seem to be specific to the PC versions of Operation Stealth. Don't know about Future Wars.

All filenames of this type start with "BOND" and the files are 15 bytes in size. Here's a list of all the 12 different IST-files I've seen so far:

  • BOND1.IST, BOND4.IST, BOND10.IST, BOND11.IST, BOND14.IST, BOND16.IST, BOND20.IST, BOND22.IST, BOND23.IST, BOND24.IST, BOND31.IST, BOND32.IST

These probably have something to do with Roland MT-32 music as there are corresponding files without the IST extension as well (BOND1, BOND4, ..., BOND32) and the corresponding files without the IST extension contain names of .H32 files (e.g. DEEPSNAR.H32, ELECGTR1.H32, SLAPBAS1.H32, STRSECT1.H32) in addition to other data.

.MIG files

"INTRO.MIG" is the only file I've seen of this type (And that's with Operation Stealth, I'd guess Future Wars hasn't got any files with this extension but that's only a guess).

I'd guess "INTRO.MIG" is Operation Stealth's introduction sequence's music file.

Here are the MD5 checksums of the two versions of the INTRO.MIG file I've seen:

  • 65613c9f127ecf8bd7988aef7e49b698 (1624 bytes, Amiga versions of Operation Stealth)
  • efb187353d289d9b85e1e455734e38f9 (3424 bytes, PC and Atari ST versions of Operation Stealth)

The Amiga version's file starts with the characters "SONG" and then a bit later there reads "A_COEUR.AMI" (AMI files seem to be samples). In the PC and Atari ST versions' file there reads "COEUR___.H32" and values "02", "03", ..., "15". BTW coeur is French and means heart.

.MSG files

A formatted file containing a list of strings.

.MSK files

An image mask (1 bit per pixel).

.OBJ files

A formatted file containing descriptions of objects in a scene. Each object has info about X, Y, Mask, Frame, Costume, Name and Part.

.PI1 files

I have no idea what these do.

.PRC files

Script files. PRC = Procedure?

.REL files

Script files. REL = Relation?

.SEQ files

Some type of animation data.

.SET files

An image set.

.SPL files

NOTE: This file type seems to be specific to Amiga and Atari ST versions of Operation Stealth. Not sure if Future Wars uses this.

Some type of animation data.

Files without any extension

Examples:

  • "BOND1", "BOND32", "CELLO2", "PRO1002", "FUZZGUIT", "ETOMSIMO"

Maybe those PRO* files without an extension are still script files? And CELLO2 sounds like something music related.

The BOND* files seem to have something to do with music. For more info on them take look at the description of the .IST files.

Part file format

NOTE: This applies to both Future Wars and Operation Stealth. 

Part file's start:

    Byte  Meaning
    ----- -----------------------------------------------------------
     0-1  Number of elements in this part file (Uint16BE)
     2-3  Entry size (Uint16BE). Normally 0x1E i.e. 30
    ----- -----------------------------------------------------------

Then comes info for each element (Entry size in length each): 

    Byte  Meaning
    ------ -----------------------------------------------------------
     0-13  Name (ASCIIZ string)
    14-17  Data's starting offset in this part file (Uint32BE)
    18-21  Packed size (Uint32BE)
    22-25  Unpacked size (Uint32BE)
    26-29  ???
    ------ -----------------------------------------------------------

After that it's the data for all the elements contained in this part file.

VOL.CNF

NOTE: This file is specific to Operation Stealth.

This file contains list of resource files (e.g. PROCS10, RSC04, SONS2, LABYBASE etc) and a list of files contained in each resource file (e.g. AUTO00.PRC and MASKG.REL in PROCS00).

If the file starts with "ABASECP" the file is compressed, otherwise it's uncompressed. 

Compressed file's start:

    Byte  Meaning
    ----- -----------------------------------------------------------
     0-7  Magic header ("ABASECP" string with the trailing zero)
     8-11 Unpacked size of the data after this header (Uint32BE)
    12-15 Packed size of the data after this header (Uint32BE)
    ----- -----------------------------------------------------------

For a compressed file just read the rest of the file after the header and uncompress it with Delphine's unpacking routine and you've got the same data as you'd have had if you'd have had an unpacked file to start with (Except the header, of course). 

Uncompressed file's start:

    Byte  Meaning
    ----- -----------------------------------------------------------
     0-1  Resource files count (Uint16BE)
     2-3  Entry size (Uint16BE). 0x14 in all tested files so far
    ----- -----------------------------------------------------------

Then come resource files count times resource file info structs of entry size bytes length each: 

Resource file info struct:
x = entry size - 1

    Byte  Meaning
    ----- -----------------------------------------------------------
     0-7  Resource file name string (Possibly no trailing zero!)
     8-x  Unknown data
    ----- -----------------------------------------------------------

Then for each resource file comes a list of files that are in it.

Each list begins with an unsigned 32-bit big endian integer telling the size of the entry. After that come (size / 11) or (size / 13) filenames of length 11 or 13 respectively. Almost as a rule compressed files use filename length of 11 and uncompressed files use filename length of 13 but it's not always so (At least some Amiga version used a compressed 'vol.cnf' file but still used filenames of length 13).

Filenames of length 11 have no separation of the extension and the basename so that's why we have to convert them first. There's no trailing zero in them either and they're always of the full length 11 with padding spaces. Extension can be always found at offset 8 onwards. Filenames of length 13 are okay as they are, no need for converting them. 

Examples of filename mappings:

    "AEROPORTMSG" -> "AEROPORT.MSG"
    "MITRAILLHP " -> "MITRAILL.HP" (Notice the trailing space after the extension)
    "BOND10     " -> "BOND10"
    "GIRL    SET" -> "GIRL.SET"

Compression format

The compression algorithm used by all Delphine's adventure games uses sliding window compression (Quite like LZ77) combined with a fixed non-adaptive entropy coding scheme (Not of any type I could recognize).

The compressed data is in big endian 32-bit chunks, working backwards from the buffer's end. So we start from the data's end and work backwards. Also outputting the unpacked data is done backwards, starting from the destination buffer's end and working backwards byte by byte. 

Compression format:

NOTE: As the whole data consists of unsigned big endian 32-bit integers, I use indexing
      in 32-bit addresses here. By -1 I mean the last 32-bits of the data
      (i.e. bytes src[srcLen-4], src[srcLen-3], src[srcLen-2] and src[srcLen-1]),
      by -2 the second to last 32-bits etc.

    Dword     Meaning
    --------- ------------------------------------------------------------------------
    -1        Unpacked length (Uint32BE).
    -2        Error code (Uint32BE). Xor of the whole packed data in Uint32BE chunks.
    0 - (-3)  The packed data (In Uint32BE chunks).
    --------  ------------------------------------------------------------------------

Bit sequences in the compressed stream

The whole packed data consists of commands and their parameters.

The source stream is read as a bit stream. Behind the scenes it is read in one unsigned big endian 32-bit integer value at a time and when such a chunk becomes depleted of bits then another chunk is read in etc. Each chunk's least significant bit is the first bit read from it, the most significant bit is the last bit read from it etc.

First 32-bit chunk

Because of the way the Delphine's decompressor routine handled the bit stream the first chunk acts in a sort of a peculiar way (Take a look at the functions rcr (Declaration, Definition) and nextBit (Declaration, Definition) in Cine's revision 32689). All other chunks in the source stream always contain full 32 bits but the first chunk contains only 0 to 31 bits. If the first chunk is zero, then we simply discard it and read another chunk. If the first chunk is not zero, then it contains as many source bits as its highest set bit's position is (e.g. 100b contains 2 bits because its highest set bit is at bit position 2, 110101b contains 5 bits, 1b contains 0 bits etc). So all the less significant bits than the first chunk's highest set bit are valid source stream bits (That means the highest set bit in the first chunk is always discarded).

Command length

First bit of a command always tells how many bits the whole command takes. If the first bit of a command is zero, then the whole command (Including the first bit) takes two bits total, otherwise it takes three bits (i.e. the first bit is one).

Command types

There are two possible types of commands:

  • unpackRawBytes(N)
    • Copies N bytes straight from the source stream and writes them to the destination
  • copyRelocatedBytes(OFFSET, N)
    • Copies N bytes from position OFFSET in the sliding window (i.e. from the unpacked buffer)

Commands may have predefined values or a restricted value range for some of their parameters. If a command's parameter isn't a predefined value then it is read from the source stream.

All possible commands

Here are all the possible commands (Including their first bit that tells the command's length): 

Bits  => Action:
0 0   => unpackRawBytes(3 bits + 1)              i.e. unpackRawBytes(1..8)
1 1 1 => unpackRawBytes(8 bits + 9)              i.e. unpackRawBytes(9..264)
0 1   => copyRelocatedBytes(8 bits, 2)           i.e. copyRelocatedBytes(0..255, 2)
1 0 0 => copyRelocatedBytes(9 bits, 3)           i.e. copyRelocatedBytes(0..511, 3)
1 0 1 => copyRelocatedBytes(10 bits, 4)          i.e. copyRelocatedBytes(0..1023, 4)
1 1 0 => copyRelocatedBytes(12 bits, 8 bits + 1) i.e. copyRelocatedBytes(0..4095, 1..256)
Examples of parsing commands

Some examples of parsing the commands:

  • We read one bit from the stream, it's 0. Okay, so the command has length of two bits. We read the second bit, it's 0, so the command is unpackRawBytes(3 bits + 1). That means we read 3 bits from the source stream, let's call that number X. Then we call unpackRawBytes(X + 1).
  • We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 1 and 0 (i.e. 10b). So we've got command copyRelocatedBytes(12 bits, 8 bits + 1). Let's first read 12 bits from the source stream and call that OFFSET. Then let's read 8 bits from the source stream and call that X. Then we'll call copyRelocatedBytes(OFFSET, X + 1).
  • We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 1 and 1 (i.e. 11b). So we've got command unpackedRawBytes(8 bits + 9). Let's then read 8 bits from the source stream and call that X. Then we'll call unpackRawBytes(X + 9).
  • We read one bit from the stream, it's 1. So the command has length of three bits. We read two more bits, they're 0 and 0 (i.e. 00b). So we've got command copyRelocatedBytes(9 bits, 3). Let's read 9 bits from the source stream and call that OFFSET. Then we'll call copyRelocatedBytes(OFFSET, 3).

End of decompression

The unpacking ends when we've written unpacked length bytes into the destination buffer. If at that point the exclusive-or of all the read source stream chunks together equals the error code from the source stream's header (i.e. their exclusive-or is zero), that means the packed data is probably intact (Only probably because the exclusive-or of 32-bit chunks isn't a very good error detection method, it lets errors through relatively easily if compared to other more robust error detection methods like SHA-1 or even CRC).

Retrieved from "https://wiki.scummvm.org/index.php?title=Cine/Specifications&oldid=9123"