|
|
| Line 1: |
Line 1: |
| === The problem === | | === Introduction === |
| Right now, the SCUMM engine suffers from an abundance of different game targets. We distinguish multiple game targets for many games. Of course this has very good and valid technical reasons, but from the users perspective, it is a nuisance.
| | This page talks about auto detection of games in ScummVM. It's mostly written from the perspective of the SCUMM engine, but a lot said here should hold true for arbitrary engines. |
|
| |
|
| Furthermore, we have a very good MD5 based auto detection system; but if the MD5 of a game is not known, we fall back to an extremly simplistic file name based detection scheme. This approach is very limited, e.g. "00.LFL" is a file name which implies a multitude of different possible games (Maniac Mansion, Zak, Indy3EGA, Indy3VGA, Indy3Towns, Loom, Loom Towns).
| |
|
| |
|
| In order to detect based on filenames, we have to generate all possible file names; some code duplication exists here between the detector and the openRoom() method; this can lead to problems if the code doesn't perfectly match (a game might be detectable, but later not usable, or the other way around; see the problems with "monkey" vs. "monkey1").
| |
|
| |
|
| Also some of the games are avaible in multiple versions which each use different filenames; most notably, the Mac versions of many games use long verbose filenames. We currently deal with these using the generateSubstResFileName() method. This works, but means yet another translation table is needed and has to be maintained.
| | === Detection vs. Instantiation === |
| | Insight: 'detect' and 'create' are extremly similar. The main difference is in their precise input and output; what happens internally, is actually very similar. |
|
| |
|
| Then, the MD5 system is not fully unified; for some games, we already distinguish multiple variants based on the MD5, so we have two seperate MD5 tables. This is not a major problem, but not nice either, and certainly somewhat arbitrary and awkward.
| | 'create' receives some hints (gameid, platform, language), and using those |
| | plus MD5 data, has to figure out for a given path which (unique) game settings are appropriate for that path. In other words, it returns either a single game setting, or none at all. |
|
| |
|
| To sum it up:
| | 'detect' is pointed at a directory, but given no further hints. It has to identify all (potential) games in that directory and returns a list of hints usable by 'create' (i.e. gameid, platform, language). |
| * Too many targets, confusing our users
| |
| * MD5 system is split, only one part of it is generated from a common source file, the rest is maintained manually
| |
| * Limited detection capabilities if no MD5 is present
| |
| * File names are generated in at least two different places in the code, resulting in potential inconsistencies
| |
|
| |
|
| | Seeing that, we should be able to combine the core parts of the two algorithm. For a 'create', we essentially run a 'detect' and compare the given game hint with the game hint(s) returned by 'detect', and if a unique match exists, proceed with that. |
|
| |
|
| | As a further consequence, gameids actually become optional: In some cases (in particular, in case of an MD5 match), ScummVM is able to determine the precise game settings given only the path to the game data and nothing else! |
|
| |
|
| === Upgrading config ===
| |
| Upgrading should be smooth and easy. Existing config files should continue to work, and be updated automatically. So we need to put into place a system which translates old target names into new ones. That shouldn't be too hard. We could even automatically update the config file, although that would mean making it unusable for older ScummVM versions, so I'd rather not do it unless we have to. It's fine to add additional keys, though, e.g. platform/language settings.
| |
|
| |
|
| A simple way to allow such backward compatibility: simply provide a table which maps old targets to the new ones, possibly containing some extra hinting info.
| |
|
| |
|
| | === Detection in the SCUMM engine === |
| | What was said in the previous section can be used to improve the SCUMM detection code. |
| | To this end, we have to restructure and unify some of our current data sets. Right now, we use MD5 checksums in the SCUMM engine for two different purposes: |
| | * Mapping an MD5 to a triple of (gameid, platform, language) |
| | * Mapping an MD5 to a specific game instances, i.e. |
| | (game desc, SCUMM/HE version, feature flags, platform, language, etc.) |
|
| |
|
| === Target descriptions ===
| | The SCUMM engine is also troubled by the fact that it currently uses the gameid to get an initial guess for the gamesettings. However, that guess can be quite inaccurate; e.g. for Monkey Island, it defaults to SCUMM version 4, even though some versions of MI are version 5 games. This leads to some annoying problems. |
| If we reduce the number of game descriptions, we'd also loose some variety in the game descriptions; not a terrible problem, but annoying. However, it would be easy to allow a system were game variants can extend the base game description. In fact, multiple_versions_md5_settings already allows for this. But instead of specifying a full new desc string for each variant, we'd just specify an (optional) addition.
| |
| Example: All indy3 variants share the base description "Indiana Jones and the Last Crusade". The EGA version could add "(EGA)", the VGA version "(VGA)", the FM-TOWNS version... you get the idea.
| |
|
| |
|
| So, we would specify the description in three ways:
| |
| # The base target ("indy3") carries the base desc
| |
| # Every MD5 table entry can optionally augment the desc with a string; that string is put into paranthesis and added to the end of the base desc, whenever the game detector assembles a GameSettings record for that game
| |
| # For non-MD5 detection, we can add a table which allows differentiating based on whether the file is OLD_BUNDLE vs. SMALL HEADER vs. MODERN; and of course based on the actual file names.
| |
|
| |
|
| | So, instead of having a table that maps a gameid to a single game setting struct, we could use a one-to-many relation instead. |
|
| |
|
| === Old bundle, USE_KEYS ===
| | More specifically: For each gameid, we would allow multiple different game settings. To the end user, everything remains transparent, because each game id still maps to a unique (basic) game description. The "multiple settings" would be internal only. |
| Some games exist in both old bundle and small header variants. Distinguishing isn't hard, we just have to look at the first few bytes of 00.LFL.
| |
|
| |
|
| Also, some game variants use XOR encrypted data files, some don't; in particular indy3ega does, while the VGA version don't use encryption. We could either try to automatically detect this
| | We then would extend the MD5Table table struct by (1) a way to specify which of the multiple variants for the game settings is meant, and (2) a field for an "extra description" string, like "CD", Floppy", "VGA", etc. (see also the section on "Standardized descriptions" below). |
|
| |
|
| | With this, it would be possible to get rid of multiple_versions_md5_settings again. (Well, actually, my explanations here are probably a bit sketchy and incomplete. Feel free to ask me, Fingolfin, for clarifications). |
|
| |
|
| === File names ===
| |
| Another note: We already support a "basename" config file entry, and base the "_gameName" on this. And in addition, we also have generateSubstResFileName. These should be unified, somehow.
| |
|
| |
|
| Now, for the mapping of filenames:
| |
| for the detection part, we could use a table like this:
| |
| string "filename" is mapped to a struct
| |
| struct {
| |
| String target; // E.g. "indy3"
| |
| String additionalDesc; // e.g. "EGA" or "Demo" or empty
| |
| int heVersion; // if not equal to -1: alternative HE version
| |
| };
| |
|
| |
|
| Of course, multiple entries may be present for a single filename; like for "00.LFL", we would have several entries.
| | === Detection done without MD5 === |
| | MD5 detection is a very powerful tool. However, it's an all-or-nothing approach: Either we know a game variant and its MD5, or we don't. If we relied on MD5 based detection exclusively, then that would shut out users of e.g. fan made translations, or so far unknown game variants. |
|
| |
|
| | | Hence, other means are necessary. Some thoughts on that follow. |
| TODO: For detection, we only need one name, e.g. "samnmax.sm0"; but to actually use the game, we often need a pattern (in this case, "samnmax.sm%d". How to handle this best
| |
| | |
| TODO: Instead of forgetting the file name we found during detection, it would be clever to remember that name, somehow. Maybe a "basename" setting like in the config?
| |
| openRoom() should take advantage of this; also the code in sound.cpp (and anything else which uses generateSubstResFileName right now).
| |
| | |
| | |
| === Detection ===
| |
| | |
| Detection should be able to take "hints" from the user; i.e. if "indy3" is specified, then we'll assume any 00.LFL we encounter to be part of "indy3".
| |
|
| |
|
| Files can be distinguished by their content as follows: | | Files can be distinguished by their content as follows: |
| Line 75: |
Line 53: |
| * For *.he0 files: Apprently they start with MAXS, encoded; sometimes also with RNAM. | | * For *.he0 files: Apprently they start with MAXS, encoded; sometimes also with RNAM. |
|
| |
|
| === Proposals ===
| |
|
| |
|
| I have no complete proposal yet, only various ideas for parts of the problem; I am still working on putting them all together into one coherent solution, so bear with me; this is all relatively new.
| | |
| | === Standardized descriptions === |
| | The following isn't quite about auto detection, but about a related subject: How to canonically construct a game description string from the following information tuple: |
| | (NAME, PLATFORM, LANGUAGE, EXTRA) |
| | |
| | The game desc then is: |
| | NAME (PLATFORM EXTRA LANGUAGE) |
| | |
| | If PLATFORM = PC, treat it as empty. If LANGUAGE=English, treat it as empty. If all three are missing, the desc becomes simply |
| | NAME |
| | |
| | For the language, the two letter abbreviation is used. For the platforms, the following short names are used: |
| | PC, DOS, Win, Mac, Amiga, Atari, FM-TOWNS |
| | |
| | Typical values for EXTRA include "Demo", "VGA", "EGA", "CD", "Floppy", "Talkie" |
| | |
| | For example: |
| | ("Loom", Macintosh, "", Unknown) |
| | becomes |
| | Loom (Mac) |
| | |
| | Another example: |
| | ("The Secret of Monkey Island", PC, "CD", French) |
| | becomes |
| | The Secret of Monkey Island (CD fr) |