Open main menu

Difference between revisions of "Advanced Detector"

304 bytes added ,  07:58, 29 February 2012
language changes
m (→‎AdvancedMetaEngine: -- formatting)
(language changes)
Line 1: Line 1:
= Advanced Detector =
= Advanced Detector =


If your engine supports a large number of games, or variants then detecting them can be tricky.
If your engine supports a large number of games (or variants of games) then detecting them can be tricky.


As some of the game variants will have files with the same name, but differing contents,  
Since some of the game variants will have files with the same name, but differing contents,
detection by filename alone is not sufficient, and a fair number of variants may only
detection by filename alone is not sufficient, and a fair number of variants may only
differ by small sections of data within the entire file, so opening the file and looking
differ by small sections of data within the entire file, so opening the file and looking
for a "magic header" is not reliable either.<br>
for a "magic header" is not reliable either.
So instead you will take a checksum or even better a hash of the file to detect the exact version, then you will need to write this code to open the files and run this check into your custom MetaEngine...
 
So instead, most engines take a checksum (or even better, a hash) of the file to detect the exact version, and to do that, you would need to write code to open the files and run this check into your custom MetaEngine...


'''Sounds like a lot of work?'''<br>
'''Sounds like a lot of work?'''<br>
Well to avoid every engine author having to do this themselves (and the codebase
Well, to avoid every engine author having to do this themselves (and the codebase
ending up with the maintenance headache of 20+ implementations of this which are almost, but not exactly the same!), the ScummVM Infrastructure Team have provided the '''Advanced Detector'''!<br>
ending up with the maintenance headache of 20+ implementations of this which are almost, but not exactly the same!), the ScummVM Infrastructure Team have provided the '''Advanced Detector'''!
 
This provides a standard framework for filename and MD5 based game detection.
This provides a standard framework for filename and MD5 based game detection.


Line 20: Line 22:
All you will have to provide is a standard data table of ADGameDescription entries describing each game variant, which is usually placed in a separate detection_tables.h header, which is included in detection.cpp for use there.
All you will have to provide is a standard data table of ADGameDescription entries describing each game variant, which is usually placed in a separate detection_tables.h header, which is included in detection.cpp for use there.


This structure plus other parameters are passed to the AdvancedMetaEngine constructor, which can also contain overrides of the default parameters for detection e.g. _md5Bytes is the number of bytes used for the MD5 hash for
This structure plus other parameters are passed to the AdvancedMetaEngine constructor, which can also contain overrides of the default parameters for detection e.g. _md5Bytes is the number of bytes used for the MD5 hash for each file, etc.
each file, etc.


It is suggested you consult the code and header comments in <tt>engines/advancedDetector.*</tt> and look at the examples provided by current engines for a more complete example.
It is suggested you consult the code and header comments in <tt>engines/advancedDetector.*</tt> and look at the examples provided by current engines for a more complete example.
Line 44: Line 45:
What you see here is several sections designated by identifiers in square brackets and set of key/value pairs belonging to each such section.
What you see here is several sections designated by identifiers in square brackets and set of key/value pairs belonging to each such section.


The main section with predefined name 'scummvm' contains global options, which are mainly editable in Options dialog in GUI. Then there go sections for each separate game. Additionally some of ports define their own service sections.
The main section with predefined name 'scummvm' contains global options, which are mainly editable in Options dialog in GUI. Then there are sections for each separate game. Additionally some of ports define their own service sections.


Name of each game is what we are calling '''target'''. Target, which is in the sample above specified as ''monkey2-vga'' is user-editable identifier unique to the specific user, and could be used for launching the game from command line.
Name of each game is what we are calling '''target'''. Target, which is in the sample above specified as ''monkey2-vga'' is user-editable identifier unique to the specific user, and could be used for launching the game from command line.
Line 54: Line 55:
== How Advanced Detector works ==
== How Advanced Detector works ==


Advanced detector tries to match files in probed directory against specified lists of file characteristics provided in an array of ''ADGameDescription'' structures. It takes into account MD5 sum of the file or its first several hundred bytes, its size and name. It creates list of candidates which then it tries to narrow down to a single ''ADGameDescription'' instance unless it is told to do otherwise. In case of disambiguates it returns list of games.
Advanced detector tries to match the files in the probed directory against specified lists of file characteristics provided in an array of ''ADGameDescription'' structures. It can take into account the md5sum of a part of the file (by default, its first several hundred bytes), its size and name. It then creates a list of candidates which then it tries to narrow down to a single ''ADGameDescription'' instance, unless it is told to do otherwise. In case of ambiguous matches, it returns a list of games.
 
It is important to know that currently there are in fact two modes of the Advanced Detector.
 
The first one is used during the game detection when user tries to add a game (detection mode), and the second one when the user launches already detected game (running mode). Both modes call the same method findGames() which could potentially return a list of games.
 
In detection mode, the user is then presented with a list of games to choose from, but in the running mode, if the findGames() method returns more than one game, only first one in the list will be used. This may lead to situation when the game gets detected but doesn't run, thus it is important to test detection and avoid any ambiguous situations.


It is important to know, that currently there are in fact two modes of Advanced Detector. First one is used during the game detection when user tries to add a game (detection mode), and second one when the user launches already detected game (running mode). Both uses call same method findGames() which potentially could return list of games. In detection mode the user is then represented with list of games to choose from, but in the running mode in case findGames() method returns more than one game, only first one in the list will be used. This may lead to situation when the game gets detected but doesn't run, thus it is important to test detection and avoid any disambiguates. This is also the main reason for existing of some features in Advanced Detector which are geared towards resolving such conflicts.
This is also the main reason for some of the features in Advanced Detector which are geared towards resolving such conflicts.


In the running mode Advanced Detector tries to match as many information stored in the config game entry as possible. Typical keys it matches against are '''gameid''', '''platform''' and '''language''', but it may also use '''extra''' when instructed to do so.
In the running mode Advanced Detector tries to match as much information stored in the config game entry as possible. The typical keys it matches against are '''gameid''', '''platform''' and '''language''', but it may also use '''extra''' when instructed to do so.


In case there are no matches in the ''ADGameDescription'' list, there are two additional fallback detection modes. One is file-based detection, which matches just the file names, and second one is a hook which gets called and could contain code of any complexity. Most prominent example of advanced fallback detection is SCI engine.
In case there are no matches in the ''ADGameDescription'' list, there are two additional fallback detection modes. One is file-based detection, which matches just the file names, and second one is a hook which gets called and could contain code of any complexity. The most prominent example of advanced fallback detection is SCI engine.


=== Generated targets ===
=== Generated targets ===
Line 70: Line 77:
</code>
</code>


The target generation is highly affected by AD flags. The flags which have influence are: ADGF_CD, ADGF_DEMO, ADGF_DROPLANGUAGE.  
The target generation is affected by AD flags. The flags which have influence are: ADGF_CD, ADGF_DEMO, ADGF_DROPLANGUAGE.  


== PlainGameDescriptor table ==
== PlainGameDescriptor table ==
Line 81: Line 88:
</syntax>
</syntax>


This table contains all gameids which are known by the engine. Also each gameid contains full human-readable description, which goes to '''description''' field in ScummVM configuration file.
This table contains all gameids which are known by the engine. Also each gameid contains a full human-readable description, which is used to provide the '''description''' field in the ScummVM configuration file.


Only '''gameid''' which are present in this table could be used in ''ADGameDescription'' table.
Only '''gameid'''s which are present in this table could be used in ''ADGameDescription'' table.


Typical PlainGameDescriptor table:
Typical PlainGameDescriptor table:
Line 96: Line 103:
</syntax>
</syntax>


Please note that it is NULL-terminated, and also contains generic gameid ''cine'' which is used by fallback detection.
Please note that it is NULL-terminated, and also contains the generic gameid ''cine'' which is used by fallback detection.


== ADGameDescription table ==
== ADGameDescription table ==
Line 114: Line 121:
</syntax>
</syntax>


'''gameid''' -- This is gameid. Mainly it is used for taking the game description from ''PlainGameDescriptor'' table.
'''gameid''' -- This is the gameid. Mainly it is used for taking the game description from the ''PlainGameDescriptor'' table.


'''extra''' -- This is used to distinguish between different variants of a game. The content of this field is inserted in the generated '''description''' for the config file game entry. In case '''kADFlagUseExtraAsHint''' ADFlag is set, contents of this field is stored in config file, and is used to additionally distinguish between game variants. In case '''ADGF_USEEXTRAASTITLE''' game flag is set, contents of this field will be put into '''description''' rather than one extracted from 'PlainGameDescriptor'' table.
'''extra''' -- This is used to distinguish between different variants of a game. The content of this field is inserted in the generated '''description''' for the config file game entry. In case the '''kADFlagUseExtraAsHint''' ADFlag is set, the contents of this field are stored in the config file, and is used to additionally distinguish between game variants. Also, if the '''ADGF_USEEXTRAASTITLE''' game flag is set, the contents of this field will be put into '''description''' rather than one extracted from 'PlainGameDescriptor'' table.


'''filesDescriptions''' -- list of individual file entries used for detection. 13 files (last is zero terminated) is the max number of files used in ScummVM currently. We are forced to specify hardcoded number due to C++ limitation for defining const arrays.
'''filesDescriptions''' -- a list of individual file entries used for detection. 13 files (last is zero terminated) is the maximum number of files currently used in ScummVM. We are forced to specify a hardcoded number, due to a C++ limitation for defining const arrays.


'''language''' -- language of the game variant.
'''language''' -- language of the game variant.


'''platform''' -- platofrm of the game variant
'''platform''' -- platform of the game variant.


'''flags''' -- game feature flags. Contains both engine-specific ones as well as global ones (see ''ADGameFlags'')
'''flags''' -- game feature flags. Contains both engine-specific ones as well as global ones (see ''ADGameFlags'')
Line 167: Line 174:
== Game Entry flags ADGameFlags ==
== Game Entry flags ADGameFlags ==


Game flags are used to tell the engine which features this particular game has. There are both engine-specific and Advanced Detector-specific game flags. The latter besides being more or less universal also affect the detection behaviour.
Game flags are used to tell the engine which features this particular game has. There are both engine-specific and Advanced Detector-specific game flags. The latter, besides being more or less universal, also affects the detection behaviour.


'''ADGF_ADDENGLISH''' -- Used for dual language games. In this case user will be present with selection between localised and English version of the game. Affects GUIOs.
'''ADGF_ADDENGLISH''' -- Used for dual language games. In this case the user will be presented with a selection between localised and English version of the game. Affects GUIOs.


'''ADGF_CD''' -- Specifies a CD version. Generated target will get '-cd' suffix.
'''ADGF_CD''' -- Specifies a CD version. Generated target will get '-cd' suffix.
Line 175: Line 182:
'''ADGF_DEMO''' -- Specifies a game demo. Generated target will get '-demo' suffix.
'''ADGF_DEMO''' -- Specifies a game demo. Generated target will get '-demo' suffix.


'''ADGF_DROPLANGUAGE''' -- generated target will not have language specified. Used mainly for multilanguage games which have language selector internally. Thus the language will be selected within the game and the setting stored in the config file, but the game entry will stay intact.
'''ADGF_DROPLANGUAGE''' -- the generated target will not have a language specified. Used mainly for multilanguage games which have language selector internally. Thus the language will be selected within the game and the setting stored in the config file, but the game entry will stay intact.


'''ADGF_MACRESFORK'''
'''ADGF_MACRESFORK'''
Line 183: Line 190:
'''ADGF_PIRATED''' -- Specifies a blacklisted game. The game will be detected but refuse to run.
'''ADGF_PIRATED''' -- Specifies a blacklisted game. The game will be detected but refuse to run.


There are known hacked variants for some of the games exist in the wild. We used to ignore user reports on them, but with the number of engines growing it became tough to remember that this particular game is really a hack. If it is widespread enough, we were getting recurrent reports that the game is not detected. To avoid this situation we now accept md5s of such games but mark them accordingly.
There are known hacked variants for some of the games exist in the wild. We used to ignore user reports on them, but with the number of engines growing, it became tough to remember that some particular game is really a hack. When it was widespread enough, we were getting recurrent reports that the game is not detected. To avoid this situation we now accept md5s of such games but mark them accordingly.


'''ADGF_TESTING''' -- Specifies game which was announced for public testing. The user will get relevant warning when launching the game.
'''ADGF_TESTING''' -- Specifies game which was announced for public testing. The user will get a relevant warning when launching the game.


'''ADGF_UNSTABLE''' -- Specifies game which is not publicly supported and is in a heavy development. The user will get relevant warning when launching the game.
'''ADGF_UNSTABLE''' -- Specifies game which is not publicly supported and is in a heavy development. The user will get a relevant warning when launching the game.


'''ADGF_USEEXTRAASTITLE''' -- Instead of '''description''' specified in ''PlainGameDescriptor'' table, '''extra''' field will be used as game description. Good example is AGI fan games where the game title is known but it is not feasible to add it to ''PlainGameDescriptor'' table, or minor composer engine demos with games combined for the same reason.
'''ADGF_USEEXTRAASTITLE''' -- Instead of '''description''' specified in ''PlainGameDescriptor'' table, '''extra''' field will be used as game description. Good example is AGI fan games where the game title is known but it is not feasible to add it to ''PlainGameDescriptor'' table, or minor composer engine demos with games combined for the same reason.
Line 193: Line 200:
== Advanced Detector flags ADFlags ==
== Advanced Detector flags ADFlags ==


'''kADFlagUseExtraAsHint''' -- Specify this flag in situation when there are more than single game stored in the same directory. E.g. there is no way to know which game the user wants to run without asking him. The typical example is VGA version of ''Lure of the Temptress'' which contained both EGA and VGA datafiles in game directory.
'''kADFlagUseExtraAsHint''' -- Specify this flag in situation when there is more than a single game stored in the same directory, with the same gameid. I.e. there is no way to know which game the user wants to run without asking him. The typical example is the VGA version of ''Lure of the Temptress'', which contained both EGA and VGA datafiles in the game directory.


== Upgrading obsolete gameids ==
== Upgrading obsolete gameids ==
Line 210: Line 217:
= AdvancedMetaEngine =
= AdvancedMetaEngine =


Is a generic MetaEngine wrapper aware of Advanced Detector. It should be used whenever AD is used.
Is a generic MetaEngine wrapper which is aware of the Advanced Detector. It should be used whenever AD is used.


== Engine constructor ==
== Engine constructor ==
Line 218: Line 225:
</syntax>
</syntax>


'''descs''' is pointed to list ''ADGameDescription'' structures, or their supersets.
'''descs''' must point to a list of ''ADGameDescription'' structures, or their supersets.


'''descItemSize''' is sizeof of the '''descs''' element used for iterating over it.
'''descItemSize''' is sizeof of the '''descs''' element used for iterating over it.


'''gameids''' is pointed to list of ''PlainGameDescriptor'' structures defining supported gameids.
'''gameids''' must point to a list of ''PlainGameDescriptor'' structures defining supported gameids.


== Additional Advanced MetaEngine parameters ==
== Additional Advanced MetaEngine parameters ==


'''_md5bytes''' -- number of bytes used to compute md5. If set to 0 then whole file will be used. Beware of long files though since that can dramatically slow down the detection. Typically sane value is 5000 bytes, but often experiment is required for many game variants with subtle differences.
'''_md5bytes''' -- number of bytes used to compute md5. If set to 0 then whole file will be used. Beware of doing this if your detection might encounter large files, since that can dramatically slow down the detection. Typically a sane value is 5000 bytes (the default), but often experimentation is required for many game variants with subtle differences.


'''_singleid''' -- Used to override gameid. A recommended setting to prevent global gameid pollution. With this option set, the gameid effectively turns into engineid.
'''_singleid''' -- Used to override gameid. A recommended setting to prevent global gameid pollution. With this option set, the gameid effectively turns into engineid.
193

edits