Difference between revisions of "HOWTO-Engines"

From ScummVM :: Wiki
Jump to navigation Jump to search
m (Mention new optional engine skeleton style)
 
(85 intermediate revisions by 20 users not shown)
Line 2: Line 2:
This page is meant as a mini-HOWTO which roughly outlines the steps needed to add a new engine to ScummVM. It does '''not''' tell you how to create an engine for a given game; rather it is meant to tell a developer how to properly "hook" into ScummVM.  
This page is meant as a mini-HOWTO which roughly outlines the steps needed to add a new engine to ScummVM. It does '''not''' tell you how to create an engine for a given game; rather it is meant to tell a developer how to properly "hook" into ScummVM.  


I will assume that you are at least roughly familiar with ScummVM, and have a fresh checkout of our Subversion repository. Note that it's strongly adviced to base your work on the current development version of ScummVM, and not on a release version. This will ease integration of your work.
I will assume that you are at least roughly familiar with ScummVM, and have a recent checkout of our source code repository. Note that it's strongly advised to base your work on the current development version of ScummVM, and not on a release version. This will ease integration of your work.
 


== Overview ==
== Overview ==
Essentially, you will have to implement a subclass of the Engine class. Our Doxygen documentation is your friend and should hopefully explain enough about this, see here: http://scummvm.org/docs/doxygen/html/class Engine.php.
Essentially, you will have to implement a subclass of the Engine class. Our Doxygen documentation is your friend and should hopefully explain enough about this: [https://doxygen.scummvm.org/d8/d9b/class_engine.html Engine class].  
 
You also must hook yourself into the regular ScummVM main build system. Actually, some ports use custom build system, but their maintainers will usually add your new engine once it has been added to ScummVM.


Finally, you need to make ScummVM aware of your new engine by updating a couple source files (see below).
You also must hook yourself into the regular ScummVM main build system. Actually, some ports use a custom build system, but their maintainers will usually add your new engine once it has been added to ScummVM.


Finally, you need to make ScummVM aware of your new engine.


== Steps ==
== Automated Method ==
In the following I assume your engine is called "quux".
There is a tool that automates creating new skeleton engines which works both with make-based systems like *nix/Mac and with Microsoft Visual Studio.


# Add a new directory <tt>engines/quux/</tt>
=== Make-based Systems ===
# Add <tt>engines/quux/module.mk</tt> (take a look at module.mk files of 2-3 existing engines to understand the content).
Run <tt>make devtools/create_engine</tt>, then run the <tt>devtools/create_engine/create_engine <engine name></tt>, where <tt>engine name</tt> is the desired engine name.
# Add <tt>engines/quux/quux.h</tt> and <tt>engines/quux/quux.cpp</tt>; this will contain your Engine subclass (or at least parts of it). It will also contain the plugin interface code (more on that in the next section).
# Modify <tt>engines/module.mk</tt> by adding your engine. It should be clear what to do by looking at what is done for the other engines there.
# Modify <tt>configure</tt>; you'll have to add your engine in multiple places. Again, just check out what is done for the existing engines.
# Modify <tt>base/plugins.cpp</tt>; in particular, you have to add your engine to the list in PluginManager::loadPlugins.


That's it. The difficult part is of course writing the Engine subclass. More on that in the next section!
=== Visual Studio ===


Important note: Use a C++ namespace for all your work, e.g. "namespace Quux" in this case.
If you go to the devtools/create_engine/ folder, you'll see a create_engine.sln solution which you should open and compile. This will create a create_engine.exe executable in the same folder. From the command line prompt, you should then run create_engine with the desired engine name provided as a single parameter. This will create a folder for your new engine as well as set up a batch file in the dists/msvc/ folder with the engine name that you can use as a shortcut to create a ScummVM solution with only that engine enabled.


=== Common Instructions ===
There are two styles of engine skeletons supported. The first, if you just specify an engine name, creates a bare bones engine with a sample event loop. The second kind is selected if you also specify -events as an extra parameter after the engine name. In this case, it creates a more sophisticated engine skeleton that has a single centralized event loop, and dispatches events to views, which can be easily switched between. This has benefits of avoiding duplicating event loops in multiple different places, each handling event processing and quit checks. Each view simply has to implement a drawing method and override different event handler methods to handle whichever events it needs.


=== Subclassing Engine ===
In either case, once you've got the skeleton engine compiling, your next step should be to update the placeholder detection entry in detection_tables.h. The easiest way is to choose a file in your game's folder that you think is unique, and change the filename in the detection entry to it. If you then run ScummVM and try to add the game, it should prompt you that an unrecognised md5 for the game has been found, and give you the option to copy it to clipboard. You can do so, and then paste it to extract the md5 and filesize for the file, which can be used to update the detection entry. With this done you should be able to add your game, and run the skeleton engine.
TODO: We should probably give some sample code, maybe even provide a full (empty) Engine demo class. Maybe even provide a real mini engine project somewhere on our site which demonstrates using events, drawing, etc. ? Not sure whether this would be worth the effort, though.


TODO: At the very least, describe the plugin interface: I.e. which functions *must* be implemented, and what they are supposed to do. Once again, sample code would be nice.
As a second optional step, by default everytime your game starts, you'll get an unsupported game warning dialog appear. If you want to suppress it, locate and open your scummvm.ini file, and find the section added for your game. Then add the following line to it:
<syntaxhighlight lang="bash">
enable_unsupported_game_warning=false
</syntaxhighlight>


== General Conventions ==


=== Example: engines/quux/quux.h ===
=== File name conventions ===
<pre>
Over the course of its existence, many people will have to deal with the source code of a given engine. Be it to fix bugs in it, modify it to still compile after changes made to the backend code, or to simply add new functionality. It is therefore expected to adhere to various conventions used throughout all engines. Besides source code conventions (see [[Code Formatting Conventions]]), this affects filenames. We suggest you use the following names for specific parts of your engine:
#ifndef QUUX_H
#define QUUX_H


#include "base/engine.h"
{| border="1" cellpadding="2" width=100%
|-
|ENGINENAME.h || Contains your (primary) Engine subclass.
|-
|ENGINENAME.cpp || Contains at least the constructor and destructor of your (primary) Engine subclass, as well as the implementations of the mandatory (i.e. pure virtual) Engine methods.
|-
|detection.cpp || Code related to game detection.
|-
|metaengine.cpp || Contains the implementation of the plugin interface, as described in <code>base/plugins.h</code>.
|-
|saveload.cpp || Code related to savegames
|-
|console.cpp || (console) debugger
|-
|gfx.cpp (alt: graphics.cpp) || Graphics code
|-
|sound.cpp || Sound code
|-
|music.cpp || Music code
|-
|inter.cpp, logic.cpp, script.cpp || Game logic, resp. script/bytecode interpreter
|}


namespace Quux {
Additionally, the files saved by each engine should be consistent.


// our engine debug levels
* '''Saves''': If you use the new loadGameStream & saveGameStream methods, these will automatically be created in the form of <targetid>.### (where ### is the slot id). If you want saves to be consistent across all game variants, you should instead provide your own implementation of loadGameState and saveGameState, and create savefiles using the format <gameid>.###.
enum {
kQuuxDebugExample = 1 << 0,
kQuuxDebugExample2 = 1 << 1
// next new level must be 1 << 2 (4)
// the current limitation is 32 debug levels (1 << 31 is the last one)
};


class QuuxEngine : public Engine {
* '''Other files''': These should be named <targetid>-<filename> or <gameid>-<filename>. Again the latter when the files can be shared across all variants of the game (an example for these type of files would be when a minigame saves a high score record).
public:
QuuxEngine(OSystem *syst);
~QuuxEngine();


virtual int init(GameDetector &detector);
Here, only use the <gameid> scheme if you are '''absolutely''' sure that such files can be shared across '''all''' (that is every game platform, every game language, every game patch version, every game release, etc.) versions. If you are not sure whether this is the case, then stick to the <targetid> based scheme.
virtual int go();
};


} // End of namespace Quux
=== Subclassing AdvancedMetaEngine ===
Let's implement the plugin interface:<br>
You'll have to create a custom AdvancedMetaEngine subclass. This provides the information and functionality related to the engine that can be used by the launcher without loading and running the game engine, which includes defining keymaps and achievements, listing savegames, and instancing the engine. You'll also have to create a custom AdvancedMetaEngineDetection subclass. This provides the information related to detecting games, which is always included in the main ScummVM executable regardless of whether or not the engine is enabled.


#endif
The following example illustrates this. It contains the necessary fundamentals of the details of the games and the code to create the engine, as well as REGISTER macros that register the meta engine with ScummVM. For the Quux example, If you create an empty file named quux.txt, the engine will detect it.
</pre>


=== Example: engines/quux/quux.cpp ===
=== Subclassing Engine ===
<pre>
The generated by the create_engine code gives a simple example of an engine. It contains a few important points:
#include "common/stdafx.h"
* It initializes the screen at a given resolution
 
* It creates a debugger class and registers it with the engine framework. Currently, it's only a skeleton defined in <engine>.h. For a full game, you'd implement it in its own debugger.cpp and .h files
#include "backends/fs/fs.h"
* It has a simple event loop
#include "base/gameDetector.h"
* It also demonstrates how to read and write savegames
#include "base/plugins.h"
 
#include "quux/quux.h"
 
 
static const GameSettings quux_setting[] = {
{ "quux", "Quux the Example Module", 0 },
{ "quuxcd", "Quux the Example Module (CD version)", 0 },
{ 0, 0, 0 }
};
 
GameList Engine_QUUX_gameList() {
GameList games;
const GameSettings *g = quux_setting;
while (g->gameid) {
games.push_back(*g);
g++;
}
 
return games;
}
 
DetectedGameList Engine_QUUX_detectGames(const FSList &fslist) {
DetectedGameList detectedGames;
 
// Iterate over all files in the given directory
for (FSList::const_iterator file = fslist.begin(); file != fslist.end(); ++file) {
if (!file->isDirectory()) {
const char *gameName = file->displayName().c_str();
 
if (0 == scumm_stricmp("README", gameName)) {
// You could check the contents of the file now if you need to.
detectedGames.push_back(quux_setting[0]);
break;
}
}
}
return detectedGames;
}


Engine *Engine_QUUX_create(GameDetector *detector, OSystem *syst) {
Miscellaneous important: If you end up using the ScummVM GUI manually in your game (g_gui and stuff) you have always to call g_gui.handleScreenChanged() if you received a OSystem::EVENT_SCREEN_CHANGED event, else it could be that your gui looks strange or even crashes ScummVM.
// At this point you may want to perform a sanity check on the
// values in 'detecetor'.
return new Quux::QuuxEngine(syst);
}


REGISTER_PLUGIN(QUUX, "Quux the Example Module")
For opening files in your engine, see the [[HOWTO-Open Files|how to open files]] page.


=== Infrastructure services ===
Header file <code>common/scummsys.h</code> provides services needed by virtually any source file:


namespace Quux {
* defines platform endianness
* defines portable types
* defines common constants and macros


Moreover, it deals with providing suitable building environment for different platforms:


QuuxEngine::QuuxEngine(OSystem *syst)
* provides common names for non-standard library functions
: Engine(syst) {
* disables bogus and/or annoying warnings
// Put your engine in a sane state, but do nothing big yet;
* provides a lean environment to build win32 executables/libraries
// in particular, do not load data from files; rather, if you
// need to do such things, do them from init().


// Do not initialize graphics here
The <code>common</code> directory contains many more useful things, e.g. various container classes (for lists, hashmaps, resizeable arrays etc.), code for dealing with transparent endianess handling, for file I/O, etc. We recommend that you browse this a bit to get an idea of what is there. Likewise you should familiarize yourself with <code>graphics</code> and <code>sound</code>; for example we already have decoders for quite some audio formats in there. Before you roll your own code for all sorts of basic things, have a look and see if we already have code for that, and maybe also ask some veterans for advice.


// However this is the place to specify all default directories
=== Common portability issues ===
File::addDefaultDirectory(_gameDataPath + "sound/");
There are wrapper around number of non-portable functions. These are:


// Here is the right place to set up the engine specific debug levels
  max() -> MAX()
Common::addSpecialDebugLevel(kQuuxDebugExample, "example", "this is just an example for a engine specific debug level");
  min() -> MIN()
Common::addSpecialDebugLevel(kQuuxDebugExample2, "example2", "also an example");
  rand() -> use Common::RandomSource class
  strcasecmp() / stricmp() -> scumm_stricmp()
  strncasecmp() / strnicmp() -> scumm_strnicmp()
  strrev() -> scumm_strrev()


printf("QuuxEngine::QuuxEngine\n");
Also we have predefined common integer types. Please, use them instead of rolling your own:
}


QuuxEngine::~QuuxEngine() {
  byte
// Dispose your resources here
  int8
printf("QuuxEngine::~QuuxEngine\n");
  uint8
  int16
  uint16
  int32
  uint32


// Remove all of our debug levels here
Do not use the following functions:
Common::clearAllSpecialDebugLevels();
}


int QuuxEngine::init(GameDetector &detector) {
  sprintf -> snprintf
// Initialize graphics using following template
  strcpy -> strncpy or Common::strlcpy
// You have to use transactions
_system->beginGFXTransaction();
// This is handled by base Engine class and processes command
// line parameters
_vm->initCommonGFX(detector);


// Specify dimensions of game graphics window
Additionally ScummVM offers a way of recording all events and then playing them back on request. That could be used for "demoplay" mode. But to ensure that it will work for your engine, you have to register your RandomSource class instance as demonstrated in the generated <engine>.cpp file.
_system->initSize(width, height);
_system->endGFXTransaction();
// Additional setup.
printf("QuuxEngine::init\n");
return 0;
}


int QuuxEngine::go() {
=== True RGB color support ===
// Your main even loop should be (invoked from) here.
If you need to support more than 256 colors in your game engine, please refer to [[API-Truecolor|the truecolor API reference page]] for specifications and initialization protocol.
printf("QuuxEngine::go: Hello, World!\n");


// This test will show up if -d1 and --debugflags=example are specified on the commandline
debugC(1, kQuuxDebugExample, "Example debug call");


// This test will show up if --debugflags=example or --debugflags=example2 or both of them and -d3 are specified on the commandline
debugC(3, kQuuxDebugExample | kQuuxDebugExample2, "Example debug call two");


return 0;
== Extended Saves ==
}


To easily support ScummVM-specific metadata in the saves, we implemented the ExtendedSaves interface.


This basically makes your saves look like this:


} // End of namespace Quux
  &lt;DATA>    Engine-specific save data
</pre>
  <METAINFO> ScummVM metainfo, including play time, screenshot, save name etc
  <OFFSET>   4-bytes offset to the the ScummVM metainfo from the end of the save


=== Example: engines/quux/module.mk ===
Thus, all generic methods like listSaves() are retrieving this information.
<pre>
MODULE := engines/quux


MODULE_OBJS := \
In order to benefit from it, do the following:
quux.o


MODULE_DIRS += \
# In MetaEngine::hasFeature(), add kSavesUseExtendedFormat, kSimpleSaveNames, kSupportsListSaves, kSupportsDeleteSave, kSavesSupportMetaInfo, kSavesSupportThumbnail, kSavesSupportCreationDate, and kSavesSupportPlayTime. Also kSupportsLoadingDuringStartup if you intend to support loading savegames directly from the launcher.
engines/quux
# Overload loadGameStream() and saveGameStream() where you parse/save only your engine-specific data
# ...
# PROFIT!


# This module can be built as a plugin
== General Tips ==
ifdef BUILD_PLUGINS
PLUGIN := 1
endif


# Include common rules
* During the initial porting process, it might be preferable to link to C++ STL. In such cases, the `FORBIDDEN_SYMBOL_ALLOW_ALL` define can be used. However, you'll eventually need to get rid of this define.
include $(srcdir)/common.rules
</pre>

Latest revision as of 03:24, 9 June 2023

Introduction

This page is meant as a mini-HOWTO which roughly outlines the steps needed to add a new engine to ScummVM. It does not tell you how to create an engine for a given game; rather it is meant to tell a developer how to properly "hook" into ScummVM.

I will assume that you are at least roughly familiar with ScummVM, and have a recent checkout of our source code repository. Note that it's strongly advised to base your work on the current development version of ScummVM, and not on a release version. This will ease integration of your work.

Overview

Essentially, you will have to implement a subclass of the Engine class. Our Doxygen documentation is your friend and should hopefully explain enough about this: Engine class.

You also must hook yourself into the regular ScummVM main build system. Actually, some ports use a custom build system, but their maintainers will usually add your new engine once it has been added to ScummVM.

Finally, you need to make ScummVM aware of your new engine.

Automated Method

There is a tool that automates creating new skeleton engines which works both with make-based systems like *nix/Mac and with Microsoft Visual Studio.

Make-based Systems

Run make devtools/create_engine, then run the devtools/create_engine/create_engine <engine name>, where engine name is the desired engine name.

Visual Studio

If you go to the devtools/create_engine/ folder, you'll see a create_engine.sln solution which you should open and compile. This will create a create_engine.exe executable in the same folder. From the command line prompt, you should then run create_engine with the desired engine name provided as a single parameter. This will create a folder for your new engine as well as set up a batch file in the dists/msvc/ folder with the engine name that you can use as a shortcut to create a ScummVM solution with only that engine enabled.

Common Instructions

There are two styles of engine skeletons supported. The first, if you just specify an engine name, creates a bare bones engine with a sample event loop. The second kind is selected if you also specify -events as an extra parameter after the engine name. In this case, it creates a more sophisticated engine skeleton that has a single centralized event loop, and dispatches events to views, which can be easily switched between. This has benefits of avoiding duplicating event loops in multiple different places, each handling event processing and quit checks. Each view simply has to implement a drawing method and override different event handler methods to handle whichever events it needs.

In either case, once you've got the skeleton engine compiling, your next step should be to update the placeholder detection entry in detection_tables.h. The easiest way is to choose a file in your game's folder that you think is unique, and change the filename in the detection entry to it. If you then run ScummVM and try to add the game, it should prompt you that an unrecognised md5 for the game has been found, and give you the option to copy it to clipboard. You can do so, and then paste it to extract the md5 and filesize for the file, which can be used to update the detection entry. With this done you should be able to add your game, and run the skeleton engine.

As a second optional step, by default everytime your game starts, you'll get an unsupported game warning dialog appear. If you want to suppress it, locate and open your scummvm.ini file, and find the section added for your game. Then add the following line to it:

enable_unsupported_game_warning=false

General Conventions

File name conventions

Over the course of its existence, many people will have to deal with the source code of a given engine. Be it to fix bugs in it, modify it to still compile after changes made to the backend code, or to simply add new functionality. It is therefore expected to adhere to various conventions used throughout all engines. Besides source code conventions (see Code Formatting Conventions), this affects filenames. We suggest you use the following names for specific parts of your engine:

ENGINENAME.h Contains your (primary) Engine subclass.
ENGINENAME.cpp Contains at least the constructor and destructor of your (primary) Engine subclass, as well as the implementations of the mandatory (i.e. pure virtual) Engine methods.
detection.cpp Code related to game detection.
metaengine.cpp Contains the implementation of the plugin interface, as described in base/plugins.h.
saveload.cpp Code related to savegames
console.cpp (console) debugger
gfx.cpp (alt: graphics.cpp) Graphics code
sound.cpp Sound code
music.cpp Music code
inter.cpp, logic.cpp, script.cpp Game logic, resp. script/bytecode interpreter

Additionally, the files saved by each engine should be consistent.

  • Saves: If you use the new loadGameStream & saveGameStream methods, these will automatically be created in the form of <targetid>.### (where ### is the slot id). If you want saves to be consistent across all game variants, you should instead provide your own implementation of loadGameState and saveGameState, and create savefiles using the format <gameid>.###.
  • Other files: These should be named <targetid>-<filename> or <gameid>-<filename>. Again the latter when the files can be shared across all variants of the game (an example for these type of files would be when a minigame saves a high score record).

Here, only use the <gameid> scheme if you are absolutely sure that such files can be shared across all (that is every game platform, every game language, every game patch version, every game release, etc.) versions. If you are not sure whether this is the case, then stick to the <targetid> based scheme.

Subclassing AdvancedMetaEngine

Let's implement the plugin interface:
You'll have to create a custom AdvancedMetaEngine subclass. This provides the information and functionality related to the engine that can be used by the launcher without loading and running the game engine, which includes defining keymaps and achievements, listing savegames, and instancing the engine. You'll also have to create a custom AdvancedMetaEngineDetection subclass. This provides the information related to detecting games, which is always included in the main ScummVM executable regardless of whether or not the engine is enabled.

The following example illustrates this. It contains the necessary fundamentals of the details of the games and the code to create the engine, as well as REGISTER macros that register the meta engine with ScummVM. For the Quux example, If you create an empty file named quux.txt, the engine will detect it.

Subclassing Engine

The generated by the create_engine code gives a simple example of an engine. It contains a few important points:

  • It initializes the screen at a given resolution
  • It creates a debugger class and registers it with the engine framework. Currently, it's only a skeleton defined in <engine>.h. For a full game, you'd implement it in its own debugger.cpp and .h files
  • It has a simple event loop
  • It also demonstrates how to read and write savegames

Miscellaneous important: If you end up using the ScummVM GUI manually in your game (g_gui and stuff) you have always to call g_gui.handleScreenChanged() if you received a OSystem::EVENT_SCREEN_CHANGED event, else it could be that your gui looks strange or even crashes ScummVM.

For opening files in your engine, see the how to open files page.

Infrastructure services

Header file common/scummsys.h provides services needed by virtually any source file:

  • defines platform endianness
  • defines portable types
  • defines common constants and macros

Moreover, it deals with providing suitable building environment for different platforms:

  • provides common names for non-standard library functions
  • disables bogus and/or annoying warnings
  • provides a lean environment to build win32 executables/libraries

The common directory contains many more useful things, e.g. various container classes (for lists, hashmaps, resizeable arrays etc.), code for dealing with transparent endianess handling, for file I/O, etc. We recommend that you browse this a bit to get an idea of what is there. Likewise you should familiarize yourself with graphics and sound; for example we already have decoders for quite some audio formats in there. Before you roll your own code for all sorts of basic things, have a look and see if we already have code for that, and maybe also ask some veterans for advice.

Common portability issues

There are wrapper around number of non-portable functions. These are:

 max() -> MAX()
 min() -> MIN()
 rand() -> use Common::RandomSource class
 strcasecmp() / stricmp() -> scumm_stricmp()
 strncasecmp() / strnicmp() -> scumm_strnicmp()
 strrev() -> scumm_strrev()

Also we have predefined common integer types. Please, use them instead of rolling your own:

 byte
 int8
 uint8
 int16
 uint16
 int32
 uint32

Do not use the following functions:

 sprintf -> snprintf
 strcpy -> strncpy or Common::strlcpy

Additionally ScummVM offers a way of recording all events and then playing them back on request. That could be used for "demoplay" mode. But to ensure that it will work for your engine, you have to register your RandomSource class instance as demonstrated in the generated <engine>.cpp file.

True RGB color support

If you need to support more than 256 colors in your game engine, please refer to the truecolor API reference page for specifications and initialization protocol.


Extended Saves

To easily support ScummVM-specific metadata in the saves, we implemented the ExtendedSaves interface.

This basically makes your saves look like this:

 <DATA>     Engine-specific save data
 <METAINFO> ScummVM metainfo, including play time, screenshot, save name etc
 <OFFSET>   4-bytes offset to the the ScummVM metainfo from the end of the save

Thus, all generic methods like listSaves() are retrieving this information.

In order to benefit from it, do the following:

  1. In MetaEngine::hasFeature(), add kSavesUseExtendedFormat, kSimpleSaveNames, kSupportsListSaves, kSupportsDeleteSave, kSavesSupportMetaInfo, kSavesSupportThumbnail, kSavesSupportCreationDate, and kSavesSupportPlayTime. Also kSupportsLoadingDuringStartup if you intend to support loading savegames directly from the launcher.
  2. Overload loadGameStream() and saveGameStream() where you parse/save only your engine-specific data
  3. ...
  4. PROFIT!

General Tips

  • During the initial porting process, it might be preferable to link to C++ STL. In such cases, the `FORBIDDEN_SYMBOL_ALLOW_ALL` define can be used. However, you'll eventually need to get rid of this define.