HOWTO-Dynamic Modules

NOTE: many parts of this HOWTO assume the "gsoc2010-plugins" branch has been integrated into trunk.

Introduction

This page is meant as a mini-HOWTO which roughly outlines the steps needed to implement dynamic engine plugins for a previously unsupported backend (port) of ScummVM. This HOWTO only applies to targets that can use the ELF file-format and will generally only be useful for smaller targets that cannot support POSIX plugins.

I will assume that you are at least roughly familiar with ScummVM, and have a fresh checkout of our Subversion repository. Note that it's strongly advised to base your work on the current development version of ScummVM, and not on a release version. This will ease integration of your work.

Overview

ScummVM has a PluginManager class that potentially keeps track of multiple plugin providers. As of this writing, a StaticPluginProvider is always added to this PluginManager that keeps track of all statically-linked plugins. Your job will be to add a dynamic plugin provider specific to the backend you're working on that will provide for any dynamic plugins that are enabled on that backend and to tell the backend to use this provider in the case that dynamic modules are enabled.

Step by step

For the rest of this document, I will assume you are attempting to implement loadable modules for the "foobar" port of ScummVM :-).

1. Create a new backends/plugins subdirectory matching the name of the port you're providing for, i.e. backends/plugins/foobar/.
2. Create a provider file in this subdirectory, such as foobar-provider.h that contains the class declaration of the provider for your port. This class should subtype ELFPluginProvider. This class should nest a class declaration that subtypes ELFPlugin but is specific to your port (i.e. FOOBARPlugin) and override ELFPluginProvider's "createPlugin" method with its own that returns a new FOOBARPlugin object (See example below).
3. The FOOBARPlugin class needs to override ELFPlugin's "makeDLObject" method and have it return a new object of an appropriate subclass of DLObject. Currently, the subclasses of DLObject are processor-specific, but depending on the port, you may find you have to subclass one of these processor-specific subclasses with a platform-specific one, such as FOOBARDLObject(). (See "Subclassing DLObject" below).
4. Find where scummvm_main(argc, argv) is invoked in your backend's main and somewhere shortly before it add the FOOBARPluginProvider to the PluginManager, i.e. PluginManager::instance().addPluginProvider(new FOOBARPluginProvider(); (Be sure to guard this call with #ifdef DYNAMIC_MODULES). This, of course, will require that you include the FOOBARPluginProvider header at the top of the file.
5. Now we need to get the backend generating custom engine plugin files that can be loaded/unloaded to/from memory at will during runtime. To make the runtime linking (loading) of plugin files easier to handle, we let ld do some of the work for us at compile-time. First, we build the non-relocatable main executable scummvm.elf (Manually implementing Dynamic Modules on a smaller target that doesn't allow the main executable to be non-relocatable has not yet been successfully done and, as such, is not covered by this HOWTO). Then we link the plugins files together with a custom ld linker script; ld uses the absolute addresses from the main executable to resolve messy jumps from the plugin back to the main code and thus does much of the more complicated dynamic work for us. I would suggest adding the necessary custom linker script into the backends/plugins/foobar/ directory you made earlier (See "Making the plugin linker script for your backend" below).
6. Modify the build system for your backend (See "Necessary Build Modifications" below).
7. Cross your fingers and be prepared to work at things for a while if it doesn't immediately function ;-)

Example of foobar-provider.h

<syntax type="C++">#if defined(DYNAMIC_MODULES) && defined(FOOBAR)

1. include "backends/plugins/elf-provider.h"
2. include "backends/plugins/foobarprocessor-loader.h"

class FOOBARPluginProvider : public ELFPluginProvider { class FOOBARPlugin : public ELFPlugin { public: FOOBARPlugin(const Common::String &filename) : ELFPlugin(filename) {}

DLObject *makeDLObject() { return new FOOBARProcessorDLObject(); } };

public: Plugin* createPlugin(const Common::FSNode &node) const { return new FOOBARPlugin(node.getPath()); } };

1. endif // defined(DYNAMIC_MODULES) && defined(FOOBAR)

</syntax>

Making the plugin linker script for your backend

To generate this script, first find the linker script used to link a static build of the target platform (usually this is the default script for the linker, which you can dump via the command foobar-compiler-ld --verbose).

TODO: Fill out this section.

Necessary Build Modifications

There will undoubtedly be a number of things specific to your platform to deal with when it comes to modifying the building, but you should definitely add the following, bordered by #ifdef guards to make sure DYNAMIC_MODULES is set to 1 (along with telling the desired engines that you want them to be separate plugin files, i.e. "ENABLE_SCUMM = DYNAMIC_PLUGIN"):

<syntax type="C++">DEFINES += ELF_LOADER_TARGET //also add any other necessary defines PLUGIN_SUFFIX := .plg //or whatever your suffix is PLUGIN_EXTRA_DEPS = $(EXECUTABLE) PLUGIN_LDFLAGS += -nostartfiles -Wl,-q,--just-symbols,$(EXECUTABLE),-T$(srcdir)/backends/plugins/foobar/plugin.ld,--retain-symbols-file,$(srcdir)/backends/plugins/plugin.syms PRE_OBJS_FLAGS := -Wl,--whole-archive POST_OBJS_FLAGS := -Wl,--no-whole-archive </syntax>

As you can see, these modifications mainly deal with making sure the plugins are dependent on the main executable and use the custom linker script.

You will also need to make sure your main executable is not being overly stripped as the thing doing the stripping may think the executable doesn't use certain symbols simply because they reference portions of code that will be linked in later in a plugin.

TODO: Add stuff about "ONE_PLUGIN_AT_A_TIME" once it is working perfectly.

Subclassing DLObject

At some point, the PluginManager will call getPlugins() on the FOOBARProvider, which will trigger a search for plugin files in your backend's file system and return a list of FOOBARPlugins based on the results of this search. The PluginManager will then be able to call loadplugin() and unloadPlugin, and on any of these FOOBARPlugins, which will involve calls to open(const char *path) on a new object that's a subclass of DLObject (which subclass is determined by how you coded makeDLObject() in the FOOBARPlugin class) with the path to the plugin file the FOOBARPlugin represents as its argument (The multiple levels of abstraction here can get confusing :-)). A subclass of DLObject, then, is ultimately responsible for opening the plugin files and loading them/unloading them from memory.

TODO: Finish explaining what the DLObject class does and how to subclass it effectively.