Jump to navigation Jump to search

HOWTO-Dynamic Modules

2,731 bytes added, 17:15, 25 October 2018
Text replacement - "</source>" to "</syntaxhighlight>"
This page is meant as a HOWTO which roughly outlines the steps needed to implement dynamic engine plugins for a previously unsupported backend (port) of ScummVM. This HOWTO only applies to targets that can use the ELF file-format and will generally only be useful for smaller targets that cannot support POSIX plugins.
I will assume that you are at least roughly familiar with ScummVM, and have a fresh checkout of our Subversion source code repository. Note that it's strongly advised to base your work on the current development version of ScummVM, and not on a release version. This will ease integration of your work.
=== Overview ===
# Now we need to get the backend generating custom engine plugin files that can be loaded/unloaded to/from memory at will during runtime. To make the runtime linking (loading) of plugin files easier to handle, we let ld do some of the work for us at compile-time. First, we build the non-relocatable main executable <tt>scummvm.elf</tt> (Manually implementing Dynamic Modules on a smaller target that doesn't allow the main executable to be non-relocatable has not yet been successfully done and, as such, is not covered by this HOWTO). Then we link the plugins files together with a custom ld linker script; ld uses the absolute addresses from the main executable to resolve messy jumps from the plugin back to the main code and thus does much of the more complicated dynamic work for us. I would suggest adding the necessary custom linker script into the <tt>backends/plugins/foobar/</tt> directory you made earlier (See "Making the plugin linker script for your backend" below).
# Modify the build system for your backend (See "Necessary Build Modifications" below).
# Some miscellaneous things: You should add a platform-specific call to a function that flushes the data cache into <code>static void flushDataCache()</code> in <tt>backends/plugins/elf-loader.cpp</tt>. You will also need to add a check for the proper machine (processor) type into <code>bool DLObject::readElfHeader(Common::SeekableReadStream* DLFile, Elf32_Ehdr *ehdr)</code> in that same file, adding the machine type to elf32.h if it's not already there.# That's the gist of it; Cross your fingers and be prepared to work at things for a while if it doesn't immediately function ;-).
==== Example of foobar-provider.h====
<syntax typesyntaxhighlight lang="C++cpp">#if defined(DYNAMIC_MODULES) && defined(FOOBAR)
#include "backends/plugins/elf-provider.h"
#endif // defined(DYNAMIC_MODULES) && defined(FOOBAR)
==== Making the plugin linker script for your backend ====
*First, you can remove the <tt>ENTRY(symbol)</tt> command. This command normally tells the linker which instruction should be the first to execute in the program, but since you're generating a plugin that will be linked in and jumped to from the main executable, it is not needed.
*Next, define an ELF program header for a "plugin" segment to be loaded from the file. Do this by adding the following before the SECTIONS command:
<syntax typesyntaxhighlight lang="C++cpp">PHDRS
plugin PT_LOAD ;
*Set the start address of the file to be 0 (since this will be linked in at a different address later), usually you can do this by replacing the first line in SECTIONS with <code>. = 0;</code>, though different ld scripts may require different modifications.
*Place the first section (as listed under the SECTIONS command) into the "plugin" segment you defined in PHDRS by appending <tt>:plugin</tt> to the first sections-command, i.e <code>.text . : { *(.text) } :plugin</code>. As long as there aren't any other PHDRS, this should ensure the entire linked file is put in the "plugin" segment, since future sections assume they are to be put in the same segment as the previous section unless specified otherwise (via <tt>:phdr</tt> or <tt>:NONE</tt>).
*Find the <tt>.ctors</tt> and <tt>.dtors</tt> sections (constructors and destructors) and delete any output-section-commands within them that reference crtbegin or crtend, i.e <code>KEEP (*crtbegin*.o(.dtors))</code> or <code>KEEP (*(EXCLUDE_FILE (*crtend*.o ) .dtors))</code>. Put <code>___plugin_ctors = .;</code> as the first output-section-command in the <tt>.ctors</tt> section and <code>___plugin_ctors_end = .;</code> as the last output-section-command. Do the same for the <tt>.dtors</tt> section but with <tt>___plugin_dtors</tt>, etc. This gives these symbols values that our run-time loader can use. In the end, <tt>.ctors</tt> and <tt>.dtors</tt> should look something like this:
<syntax typesyntaxhighlight lang="C++cpp">.ctors :
___plugin_ctors = .;
KEEP (*(.dtors))
___plugin_dtors_end = .;
*That's it! If you have trouble with any of these instructions or need to further modify the linker script for something specific to your platform, see [] for linker script documentation and peruse the various <tt>plugin.ld</tt> files in the subdirectories of <tt>backends/plugins/</tt>. TODO: Add stuff about MIPS-specific linker script modifications, namely the "shorts" segment.
There will undoubtedly be a number of things specific to your platform to deal with when it comes to modifying the building, but you should definitely add the following, bordered by #ifdef guards to make sure DYNAMIC_MODULES is set to 1 (along with telling the desired engines that you want them to be separate plugin files, i.e. "ENABLE_SCUMM = DYNAMIC_PLUGIN"):
<syntax typesyntaxhighlight lang="C++Makefile">DEFINES += ELF_LOADER_TARGET //also add any other necessary defines
PLUGIN_SUFFIX := .plg //or whatever your suffix is
PRE_OBJS_FLAGS := -Wl,--whole-archive
POST_OBJS_FLAGS := -Wl,--no-whole-archive
As you can see, these modifications mainly deal with making sure the plugins are dependent on the main executable and use the custom linker script.
==== Subclassing DLObject ====
At some point, the PluginManager will call <code>getPlugins()</code> on the FOOBARProvider, which will trigger a search for plugin files in your backend's file system and return a list of FOOBARPlugins based on the results of this search. The PluginManager will then be able to call <code>loadplugin()</code> and <code>unloadPlugin</code>, and on any of these FOOBARPlugins, which will involve calls to <code>open(const char *path)</code> on a new object that's a subclass of DLObject (which subclass this will be is determined by how you coded <code>makeDLObject()</code> in the your FOOBARPlugin class) with the path to the plugin file the FOOBARPlugin represents as its argument (The multiple levels of abstraction here can get confusing , I know :-)). A subclass of DLObject, then, is ultimately responsible for opening the plugin files and loading them/unloading them from memory.
TODOIf there is already a DLObject subclass that matches the processor for your smaller target (i.e <tt>MIPSDLObject</tt>, <tt>ARMDLObject</tt>), use it! Assuming that class is in use for another working backend with dynamic modules implemented, most everything should work correctly. The one function where one should expect problems is <code>bool DLObject::relocate(Common: Finish explaining what :SeekableReadStream* DLFile, unsigned long offset, unsigned long size, void *relSegment)</code> in that your platform may generate plugins that use relocation types as of yet unsupported by that subclass of DLObject. If this is the case, you will hopefully get a helpful message at runtime, such as "Unknown relocation type 12" or something like that. You should then look up which relocation type this number refers to in the ABI for your processor (relocation types are processor-specific), add a define for the type into elf32.h next to the other relocation types for your processor (i.e. <code>#define R_ARM_ABS32 2</code>), and then add code to handle it into the relocate function (again, the ABI for your processor and pre-existing relocation code will be your friend when trying to figure this out). If there isn't already a DLObject class does subclass that matches the processor for your smaller target, you will have to make one that overrides <tt>relocate</tt> and <tt>relocateRels</tt>. You can look to these functions in <tt>backends/plugins/arm-loader.cpp</tt> and <tt>backends/plugins/mips-loader.cpp</tt> for a template of how to code for them. Again, the hardest part will be the code for the actual relocations. You'll want to first detect what these relocations are (you can do this using objdump on generated plugin files), then look up how they are supposed to subclass it effectivelybe handled in the ABI for your processor. Since we pre-link to let ld do some of this relocation work for us, you should make builds that don't do this pre-linking with the plugins and ones that do and compare to see exactly what this work is. That way, you won't write relocation code for difficult things the linker already does for you (a lot of the time for absolute jumps, all you have to do is add the main offset of the plugin).Careful reading and persistence will pay off at this point :-)


Navigation menu