Open main menu
ScummVM :: Wiki

OpenTasks

Revision as of 19:40, 15 February 2009 by LordHoto (talk | contribs) (→‎Generic infrastructure tasks: changed order of tasks and removed "recorded play" ideas from "Add a testing framework for ScummVM's subsystems" (as proposed by Max on -devel))

This page contains a list of open tasks. Completing any of them would benefit the ScummVM project a lot. At least in theory, all of them should be doable even by somebody relatively new to the project, in particular, as part of the Google Summer of Code. Besides the tasks listed here, there is of course also the TODO page listing more things that need to be done (but with far less details).

Introduction

Some basic rules

Below follow some basic rules that anybody interested in one of these tasks should adhere to. Sometimes exceptions may be possible -- as always, common sense applies, and if in doubt, ask.

The projects below are sketches and ideas. Plus, things evolve over time, so the descriptions might be slightly outdated by the time you read them (although we strive to keep them up-to-date). Hence, you should talk with somebody from the team, probably the person(s) listed as Tech Contact, before starting work on any of them.

All code, unless stated differently, must be written in clean and portable C++, in particular, GCC must be able to compile it (portability exceptions can be made for platform specific code, of course). We also have some Code Formatting Conventions. Using the standard C++ lib for code used inside ScummVM is at this time not possible. Using it inside non-essential tool should be fine, though.

All of the code submitted must be contributed under the terms of the GPL v2+.

We only accept clean and maintainable code. This is a somewhat vague requirement, but as a rule of thumb, if the code does what it is supposed to do, but is not extensible, a quick hack, or we need to rewrite it from scratch to properly integrate it, we will not accept it. In particular, we would rather have a maintainable, clean and incomplete piece of code that we could extend than a complete but unmaintainable piece of code.

Some good advice

The PostgrSQL folks have some real good Advice to Students on Submitting SoC Applications. We recommend all students interested in applying with us (or any other SoC project, for that matter) to read this.

Student application template

The following was adapted from the FreeBSD Proposal Guidelines.

  • Name
  • Email
  • Project Title
  • Possible Mentor (optional)
  • Benefits to the ScummVM Community - a good project will not just be fun to work on, but also generally useful to others.
  • Deliverables - It is very important to list quantifiable results here e.g.
    • "Improve X modules in ways Y and Z."
    • "Write 3 new man pages for the new interfaces."
    • "Improve test coverage by writing X more unit/regression tests."
    • "Improve performance in FOO by X%."
  • Project Schedule - How long will the project take? When can you begin work?
  • Availability - How many hours per week can you spend working on this? What other obligations do you have this summer?
  • Bio - Who are you? What makes you the best person to work on this project?

Your task

Anything you can dream of

Technical Contact: Our IRC channel, our mailing list, or contact Eugene Sandulenko, Max Horn

The Task:

Come up with your personal clever way to improve ScummVM and its various side projects. Be creative. Incorporate the ideas listed below and in our TODO, but don't let yourself be limited by them. Come up with something totally new, or enhance existing features. It's up to you.

But of course like with all the other tasks, we recommend that you first talk to us (see above).

Generic infrastructure tasks

Add possibility for compiling externally to ScummVM tree

Technical Contact: Max Horn, Eugene Sandulenko

Background:

OSystem (ScummVM core) turned into nice portable infrastructure for creating graphics-related applications. We already have our own full blown GUI and couple of dozens of engines built on top of it.

The Task:

Modify ScummVM build system to add ability to build engines ourside of the main source tree. It may require turning ScummVM OSystem into a library.

Required Skills:

  • Good C++ skills.
  • Excellent Makefile skills


Implemented support for loadable modules on Nintendo DS and other ports

Technical Contact: Max Horn

Background:

TODO engines take up lots of storage space and RAM. Putting them into lodable modules reduces memory usage, as only needed module needs to be loaded. For DS port, this is is esp. nice, because it right now has to be shipped in multiple versions, combining different sets of engines. Ugly.

The Task:

TODO Implement infrastructure that allows generating loadable modules for the Nintendo DS, and code which then allows to (un)load these during runtime. For our Dreamcast code, we already have code doing that, which could be used as a reference, and maybe even some of its code could be reduced. Ultimately, the goal would be to not just target the Nintendo DS, but all target platforms which use the ELF binary format and an ARM processor. Bonus: Support for MIPS processors to cover Playstation 2 / PSP ports.

TODO: Add reference to the DS build chain, DevKitPro, etc.

Required Skills:

  • Good C++ skills.
  • Some knowledge of the ELF binary format (great documentation is available on the net, start e.g. at Wikipedia).
  • A Nintendo DS for testing would be ideal.

Small Devices Backend

Technical Contact: John Willis, Joost Peters

Background:

ScummVM has been ported to many platforms, often by simply re-using the SDL backend (which is based on SDL, which by itself has been ported to many platforms, making it fairly easy to port ScummVM to any of these platforms).

But for some platforms, dedicated backends are required, either because SDL doesn't support them, or because we can't achieve all our needs by using the SDL port (e.g. because we need more speed, more control, etc.). These backends are typically made for what we call "small systems" -- systems like PDAs, SmartPhones, Linux Tablets (PalmOS, SymbianOS, Windows CE, Maemo), or game consoles (Dreamcast, GP2X, Nintendo DS, PlayStation 2, PlayStation Portable).

These systems share many features. In particular they often have no (full) keyboard and quite limited resources: Little RAM, little permanent storage space, not that much CPU power, or a limited screen resolution.

This makes it often necessary to (re)implement certain functionality, like virtual keyboards (see above task), or graphic downscalers and the like.

The Task:

Since the same needs occur again and again, it would be nice to implement such functionality only once in a sufficiently portable and flexible way, making it possible for backends to pick and use whatever they need.

Details, scope and further suggestions as to how to achieve this can be found on the Small Devices Backend page.

Besides this, optimizing code for speed and memory usage benefits all our targets. In particular these "small devices". Hence doing this is a worthy goal on its own.

Required Skills:

  • Reasonable C++ skills.
  • Refactoring skills.
  • Knowledge of one or more ScummVM platform backends is desirable.

Improve the overlay API

Technical Contact: Max Horn

Background:

ScummVM is a highly portable application, running on a multitude of devices, ranging from desktop PCs running Linux, Mac OS X, Windows and other systems, over game consoles like the PlayStation or the Nintendo DS, down to small mobile devices like PDAs and smartphones running PalmOS, WinCE or Symbian. One key component which makes this possible is our relatively strict separation between "backend code" (which implements the device specific functionality, like graphics drawing or sound output), and "frontend code" (which implements support for certain games). These two parts are held together by a bit of middle layer code. The key API for all backends is the OSystem class (you can use it as a great starting point to figure out most of the rest of the backend API).

Two key components of the backend API are the game graphics (used for drawing the game graphics, surprise surprise), which are currently limited to 8bit palette mode, and the overlay graphics (used to e.g. draw the GUI, and currently either 16bit or 8bit palette, fixed at compile time). For the overlay API, it would be nice if we could support arbitrary graphics modes, e.g. also 24/32bit modes.

The current overlay API is rather inflexible when it comes to supporting different modes. Most code assumes that the overlay is in 16 bit mode. For 8bit mode, a compile time switch has to be used, and supporting 24/32bit mode is virtually impossible. Furthermore, which variant of 16bit mode is determined through an ugly global variable gBitFormat (set by the InitScalers() function in graphics/scaler.cpp): This variable is set to values like 555, 565 to indicate the 16bit mode variant (555 meaning that 5 bits are used for each color component, while 565 means 5bits for red, 6 for green, 5 for blue). Some systems use other modes, like 1555, 4444, or even use BGR instead of RGB. For 32bit modes, things are even more complicated.

The Task:

Get rid of the evil gBitFormat variable. Instead, introduce a new Graphics::PixelFormat struct (modeled after SDL_PixelFormat). Enhance Graphics::Surface to use it (similar to SDL_Surface). Add a new getOverlayPixelFormat method to OSystem, and implement it in the SDL backend. Rewrite the GUI code and other code accessing the overlay to make use of the PixelFormat data, instead of using gBitFormat. This also includes updating some of the scalers (at least the simple ones; for the special ones, like HQ2x, we have to determine how to treat them best).

For all this, API flexibility has to be carefully weight against API complexity, and efficiency -- since we have to support small devices with limited resources, we don't want to needlessly waste CPU cycles nor memory.

Also, the PixelFormat code and its usage have to be documented (using Doxygen comments, and maybe a concept document), as our porters will have to implement these changes, and it must be as clear and easy as possible for them to do so.

Required Skills:

  • Good C++ skills.
  • Refactoring skills.
  • Knowledge of one or more ScummVM platform backends is desirable.

Add a testing framework for ScummVM's subsystems

Technical Contact: Johannes Schickel, Max Horn

Background:

ScummVM is a highly portable application. To achieve that there are different interfaces to access platform specific functionality. Our APIs are usually well documented, thus giving porters and engine authors a clear vision of how different functions / methods are supposed to work. Apart from the lower level platform abstraction APIs we usually offer more high level APIs for engine authors.

For example we have a low-level filesystem abstraction API: FSNode. This can be used to browse directories, open files, create new file for writing etc. Since the API is pretty much low level, engine authors usually do not use it directly, but use some wrapper code to access it. To prevent code duplication and take work from engine authors we introduced the Archive and SearchMan classes. SearchMan can be easily used to access game data files without fiddling with FSNode.

However, experience has shown that the documentation of all these classes is not always as clear and complete as it should be, and sometimes might be misunderstood. In addition, the implementations for the various systems we run on may not adhere to the documented specification and/or regressions were introduced when doing changes to the code. Currently we have only a very minimalistic test suite in "tests/", testing only a tiny subset of our code: container classes, string handling and some stream code. Currently it is hard to do proper regression / implementation testing for other APIs, for example FSNode API.

The Task:

As visible from the description above, we need a automated test system to help ensure our implementations are bug free. We need both a framework for the tests, as well as an extensive test suite. There are several different approaches to testing which all are of independent interest and complement each other:

  1. Unit tests: Add many more tests to our test suite in "tests/". This would also require some enhancements to our build system, since currently we do not link against all of our code for the test suite. Also new test mechanisms are needed to make it possible to e.g. test audio & graphics output (e.g. code that can load an existing reference image and compare it to the output of a graphics primitive). It might also be helpful to enhance the "null" backend so that all unit tests can be run against that, headless (good for automated tests on a server with actual graphics/audio output.
  2. A "test" engine: Unit tests probably would likely only be runnable on "desktop" ports like Windows and Linux, and not on ports like PSP or NDS, due to run environment restrictions. To allow extended coverage on *all* ports, implementing a "test" engine would be helpful. In addition to running on all ports, the "test" engine also provides an environment to test things in an integrate, non-isolated way, unlike unit tests.It should test as many features as possible: game detection, file loading/saving, navigating the file system, testing graphics and audio output, etc. -- and it must be possible to verify everything. I.e. compare results to reference data, output progress data on the screen, create dump logs, and so on. Lots can be done here.

The test suite extension would be easiest to use on Linux etc. and there is already some existing code to build on. An test engine would work on all ports, including game consoles and phones, and is nice for testing in a more "holistic" and complete way. Of course best would be to have both, and also make it possible to run all unit test from inside the test engine.

Overall the test engine has the highest priority, since it would give us the possibility to test all ports without additional work. Thus the primary task would be to implement and document at least a test engine, which tests (preferably) all of ScummVM's subsystems extensively.

Required Skills:

  • Reasonable C++ skills.
  • Experience with unit tests.
  • Knowledge about various ScummVM subsystems would be useful, but not required.
  • Knowledge about how to create a new engine for ScummVM would also be useful. See also our Engines HOWTO.

Also required:

  • Access to a non-desktop testing device (PSP, NDS, PS2, WinCE or Symbian mobile, etc.) and the ability to compile and run ScummVM on it (the ScummVM team would help with compiling and running).

Add a testing framework for ScummVM's engines

Technical Contact: Eugene Sandulenko

Background:

Number of ScummVM supported games grows with each release. Since 0.8.0 we are performing extensive prerelease testing. Basically it is game replaying for finding regressions. This becomes somewhat tiresome process and takes much time.

In fact there should be no changes in the gameplay in most cases, so event recording and proper replaying should be enough. There is some code for that based on Action recorder patch, but the recordings are not played consistently. The main problem is that there is no guarantee of the events to be in sync with other engine parts, so the playback may deviate, depending on the machine speed.

The Task:

Implement proper event recorder and player. It should be multiplatform and not depend on the CPU power. I.e. it should guarantee proper event sequence and time. It should store mouse, keyboard events, as well as generated random numbers and any other functions dealing with extern world such as time. Basically it should be implemented as some globally blocking cycle which will be driven by recorded events.

The format should contain also ways to give name to the recording, keep a screenshot, and optional savegame, so arbitrary portions of the game could be recorded.

Additionally there should be implemented possibility to create screenshots at specified intervals, so the output could be compared.

The GUI should preferably include an overlay image which will indicate that the game is being recorded or played back. But it will require some work in backends.

Optionally there could be implemented step-by-step mode with ability to pause the playback at any place. And as a big bonus ability to annotate the recordings with on-screen messages could be also implemented.

Required Skills:

  • Good C++ skills.
  • Access to slow(er) devices such as handhelds is preferable.

Audio related tasks

Improve sound support in SCUMM games

Technical Contact: Eugene Sandulenko, Travis Howell

  • Add support for sound format (SID format?) used by Commodore 64 versions of Maniac Mansion and Zak McKracken to SCUMM engine. Unfortunately there is no demo versions are available, so you need to own one of the games.
  • Add support for sound format used by Macintosh version of Loom to SCUMM engine. Known information about the structure of the sound resources used is available. If you don't own the game, the LucasArts Mac CD Game pack is usually available via eBay.

MIDI enhancements

Many of the adventures supported by ScummVM make use of MIDI music. Which is why we already include several device drivers for various MIDI APIs and emulators (e.g. ALSA, Windows MIDI, Mac OS X CoreAudio/CoreMIDI, fluidsynth...).

MIDI device configuration

Technical Contact: Max Horn

Background:

Right now, the MIDI drivers are treated by ScummVM in a rather single minded fashion: Either a driver is linked in and hence "available", or not. It's not possible to configure anything about them (like ports to be used etc.), nor does it ever take into account that a single driver might correspond to multiple devices (after all, you can plug several sequencers into your MIDI port; or you could have configured several different sound font settings in your MIDI emulator).

The Task:

  • Add an API for querying the OSystem backend for a list of available MIDI devices (not drivers)
  • Information about the selected device must be serializable, so that it can be stored in the config file
  • Selection of devices via command line should be possible
  • It must deal with devices being added/removed (at least between runs of ScummVM, ideally also while ScummVM is running)
  • Devices should be configurable via the GUI; this needs to be done in a flexible (different devices/drivers offer different settings) and portable fashion.

Accolade MIDI parser

Technical Contact: Travis Howell

Background:

The Elvira 1 (DOS), Elvira 2 (DOS), Waxworks (DOS) games of the AGOS engine use the Accolade MIDI format for music. The current parser (see engines/agos/midiparser_s1d.cpp), is based off looking at the music in the DOS Floppy Demo of Simon the Sorcerer 1 (Which was based Waxworks engine) and guess work. The current code frequently crashes when changing locations in Waxworks, due to invalid MIDI data been passed along.

The Task:

Updated the current parser, to completely support the Accolade MIDI format used by these games. There are already comments in the current code, about where each additional MIDI event is triggered in the music of these games.

XMIDI parser

Technical Contact: Max Horn

Background:

Several of our games make use of the XMIDI format. We already have a parser for it (see sound/midiparser_xmidi.cpp), which was based on code from the Exult project, but it is incomplete.

The Task:

Specifically, we require support for XMIDI_CONTROLLER_FOR_LOOP and XMIDI_CONTROLLER_NEXT_BREAK. There are many more XMIDI controllers, however, and ideally support for all of them would be great. Some references:

  • The XMIDI code from the Exult project (see Exult's audio/midi_drivers/XMidiSequence.cpp) could be used as a reference again.
  • In addition, Pentagram has XMidi code based on the Exult code, but may be cleaner (see here.)
  • Another good reference for this task is the AIL library which has been recently open-sourced by its author. This is infact where XMIDI comes from, and it even contains specs. To find them, unzip AIL2.ZIP; in there you find A214_D2.ZIP, which when unzipped contains a dir DOC, which in turn contains ASCIIDOC.ZIP, which then contains XMIDI.TXT and some other files which might be of use (e.g. NOTES.TXT).

Support for AMIGA Audio Formats: TFMX and MaxTrax

Technical Contact: Eugene Sandulenko, Sven Hesse, Kostas Nakos

Background:

Since we're adding (and have added so far) support for different Amiga game versions, we need support for the sound files too. Currently we need support for following formats:

  • Desktop Tracker format used by music in the Acorn disk version of Simon the Sorcerer 1. Information about the specifications of the Desktop Track format and sample mods are available.
  • MaxTrax support for Kyrandia 1. The Amiga version isn't supported yet, but LordHoto will be working on it when his copy arrives. A Motorola 68000 (short: 68k) assembler implementation by Joe Pearce can be found here. It would be necessary to re-implement it in proper C++, naturally a must for this task is 68k assembler knowledge. If you don't have the Amiga version of the Legend of Kyrandia, there are sample modules available.
  • TFMX a (non-MOD) format used by Monkey Island 1. There is a player for it already, which is written in rather poor C, also it misses several macros implementation. This documentation may be useful, it isn't complete though. If you don't have the Amiga version of Monkey Island 1 you can use the demo of Monkey Island 1. The main objective of this task is to write a TFMX player with all the effects and macros needed by the Monkey Island music tunes. Work can be based on the original - uncommented - replayer routine disassembly (Motorola 68k).

The Task:

  • Extend our general MOD player to support mentioned formats

Required Skills:

  • Good C++ skills.
  • Understanding of how audio is generated.
  • m68k assembler knowledge is a big plus.
  • Amiga knowledge is a plus.

Tools

Tools: Create a great user interface for the compression tools

Technical Contact: Max Horn

Background:

We offer a multitude of command line tools in a separate package (scummvm-tools). The majority of these tools are used to (re)compress audio data. This greatly helps users who want to play their games on devices with limited storage, like PDAs and smart phones.

The user can choose between MP3, Ogg Vorbis and (in those cases where it makes sense) FLAC compression. The tools take the original data files, extract sound data, compress them, and reassemble everything into new (smaller) data files.

The Task:

Right now, those tools are mainly for command line users. Needless to say, this makes it very difficult to use for many people who just want to play a game, and who do not have a strong technical background.

During GSoC 2007, progress was made in unifying the internal code of these tools (mainly the compression tools). Also a basic GUI wrapper using wxWidget was added, as a first step towards making the tools usable for non-experts. However, this wrapper is right now a very thin shell around the true complexity of the tools.

Your task would be to turn this GUI tool into a truly amazing and portable tool (usable on Linux, Windows and Mac OS X at least, ideally on more) which would be very easy to use for beginners, yet offers (optionally) the full power of the compression tools for experts.

In particular, the user should be able to just drop a file onto the tool icon (or one of its windows), the tool would detect which game it is looking at, and offer a simple "one-click-and-done" start button to the user. Optionally, the user could tweak his default settings before starting the conversion (like choosing a different compression level, a custom output directory, etc.). The GUI would be very forgiving to the user (e.g. if selecting data from a read-only media or a directory with not enough free space, it would automatically ask the user for an alternate output location of the generated files. It would upon startup show a nice friendly window with instructions on how to use it, etc.). The exact desired feature set would have to be determined at the start of the project, in discussions with the ScummVM team members and users on our forums. This would be turned into a rough set of mockups and/or texts describing the planned features. Many ideas for what could be done here already exist, but you are most welcome to also develop and contribute your own!

Engine/game specific

Add support fot Dragon's History game

Technical Contact: Eugene Sandulenko

Background:

Dragon's History game sources were released under GPL. They are written in Turbo Pascal and have extensive Czech comments in it. Game assets were also released as freeware.

The Task:

Create from scratch an engine that will support all freeware variants of the game.

Required Skills:

  • Strong C++ skills.
  • Basic Pascal Knowledge
  • Work more than 12 weeks

Objectify CruisE engine

Technical Contact: Eugene Sandulenko

Background:

The cruisE engine also started out as an external project started by Yaz0r. Originally it was written in plain C. ScummVM is a C++ project, so we need to objectify this engine without changing/breaking its behavior. The engine is fairy complete, but also suffers from portability problems, that is it works correctly only under little-endian, alignment-agnostic CPUs.

The engine itself is well-structured, hence many functions / variables which might be good candidates for being grouped together into a C++ class are already grouped by files.

No deep knowledge of the engine internals is required too.

We have previously "objectified" several other engines, namely SAGA, Gob and AGI, so one can learn a lot about various approaches how to do this by tracing through our SVN repository.

Add 16bit graphics support to SCUMM engine

Technical Contact: Travis Howell, Eugene Sandulenko

Background:

The SCUMM engine was originally developed for palette-based graphics. At version 6 it was forked by Humongous Entertainment, which extended it significantly. Their later games started to use 16bit graphics for backgrounds and actors. See here for more detailed information.

This task requires good knowledge of C++, as we need a solution which will not clobber our code, will have minimal impact on 8bit games, and can be optionally turned off at compilation stage.

If you don't have any of the required games, there are several demos available. A 16bit graphics demo example would be the Freddi5 demo.

Improve AGI engine

Technical Contact: Eugene Sandulenko

Background:

Our AGI engine is in quite good shape. Still it misses some important aspects.

The Task:

  • NAGI has better Tandy music support, particularly noise channel emulation is better there. NAGI license lets use its code directly in ScummVM. For the most authentic model MAME could be referenced, though its license is not GPL-compatible
  • Add support for AGI 2.01 games, such as Donald Duck Playground
  • Add support for AGI0 and AGI1 (bootable floppies) games
  • Document all differences between AGI versions on Wiki and probably in code
  • Cover more tasks from AGI/TODO page

Required Skills:

  • Good C++ skills.
  • Engine internals could be studied within a week or less.
Retrieved from "https://wiki.scummvm.org/index.php?title=OpenTasks&oldid=10182"