736
edits
Difference between revisions of "OpenTasks"
m
Waffeln - viele Waffeln!
m (Waffeln - viele Waffeln!) |
|||
| Line 10: | Line 10: | ||
The projects below are sketches and ideas. Plus, things evolve over time, so the descriptions might be slightly outdated by the time you read them (although we strive to keep them up-to-date). Hence, you should talk with somebody from the team, probably the person(s) listed as Tech Contact, before starting work on any of them. | The projects below are sketches and ideas. Plus, things evolve over time, so the descriptions might be slightly outdated by the time you read them (although we strive to keep them up-to-date). Hence, you should talk with somebody from the team, probably the person(s) listed as Tech Contact, before starting work on any of them. | ||
All code, unless stated differently, must be written in clean and portable C++, in particular, GCC must be able to compile it (portability exceptions can be made for platform specific code, of course). We also have some [[Code Formatting Conventions]]. Using the standard C++ lib for code used inside ScummVM is at this time not possible. Using it inside non-essential tool should be fine, though. | All code, unless stated differently, must be written in clean and portable C++, in particular, GCC must be able to compile it (portability exceptions can be made for platform specific code, of course). We also have some [[Code Formatting Conventions]]. Using the standard C++ lib for code used inside ScummVM is at this time not possible. Using it inside a non-essential tool should be fine, though. | ||
All of the code submitted must be contributed under the terms of the GPL v2+. | All of the code submitted must be contributed under the terms of the GPL v2+. | ||
| Line 17: | Line 17: | ||
=== Some good advice === | === Some good advice === | ||
The | The PostgreSQL folks have some real good | ||
[http://www.postgresql.org/developer/summerofcodeadvice.html Advice to Students on Submitting SoC Applications]. We recommend all students interested in applying with us (or any other SoC project, for that matter) to read this. | [http://www.postgresql.org/developer/summerofcodeadvice.html Advice to Students on Submitting SoC Applications]. We recommend all students interested in applying with us (or any other SoC project, for that matter) to read this. | ||
| Line 27: | Line 27: | ||
* '''Project Title''' | * '''Project Title''' | ||
* '''Possible Mentor''' (optional) | * '''Possible Mentor''' (optional) | ||
* '''Benefits to the ScummVM Community''' - | * '''Benefits to the ScummVM Community''' - A good project will not just be fun to work on, but also generally useful to others. | ||
* '''Deliverables''' - It is very important to list quantifiable results here e.g. | * '''Deliverables''' - It is very important to list quantifiable results here e.g. | ||
** "Improve X modules in ways Y and Z." | ** "Improve X modules in ways Y and Z." | ||
| Line 37: | Line 37: | ||
* '''Skype ID''' - If you don't use Skype, install it. | * '''Skype ID''' - If you don't use Skype, install it. | ||
* '''Phone Number''' - Celullar is preferable, for emergency contacts. | * '''Phone Number''' - Celullar is preferable, for emergency contacts. | ||
* '''Timezone''' - | * '''Timezone''' - Where do you live. | ||
* '''Bio''' - Who are you? What makes you the best person to work on this project? | * '''Bio''' - Who are you? What makes you the best person to work on this project? | ||
| Line 54: | Line 54: | ||
=== Improve the ScummVM build system === | === Improve the ScummVM build system === | ||
''Technical Contact'': [[User:Fingolfin|Max Horn]], [[User:Sev|Eugene Sandulenko]] | ''Technical Contact'': [[User:Fingolfin|Max Horn]], [[User:Sev|Eugene Sandulenko]] | ||
| Line 61: | Line 60: | ||
ScummVM uses a custom hand-made Makefile based build system by default. In addition, we maintain a set of project files for various versions of MSVC, and some other development environments. Some of our ports tie into the default Makefile build system, but others need yet other project files (e.g. the PalmOS port). | ScummVM uses a custom hand-made Makefile based build system by default. In addition, we maintain a set of project files for various versions of MSVC, and some other development environments. Some of our ports tie into the default Makefile build system, but others need yet other project files (e.g. the PalmOS port). | ||
This leads to the following problem. Whenever we add, move or delete files from our repository, the default build system is updated, but the other project files in general are not -- they have to be maintained separately. This is a manual task, and if forgotten, causes these project files to become out of date and hence more or less unusable. Therefore, we would like to be able to generate these project files from the master build system. For MSVC, we already do this with the help our custom C++ tool [code]create_msvc[/code]. | This leads to the following problem. Whenever we add, move or delete files from our repository, the default build system is updated, but the other project files in general are not -- they have to be maintained separately. This is a manual task, and if forgotten, causes these project files to become out of date and hence more or less unusable. Therefore, we would like to be able to generate these project files from the master build system. For MSVC, we already do this with the help of our custom C++ tool [code]create_msvc[/code]. | ||
Another issue is the following: There are several people who develop 3rd party engines, which use the core code from ScummVM, but are not hosted within the ScummVM repository. Since ScummVM allows building engines as loadable modules, that is fine. However, in order to ''build'' these engines, one currently has to copy them into a checkout of the ScummVM trunk, and then hook them into the build system (by modifying the [code]engines/engines.mk[/code] file). It would be so much easier if one could build an engine as a loadable module *outside* of the ScummVM source tree, and without any need to modify files in the ScummVM source tree. This is definitely doable, but needs somebody to implement it. | Another issue is the following: There are several people who develop 3rd party engines, which use the core code from ScummVM, but are not hosted within the ScummVM repository. Since ScummVM allows building engines as loadable modules, that is fine. However, in order to ''build'' these engines, one currently has to copy them into a checkout of the ScummVM trunk, and then hook them into the build system (by modifying the [code]engines/engines.mk[/code] file). It would be so much easier if one could build an engine as a loadable module *outside* of the ScummVM source tree, and without any need to modify files in the ScummVM source tree. This is definitely doable, but needs somebody to implement it. | ||
| Line 73: | Line 72: | ||
* Generate project files for various IDEs (XCode, Eclipse, ...) from the default build system. (MSVC has already been done.) | * Generate project files for various IDEs (XCode, Eclipse, ...) from the default build system. (MSVC has already been done.) | ||
* Modify ScummVM build system | * Modify the ScummVM build system with the ability to build engines outside of the main source tree. | ||
* Enhance the build system of the ScummVM tools (they are in a separate Subversion module, and not to be confused with the tools included in the ScummVM source tree itself). | * Enhance the build system of the ScummVM tools (they are in a separate Subversion module, and not to be confused with the tools included in the ScummVM source tree itself). | ||
An alternative approach to hand-rolling everything described above would be to switch our | An alternative approach to hand-rolling everything described above would be to switch our build system to [http://www.cmake.org/ CMake]. However, since this would be a major change for us, it would have to be carefully researched first, in particular with regards to the next point: | ||
A prime requirement is that the resulting system must still work on Linux, Mac OS X and Windows, and all other platforms that use the default build system (e.g. WinCE, Nintendo DS, ...). Any negative impact on portability should be avoided. | A prime requirement is that the resulting system must still work on Linux, Mac OS X and Windows, and all other platforms that use the default build system (e.g. WinCE, Nintendo DS, ...). Any negative impact on portability should be avoided. | ||
| Line 83: | Line 82: | ||
* Good C++ skills. | * Good C++ skills. | ||
* Excellent Makefile skills | * Excellent Makefile skills. | ||
* Possibly experience with CMake, SCons or another meta-build system | * Possibly experience with CMake, SCons or another meta-build system. | ||
=== Implemented support for loadable modules on small devices === | === Implemented support for loadable modules on small devices === | ||
| Line 101: | Line 100: | ||
The module loader consists of a [http://sourceware.org/binutils/docs-2.19/ld/Scripts.html#Scripts linker script] which is used to generate custom loadable modules "manually". Furthermore, we use a custom loader for [http://en.wikipedia.org/wiki/Executable_and_Linkable_Format ELF binaries] which takes care of the run-time linking. | The module loader consists of a [http://sourceware.org/binutils/docs-2.19/ld/Scripts.html#Scripts linker script] which is used to generate custom loadable modules "manually". Furthermore, we use a custom loader for [http://en.wikipedia.org/wiki/Executable_and_Linkable_Format ELF binaries] which takes care of the run-time linking. | ||
The original Dreamcast code is for | The original Dreamcast code is for SuperH processors, and the PSP code runs on a MIPS processor. You would have to adapt these to generate and load ELF binaries for ARM processors (for Nintendo DS, WinCE, Symbian, GP2x), PowerPC (Nintendo Wii & Gamecube), or the PS2's MIPS version. | ||
Ideally, your code should be generic enough to be usable for all platforms. With two examples of the loader already in existence, this should quite doable. | Ideally, your code should be generic enough to be usable for all platforms. With two examples of the loader already in existence, this should be quite doable. | ||
Another factor to consider is that should you wish to build the loader for very memory-starved platforms such as the Nintendo DS, you will have to deal with an issue within the ScummVM architecture. Currently, the loadable modules are all initially loaded into memory. When a game is chosen by the user, the irrelevant modules are erased from memory. In order for the modules to fit into the limited Nintendo DS RAM, this method will have to change -- only one module should be loaded into memory at a time. We have some ideas on how to achieve that, talk to us. | Another factor to consider is that should you wish to build the loader for very memory-starved platforms such as the Nintendo DS, you will have to deal with an issue within the ScummVM architecture. Currently, the loadable modules are all initially loaded into memory. When a game is chosen by the user, the irrelevant modules are erased from memory. In order for the modules to fit into the limited Nintendo DS RAM, this method will have to change -- only one module should be loaded into memory at a time. We have some ideas on how to achieve that, talk to us. | ||
| Line 123: | Line 122: | ||
ScummVM has been ported to many platforms, often by simply re-using the SDL backend (which is based on [http://www.libsdl.org SDL], which by itself has been ported to many platforms, making it fairly easy to port ScummVM to any of these platforms). | ScummVM has been ported to many platforms, often by simply re-using the SDL backend (which is based on [http://www.libsdl.org SDL], which by itself has been ported to many platforms, making it fairly easy to port ScummVM to any of these platforms). | ||
But for some [[Platforms|platforms]], dedicated backends are required, either because SDL doesn't support them, or because we can't achieve all our needs by using the SDL port (e.g. because we need more speed, more control, etc.). These backends are typically made for what we call "small systems" -- systems like PDAs, SmartPhones, | But for some [[Platforms|platforms]], dedicated backends are required, either because SDL doesn't support them, or because we can't achieve all our needs by using the SDL port (e.g. because we need more speed, more control, etc.). These backends are typically made for what we call "small systems" -- systems like PDAs, SmartPhones, Internet Tablets ([[PalmOS]], [[SymbianOS]], [[Windows CE]], [[Maemo]]), or game consoles ([[Dreamcast]], [[GP2X]], [[Nintendo DS]], [[PlayStation 2]], [[PlayStation Portable]]). | ||
These systems share many features. In particular they often have no (full) keyboard and quite limited resources: Little RAM, little permanent storage space, not that much CPU power, or a limited screen resolution. | These systems share many features. In particular they often have no (full) keyboard and quite limited resources: Little RAM, little permanent storage space, not that much CPU power, or a limited screen resolution. | ||
| Line 148: | Line 147: | ||
''Background:'' | ''Background:'' | ||
ScummVM is a highly portable application. To achieve that there are different interfaces to access platform specific functionality. Our APIs are usually well documented, thus giving porters and engine authors a clear vision of how different functions / methods are supposed to work. Apart from the lower level platform abstraction APIs we usually offer more high level APIs for engine authors. | ScummVM is a highly portable application. To achieve that, there are different interfaces to access platform specific functionality. Our APIs are usually well documented, thus giving porters and engine authors a clear vision of how different functions / methods are supposed to work. Apart from the lower level platform abstraction APIs we usually offer more high level APIs for engine authors. | ||
For example we have a low-level filesystem abstraction API: FSNode. This can be used to browse directories, open files, create new | For example we have a low-level filesystem abstraction API: FSNode. This can be used to browse directories, open files, create new files for writing etc. Since the API is pretty much low level, engine authors usually do not use it directly, but use some wrapper code to access it. To prevent code duplication and take work from engine authors we introduced the Archive and SearchMan classes. SearchMan can be easily used to access game data files without fiddling with FSNode. | ||
However, experience has shown that the documentation of all these classes is not always as clear and complete as it should be, and sometimes might be misunderstood. In addition, the implementations for the various systems we run on may not adhere to the documented specification and/or regressions were introduced when doing changes to the code. Currently we have only a very minimalistic test suite in "tests/", testing only a tiny subset of our code: container classes, string handling and some stream code. Currently it is hard to do proper regression / implementation testing for other APIs, for example FSNode API. | However, experience has shown that the documentation of all these classes is not always as clear and complete as it should be, and sometimes might be misunderstood. In addition, the implementations for the various systems we run on may not adhere to the documented specification and/or regressions were introduced when doing changes to the code. Currently we have only a very minimalistic test suite in "tests/", testing only a tiny subset of our code: container classes, string handling and some stream code. Currently it is hard to do proper regression/implementation testing for other APIs, for example FSNode API. | ||
''The Task:'' | ''The Task:'' | ||
As visible from the description above, we need | As visible from the description above, we need an automated test system to help ensure our implementations are bug free. We need both a framework for the tests, as well as an extensive test suite. There are several different approaches to testing which all are of independent interest and complement each other: | ||
# Unit tests: Add many more tests to our test suite in "tests/". This would also require some enhancements to our build system, since currently we do not link against all of our code for the test suite. Also new test mechanisms are needed to make it possible to e.g. test audio & graphics output (e.g. code that can load an existing reference image and compare it to the output of a graphics primitive). It might also be helpful to enhance the "null" backend so that all unit tests can be run against that, headless (good for automated tests on a server with actual graphics/audio output. | # Unit tests: Add many more tests to our test suite in "tests/". This would also require some enhancements to our build system, since currently we do not link against all of our code for the test suite. Also new test mechanisms are needed to make it possible to e.g. test audio & graphics output (e.g. code that can load an existing reference image and compare it to the output of a graphics primitive). It might also be helpful to enhance the "null" backend so that all unit tests can be run against that, headless (good for automated tests on a server with actual graphics/audio output). | ||
# A "test" engine: Unit tests probably | # A "test" engine: Unit tests would probably only be runnable on "desktop" ports like Windows and Linux, and not on ports like PSP or NDS, due to run environment restrictions. To allow extended coverage on *all* ports, [[HOWTO-Engines|implementing a "test" engine]] would be helpful. In addition to running on all ports, the "test" engine also provides an environment to test things in an integrated, non-isolated way, unlike unit tests. It should test as many features as possible: Game detection, file loading/saving, navigating the file system, testing graphics and audio output, etc. -- and it must be possible to verify everything. I.e. compare results to reference data, output progress data on the screen, create dump logs, and so on. Lots can be done here. | ||
The | The test suite extension would be easiest to use on Linux etc. and there is already some existing code to build on. A test engine would work on all ports, including game consoles and phones, and is nice for testing in a more "holistic" and complete way. Of course best would be to have both, and also make it possible to run all unit tests from inside the test engine. | ||
Overall the test engine has the highest priority, since it would give us the possibility to test ''all'' ports without additional work. Thus the primary task would be to implement and document at least a test engine, which tests (preferably) all of ScummVM's subsystems extensively. | Overall the test engine has the highest priority, since it would give us the possibility to test ''all'' ports without additional work. Thus the primary task would be to implement and document at least a test engine, which tests (preferably) all of ScummVM's subsystems extensively. | ||
| Line 176: | Line 175: | ||
=== Add a testing framework for ScummVM's engines === | === Add a testing framework for ScummVM's engines === | ||
''Technical Contact'': [[User:Sev|Eugene Sandulenko]] | ''Technical Contact'': [[User:Sev|Eugene Sandulenko]] | ||
''Background:'' | ''Background:'' | ||
The number of ScummVM supported games grows with each release. Since 0.8.0 we are performing extensive prerelease testing. Basically, it is game replaying to find regressions. This becomes a somewhat tiresome process and takes up a lot of time. | |||
In fact there should be no changes in the gameplay in most cases, so event recording and proper replaying should be enough. There is some code for that based on [http://sourceforge.net/tracker/index.php?func=detail&aid=1738058&group_id=37116&atid=418822 Action recorder patch], but the recordings are not played consistently. The main problem is that there is no guarantee of the events to be in sync with other engine parts, so the playback may deviate, depending on the machine speed. | In fact there should be no changes in the gameplay in most cases, so event recording and proper replaying should be enough. There is some code for that based on the [http://sourceforge.net/tracker/index.php?func=detail&aid=1738058&group_id=37116&atid=418822 Action recorder patch], but the recordings are not played consistently. The main problem is that there is no guarantee of the events to be in sync with other engine parts, so the playback may deviate, depending on the machine speed. | ||
''The Task:'' | ''The Task:'' | ||
Implement proper event recorder and player. It should be multiplatform and not depend on the CPU power. I.e. it should ''guarantee'' proper event sequence and time. It should store mouse, keyboard events, as well as generated random numbers and any other functions dealing with extern world such as time. Basically it should be implemented as some globally blocking cycle which will be driven by recorded events. | Implement proper event recorder and player. It should be multiplatform and not depend on the CPU power. I.e. it should ''guarantee'' proper event sequence and time. It should store mouse, keyboard events, as well as generated random numbers and any other functions dealing with the extern world such as time. Basically it should be implemented as some globally blocking cycle which will be driven by recorded events. | ||
The format should | The format should also include to give a name to the recording, keep a screenshot, and optional savegame, so arbitrary portions of the game could be recorded. | ||
Additionally | Additionally, an automation to create screenshots at specified intervals should be implemented, so the output could be compared. | ||
The GUI should preferably include an overlay image which will indicate that the game is being recorded or played back. But it will require some work in backends. | The GUI should preferably include an overlay image which will indicate that the game is being recorded or played back. But it will require some work in backends. | ||
Optionally | Optionally, a step-by-step mode with the ability to pause the playback at any place could be implemented. And a big bonus would be the ability to annotate the recordings with on-screen messages. | ||
''Required Skills:'' | ''Required Skills:'' | ||
| Line 206: | Line 204: | ||
* Access to a non-desktop testing device (PSP, NDS, PS2, WinCE or Symbian mobile, etc.) and the ability to compile and run ScummVM on it (the ScummVM team would help with compiling and running). | * Access to a non-desktop testing device (PSP, NDS, PS2, WinCE or Symbian mobile, etc.) and the ability to compile and run ScummVM on it (the ScummVM team would help with compiling and running). | ||
=== Improve the default ScummVM backend by adding OpenGL support and providing a common base class | === Improve the default ScummVM backend by adding OpenGL support and providing a common base class === | ||
''Technical Contact:'' [[User:DJWillis|John Willis]] | ''Technical Contact:'' [[User:DJWillis|John Willis]] | ||
| Line 214: | Line 211: | ||
Since its inception, ScummVM has offered a default cross platform backend that makes use of the extensive and prolific SDL API. We have no intention to change this but we would like to extend the default backend to take advantage of things like OpenGL, and its little brother OpenGL ES, that is becoming commonplace on the latest generations of devices. | Since its inception, ScummVM has offered a default cross platform backend that makes use of the extensive and prolific SDL API. We have no intention to change this but we would like to extend the default backend to take advantage of things like OpenGL, and its little brother OpenGL ES, that is becoming commonplace on the latest generations of devices. | ||
We would also like to refactor the common aspects of the SDL backend into a common base class to better support backends that derive from it. Architecturally this could be very elegant and allow platforms to derive from the common SDL base (inc. the OpenGL and default existing one) and future platforms could also derive from the OpenGL base etc. | We would also like to refactor the common aspects of the SDL backend into a common base class to better support backends that derive from it. Architecturally, this could be very elegant and allow platforms to derive from the common SDL base (inc. the OpenGL and default existing one) and future platforms could also derive from the OpenGL base etc. | ||
There are a number of required features that the extended backend must have. | There are a number of required features that the extended backend must have. | ||
| Line 228: | Line 225: | ||
''Also required:'' | ''Also required:'' | ||
* This task will require testing under both OpenGL and OpenGL ES | * This task will require testing under both OpenGL and OpenGL ES APIs. | ||
** There are sample OpenGL ES and OpenGL implementations for development available from several sources. For OpenGL ES the | ** There are sample OpenGL ES and OpenGL implementations for development available from several sources. (For OpenGL ES, the [http://www.imgtec.com/powervr/insider/powervr-sdk.asp PowerVR SGX SDK] should be of interest.) | ||
''Required Skills:'' | ''Required Skills:'' | ||
| Line 235: | Line 232: | ||
* Reasonable C++ skills. | * Reasonable C++ skills. | ||
* Refactoring skills. | * Refactoring skills. | ||
* Knowledge of the OpenGL and SDL | * Knowledge of the OpenGL and SDL APIs. | ||
* Knowledge of the default SDL ScummVM platform backend is desirable. | * Knowledge of the default SDL ScummVM platform backend is desirable. | ||
| Line 254: | Line 251: | ||
Let's start by reviewing the current state of affairs. Engines specify during a startup a set of [http://doxygen.scummvm.org/dd/d2e/mididrv_8h.html#0b8636a2c41edcf47a84282ba7a00971 audio output flags]. These flags try to describe the output ''types'' supported by the engine resp. the current game. We then use some ( | Let's start by reviewing the current state of affairs. Engines specify during a startup a set of [http://doxygen.scummvm.org/dd/d2e/mididrv_8h.html#0b8636a2c41edcf47a84282ba7a00971 audio output flags]. These flags try to describe the output ''types'' supported by the engine resp. the current game. We then use some (rather simple and stupid) "detection" method based on these to determine the output ''driver'' to use. (There is no support for output ''devices'' in ScummVM right now, and adding support for this is not part of this task, but of course we won't complain if it happens either.) Unfortunately, this detection does not include all the above mentioned output types, for example it does not allow to specify "Amiga", which recently turned out to be a major problem for the WIP SCI engine, since its Amiga games support both MIDI (in this case, MT-32 MIDI) and Amiga sound output. Now there is no (clean) way of checking whether the user wanted to use Amiga sound or MT-32 MIDI. Next, it does directly map to some concrete audio ''driver'' (like ALSA or Windows MIDI for MIDI). This results in an unclean way of checking in the engine what exactly the selected output type is (this might be required to load special music data files for the different output types). To further complicate things, the current "AdLib" music driver is also used to emulate MIDI sound (useful on low end system which don't properly support MIDI output on their own), causing several engines which only use MIDI for audio output, and which don't contain any AdLib data, to still specify the MDT_ADLIB audio flag. | ||
For the user we currently always show all concrete output implementations (except digital sound!) in the GUI, without distinguishing between output types and output drivers. We also always show everything, without considering what a given game actually supports. So currently the user can select "AdLib" sound for a game which does not contain any AdLib sound data. This is rather annoying. | For the user, we currently always show all concrete output implementations (except digital sound!) in the GUI, without distinguishing between output types and output drivers. We also always show everything, without considering what a given game actually supports. So currently the user can select "AdLib" sound for a game which does not contain any AdLib sound data. This is rather annoying. | ||
| Line 267: | Line 264: | ||
So if you plan to take this task, you will still have a lot of research to do, to find the best separation :-). Of course the technical contacts, as well as the rest of the ScummVM team, would collaborate with you on this. | So if you plan to take this task, you will still have a lot of research to do, to find the best separation :-). Of course the technical contacts, as well as the rest of the ScummVM team, would collaborate with you on this. | ||
Roughly the same layers should also be visible in the GUI configuration to allow for a much more user friendly output configuration. The user might additionally configure a concrete output driver like ALSA to use a | Roughly the same layers should also be visible in the GUI configuration to allow for a much more user friendly output configuration. The user might additionally configure a concrete output driver like ALSA to use a specific setting (like in the output port for ALSA); think of this as a baby version of output devices (for true output devices, one would have to support multiple such configurations for each driver, configurable and selectable via the command line, GUI and the internal API). See the [[OpenTasks#MIDI device configuration|MIDI device configuration]] task for more thoughts about this. (Note that it would probably be best to tackle both tasks in a single Google Summer of Code project!) | ||
Of course this is not set in stone yet; if you have some other great idea how to tackle this, we are happy for your feedback :-). | Of course this is not set in stone yet; if you have some other great idea how to tackle this, we are happy for your feedback :-). | ||
| Line 276: | Line 273: | ||
* Adopt all engines to use it. | * Adopt all engines to use it. | ||
* Adopt the GUI to reflect the changes. | * Adopt the GUI to reflect the changes. | ||
* (Optionally) also extend all engines to supply a list of supported audio devices for a specific game target | * (Optionally) also extend all engines to supply a list of supported audio devices for a specific game target (this would be done via the help of engine authors). | ||
* All configuration must be serializable for our configuration file. (And it would be nice to have some auto update feature for old configuration files!) | * All configuration must be serializable for our configuration file. (And it would be nice to have some auto update feature for old configuration files!) | ||
| Line 286: | Line 283: | ||
Many of the adventures supported by ScummVM make use of MIDI music. Which is why we already include several device drivers for various MIDI APIs and emulators (e.g. ALSA, Windows MIDI, Mac OS X CoreAudio/CoreMIDI, fluidsynth...). | Many of the adventures supported by ScummVM make use of MIDI music. Which is why we already include several device drivers for various MIDI APIs and emulators (e.g. ALSA, Windows MIDI, Mac OS X CoreAudio/CoreMIDI, fluidsynth...). | ||
Right now, the MIDI drivers (and more generally, audio drivers, see the previous task) are treated by ScummVM in a rather | Right now, the MIDI drivers (and more generally, audio drivers, see the previous task) are treated by ScummVM in a rather simple minded fashion: Either a driver is linked in and hence "available", or not. It's not possible to configure anything about them (like ports to be used etc.), nor does it ever take into account that a single driver might correspond to multiple devices (after all, you can plug several sequencers into your MIDI port; or you could have configured several different sound font settings in your MIDI emulator). These two issues are very tightly related. | ||
<span style="color:red">TODO improve and extend description, and integrate it better with the Audio output selection task</span> | <span style="color:red">TODO improve and extend description, and integrate it better with the Audio output selection task</span> | ||
| Line 294: | Line 291: | ||
''The Task:'' | ''The Task:'' | ||
* Add an API for querying the OSystem backend for a list of available MIDI devices (not drivers) | * Add an API for querying the OSystem backend for a list of available MIDI devices (not drivers). | ||
* Information about the selected device must be serializable, so that it can be stored in the config file | * Information about the selected device must be serializable, so that it can be stored in the config file. | ||
* Selection of devices via command line should be possible | * Selection of devices via command line should be possible. | ||
* It must deal with devices being added/removed (at least between runs of ScummVM, ideally also while ScummVM is running) | * It must deal with devices being added/removed (at least between runs of ScummVM, ideally also while ScummVM is running). | ||
* Devices should be configurable via the GUI; this needs to be done in a flexible (different devices/drivers offer different settings) and portable fashion. | * Devices should be configurable via the GUI; this needs to be done in a flexible (different devices/drivers offer different settings) and portable fashion. | ||
| Line 305: | Line 302: | ||
''Background:'' | ''Background:'' | ||
Several of our games make use of the XMIDI format. We already have a parser for it (see sound/midiparser_xmidi.cpp), which was based on code from the [http://exult. | Several of our games make use of the XMIDI format. We already have a parser for it (see sound/midiparser_xmidi.cpp), which was based on code from the [http://exult.sourceforge.net/ Exult] project, but it is incomplete. | ||
''The Task:'' | ''The Task:'' | ||
| Line 311: | Line 308: | ||
Specifically, we require support for XMIDI_CONTROLLER_FOR_LOOP and XMIDI_CONTROLLER_NEXT_BREAK. There are many more XMIDI controllers, however, and ideally support for all of them would be great. Some references: | Specifically, we require support for XMIDI_CONTROLLER_FOR_LOOP and XMIDI_CONTROLLER_NEXT_BREAK. There are many more XMIDI controllers, however, and ideally support for all of them would be great. Some references: | ||
* The XMIDI code from the [http://exult. | * The XMIDI code from the [http://exult.sourceforge.net/ Exult] project (see Exult's audio/midi_drivers/XMidiSequence.cpp) could be used as a reference again. | ||
* In addition, [http://pentagram. | * In addition, [http://pentagram.sourceforge.net/ Pentagram] has XMidi code based on the Exult code, but may be cleaner (see [http://pentagram.svn.sourceforge.net/viewvc/pentagram/pentagram/trunk/audio/midi/ here]). | ||
* Another good reference for this task is the AIL library which has been recently open-sourced by its [http://www.thegleam.com/ke5fx/ author]. This is | * Another good reference for this task is the AIL library which has been recently open-sourced by its [http://www.thegleam.com/ke5fx/ author]. This is in fact where XMIDI comes from, and it even contains specs. To find them, unzip AIL2.ZIP; in there you find A214_D2.ZIP, which when unzipped contains a directory DOC, which in turn contains ASCIIDOC.ZIP, which then contains XMIDI.TXT and some other files which might be of use (e.g. NOTES.TXT). | ||
=== Accolade | === Accolade MIDI parser === | ||
''Technical Contact'': [[User:Kirben|Travis Howell]] | ''Technical Contact'': [[User:Kirben|Travis Howell]] | ||
''Background:'' | ''Background:'' | ||
The Elvira 1 (DOS), Elvira 2 (DOS), Waxworks (DOS) games of the AGOS engine use the [http://adplug.sourceforge.net/library/entry.php?file=db/AccoladeMidi.txt Accolade MIDI] format for music. The current parser (see engines/agos/midiparser_s1d.cpp), is based off looking at the music in the DOS Floppy Demo of Simon the Sorcerer 1 ( | The Elvira 1 (DOS), Elvira 2 (DOS), Waxworks (DOS) games of the AGOS engine use the [http://adplug.sourceforge.net/library/entry.php?file=db/AccoladeMidi.txt Accolade MIDI] format for music. The current parser (see engines/agos/midiparser_s1d.cpp), is based off looking at the music in the DOS Floppy Demo of Simon the Sorcerer 1 (which was based on the Waxworks engine) and guesswork. The current code frequently crashes when changing locations in Waxworks, due to invalid MIDI data being passed along. | ||
''The Task:'' | ''The Task:'' | ||
Update the current parser, to completely support the [http://adplug.sourceforge.net/library/entry.php?file=db/AccoladeMidi.txt Accolade MIDI] format used by these games. There are already comments in the current code, about where each additional MIDI event is triggered in the music of these games. | |||
=== Support for Acorn Audio Format: Desktop Tracker === | === Support for Acorn Audio Format: Desktop Tracker === | ||
''Technical Contact'': [[User:Sev|Eugene Sandulenko]], [[User:DrMcCoy|Sven Hesse]], [[User:Jubanka|Kostas Nakos]] | ''Technical Contact'': [[User:Sev|Eugene Sandulenko]], [[User:DrMcCoy|Sven Hesse]], [[User:Jubanka|Kostas Nakos]] | ||
| Line 333: | Line 329: | ||
Since we're adding (and have added so far) support for different platform game versions, we need support for the sound files too. | Since we're adding (and have added so far) support for different platform game versions, we need support for the sound files too. | ||
Currently we need support for following format: | Currently we need support for the following format: | ||
* Desktop Tracker format used by music in the Acorn disk version of Simon the Sorcerer 1. [http://www.tribbeck.com/software/sonor/dtt/manual/appx_f.html | * Desktop Tracker format used by the music in the Acorn disk version of Simon the Sorcerer 1. [http://www.tribbeck.com/software/sonor/dtt/manual/appx_f.html Information] about the specifications of the Desktop Tracker format and [http://members.optusnet.com.au/wormmon/mods/ sample mods] are available. | ||
''The Task:'' | ''The Task:'' | ||
* Extend our general MOD player to support the mentioned format | * Extend our general MOD player to support the mentioned format. | ||
''Required Skills:'' | ''Required Skills:'' | ||
| Line 353: | Line 349: | ||
<span style="color:red">TODO: Is this task really suitable for this wiki page? It seems to be very specific and has a high entry barrier (esp. for the Apple II part). At the very least, a more detailed description is necessary.</span> | <span style="color:red">TODO: Is this task really suitable for this wiki page? It seems to be very specific and has a high entry barrier (esp. for the Apple II part). At the very least, a more detailed description is necessary.</span> | ||
* Add support for beeper sound used by the Apple II version of Maniac Mansion to the SCUMM engine. Unfortunately | * Add support for the beeper sound used by the Apple II version of Maniac Mansion to the SCUMM engine. Unfortunately, no demo versions are available, so you need to own the game. Known information about the resource format can be found [[SCUMM/V0/Maniac Mansion Apple II|here]]. | ||
* Add support for sound format used by Macintosh version of Loom to SCUMM engine. Known [http://sourceforge.net/tracker/index.php?func=detail&aid=824221&group_id=37116&atid=418823 | * Add support for the sound format used by the Macintosh version of Loom to the SCUMM engine. Known [http://sourceforge.net/tracker/index.php?func=detail&aid=824221&group_id=37116&atid=418823 information] about the structure of the sound resources used is available. If you don't own the game, the LucasArts Mac CD Game pack is usually available via eBay. | ||
== Tools == | == Tools == | ||
=== Tools: Game script (bytecode) decompiler === | === Tools: Game script (bytecode) decompiler === | ||
''Technical Contact'': [[User:Fingolfin|Max Horn]], | ''Technical Contact'': [[User:Fingolfin|Max Horn]], [[User:LordHoto|Johannes Schickel]] | ||
''Background:'' | ''Background:'' | ||
| Line 366: | Line 362: | ||
Most adventure engines ScummVM supports are driven by bytecode scripts, which control the game behavior. For various reasons (in particular, debugging), it's very useful to be able to take such a script and decompile it into something resembling regular code (with variables, loops, if statements etc.). We have a tool for doing this for the bytecode used by SCUMM games. This tool is called descumm. We also have similar (but less advanced) tools for various other engines, in our [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/trunk tools module in SVN]. | Most adventure engines ScummVM supports are driven by bytecode scripts, which control the game behavior. For various reasons (in particular, debugging), it's very useful to be able to take such a script and decompile it into something resembling regular code (with variables, loops, if statements etc.). We have a tool for doing this for the bytecode used by SCUMM games. This tool is called descumm. We also have similar (but less advanced) tools for various other engines, in our [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/trunk tools module in SVN]. | ||
Currently, descumm | Currently, descumm uses a relatively crude heuristic to detect "if" statements, loops and so on. In particular, it only supports loops with the condition at the start (think "while(...) { ... }" ), not such with the condition at the end (as in "do { ... } while(...)" and also suffers from various other limitations (like not being able to correctly recognize "continue" and "break" statements. | ||
The whole thing is somewhat complicated by the fact that there are two main versions of this bytecode: V5 and older (register based bytecode), and V6 and newer (stack based). We handle the registed based code quite well, but not so the stack based one. And in fact most non-SCUMM engines use a stack based virtual machine. | The whole thing is somewhat complicated by the fact that there are two main versions of this bytecode: V5 and older (register based bytecode), and V6 and newer (stack based). We handle the registed based code quite well, but not so the stack based one. And in fact most non-SCUMM engines use a stack based virtual machine. | ||
| Line 374: | Line 370: | ||
Write a ''generic'' bytecode decompiler with the goal of ultimately being able to decompile bytecode of all games which use a stack based bytecode virtual machine. You can look at existing work from GSoC 2007, our descumm compiler (only SCUMM v6 and newer is stack based) and some other de*.cpp files in our [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/trunk/ tools SVN module]. Your goal should be to support at least two different bytecode systems (i.e., from two different ScummVM engines). But everything should be designed and implemented with the ultimate goal of being able to support all such bytecode systems (e.g. by providing a suitable API or subclassing facilities that make it possible to hook in parsers for arbitrary bytecode variants). Engines which use a stack based virtual machine include SCUMM v6+, KYRA, SWORD1, SWORD2, ... For many of these, free game demos are available, too. We would help you by providing dumps of scripts etc. | Write a ''generic'' bytecode decompiler with the goal of ultimately being able to decompile bytecode of all games which use a stack based bytecode virtual machine. You can look at existing work from GSoC 2007, our descumm compiler (only SCUMM v6 and newer is stack based) and some other de*.cpp files in our [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/trunk/ tools SVN module]. Your goal should be to support at least two different bytecode systems (i.e., from two different ScummVM engines). But everything should be designed and implemented with the ultimate goal of being able to support all such bytecode systems (e.g. by providing a suitable API or subclassing facilities that make it possible to hook in parsers for arbitrary bytecode variants). Engines which use a stack based virtual machine include SCUMM v6+, KYRA, SWORD1, SWORD2, ... For many of these, free game demos are available, too. We would help you by providing dumps of scripts etc. | ||
A good starting point for this might be the | A good starting point for this might be the [http://jode.sourceforge.net/ Jode] Java bytecode decompiler which does a rather good job at decompiling. Another useful site might be [http://www.program-transformation.org/Transform/DeCompilation], and [http://vanemmerikfamily.com/mike/master.pdf this thesis]. The technical contact has several ideas on how to approach this project, too -- as usual, you are expected to talk to us and ask for help and ideas! | ||
The tool should run on at least | The tool should run on at least Linux, Mac OS X and Windows. Acceptable languages include C/C++, Python, Perl -- other languages might be OK, but please consult with us first -- after all, other developers also will have to use and maintain your code. For the same reason it is obviously mandatory that your code be well documented. | ||
Actually, we probably would prefer if it was written in C++ and was written as kind of a library, with the command line tool just a frontend to the library. This "library" approach would make it possible to include the decompiler into ScummVM itself, making it possible to decompile scripts on the fly from ScummVM's built-in debugger console. | Actually, we probably would prefer if it was written in C++ and was written as kind of a library, with the command line tool just a frontend to the library. This "library" approach would make it possible to include the decompiler into ScummVM itself, making it possible to decompile scripts on the fly from ScummVM's built-in debugger console. | ||
| Line 383: | Line 379: | ||
''Note:'' | ''Note:'' | ||
This task was being worked on as part of the Google Summer of Code 2007 and again 2009, but unfortunately in both cases the result sadly is incomplete and never reached a status we | This task was being worked on as part of the Google Summer of Code 2007 and again 2009, but unfortunately in both cases the result sadly is incomplete and never reached a status we were satisfactory with. It is probably best to start from scratch, but you can find the [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/branches/gsoc2007-decompiler/ code from 2007 here] and the [http://scummvm.svn.sourceforge.net/viewvc/scummvm/tools/branches/gsoc2009-decompiler/decompiler/ code from 2009 here]. | ||
== Engine/game specific == | == Engine/game specific == | ||
| Line 392: | Line 388: | ||
''Background:'' | ''Background:'' | ||
The | The CruisE engine also started out as an external project by [[User:Yazoo|Yaz0r]]. Originally it was written in plain C. | ||
ScummVM is a C++ project, so we need to objectify this engine without changing/breaking its behavior. The engine is fairly complete, but also suffers from portability problems, that is it works correctly only under little-endian, alignment-agnostic CPUs. | ScummVM is a C++ project, so we need to objectify this engine without changing/breaking its behavior. The engine is fairly complete, but also suffers from portability problems, that is, it works correctly only under little-endian, alignment-agnostic CPUs. | ||
The engine itself is well-structured, hence many functions / variables which might be good candidates for being grouped together into a C++ class are already grouped by files. | The engine itself is well-structured, hence many functions/variables which might be good candidates for being grouped together into a C++ class are already grouped by files. | ||
No deep knowledge of the engine internals is required | No deep knowledge of the engine internals is required either. | ||
We have previously "objectified" several other engines, namely SAGA, Gob and AGI, so one can learn a lot about various approaches how to do this by tracing through our SVN repository. | We have previously "objectified" several other engines, namely SAGA, Gob and AGI, so one can learn a lot about various approaches how to do this by tracing through our SVN repository. | ||
| Line 406: | Line 402: | ||
''Background:'' | ''Background:'' | ||
Our AGI engine is in quite good shape. Still it misses some important aspects. | Our AGI engine is in quite a good shape. Still it misses some important aspects. | ||
''The Task:'' | ''The Task:'' | ||
* NAGI has better Tandy music support, particularly noise channel emulation is better there. NAGI license lets use its code directly in ScummVM. For the most authentic model MAME could be referenced, though its license is not GPL-compatible | * NAGI has better Tandy music support, particularly noise channel emulation is better there. The NAGI license lets us use its code directly in ScummVM. For the most authentic model, MAME could be referenced, though its license is not GPL-compatible. | ||
* Add support for AGI 2.01 games, such as Donald Duck Playground | * Add support for AGI 2.01 games, such as Donald Duck's Playground. | ||
* Add support for AGI0 and AGI1 (bootable floppies) games | * Add support for AGI0 and AGI1 (bootable floppies) games. | ||
* Document all differences between AGI versions on Wiki and probably in code | * Document all differences between AGI versions on Wiki and probably in code. | ||
* Cover more tasks from [[AGI/TODO]] page | * Cover more tasks from the [[AGI/TODO]] page. | ||
''Required Skills:'' | ''Required Skills:'' | ||