OpenTasks/Audio/Audio Output Selection

Open Task
Task Name	Audio Output Selection
Technical Contact(s)	Johannes Schickel
Subsystem	Audio

Background:

Right now, our user interface and our internal API for selecting and controlling audio output are very limited. They do not properly differentiate between the output type, the output driver and the output device. To explain the difference:

An output type could be MIDI, AdLib, (PC) speaker, Amiga, etc.; MIDI could be further split into GM (General MIDI), MT-32, Roland, etc.
An output driver is a concrete implementation of an output type. For example, we have two AdLib/OPL drivers right now; and many MIDI drivers.
Finally, a single driver could control multiple devices; e.g. you might have a General MIDI synthesizer *and* an MT-32 attached to your computer; or you might want to use fluidsynth with two different sound fonts, each shown as a separate "device".

These different concepts are currently not properly distinguished. Furthermore, the approach we use to deal with MIDI variants like GM, MT-32 and Roland, is rather weird and irregular. This all results in a rather confusing way to select the output device -- for engine developers, for audio output developers, and for the user in the GUI. Clearly, this is a bad thing for all involved parties and hence the goal of this task is to fix this mess.

Let's start by reviewing the current state of affairs. Engines specify during a startup a set of audio output flags. These flags try to describe the output types supported by the engine resp. the current game. We then use some (rather simple and stupid) "detection" method based on these to determine the output device to use. Unfortunately, this detection does not include all the above mentioned output types, for example it does not allow to specify "Amiga", which recently turned out to be a major problem for the work-in-progress SCI engine, since its Amiga games support both MIDI (in this case, MT-32 MIDI) and Amiga sound output. Now there is no (clean) way of checking whether the user wanted to use Amiga sound or MT-32 MIDI. This results in an unclean way of checking in the engine what exactly the selected output type is (this might be required to load special music data files for the different output types). To further complicate things, the current "AdLib" music device is actually an MIDI emulation device (useful on low end system which don't properly support MIDI output on their own), but on the other hand says it's an AdLib type device, causing several engines which only use MIDI for audio output, and which don't contain any AdLib data, to still specify the MDT_ADLIB audio flag.

For the user, we currently always show all concrete output devices (except digital sound!) in the GUI, without distinguishing between output types and output drivers. We also always show everything, without considering what a given game actually supports. So currently the user can select "AdLib" sound for a game which does not contain any AdLib sound data. This is rather annoying.

To overcome these issues it would be best to change our audio output configuration API to use different layers of configuration:

The first layer would be the output type. As described above, this could include General MIDI, MT-32 MIDI, AdLib (or generic OPL* output), (PC) Speaker (this might also include PCjr and CMS, to be determined later on), Amiga, FM-TOWNS, Digital Output etc. Note that this is no final list; making this list concrete is part of the task! For example, one might want to have a single MIDI output type, and introduce a notion of subtype to distinguish between GM, MT-32 and Roland MIDI.
The next layer would probably be a concrete output implementation, i.e. output drivers. Examples include Windows MIDI or ALSA for MIDI. Note that a single driver could potentially support multiple types: A user may usually use the Windows MIDI driver for GM output, via software synthesis, but may also own a real MT-32. This user may then wish to use the MT-32 via the Windows MIDI driver. Hence, depending on the output device resp. the user choice, the Windows MIDI driver would accept any MIDI variant. On the other hand, the MT-32 software emulator might only support MT-32 data.
One could also insert an extra conversion layer which tries to (optionally) translate between the various MIDI variants: So if a game requires MT-32 output but the user only has a GM device available, the MT-32 data could be translated on the fly (as good as possible) to GM before being sent to the actual output driver. Such conversions currently happen in engines. Sharing this code might be beneficial; on the other hand, it might turn out that each engine knows best how to perform this translation, in which case we should leave it up to the engines to perform such a conversion (or not). Once again, deciding this is part of the task.

So if you plan to take this task, you will still have a lot of research to do, to find the best separation :-). Of course the technical contacts, as well as the rest of the ScummVM team, would collaborate with you on this.

Roughly the same layers should also be visible in the GUI configuration to allow for a much more user friendly output configuration. The user might additionally configure a concrete output driver like ALSA to use a specific setting (like in the output port for ALSA); think of this as a baby version of output devices (for true output devices, one would have to support multiple such configurations for each driver, configurable and selectable via the command line, GUI and the internal API). See the MIDI device configuration task for more thoughts about this. (Note that it would probably be best to tackle both tasks in a single Google Summer of Code project!)

Of course this is not set in stone yet; if you have some other great idea how to tackle this, we are happy for your feedback :-).

The Task:

Design a proper layer based audio output system.
Implement the API.
Adopt all engines to use it.
Adopt the GUI to reflect the changes.
(Optionally) also extend all engines to supply a list of supported audio devices for a specific game target (this would be done via the help of engine authors).
All configuration must be serializable for our configuration file. (And it would be nice to have some auto update feature for old configuration files!)