Difference between revisions of "SCI/SCI32 Mac Support Status"

Latest revision as of 16:15, 22 May 2022

Status

This page is an attempt to track the state of ScummVM support for SCI32 Mac games. Aside from Gabriel Knight 1, they are not supported yet but we're making good progress. Many are now playable and all of them can at least boot to the title screen.

Data Files

Working with data files from SCI32 Macintosh CDs is difficult. Some of these difficulties are for users while others are problems for us that we haven't solved yet. It's not just one thing, it's the combination:

Resource forks: Game data files use Macintosh resource forks while individual resource files use data forks.
Hidden files: Most data files and directories are flagged Hidden. This is an extraction obstacle for software that respects this flag, such as classic Finder.
Thousands of files: Each speech audio resource is stored as an individual file. Some games have up to 10,000 files. Our SCI resource loader is not designed for this large amount. On startup it performs work on each file and this creates delays that can last minutes.
Folder structure: Data files are scattered across folders, some hidden and some not, with duplicates. Some of these duplicates appear to contain no resources. The idea was to present a folder that users would copy to their hard drive with the executable and one or two data files, leaving the others on disc and invisible.
Many data files: Games have many small Data## files instead of a few large volume files. Our Advanced Detector only supports 13 files.
Multi-disc: The structure of multi-disc games is at odds with our detection model. Each disc has many small Data## files with the same filenames as those on other discs. They can't all be placed in the same directory with those names, but there are also far too many to require manual renaming. It seems like some kind of subdirectory scheme will be required. As mentioned above, the Advanced Detector doesn't have support enough entries for even one disc of these games, let alone six. Given that setting up these files properly could be error prone, not being able to detect missing files would be a problem.

Update: The relatively recent ScummVM Dumper Companion helps with a lot of this. Multi-disc is currently the biggest obstacle.

Real Talk

I (sluicebox) have decompiled the scripts of every SCI32 Mac game and compared them to the PC versions. Aside from one joke in Phantasmagoria and an updated birthday in Space Quest 6, the Mac versions don't seem to offer interesting differences beyond native (worse) cursors and some different audio formats. It's fun to add support for them where possible, but it's probably not worth significantly complicating the codebase just to for these rare versions that are more difficult to setup and often more limited than the PC versions. For example, Space Quest 6 Mac stores music in a unique way (regular MIDI files!) that breaks core assumptions in our audio code. No other game does this. If that requires a major audio refactor, it would be hard to justify. So far, I've been able to fix Mac incompatibilities and add Mac features with low impact changes. What's left is the hard stuff.

Later SCI32 games were mostly distributed on CDs with both PC and Macintosh partitions so these discs already have the working PC files anyway. This reduces the practical benefits of Mac support for these games even further.

General TODOs

Determine how to handle multi-disc data files and implement it. (Current blocker for Phant1 and GK2)
Improve load times when faced with thousands of patch files. (This would help PC fan-patches too: [Bug #11353] )
Add engine support for a library of standard MIDI files (SQ6 only)
Implement kPlatform Mac subops for saving and restoring in Lighthouse.
Speed throttling (Mac games don't call kGameIsRestarting, it's luck that PC games did since it was a no-op in SCI32)
Document game file extraction for users

SCI32 Mac Games

Game	Status
GK1	Supported!
KQ7	Needs testing, the game seems playable.
LSL6 HI-RES	Game has been successfully completed!
PQ4	Needs testing, the game seems playable.
GK2	Determine how to name/detect the multi-disc Data files
HOYLE5	Needs testing, the game seems playable. (I won at Hearts and Gin Rummy a few times)
MOTHERGOOSE HI-RES	Needs testing, the game seems playable.
PHANTASMAGORIA	Determine how to name/detect the multi-disc Data files
SHIVERS	Needs testing, the game seems playable.
SQ6	No music: it's stored in resource fork of "Midi Library" as standard MIDI files (unlike any other game)
TORIN	Needs testing, the game seems playable.
LSL7	Needs testing, the game seems playable.
LIGHTHOUSE	Script crash when starting new game (multiple ExitFeature classes and one is broken) Audio crash at end of opening credits Requires kPlatform subops for saving and loading (see below) Determine how to name/detect the multi-disc Data files
RAMA	Determine how to name/detect the multi-disc Data files Error when saving additional saved games No transparencies on title screen
PQSWAT	Determine how to name/detect the multi-disc Data files No transparencies on title intro screen

kPlatform subops

kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts. I have implemented these for every game except Lighthouse since it's not playable enough to test this yet. I'm leaving all of these kPlatform notes here until Lighthouse's subops are done:

KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. These subops take parameters similar to kSave and kRestore.
Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.

HOYLE5 Floppy

There is a Mac floppy version of HOYLE5 but currently not much is known about it.

@@ Line 1: / Line 1: @@
 ==Status==
-This page is an attempt to track the state of ScummVM support for SCI32 Mac games. They are not supported yet. The work stalled for several years and it is unclear how much is left to do. I (sluicebox) gathered most of the Mac games to determine their current blockers. In the process I found several easy fixes that got some games to boot and play audio for the first time. Many of the others are one or two issues away from booting. We could be closer than we think.
+This page is an attempt to track the state of ScummVM support for SCI32 Mac games. Aside from Gabriel Knight 1, they are not supported yet but we're making good progress. Many are now playable and all of them can at least boot to the title screen.
 ==Data Files==
-Extracting data files from SCI32 Macintosh CDs is particularly difficult. Unless a reasonable method is discovered and documented, it seems unlikely that the average user will be able to take advantage of ScummVM support.
+Working with data files from SCI32 Macintosh CDs is difficult. Some of these difficulties are for users while others are problems for us that we haven't solved yet. It's not just one thing, it's the combination:
-Extracting Macintosh files is already complicated by resource and data forks. SCI resources are in the resource fork and many users have trouble with just that part. It's often necessary to use a real or emulated classic Macintosh at some point in the process. SCI32 CDs complicate this further by setting the Hidden flag on files and directories. The Macintosh Hidden file system flag is like the Windows one in that it prevents a file from appearing in Finder. The difference is that Macintosh has no Show Hidden Files setting. Instead, third party programs such as File Buddy must be used to reveal and copy these otherwise invisible files. Sierra hid all files that didn't need to be copied to the hard disk, which is all of the media files and most of the data files. As a final complication, some data files were duplicated between these hidden directories and the visible ones, and different data files use the same name on different CDs.
+*Resource forks: Game data files use Macintosh resource forks while individual resource files use data forks.
+* Hidden files: Most data files and directories are flagged Hidden. This is an extraction obstacle for software that respects this flag, such as classic Finder.
+*Thousands of files: Each speech audio resource is stored as an individual file. Some games have up to 10,000 files. Our SCI resource loader is not designed for this large amount. On startup it performs work on each file and this creates delays that can last minutes.
+*Folder structure: Data files are scattered across folders, some hidden and some not, with duplicates. Some of these duplicates appear to contain no resources. The idea was to present a folder that users would copy to their hard drive with the executable and one or two data files, leaving the others on disc and invisible.
+*Many data files: Games have many small Data## files instead of a few large volume files. Our Advanced Detector only supports 13 files.
+*Multi-disc: The structure of multi-disc games is at odds with our detection model. Each disc has many small Data## files with the same filenames as those on other discs. They can't all be placed in the same directory with those names, but there are also far too many to require manual renaming. It seems like some kind of subdirectory scheme will be required. As mentioned above, the Advanced Detector doesn't have support enough entries for even one disc of these games, let alone six. Given that setting up these files properly could be error prone, not being able to detect missing files would be a problem.
-There is also a performance problem that needs to be solved for SCI32 Mac games. The CD versions contain as many as ''ten thousand'' media files which slows the loader to a crawl when starting a game. Each audio resource is in its own file instead of a resource volume and the loader is doing work on each.
+''Update'': The relatively recent [https://www.scummvm.org/dumper-companion ScummVM Dumper Companion] helps with a lot of this. Multi-disc is currently the biggest obstacle.
+==Real Talk==
+I (sluicebox) have decompiled the scripts of every SCI32 Mac game and compared them to the PC versions. Aside from one joke in Phantasmagoria and an updated birthday in Space Quest 6, the Mac versions don't seem to offer interesting differences beyond native (worse) cursors and some different audio formats. It's fun to add support for them where possible, but it's probably not worth significantly complicating the codebase just to for these rare versions that are more difficult to setup and often more limited than the PC versions. For example, Space Quest 6 Mac stores music in a unique way (regular MIDI files!) that breaks core assumptions in our audio code. No other game does this. If that requires a major audio refactor, it would be hard to justify. So far, I've been able to fix Mac incompatibilities and add Mac features with low impact changes. What's left is the hard stuff.
+Later SCI32 games were mostly distributed on CDs with both PC and Macintosh partitions so these discs already have the working PC files anyway. This reduces the practical benefits of Mac support for these games even further.
 ==General TODOs==
-* Fix "Failed to initialize sound driver!" error when loading SCI2.1+ games (see patch below for bypassing this)
+* Determine how to handle multi-disc data files and implement it. (Current blocker for Phant1 and GK2)
-* Fix screen items not being cleared when restoring
+* Improve load times when faced with thousands of patch files. (This would help PC fan-patches too: {{Tracker|id=11353}} )
-* Fix palette handling. Games either have corrupt palettes at startup or soon afterwards.
+* Add engine support for a library of standard MIDI files (SQ6 only)
-* Use correct kDoSound subop numbers. Mac versions of SCI2.1 Middle and later continue using older subop numbers.
+* Implement kPlatform Mac subops for saving and restoring in Lighthouse.
-* Implement support for Mac color cursors. They are currently black and white and not upscaled.
+* Speed throttling (Mac games don't call kGameIsRestarting, it's luck that PC games did since it was a no-op in SCI32)
-* Implement kPlatform Mac subops for saving and restoring in KQ7, Shivers, Lighthouse, and Mother Goose.
+* Document game file extraction for users
-* Implement, or workaround, kPlatform Mac subop 12 for setting music volume (Phantasmagoria only)
-* Games seem too dark
-* Verify accuracy of music playback
 ==SCI32 Mac Games==
-{|border="1"  cellspacing="0" cellpadding="5" width=75%
+{| width="75%" cellspacing="0" cellpadding="5" border="1"
-! style="text-align:left;"| Game
+! style="text-align:left;" |Game
-! style="text-align:left;"| Blockers
+! style="text-align:left;" |Status
 |-
 |GK1
 |
-* Palette corruption after a while.
+*Supported!
 |-
 |KQ7
 |
-* Requires kPlatform subops for saving and loading (see below)
+* Needs testing, the game seems playable.
-* Intro and other movies require QuickTime to be implemented in video player
 |-
 |LSL6 HI-RES
 |
-* Palette corruption.
+*Game has been successfully completed!
 |-
 |PQ4
 |
-* Palette corruption.
+*Needs testing, the game seems playable.
-* Introduction error about too many screen items.
 |-
-|GK2
+| GK2
 |
-* "Failed to initialize sound driver!"
+*Determine how to name/detect the multi-disc Data files
-* Palette corruption.
 |-
 |HOYLE5
 |
-* "Failed to initialize sound driver!"
+* Needs testing, the game seems playable. (I won at Hearts and Gin Rummy a few times)
-* Wrong kDoSound subops.
 |-
 |MOTHERGOOSE HI-RES
 |
-* Requires kPlatform 0 6,7,8,9,10,11 subops for saving and loading, prevents booting
+*Needs testing, the game seems playable.
 |-
-|PHANTASMAGORIA
+| PHANTASMAGORIA
 |
-* "Failed to initialize sound driver!"
+*Determine how to name/detect the multi-disc Data files
-* "kArrayFill signature mismatch"
-* Requires a kPlatform subop for setting music volume (see below)
 |-
 |SHIVERS
 |
-* "Failed to initialize sound driver!"
+*Needs testing, the game seems playable.
-* Wrong kDoSound subops.
-* Requires kPlatform subops for saving and loading (see below)
 |-
-|SQ6
+| SQ6
 |
-* "Failed to initialize sound driver!"
+*No music: it's stored in resource fork of "Midi Library" as standard MIDI files (unlike any other game)
-* Wrong kDoSound subops.
 |-
 |TORIN
 |
-* "Failed to initialize sound driver!"
+*Needs testing, the game seems playable.
 |-
 |LSL7
 |
-* "Failed to initialize sound driver!"
+*Needs testing, the game seems playable.
 |-
 |LIGHTHOUSE
 |
-* "Failed to initialize sound driver!"
+*Script crash when starting new game (multiple ExitFeature classes and one is broken)
-* Lite:init errors
+*Audio crash at end of opening credits
-* Requires kPlatform subops for saving and loading (see below)
+*Requires kPlatform subops for saving and loading (see below)
+*Determine how to name/detect the multi-disc Data files
 |-
 |RAMA
 |
-* "Failed to initialize sound driver!"
+*Determine how to name/detect the multi-disc Data files
+*Error when saving additional saved games
+*No transparencies on title screen
 |-
 |PQSWAT
 |
-* "Failed to initialize sound driver!"
+*Determine how to name/detect the multi-disc Data files
-* Wrong kDoSound subops.
+*No transparencies on title intro screen
 |}
-=="Failed to initialize sound driver!"==
+==kPlatform subops==
-The "Failed to initialize sound driver!" error occurs within SciMusic::init.
-You can temporarily bypass that with this patch:
-<syntaxhighlight lang="diff">
---- a/engines/sci/sound/music.cpp
-+++ b/engines/sci/sound/music.cpp
-@@ -74,6 +74,8 @@ void SciMusic::init() {
- #ifdef ENABLE_SCI32
-        if (g_sci->_features->generalMidiOnly()) {
-                deviceFlags = MDT_MIDI;
-+       } else if (platform == Common::kPlatformMacintosh) {
-+               deviceFlags = MDT_MIDI;
-        } else {
- #endif
-                deviceFlags = MDT_PCSPK | MDT_PCJR | MDT_ADLIB | MDT_MIDI;
-</syntaxhighlight>
-...Or even better, you could figure out what it's supposed to do! =)
+kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts. I have implemented these for every game except Lighthouse since it's not playable enough to test this yet. I'm leaving all of these kPlatform notes here until Lighthouse's subops are done:
-== kPlatform subops ==
+*KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open  dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
+*Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
+*Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
+*Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. These subops take parameters similar to kSave and kRestore.
+*Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.
-kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts, so this is a complete list of who uses what outstanding operations:
+==HOYLE5 Floppy==
-* KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open  dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
+There is a Mac floppy version of HOYLE5 but currently not much is known about it.
-* Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
-* Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
-* Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. This includes delete functionality. These subops take parameters similar to kSave and kRestore.
-* Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.