Difference between revisions of "SCI/SCI32 Mac Support Status"

From ScummVM :: Wiki
< SCI
Jump to navigation Jump to search
m (Update TODOs)
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Status==
==Status==


This page is an attempt to track the state of ScummVM support for SCI32 Mac games. They are not supported yet. The work stalled for several years and it is unclear how much is left to do. I (sluicebox) gathered most of the Mac games to determine their current blockers. In the process I found several easy fixes that got some games to boot and play audio for the first time. Many of the others are one or two issues away from booting. We could be closer than we think.
This page is an attempt to track the state of ScummVM support for SCI32 Mac games. Aside from Gabriel Knight 1, they are not supported yet but we're making good progress. Many are now playable and all of them can at least boot to the title screen.


==Data Files==
==Data Files==


Extracting data files from SCI32 Macintosh CDs is particularly difficult. Unless a reasonable method is discovered and documented, it seems unlikely that the average user will be able to take advantage of ScummVM support.
Working with data files from SCI32 Macintosh CDs is difficult. Some of these difficulties are for users while others are problems for us that we haven't solved yet. It's not just one thing, it's the combination:


Extracting Macintosh files is already complicated by resource and data forks. SCI resources are in the resource fork and many users have trouble with just that part. It's often necessary to use a real or emulated classic Macintosh at some point in the process. SCI32 CDs complicate this further by setting the Hidden flag on files and directories. The Macintosh Hidden file system flag is like the Windows one in that it prevents a file from appearing in Finder. The difference is that Macintosh has no Show Hidden Files setting. Instead, third party programs such as File Buddy must be used to reveal and copy these otherwise invisible files. Sierra hid all files that didn't need to be copied to the hard disk, which is all of the media files and most of the data files. As a final complication, some data files were duplicated between these hidden directories and the visible ones, and different data files use the same name on different CDs.
*Resource forks: Game data files use Macintosh resource forks while individual resource files use data forks.
* Hidden files: Most data files and directories are flagged Hidden. This is an extraction obstacle for software that respects this flag, such as classic Finder.
*Thousands of files: Each speech audio resource is stored as an individual file. Some games have up to 10,000 files. Our SCI resource loader is not designed for this large amount. On startup it performs work on each file and this creates delays that can last minutes.
*Folder structure: Data files are scattered across folders, some hidden and some not, with duplicates. Some of these duplicates appear to contain no resources. The idea was to present a folder that users would copy to their hard drive with the executable and one or two data files, leaving the others on disc and invisible.
*Many data files: Games have many small Data## files instead of a few large volume files. Our Advanced Detector only supports 13 files.
*Multi-disc: The structure of multi-disc games is at odds with our detection model. Each disc has many small Data## files with the same filenames as those on other discs. They can't all be placed in the same directory with those names, but there are also far too many to require manual renaming. It seems like some kind of subdirectory scheme will be required. As mentioned above, the Advanced Detector doesn't have support enough entries for even one disc of these games, let alone six. Given that setting up these files properly could be error prone, not being able to detect missing files would be a problem.


There is also a performance problem that needs to be solved for SCI32 Mac games. The CD versions contain as many as ''ten thousand'' media files which slows the loader to a crawl when starting a game. Each audio resource is in its own file instead of a resource volume and the loader is doing work on each.
''Update'': The relatively recent [https://www.scummvm.org/dumper-companion ScummVM Dumper Companion] helps with a lot of this. Multi-disc is currently the biggest obstacle.
 
==Real Talk==
 
I (sluicebox) have decompiled the scripts of every SCI32 Mac game and compared them to the PC versions. Aside from one joke in Phantasmagoria and an updated birthday in Space Quest 6, the Mac versions don't seem to offer interesting differences beyond native (worse) cursors and some different audio formats. It's fun to add support for them where possible, but it's probably not worth significantly complicating the codebase just to for these rare versions that are more difficult to setup and often more limited than the PC versions. For example, Space Quest 6 Mac stores music in a unique way (regular MIDI files!) that breaks core assumptions in our audio code. No other game does this. If that requires a major audio refactor, it would be hard to justify. So far, I've been able to fix Mac incompatibilities and add Mac features with low impact changes. What's left is the hard stuff.
 
Later SCI32 games were mostly distributed on CDs with both PC and Macintosh partitions so these discs already have the working PC files anyway. This reduces the practical benefits of Mac support for these games even further.


==General TODOs==
==General TODOs==


* Upscale native Mac cursors.
* Determine how to handle multi-disc data files and implement it. (Current blocker for Phant1 and GK2)
* Implement kPlatform Mac subops for saving and restoring in KQ7, Shivers, Lighthouse, and Mother Goose.
* Improve load times when faced with thousands of patch files. (This would help PC fan-patches too: {{Tracker|id=11353}} )
* Implement, or workaround, kPlatform Mac subop 12 for setting music volume (Phantasmagoria only)
* Add engine support for a library of standard MIDI files (SQ6 only)
* Use SCI 2.1 resource type numbers for most SCI32 Mac games in ResourceManager::convertResType()
* Implement kPlatform Mac subops for saving and restoring in Lighthouse.
* Games seem too dark
* Speed throttling (Mac games don't call kGameIsRestarting, it's luck that PC games did since it was a no-op in SCI32)
* Verify accuracy of music playback
* Document game file extraction for users


==SCI32 Mac Games==
==SCI32 Mac Games==


{|border="1" cellspacing="0" cellpadding="5" width=75%
{| width="75%" cellspacing="0" cellpadding="5" border="1"
! style="text-align:left;"| Game
! style="text-align:left;" |Game
! style="text-align:left;"| Blockers
! style="text-align:left;" |Status
|-
|-
|GK1
|GK1
|
|  
* Testing, the game seems playable.
*Supported!
|-
|-
|KQ7
|KQ7
|
|
* Requires kPlatform subops for saving and loading (see below)
* Needs testing, the game seems playable.
* Intro and other movies require QuickTime to be implemented in video player
|-
|-
|LSL6 HI-RES
|LSL6 HI-RES
|
|
* Testing, the game seems playable.
*Game has been successfully completed!
|-
|-
|PQ4
|PQ4
|
|  
* Introduction error about too many screen items.
*Needs testing, the game seems playable.
|-
|-
|GK2
| GK2
|
|
* Videos don't play due to ResourceManager::convertResType() using wrong type map
*Determine how to name/detect the multi-disc Data files  
* Determine how to name/detect the multi-disc Data files
|-
|-
|HOYLE5
|HOYLE5  
|
|
* Testing, game seems playable.
* Needs testing, the game seems playable. (I won at Hearts and Gin Rummy a few times)
|-
|-
|MOTHERGOOSE HI-RES
|MOTHERGOOSE HI-RES
|
|
* Requires kPlatform 0 6,7,8,9,10,11 subops for saving and loading, prevents booting
*Needs testing, the game seems playable.
|-
|-
|PHANTASMAGORIA
| PHANTASMAGORIA
|
|
* Determine how to name/detect the multi-disc Data files
*Determine how to name/detect the multi-disc Data files  
|-
|-
|SHIVERS
|SHIVERS
|
|  
* Requires kPlatform subops for saving and loading (see below)
*Needs testing, the game seems playable.
|-
|-
|SQ6
| SQ6
|
|
* Music stored in resource fork of "Midi Library" as standard MIDI files
*No music: it's stored in resource fork of "Midi Library" as standard MIDI files (unlike any other game)
* Crash when displaying control panel after restoring game
|-
|-
|TORIN
|TORIN
|
|  
* Saving/restoring crashes, kScummVMSaveLoad script patch not applied correctly
*Needs testing, the game seems playable.
|-
|-
|LSL7
|LSL7
|
|
* Saving/restoring crashes, kScummVMSaveLoad script patch not applied correctly
*Needs testing, the game seems playable.
|-
|-
|LIGHTHOUSE
|LIGHTHOUSE
|
|
* Lite:init errors if VERSION file is missing
*Script crash when starting new game (multiple ExitFeature classes and one is broken)
* Script crash when starting new game
*Audio crash at end of opening credits
* Requires kPlatform subops for saving and loading (see below)
*Requires kPlatform subops for saving and loading (see below)
* Determine how to name/detect the multi-disc Data files
*Determine how to name/detect the multi-disc Data files  
|-
|-
|RAMA
|RAMA
|
|
* Determine how to name/detect the multi-disc Data files
*Determine how to name/detect the multi-disc Data files
*Error when saving additional saved games
*No transparencies on title screen
|-
|-
|PQSWAT
|PQSWAT  
|
|
* "Unable to load file" error after video when starting new game
*Determine how to name/detect the multi-disc Data files
* Determine how to name/detect the multi-disc Data files
*No transparencies on title intro screen
|}
|}


== kPlatform subops ==
==kPlatform subops==
 
kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts. I have implemented these for every game except Lighthouse since it's not playable enough to test this yet. I'm leaving all of these kPlatform notes here until Lighthouse's subops are done:
 
*KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open  dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
*Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
*Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
*Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. These subops take parameters similar to kSave and kRestore.
*Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.


kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts, so this is a complete list of who uses what outstanding operations:
==HOYLE5 Floppy==


* KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open  dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
There is a Mac floppy version of HOYLE5 but currently not much is known about it.
* Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
* Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
* Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. This includes delete functionality. These subops take parameters similar to kSave and kRestore.
* Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.

Latest revision as of 16:15, 22 May 2022

Status

This page is an attempt to track the state of ScummVM support for SCI32 Mac games. Aside from Gabriel Knight 1, they are not supported yet but we're making good progress. Many are now playable and all of them can at least boot to the title screen.

Data Files

Working with data files from SCI32 Macintosh CDs is difficult. Some of these difficulties are for users while others are problems for us that we haven't solved yet. It's not just one thing, it's the combination:

  • Resource forks: Game data files use Macintosh resource forks while individual resource files use data forks.
  • Hidden files: Most data files and directories are flagged Hidden. This is an extraction obstacle for software that respects this flag, such as classic Finder.
  • Thousands of files: Each speech audio resource is stored as an individual file. Some games have up to 10,000 files. Our SCI resource loader is not designed for this large amount. On startup it performs work on each file and this creates delays that can last minutes.
  • Folder structure: Data files are scattered across folders, some hidden and some not, with duplicates. Some of these duplicates appear to contain no resources. The idea was to present a folder that users would copy to their hard drive with the executable and one or two data files, leaving the others on disc and invisible.
  • Many data files: Games have many small Data## files instead of a few large volume files. Our Advanced Detector only supports 13 files.
  • Multi-disc: The structure of multi-disc games is at odds with our detection model. Each disc has many small Data## files with the same filenames as those on other discs. They can't all be placed in the same directory with those names, but there are also far too many to require manual renaming. It seems like some kind of subdirectory scheme will be required. As mentioned above, the Advanced Detector doesn't have support enough entries for even one disc of these games, let alone six. Given that setting up these files properly could be error prone, not being able to detect missing files would be a problem.

Update: The relatively recent ScummVM Dumper Companion helps with a lot of this. Multi-disc is currently the biggest obstacle.

Real Talk

I (sluicebox) have decompiled the scripts of every SCI32 Mac game and compared them to the PC versions. Aside from one joke in Phantasmagoria and an updated birthday in Space Quest 6, the Mac versions don't seem to offer interesting differences beyond native (worse) cursors and some different audio formats. It's fun to add support for them where possible, but it's probably not worth significantly complicating the codebase just to for these rare versions that are more difficult to setup and often more limited than the PC versions. For example, Space Quest 6 Mac stores music in a unique way (regular MIDI files!) that breaks core assumptions in our audio code. No other game does this. If that requires a major audio refactor, it would be hard to justify. So far, I've been able to fix Mac incompatibilities and add Mac features with low impact changes. What's left is the hard stuff.

Later SCI32 games were mostly distributed on CDs with both PC and Macintosh partitions so these discs already have the working PC files anyway. This reduces the practical benefits of Mac support for these games even further.

General TODOs

  • Determine how to handle multi-disc data files and implement it. (Current blocker for Phant1 and GK2)
  • Improve load times when faced with thousands of patch files. (This would help PC fan-patches too: [Bug #11353] )
  • Add engine support for a library of standard MIDI files (SQ6 only)
  • Implement kPlatform Mac subops for saving and restoring in Lighthouse.
  • Speed throttling (Mac games don't call kGameIsRestarting, it's luck that PC games did since it was a no-op in SCI32)
  • Document game file extraction for users

SCI32 Mac Games

Game Status
GK1
  • Supported!
KQ7
  • Needs testing, the game seems playable.
LSL6 HI-RES
  • Game has been successfully completed!
PQ4
  • Needs testing, the game seems playable.
GK2
  • Determine how to name/detect the multi-disc Data files
HOYLE5
  • Needs testing, the game seems playable. (I won at Hearts and Gin Rummy a few times)
MOTHERGOOSE HI-RES
  • Needs testing, the game seems playable.
PHANTASMAGORIA
  • Determine how to name/detect the multi-disc Data files
SHIVERS
  • Needs testing, the game seems playable.
SQ6
  • No music: it's stored in resource fork of "Midi Library" as standard MIDI files (unlike any other game)
TORIN
  • Needs testing, the game seems playable.
LSL7
  • Needs testing, the game seems playable.
LIGHTHOUSE
  • Script crash when starting new game (multiple ExitFeature classes and one is broken)
  • Audio crash at end of opening credits
  • Requires kPlatform subops for saving and loading (see below)
  • Determine how to name/detect the multi-disc Data files
RAMA
  • Determine how to name/detect the multi-disc Data files
  • Error when saving additional saved games
  • No transparencies on title screen
PQSWAT
  • Determine how to name/detect the multi-disc Data files
  • No transparencies on title intro screen

kPlatform subops

kPlatform supports a handful of miscellaneous Mac-specific sub operations. The SCI32 ones are different than SCI16, and some changed between SCI32 versions. I have decompiled and audited all kPlatform calls in all SCI32 Mac scripts. I have implemented these for every game except Lighthouse since it's not playable enough to test this yet. I'm leaving all of these kPlatform notes here until Lighthouse's subops are done:

  • KQ7 does custom saving and restoring with subops 3, 4, and 5 to initiate, save, and restore files. Initiate is called when starting a new game. Initiate and Restore would display a native Mac save or open dialog with a hard-coded and misspelled prompt such as "Who's game?". The interpreter would remember the current save file, not the game scripts. These subops take no parameters.
  • Shivers also does custom saving and restoring with subops 3, 4, and 5 but they take parameters similar to the normal kSave and kRestore functions. It appears the interpreter is no longer in charge of tracking the current save file.
  • Lighthouse also does custom saving and restoring but just for fun it only uses subop 3 but adds a sub-subop parameter for the three operations. It appears that -1 initializes, 1 saves, and 0 restores.
  • Mother Goose uses subops 6, 7, 8, 9, 10, and 11 for saving and restoring. These subops take parameters similar to kSave and kRestore.
  • Phantasmagoria sets the music volume from the control panel with subop 12 and passes a volume parameter between 0 and 15.

HOYLE5 Floppy

There is a Mac floppy version of HOYLE5 but currently not much is known about it.