151
edits
Difference between revisions of "HOWTO-Debug-Endian-Issues"
Jump to navigation
Jump to search
HOWTO-Debug-Endian-Issues (view source)
Revision as of 09:13, 8 January 2023
, 09:13, 8 January 2023Mention the Raspberry Pi being able to run big-endian NetBSD, just in case… untested
(Update this page for 2022 and the tweaked VM) |
(Mention the Raspberry Pi being able to run big-endian NetBSD, just in case… untested) |
||
| (18 intermediate revisions by the same user not shown) | |||
| Line 8: | Line 8: | ||
#* Examples include the [https://www.raptorcs.com/content/BK1B01/intro.html Raptor Blackbird™] workstation. It is a powerful system which can run up-to-date Linux/BSD distributions, both in little-endian and big-endian modes. | #* Examples include the [https://www.raptorcs.com/content/BK1B01/intro.html Raptor Blackbird™] workstation. It is a powerful system which can run up-to-date Linux/BSD distributions, both in little-endian and big-endian modes. | ||
#* Unfortunately, it's a pricey niche system, and there's no guarantee that big-endian OS options will be maintained for long. | #* Unfortunately, it's a pricey niche system, and there's no guarantee that big-endian OS options will be maintained for long. | ||
#* NetBSD also allows [https://mail-index.netbsd.org/port-arm/2020/12/03/msg007117.html running a Raspberry Pi in big-endian mode], but this isn't been tested for ScummVM development yet. | |||
# Running a native, older big-endian development system: | # Running a native, older big-endian development system: | ||
#* Examples include buying an older G4 or G5 Apple PowerPC system, or an older SPARC64 Sun station. They can often be bought second hand at very reasonable prices. | #* Examples include buying an older G4 or G5 Apple PowerPC system, or an older SPARC64 Sun station<ref>Some MIPS and ARM boards also exist, but their quality can vary a lot, and although the MIPS and ARM architectures are theoretically bi-endian, in practice these development boards often only run in little-endian mode, nowadays. They can be useful for strict-alignment testing, though (but <code>-fsanitize=alignment -DSCUMM_NEED_ALIGNMENT</code> in UBSan on your regular desktop will also catch a lot of these issues).</ref>. They can often be bought second hand at very reasonable prices. | ||
#* However, running a modern development environment on them in getting harder (but not impossible), because the big-endian desktop ecosystem receives less and less maintenance, so things often tend to break. G5 systems (in particular) also require careful maintenance and can be very power-hungry. | #* However, running a modern development environment on them in getting harder (but not impossible), because the big-endian desktop ecosystem receives less and less maintenance, so things often tend to break. G5 systems (in particular) also require careful maintenance and can be very power-hungry. | ||
# '''Emulating a big-endian development system from your regular development machine''': | # '''Emulating a big-endian development system from your regular development machine''': | ||
#* Any reasonably powerful desktop system should be able to emulate a big-endian architecture, thanks to [https://www.qemu.org QEMU]. | #* Any reasonably powerful desktop system should be able to emulate a big-endian architecture, thanks to [https://www.qemu.org QEMU]. | ||
#* The main drawback is that this currently requires running some older/unmaintained Linux distributions, and, as of late 2022, | #* The main drawback is that this currently requires running some older/unmaintained Linux distributions, and, as of late 2022, graphics acceleration support is missing. Since this is emulation (and not virtualization), there is also a noticeable (but usually tolerable) performance impact. | ||
This howto focuses on the last option, since it is the most accessible one, and it's still a way of fixing the majority of endianness issues we encounter. | This howto focuses on the last option, since it is the most accessible one, and it's still a way of fixing the majority of endianness issues we encounter. | ||
| Line 20: | Line 20: | ||
== Linux big-endian PowerPC emulation with QEMU == | == Linux big-endian PowerPC emulation with QEMU == | ||
The current reference VM is a pre-configured Debian 8.11 PowerPC system. It has been modified to feature an updated C++11 toolchain (GCC 5.5.0) | The current reference VM is a pre-configured Debian 8.11 PowerPC system<ref>Modern versions of Debian are actually still built for big-endian PowerPC, but it's not a ''release'' architecture anymore, which means that it's only available through Debian ''unstable''. Debian unstable is harder to maintain than a stable release, and bugs/reliability issues often appear (e.g. Valgrind has been having PPC SDL compatibility problems for years, GRUB installation being much less reliable than the older Yaboot…), especially on non-mainstream architectures. This is why we're sticking with a Debian 8 VM for now.</ref>. It has been modified to feature an updated C++11 toolchain (GCC 5.5.0). | ||
Some important notes: | Some important notes: | ||
* Security support updates have been discontinued for Debian 8 in late 2018. Older cryptographic ciphers and certificates (such as in TLS or SSH) in the base system may also cause various issues. For this reason, this VM should only be run ''on a local, trusted environment''. | * Security support updates have been discontinued for Debian 8 in late 2018. Older cryptographic ciphers and certificates (such as in TLS or SSH) in the base system may also cause various issues. For this reason, this VM should only be run ''on a local, trusted environment''. | ||
* | * 3D games will have a slow framerate, since QEMU only provides a limited, unaccelerated framebuffer for PPC<ref>Using the <code>-device ati-vga</code> QEMU option may bring an improvement at some point, but it's experimental and currently broken, especially with the old Debian 8 kernel.</ref>. | ||
* '''The bigger your host CPU clock rate, the better''': a 4 GHz CPU will bring some improvement over a 3 GHz CPU, which is itself much better than a 2 GHz CPU, and so on<ref>For reference, a full build of ScummVM with only the SCUMM engine takes around 26 minutes in QEMU on an Intel i7 or an Apple M1, while the same build on a native PowerPC G4 7447A takes 13 minutes (all single-threaded).</ref>. Note that QEMU emulation is mostly single-threaded, so having many CPU cores isn't really useful for this. | |||
* The bigger your CPU clock rate, the better: a 4 GHz CPU will bring | |||
=== Starting the VM === | === Starting the VM === | ||
The VM image is quite large, so ask the rest of the team for its URL. | |||
Once you've downloaded and extracted the VM archive, you'll need to install QEMU for your system with your usual package manager (Windows builds are available [https://qemu.weilnetz.de/w64/ here]). | Once you've downloaded and extracted the VM archive, you'll need to install QEMU for your system with your usual package manager (Windows builds are available [https://qemu.weilnetz.de/w64/ here]). | ||
==== QEMU options ==== | |||
Then, create one of the following scripts in the same directory as the VM image. | Then, create one of the following scripts in the same directory as the VM image. | ||
| Line 45: | Line 46: | ||
-m 2048 ^ | -m 2048 ^ | ||
-g 1024x750x32 ^ | -g 1024x750x32 ^ | ||
-device ES1370 ^ | |||
-no-reboot ^ | -no-reboot ^ | ||
-boot c ^ | -boot c ^ | ||
| Line 61: | Line 63: | ||
-m 2048 \ | -m 2048 \ | ||
-g 1024x750x32 \ | -g 1024x750x32 \ | ||
-device ES1370 \ | |||
-no-reboot \ | -no-reboot \ | ||
-boot c \ | -boot c \ | ||
| Line 73: | Line 76: | ||
* <code>-g 1024x750x32</code>: VM screen resolution and bit depth. You can try suiting it to your needs, but strange results may happen with some resolutions, and, since there's no graphics acceleration, making the window too big may worsen performance. Reducing the bit-depth from <code>32</code> to <code>24</code> bits could help in some cases. | * <code>-g 1024x750x32</code>: VM screen resolution and bit depth. You can try suiting it to your needs, but strange results may happen with some resolutions, and, since there's no graphics acceleration, making the window too big may worsen performance. Reducing the bit-depth from <code>32</code> to <code>24</code> bits could help in some cases. | ||
* <code>-cdrom /path/to/host/game.iso</code>: this is a quick way of sharing some game or development files from your host to the VM (it will then appear in its file manager). Other file-sharing options between the two systems are possible ([[#Various options for a more convenient setup|see below]]). | * <code>-cdrom /path/to/host/game.iso</code>: this is a quick way of sharing some game or development files from your host to the VM (it will then appear in its file manager). Other file-sharing options between the two systems are possible ([[#Various options for a more convenient setup|see below]]). | ||
==== Booting Debian ppc ==== | |||
Then, run that script. A QEMU window should appear, and a Linux system should boot. Wait until a full [https://docs.xfce.org/4.10/start XFCE desktop] appears (this may take a couple of minutes, depending on your host system performance). | Then, run that script. A QEMU window should appear, and a Linux system should boot. Wait until a full [https://docs.xfce.org/4.10/start XFCE desktop] appears (this may take a couple of minutes, depending on your host system performance). | ||
==== Keyboard layout ==== | |||
Keyboard layout can be changed in Applications Menu > Settings > Keyboard > Layout, or with <code>setxkbmap</code> (see also https://wiki.debian.org/Keyboard), if necessary. Default credentials are <code>scummvm</code> / <code>scummvm</code>, since this is just a local development environment. | Keyboard layout can be changed in Applications Menu > Settings > Keyboard > Layout, or with <code>setxkbmap</code> (see also https://wiki.debian.org/Keyboard), if necessary. Default credentials are <code>scummvm</code> / <code>scummvm</code>, since this is just a local development environment. | ||
| Line 86: | Line 93: | ||
Don't expect the included web browser to be remotely useful for anything. | Don't expect the included web browser to be remotely useful for anything. | ||
For development purposes, you may want to install the following set of tools, if | ==== Build dependencies ==== | ||
For development purposes, you may want to install the following set of tools, if they are useful for your use case (note that Valgrind is quite large, though): | |||
<syntaxhighlight lang="shell"> | <syntaxhighlight lang="shell"> | ||
| Line 92: | Line 101: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Compilers, GNU Make, GDB, Git and SDL development files are already installed in this image. The APT package manager is also pre-configured to use the older [https://www.debian.org/distrib/archive.en.html Debian 8 archive | Compilers, GNU Make, GDB, Git and SDL development files are already installed in this image. The APT package manager is also pre-configured to use the older [https://www.debian.org/distrib/archive.en.html Debian 8 archive] files. Some Debian 8 GPG keys have expired since then, though, so APT will print some security warnings. | ||
==== Building ==== | |||
Cloning the repository and compiling ScummVM is done through the [[Compiling_ScummVM/GCC|usual means]], with some important points: | Cloning the repository and compiling ScummVM is done through the [[Compiling_ScummVM/GCC|usual means]], with some important points: | ||
* Everything is going to be slower than your usual environment | * Everything is going to be slower than your usual environment. | ||
* Since QEMU emulation is single-threaded, there is no point is running <code>make</code> with any <code>-j</code> flag for parallel compilation. | * Since QEMU emulation is single-threaded, there is no point is running <code>make</code> with any <code>-j</code> flag for parallel compilation. | ||
* It is highly suggested to only enable the engines and features that you need for your test, e.g. <code>./configure --disable-detection-full --disable-all-engines --enable-engine=tinsel --disable-lua --disable-tinygl --disable-cloud --disable-hq-scalers --disable-optimizations --enable-debug</code> will save you a lot of time if you're only interested in testing the Tinsel engine. | * It is highly suggested to only enable the engines and features that you need for your test, e.g. <code>./configure --disable-detection-full --disable-all-engines --enable-engine=tinsel --disable-lua --disable-tinygl --disable-cloud --disable-hq-scalers --disable-optimizations --enable-debug</code> will save you a lot of time if you're only interested in testing the Tinsel engine. | ||
* Some tools such as Valgrind become unbearably slow when emulated. <code>--enable-asan</code> is available, but its implementation is from 2015. | * Some tools such as Valgrind become unbearably slow when emulated. <code>--enable-asan</code> is available, but its implementation is from 2015. | ||
* The older 1.2 branch of SDL is used, since rendering is done through an unaccelerated framebuffer at the moment, and SDL1.2 is a better option than SDL2 for this case. | * The older 1.2 branch of SDL is used, since rendering is done through an unaccelerated framebuffer at the moment, and SDL1.2 is a better option than SDL2 for this case. | ||
==== Testing ==== | |||
You can then run the resulting <code>scummvm</code> binary, while making sure that some ScummVM options such as Global Options > Paths > Extra Path are properly configured for development. Then, add your game, and see how it behaves on big-endian! Run <code>gdb</code> on it if necessary, make the appropriate code changes, and iterate with <code>make</code> until it works as expected. | You can then run the resulting <code>scummvm</code> binary, while making sure that some ScummVM options such as Global Options > Paths > Extra Path are properly configured for development. Then, add your game, and see how it behaves on big-endian! Run <code>gdb</code> on it if necessary, make the appropriate code changes, and iterate with <code>make</code> until it works as expected. | ||
=== Various options for a more convenient setup === | === Various options for a more convenient setup === | ||
==== Unmuting audio ==== | |||
If you can't hear any sound, make sure that your QEMU run script has the <code>-device ES1370</code> option. | |||
It's also possible that the emulated audio card is muted inside the Linux VM, for some reason. To fix this, start the Applications > Multimedia > Audio Mixer program and check that no main output is muted (you can also run the <code>alsamixer</code> program in the Terminal and type <code>M</code> and then <code>Esc</code> to unmute the main output). | |||
==== Modern IDE integration ==== | |||
If you need a powerful and user-friendly IDE inside the VM, you won't have many options, because this is an old and limited Linux system. If you don't like console text editors, you're probably out of luck (unless you want to try <code>sudo apt-get install codeblocks</code>). One option could be to work from your usual IDE on your regular desktop, and share its files with the VM. | If you need a powerful and user-friendly IDE inside the VM, you won't have many options, because this is an old and limited Linux system. If you don't like console text editors, you're probably out of luck (unless you want to try <code>sudo apt-get install codeblocks</code>). One option could be to work from your usual IDE on your regular desktop, and share its files with the VM. | ||
==== Sharing resources ==== | |||
The VM can access its host system via the <code>10.0.2.2</code> IP address (useful if you want to share files from the host to the VM with an NFS, SMB, HTTP, or FTP server). | The VM can access its host system via the <code>10.0.2.2</code> IP address (useful if you want to share files from the host to the VM with an NFS, SMB, HTTP, or FTP server). | ||
| Line 117: | Line 140: | ||
[https://wiki.archlinux.org/title/QEMU#VNC Connecting through VNC] is also possible, but it's probably not going to be a great experience. | [https://wiki.archlinux.org/title/QEMU#VNC Connecting through VNC] is also possible, but it's probably not going to be a great experience. | ||
==== Reclaiming some storage space ==== | |||
If you need a bit more storage space inside the VM, some big and unnecessary tools can be removed: | If you need a bit more storage space inside the VM, some big and unnecessary tools can be removed: | ||
| Line 130: | Line 155: | ||
However, there are a few architectures which QEMU does not support, notably [https://en.wikipedia.org/wiki/SuperH SH]. However, there is another general purpose CPU/machine emulator called [http://gxemul.sourceforge.net/ GXemul] which does support this and some other more esoteric platforms. This is less supported than QEMU, but this procedure should be possible with some modifications. Any notes on this would be gratefully received by the team. | However, there are a few architectures which QEMU does not support, notably [https://en.wikipedia.org/wiki/SuperH SH]. However, there is another general purpose CPU/machine emulator called [http://gxemul.sourceforge.net/ GXemul] which does support this and some other more esoteric platforms. This is less supported than QEMU, but this procedure should be possible with some modifications. Any notes on this would be gratefully received by the team. | ||
== Notes == | |||
<references /> | |||