Difference between revisions of "Compiling ScummVM/Android"

From ScummVM :: Wiki
Jump to navigation Jump to search
(Update instructions - use the Dockerized buildbot toolchains)
Line 2: Line 2:
  
 
This page describes how to build Android packages from the ScummVM source tree.
 
This page describes how to build Android packages from the ScummVM source tree.
 +
While one can manually go through the process of installing and configuring the toolchains for the target Android architectures, and cross-compiling any third party libraries (again for each target architecture), currently the recommended method is to use a Docker container that can be built using scripts from our [https://github.com/scummvm/dockerized-bb Dockerised Buildbot repository]. This page will cover how to build ScummVM for the supported target Android architecture using the Docker container method.
  
{| border="0" cellpadding="1" cellspacing="2" style="margin-left:5em; background:#FF9999; width:80%"
+
However, one can still attempt the manual way of setting up the Android toolchain(s), without using Docker containers, by following the logic and commands in our scripts for building the Docker images.  
|- align="center"
 
| '''NOTE'''
 
|- style="margin-left:1em; background:#ffffff;"
 
|- style="background:#ffffff"
 
| Please note that information on this page may be outdated; The Optional Libraries  section in particular needs updating.  
 
|}
 
  
= Mandatory tools =
+
= Preparatory work =
  
Android SDK Manager from Android Studio or from Command line tools - https://developer.android.com/studio
+
== On a Linux 64bit distribution ==
  
Use the SDK Manager to install:
+
You will to run a recent Linux 64bit distribution, eg. Ubuntu 16.04 LTS or higher (or equivalent). These instructions were tested on a host running Ubuntu x64 20.04 LTS.
* Platform tools
 
* Build Tools 29.0.
 
* SDK Platform API 29
 
* NDK 21.3.6528147 (r21d)
 
  
Set the <tt>ANDROID_SDK_ROOT</tt> environment variable to the root directory of the SDK, and <tt>ANDROID_NDK_ROOT</tt> to the root directory of the NDK - default installation: <tt>${ANDROID_SDK_ROOT}/ndk/21.3.6528147</tt>
+
Install the <tt>git</tt> package in order to clone the ScummVM source code repository and keep it up-to-date, and also some additional packages to build the Docker toolchain containers:
 +
sudo apt-get update
 +
sudo apt-get upgrade
 +
sudo apt-get install git m4 make
  
= Optional libraries =
+
Follow the instructions on the [https://docs.docker.com/engine/install/ubuntu/ official Docker documentation page to install the Docker Engine on Linux]. We tested with Docker Engine Community Edition 19.03.12, (build 48a66213fe) using containerd.io version 1.2.13 on Linux Ubuntu x64 20.04 LTS.
* [http://svn.xiph.org/branches/lowmem-branch/Tremor/ Tremor] (lowmem branch) to play ''.ogg'' audio files
 
* [http://www.underbit.com/products/mad/ MAD] to play ''.mp3'' audio files
 
* [http://flac.sourceforge.net/ FLAC] to play lossless ''.flac'' files
 
  
Again, you have two options:
+
== On Windows 10 (version 1904 or higher) using WSL2 ==
  
== Precompiled binaries ==
+
Please, follow the instructions on the [https://docs.docker.com/docker-for-windows/wsl/ official Docker documentation page to install the Docker Engine on Windows 10 using WSL2]. This process requires rebooting your PC at least a couple of times. The instructions also link to appropriate documentation on how to enable WSL on Windows and install a Linux distribution from the Windows Store. These instructions were tested with the Ubuntu 20.04 LTS app installed.
  
Get [http://sourceforge.net/projects/scummvm/files/build/3rd-android-4.tar.bz2/download this archive] and extract it to e.g. <tt>/opt/android</tt>.
+
After enabling WSL2, and running Docker Desktop (with WSL integration), launch a bash shell for your installed Linux distribution by running the respective Windows app (here, Ubuntu 20.04 LTS). You can launch multiple bash shells by running the Linux distribution app multiple times.  
  
== Build the libraries yourself ==
+
From the Linux bash shell do:
 +
sudo apt-get update
 +
sudo apt-get upgrade
 +
sudo apt-get install git m4 make
  
Warning: this sections is not yet updated for new Android toolchain
+
*Important note: The instructions below assume that you are running the commands from a Linux bash shell. If you're on Windows, please make sure you 'always have Docker Desktop up and running' before launching your Linux bash shell(s).
  
Here's a a simple script to set up a shell for crosscompiling for <tt>arm-v7a</tt>:
+
= Setting up the environment =
 +
In this step we will fetch the latest code from the ScummVM source code repository, and create a few additional folders and files that will help in the building process and/or improve its ease of use (eg. by maintaining the history of submitted commands).
  
  #!/bin/sh
+
From your Linux host shell do:
   
+
  mkdir -p ~/Workspace
PREFIX=${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-
+
  mkdir -p ~/Workspace/android-scummvm-bb/bshomes
  export CC=${PREFIX}gcc
+
  touch ~/Workspace/android-scummvm-bb/.bash_history
export CXX=${PREFIX}g++
+
  sudo chmod o+rw ~/Workspace/android-scummvm-bb/.bash_history
export AS=${PREFIX}gcc
 
export LD=${PREFIX}gcc
 
export AR=${PREFIX}ar cru
 
export RANLIB=${PREFIX}ranlib
 
export STRIP=${PREFIX}strip
 
export OBJCOPY=${PREFIX}objcopy
 
export CPP=${PREFIX}cpp
 
 
export MACHDEP="--sysroot=$ANDROID_NDK/platforms/android-4/arch-arm -g -mandroid -mthumb-interwork"
 
   
 
export CFLAGS="${MACHDEP} -Os -Wall -mandroid -msoft-float -march=armv5te -mtune=xscale"
 
 
export CXXFLAGS="${CFLAGS}"
 
export CPPFLAGS="${CFLAGS}"
 
 
export LDFLAGS="${MACHDEP}"
 
 
echo "./configure --host=arm --prefix=/opt/android/3rd-android-4-armeabi"
 
  
Save it to a file (like <tt>android-4-eabi.sh</tt>) and run
+
Now clone the ScummVM repository:
 +
cd ~/Workspace
 +
git clone https://github.com/scummvm/scummvm.git
  
  . ./android-4-eabi.sh
+
Also clone the ScummVM dockerized buildbot repository and build the Docker containers:
 +
  git clone https://github.com/scummvm/dockerized-bb.git ~/Workspace/android-scummvm-bb/dockerized-bb
 +
cd ~/Workspace/android-scummvm-bb/dockerized-bb
 +
make toolchains/android
  
in your shell. Build systems should now use the correct tools.
+
At this point the 'docker images' command should return something like the following:
 +
docker images
 +
REPOSITORY                  TAG      IMAGE ID      CREATED            SIZE
 +
toolchains/android          latest    2005faedc472  22 minutes ago      3.9GB
 +
toolchains/android-common  latest    c726cca9065b  About an hour ago  30kB
 +
toolchains/common          latest    c751ec410582  About an hour ago  24.7kB
  
=== Tremor ===
+
*Important note: From time to time, you may need get updates from our Dockerized buildbot code (using git pull) and rebuild the toolchain containers, in order to make use of the latest improvements and any additional third party cross-compiled libraries.
  
Get a SVN checkout (r17852. as of this writing):
+
= Launch the Docker container =
  
  svn co http://svn.xiph.org/trunk/Tremor
+
Now launch the Docker container shell:
 +
  docker run -v ~/Workspace/scummvm:/data/scummvm \
 +
-v ~/android-scummvm-bb/bshomes:/data/bshomes/android \
 +
-v ~/android-scummvm-bb/.bash_history:/root/.bash_history \
 +
-w /data/scummvm \
 +
-it toolchains/android \
 +
/bin/bash
  
configure it:
+
= Compiling =
 
 
./autogen.sh --host=arm --prefix=/opt/android/3rd-android-4-armeabi --enable-low-accuracy --enable-64kHz-limit
 
 
 
and finally build & install it:
 
  
  make
+
== Set the environment variables ==
  make install
+
First, make sure that the required environment variables are set with the correct values:
 +
  export RO_ANDROID_ROOT=/opt/android \
 +
ANDROID_EXTERNAL_ROOT=/data/bshomes/android \
 +
  HOST_TAG=linux-x86_64
  
=== MAD ===
+
Then run:
 +
export ANDROID_NDK_ROOT=${RO_ANDROID_ROOT}/ndk \
 +
ANDROID_TOOLCHAIN=${RO_ANDROID_ROOT}/ndk/toolchains/llvm/prebuilt/linux-x86_64 \
 +
ANDROID_SDK_ROOT=${ANDROID_EXTERNAL_ROOT}/sdk \
 +
ANDROID_SDK_HOME=${ANDROID_EXTERNAL_ROOT}/sdk-home \
 +
GRADLE_USER_HOME=${ANDROID_EXTERNAL_ROOT}/gradle \
 +
ANDROID_SDK_BTOOLS_VERSION=29.0.3
  
Get a tarball of the latest stable version [http://sourceforge.net/projects/mad/files/libmad/ here] (v0.15.1b as of this writing) and extract it.
+
*Important note: The above command reflects the SDK build tools version installed by the container at the time of writing. This version string may need updating if the container is updated to set up a more recent version of the SDK build tools.  
  
configure it:
+
Additionally, run:
 +
export ANDROID_SDK_TOOLS=${ANDROID_SDK_ROOT}/tools \
 +
ANDROID_SDK_BTOOLS=${ANDROID_SDK_ROOT}/build-tools/${ANDROID_SDK_BTOOLS_VERSION} \
 +
PATH=${ANDROID_SDK_TOOLS}:${ANDROID_SDK_BTOOLS}:${PATH} \
 +
NDK_LOG=1 \
 +
LC_ALL=C \
 +
CCACHE_COMPRESS=1
  
  ./autogen.sh --host=arm --prefix=/opt/android/3rd-android-4-armeabi --enable-speed --enable-fpm=arm --disable-aso
+
Also, after setting up these variables, run the following command:
 +
  mkdir -p "${ANDROID_SDK_ROOT}" "${ANDROID_SDK_HOME}" "${GRADLE_USER_HOME}"
 +
cp -R "${RO_ANDROID_ROOT}"/sdk/licenses "${ANDROID_SDK_ROOT}"/licenses
  
and finally build & install it:
+
*Important note: The above command is only required to be run once, since it sets up some required folders and copies required SDK licenses to the external mounted folders (on the host Linux, outside the container itself)
  
  make
+
== Configuring and Compiling ScummVM ==
  make install
+
From your Linux host (ie. not from within the Container shell):
 +
You should probably first make sure you are working with the latest ScummVM code. Please, run:
 +
  cd ~/Workspace/scummvm
 +
  git pull
  
=== FLAC ===
+
'''TODO''' The following instructions build for the 'android-arm64-v8a' architecture. A slightly different configure command is required for the other target architectures
  
Get a tarball of the latest stable version [http://flac.sourceforge.net/download.html here] (v1.2.1 as of this writing) and extract it.
+
From the Docker container shell:
 +
* For a debug build, run the following series of commands from inside the local scummvm repository folder (you should be in there by default after running the 'docker run' command):
 +
rm -rf android_project
 +
rm -rf debug
 +
rm -rf release
 +
CXX="ccache ${ANDROID_TOOLCHAIN}/bin/aarch64-linux-android21-clang++" PKG_CONFIG_LIBDIR="${ANDROID_TOOLCHAIN}/sysroot/usr/lib/aarch64-linux-android/21/pkgconfig" PATH=${PATH}:"${ANDROID_TOOLCHAIN}/sysroot/usr/bin/aarch64-linux-android/21" make clean; ./configure --enable-verbose-build --enable-all-engines --host=android-arm64-v8a  --enable-debug
 +
make -j$(nproc) androiddistdebug
  
configure it:
+
A debug build will be signed automatically by a debug certificate. This certificate will be created by default inside a '.android' subfolder of your ANDROID_SDK_HOME path, which for us is the host path: <tt>~/Workspace/android-scummvm-bb/.android/</tt>. The debug keystore is named appropriately 'debug.keystore'. For instructions on how to create a debug signing certificate manually, please refer to the [https://developer.android.com/studio/publish/app-signing#debugmode Android Developers instructions here]
  
  ./configure --prefix=/opt/android/3rd-android-4-armeabi --disable-largefile --disable-thorough-tests --disable-doxygen-docs --disable-xmms-plugin --disable-cpplibs --disable-ogg --disable-oggtest --disable-rpath
+
* For a release build, run the following series of commands from inside the local scummvm repository folder (you should be in there by default after running the 'docker run' command):
 +
  rm -rf android_project
 +
rm -rf debug
 +
rm -rf release
 +
CXX="ccache ${ANDROID_TOOLCHAIN}/bin/aarch64-linux-android21-clang++" PKG_CONFIG_LIBDIR="${ANDROID_TOOLCHAIN}/sysroot/usr/lib/aarch64-linux-android/21/pkgconfig" PATH=${PATH}:"${ANDROID_TOOLCHAIN}/sysroot/usr/bin/aarch64-linux-android/21" make clean; ./configure --enable-verbose-build --enable-all-engines --host=android-arm64-v8a  --disable-debug --enable-release
 +
make -j$(nproc) androiddistrelease
  
and build & install the required parts:
+
Please note that a release build must be signed with a non-debug key in order to be installed on a Android device.
  
make -C src/libFLAC
+
Supported target architectures for the configure command:
make -C src/libFLAC install
 
make -C include install
 
 
 
= Compiling =
 
 
 
The Android port can be compiled with the <tt>configure</tt> script.
 
 
 
The most basic way to accomplish this is to run:
 
 
 
./configure --host=android-arm-v7a
 
make
 
 
 
The script supports various arguments, which you can use to en- or disable features, see <tt>./configure --help</tt>.
 
 
 
If you want support for additional libraries, you need cross-compiled binaries.
 
If you followed the [[#Optional libraries|above description]] and have those libraries in <tt>/opt/android/3rd-android-4-armeabi</tt>, the <tt>configure</tt> command line would look like this:
 
 
 
./configure --host=android-arm-v7a --with-tremor-prefix=/opt/android/3rd-android-4-armeabi --with-mad-prefix=/opt/android/3rd-android-4-armeabi --with-flac-prefix=/opt/android/3rd-android-4-armeabi
 
 
 
Supported hosts:
 
 
* <tt>android-arm-v7a</tt> - for older Android devices
 
* <tt>android-arm-v7a</tt> - for older Android devices
 
* <tt>android-arm64-v8a</tt> - for all new Android devices
 
* <tt>android-arm64-v8a</tt> - for all new Android devices
Line 138: Line 138:
 
After compiling there in the build directory (if used) there will be a new directory called <tt>android-project</tt>, this can be opened by Android Studio. C files are not yet available, but it is possible to edit the Java source files and debug both Java & native code (in emulator or on a real device via <tt>adb</tt> connection).
 
After compiling there in the build directory (if used) there will be a new directory called <tt>android-project</tt>, this can be opened by Android Studio. C files are not yet available, but it is possible to edit the Java source files and debug both Java & native code (in emulator or on a real device via <tt>adb</tt> connection).
 
To enable debugging, after opening the <tt>android-project</tt> directory change <tt>Project configuration</tt>, under tab <tt>Debugger</tt>, set field <tt>Debug type</tt> to <tt>Dual</tt>
 
To enable debugging, after opening the <tt>android-project</tt> directory change <tt>Project configuration</tt>, under tab <tt>Debugger</tt>, set field <tt>Debug type</tt> to <tt>Dual</tt>
 +
 +
= References  =
 +
* [[Compiling_ScummVM/Docker#Using_the_docker_images | Using the Docker images]]

Revision as of 09:49, 3 May 2021

Compiling ScummVM for Android

This page describes how to build Android packages from the ScummVM source tree. While one can manually go through the process of installing and configuring the toolchains for the target Android architectures, and cross-compiling any third party libraries (again for each target architecture), currently the recommended method is to use a Docker container that can be built using scripts from our Dockerised Buildbot repository. This page will cover how to build ScummVM for the supported target Android architecture using the Docker container method.

However, one can still attempt the manual way of setting up the Android toolchain(s), without using Docker containers, by following the logic and commands in our scripts for building the Docker images.

Preparatory work

On a Linux 64bit distribution

You will to run a recent Linux 64bit distribution, eg. Ubuntu 16.04 LTS or higher (or equivalent). These instructions were tested on a host running Ubuntu x64 20.04 LTS.

Install the git package in order to clone the ScummVM source code repository and keep it up-to-date, and also some additional packages to build the Docker toolchain containers:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git m4 make

Follow the instructions on the official Docker documentation page to install the Docker Engine on Linux. We tested with Docker Engine Community Edition 19.03.12, (build 48a66213fe) using containerd.io version 1.2.13 on Linux Ubuntu x64 20.04 LTS.

On Windows 10 (version 1904 or higher) using WSL2

Please, follow the instructions on the official Docker documentation page to install the Docker Engine on Windows 10 using WSL2. This process requires rebooting your PC at least a couple of times. The instructions also link to appropriate documentation on how to enable WSL on Windows and install a Linux distribution from the Windows Store. These instructions were tested with the Ubuntu 20.04 LTS app installed.

After enabling WSL2, and running Docker Desktop (with WSL integration), launch a bash shell for your installed Linux distribution by running the respective Windows app (here, Ubuntu 20.04 LTS). You can launch multiple bash shells by running the Linux distribution app multiple times.

From the Linux bash shell do:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install git m4 make
  • Important note: The instructions below assume that you are running the commands from a Linux bash shell. If you're on Windows, please make sure you 'always have Docker Desktop up and running' before launching your Linux bash shell(s).

Setting up the environment

In this step we will fetch the latest code from the ScummVM source code repository, and create a few additional folders and files that will help in the building process and/or improve its ease of use (eg. by maintaining the history of submitted commands).

From your Linux host shell do:

mkdir -p ~/Workspace
mkdir -p ~/Workspace/android-scummvm-bb/bshomes
touch ~/Workspace/android-scummvm-bb/.bash_history
sudo chmod o+rw ~/Workspace/android-scummvm-bb/.bash_history

Now clone the ScummVM repository:

cd ~/Workspace
git clone https://github.com/scummvm/scummvm.git

Also clone the ScummVM dockerized buildbot repository and build the Docker containers:

git clone https://github.com/scummvm/dockerized-bb.git ~/Workspace/android-scummvm-bb/dockerized-bb
cd ~/Workspace/android-scummvm-bb/dockerized-bb
make toolchains/android

At this point the 'docker images' command should return something like the following:

docker images
REPOSITORY                  TAG       IMAGE ID       CREATED             SIZE
toolchains/android          latest    2005faedc472   22 minutes ago      3.9GB
toolchains/android-common   latest    c726cca9065b   About an hour ago   30kB
toolchains/common           latest    c751ec410582   About an hour ago   24.7kB
  • Important note: From time to time, you may need get updates from our Dockerized buildbot code (using git pull) and rebuild the toolchain containers, in order to make use of the latest improvements and any additional third party cross-compiled libraries.

Launch the Docker container

Now launch the Docker container shell:

docker run -v ~/Workspace/scummvm:/data/scummvm \
-v ~/android-scummvm-bb/bshomes:/data/bshomes/android \
-v ~/android-scummvm-bb/.bash_history:/root/.bash_history \
-w /data/scummvm \
-it toolchains/android \
/bin/bash

Compiling

Set the environment variables

First, make sure that the required environment variables are set with the correct values:

export RO_ANDROID_ROOT=/opt/android \
ANDROID_EXTERNAL_ROOT=/data/bshomes/android \
HOST_TAG=linux-x86_64

Then run:

export ANDROID_NDK_ROOT=${RO_ANDROID_ROOT}/ndk \
ANDROID_TOOLCHAIN=${RO_ANDROID_ROOT}/ndk/toolchains/llvm/prebuilt/linux-x86_64 \
ANDROID_SDK_ROOT=${ANDROID_EXTERNAL_ROOT}/sdk \
ANDROID_SDK_HOME=${ANDROID_EXTERNAL_ROOT}/sdk-home \
GRADLE_USER_HOME=${ANDROID_EXTERNAL_ROOT}/gradle \
ANDROID_SDK_BTOOLS_VERSION=29.0.3
  • Important note: The above command reflects the SDK build tools version installed by the container at the time of writing. This version string may need updating if the container is updated to set up a more recent version of the SDK build tools.

Additionally, run:

export ANDROID_SDK_TOOLS=${ANDROID_SDK_ROOT}/tools \
ANDROID_SDK_BTOOLS=${ANDROID_SDK_ROOT}/build-tools/${ANDROID_SDK_BTOOLS_VERSION} \
PATH=${ANDROID_SDK_TOOLS}:${ANDROID_SDK_BTOOLS}:${PATH} \
NDK_LOG=1 \
LC_ALL=C \
CCACHE_COMPRESS=1

Also, after setting up these variables, run the following command:

mkdir -p "${ANDROID_SDK_ROOT}" "${ANDROID_SDK_HOME}" "${GRADLE_USER_HOME}"
cp -R "${RO_ANDROID_ROOT}"/sdk/licenses "${ANDROID_SDK_ROOT}"/licenses
  • Important note: The above command is only required to be run once, since it sets up some required folders and copies required SDK licenses to the external mounted folders (on the host Linux, outside the container itself)

Configuring and Compiling ScummVM

From your Linux host (ie. not from within the Container shell): You should probably first make sure you are working with the latest ScummVM code. Please, run:

cd ~/Workspace/scummvm
git pull

TODO The following instructions build for the 'android-arm64-v8a' architecture. A slightly different configure command is required for the other target architectures

From the Docker container shell:

  • For a debug build, run the following series of commands from inside the local scummvm repository folder (you should be in there by default after running the 'docker run' command):
rm -rf android_project
rm -rf debug
rm -rf release
CXX="ccache ${ANDROID_TOOLCHAIN}/bin/aarch64-linux-android21-clang++" PKG_CONFIG_LIBDIR="${ANDROID_TOOLCHAIN}/sysroot/usr/lib/aarch64-linux-android/21/pkgconfig" PATH=${PATH}:"${ANDROID_TOOLCHAIN}/sysroot/usr/bin/aarch64-linux-android/21" make clean; ./configure --enable-verbose-build --enable-all-engines --host=android-arm64-v8a  --enable-debug
make -j$(nproc) androiddistdebug

A debug build will be signed automatically by a debug certificate. This certificate will be created by default inside a '.android' subfolder of your ANDROID_SDK_HOME path, which for us is the host path: ~/Workspace/android-scummvm-bb/.android/. The debug keystore is named appropriately 'debug.keystore'. For instructions on how to create a debug signing certificate manually, please refer to the Android Developers instructions here

  • For a release build, run the following series of commands from inside the local scummvm repository folder (you should be in there by default after running the 'docker run' command):
rm -rf android_project
rm -rf debug
rm -rf release
CXX="ccache ${ANDROID_TOOLCHAIN}/bin/aarch64-linux-android21-clang++" PKG_CONFIG_LIBDIR="${ANDROID_TOOLCHAIN}/sysroot/usr/lib/aarch64-linux-android/21/pkgconfig" PATH=${PATH}:"${ANDROID_TOOLCHAIN}/sysroot/usr/bin/aarch64-linux-android/21" make clean; ./configure --enable-verbose-build --enable-all-engines --host=android-arm64-v8a  --disable-debug --enable-release
make -j$(nproc) androiddistrelease

Please note that a release build must be signed with a non-debug key in order to be installed on a Android device.

Supported target architectures for the configure command:

  • android-arm-v7a - for older Android devices
  • android-arm64-v8a - for all new Android devices
  • android-x86 - ideal for emulator with 32bit images
  • android-x86_64 - ideal for emulator with 64bit images

Android Studio setup

After compiling there in the build directory (if used) there will be a new directory called android-project, this can be opened by Android Studio. C files are not yet available, but it is possible to edit the Java source files and debug both Java & native code (in emulator or on a real device via adb connection). To enable debugging, after opening the android-project directory change Project configuration, under tab Debugger, set field Debug type to Dual

References