Buildbot

Buildbot Project Service
URL(s)	http://buildbot.scummvm.org
Purpose	Provides automated build services for an increasing number of our supported platforms.
Maintainer(s)	Andre Heider (Dhewg) John Willis (DJWillis)

Introduction

From the buildbot homepage:

The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

The overall goal is to reduce tree breakage and provide a platform to run tests or code-quality checks that are too annoying or pedantic for any human to waste their time with. Developers get immediate (and potentially public) feedback about their changes, encouraging them to be more careful about testing before checkin.

You can find the ScummVM buildbot page here.

Goals

Compile all ports when a commit is done (both trunk and the active branch) to check if it broke one or more ports.
Notify developers about breakage (and if a following commit fixes it again).
Provide useful statistics.
Provide nightly builds of all ports.

Status

What we have so far:

The bot already polls the Git Server for changes on trunk and the current branch.
When a change occurs, all ports are being built incrementally.
Once every 24h (currently 4am CET), a full and clean rebuild is compiled for all ports.
Provide the nightly builds via HTTP.
Notify developers upon problems through an IRC bot in IRC Channel.

What's missing:

Add more toolchains for all missing ports - when possible (see below).
Notify developers upon problems via mail.

Toolchains

A few requirements must be met to add a toolchain to the buildbot server:

It must be possible to cross-compile the port from Linux (that might change in the future).
No custom Makefiles from the backend/ folder, the port has to use the ./configure script.
The compile process must neither modify nor add anything in the source tree. All builds are performed in external build directories, by invoking the ./configure script in these external directories.
The port must be fully buildable from scratch with only ./configure arguments and environment variables for it.

The toolchain should be primarily maintained by the port maintainer, but since this requires a little Linux experience, its not a must ;). We will provide assistance with this where possible.

If your toolchain/port is ready to be added, ask sev, joostp or dhewg for an account.

Administration

When the buildbot config files are changed, a user with shell access and sudo privileges needs to run the following commands. It's good practice to do this once buildbot is idle:

sudo -s
systemctl stop buildslave
systemctl stop buildmaster
cd ~buildbot/scummvm-sites
sudo -u buildbot git pull --ff-only
systemctl start buildmaster
systemctl start buildslave

If problems arise, check if all files in the master (~buildbot/scummvm-master) and slave (~buildbot/scummvm-slave) directories are owned by the correct user (buildbot:buildbot). If the daemons were incorrectly started, new files might have been created under another user account. In that case:

stop the daemons (see above)
fix the permissions with

chown -R buildbot:buildbot ~buildbot/scummvm-master/*
chown -R buildbot:buildbot ~buildbot/scummvm-slave/*

start the daemons

Rebuilding Toolchain Libraries

It is periodically necessary to update the toolchain libraries for newer version releases or to add new library dependencies. This is a basic outline procedure for how to build these on the buildbot.

Toolchains are located at /opt/toolchains/<host target> in general, with the following exceptions:
- Though some of the MinGW 32 toolchain is present in

/opt/toolchains/i586-mingw32msvc

which should be used as the host/prefix in the following instructions, the binaries i.e. i586-mingw32msvc-gcc are located at /usr/bin. This is probably as they are precompiled to be installed at this location, so this is necessary.

However, if the PATH step to the bin is omitted in the following, this will still work for this target, which is confusing and could result in odd builds as some of toolchain bin contents may be needed.

All source tarballs used and any patches should be left in /opt/toolchains for reference.

/opt/toolchains is actually a symlink to the real location of the external storage drive, but /opt/toolchains should be used in all configuration to ensure that this can be freely moved in future if necessary.

libSDL:

Decompress the latest libSDL tarball to a temporary location i.e.

tar -xvzf SDL-1.2.15.tar.gz -d /home/digitall

Apply any required patches i.e. Currently libSDL-1.2.15 snapshot has a error in the Win32 version strings.. This may occur again on future releases :/

patch -p0 < SDL-1.2.15-Fix-Windows-DLL-Version.patch

Add toolchain binaries directory to path.

PATH=/opt/toolchains/x86_64-w64-mingw32/bin:$PATH
export PATH

Add out of tree build directory and configure the SDL codebase here with the host target and prefix to use the correct cross toolchain and target for make install.

mkdir SDL-1.2.15-x86_64-w64-mingw32
cd SDL-1.2.15-x86_64-w64-mingw32
../SDL-1.2.15/configure --host=x86_64-w64-mingw32 --prefix=/opt/toolchains/x86_64-w64-mingw32

Compile the codebase.

make

Install the compiled files. As you become root, you need to make the same modification to the path to get the required cross toolchain tools such as ranlib.

sudo bash
PATH=/opt/toolchains/x86_64-w64-mingw32/bin:$PATH
export PATH
make install