TrustedUser
2,147
edits
Difference between revisions of "Buildbot"
Jump to navigation
Jump to search
m
Text replacement - "<source lang=" to "<syntaxhighlight lang="
(spelling/typo fixes) |
m (Text replacement - "<source lang=" to "<syntaxhighlight lang=") |
||
| (19 intermediate revisions by 9 users not shown) | |||
| Line 1: | Line 1: | ||
{{Infobox Project Service Information| | |||
url=http://buildbot.scummvm.org| | |||
purpose=Provides automated build services for an increasing number of our supported platforms.| | |||
maintainer=Andre Heider ([[User:Dhewg|Dhewg]])<br>John Willis ([[User:DJWillis|DJWillis]]) | |||
}} | |||
== Introduction == | == Introduction == | ||
| Line 18: | Line 24: | ||
== Status == | == Status == | ||
What we | What we have so far: | ||
* The bot already polls the | * The bot already polls the [[Git|Git Server]] for changes on trunk and the current branch. | ||
* When a change occurs, all ports are being built incrementally. | * When a change occurs, all ports are being built incrementally. | ||
* Once every 24h (currently 4am CET), a full and clean rebuild is compiled for all ports. | * Once every 24h (currently 4am CET), a full and clean rebuild is compiled for all ports. | ||
* Provide the [http://buildbot.scummvm.org/builds.html nightly builds via HTTP]. | |||
* Notify developers upon problems through an IRC bot in [[IRC Channel]]. | |||
What's missing: | What's missing: | ||
* Add more toolchains for all missing ports - when possible (see below). | * Add more toolchains for all missing ports - when possible (see below). | ||
* Notify developers upon problems | * Notify developers upon problems via mail. | ||
== Toolchains == | == Toolchains == | ||
A few requirements must be met to add a toolchain to the buildbot server: | A few requirements must be met to add a toolchain to the buildbot server: | ||
* It | * It must be possible to cross-compile the port from Linux (that might change in the future). | ||
* No custom Makefiles from the ''backend/'' folder, the port has to use the ''./configure'' script. | * No custom Makefiles from the ''backend/'' folder, the port has to use the ''./configure'' script. | ||
* The compile process | * The compile process must neither modify nor add anything in the source tree. All builds are performed in external build directories, by invoking the ''./configure'' script in these external directories. | ||
* The port | * The port must be fully buildable from scratch with only ''./configure'' arguments and environment variables for it. | ||
The toolchain should be maintained by the port maintainer, but since this requires a little Linux experience, its not a must ;) | The toolchain should be primarily maintained by the port maintainer, but since this requires a little Linux experience, its not a must ;). We will provide assistance with this where possible. | ||
If your toolchain/port is ready to be added, ask ''sev'', ''joostp'' or ''dhewg'' for an account. | If your toolchain/port is ready to be added, ask ''sev'', ''joostp'' or ''dhewg'' for an account. | ||
== Administration == | |||
When the buildbot config files are changed, a user with shell access and sudo privileges needs to run the following commands. It's good practice to do this once buildbot is idle: | |||
<syntaxhighlight lang="bash"> | |||
sudo -s | |||
systemctl stop buildslave | |||
systemctl stop buildmaster | |||
cd ~buildbot/scummvm-sites | |||
sudo -u buildbot git pull --ff-only | |||
systemctl start buildmaster | |||
systemctl start buildslave | |||
</syntaxhighlight> | |||
If problems arise, check if all files in the master (~buildbot/scummvm-master) and slave (~buildbot/scummvm-slave) directories are owned by the correct user (buildbot:buildbot). If the daemons were incorrectly started, new files might have been created under another user account. In that case: | |||
* stop the daemons (see above) | |||
* fix the permissions with | |||
<syntaxhighlight lang="bash"> | |||
chown -R buildbot:buildbot ~buildbot/scummvm-master/* | |||
chown -R buildbot:buildbot ~buildbot/scummvm-slave/* | |||
</syntaxhighlight> | |||
* start the daemons | |||
=== Rebuilding Toolchain Libraries === | |||
It is periodically necessary to update the toolchain libraries for | |||
newer version releases or to add new library dependencies. | |||
This is a basic outline procedure for how to build these on the | |||
buildbot. | |||
* Toolchains are located at /opt/toolchains/<host target> in general, with the following exceptions: | |||
** Though some of the MinGW 32 toolchain is present in | |||
/opt/toolchains/i586-mingw32msvc | |||
which should be used as the host/prefix in the following | |||
instructions, the binaries i.e. i586-mingw32msvc-gcc are located | |||
at /usr/bin. This is probably as they are precompiled to be | |||
installed at this location, so this is necessary. | |||
However, if the PATH step to the bin is omitted in the following, | |||
this will still work for this target, which is confusing and could | |||
result in odd builds as some of toolchain bin contents may be needed. | |||
* All source tarballs used and any patches should be left in /opt/toolchains for reference. | |||
* /opt/toolchains is actually a symlink to the real location of the external storage drive, but /opt/toolchains should be used in all configuration to ensure that this can be freely moved in future if necessary. | |||
libSDL: | |||
* Decompress the latest libSDL tarball to a temporary location i.e. | |||
<syntaxhighlight lang="bash"> | |||
tar -xvzf SDL-1.2.15.tar.gz -d /home/digitall | |||
</syntaxhighlight> | |||
* Apply any required patches i.e. Currently libSDL-1.2.15 snapshot has a error in the Win32 version strings.. This may occur again on future releases :/ | |||
<syntaxhighlight lang="bash"> | |||
patch -p0 < SDL-1.2.15-Fix-Windows-DLL-Version.patch | |||
</syntaxhighlight> | |||
* Add toolchain binaries directory to path. | |||
<syntaxhighlight lang="bash"> | |||
PATH=/opt/toolchains/x86_64-w64-mingw32/bin:$PATH | |||
export PATH | |||
</syntaxhighlight> | |||
* Add out of tree build directory and configure the SDL codebase here with the host target and prefix to use the correct cross toolchain and target for make install. | |||
<syntaxhighlight lang="bash"> | |||
mkdir SDL-1.2.15-x86_64-w64-mingw32 | |||
cd SDL-1.2.15-x86_64-w64-mingw32 | |||
../SDL-1.2.15/configure --host=x86_64-w64-mingw32 --prefix=/opt/toolchains/x86_64-w64-mingw32 | |||
</syntaxhighlight> | |||
* Compile the codebase. | |||
<syntaxhighlight lang="bash"> | |||
make | |||
</syntaxhighlight> | |||
* Install the compiled files. As you become root, you need to make the same modification to the path to get the required cross toolchain tools such as ranlib. | |||
<syntaxhighlight lang="bash"> | |||
sudo bash | |||
PATH=/opt/toolchains/x86_64-w64-mingw32/bin:$PATH | |||
export PATH | |||
make install | |||
</syntaxhighlight> | |||