HOWTO-Release

How to make a ScummVM release.

NOTE: This checklist simply works from top to bottom. No deviations, not jumping around, just follow it. And yes, I wrote this for myself (rootfather)

This is work in progress and needs to be completed!

All examples are based on the 2.7 and 2.7.0 version. Don't forget to adapt with the adequate version number.

For "x.y.0" releases of ScummVM, we also release a new version of the ScummVM Tools set.

Definitions

• Major release: A major release marks the leap from e.g. ScummVM 2.6.x to 2.7.x
• Minor release: A minor release marks the exact version we release, e.g. ScummVM 2.6.0 or 2.6.1.

Preparations

• Make the release plan, normally 1-2 weeks before branching, 3-4 weeks for bug fixing, 2 weeks after tagging before the release.
  • For reference, ScummVM 2.3.0 was built according to the following schedule:

August 14 (Sat): Release testing announced, feature freeze in master
August 28 (Sat): Release branch created, master unfrozen
September 17 (Fri): Release tagged, porters asked to submit their builds
September 27th (Mon): Release

• e-mail -devel with the proposed schedule, ask if someone would like to finish some functionality
• Look through all the open issues on the bugtracker and identify release blockers and nice-to-have-fixed bugs
• Update the lists of files below, adding new engine data files and such
• If some games must be fixed before the release, identify those and create a list at Release Testing
• Announce the testing period on the main website and copy to the forums and FB

During the release cycle

• Create a thread in the #team-talk channel on Discord: "X.Y.Z Release Preparations" for easier communication
• E-mail the scummvm-devel list at least weekly with the list of release blockers and nice-to-have-fixed bugs
• Share those e-mails in the release discussions thread on Discord
• Track daily testing progress and update Wiki if applicable. Normally, we collect playtests on the forums
• Watch for any potential blockers and pat developers who own the identified blockers
• Squash as many bugs as possible

Branching for major release

• Before creating the branch make sure there is no pending changes on weblate
  • Go to repository maintenance for the ScummVM/scummvm component on the translations website: https://translations.scummvm.org/projects/scummvm/scummvm/#repository
    • Click on the Lock the component button
    • Commit pending changes, and push.
• Create a branch for the 2.7.x releases from master, named branch-2-7
  • git checkout -b branch-2-7 master
• Increase version on branch-2-7 to 2.7.0pre (using devtools/update-version.pl; see below), and commit
  • devtools/update-version.pl 2 7 0 pre
  • git commit -m "RELEASE: This is 2.7.0pre" -a
• Create annotated tag desc/2.7.0pre on that commit and push
  • git tag -a -m "Mark 2.7.0pre" desc/2.7.0pre
  • git push origin tag desc/2.7.0pre branch-2-7:branch-2-7
• Update doc/docportal/other_platforms/ios_build.rst to clone the new branch
  • Change git clone https://github.com/scummvm/scummvm.git to git clone --depth 1 -b branch-2-7 https://github.com/scummvm/scummvm.git
  • git add doc/docportal/other_platforms/ios_build.rst
  • git commit -m 'DOC: Update iOS clone command for 2.6 branch'
  • git push
• Increase version on master to 2.8.0git, and commit
  • git checkout master
  • devtools/update-version.pl 2 8 0 git
  • git commit -m "RELEASE: This is 2.8.0git" -a
• Create annotated tag desc/2.7.0git on that commit and push
  • git tag -a -m "Mark 2.8.0git" desc/2.8.0git
  • git push origin tag desc/2.8.0git master
• Switch Weblate to the new branch
  • Go to the component Version control Settings: https://translations.scummvm.org/settings/scummvm/scummvm/#vcs
    • Change the Repository branch setting from master to the new branch
  • Go to repository maintenance for the ScummVM/scummvm component on the translations website: https://translations.scummvm.org/projects/scummvm/scummvm/#repository
    • Unlock the component
• Update buildbot
  • Edit at the end of builds.py the line starting with builds.append(ScummVMStableBuild("stable",.
  • Remove the customizations in class ScummVMStableBuild if there were any to make the code inherit from ScummVMBuild without any difference.
  • Edit the file platforms.py and remove all specific rules mentioning ScummVMStableBuild
  • Update the code on the server and reload the buildbot as described here
  • Remove the old snapshots: rm /home/buildbot/dockerized-bb/buildbot-data/packages/snapshots/stable/*
  • Login on Buildbot, force a build of fetch-stable with cleaning and packaging to create packages for the new stable.
• Communication
  • Update IRC and Discord channel description
  • Write a mail to scummvm-devel

Semi-automated Checks

• Run

  make release-checks

  for performing several routine checks covered below:
  • Icons and Shader packs
  • ADGF_TESTING
  • IDE Projects
  • Translations
  • Release Notes on FRS
  • Engine data for MM and Ultima

Testing

• Ensure ADGF_TESTING/ADGF_UNSTABLE flags are correct for the release.
• Identify release critical (show stopper) bugs and get people to fix them.
• Identify other important bugs and get people to fix them.
• Get even more people to fix all the other bugs, if possible :-).
• Get people to do lots of testing, possibly by asking for help via a news item.
• Collect all that information (e.g. in the Wiki). See Release Testing

Preparing the source

• Browse 'git log' since the previous release and add all significant changes to the NEWS file
• For a major release, make sure the translations.dat file is up to date.
  • Go to repository maintenance for the ScummVM/scummvm component on the translations website: https://translations.scummvm.org/projects/scummvm/scummvm/#repository
    • Lock the component
    • Commit and push pending changes
  • Use make translations-dat to update the translations data file (and then commit it if it was changed).
  • At this point Weblate can be switched back to master:
    • Copy all po/*po file from the release branch
    • In master branch run make updatepot ; make po/*.po
    • Commit and push: git add po/scummvm.pot po/*.po ; git commit -m "I18N: Merge translations from branch-2-6" ; git push
    • On https://translations.scummvm.org/settings/scummvm/scummvm/#vcs switch the repository branch setting back to master
    • On https://translations.scummvm.org/projects/scummvm/scummvm/#repository unlock the component
• At this point, create the minor release branch according to "Branching for minor releases"
• Note: Make sure that you update the version strings on _both_ the major and the minor branch accordingly!
• You have to update the versions in all relevant files. There is a script, devtools/update-version.pl meant to help you with this. It will generate/update the following files:
  • base/internal_version.h
  • dists/macosx/Info.plist
  • dists/redhat/scummvm.spec
  • dists/scummvm.rc
• Review the following documentation files for version and copyright year updates:
  • README(.md)
  • NEWS(.md)
  • COPYRIGHT
• Make sure ADGF_TESTING flag is cleared on all games.
• Review the following port specific files for version updates:
  • backends/platform/psp/README.PSP
  • backends/platform/symbian/README
• Review the release dates in the following port specific files:
  • dists/win32/scummvm.gdf.xml
• Make sure package descriptions are up-to-date:
  • dists/redhat/scummvm.spec
• Make sure there are no any significant pending GUI changes in the default icons set:
  • Check scummvm-icons repo
  • Compare the creation date of the gui/themes/gui-icons.dat with the directory above.
  • If there are changes, run zip -r9 gui-icons.dat * and the commit it to the gui/themes
• Create MSVC project files in the release branch (for example branch-1-7-0, where the tarballs are created from)
  • make ideprojects
  • git commit -m "DISTS: Generated Code::Blocks and MSVC project files"
  • And last but not least don't forget to push the results to our github repo via "git push"
• Make source tarballs, put them in the FRS directory.
  • There is a script devtools/dist-scummvm.sh, which creates them automatically.
  • Make sure that porters have write access to the release folder on FRS: chmod 775 <folder>
• Follow the instructions at HOWTO-Release#Signing binaries in order to enable updates via (Win)Sparkle

Branching for minor releases

• Create a branch for 1.7.0 from branch-1-7, named branch-1-7-0
  • git checkout -b branch-1-7-0 branch-1-7
• Increase version on branch-1-7-0 to 1.7.0 and commit, push
  • devtools/update-version.pl 1 7 0
  • Manually edit backends/platform/maemo/debian/changelog to reflect correct release date for 1.7.0 entry
  • git commit -m "RELEASE: This is 1.7.0" -a
  • git push origin branch-1-7-0:branch-1-7-0

Binaries

• Get porters to make binaries.
  • Porter must follow our filename convention, which goes like this: "scummvm-VERSION-PORT.EXTENSION". For example: "scummvm-1.1.0-macosx.dmg". Do not use special names like "EScummVM" or "ScummVMDS". The only exclusion from this requirement is Debian.
  • Porters should pass "--enable-release" to configure in case their port is using the configure/make based build system. If their port is not using the configure/make based build system they should make sure they properly define RELEASE_BUILD when compiling their release binary. It is also noteworthy that "--disable-debug" should be passed in order to remove debugging information from the binary.
  • Here is a list of files porters should include in their release archives:
    • AUTHORS
    • COPYING
    • LICENSES/* (all additional licenses)
    • COPYRIGHT
    • NEWS(.md)
    • README(.md) (generic ScummVM one)
    • localized NEWS and README files (from doc/) (optional)
    • if your port uses new GUI:
      • scummmodern.zip (from gui/themes directory)
      • scummclassic.zip (from gui/themes directory)
      • scummremastered.zip (from gui/themes directory)
      • residualvm.zip (from gui/themes directory)
    • translations.dat (from gui/themes directory)
    • engine data from dists/engine-data
      • *.dat
      • grim-patch.lab
      • monkey4-patch.m4b
      • pred.dic
      • queen.tbl
      • sky.cpt
      • wintermute.zip
      • xeen.ccs
    • your build binary
    • your build-specific README (optional)
    • ...
• Porters need to upload binaries. To do this, we will use our own Nextcloud instance, so porters don't have to rely on third-party services. To do this:
  • Create a new shared folder with upload permissions in the Nextcloud (ask rootfather when in doubt)
  • Set a new password for this shared folder
  • Mention this password in the mail to the mailing list asking for the porters to submit their builds

Updating the Website

Update the website internally (i.e. update the version in the repository), but do not yet put these changes online.

• Edit include/config.inc.php to update the global version.
• Add a news item.
• Follow the instructions at HOWTO-Release#Signing binaries in order to enable updates via (Win)Sparkle
• Copy over the release files to FRS
  • Set the OS for the newly uploaded binaries (click on the (i) next to each binary).
    • -win32.exe as Windows binary
    • -macosx.dmg as Mac binary
    • .tar.bz2 as Linux, FreeBSD and Other binary
    • -solaris-x86.tar.gz as Solaris binary
• On the ScummVM Data spreadsheet
  • On the versions tab, add the new release to the bottom of the list
  • On the compatibility tab, replace DEV with the release number for all newly added games
  • On the scummvm_downloads tab, for any releases that have been uploaded to FRS, change the version column to match the latest version. Verify that all ports that don't have the new version (yet) are set to "old", so they appear in the right section of our website.
  • WARNING: The website update script keeps track of the different sheets in the spreadsheet file based on an internal ID. You must use the exact same sheet for every release. This means that you have to temporarily duplicate the downloads sheet and constantly update it while the porters submit their builds - you don't want to all builds at once right before the release. As soon as the release is about to get published, copy over all content of the temporary scummvm_downloads sheet to the productive one. You must not rename the old sheet under any circumstances, it will break the website!

Tagging a version

• When 1.7.0 is ready to be announced, make a permanent v1.7.0 tag in git to mark the release
  • git checkout branch-1-7-0
  • git tag -a -m "Tag v1.7.0" v1.7.0
  • git push origin tag v1.7.0
• Also increase the version number on the 1.7.x branch to 1.7.1pre
  • git checkout branch-1-7
  • devtools/update-version.pl 1 7 1 pre
  • git commit -m "RELEASE: This is 1.7.1pre" -a
• Create annotated tag desc/1.7.1pre on that commit and push
  • git tag -a -m "Mark 1.7.1pre" desc/1.7.1pre
  • git push origin tag desc/1.7.1pre branch-1-7:branch-1-7
• Add this doc to the Read The Doc admin settings to generate the documentation for this version.
  • Open Read The Docs Portal
    1. Click on ScummVM
    2. Click on Versions
    3. Find the relevant tag like 'v2.6.0'
    4. Press Activate

Creating source tarballs

Then you need to create and upload the tarballs.

devtools/dist-scummvm.sh scummvm 1.9.0
devtools/dist-scummvm.sh scummvm-tools 1.9.0

Then upload the resulting scummvm-1.9.0.tar.gz, scummvm-1.9.0.tar.bz2, scummvm-1.9.0.tar.xz and scummvm-1.9.0.tar.zip to the FRS.

Signing binaries

Put tarballs into directory archives/, run:

bin/generate_appcast archives/
bin/old_dsa_scripts/sign_update archives/scummvm-2.1.1-win32.exe dsa_priv.pem
bin/old_dsa_scripts/sign_update archives/scummvm-2.1.1-macosx.dmg dsa_priv.pem

The first line will generate file archives/release.xml. The following ones will just spit out DSA signatures.

Copy/paste signatures to dists/macosx/scummvm_appcast.xml in the main git repo.

Then run:

make publish-appcast
cd ../scummvm-web
git commit -m 'WEB: Sync appcast' -a

Generating the Icons and Shader packs

• Run https://github.com/scummvm/scummvm-icons/blob/master/gen-set.py to generate icons pack
• Then upload the generated LIST and the gui-icons-DDDDDDDD.dat files to /.0/frs/icons/ on www.scummvm.org
• Run. TODO, to generate shaders pack
• Then upload the generated LIST-SHADERS and the shaders-DDDDDDDD.dat files to /.0/frs/icons/ on www.scummvm.org

News & Notification

Write a news blurb (or rather multiple to be used in the following). Usually, you want a bit longer one for the website, a list for Versiontracker, something with proper english sentences for Freshmeat, etc....

• Write a news item for our website (also to be used as a template for news items on other sites).
  • Change version constants in include/Constants.php
  • Put Release Notes into two files in the release directory, one is plain text, named "ReleaseNotes", another one is HTML, use markdowntohtml.com service but copy the HTML body tags from the previous release.
  • Make sure to update the Release information in the datasheet
  • Make sure Sparkle appcast is updated (see #Signing binaries)
• Update / notify various sites:
  • Post about the release on our forums
  • Post about the release on Facebook
  • Post link to the news item to Twitter
• Put the updated web site online
• Update IRC and Discord channel topics

GitHub release

Use https://github.com/scummvm/scummvm/releases/new for creating a new GitHub release:

• Use the current tag as "base" for the release
• Use the current version number as "Release title" (format: 1.2.3, no leading "v")
• Use the current ReleaseNotes in Markdown format for the "Describe this release" field
• GitHub automatically attaches the matching source tarball in .zip and .tar.gz format, so no need to upload it on your own

Updating the wiki

• Update Template:StableVersion. This is used for links to the Compatibility page for the stable version and for the latest stable version info field for each platform.
• Move newly supported engines in the Complete engines in the latest stable version section on the Engines page.
• Add the first release field on the info box of the newly supported engines.
• Update Platforms page if status changed for some of the platforms (e.g. no release for the latest stable version). For these platforms the latest stable version info will also need to be changed if it was using the StableVersion template.
• Set the First Official Version field for new platforms.
• Update the support field on the info box of the newly supported games. Also change these games from Unsupported Games category to Supported Games category.