Difference between revisions of "Release Testing"

From ScummVM :: Wiki
Jump to navigation Jump to search
(1.6.0 testing is no longer current)
 
(15 intermediate revisions by 8 users not shown)
Line 1: Line 1:
== Release Testing for ScummVM ==
== Release Testing for ScummVM ==


* [[Release Testing/2.7.0|2.7.0]]
* [[Release Testing/2.6.0|2.6.0]]
* [[Release Testing/2.3.0|2.3.0]]
* [[Release Testing/2.1.0|2.1.0]]
* [[Release Testing/2.0.0|2.0.0]]
* [[Release Testing/1.9.0|1.9.0]]
* [[Release Testing/1.8.1|1.8.1]]
* [[Release Testing/1.8.0|1.8.0]]
* [[Release Testing/1.7.0|1.7.0]]
* [[Release Testing/1.6.0|1.6.0]]
* [[Release Testing/1.6.0|1.6.0]]
* [[Release Testing/1.5.0|1.5.0]]
* [[Release Testing/1.5.0|1.5.0]]
Line 17: Line 26:
== Testing Guidelines ==
== Testing Guidelines ==


* Use the latest build which you may find at the bottom of our [http://scummvm.org/downloads/ download page].
* Use the latest build which you may find at the bottom of our [http://scummvm.org/downloads/ download page]. After we branched please use the latest '''stable''' builds when downloading from our [http://buildbot.scummvm.org/builds.html Buildbot].
* Use the launcher to add your game and see if it was detected normally, even if you already have this game already added. Start newly added target.
* Use the launcher to add your game and see if it was detected normally, even if you already have this game already added. Start newly added target.
* Preferably test detection of every game/demo you have, even when you're not going to play them.
* Preferably test detection of every game/demo you have, even when you're not going to play them.
Line 23: Line 32:
* Preferably use of save/load facility.
* Preferably use of save/load facility.
* '''Do not''' skip cutscenes.
* '''Do not''' skip cutscenes.
* If you encounter any problems, please, submit a bug report at our [http://sourceforge.net/tracker/?atid=418820&group_id=37116&func=browse bug tracker].
* If you encounter any problems, please, submit a bug report at our [https://bugs.scummvm.org/ bug tracker].
* If a game has several branches, test as many variants as you can.
* If a game has several branches, test as many variants as you can.
* At least 1 user should test each game, preferably but not obligatory on different platforms.
* At least 1 user should test each game, preferably but not obligatory on different platforms.
** However, more than one user should test games that satisfy one or more of the following conditions:
** However, more than one user should test games that satisfy one or more of the following conditions:
*** It is a multiple path game (such as ''Fate of Atlantis'' or ''Maniac Mansion'').
*** It is a multiple path game (such as ''Fate of Atlantis'' or ''Maniac Mansion''). We designate such games with a '''''#''''' (hash symbol).
*** It is a new game for this release.
*** It is a new game for this release. We designate such games with a '''''*''''' (star symbol).
*** We designate such games with a '''''#''''' (hash symbol).


== What to provide when you encounter a bug ==
== What to provide when you encounter a bug ==
Line 35: Line 43:
Please provide information on our bug tracker. Please, do not post bugs to our forums as that makes discussions and bug fixing progress tracking practically impossible.
Please provide information on our bug tracker. Please, do not post bugs to our forums as that makes discussions and bug fixing progress tracking practically impossible.


* Make sure that you have a working mail set up at your sf.net account. In this case you will receive our reply to your bug report.
* Make sure that you have a working mail set up at your GitHub account. In this case you will receive our reply to your bug report.
* Snapshot date (but please, always test with the latest one).
* Snapshot date (but please, always test with the latest one).
* Bug details, including instructions on reproducing it.
* Bug details, including instructions on reproducing it.
Line 42: Line 50:
* Version of game (talkie, floppy...).
* Version of game (talkie, floppy...).
* Whether the sound was compressed or not.
* Whether the sound was compressed or not.
* Platform and compiler (Linux, Mac OS X, Win32...).
* Platform and compiler (Linux, macOS, Windows, etc).
* Attach a save game if possible.
* Attach a save game if possible.
* Attach a screenshot to demonstrate graphics bugs if applicable.
* Attach a screenshot to demonstrate graphics bugs if applicable.

Latest revision as of 11:07, 7 January 2023

Release Testing for ScummVM

Testing Guidelines

  • Use the latest build which you may find at the bottom of our download page. After we branched please use the latest stable builds when downloading from our Buildbot.
  • Use the launcher to add your game and see if it was detected normally, even if you already have this game already added. Start newly added target.
  • Preferably test detection of every game/demo you have, even when you're not going to play them.
  • Play game from beginning through the end.
  • Preferably use of save/load facility.
  • Do not skip cutscenes.
  • If you encounter any problems, please, submit a bug report at our bug tracker.
  • If a game has several branches, test as many variants as you can.
  • At least 1 user should test each game, preferably but not obligatory on different platforms.
    • However, more than one user should test games that satisfy one or more of the following conditions:
      • It is a multiple path game (such as Fate of Atlantis or Maniac Mansion). We designate such games with a # (hash symbol).
      • It is a new game for this release. We designate such games with a * (star symbol).

What to provide when you encounter a bug

Please provide information on our bug tracker. Please, do not post bugs to our forums as that makes discussions and bug fixing progress tracking practically impossible.

  • Make sure that you have a working mail set up at your GitHub account. In this case you will receive our reply to your bug report.
  • Snapshot date (but please, always test with the latest one).
  • Bug details, including instructions on reproducing it.
  • Language of game (English, German...).
  • If it is a fan translation, mention it.
  • Version of game (talkie, floppy...).
  • Whether the sound was compressed or not.
  • Platform and compiler (Linux, macOS, Windows, etc).
  • Attach a save game if possible.
  • Attach a screenshot to demonstrate graphics bugs if applicable.

It may happen that at some point of our testing we break savegame compatibility, or do something which affects a specific engine deeply. In that case, you could be asked to restart the game you're currently playing from the beginning. You will be explicitly told to do so, so don't worry if we say nothing in this aspect.

Please, report about your completion on our forums (preferably in the thread linked by the testing announcement). If you encountered some problem, provide the bug report number as well.

Thanks for your help.