Manual Tests of Ubuntu Studio Packages

We have been caught out a few times in the lead up to some of the recent releases of Ubuntu Studio, where we discovered very late that there were problems with a particular package. If you are an experienced Ubuntu Studio user, or you would like to begin helping out in the Ubuntu Studio Developers Team, why not start testing packages for the next release (Zesty 17.04)?

Step 1 – Install the Ubuntu Studio Development Release

It is not recommended to install the development release on a computer where you cannot afford to loose important data. In order of preference, install it on:

  1. A spare computer with lots of audio/video hardware plugged in.
  2. A spare computer.
  3. Your main desktop/laptop computer with a spare hard disk plugged in.
  4. A Virtual Machine on your main desktop/laptop (not really suitable for audio/video applications).

Instructions for installing the Ubuntu Studio Development Release can be found here.

Step 2 – Choose a package to test

The list of Test Cases for Ubuntu Studio Zesty 17.04 can be found on the QA Package Tracker.

screenshot-from-2016-12-11-200300

Step 3 – Check package versions

It is a good idea to note down the version number of the package in the Ubuntu development release (you will need it when reporting any bugs you find), and also in Debian (and also upstream if you are keen). Let us in the Ubuntu Studio Development Team know if our package is way out of date so that we can look into what is blocking the newer version.

To find the version in Ubuntu use the search form at the bottom of this page. For Debian, use the search form at the bottom of this page. Make sure you search in the right distribution (Ubuntu – Zesty at the moment, Debian – unstable).

Step 4 – Run the test

Click on the package you want to test in the QA package tracker (see screenshot above), and the test case should appear.

Screenshot from 2016-12-11 20:20:02.png

Follow the steps of the test case. It is as simple as that. If you are an experienced user of that package, feel free to test further functions. The more bugs we find early in the release cycle, the more chance they will be fixed before the release.

Step 5 – Record the results & report bugs

For this step you will need to have a Launchpad login. Log into the package tracker. You can see the button on the above screen-shot. Record your results (hopefully a “pass”) in the bottom of the tracker. The results will be stored, so feel free to come back and test the same package later and add another result. If you spot a minor bug, then see if it has already been reported in Launchpad, and if not then report it. Add the bug number to the applicable column in your test result. If you cannot complete the test case due to a bug, please mark the test as failed (and add the bug number to the report). Feel free to add as many comments to the test result as you like. In particular, we are interested in your test environment (e.g. laptop/desktop/Virtual Machine), and the version of the package when you tested it.

Step 6 – Improve the Test Cases

If you have got this far, and finished a test, then well done and thank you! You deserve a break. But why stop there? Test a different package. We also need help maintaining the Test Cases. If you spot a mistake in a Test Case, or a note a possible improvement, then report a bug against the manual-tests in Launchpad. If you think we are missing a Test Case for an Ubuntu Studio package, then please also report a bug (after checking that there isn’t already one).

You could also help out further by actually correcting, or creating the Test Case yourself. There are excellent documents on how to do this on the QA wiki here:

Contributing Manual Test Cases

Leave a comment