How do I get involved with testing and QA with Ubuntu?
Solution 1:
The QA team does extensive automated testing - it's part of the requirements for hardware to get Ubuntu certified.
The project they use is Checkbox. Don't be fooled by the quite cut-down version shipped in the checkbox-gtk package - the full suite contains a huge range of tests.
The Ubuntu QA mailing list is where you want to go for checkbox questions, or to discuss merge requests. This link discusses about the complete set of tools used by the QA team for automation.
Solution 2:
There are a couple ways to get involved in writing automated tests. The quality team is involved in writing autopkg and autopilot tests for ubuntu packages.
Autopkg
Autopkg tests are run at build time automatically by the buildbots for the package. The goal of these tests is to provide system and integration testing to guarantee basic functionality. You can see the live output of the current autopkg tests here. If your interested in writing these tests, see:
- How do I contribute an autopkg test to ubuntu?
Autopilot
These tests are written in python using the autopilot framework. The goal of these tests is to provide higher level application and functional testing. The tests are written in python and can be user executed or scripted via a test runner such as UTAH or jenkins to run and publish results automatically. For more information on autopilot, check out the project on launchpad as well as the documentation for the project. If your interested in writing these tests, see:
- How do I contribute an autopilot test?
Getting Help
For more information or to get help, email the ubuntu-quality list or visit #ubuntu-quality on freenode.
Solution 3:
A project does exist at https://wiki.ubuntu.com/Testing/Automation for this - I'm not sure how much it currently covers, but there is also automated upgrade testing done to test out upgrading various package combinations.
Solution 4:
You can get involved with testing by joining the QA team, their mailing list - ubuntu-quality, and their launchpad page.
The last link will take you to the autopilot launchpad site which is thier automatated testing package.
See my answer here for additional details on how Ubuntu automated testing works.
Solution 5:
Usually automated testing is done on a per project basis and the quality of those tests are highly dependant on the project's organisation and quality control. Tests can be tied into the building of debs and such but as far as I know I've never heard of any external testing framework outside of any particular project.