16 Quality Assurance(QA) Hooks

OBS provides multiple hooks to place automated or manual tests at different points of time.

This chapter describes the different possibilities to provide and execute QA checks. The order of the items is sorted by the order in a typical development process. It is preferred to add a check as early as possible in the process to keep turn-around times small.

16.1 Source Related Checks

Things which can be verified based on a given source can be checked even before commit time on the developers workstation. This is the earliest possible point of time to add a check. But it can also optionally be enforced on the server side.

Automated source processing is done by source services in OBS world. Check the source service chapter how to use or write one. It is important to decide if the test case shall output warning messages and when it shall report an error by exit status.

Test cases in source services get usually applied to all packages of a project. (It is possible to execute it only for specific packages though.)

16.2 Build Time Checks

16.2.1 In-Package Checks

Checks running during the build of a package are usually test cases provided by the author of a package. However, also the packager might add simple checks, esp. for stuff which is known to break on version updates for example and might be forgotten when the package gets touched the next time.

These test are often specific for a concrete package only. So it is typically executed in %check section of rpm spec files directly. In case the check can be used with multiple package source, it is a good idea to package the test case in an own package and just call it from the other packages.

rpm calls %check after %install section and before creating the actual checks.

SUSE distros is also providing build time checks for testing the installed files inside of the build root. It is to be used for test cases which shall run on all packages which are build inside of a distribution. This hook can be used by installing a file to /usr/lib/rpm/brp-suse.d/ directory. These scripts have also the power to modify the installed files if needed.

16.2.2 Post Build Checks

The standard to test built package binaries is rpmlint for rpm based distros. Debian distributions are using the lintian tool.

These checks are executed by the build script after a successful build. Note that these are executed as the standard user by default.

16.2.3 Post Build Root Checks

Files in /usr/lib/build/checks/* are executed as root user. Typical use cases are install tests of the build packages to ensure that the scripts inside of the packages are working in general.

16.2.4 KIWI Specific Post Build Root Checks

The file /usr/lib/build/kiwi_post_run is executed after KIWI jobs have finished. It can be used to run the appliance or to modify it. For example to package an appliance into an rpm.

16.3 Workflow Checks

Workflow steps, for example transferring packages from one project to another, are done via requests in OBS. At least when multiple parties are involved. One or more of these parties can be automated test cases. Or human manual approval steps.

Default reviews can be defined inside of projects and packages. A new request to a certain package does get the reviewers added defined in target projects and packages. Reviewers can be currently users, groups or the maintainers of a specified project or package.

16.3.1 Automated Test Cases

Open requests can be requested in an XML parseable way via the API running

osc api /request?states=review&user=auto-review-user&roles=reviewer&reviewstates=new&view=collection

osc can also be used to accept or decline the reviews of these requests after running the automated test. Also a comment can be given to show errors which appeared. It is important that requests, which are not tested, for example because they are of a not matching type (eg. deleting packages) needs to get also a review accept. Otherwise the process would get blocked.

Print this page