37 Quality Assurance(QA) Hooks #
OBS provides multiple hooks to place automated or manual tests at different points of time.
This chapter describes the different possibilities to provide and execute QA checks. The order of the items is sorted by the order in a typical development process. It is preferred to add a check as early as possible in the process to keep turn-around times small.
37.1 Source Related Checks #
Things which can be verified based on a given source can be checked even before commit time on the developers workstation. This is the earliest possible point of time to add a check. But it can also optionally be enforced on the server side.
Automated source processing is done by source services in OBS world. Check the source service chapter how to use or write one. It is important to decide if the test case shall output warning messages and when it shall report an error by exit status.
Test cases in source services get usually applied to all packages of a project. (It is possible to execute it only for specific packages though.)
37.2 Build Time Checks #
37.2.1 In-Package Checks #
Checks running during the build of a package are usually test cases provided by the author of a package. However, the packager can also add simple checks, for example, for code that is known to break on version updates and might be forgotten when the package is touched the next time.
These test are often specific for a concrete package only. So it is typically executed in %check section of rpm spec files directly. In case the check can be used with multiple package source, it is a good idea to package the test case in an own package and just call it from the other packages.
rpm calls %check after %install section and before creating the actual checks.
SUSE distributions also provide build time checks to test the installed files inside of the build root. It is to be used for test cases which shall run on all packages which are built inside of a distribution. This hook can be used by installing a file to /usr/lib/rpm/brp-suse.d/ directory. These scripts also have the power to modify installed files if needed.
37.2.2 Post Build Checks #
The standard tool to test binary packages for RPM-based distributions
is rpmlint. DEB-based distributions use the
lintian tool instead.
These checks are executed by the build script after a successful build. Note that these are executed as the standard user by default.
37.2.3 Post Build Root Checks #
Files in /usr/lib/build/checks/* are executed as root user. Typical use cases are install tests of the build packages to ensure that the scripts inside of the packages are working in general.
37.2.4 KIWI Specific Post Build Root Checks #
The file /usr/lib/build/kiwi_post_run is executed after KIWI jobs have finished. It can be used to run the appliance or to modify it. For example to package an appliance into an rpm.
37.3 Workflow Checks #
Workflow steps, for example transferring packages from one project to another, are done via requests in OBS. At least when multiple parties are involved. One or more of these parties can be automated test cases. Or human manual approval steps.
Default reviews can be defined inside of projects and packages. A new request to a certain package does get the reviewers added defined in target projects and packages. Reviewers can be currently users, groups or the maintainers of a specified project or package.
37.3.1 Automated Test Cases #
Open requests can be requested in an XML parseable way via the API running
osc api /request?states=review&user=auto-review-user&roles=reviewer&reviewstates=new&view=collection
osc can be used to accept or decline requests
after running the automated test. It can also add a review comment which you can use to give a reason (for example, error messages) for accepting or declining a request.
Requests, which are not
tested, for example because they are of a not matching type (for example, deleting
packages) needs to get also a review accept. Otherwise, this would block the process.