Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]

8 SCM/CI Workflow Integration

8.1 SCM/CI Workflow Integration Setup

8.1.1 Introduction

With this integration, you can take advantage of source code management (SCM) systems like GitHub, GitLab or Gitea to manage your packages sources. Then, you can integrate those sources with OBS to run different workflows, for instance, to build a package and report back the result to the SCM.

In the following sections, you will find the instructions to set up the integration between SCMs and OBS.

This chapter talks in GitHub jargon to simplify the text. As constantly mentioning all the names for the same things, e.g. Pull Requests/Merge Requests, is tiresome and confusing. However, every aspect has its correspondence in GitLab and Gitea. Refer to Section 8.1.10, “Equivalence Table” for clarification of terminology.

8.1.2 Prerequisites

Before you start, you need to

  • have a repository on GitHub.

  • have a package on an OBS Instance.

8.1.3 Supported SCMs

We support the GitHub.com and GitLab.com instances.

We also support Self-Hosted instances from GitHub, GitLab and Gitea. As long as the network connectivity works, OBS will be able to interact with that SCM.

8.1.4 Token Authentication

OBS and GitHub need to talk to each other. Tokens are the way to make this happen.

8.1.4.1 How to Authenticate OBS with SCMs

You have to create a GitHub personal access token. OBS is going to use it to talk to GitHub on your behalf.

The personal access token needs, at least, the following scopes assigned:

  • GitHub: repo

  • GitLab: api

  • Gitea does not support scopes for access tokens

Check GitHub’s , GitLab's and Gitea's documentation to learn how.

8.1.4.2 How to Authenticate SCMs with OBS

You have to create an OBS workflow token. Github is going to use it to trigger actions on OBS on your behalf.

8.1.4.2.1 Create Token

You can create the OBS token via WebUI in Profile > Manage Your Tokens.

You can also use osc for this:

osc token --create --operation workflow --scm-token long_ascii_salad

Example of response:

<status code="ok">
    <summary>Ok</summary>
    <data name="token">long_ascii_salad</data>
    <data name="id">12345</data>
</status>

Make sure you replace long_ascii_salad with your real GitHub personal access token created in Section 8.1.4.1, “How to Authenticate OBS with SCMs”

Warning
Warning

Don't forget to keep your token secret to prevent someone else from triggering operations in your name!

8.1.4.2.2 Regenerating Secrets and Deleting Tokens

If you suspect your OBS token secret was leaked, you can regenerate the secret or delete the whole token to secure it again:

a) Regenerate the token secret

Through the WebUI in Profile > Manage Your Tokens > Edit > Regenerate Token.

b) Delete the token

You can always delete your token via WebUI, in Profile > Manage Your Tokens, or with these commands:

osc token # list all your tokens
osc token --delete $token_id # remove the token with the given id

Then you can create a new one as explained in Section 8.1.4.2, “How to Authenticate SCMs with OBS” and replace it wherever you use it.

8.1.5 Webhooks

Once OBS and GitHub are allowed to speak to each other, they can start talking via webhooks.

8.1.5.1 SCM Events

On a GitHub repository, events are happening all the time: a pull request is created, somebody pushes a commit, a pull request is merged etc. When you set up a webhook on GitHub, you can specify which events you are interested in. Only when those events happen, the webhook will be sent to OBS.

This is the list of SCM events supported by the existing workflows in OBS:

  • Pull requests

  • Pushes

  • Tag Pushes

Other SCMs like GitLab have similar behaviour; the main difference is how they name the events. Refer to the Equivalence Table for more details or read more about GitHub events, GitLab events and Gitea events.

8.1.5.2 How to Set Up a Webhook on Github

Go to the project you want to set the integration on, then under Settings > Webhooks.

Webhooks on GitHub.
Figure 8.1: Webhooks on GitHub.

You have to fill in the form with:

  • Payload URL: https://build.opensuse.org/trigger/workflow?id=12345. Replace 12345 with the OBS token numerical ID previously obtained.

  • Content type: application/json.

  • Secret: uvwxyz. Replace uvwxyz with the OBS token secret string previously obtained.

  • Enable SSL verification.

  • Let me select individual events: Pull requests, Pushes.

8.1.5.3 How to Set Up a Webhook on GitLab

Go to the project you want to set the integration on, under the Settings > Webhooks.

Webhooks on GitLab.
Figure 8.2: Webhooks on GitLab.

Fill in the following fields:

  • URL: https://build.opensuse.org/trigger/workflow?id=12345. Replace 12345 with the OBS token numerical ID previously obtained.

  • Secret Token: uvwxyz Replace uvwxyz with the OBS token secret string previously obtained.

  • Trigger: Merge request events, Push events, Tag push events.

8.1.5.4 How to Set Up a Webhook on Gitea

Go to the repository you want to set the integration on, then under Settings > Webhooks.

Webhooks on Gitea.
Figure 8.3: Webhooks on Gitea.

You have to fill in the form with:

  • Target URL: https://build.opensuse.org/trigger/workflow?id=12345. Replace 12345 with the OBS token numerical ID previously obtained.

  • HTTP Method: POST.

  • POST Content Type: application/json.

  • Secret: uvwxyz. Replace uvwxyz with the OBS token secret string previously obtained.

  • Custom events...: Pull Request, Push.

8.1.6 OBS Workflows

A GitHub event occurs and OBS receives the corresponding webhook. Now is when the OBS workflows come into play.

A workflow is nothing else than a sequence of steps you want to perform in OBS. You can describe the steps to run in a YAML configuration file.

To do so, in the root directory of your GitHub repository, create a directory .obs which contains a file called workflows.yml.

The content of .obs/workflows.yml could look like this:

rebuild_master:
  steps:
    - rebuild_package:
        project: home:Admin
        package: ctris
  filters:
    event: push

You can also define multiple workflows, each one needs an unique name. The following example contains two workflows: main_workflow and rebuild_master.

main_workflow:
  steps:
    - branch_package:
        source_project: OBS:Server:Unstable
        source_package: obs-server
        target_project: OBS:Server:Unstable:CI
  filters:
    event: pull_request
rebuild_master:
  steps:
    - rebuild_package:
        project: home:Admin
        package: ctris
  filters:
    event: push
    branches:
      only:
        - master

OBS assumes the configuration file is placed in your target branch and not in other branches or forks.

8.1.6.1 OBS Workflow Steps

We support the following steps (the keys used in the configuration file appears surrounded with parenthesis):

  • Branch a package in a project (branch_package).

  • Link a package to a project (link_package).

  • Configure repositories/architectures for a project (configure_repositories)

  • Rebuild a package (rebuild_package)

  • Set flags for projects, packages, repositories or architectures (set_flags)

  • Trigger services of a package (trigger_services)

Warning
Warning

The user the token belongs to needs to have permissions to branch a package, link packages, configure repositories/architectures, rebuild packages and trigger services of a package.

8.1.6.1.1 Branch a Package in a Project

Given we have a source package called ctris coming from a source project called games, and a target project called home:jane, this step will branch that package onto the target project, keeping in mind that:

  • With a pull request event, it will go to e.g.: home:jane:github:jane:ctris:PR-1/ctris. PR-1 being the pull request number.

  • With a push event for commits, it will go to e.g.: home:jane/ctris-66f2acfbded89a19935ee6d481b7cf2ab95427f6. 66f2acfbded89a19935ee6d481b7cf2ab95427f6 being the SHA of the latest commit that triggered the event.

  • With a push event for tags, it will go to e.g.: home:jane/ctris-release_1. release_1 being the name of the tag that triggered the event.

This is an example of a configuration file with a branch package step:

workflow:
  steps:
    - branch_package:
        source_project: games
        source_package: ctris
        target_project: home:jane
8.1.6.1.3 Configure Repositories/Architectures for a Project

Given a project called home:jane, the step will configure a number of repositories and architectures for:

  • home:jane:jane_github:repo123:PR-1 when the event is a pull request. jane_github being the username/organization which owns the SCM repository. repo123 being the name of the SCM repository. PR-1 being the pull request number.

  • home:jane when the event is a commit or tag push.

Each repository needs:

  • a name, e.g.: openSUSE_Tumbleweed

  • a list of paths, each having a target project (e.g: openSUSE:Factory) and target repository (e.g: snapshot)

  • a list of architectures to be defined for each repository. e.g.: x86_64 and i586

This is an example of a configuration file with a configure repositories step:

workflow:
  steps:
    - configure_repositories:
        project: home:jane
        repositories:
          - name: openSUSE_Tumbleweed
            paths:
              - target_project: openSUSE:Factory
                target_repository: snapshot
              - target_project: openSUSE:Tumbleweed
                target_repository: standard
            architectures:
              - x86_64
              - i586
          - name: openSUSE_Leap_15.2
            paths:
              - target_project: openSUSE:Leap:15.2
                target_repository: standard
            architectures:
              - x86_64
8.1.6.1.4 Rebuild a Package

Given a project called home:Admin and a package ctris, the step will rebuild the package home:Admin/ctris.

This is an example of a configuration file with a rebuild package step.

workflow:
  steps:
    - rebuild_package:
        project: home:Admin
        package: ctris
8.1.6.1.5 Set Flags for Projects, Packages, Repositories or Architectures

There are OBS-wide defaults for each flag type. This step is only necessary if you want to diverge from those defaults.

Providing the type build, the status enable and the project home:Admin, OBS will enable all builds:

  • for the home:Admin:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER project when the webhook event is a pull request.

  • for the home:Admin project when the webhook event is a push.

Providing multiple flags is supported as noted in the configuration file below:

workflow:
  steps:
    - set_flags:
        flags:
          - type: build
            status: enable
            project: home:Admin
          - type: publish
            status: disable
            project: home:Admin

The type, status and project keys are always required. Optional keys are also available to limit the flag to a package, repository or architecture.

The project, package, repository and architecture should exist before a flag is set for them. They can be created in steps preceding a set_flags step, although this isn't necessary as long as they exist.

The type has to be one of the following values:

  • lock (default status disable)

  • build (default status enable)

  • publish (default status enable)

  • debuginfo (default status disable)

  • useforbuild (default status enable)

  • binarydownload (default status enable)

  • sourceaccess (default status enable)

  • access (default status enable)

The status is either disable or enable.

So with the configuration file provided below and a pull request event, builds of the home:Admin:$MY_SCM_ORG:$MY_SCM_PROJECT:PR-$MY_PR_NUMBER/ctris package will be disabled for the openSUSE_Tumbleweed repository and x86_64 architecture. For a push event, it's exactly the same, except for the package which is home:Admin/ctris-$MY_COMMIT_SHA_OR_TAG_NAME.

workflow:
  steps:
    - set_flags:
        flags:
          - type: build
            status: disable
            project: home:Admin
            package: ctris
            repository: openSUSE_Tumbleweed
            architecture: x86_64
8.1.6.1.6 Trigger Services of a Package

Given a project called home:Admin and a package ctris, the step will trigger services of the package home:Admin/ctris.

Be sure to have a _service file in the package home:Admin/ctris.

This is an example of a configuration file with a trigger services step:

workflow:
  steps:
    - trigger_services:
        project: home:Admin
        package: ctris

8.1.6.2 Filters

You can customize when workflows run by declaring branch or Event filters. They will make workflows run or not for specific branches/events.

You can define them in the configuration file .obs/workflows.yml. Here's an example:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    event: pull_request
    branches:
      only:
        - master
        - staging
8.1.6.2.1 Filters Delimiters: only and ignore

Some steps can affect a group of elements (branches) You can use filter delimiters like only and ignore to specify which elements should be affected, or not, by the step.

The available filters delimiters are:

  • only: the step only affects the elements in the list.

  • ignore: the step affects all the elements except those in the list.

Note
Note

only has precedence over ignore, so if both are defined, ignore is not considered.

This is an example to run a workflow only for the target branches master:

workflow:
  steps:
    - rebuild_package:
        project: games
        package: ctris
  filters:
    branches:
      only:
        - master

This is an example to run a workflow for all the target branches except for the branch staging:

workflow:
  steps:
    - rebuild_package:
        project: games
        package: ctris
  filters:
    branches:
      ignore:
        - staging
8.1.6.2.2 Event Filter

Setting an event filter will run the workflow only for this event. The event filter doesn't accept multiple events. Documentation on the SCM events is here.

The available event filters are:

  • pull_request is for pull request events.

  • push is for push events related to commits.

  • tag_push is for push events related to tags.

The following is an example to run a workflow only for a pull request event:

workflow:
  steps:
    - branch_package:
        source_project: games
        source_package: ctris
        target_project: home:jane_doe
  filters: 
    event: pull_request
8.1.6.2.3 Branches Filter

Matches target branches based on their names and runs a workflow only for those branches.

This is an example to run a workflow for all target branches, except master and final:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: ctris
        target_project: games
  filters:
    branches:
      ignore:
        - master
        - final

Learn more about Filters Delimiters: only and ignore.

8.1.7 Status Reporting

Once all the steps in the workflow are done, OBS will report the build results back to GitHub.

OBS will show detailed package build status for each distribution and architecture you have set up in the configuration file.

Build Status
Figure 8.4: Build Status

Moreover, if your package builds several multibuild flavors, the status will have the flavor name appended to the package name:

Build Status for Several Multibuild Flavors
Figure 8.5: Build Status for Several Multibuild Flavors
Note
Note

Due to a limitation, the initial "pending" build status of packages with multibuild flavors is not reported. The build status for those flavors will however still be reported when the build finishes.

8.1.8 Workflow Runs

For every SCM event, OBS compiles relevant information about the workflows running on the system. You can find the so-called "Workflow Runs" under the list of tokens.

Tokens list
Figure 8.6: Tokens list

From the list of workflow runs, you can get information like:

  • the status (running/fail/success) represented by icons,

  • the type of event and action,

  • links to the SCM repository, pull/merge request or commit involved,

  • the time when the workflow was triggered

Workflow Runs
Figure 8.7: Workflow Runs

Click on each workflow run to get detailed information about it. OBS records the request received from the SCM,

Workflow Runs - Request
Figure 8.8: Workflow Runs - Request

the response sent back to the SCM,

Workflow Runs - Response
Figure 8.9: Workflow Runs - Response

and the artifacts used or generated during the run of each workflow.

Workflow Runs - Artifacts
Figure 8.10: Workflow Runs - Artifacts

These records will help with debugging workflows. If an error occurs in any of the workflow steps, the workflow run will record error messages. And reading the artifacts will help to understand what happened behind the scenes.

8.1.9 Errors

Table 8.1: Common Errors
ErrorReason
"Project not found"Make sure the projects you declared in the .obs/workflows.yml file exist in your OBS instance.
"Package not found"Make sure the packages you declared in the .obs/workflows.yml file exist in your OBS instance.
No build result updates are displayed in your PR/MRMake sure there are repositories defined on your source project. Another reason can be that the build did not start because your package is "unresolvable" or "broken".
The project in OBS doesn't get updated with the latest changes in the SCM.For certain steps you need to set up a _service file. Follow the obs-service-tar_scm documentation.

8.1.10 Equivalence Table

Table 8.2: Equivalence table
GitHubGitLabGitea
RepositoryProjectRepository
Pull requestMerge requestPull request
PRMRPR
PushPush HookPush
Pull requests (in webhook configuration)Merge request events (in webhook configuration)Pull Request (in webhook configuration)
Pushes (in webhook configuration)Push events (in webhook configuration)Push (in webhook configuration)

8.2 SCM/CI Workflow Integration Use-Cases

8.2.1 OBS SCM Service

For some of the use cases, you might want the OBS package to get the sources from the pull request in GitHub.

For this, you can make use of the existing obs-service-tar_scm service. Your package should include a properly defined _service file. obs-service-tar_scm will automatically use the sources of the pull request that triggered it.

8.2.2 Test Build a Package For Every Pull Request on GitHub

You decide to manage your package sources from a GitHub repository. However, every time someone tries to add changes to your sources by opening a pull request, you need to verify that your package still builds for certain repositories and architectures in OBS. You can have the best of both worlds with the SCM/CI workflows integration in OBS.

You will need:

  • A project in OBS that you own, it will be the target project. Let's say: home:jane:playground.

  • A package in OBS that you want to test build, it will be the source package inside the source project. E.g.: GNOME:Factory/gnome-shell.

  • A repository in GitHub with the source code that will receive the pull requests, e.g.: https://github.com/GNOME/gnome-shell.

  • The required tokens to allow OBS and GitHub talk each other as explained in Section 8.1.4, “Token Authentication”

  • The required webhooks so GitHub notifies OBS of any event as explained in Section 8.1.5, “Webhooks”

This is obviously a good candidate to use the OBS SCM Service.

There are two different strategies to do this: branching the package or linking to it.

8.2.2.1 Branch

If you decide to branch the package for the test build, the configuration file should be something like this:

workflow:
  steps:
    - branch_package:
        source_project: GNOME:Factory
        source_package: gnome-shell
        target_project: home:jane:playground
  filters: 
    event: pull_request

Whenever someone opens a new pull request in the repository, OBS will branch the source package onto the target project, trigger the build, and report the results in the pull request's status checks.

Keep in mind that, when OBS branches a package, it copies the repositories from the source project to the target project, so everything is ready to start building.

Once the pull request is accepted or closed, the branched package will be deleted.

Read how to set up the workflows and, specifically, the workflow steps.

8.2.3 Rebuild a Package for Every Change in a Branch

You have a test build set up and you want it to keep up to date with the new changes you add to the PR. One way to do it, is to configure a rebuild package step with a push event filter.

You need:

The workflow configuration should be like this one:

workflow:
  steps:
    - rebuild_package:
        project: home:jane
        package: rust
  filters:
    event: push

8.2.4 Set Flags on a Package to Disable Builds for an Architecture

When you branch a package, all its repositories and their architectures will be copied over. For this package, you might want to disable builds for a certain repository or architecture. This is possible with the set_flags step.

The workflow configuration should be like this one:

workflow:
  steps:
    - branch_package:
        source_project: home:jane_doe
        source_package: rust
        target_project: home:jane_doe:CI
    - set_flags:
        flags:
          - type: build
            status: disable
            project: home:jane_doe:CI
            package: rust
            architecture: x86_64

8.2.5 Create Package on OBS for Every Software Release With Git Tags

You have a software project for which you mark releases with Git tags. For every release, you want to create a package on OBS. This can be automated in a workflow with the branch_package step and the tag_push event filter. Once the workflow is in place, every tag you push to your SCM repository will branch a package on OBS and create, then build a package for the source code associated to the tag's commit. This way, your users can always install a versioned release of your software project. You can also link one of those versioned releases to another project on OBS if you need it as a dependency.

After the usual setup for OBS workflows with tokens and webhooks, you will need:

  • A package in OBS that you own, and for which you want to create releases. It will be the source package (e.g.: home:jane/my_package) and it will contain a _service file. When this package is branched by the branch_package step, the branched package name will end with the name of the tag which was pushed (e.g.: my_package-1.0).

  • A project in OBS that you own, and which will contain all packages created by the branch_package step. It will be the target project (e.g.: home:jane:releases).

  • A SCM repository for your software project with the source code and spec file in which you will create Git tags to mark releases, e.g.: https://github.com/jane/my_package.

The workflow configuration should be like this one:

workflow:
  steps:
    - branch_package:
        source_project: home:jane
        source_package: my_package
        target_project: home:jane:releases
  filters:
    event: tag_push

The _service file in your source package should be like:

<?xml version="1.0"?>
  <services>
    <service name="obs_scm">
      <param name="versionformat">@PARENT_TAG@</param>
      <param name="url">https://github.com/jane/my_package.git</param>
      <param name="scm">git</param>
      <param name="revision">@PARENT_TAG@</param>
      <param name="extract">my_package.spec</param>
    </service>
    <service name="set_version"/>
    <service name="tar" mode="buildtime"/>
  </services>

Here's an explanation of the services involved:

  • The obs_scm service fetches the source code from your SCM repository for a specific revision, which in this case, is for the latest tag (@PARENT_TAG@). Don't forget to extract the spec file with a extract element.

    Set the versionformat to match how you want to name your releases. This is typically the Git tag name, so @PARENT_TAG@ is what you should use. For other options, refer to git log and its format:<format-string> documentation.

  • The set_version service updates the Version directive in the spec file downloaded by the obs_scm service. The version format comes from the versionformat element under the obs_scm service.

  • The tar service creates a tarball out of the source code fetched by the obs_scm service.

For the spec file in your SCM repository, pay attention to this:

  • The Source0 directive is based on values you provided to the obs_scm service in the _service file and it should be like this: my_package-%{version}.tar.

    The first part is the SCM repository name (e.g: my_package). The second part is the version macro which will be expanded to match what you defined in the versionformat of the obs_scm service in the _service file. The third part is the archive extension (.tar) since a tarball was created by the tar service.

  • Under the %prep directive, you might have to update the %setup directive if your source package name doesn't match the name of your SCM repository. Here's how, with my_package being the SCM repository name:

    %prep
      %setup -q -n my_package-%version
Print this page