Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
User Guide / Reference / SCM Bridge

26 SCM Bridge

26.1 SCM Bridge

26.1.1 Introduction

The SCM bridge allows the sources of a single package or an entire project to be stored and maintained in any trusted SCM repositories. However, git is currently the only supported SCM system.

This allows to manage all packaging-relevant source changes in an external SCM repository using the native tooling of the SCM (e.g. git). It is recommended to have this SCM server on the same trust level as the OBS instance as it becomes the authoritative source. However, any SCM server reachable via network could be used.

osc can still be used to checkout sources. This will create an SCM repository, so any modification needs to be done using the native SCM tooling (e.g. git). osc can still be used for local building and checking server side build results.

Be aware that any source modification mechanics, submit requests or link behaviour may work differently or not at all. For example you can not use automatic package source merging via _link files anymore. This means also that workflows inside of OBS are not working anymore or are limited. A submit or release request would still work with a scmsync (source control management system sync) source, but the user would always need to manually specify the target. Functionality for when an entire project is managed via scmsync is even more limited.

Furthermore it is recommended to configure a global notification hook in the SCM server to allow OBS to follow the sources automatically.

26.1.2 Setup a package using the scm bridge

The setup is purely done in package meta by defining the SCM URL inside of the scmsync tag:

<scmsync>https://gitlab.com/some/repository</scmsync>

The repository is cloned including all subdirectories and submodules. However the build descriptions need to be placed in the top-level directory.

Large binary files may be stored using git LFS or by referencing external build assets, using asset management. These assets will get downloaded and verified as well. The advantage is that this information can also be used to compile Software Bills of Materials (SBOMs). Have a look in the pbuild documentation for further details on this.

In another typical scenario, a package maintainer owns a git repository where a build description and distribution specific files are stored. The sources from the upstream project may be added via a git submodule. However, for many build types like rpm builds it is recommended to build a tar ball at buildtime for the source rpm. This can be achieved with buildtime source services.

26.1.3 Setup an entire project using the SCM bridge

An entire project can be setup by defining the scmsync tag in project meta data. Every top level subdirectory of the scm repository is considered as a package.

Large projects may use git submodules for each package. This avoids the need to clone the entire project to modify a single package.

The build configuration part can get stored as '_config' file in the top level directory.

It is possible to limit the actual used package sources by specifying an "onlybuild" cgi parameter as part of the scmsync url. This parameter can be used multiple times. This is useful, for example, when trying to do a test build of just a few packages without the need to modify the source. In that way an open merge request may be tested before merging. Example URL:

<scmsync>https://gitlab.com/some/repository?onlybuild=glibc&onlybuild=kernel</scmsync>

26.1.4 Implementation and Limitations

The sources are currently cloned for the OBS source server to allow OBS to process them. This is to be considered an internal implementation detail and may change in future.

  • Git submodules: are cloned by default

  • Git VCS history: is not included for server side builds by default to save storage space. It can be included by adding a keepmeta=1 CGI parameter to the URL. This is not needed for local building when using osc.

  • Git LFS files: are included by default. They can be excluded by adding a lfs=0 CGI parameter to the URL.

  • Using just a subdirectory of the git repository can be done by adding a subdir=DIRECTORY CGI parameter to the URL.

  • Architecture specific remote assets can be selected by adding an arch=ARCH CGI parameter to the URL.

As an example, for the subdirectory case the URL would look like this:

<scmsync>https://gitlab.com/some/repository?subdir=MY_SUBDIRECTORY</scmsync>

26.1.4.1 Using a specific revision, tag or branch

The URL can define a revision, tag or branch via an URL fragment. This means the URL can get extended by a hash character and the revision, tag or branch.

<scmsync>https://gitlab.com/some/repository#MY_REVISION</scmsync>

This allows to set up multiple projects building for different branches. It is possible to use branches for implementing CI workflows. For example a submission test building just a subset, a clean build and a final reviewed build.

26.1.4.2 Converting to a project git

A project git repository is mostly the same as any project for the pbuild tool. The command

# osc create-pbuild-config

creates the _config and _pbuild files for a given repository and architecture. The _config file is used by OBS as well as by the build configuration (prjconf). The additional _pbuild file is analogous to the OBS project meta, but is not yet honored by OBS. Still, it can be included in the git tree for pbuild users.

26.1.4.3 Forking a scmsync package

The OBS server allows to create a cloned project with another package using a specified SCM repository. This can be used by tooling to create forked test builds for merge requests. The api POST route for this is

/source/PROJECT/PACKAGE?cmd=fork&scmsync=NEW_SCM_URL

26.1.5 SCM Source Updates

The OBS instance needs to get notified on any change in the SCM server. There are two ways to achieve this. One way is via single configurations for each git repository as documented in the Section 7.6, “Trigger a service run via a webhook” documentation. An alternative way is to configure it globally. This requires admin permissions on the git hosting side and another bridge implementation. The advantage is that any used repository will be synced automatically just by referencing it via the scmsync meta tag.

The update notification has been only implemented for gitea atm, you can find details in the obs-gitea-bridge documentation.