We describe here the high-level concepts: how Open Build Service is designed, manages its content and is supposed to work.
All sources and binaries which are hosted inside of OBS are organized into projects. A project is the container defining a larger task. It defines who is working there.
A project is configured in the project
/source/$PROJECT/_meta
path. It can be edited in the
web interface using the RAW Config
tab or via command
line with
osc meta prj -e $PROJECT
This file contains:
Generic description data in title
and
description
elements.
An ACL list of users and groups connected with a role. The
maintainer
role defines the list of users permitted
to commit changes to the project.
A number of flags controlling the build and publishing process and possible read access protections.
A list of repositories to be created. This list defines what other repositories should be used, which architectures shall be built and build job scheduling parameters.
The following flags can be used to control the behavior of a package or project. Most of them can also be limited to specified repositories or architectures.
build
defines whether package sources should
get built. If enabled, it signals the scheduler to trigger server-side
builds based on events like source changes, changes of packages used in
the build environment or manual rebuild triggers. A local build via CLI
is possible independent of this flag. Default is enabled.
publish
can be used to enable or disable
publishing the build result as repository. This happens after an entire
repository has finished building for an architecture. A publish also gets
triggered when the publish flag is enabled after a repository finishes
the build. Default is enabled.
debuginfo
can be used to modify the build
process to create debuginfo data along with the package build for later
debugging purposes. Changing this flag does not trigger rebuilds, it just
affects the next build. Default is disabled.
useforbuild
is used to control if a built
result shall be copied to the build pool. This means it will get used for
other builds in their build environment. When this is disabled, the build
has no influence on builds of other packages using this repository. In
case a former build exists the old binaries will be used. Disabling this
flag also means that "wipe" commands to remove binary files will have no
effect on the build pool. Changing this flag does not trigger rebuilds,
it just affects the next build. Default is enabled.
access
flag can be used to hide an entire
project. This includes binaries and sources. It can only be used at
project creation time and can just be enabled (making it public again)
afterwards. This flag can only be used on projects. Default is enabled.
sourceaccess
flag can be used to hide the
sources, but still show the existence of a project or package. This also
includes debug packages in case the distribution is supporting this
correctly. This flag can only be used at package creation time. There is
no code yet which checks for possible references to this package. Default
is enabled.
downloadbinary
permission still exists like
before. However, unlike "access" and "sourceaccess" this is not a
security feature. It is just a convenience feature, which makes it
impossible to get the binaries via the API directly. But it is still
possible to get the binaries via build time in any case. Default is
enabled.
A project is configured in the project
/source/$PROJECT/_config
path. It can be edited in web
interface in the Project Config
tab or via one of the
following command lines
osc meta prjconf -e $PROJECT
osc co $PROJECT _project
This file contains information on how to set up a build environment.
The macro configuration is part of the build configuration in
/source/$PROJECT/_config
. It can be added at the end
after a Macros:
line.
An OBS Package is a sub-namespace below a project. It contains the specification of a single package build for all specified repositories.
The OBS interconnect is a mechanism to connect two OBS instances. All content, including sources and binary build results, will be available in the connecting instance. Unlike other methods the instances will also notify each other about changes.
In a DoD repository external software repositories can be configured which are used for dependency resolution and where packages will be downloaded at build time. A DoD repository has some main advantages in comparison to binary import projects:
less disk usage as only really required packages will be downloaded
automatic package updates when new upstream releases are available
simple to configure in project meta with no for shell access to repo servers
In download repotypes where package checksums can be verified (e.g.
susetags, rpmmd and deb), we recommend that you use a mirror server URL in
<download> in order to reduce traffic on the master server and
configure a <master> with an https
url and a
sslfinger
in order to avoid man in the middle attacks by
peer verification.
<project> <repository> <download> <master/> (optional) <pubkey/> (optional) </download> </repository> </project>
The bs_dodup daemon periodically checks for new metadata in remote repositories. This daemon can be enabled for startup with the command
systemctl enable obsdodup.service
and can be started with
systemctl start obsdodup.service
master
Subelement # The <master>
tag as shown in the rpmmd
example below is optional but strongly recommended for security reasons.
Verification is supported in the following repotypes
susetags
rpmmd
deb
This option could be defined by any valid URL (HTTP and HTTPS) to the
origin of the repository but it is strongly recommended to use
https
with a sslfingerprint
to
bs_dodup possibility to verify its peer in order to avoid man-in-the-middle
attacks. The download URL can be a mirror as we validate package checksums
found in repo data.
You can easily query the SSL fingerprint of a remote server with the following command:
openssl s_client -connect <host>:<port> < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout
pubkey
Subelement #The pubkey element contains one or more GPG public keys in order to verify repository information but not packages. For an example, look at the repotype "deb" documentation below.
Example:
<project name="My::SuSE::CD"> [...] <repository name="standard"> <download arch="x86_64" url="http://mirror.example.org/path/to/iso" repotype="susetags" /> <download arch="i586" url="http://mirror.example.org/path/to/iso" repotype="susetags" /> <arch>x86_64</arch> <arch>i586</arch> </repository> </project>
Example:
<project name="Fedora:Rawhide"> [...] <repository name="standard"> <download arch="x86_64" url="http://mirror.example.org/fedora/rawhide/x86_64/os" repotype="rpmmd"> <master url="https://master.example.org/whereever/fedora/rawhide/x86_64/os" sslfingerprint="sha256:0a64..0303"/> </download> <download arch="i586" url="http://mirror.example.org/fedora/rawhide/i386/os" repotype="rpmmd"> <master url="https://master.example.org/whereever/fedora/rawhide/i386/os" sslfingerprint="sha256:0a64..0303"/> </download> <arch>x86_64</arch> <arch>i586</arch> </repository> </project>
Apt supports two repository types, flat repositories and distribution repositories.
The download url syntax for them is:
<baseurl>/<distribution>/<components>
<flat_url>/.[/<components>]
You can specify multiple components separated by a comma.
An empty components string is parsed as "main".
Example:
<project name="Debian:8"> [...] <repository name="ga"> <download arch="x86_64" url="http://ftp.de.debian.org/debian/jessie/main" repotype="deb"> <pubkey> -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.12 (GNU/Linux) [...] </pubkey> </download> <download arch="i586" url="http://ftp.de.debian.org/debian/jessie/main" repotype="deb"> <pubkey> -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.12 (GNU/Linux) [...] </pubkey> </download> <arch>x86_64</arch> <arch>i586</arch> </repository> </project>
Be aware that there is currently no way to verify the origin of repository for Arch.
Example:
<project name="Arch:Core"> [...] <repository name="standard"> <download arch="x86_64" url="http://ftp5.gwdg.de/pub/linux/archlinux/core/os/x86_64" repotype="arch"/> <download arch="i586" url="http://ftp5.gwdg.de/pub/linux/archlinux/core/os/i686" repotype="arch"/> <arch>x86_64</arch> <arch>i586</arch> </repository> </project>
Example:
<project name="Mageia:5"> [...] <repository name="standard"> <download arch="x86_64" url="http://mirror.example.org/Mageia/distrib/5/x86_64/media/core/release" repotype="mdk"/> <download arch="i586" url="http://mirror.example.org/mirrors/Mageia/distrib/5/i586/media/core/release" repotype="mdk"/> <arch>x86_64</arch> <arch>i586</arch> </repository> </project>
This chapter makes some recommendations how upstream resources can be integrated into the build process. SCM stands for source control management. git, subversion or CVS are concrete implementations of an SCM. The OBS itself comes also with an own SCM, but this is only intended to manage the files needed for packaging. However, you can add references to external SCM systems. The source service system will mirror the sources and provide it to the build systems. OBS makes sure that you can access the sources of all builds also in the future, even when the upstream server delivers different or no content at all anymore. Using external SCM references has the following advantages:
It is documented where a source comes from and how to create the archive.
Working on the upstream sources can be done directly in local checkouts and changes can be tested via local builds before pushing to the SCM server.
The sources can be stored incrementally and need less storage on the server.
External references are defined in _service files. The file can look like this:
<services> <service name="obs_scm"> <param name="url">git://...</param> <param name="scm">git</param> </service> <service name="tar" mode="buildtime"/> <service name="recompress" mode="buildtime"> <param name="file">*.tar</param> <param name="compression">xz</param> </service> <service name="set_version" mode="buildtime" /> </services>
The services do the following:
obs_scm
: mirrors the source. It stores it as a
cpio archive, but for the build process this looks like a directory. It
also stores additional information from the metadata to a file with
obsinfo suffix.
tar
: creates a tar file from the directory
recompress
: applies a compression on the tar file
set_version
: reads the version from the obsinfo
file and adapts the build descriptions to it.
Note that only the first service (obs_scm
)
runs on the OBS server. The other services run during the build process.
They can also be replaced by any user by providing alternative
implementations of them, or by writing their own service from scratch.
Using osc build
in any package with such a
definition will do the same process locally. The only difference is that you
get a local subdirectory with the SCM content. You can go inside and work as
you are used to. Any changes inside will be used for your next local build,
whether they were pushed to the upstream server or not. However, you need to
push it upstream when you let the OBS server re-fetch the changes from
upstream. The only way out would be to set the
obs_scm
service to mode disabled and upload your
local archive.
The obs_scm
service allows you to export
files next to the archive. You can specify one or more files using the
extract
parameter. Use it for your build recipe
files.