I am going to show here how it is possible to build an application
for Snappy and Flatpak at the same time.
This is not about converting Flatpak to run on Snappy.
Motivation
Flatpak and Snappy
are both package management systems where applications come with their
userland runtime. This is allows those applications to run on any
distribution.
With Snappy and Flatpak, developers can finally distribute their
applications without a need to build for multiple distributions (and
multiple versions of them).
However both Snappy and Flatpak both come with their development
tools, respectively Snapcraft
and FlatpakBuilder. The choice of the
developer’s tool affects what the user is going to use. This forces
users to have both Snappy and Flatpak installed.
But since the applications come with their own userland runtime, the same
build should be able to be packaged for both Snappy and Flatpak.
Base runtimes
Applications runtimes are not fully built from scratch. A base system
is needed to build the rest on top of it.
Snappy applications run on Ubuntu
Core. Ubuntu Core is very
minimal. When building the application, dependencies can be built from source,
or imported from prebuilt packages.
Flatpak applications run on Freedesktop
SDK. Freedesktop SDK contains the
minimum runtime to run a desktop application. Other runtimes are also
available: Gnome
or KDE. Those are built
on top of Freedesktop SDK. Other dependencies can be built from
sources.
To be able to run an application built on both Flatpak and Snappy, you
need to be able to use the same base runtime on both. As a developer
of Freedesktop SDK, I have been looking shipping it to
Snappy. However, it should be possible to do the same work on Ubuntu
Core to run on Flatpak.
BuildStream
Since version
18.08,
Freedesktop SDK is being built with
BuildStream rather than
FlatpakBuilder. BuildStream is also used to build the Gnome SDK.
BuildStream is independent from Flatpak. There is nothing Flatpak
specific about it. We use plugins in order to generate Flatpak
packages.
For that reason we can also use BuildStream to build bootable images,
OCI/Docker images, and Snappy images.
Demo
As a demonstration, I made a build of Firefox.
The source code for the build is available at
https://gitlab.com/valentindavid/firefox-buildstream.
Structure
The main file to build Firefox itself is elements/firefox.bst
. We also need
to build dependencies and package the application. Here is a description
of the directory structure of the project.
elements/freedesktop-sdk.bst
Describe which version of Freedesktop Sdk we
depend on.
elements/
Contains .bst
files describing how to compile firefox and dependencies.
elements/snap/
Contains element files describing how to package the application for Snappy.
elements/flatpak/
Contains element files describing how to package the application for Flatpak.
files/
contains extra source files.
patches/
contains patches referenced by elements.
plugins/
contains extra BuildStream plugins.
app.yml
is the generic BuildStream configuration for application.
project.conf
is the BuildStream configuration for this application.
Note that many files will be common to every application and will be
either merged to Freedesktop SDK or
bst-external (the
BuildStream plugin collection).
Prerequisites
You will need to install BuildStream 1.2 and bst-external. One way
to do it is:
pip3 install --user git+https://gitlab.com/BuildStream/buildstream.git@bst-1.2
pip3 install --user git+https://gitlab.com/BuildStream/bst-external.git@master
There are other ways to install it. Please read the BuildStream’s
installation page for more
information.
Building and installing Freedesktop SDK as snap
git clone https://gitlab.com/freedesktop-sdk/freedesktop-sdk.git freedesktop-sdk-snap --branch valentindavid/snap
cd freedesktop-sdk-snap
make export-snap
snap install --dangerous snap/platform.snap
Do not worry, this will download already built artifacts from the
Freedesktop SDK cache server.
Building and testing Firefox
First we will build Firefox itself.
git clone https://gitlab.com/valentindavid/firefox-buildstream.git
cd firefox-buildstream
bst build firefox.bst
This will take some time. Some dependencies not in Freedesktop SDK as
well as Firefox itself will need to build. Of course applications
can use artifact cache servers to speed up build time.
Now we can run it directly with BuildStream to test it.
bst shell firefox.bst -- firefox --new-instance
The container will run with an empty fresh home directory. So it will not
affect your existing Firefox profiles.
And it should run:
In order to do some debugging, you can create a new element that depends on
firefox.bst
as well as your required debugging tool from Freedestkop SDK,
for example gdb.bst
. After building it, just bst shell
that element.
kind: stack
depends:
- filename: base/gdb.bst
junction: freedesktop.bst
- filename: firefox.bst
Building the snap package
To build, install and run Snappy packages:
bst build snap/image.bst
bst checkout snap/image.bst snap/
snap install --dangerous snap/firefox.snap
/snap/bin/firefox --new-instance
Elements related to making a snap elements are in directory
elements/snap
.
Element
snap/firefox.bst
creates a directory ready for Snappy with the meta/snap.yaml
file. We do not provide a static file for meta/snap.yaml
as we may
use BuildStream variables to be expanded.
Element
snap/image.bst
takes this directory and builds a proper .snap
file using
mksquashfs
.
Building the flatpak package
To build, install and run Flatpak packages:
bst build flatpak/repo.bst
bst checkout flatpak/repo.bst repo/
flatpak remote-add --no-gpg-verify local-firefox repo/
flatpak install local-firefox org.mozilla.Firefox
It is possible to export a bundle. Please see the
documentation.
Elements related to building the Flatpak are in
elements/flatpak
.
The main element is
flatpak/org.mozilla.Firefox.bst
which builds the built directory for flatpak.
This element is then taken in
flatpak/repo.bst
which creates a flatpak repository using flatpak build-export
(It is
Flatpak, not FlatpakBuilder).
Application path
Flatpak applications expect to be mounted at /app
. However, this is not
the case with Snappy where it is available at $SNAP
. Flatpak runs
ldconfig
on application with /app/lib
activate. For Snappy, wrapper
scripts are used to set LD_LIBRARY_PATH
to $SNAP/lib
. For loading library
that works.
When it comes to other data to load however, the application needs to be
aware of the path. The wrapper may define more paths using environment
variables. But the application itself needs to be able to read them.
That might mean modifying it, and patching some dependencies.
Current status
At the time of writing the work is a merge
request
being reviewed on the Freedesktop SDK.
The CI is ready to publish the Freedesktop SDK runtime as soon as we have a new
release. There will also be edge builds automatically pushed at each
new merge. The name is reserved to be
https://snapcraft.io/freedesktop-sdk-runtime-18-08.
Because the Freedesktop SDK runtime is of “base” type, it will require
manual review from Snapcraft. I am waiting for finished review from
Freedesktop SDK before requesting one on Snapcraft’s store.
Challenges and work ahead
The image of Freedesktop SDK for Snappy is soon ready to publish. But
there is still work ahead. I will give here a list of things I have in
mind that need to be done.
snap-confine
and /etc
snap-confine
mounts /etc
from host. And it remounts few exceptions from
the runtime image. Flatpak does the opposite by it takes most of things from
/etc
from the image but has few exceptions taken from host or generated.
Freedesktop SDK has been made for Flatpak and for example, font
configuration in /etc/fonts
is expected to be from the image to match
the fonts we provide.
It would be possible to selectively tell packages to use /etc
or
/usr/etc
depending whether they want to read respectively host or
image configuration. But this will be some work. We will probably fix
it case by case when bugs are filed.
For the Firefox demo, we worked around the font issue by copying
/etc/fonts
in the application image, and defined FONTCONFIG_PATH
in the wrapper script.
Forced to provide Dash
snap-confine
uses apparmor
. snap-confine
will run /bin/sh
and
expects it to be Dash. That forces use to provide Dash as /bin/sh
on
the Snap version of the Freedesktop SDK. We would like of course that
snap-confine
lets us use what Bourne Shell we want.
BuildStream boiler-plate
Because BuildStream is very generic it can be relatively verbose.
However applications will have mostly the same configuration and
plugins.
We have added support for configuration import through external
projects for this purpose. So soon we should see application configuration
provided directly by the Freedesktop SDK.
Nvidia proprietary drivers
I have not dealt with Nvidia proprietary drivers yet. My understanding
is that snap-confine
mounts them. Freedesktop SDK already dispatches
with libglvnd, the Vulkan loader, and OpenCL loader. Drivers in
flatpak are provided through extensions. So I am guessing it will need
some tweaks for the paths.
KDE and Gnome SDKs to Snappy
The Gnome SDK is already using BuildStream, so once we have a relatively
stable Snap version of Freedesktop SDK, we will look at providing this
GNOME Build Meta.
KDE SDK is still built using FlatpakBuilder. So it would need to be
rewritten using BuildStream.
Aknowledgements
This work has been sponsored by Codethink Ltd.