random notes

Turn podman containers into jails
Login

Turn podman containers into jails

Context

While hacking on forgejo extending LDAP support to match with some of the FreeBSD's cluster peculiar needs. I had to:

  1. make sure I have not broken the existing LDAP features.
  2. add tests to make sure the new feature I had will not be broken by someone else later by accident.

Upstream is extensively using docker as part of their CI in order to be able run test and in particular integration test.

As for the LDAP part they do use a preseeded docker image they inherited from the gitea project.

Thanks many people, on FreeBSD we can natively run docker images via podman including native linux docker images.

First naive attempt, direct podman

Let's just install podman and directly run the container

$ pkg install -y podman

Activate the linuxulator (linux compatibility layer for FreeBSD

$ service linux enable
$ service linux start

and run the instance I wanted to get:

$ podman run -t --os=linux --rmi docker.io/gitea/test-openldap

Everything seems to work, I can even see the prompt telling me:

starting slapd on port 389 and 636...
6630971b @(#) $OpenLDAP: slapd  (May 23 2018 04:25:19) $
	Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>

But running the Forgejo's LDAP integration tests shows me it does not seems to be able to find this LDAP instance.

Sockstat seems to confirm there is nothing listening locally on ports 389 and 636.

My approach has been too naive, I need to learn how network is implemented on podman for FreeBSD to actually be able to access this VMs.

As the only reason here for me to run podman was not to learn how to use podman on FreeBSD but actually be able to run that openldap container, I decided to take another road, let's directly use jails.

Native FreeBSD way

After all on FreeBSD podman uses jail under the hood, so why not just run a jail

When one run a podman container like explained above, in the end a jail with vnet support is actually being started as shown by the jls(8) command:

$ jls -n
...
devfs_ruleset=4 nodying enforce_statfs=1 host=new ip4=inherit ip6=inherit jid=2 linux=new name=vnet-b408f89e-ab11-e30e-df9c-92ac4b767492.ef4f990a8e867d16e09e121703a563a0366e8f6c90373dd14ffd7839d6ea8593 osreldate=1500018 osrelease=15.0-CURRENT parent=1 path=/var/db/containers/storage/zfs/graph/e14bedcac335a859113a833bb4ddca8e6631fbfa37e09bf2856d6ae063acbca8 persist securelevel=-1 sysvmsg=disable sysvsem=disable sysvshm=disable vnet=inherit zfs=new allow.chflags allow.noextattr allow.nomlock allow.nomount allow.mount.nodevfs allow.mount.nofdescfs allow.mount.nofusefs allow.mount.nolindebugfs allow.mount.nolinprocfs allow.mount.nolinsysfs allow.mount.nonullfs allow.mount.noprocfs allow.mount.notmpfs allow.mount.nozfs allow.nonfsd allow.noquotas allow.raw_sockets allow.noread_msgbuf allow.reserved_ports allow.set_hostname allow.nosocket_af allow.suser allow.nosysvipc allow.unprivileged_proc_debug children.cur=0 children.max=0 cpuset.id=4 host.domainname="" host.hostid=0 host.hostname=ef4f990a8e86 host.hostuuid=00000000-0000-0000-0000-000000000000 ip4.addr= ip4.saddrsel ip6.addr= ip6.saddrsel linux.osname=Linux linux.osrelease=5.15.0 linux.oss_version=198144 zfs.mount_snapshot=0

It means I can get a copy of the image which I could save somewhere else, and run it directly with the jail command line or a couple of config in jail.conf

Via the mount command I can see the podman actually put the continer into a zfs filesystem which is convenient for me.

$ mount | grep containers
zroot/ROOT/pkgbase/e14bedcac335a859113a833bb4ddca8e6631fbfa37e09bf2856d6ae063acbca8 on /var/db/containers/storage/zfs/graph/e14bedcac335a859113a833bb4ddca8e6631fbfa37e09bf2856d6ae063acbca8 (zfs, local, noatime, nfsv4acls)

let's snaphot a copy if of this filesystem and save it somewhere else.

$ zfs snapshot zroot/ROOT/pkgbase/e14bedcac335a859113a833bb4ddca8e6631fbfa37e09bf2856d6ae063acbca8@save
$ zfs send zroot/ROOT/pkgbase/e14bedcac335a859113a833bb4ddca8e6631fbfa37e09bf2856d6ae063acbca8@save | zfs receive zroot/jails/gitea-openldap
$ zfs set mountpoint=/jails/gitea-openldap zroot/jails/gitea-openldap

now I can run a jail command directly.

$ jail -c -n gitea-openldap path=/jails/gitea-openldap mount.devfs mount.fdescfs ip4=inherit ip6=inherit exec.start="env LDAP_DEBUG_LEVEL=none /bin/bash /run.sh"

As I want to be able to execute the jail each time I want to run forgejo LDAP integration tests, I will use directly jails.conf

Here is my jail.conf:

mount.devfs;
mount.fdescfs;
path = /jails/$name;

gitea-openldap {
    ip4=inherit;
    ip6=inherit;
    exec.start += "env LDAP_DEBUG_LEVEL=none /bin/bash /run.sh";
}

Now I can start the jail

$ jail -c gitea-openldap
gitea-openldap: created
starting slapd on port 389 and 636...
66309b39 @(#) $OpenLDAP: slapd  (May 23 2018 04:25:19) $
	Debian OpenLDAP Maintainers <pkg-openldap-devel@lists.alioth.debian.org>
66309b39 slapd starting

I can see from the prompt that it seems to be working as much as the podman version, but this time the slapd binary is really listening on the socket:

$ sockstat | grep slapd
101      slapd      10295 6   tcp4   *:389                 *:*
101      slapd      10295 7   stream /var/run/slapd/ldapi
101      slapd      10295 8   tcp4   *:636                 *:*

And running the Forgejo's testsuite confirms it can find the LDAP setup just fine.

Conclusion

I would probably have reached the same directly running podman, If I had taken the time to actually read some documentation which I haven't done to configure podman network correctly, but I know jails pretty well and it was easier for me to just turn this podman container into a simple jail.