systemd-nspawn is like the chroot command, but it is a chroot on steroids. See systemd-nspawn - ArchWiki ( and nspawn - Debian Wiki for introduction.


We can bootstrap a Debian machine using debootstrap, but also try mkosi.

For example, bootstrap a openSUSE image:

python3 -m pip install --user git+git://
sudo .local/bin/mkosi -d opensuse -t directory -p systemd-container --checksum --password password -o /var/lib/machines/opensuse-test



Although there is no document for systemd-nspawn, we can refer to How-to: Deploy RDMA accelerated Docker container over InfiniBand fabric.

Make sure these tools has the same version as host.

We only need to install userspace tools into nspawn container without updating firmware:

./mlnxofedinstall --user-space-only --without-fw-update

Edit .nspawn file

Edit .nspawn file of the container, which is located at /etc/systemd/nspawn/<machine-name>.nspawn. If such a file does not exist, create one.

Then, add following content



Also consider use host network by


Add DeviceAllow

Create a drop-in file use command

sudo systemctl edit systemd-nspawn@<machine-name>

with content of

DeviceAllow=/dev/infiniband/uverbs0 rwm
DeviceAllow=/dev/infiniband/uverbs1 rwm

Put all of devices you want to allow there.


Show status with ibstat. Test RDMA with perftest.

If you find tools like perftest does not work, it may releated to

  • Limit on memlock, see below for solution.

Disable memlock limit

IB tools may fail to allocate memory if memlock limit is too small. To show current memlock limit, use

sudo systemctl show systemd-nspawn@<machine-name> --property LimitMEMLOCK

To disable limit, use

sudo systemctl edit systemd-nspawn@<machine-name>

And add LimitMEMLOCK=infinity to [Service] section, then restart your container.


No color in terminal

See Arch wiki for "broken colors" problem.

Create file /etc/systemd/system/container-getty@.service.d/term.conf in container with following contents: