a527bb251a
We offer full cgroup delegation to our sandbox now, required for running containers inside the sandbox. To run systemd-nspawn or containers managers inside the sandbox, there is a need for one extra ingredient now: control over your own cgroup subtree inside the sandbox. If, in addition, you need multiple UIDs, for e.g. rootless usecases, you need to run with the `uid-range` system feature. Therefore, when the daemon or Nix runs under the right condition, e.g. systemd-style delegation of the cgroup subtree while placing the nix-daemon in a supervisor sub-cgroup, we create a new sub-cgroup for each build based on the build UID and delegate that sub-cgroup to the builder's process. Additionally, `uid-range` always request the `cgroups` feature now, as `uid-range` builds would probably always benefit from having cgroups delegated, but the converse is not true. Inspired from https://github.com/NixOS/nix/pull/11412 with a different design that does not use function-local statics to derive the root cgroup. Co-authored-by: Linus Heckemann <git@sphalerite.org> Co-authored-by: Parker Hoyes <contact@parkerhoyes.com> Change-Id: Ic8947c5adaf4b5bbd153386e05fad65a935274fa Signed-off-by: Raito Bezarius <raito@lix.systems>
2.5 KiB
synopsis, issues, cls, category, credits
| synopsis | issues | cls | category | credits | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| New cgroup delegation model |
|
|
Breaking Changes |
|
Builds using cgroups (i.e. use-cgroups = true and the experimental feature
cgroups) now always delegate a cgroup tree to the sandbox.
Compared to the original C++ Nix project, our delegation includes the
subtree_control file as well, which means that the sandbox can disable
certain controllers in its own cgroup tree.
This is a breaking change because this requires the Nix daemon to run with an already delegated cgroup tree by the service manager.
How to setup the cgroup tree with systemd?
systemd offers knobs to perform the required setup using:
[Unit]
Delegate=yes
DelegateSubtree=supervisor
These directives are now included in our systemd packaging.
What about using Nix as root without connecting to the daemon?
Builds run as root without connecting to the daemon relying on the cgroup
feature are now broken, i.e.
# nix-build --use-cgroups --sandbox ... # will not work
Consider doing instead:
# systemd-run --same-dir --wait -p Delegate=yes -p DelegateSubgroup=supervisor nix-build --use-cgroups ...
If you need to disable cgroups temporarily, remember that you can do
NIX_CONF='include /etc/nix/nix.conf\nuse-cgroups = false' nix-build ... or
nix-build --no-use-cgroups ....
What about other service managers than systemd?
systemd has a documentation on how to handle cgroup delegation from service management perspective.
If your service manager adheres to systemd semantics, e.g. writing an extended
attribute user.delegate=1 on the delegated cgroup tree directory and moving
the nix-daemon process inside a cgroup tree to respect the inner process
rule, then, the feature will work as well.
Why is the cgroup feature still experimental?
While the cgroup feature unlocks many use cases, its behavior and integration (e.g. user experience), especially at scale on build farms or in multi-tenant environments, are not yet fully matured. There’s also potential for deeper systemd integration (e.g. using slices and scopes) that has not been fully explored.
To avoid locking in an unstable interface, we’re keeping the experimental flag until we have validated the feature across a broader range of scenarios, including but not limited to:
- Nix as root
- Hydra-style build farms
- Forgejo CI runners
- Shared remote builders