• Bug#1079329: closed by Debian FTP Masters

    From Helmut Grohne@21:1/5 to Debian Bug Tracking System on Wed Apr 2 12:10:02 2025
    Control: reopen -1

    On Sat, Mar 29, 2025 at 01:39:02AM +0000, Debian Bug Tracking System wrote:
    It has been closed by Debian FTP Masters <ftpmaster@ftp-master.debian.org> (reply to Luca Boccassi <bluca@debian.org>).

    I'm saddened that rather than addressing the root cause, you declare incompatibility with other components that happen to expose the faulty behavior.

    The actual problem resides in systemctl switch-root (or rather the implementation of the resulting ipc request). dracut is just one
    initramfs generator that happens to employ this functionality, but it is
    not the package causing the problem. I argue that your declared conflict
    is misdirected.

    We may easily reproduce the issue using mkosi. In order to make the
    exercise simple to reproduce locally, I demonstrate it inside a minimal
    virtual machine created using debvm.

    debvm-create -z 10G -- --architecture=arm64 --include=mkosi,firmware-linux-free
    debvm-run

    Note that mkosi-initrd is unhappy if /usr/lib/firmware does not exist
    and hence I add firmware-linux-free to populate it with something. That
    minor issue may be worth fixing in mkosi as well.

    Once booted, we may use mkosi to create a new initrd.

    mkosi-initrd -O /boot/ -o initrd.img-$(uname -r)

    Since debvm cannot notice initrd updates, the machine must be powered
    off and started again rather than rebooted to experience the problem.
    Once booted, /lib64 points to usr/lib.

    I am reproducing the problem with mkosi now, but again it is not mkosi
    which is at fault here. I ask you to refrain from declaring a conflict
    with it as well.

    Helmut

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luca Boccassi@21:1/5 to All on Wed Apr 2 13:20:02 2025
    On Wed, 2 Apr 2025 12:05:17 +0200 Helmut Grohne <helmut@subdivi.de>
    wrote:
    Control: reopen -1

    On Sat, Mar 29, 2025 at 01:39:02AM +0000, Debian Bug Tracking System
    wrote:
    It has been closed by Debian FTP Masters
    <ftpmaster@ftp-master.debian.org> (reply to Luca Boccassi
    <bluca@debian.org>).

    I'm saddened that rather than addressing the root cause, you declare incompatibility with other components that happen to expose the
    faulty
    behavior.

    Sorry, but this is not a supported combination, and the intention was
    to make that explicit, in the simplest way possible (ie, so that's it's
    noticed at build time already).
    The default initrd generator in Debian is initramfs-tools, that's fully supported.

    If you wish to experiment with dracut, that's great! You can use it
    with many init systems (or no init system at all). The particular
    combination of arm64+dracut+systemd is known bad and unsupported, and
    any one of those 3 factors can be swapped with something else to get a
    working solution.

    You can expect support for defaults, but requiring volunteers to go out
    of their way and do a huge amount of extra work to support non-
    standard, non-default corner-case combinations is not ok, sorry. One
    can ask of course, but one can also say "sorry, but -ENOTIME".

    Do you have other suggestions on how to best encode this? If not, I
    could perhaps add an AssertArchitecture=!arm64 to the initrd-only
    switchroot unit? That is a runtime setting so less ideal as the
    feedback is not immediate, but it would allow tinkerers to break the
    warranty seal, mask it and do strange things on the other hand, which
    is nicer.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luca Boccassi@21:1/5 to Antonio Terceiro on Wed Apr 2 20:20:02 2025
    On Wed, 2 Apr 2025 at 18:47, Antonio Terceiro <terceiro@debian.org> wrote:

    On Wed, Apr 02, 2025 at 12:09:13PM +0100, Luca Boccassi wrote:
    On Wed, 2 Apr 2025 12:05:17 +0200 Helmut Grohne <helmut@subdivi.de>
    wrote:
    Control: reopen -1

    On Sat, Mar 29, 2025 at 01:39:02AM +0000, Debian Bug Tracking System
    wrote:
    It has been closed by Debian FTP Masters
    <ftpmaster@ftp-master.debian.org> (reply to Luca Boccassi <bluca@debian.org>).

    I'm saddened that rather than addressing the root cause, you declare incompatibility with other components that happen to expose the
    faulty
    behavior.

    Sorry, but this is not a supported combination, and the intention was
    to make that explicit, in the simplest way possible (ie, so that's it's noticed at build time already).
    The default initrd generator in Debian is initramfs-tools, that's fully supported.

    If you wish to experiment with dracut, that's great! You can use it
    with many init systems (or no init system at all). The particular combination of arm64+dracut+systemd is known bad and unsupported, and
    any one of those 3 factors can be swapped with something else to get a working solution.

    I think you are talking past each other.

    This issue is not related to dracut at all. The issue is that when you
    start a systemd-nspwn container on arm64, /lib64 is created while it
    should not be. I assume that it would do the same on any system that is booted from a rootfs that contains only /usr.

    A simple reproducer is this:

    # ls -l /var/lib/lxc/autopkgtest-testing/rootfs/ | grep lib64
    # systemd-nspawn -D /var/lib/lxc/autopkgtest-testing/rootfs/ --console=pipe -- ls -l / | grep lib64
    lrwxrwxrwx 1 0 0 7 Apr 2 14:29 lib64 -> usr/lib

    (I used a lxc container rootfs that I have here, but the behavior will
    likely be the same with any rootfs).

    AIUI, /lib64 should no longer exist on Debian arm64 systems, or if it
    exists, it should point exactly at usr/lib64 (which no longer exists
    AFAICT). systemd, instead, is looking for the arm64 linker and
    symlinking /lib64 exactly at where it is found.

    This seems to be implemented in src/shared/base-filesystem.c. Now,
    whether or not to create /lib64, and where it should point to, depends
    on what OS is in the rootfs. I'm not sure where to go from here.

    Yes, essentially on arm64 the expectations of Debian/base-files and
    systemd have diverged irremediably on this aspect, and should not be
    mixed and matched anymore as it's way too costly to support, and that
    includes nspawn+arm64 and dracut+systemd+arm64, and I've taken steps
    to try and make sure it doesn't happen (as per your ruling: no
    incompatible symlink shall be created), even though there are some
    small gaps evidently, but I believe they can be filled.

    The standard, default and common use cases such as booting with initramfs-tools, or with vastly more popular container managers like
    Docker or LXD or Podman are not affected and continue to work
    correctly, to be clear.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Helmut Grohne@21:1/5 to Luca Boccassi on Fri Apr 4 12:10:01 2025
    Hi Luca,

    On Thu, Apr 03, 2025 at 08:29:03PM +0000, Luca Boccassi wrote:
    On Wed, 2 Apr 2025 at 12:09, Luca Boccassi <bluca@debian.org> wrote:
    Do you have other suggestions on how to best encode this? If not, I
    could perhaps add an AssertArchitecture=!arm64 to the initrd-only switchroot unit? That is a runtime setting so less ideal as the
    feedback is not immediate, but it would allow tinkerers to break the warranty seal, mask it and do strange things on the other hand, which
    is nicer.

    Gentle ping on this?

    Thanks for your answer to my upstream followup at https://github.com/systemd/systemd/pull/34804#issuecomment-2777164830.
    I think this is the most promising way forward. Unfortunately, I will
    not be able to work on it today. I prefer providing a working and tested
    PR than waste your time by taking several iterations.

    In the mean time, I would like to remind you that a decision has been
    reached in accordance with Debian's procedures, and given the 3:1
    ruling, it needs to be implemented despite upstream choices and local
    packaging policies. You expressed concern with the patch in time, but clarifying questions from CTTE were left unanswered and therefore your
    concern did not tip the balance in reaching the decision. The result of
    that decision was made with the information the CTTE had at the time and
    by selecting among available solutions rather than doing design work.

    I am looking forward to reaching a consensual solution on the next
    upstream PR and swapping it out then.

    Helmut

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Luca Boccassi@21:1/5 to All on Mon Apr 7 17:10:02 2025
    On Fri, 4 Apr 2025 11:59:29 +0200 Helmut Grohne <helmut@subdivi.de>
    wrote:
    I am looking forward to reaching a consensual solution on the next
    upstream PR and swapping it out then.

    As agreed privately: the backport of the new patch is now queued in
    Salsa for the next upload, waiting for the current version (with the
    fix for #1098914) to migrate to testing first, which should happen
    tomorrow evening or so.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)