XPost: linux.debian.devel.testing

Package: upgrade-reports
Severity: important
X-Debbugs-Cc: martin-eric.racine@iki.fi

After upgrading an amd64 host from Bookworm to Trixie (2025-04-05 EEST), on reboot, the host gets a kernel panic with:

initramfs unpacking failed invalid magic at start of compressed archive

The partition table:

sda1 vfat A28F-40DC
sda2 btrfs 960a40df-165b-408e-b47a-69df71aa7300
sda3 swap 02909177-67bb-4097-b1d8-8d18b07b521f

Booting from external media, I chroot myself in with:

sudo mount -o subvol=@rootfs /dev/sda2 /mnt/
sudo mount /dev/sda1 /mnt/boot/efi/
sudo mount --bind /proc/ /mnt/proc/
sudo mount --bind /dev/ /mnt/dev/
sudo mount --bind /dev/pts/ /mnt/dev/pts/
sudo mount --bind /sys/ /mnt/sys/

dpkg -l | grep linux-image
ii linux-image-6.12.20-amd64 6.12.20-1 amd64 Linux 6.12 for 64-bit PCs (signed)
ii linux-image-amd64 6.12.20-1 amd64 Linux for 64-bit PCs (meta-package)

file /boot/initrd.img-6.12.20-amd64
/boot/initrd.img-6.12.20-amd64: ASCII cpio archive (SVR4 with no CRC)

Relevant /etc/default/grub variables:

GRUB_CMDLINE_LINUX_DEFAULT="quiet loglevel=3 splash" GRUB_CMDLINE_LINUX="panic=15 enable_mtrr_cleanup=1 intel_iommu=off mds=full"

The resulting /boot/grub/grub.cfg:
---------------------------------
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
set have_grubenv=true
load_env
fi
if [ "${next_entry}" ] ; then
set default="${next_entry}"
set next_entry=
save_env next_entry
set boot_once=true
else
set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
menuentry_id_option="--id"
else
menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
set saved_entry="${prev_saved_entry}"
save_env saved_entry
set prev_saved_entry=
save_env prev_saved_entry
set boot_once=true
fi

function savedefault {
if [ -z "${boot_once}" ]; then
saved_entry="${chosen}"
save_env saved_entry
fi
}
function load_video {
if [ x$feature_all_video_module = xy ]; then
insmod all_video
else
insmod efi_gop
insmod efi_uga
insmod ieee1275_fb
insmod vbe
insmod vga
insmod video_bochs
insmod video_cirrus
fi
}

if [ x$feature_default_font_path = xy ] ; then
font=unicode
else
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300 fi
font="/@rootfs/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
set gfxmode=auto
load_video
insmod gfxterm
set locale_dir=$prefix/locale
set lang=fi_FI
insmod gettext
fi
terminal_output gfxterm
if [ "${recordfail}" = 1 ] ; then
set timeout=30
else
if [ x$feature_timeout_style = xy ] ; then
set timeout_style=menu
set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
set timeout=5
fi
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/05_debian_theme ###
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300 fi
insmod png
if background_image /@rootfs/usr/share/desktop-base/emerald-theme/grub/grub-16x9.png; then
set color_normal=white/black
set color_highlight=black/white
else
set menu_color_normal=cyan/blue
set menu_color_highlight=white/blue
fi
### END /etc/grub.d/05_debian_theme ###

### BEGIN /etc/grub.d/10_linux ###
function gfxmode {
set gfxpayload="${1}"
}
set linux_gfx_mode=keep
export linux_gfx_mode
menuentry 'Debian GNU/Linux GNU/Linux, with Linux 6.12.20-amd64' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.12.20-amd64-advanced-960a40df-165b-408e-b47a-69df71aa7300' {
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
echo 'Loading Linux 6.12.20-amd64 ...'
linux /@rootfs/boot/vmlinuz-6.12.20-amd64 root=UUID=960a40df-165b-408e-b47a-69df71aa7300 ro rootflags=subvol=@rootfs panic=15 enable_mtrr_cleanup=1 intel_iommu=off mds=full quiet loglevel=3 splash
echo 'Loading initial ramdisk ...'
initrd /@rootfs/boot/initrd.img-6.12.20-amd64
}
menuentry 'Debian GNU/Linux GNU/Linux, with Linux 6.12.20-amd64 (recovery mode)' --class debian --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.12.20-amd64-recovery-960a40df-165b-408e-b47a-69df71aa7300' {
load_video
gfxmode $linux_gfx_mode
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
echo 'Loading Linux 6.12.20-amd64 ...'
linux /@rootfs/boot/vmlinuz-6.12.20-amd64 root=UUID=960a40df-165b-408e-b47a-69df71aa7300 ro single dis_ucode_ldr rootflags=subvol=@rootfs panic=15 enable_mtrr_cleanup=1 intel_iommu=off mds=full
echo 'Loading initial ramdisk ...'
initrd /@rootfs/boot/initrd.img-6.12.20-amd64
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_ipxe ###
menuentry "Network boot (iPXE)" --users "" --class network --id ipxe {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
if [ "$grub_platform" = "efi" ]; then
chainloader /@rootfs/boot/ipxe.efi
else
linux16 /@rootfs/boot/ipxe.lkrn
# If the user provided an iPXE script, load it
if [ -f /@rootfs/boot/ipxe.ipxe ]; then
initrd16 /@rootfs/boot/ipxe.ipxe
fi
fi
}
### END /etc/grub.d/20_ipxe ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/20_memtest86+ ###
if [ "$grub_platform" = efi -a "$grub_cpu" = x86_64 ]; then
menuentry "Memory test (memtest86+x64.efi)" --class memtest $menuentry_id_option "memtest86+" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+x64.efi
}
menuentry "Memory test (memtest86+x64.efi, serial console)" --class memtest $menuentry_id_option "memtest86+-serial" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+x64.efi console=ttyS0,115200
}
fi
if [ "$grub_platform" = efi -a "$grub_cpu" = i386 ]; then
menuentry "Memory test (memtest86+ia32.efi)" --class memtest $menuentry_id_option "memtest86+" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+ia32.efi
}
menuentry "Memory test (memtest86+ia32.efi, serial console)" --class memtest $menuentry_id_option "memtest86+-serial" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+ia32.efi console=ttyS0,115200
}
fi
if [ "$grub_platform" = pc ]; then if cpuid -l ; then
menuentry "Memory test (memtest86+x64.bin)" --class memtest $menuentry_id_option "memtest86+" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+x64.bin
}
menuentry "Memory test (memtest86+x64.bin, serial console)" --class memtest $menuentry_id_option "memtest86+-serial" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+x64.bin console=ttyS0,115200
}
fi ; fi
if [ "$grub_platform" = pc ]; then if ! cpuid -l ; then
menuentry "Memory test (memtest86+ia32.bin)" --class memtest $menuentry_id_option "memtest86+" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+ia32.bin
}
menuentry "Memory test (memtest86+ia32.bin, serial console)" --class memtest $menuentry_id_option "memtest86+-serial" {
insmod part_gpt
insmod btrfs
set root='hd0,gpt2'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt2 --hint-efi=hd0,gpt2 --hint-baremetal=ahci0,gpt2 960a40df-165b-408e-b47a-69df71aa7300
else
search --no-floppy --fs-uuid --set=root 960a40df-165b-408e-b47a-69df71aa7300
fi
linux /@rootfs/boot/memtest86+ia32.bin console=ttyS0,115200
}
fi ; fi
### END /etc/grub.d/20_memtest86+ ###

### BEGIN /etc/grub.d/25_bli ###
if [ "$grub_platform" = "efi" ]; then
insmod bli
fi
### END /etc/grub.d/25_bli ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
if [ "$grub_platform" = "efi" ]; then
fwsetup --is-supported
if [ "$?" = 0 ]; then
menuentry 'UEFI Firmware Settings' $menuentry_id_option 'uefi-firmware' {
fwsetup
}
fi
fi
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/35_fwupd ###
### END /etc/grub.d/35_fwupd ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f ${config_directory}/custom.cfg ]; then
source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then
source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###
---------------------------------

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

XPost: linux.debian.devel.testing

On Sun, 6 Apr 2025 10:00:45 +0300
=?UTF-8?Q?Martin=2D=C3=89ric_Racine?= <martin-eric.racine@iki.fi>
wrote:

On Sat, 05 Apr 2025 22:56:46 +0300 =?utf-8?q?Martin-=C3=89ric_Racine?= <martin-eric.racine@iki.fi> wrote:

Package: upgrade-reports
Severity: important
X-Debbugs-Cc: martin-eric.racine@iki.fi

After upgrading an amd64 host from Bookworm to Trixie (2025-04-05 EEST), on reboot, the host gets a kernel panic with:

initramfs unpacking failed invalid magic at start of compressed archive

I just checked whether I can rescue the host using the Trixie mini.iso
on a USB stick:

Debian GNU/Linux 13 (trixie) amd64 - netboot mini.iso 20241227

I get the same kernel panic as soon as GRUB loads the kernel.

If I try with the Bookworm mini.iso, the kernel boots just fine and
gets me to the rescue menu.

Debian GNU/Linux 12 (bookworm) amd64 - netboot mini.iso 20230607+deb12u9

However, using the Bookworm rescue mode is a PITA, because it doesn't
know how to mount brtfs sub-volumes (not even with Debian's default
@rootfs) so even just chroot'ing myself in using the above recipe
requires a LOT of command typing inside the rescue mode's shell.

As a further test, I've tried booting the following Trixie USB stick
in BIOS mode via the rescue menu:

Debian GNU/Linux trixie-DI-alpha1 "Trixie" - Official Alpha amd64
NETINST with firmware 20241230-11:26

This succesfully gets me to the rescue process and allows me to chroot
into the host after typing the long commandline series above.

Basically, a/b testing shows that Trixie kernels per-se work fine, but something fishy is going on with the grub-efi-amd64 in Trixie. The
same host was originally installed with a Bookworm mini.iso on USB and
booted fine via EFI, so the problem really does seem to be the
grub-efi-amd64 in Trixie.

Martin-Éric

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 6 Apr 2025 13:02:29 +0300
=?UTF-8?Q?Martin=2D=C3=89ric_Racine?= <martin-eric.racine@iki.fi>
wrote:

On Sun, 6 Apr 2025 10:00:45 +0300
=?UTF-8?Q?Martin=2D=C3=89ric_Racine?= <martin-eric.racine@iki.fi>
wrote:

On Sat, 05 Apr 2025 22:56:46 +0300 =?utf-8?q?Martin-=C3=89ric_Racine?= <martin-eric.racine@iki.fi> wrote:

Package: upgrade-reports
Severity: important
X-Debbugs-Cc: martin-eric.racine@iki.fi

After upgrading an amd64 host from Bookworm to Trixie (2025-04-05 EEST), on reboot, the host gets a kernel panic with:

initramfs unpacking failed invalid magic at start of compressed archive

I just checked whether I can rescue the host using the Trixie mini.iso
on a USB stick:

Debian GNU/Linux 13 (trixie) amd64 - netboot mini.iso 20241227

I get the same kernel panic as soon as GRUB loads the kernel.

If I try with the Bookworm mini.iso, the kernel boots just fine and
gets me to the rescue menu.

Debian GNU/Linux 12 (bookworm) amd64 - netboot mini.iso 20230607+deb12u9

However, using the Bookworm rescue mode is a PITA, because it doesn't
know how to mount brtfs sub-volumes (not even with Debian's default @rootfs) so even just chroot'ing myself in using the above recipe
requires a LOT of command typing inside the rescue mode's shell.

As a further test, I've tried booting the following Trixie USB stick
in BIOS mode via the rescue menu:

Debian GNU/Linux trixie-DI-alpha1 "Trixie" - Official Alpha amd64
NETINST with firmware 20241230-11:26

This succesfully gets me to the rescue process and allows me to chroot
into the host after typing the long commandline series above.

Basically, a/b testing shows that Trixie kernels per-se work fine, but something fishy is going on with the grub-efi-amd64 in Trixie. The
same host was originally installed with a Bookworm mini.iso on USB and
booted fine via EFI, so the problem really does seem to be the
grub-efi-amd64 in Trixie.

Given the above, reassigned to bin:grub-efi-amd64.

Martin-Éric

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 16/04/2025 at 13:38, Martin-Éric Racine wrote:

As the attached screenshot shows, 'fuseblk' seemingly doesn't know
anything about the 'subvol' option for btrfs, which is what causes the failure to mount root. As to whether this implies a GRUB or Linux
module, I wouldn't know.

The failure has nothing to do with fuse.
I guess the kernel tries to mount the root filesystem with fuseblk as a
last resort *after* the initramfs unpacking failure because it is the
only built-in driver, all other block device/filesystem drivers being
modules in the initramfs.

If you start grub shell and load the kernel and initramfs by hand, does
it show an error when loading the initramfs ?

ls # show drives and partitions
set root=hd0,gpt2 # adjust with actual root partition
linux /@rootfs/boot/vmlinuz-6.12.20-amd64 # load kernel
initrd /@rootfs/boot/initrd.img-6.12.20-amd64 # load initramfs

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ke 16.4.2025 klo 15.41 Pascal Hambourg (pascal@plouf.fr.eu.org) kirjoitti:

On 16/04/2025 at 13:38, Martin-Éric Racine wrote:

As the attached screenshot shows, 'fuseblk' seemingly doesn't know
anything about the 'subvol' option for btrfs, which is what causes the failure to mount root. As to whether this implies a GRUB or Linux
module, I wouldn't know.

The failure has nothing to do with fuse.
I guess the kernel tries to mount the root filesystem with fuseblk as a
last resort *after* the initramfs unpacking failure because it is the
only built-in driver, all other block device/filesystem drivers being
modules in the initramfs.

If you start grub shell and load the kernel and initramfs by hand, does
it show an error when loading the initramfs ?

ls # show drives and partitions
set root=hd0,gpt2 # adjust with actual root partition
linux /@rootfs/boot/vmlinuz-6.12.20-amd64 # load kernel
initrd /@rootfs/boot/initrd.img-6.12.20-amd64 # load initramfs

No error.

Martin-Éric

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 17/04/2025 at 06:01, Martin-Éric Racine wrote:

ke 16.4.2025 klo 20.40 Pascal Hambourg (pascal@plouf.fr.eu.org) kirjoitti:

There may be some silent data corruption when grub reads the initramfs.
Can you test with the initramfs on another filesystem type (fat, ext4) ?
Can you test with grub for BIOS/legacy boot ?

See above. GRUB-EFI is clearly the culprit. Booting the Trixie d-i in
rescue mode works in BIOS mode, but not in EFI mode.

d-i boots in BIOS mode with isolinux, not grub.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hey Felix,

pe 18.4.2025 klo 19.39 Felix Zielcke (fzielcke@z-51.de) kirjoitti:

Am Freitag, dem 18.04.2025 um 19:10 +0300 schrieb Martin-Éric Racine:

The Debian changelog suggests what caused this:

Someone enabled compiling GRUB with FUSE3 to enhance QEMU support.
The problem with FUSE3 is that it only has very basic support for
btrfs, lacking among other things support for subvolumes. This is a
serious issue considering how widely spread btrfs has become and how debian-installer uses subvolumes by default whenever someone selects
brtfs as the filesystem type. Rendering hosts unbootable is a big
no-no. I would really hope the release team to step in on this one.

FUSE support in GRUB 2 is only used for the grub-mount binary. And not
at all in the actual booting code loaded from EFI/BIOS.

You could though test the fs drivers of it with grub-mount:

grub-mount /dev/sda2 /mnt (replace sda2 with your / partition)
sha1sum /mnt/@rootfs/boot/initrd.img-6.12.20-amd64
sha1sum /@rootfs/boot/initrd.img-6.12.20-amd64

both commands should output the same hash sum.
And yes, this is safe with a mounted fs because the GRUB fs code is
100% complete read-only.

Exactly how does the above address the issue I reported?

Martin-Éric

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Am Samstag, dem 19.04.2025 um 09:02 +0300 schrieb Martin-Éric Racine:

There seems to be two issues here:

1) A magic error upon loading initrd. It also affects the Trixie
rescue mode via UEFI (but not via BIOS). Bookworm rescue mode boots
both ways.
2) fuseblk reporting an unsupported subvol option on Trixie.

It's a simply a/b test: Downgrading grub* to Bookworm made the host
bootable again.

Martin-Éric

Then it would be good if you could bisect GRUB 2 between 2.06 and 2.12.
Then hopefully we get the commit which broke it for you.

It's not that easy though. If you use SecureBoot then you'll need to
sign it with your own MOK key.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 19/04/2025 at 08:02, Martin-Éric Racine wrote:

la 19.4.2025 klo 8.43 Felix Zielcke (fzielcke@z-51.de) kirjoitti:

Am Freitag, dem 18.04.2025 um 21:17 +0300 schrieb Martin-Éric Racine:

pe 18.4.2025 klo 19.39 Felix Zielcke (fzielcke@z-51.de) kirjoitti:

You could though test the fs drivers of it with grub-mount:

grub-mount /dev/sda2 /mnt (replace sda2 with your / partition)
sha1sum /mnt/@rootfs/boot/initrd.img-6.12.20-amd64
sha1sum /@rootfs/boot/initrd.img-6.12.20-amd64

both commands should output the same hash sum.
And yes, this is safe with a mounted fs because the GRUB fs code is
100% complete read-only.

Same checksum. However, manually executing this tells us nothing. It's performed on a filesystem that's already available. It doesn't
simulate GRUB attempting to cold mount a system on a booting host.

It does not matter whether the filesystem is already mounted and
available. grub-mount does not use the kernel filesystem layer, it reads directly the partition (or disk ?) block devices and uses GRUB's own
filesystem drivers.

There seems to be two issues here:

1) A magic error upon loading initrd. It also affects the Trixie
rescue mode via UEFI (but not via BIOS).

It is unclear what you mean here. Does it affect
- boot trixie installer kernel and initramfs
or
- boot your system kernel and initramfs from trixie installer GRUB ?

2) fuseblk reporting an unsupported subvol option on Trixie.

I already explained this is a red herring.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, Apr 13, 2025 at 10:23:17AM +0300, Martin-Éric Racine wrote:

Is there any way I can help the maintainers pinpoint the source of the problem?

This bug needs two things:

1) A full reproducer step list, starting from "I have nothing but an empty amd64 VM".

and

2) A bisect of GRUB between 2.06 and 2.12.


Until at least 1) shows up, this bug is not actionable.


Best,
Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Control: tags -1 + moreinfo unreproducible

On Fri, May 02, 2025 at 08:14:37PM +0300, Martin-Éric Racine wrote:

pe 2.5.2025 klo 19.44 Chris Hofstaedtler (zeha@debian.org) kirjoitti:

On Sun, Apr 13, 2025 at 10:23:17AM +0300, Martin-Éric Racine wrote:

Is there any way I can help the maintainers pinpoint the source of the problem?

This bug needs two things:

1) A full reproducer step list, starting from "I have nothing but an empty amd64 VM".

There isn't much to reproduce:
1) Start with a fully functional host running Bookworm with GRUB-EFI
on a brtfs filesystem created using d-i/Bookworm's default @rootfs
subvolume name.
2) Change APT sources from Bookworm to Trixie.
3) Dist-upgrade.
4) Reboot.
5) Find the above kernel panic.
6) Using Bookworm d-i's rescue mode via EFI, APT pin and downgrade
grub* to Bookworm. All other packages remain at Trixie versions.
7) Reboot.
8) The host boots normally.

I've followed these steps today, and cannot reproduce the problem.
The upgraded VM boots successfully.

Chris

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 3 May 2025 19:43:03 +0300 =?UTF-8?Q?Martin=2D=C3=89ric_Racine?= <martin-eric.racine@iki.fi> wrote:

la 3.5.2025 klo 17.47 Martin-Éric Racine (martin-eric.racine@iki.fi)

kirjoitti:

la 3.5.2025 klo 17.41 Julian Andres Klode (jak@debian.org) kirjoitti:

On 3 May 2025 15:03:18 CEST, "Martin-Éric Racine"

<martin-eric.racine@iki.fi> wrote:

la 3.5.2025 klo 14.42 Chris Hofstaedtler (zeha@debian.org)

kirjoitti:

Control: tags -1 + moreinfo unreproducible

On Fri, May 02, 2025 at 08:14:37PM +0300, Martin-Éric Racine

wrote:

pe 2.5.2025 klo 19.44 Chris Hofstaedtler (zeha@debian.org)

kirjoitti:

On Sun, Apr 13, 2025 at 10:23:17AM +0300, Martin-Éric

Racine wrote:

Is there any way I can help the maintainers pinpoint the

source of the problem?

This bug needs two things:

1) A full reproducer step list, starting from "I have

nothing but an empty

amd64 VM".

There isn't much to reproduce:
1) Start with a fully functional host running Bookworm with

GRUB-EFI

on a brtfs filesystem created using d-i/Bookworm's default

@rootfs

subvolume name.
2) Change APT sources from Bookworm to Trixie.
3) Dist-upgrade.
4) Reboot.
5) Find the above kernel panic.
6) Using Bookworm d-i's rescue mode via EFI, APT pin and

downgrade

grub* to Bookworm. All other packages remain at Trixie versions.
7) Reboot.
8) The host boots normally.

I've followed these steps today, and cannot reproduce the problem.
The upgraded VM boots successfully.

I beleive you. Again, googling this exact error mostly pulls up >reports of this failing on ASUS motherboards of various models.

Unfortunately some firmware is just broken and there isn't much

that can be done about that.

Sorry, but that's a really pitiful excuse for breaking something that worked fine until now.

As we're moving towards more upstream solutions that use more

parts of the firmware, such as EFI LoadFile2 protocols for loading the
initrd, more hardware will break.

I cannot help but wonder what is the point of breaking something as fundamental as a bootloader, even just for the sake of adopting new
ways of doing things. It can only result in fewer and fewer people
using Debian.

Check this post out:

https://forum.level1techs.com/t/ubuntu-upgrade-bricked-os-initramfs-unpacking-failed-invalid-magic-at-start-of-compressed-archive/215981

It's on Ubuntu, but the error is similar. Key point: something among
the boot messages (see screenshot there) suggests that either GRUB or
Linux don't know how to boot off a btrfs partition anymore. Sure

enough, someone in the thread says that once they made a fresh install

Hello,

I am running two very similiar pre-upgrade setups to Martin-Éric
(Bookworm, btrfs install via Debian Installer with @rootfs) but on
vastly different platforms - an Asus Ryzen B550 and a Lenovo 11th Gen
Tiger Lake-LP laptop. I also have Asrock Ryzen B450 and B550 machines
standing by for additional testings if applicable. For at least one of
these I was planning a fresh Trixie btrfs install once it is out.

Seems to me this bug is for now limited to Asus boards (so far I saw
Asus H61 and Z68 chipsets mentioned in the forum links you provided)
with Sandy bridge CPUs and btrfs. Anyway, my question for you
Martin-Éric, since I got a bit lost in your replies with mantainers, how
would I try to verify if my machines will survive the Trixie upgrade or
not (without actually fully upgrading for now)?

If I gather you correctly, if "Debian GNU/Linux 13 (trixie) amd64 -
netboot mini.iso 20241227" boots in UEFI mode, the system will also
probably boot after upgrade to Trixie?

BTW I feel your frustration, considering all the architectures Debian
supports, that your somewhat-aged but pretty mainstream system is now (hopefully just temporarily) not supported.

Regards,

Staš

P.S. We still had some retired Sandy/Ivy Bridge H61/H67 workstations
(mostly Gigabyte boards though) lying around a couple weeks ago, but we
donated them to charity, shame as they could have come handy now.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)