Johannes Schauer Marin Rodrigues <josch@debian.org> (2025-04-12):
Package: flash-kernel
Version: 3.109+reform1
This problem occurred with a patched flash-kernel but you know that we only patch the machines file with more entries and do not patch the scripts: https://source.mnt.re/reform/reform-debian-packages/-/blob/main/patches/flash-kernel
Might want to adjust the version for the BTS though, I'd think?
$ apt-cache policy linux-image-arm64...
linux-image-arm64:
Installed: 6.12.19-1+reform20250322T135019Z
Candidate: 6.12.22-1+reform20250411T222458Z
$ sudo apt full-upgrade...
Removing linux-headers-6.12.16-mnt-reform-arm64 (6.12.16-1+reform20250219T175041Z) ......
flash-kernel: A higher version (6.12.19-mnt-reform-arm64) is still installed, no reflashing required....
Setting up linux-image-6.12.22-mnt-reform-arm64 (6.12.22-1+reform20250411T222458Z) ......
Installing /usr/lib/linux-image-6.12.22-mnt-reform-arm64/amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb into /boot/dtbs/6.12.22-mnt-reform-arm64/amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb...
Setting up flash-kernel (3.109+reform1) ...
===========================================
REBOOT
===========================================
$ uname -a
Linux kodi 6.12.19-mnt-reform-arm64 #1 SMP Debian 6.12.19-1+reform20250322T135019Z (2025-03-22) aarch64 GNU/Linux
$ sudo apt remove linux-image-6.12.19-mnt-reform-arm64 linux-headers-6.12.19-mnt-reform-arm64
[...]
/etc/kernel/postrm.d/zz-flash-kernel:
Using DTB: amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb
flash-kernel: Kernel 6.12.19-mnt-reform-arm64 has been removed.
flash-kernel: A higher version (6.12.22-mnt-reform-arm64) is still
installed, no reflashing required.
$ grep -a 6.12.19 /boot/boot.scr
setenv fk_kvers '6.12.19-mnt-reform-arm64'
$ sudo flash-kernel
Using DTB: amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb
Installing /usr/lib/linux-image-6.12.22-mnt-reform-arm64/amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb into /boot/dtbs/6.12.22-mnt-reform-arm64/amlogic/meson-g12b-bananapi-cm4-mnt-reform2.dtb
Taking backup of meson-g12b-bananapi-cm4-mnt-reform2.dtb.
Installing new meson-g12b-bananapi-cm4-mnt-reform2.dtb.
flash-kernel: installing version 6.12.22-mnt-reform-arm64
Generating boot script u-boot image... done.
Taking backup of boot.scr.
Installing new boot.scr.
$ grep -a 6.12.22 /boot/boot.scr
setenv fk_kvers '6.12.22-mnt-reform-arm64'
Luckily, running flash-kernel manually fixed the issue. But had I not noticed that /boot/boot.scr still contained a version of a kernel that I had just removed, my system would've become unbootable.
Presuming this isn't some bizarre fluke, then this bug is likely present in most versions of flash-kernel, as that code has not been touched for at least a 2-5 years...
I vaguely recall a bug or merge request coming from Ubuntu that might be related...
I will try to reproduce this issue later using Debian kernels. My hunch is, that the problem is that a new kernel version got installed at the same time that flash-kernel got upgraded. Because at the time that 6.12.22 is installed,
flash-kernel should have been run but instead we see this in the log:
flash-kernel: deferring update (trigger activated)
And then the only other flash-kernel related message is:
Setting up flash-kernel (3.109+reform1)
So maybe this is about the order of triggers? Instead of "deferring update", flash-kernel should've been run at the point of "Setting up linux-image-6.12.22", no?
On 2025-04-12, Vagrant Cascadian wrote:
On 2025-04-12, Johannes Schauer Marin Rodrigues wrote:
Luckily, running flash-kernel manually fixed the issue. But had I not
noticed
that /boot/boot.scr still contained a version of a kernel that I had
just
removed, my system would've become unbootable.
Presuming this isn't some bizarre fluke, then this bug is likely
present
in most versions of flash-kernel, as that code has not been touched
for
at least a 2-5 years...
I vaguely recall a bug or merge request coming from Ubuntu that might
be
related...
Does this by chance help at all:
https://salsa.debian.org/installer-team/flash-kernel/-/merge_requests/41
It is about partially installed kernels,
but perhaps might fix this as a
side-effect?
This is certainly worth looking into at some point ... although maybe a
bit too large changeset during the bookworm freeze right now...
It seems that, if flash-kernel gets upgraded together with the kernel, then flash-kernel is not run at the end of the installation and thus, the old kernel stays referenced in /boot/boot.scr. Steps to reproduce:share/perl5 /usr/lib/aarch64-linux-gnu/perl-base /usr/lib/aarch64-linux-gnu/perl/5.40 /usr/share/perl/5.40 /usr/local/lib/site_perl) at /usr/share/perl5/Debconf/FrontEnd/Readline.pm line 8.)
$ debvm-create -r unstable -- \
http://snapshot.debian.org/archive/debian/20240401T000000Z/ \
--aptopt='Acquire::Check-Valid-Until "false"' \
--dpkgopt=force-confold \
--include flash-kernel,u-boot-tools \
--setup-hook='mkdir -p "$1"/etc/flash-kernel' \
--setup-hook='printf "Machine: linux,dummy-virt\nKernel-Flavors: any\nBoot-Script-Path: /boot/boot.scr\nU-Boot-Script-Name: bootscr.uboot-generic\nRequired-Packages: u-boot-tools\n" > "$1"/etc/flash-kernel/db' \
--setup-hook='echo linux,dummy-virt > "$1"/etc/flash-kernel/machine' [...]
$ debv-run
[...]
root@testvm:~# echo "deb http://snapshot.debian.org/archive/debian/20250401T000000Z/ unstable main" >> /etc/apt/sources.list && apt update && apt full-upgrade --yes
[...]
Setting up linux-image-6.12.21-cloud-arm64 (6.12.21-1) ...
I: /vmlinuz is now a symlink to boot/vmlinuz-6.12.21-cloud-arm64
I: /initrd.img is now a symlink to boot/initrd.img-6.12.21-cloud-arm64 /etc/kernel/postinst.d/initramfs-tools:
update-initramfs: Generating /boot/initrd.img-6.12.21-cloud-arm64
W: No zstd in /usr/bin:/sbin:/bin, using gzip
W: /sbin/fsck.ext4 doesn't exist, can't install to initramfs
flash-kernel: deferring update (trigger activated) /etc/kernel/postinst.d/zz-flash-kernel:
flash-kernel: deferring update (trigger activated)
Setting up linux-image-cloud-arm64 (6.12.21-1) ...
Setting up flash-kernel (3.108) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 79.)
debconf: falling back to frontend: Readline
debconf: unable to initialize frontend: Readline
debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module) (@INC entries checked: /etc/perl /usr/local/lib/aarch64-linux-gnu/perl/5.40.1 /usr/local/share/perl/5.40.1 /usr/lib/aarch64-linux-gnu/perl5/5.40 /usr/
debconf: falling back to frontend: Teletype
Setting up dmsetup (2:1.02.205-1) ...
Setting up libdevmapper1.02.1:arm64 (2:1.02.205-1) ...
Setting up libcryptsetup12:arm64 (2:2.7.5-1) ...
Setting up systemd-cryptsetup (257.4-7) ...
Processing triggers for debianutils (5.21) ...
Processing triggers for libc-bin (2.41-6) ...
Processing triggers for systemd (257.4-7) ...
Processing triggers for initramfs-tools (0.147) ...
update-initramfs: /boot/initrd.img-6.12.21-cloud-arm64 has already been updated since Sun Apr 13 21:19:34 2025.
root@testvm:~# grep -a 6.12.21 /boot/boot.scr
root@testvm:~# grep -a 6.7.9 /boot/boot.scr
setenv fk_kvers '6.7.9-cloud-arm64'
So bottom line: if I upgrade flash-kernel together with the kernel, then flash-kernel will not put the new kernel into /boot/boot.scr. This is of course
fatal when at the same time that the new kernel got installed the new kernel (the one still referenced by /boot/boot.scr) got removed.
I hope having this easily reproducible will help you figure out what's going on.
I patched flash-kernel with the changes from this MR: https://salsa.debian.org/installer-team/flash-kernel/-/merge_requests/41
I can confirm that this patch stack fixes the problem that I had and which can be demonstrated with above reproducer. Thus, I'm tagging this bug with "patch".
Sysop: | Keyop |
---|---|
Location: | Huddersfield, West Yorkshire, UK |
Users: | 480 |
Nodes: | 16 (2 / 14) |
Uptime: | 250:19:40 |
Calls: | 9,532 |
Files: | 13,650 |
Messages: | 6,137,985 |