Package: incus
Version: 6.0.4-1
Severity: important

Dear Maintainer,

On arm64 an up to date trixie/arm64 system, I cannot get a shell after launching a VM. No longer how much I wait, the agent is never currently running:

$ incus launch images:debian/12 --vm foobar
Launching foobar
$ incus shell foobar
Error while executing alias expansion: incus exec foobar -- su -l
Error: VM agent isn't currently running
$ incus shell foobar
Error while executing alias expansion: incus exec foobar -- su -l
Error: VM agent isn't currently running
$ incus shell foobar
Error while executing alias expansion: incus exec foobar -- su -l
Error: VM agent isn't currently running
$ incus shell foobar
Error while executing alias expansion: incus exec foobar -- su -l
Error: VM agent isn't currently running
$ incus shell foobar
Error while executing alias expansion: incus exec foobar -- su -l
Error: VM agent isn't currently running

I tried debian/13 and it worked fine.

I tried debian/12 on an up to date trixie/amd64 (x86_64) and it also works.

What else can I provide to help debug this?

-- System Information:
Debian Release: trixie/sid
APT prefers testing
APT policy: (900, 'testing'), (500, 'stable-security'), (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: arm64 (aarch64)

Kernel: Linux 6.1.0-33-arm64 (SMP w/32 CPU threads)
Locale: LANG=pt_BR.UTF-8, LC_CTYPE=pt_BR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages incus depends on:
ii incus-agent 6.0.4-1
ii incus-base 6.0.4-1
ii qemu-system-arm 1:10.0.0~rc2+ds-2
ii swtpm 0.7.1-1.5
ii virtiofsd 1.13.0-5

incus recommends no packages.

Versions of packages incus suggests:
ii gdisk 1.0.10-2+b1

-- no debconf information

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmgH2JcACgkQ/A2xu81G C97EixAAsuULIOgitzNTAjHGWk8VkB5xPw44vZcyHpNEZiwYryqopJUUHpKwdZmK G2xZBl6NIUy2vleod0leNchSLk089vZGg1FKXBW2FdOK+mRxs15up0zHSuW2urhX E7GdYTWn2r8pv4hz+81Z+MdS50NBcsiLfr30/zaccXoHA08euVNKkBvspus/NAf6 Lom4V898mW5c/37Ot+5OqX6Vml3wEzZNciPwqIo5wpsLaEiya+lbKUurnrYTrTUi x49z47vF1CFdZMYirWclDB06LP/g9l/THwafrC8owXNfAeCNa1e+ApjxHI//GF+b p27wdS70oCSIERaFzzQU6lnCJmLD0pPC0ks0FBQx8cUDtUIRkIs2x2RvP34MjXBB I8Nbxv4Mx9Cw8kr+hSFAqysEhigAY/auRJTOzAUis5DssNh2pPkiSqIb/t4F1gMD R07yKnVTYJJMNMs5PzLh5STVX0QV4wWTyH8UZlEBoJeD5s1V4yo5l9wjt1U1G4Ke uK53iW9HtIX/m2mQxZ6NFOA1U0HcQFBpxVa2SCwnQ6lb88sHeS5shRMrCg+PngYW +ynIkhS9fKQl1n/NIVK2l7YLOU22M4wuvri2AtPt+dkkfRc96cvtBFww30pR0BvX 1y59mvnS7UQ8N/8UW2eS4HmRYw/Rp1EOOkPKYeS8Y0NzmJMNi1w=
=XbRk
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

control: tags -1 + confirmed

Hi Antonio,

I have successfully reproduced the issue on a RaspberryPi 5 with its
apt sources set to trixie. While not a pure Debian environment, it's
the only arm64 system I have full access to. :)

TL;DR: As a workaround, try disabling secure boot:

$ incus create images:debian/12 --vm foobar
$ incus config set foobar security.secureboot=false
$ incus start foobar

With kernel 1:6.12.20-1+rpt1 and arm64 bookworm VM image
20250422_05:24 (fingerprint f6e1ffebdc1c), I get the following boot
error when running `incus launch images:debian/12 --vm deb12 --console=vga`:

Booting `Debian GNU/Linux'

Loading Linux 6.1.0-33-arm64 ...
Loading initial ramdisk ...

Synchronous Exception at 0x000000006D79AF60

Can you try starting your bookworm VM with the `--console=vga` option
and see if you get the same error? (You might have to install spice-
client-gtk or similar spice client first.)

Some googling pointed me to shim and I found bug #1076724. _Maybe_
that's related; there's a fix upstream[0] that's not present in Debian. (However, the version of shim looks essentially the same for both
bookworm and trixie. But there could be lots of moving pieces that
would result in a different alignment of produced boot artifacts.)

I did a local build of shim with the upstream commit in an attempt to
test the fix, but in so doing I had to disable secure boot for the VM.
I then tested disabling secure boot in the original bookworm VM, and
found that it booted properly too. (I didn't bother to try to enroll
local secure boot keys and sign my modified shim; maybe that would be a
useful datapoint for someone else to do?)

So for the moment, I think that disabling secure boot is a viable
workaround. It would still be nice to get whatever the underlying issue
is fixed, but I'm not sure if I was going down the right path or if the
issue lies elsewhere.

Out of curiosity while debugging things, I also tried a handful of
older qemu versions, all of which encountered the same error: 1:10.0.0~rc2+ds-2, 1:9.2.1+ds-1, 1:9.1.1+ds-5, and 1:9.0.2+ds-2+b1.

Testing some other images, ubuntu/24.04 works, while ubuntu/22.04
encounters a similar "Synchronous Exception" error at boot.

Mathias

[0] -- https://github.com/rhboot/shim/pull/673

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE1Bp60H32xfynSJ8cKe7i1uz0QvkFAmgILpYACgkQKe7i1uz0 QvlbLhAAoj/aacz7tbeJd3+lPXN9DZ+HaEhYdk/pDEPoTpBDcWlmb5KkgV2fy37L C2aqr+3lLXIq/XUiYJ49Jufsttrqywg75YNlW+bPrw7Nw83ORB4Izewbl5Y6oLIk DexE5wrlWqUNVrbBY+y5I3FmomZTJyNSi1pEXmjvGGnW34pw80AG8PQ21qPpLPsn o0+cwq8KYYDG8S3nUv716KVRQ0uc2wcgZhVtwVa70dyh0O8kyI8G+8xwfRZW3kEW Ei3h1KQK5k0sJC2XsHmY9nm0n1lWZzVgtwXoAU27+ku1AF6iuGuhFdTWQrHpon8X edu/Wsn+RfUWJohEhV3ugtbFtmDpQV6nYFYZX/7CXLzSadRJHGJ25vXUiUHcX1NK HUKQB9JSzy45VO9RbAKDVYtxaCFWjtw5S8aLynDW5t6VT10UjAq3JEdAeS3SZt8Y LeVs1+yhZXgyyXtZvZBmpFy/mMYwmnsO4vfUphIcpTCoWsiqRK0vEZDnmHNjklhY jDlQg5l/WH3YJmYTFqv7DEllLqSiaLAf5Qjl1JsanB2KvWtrJwVR+cFwm+AUXyd8 kSastNsFNbUIDX3Xx7WBf6GtiICHcpiTilvM9k3CamXHCuhikcWFaVgpqifizMhQ WG3LgKPHqahSbOqMC1PDnM1Tgczc7KWPpn4+SSJVMmbKHdPW7OI=
=WOl8
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

--=-z0KrJZHibi4Qu0+Vawsm
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

control: retitle -1 qemu-efi-aarch64: Secure Boot regression for some arm64 VMs control: reassign -1 qemu-efi-aarch64 2025.02-7
control: severity -1 serious
control: affects -1 incus

Release 2025.02-5 of src:edk2 dropped the patch Revert-ArmVirtPkg- make-EFI_LOADER_DATA-non-executabl.patch. This has caused a regression
for (at least) Debian bookworm and Ubuntu 22.04 arm64 VMs when booting
with Secure Boot enabled.

I have verified that re-applying that patch to src:edk2 2025.02-7
allows these VMs to boot once again.

Mathias

--=-z0KrJZHibi4Qu0+Vawsm
Content-Disposition: attachment;
filename*0ík2-0005-Revert-ArmVirtPkg-make-EFI_LOADER_DATA-non-executab;
filename*1=l.patch
Content-Type: text/x-patch;
name*0ík2-0005-Revert-ArmVirtPkg-make-EFI_LOADER_DATA-non-executabl.pa;
name*1=tch; charset="ISO-8859-1"
Content-Transfer-Encoding: base64

RnJvbSBjMmRmNjIwM2RhNWRmMWFiMDE1ZTUxZjZkNmMzMDQ5Y2U3ODY3ZDY5IE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/U3Q9QzM9QTlwaGFuZT0yMEdyYWJlcj89IDxz dGdyYWJlckBzdGdyYWJlci5vcmc+CkRhdGU6IEZyaSwgMTUgU2VwIDIwMjMgMTE6MTM6MjMgLTA0 MDAKU3ViamVjdDogW1BBVENIXSBSZXZlcnQgIkFybVZpcnRQa2c6IG1ha2UgRUZJX0xPQURFUl9E QVRBIG5vbi1leGVjdXRhYmxlIgpNSU1FLVZlcnNpb246IDEuMApDb250ZW50LVR5cGU6IHRleHQv cGxhaW47IGNoYXJzZXQ9VVRGLTgKQ29udGVudC1UcmFuc2Zlci1FbmNvZGluZzogOGJpdAoKVGhp cyByZXZlcnRzIGNvbW1pdCAyOTk3YWUzODczOTc1NmVjYmE5YjBkZTE5ZTg2MDMyZWJjNjg5ZWY5 LgoKU2lnbmVkLW9mZi1ieTogU3TDqXBoYW5lIEdyYWJlciA8c3RncmFiZXJAc3RncmFiZXIub3Jn PgotLS0KIEFybVZpcnRQa2cvQXJtVmlydC5kc2MuaW5jIHwgMiArLQogMSBmaWxlIGNoYW5nZWQs IDEgaW5zZXJ0aW9uKCspLCAxIGRlbGV0aW9uKC0pCgpkaWZmIC0tZ2l0IGEvQXJtVmlydFBrZy9B cm1WaXJ0LmRzYy5pbmMgYi9Bcm1WaXJ0UGtnL0FybVZpcnQuZHNjLmluYwppbmRleCA1Mzg0YTQx ODE4Li5mZTFhZTI1YTkzIDEwMDY0NAotLS0gYS9Bcm1WaXJ0UGtnL0FybVZpcnQuZHNjLmluYwor KysgYi9Bcm1WaXJ0UGtnL0FybVZpcnQuZHNjLmluYwpAQCAtMzgzLDcgKzM4Myw3IEBACiAgICMg YnVpbGQgY29tbWFuZCBsaW5lIHlvdSBjYW4gYWxsb3cgY29kZSBleGVjdXRpb24gaW4gRWZpTG9h ZGVyRGF0YS4gVGhpcyBpcw0KICAgIyByZXF1aXJlZCB3aGVuIHVzaW5nIHNvbWUgb3V0ZGF0ZWQg R1JVQiB2ZXJzaW9ucy4NCiAgICMNCi0gIGdFZmlNZGVNb2R1bGVQa2dUb2tlblNwYWNlR3VpZC5Q Y2REeGVOeE1lbW9yeVByb3RlY3Rpb25Qb2xpY3l8MHhDMDAwMDAwMDAwMDA3RkQ1DQorICBnRWZp TWRlTW9kdWxlUGtnVG9rZW5TcGFjZUd1aWQuUGNkRHhlTnhNZW1vcnlQcm90ZWN0aW9uUG9saWN5 fDB4QzAwMDAwMDAwMDAwN0ZEMQ0KIA0KICAgZ0VmaU1kZU1vZHVsZVBrZ1Rva2VuU3BhY2VHdWlk LlBjZENwdVN0YWNrR3VhcmR8VFJVRQ0KIA0K


--=-z0KrJZHibi4Qu0+Vawsm--

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEE1Bp60H32xfynSJ8cKe7i1uz0QvkFAmgJergACgkQKe7i1uz0 Qvkj9w//f6ZyYbBy7qbfDzHyKyp4qSfSnG+uZaVs+dA5TziWVtCKnFOSIdY66yYC 6zo90+WrhiHPdfaVXNoKXsrVr7scOeaICbo9EgDjEdM0rOBcd6wIvCTxyTBrjdjN e3QX2ZjaAGvPIYC4rWAcArajkiKi32NBZgsZ+7oBjrwvjG41hjwbUog2QaBuAzo8 HLEPTtcxd0uDQSnLgzvNY/MOvUlzHNb0xrtfpFq20Dus+oEetIynO1CYA6jYyNE4 sZ9Y3/W2RaYyfW0BMBKbIr3dXsUIgkOMXqaDXIRfd609jOcnfcP1HrGXToGV0a4N j/BmTqEQZ2M7wD4c/5wWqhFxc27fGVurX3+JDzK0ylHjM3kFtL5dqFUtoPKSSXlo wbE4tU1N7B8WjDQ4woPxhLtYcWXqTt4g8rFqUZsBmPk0RvEfV39eUX2k6cyLPWJG Hy+BRmIuBqSrvvzscMzvF2hmhIqsvkLHcOOUHUqlkUFY1aRFTYXOTaKa66nsImha j1lC6eu0U5jb5FJ3vAIF4aypoCnzmRKPIi1aHBg3JzrGLJT25dEZV6v6u8qyRDLY zpAgdLwe54cQWztzrM5eSP5zZm/NR1uUqYNdDGQHPRrRTWnrwlpCZmvRxCgE849j 531GM58BYXkzDWqBOK2bKlPJopZzzba9nhkYAiKpjSuDX2I/amU=
=n1aJ
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Apr 23, 2025, 17:41 Mathias Gibbens <gibmat@debian.org> wrote:

control: retitle -1 qemu-efi-aarch64: Secure Boot regression for some
arm64 VMs
control: reassign -1 qemu-efi-aarch64 2025.02-7
control: severity -1 serious
control: affects -1 incus

Release 2025.02-5 of src:edk2 dropped the patch Revert-ArmVirtPkg- make-EFI_LOADER_DATA-non-executabl.patch. This has caused a regression
for (at least) Debian bookworm and Ubuntu 22.04 arm64 VMs when booting
with Secure Boot enabled.

I have verified that re-applying that patch to src:edk2 2025.02-7
allows these VMs to boot once again.

Please see NEWS.Debian:

https://salsa.debian.org/qemu-team/edk2/-/blob/08d4411d458eefc4df5d48acce4f995d4ae6087d/debian/qemu-efi-aarch64.NEWS

<div dir="auto"><div><br><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Wed, Apr 23, 2025, 17:41 Mathias Gibbens &lt;<a href="mailto:gibmat@debian.org">gibmat@debian.org</a>&gt; wrote:<br></div><blockquote class="
gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">control: retitle -1 qemu-efi-aarch64: Secure Boot regression for some arm64 VMs<br>
control: reassign -1 qemu-efi-aarch64 2025.02-7<br>
control: severity -1 serious<br>
control: affects -1 incus<br>

  Release 2025.02-5 of src:edk2 dropped the patch Revert-ArmVirtPkg-<br> make-EFI_LOADER_DATA-non-executabl.patch. This has caused a regression<br>
for (at least) Debian bookworm and Ubuntu 22.04 arm64 VMs when booting<br>
with Secure Boot enabled.<br>

  I have verified that re-applying that patch to src:edk2 2025.02-7<br>
allows these VMs to boot once again.<br></blockquote></div></div><div dir="auto"><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br></blockquote></div></

<div dir="auto">Please see NEWS.Debian:</div><div dir="auto"></div><div dir="auto"><a href="https://salsa.debian.org/qemu-team/edk2/-/blob/08d4411d458eefc4df5d48acce4f995d4ae6087d/debian/qemu-efi-aarch64.NEWS">https://salsa.debian.org/qemu-team/edk2/-

/blob/08d4411d458eefc4df5d48acce4f995d4ae6087d/debian/qemu-efi-aarch64.NEWS</a></div><div dir="auto"><br></div><div dir="auto"></div></div>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Apr 23, 2025 at 10:01:36PM -0600, dann frazier wrote:

On Wed, Apr 23, 2025, 17:41 Mathias Gibbens <gibmat@debian.org> wrote:

control: retitle -1 qemu-efi-aarch64: Secure Boot regression for some
arm64 VMs
control: reassign -1 qemu-efi-aarch64 2025.02-7
control: severity -1 serious
control: affects -1 incus

Release 2025.02-5 of src:edk2 dropped the patch Revert-ArmVirtPkg- make-EFI_LOADER_DATA-non-executabl.patch. This has caused a regression
for (at least) Debian bookworm and Ubuntu 22.04 arm64 VMs when booting
with Secure Boot enabled.

I have verified that re-applying that patch to src:edk2 2025.02-7
allows these VMs to boot once again.

Please see NEWS.Debian:

https://salsa.debian.org/qemu-team/edk2/-/blob/08d4411d458eefc4df5d48acce4f995d4ae6087d/debian/qemu-efi-aarch64.NEWS

It says this:

The EFI_MEMORY_ATTRIBUTE_PROTOCOL is now enabled by default in the AAVMF_CODE.secboot.fd image. This is a security feature that will
cause crashes for operating systems with bootloaders that do not
observe proper memory access semantics. Users that experience issues
with such bootloaders have the options to either append the following
to the qemu-system-aarch64 command line:
-fw_cfg name=opt/org.tianocore/UninstallMemAttrProtocol,string=y
or switch to the no-secboot image, which uninstalls this protocol by
default.

This would mean having to change software like libvirt, incus, etc, to
pass this parameter but only for debian 12 images, which might not be
easy to do. Since debian 13 just works, do you happen to known what
bits need to be backported to fix debian 12 images to work out of the
box without having to turn off this security feature?

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmgKJZIACgkQ/A2xu81G C97yxw/+NaEXlUeOKKJtHovMEw+Wv7d7H20xhZL8yCOiK0lBE8iIF8R2XOlAh/jb bQDbKx/zh0o9eiaF+4EVm9lMbb4eia5DsZtqO7TeSpQt8RqVnV4ppiW3FAckmYZO WCHQIml7GWwkgr1YU2gDJnw6/qGgb+VkrqDZqsvIA8buuXZM0jqk51v7Xpp4LRY6 wd8ux8WGIZ+on0ppjAU0UKt/AmsRwoGjAhUIE39r/DBYcLNlIW0nuSUiYcedAKUM sOx0ZEgh7kU5AS36tvnsZBGtgs3pmXmKpOjFgYO92ofFms2KrubeAkoKx/G4P2Wd XkWaKF8HmQ/R+NpHI+ha34ei//w6yQymTtztXvz9JUgjPdCyTO4IzzjBoAQJFcJr yV2SRFAteAT/SO0dg3TVhnfcN/DcLY/syalEDEkYe/Kz6Ktuwe3VKCa+FZ2M1uws cWGfu7/sB7LwxXk0k6nA/9+zNqDkqeniQmAXIHmwnp1Rcr34A9l+wXPVw3PkLrXW CatBuZsG2bo7lj8adi1oyBXDBtlPRxXTxGH8tnUreCfu1vY0EZ0tXVeDWtI0jV/T Du2IHxNGb6V2AfBqubAUifR+0WUeL4eWImBtk/D1MpzRsc7QVGJwvDIStVqMI7ZY A1BFN1vU4ld4N9+Oc3ZfKqG0gf25ih76Ej7VaRBXlzzSgt+Ehvs=
=ZbQ2
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

On Wed, Apr 23, 2025 at 12:04:38AM +0000, Mathias Gibbens wrote:

control: tags -1 + confirmed

Hi Antonio,

I have successfully reproduced the issue on a RaspberryPi 5 with its
apt sources set to trixie. While not a pure Debian environment, it's
the only arm64 system I have full access to. :)

TL;DR: As a workaround, try disabling secure boot:

$ incus create images:debian/12 --vm foobar
$ incus config set foobar security.secureboot=false
$ incus start foobar

With kernel 1:6.12.20-1+rpt1 and arm64 bookworm VM image
20250422_05:24 (fingerprint f6e1ffebdc1c), I get the following boot
error when running `incus launch images:debian/12 --vm deb12 --console=vga`:

Booting `Debian GNU/Linux'

Loading Linux 6.1.0-33-arm64 ...
Loading initial ramdisk ...

Synchronous Exception at 0x000000006D79AF60

Can you try starting your bookworm VM with the `--console=vga` option
and see if you get the same error? (You might have to install spice- client-gtk or similar spice client first.)

FWIW I confirm all of these. The VM gives the exact same failure
(Synchronous Exception at ADDRESS). Copying the image out of the incus
storage and trying to boot it with libvirt/virt-manager also causes the
same error.

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEst7mYDbECCn80PEM/A2xu81GC94FAmgKJgwACgkQ/A2xu81G C97O4w//TDyuJgvdsnANCIPFpT5vPt2L/6RHNB9AWqqqecYQ+GraO7z9lHL2/UWD /wzvVmJIF3+18lYre/Y16LrfYar6ZtMo2KnxJu+Xvds1mqP+0zkWzOj6dWed9K9O ySuX8jkb9oHxLSAK4xbn2pZhvvO/NC+lRZzk1070pp8u6k0hS/DE8w9NZeLpVPKX qhvF1uB8nfBgess2DhDJb1r31E9OavJ3Pql3MGsRuAxQEI0zqyiRxLunSVWm71YC bXPxKs50thy1XaRnEOc/m+Qfd+Syd2S5bA1rUG2dUGNGU/ipq/mbuvg0JZyXKWLl dCmfY+rrDgRZ8iPbF1jqJOJym6Kwn0CdOYkQpPY+vvEwsWM4v42DbnWaJd95Aaqk eumJJ33YSpHfgkwWO6VyzVm2KGFL2AlEVUgBYyEd6N2qRcktpkAyzuOZLv7isF4F pJCQTLmfVyTrUd3oDbVYW9t9LHCnU6npouvY66v/EOTdv9V7H9SVBsOisZinZTph tMKZputT/5QRlQdd7zWE49uVwaDMxkIhqdrFHmrfVHyCQzU8mWjRVxlhBtSvI4/X hPq7SV78Uznh9ZmqE7i3V8Tm9P3mndb2XZxObb/ftdxg6J+wgGAMyBHtz3BN4rS1 wkV/Un2tJldTtM2fttfVW6AH5aCYmf5O5+V2uGSeJdIZCf08KX0=
=ISpO
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)