• Re: Slow 'daily-build' command (Was: Bug#1091366: debian-installer: No

    From Cyril Brulebois@21:1/5 to All on Wed Dec 25 08:40:01 2024
    Hi,

    Roland Clobus <rclobus@rclobus.nl> (2024-12-24):
    On 24/12/2024 20:19, Cyril Brulebois wrote:
    Building with daily-build seems super slow for no obvious reasons, so I've just tried a regular `dpkg-buildpackage -b` instead, and I can replicate the issue with files below build/dest/cdrom/gtk after the build.

    I've seen this slow building too.
    It looks like the Makefile (despite the ifndef-guard) is calling 'dpkg-architecture' really often.

    This runs within a second:
    LC_ALL=C DEB_HOST_ARCH=amd64 DEB_HOST_ARCH_OS=linux DEB_HOST_GNU_CPU=x86_64 DEB_HOST_GNU_SYSTEM=linux-gnu DEB_HOST_MULTIARCH=x86_64-linux-gnu time make reallyclean

    whereas this takes minutes (even though it was run after the previous command):
    LC_ALL=C time make reallyclean

    daily-build calls `make all_list`, which is nice to reproduce the issue
    (it's mostly Makefile-based introspection).

    With the brand new make in unstable, that looked like an easy target,
    and it is indeed: after downgrading make to testing's version in my sid
    chroot, `make all_list` only takes a few seconds, as expected.

    If someone wants to pick that up, `make -d all_list` gives a lot of
    details about what's happening inside make.


    Cheers,
    --
    Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
    D-I release manager -- Release team member -- Freelance Consultant

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmdrtkwACgkQ/5FK8MKz VSAkwg//T5PpVPxhC4WhNp6GIpOic2FWxI+NXuHL/T2BFSvjCkDDPjoTyPooY47B V7ma0lRrOi6gzduUvDSOmnNdnyd+YMebM+rOP9zTDEuBP/rY731i+J0IfoeChUX/ p17hGN++hMsn0QgWNp/orDEjjATCrp7IoU2m5V65JwqA1gUlgZL37Yk/33nRJW0B N/K9OrAAXt1CT/3nlRHQ0J0GezAG0M0Kn1h+UmWGcbzk8k8F7QeOcQt5uBqsVKHX psRe/exqD7BeYP/izdIZDAqSU3rIGKbFZuwFIDrej5UX1bgMqpwtOBd/TVVHiGIW D0sk80C6Z0xp//UCvfoSIttdwJjJX685wyLvlztL2MPwvmv4d7RUjlvum2j9C8OJ m454AuXfURUN8nNx1vtB81vsA2DNmle+/6vt37lQ7/4i0J30NXvjnKvYvngvNR9G lHrb9QTc8nqwo50KQxiWC+VSd95Dp9oCymSwBX61xJ4KNAnPpulwsaGJvecz6ThX WqrqyEBEeTgKW74acAKBoJ/89KQaDekECWmrI11TvuJEuIJh70P7te0LWY0FE6jG FsieYyMV7Kme0CugB5XPXCP8JRhKrABLNIvqkTM/5eeWT2KmvhmMo1gx9jMfegO+ snwvRroqgc+En6wQbIGDJw7ok36nkGtkzN2SwHmUB24eYM0EyMQ=
    =JMCZ
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    *
  • From Cyril Brulebois@21:1/5 to All on Sat Dec 28 02:10:01 2024
    Hi,

    Cyril Brulebois <kibi@debian.org> (2024-12-25):
    daily-build calls `make all_list`, which is nice to reproduce the issue
    (it's mostly Makefile-based introspection).

    With the brand new make in unstable, that looked like an easy target,
    and it is indeed: after downgrading make to testing's version in my sid chroot, `make all_list` only takes a few seconds, as expected.

    If someone wants to pick that up, `make -d all_list` gives a lot of
    details about what's happening inside make.

    Having a quick look at whether that's impacting daily builds being
    performed on porterboxes, I'm seeing a huge penalty but it's much more
    recent than the make(-dfsg) update in unstable:

    The “Accepted” mail is dated: Wed, 18 Dec 2024 18:22:14 +0000.

    https://tracker.debian.org/news/1595412/accepted-make-dfsg-441-1-source-into-unstable/

    barriere:/home/d-i/di/logs/di-autobuild_daily-amd64-20241222-0000 ends
    with an unhide call at 20241222-00:14

    barriere:/home/d-i/di/logs/di-autobuild_daily-amd64-20241223-0000 ends
    with an unhide call at 20241223-01:42, meaning 90 extra minutes.

    I'm not going to dive into that right now (my focus is trying to get the
    Trixie Alpha 1 train rolling), but I suspect this might be a case of
    make's being installed already (as opposed to build-deps), and maybe not getting upgraded by our tooling. Yet at some point we catch up with
    newer versions, possibly when chroots are getting rebuilt (once a week
    if memory serves). If that's confirmed, maybe it would be best to call dist-upgrade in the first place, to avoid delaying seeing the impacts of changes/regressions, and to make it easier to pinpoint/confirm theories
    when we see something strange?

    The same effect can be seen on at least platti and zelenka (there the
    build is very fast, but got bumped from 1-2 minutes to 10+), between the
    22nd and the 23rd as well.

    TL;DR: Yes, this issue impacts regular d-i daily builds and wastes time
    on porterboxes every night, it'd nice to rewrite the make introspection soon-ish. As mentioned (I think) earlier that doesn't affect regular
    builds, and therefore debian-installer uploads, so this can definitely
    wait until Trixie Alpha 1 (I have no idea yet whether other uploads are
    going to be required).


    Cheers,
    --
    Cyril Brulebois (kibi@debian.org) <https://debamax.com/>
    D-I release manager -- Release team member -- Freelance Consultant

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEtg6/KYRFPHDXTPR4/5FK8MKzVSAFAmdvT7MACgkQ/5FK8MKz VSCK2g/9GHfDWHsB8deZ2ZhiaUJoCR4nYNd8dPlyaJoWpHeKLXFDQhe2jCz4jdQN pQugvgROCttQ2GF3BrQwSC6kSsjsckdaznD5lU4lfNNyp7f0YiOlVbF42umo2dXq oy01GaYUdI3WjuAkO3UiAdCMXDcSO0TN7F+jOEIItXs1wTOYYnOvJPfhN6ehIKDP t4o9aG9uQyeY6FCTd6rkNFhsCn8uhpMXJNbc+ajxNv/lqhd8pTCmy6orNPzggumb 9hw+QWVU8ixHmu5Li2VRl+29eJWhvnCQ1ecRET8alnCDeRPjpAppEptqG14KD61V Jv8Gx7zWxgOpviguw3v/UWqRquO557MbClggeivB5YPl9yvptG+8Wp4kh3Gwx7sM 5k3TkReIptX6g4koL+JaVUQ0anKr0fIOmJ7M6hcf2qzqTaoUseCotkdVqpbyGAL/ K0tFC1kzCuq1mwjc2RFnh/bOJCeSsOSfUylmRHTnV8wUT9hd02vbUtHkwf480dAq Fvmnc0tJtZdmSbZk/ZmiZb8CV1p3otdhi+o4o4SMuj4KGLorIdgasTrU8heGnHUz 46f2oRr5n1ilXQ9UmZacqmjy4wwX9csMKQ6v0GG+Tr1s8gXmkwVgOJ2rxpcDL0d9 0VqijuP3t6P/RFD9Cgwy4MESsqRI/78+W7YXmApLds9zANdU1DY=
    =y2q1
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    *