• [gentoo-dev] RFC: Block ebuilds installing tests to ${D} by default

    From Robin H. Johnson@21:1/5 to All on Thu Feb 1 09:20:01 2024
    TL;DR:
    I'd like to propose a change where packages should NOT install their
    tests to ${D} by default. Such an install may optionally enabled with
    USE=test, which should be decoupled from FEATURES=test. Or depending on
    the color of the bikeshed, we add something new like USE=install-tests.

    Background:
    Python packages install a number of _test.py files, and related .pyc
    files. The files are generally useful for running tests after the
    package is installed, and may have additional testing dependencies that
    are not installed via RDEPEND.

    As an example, on the livegui install media, these files take 100MB+
    before squashfs compression.

    Some users MAY wish to verify that a package continues to function
    correctly, and they should have the USE=test dependencies available at
    runtime, and the tests installed.

    Such post-install testing may also require other files to be present, to configure the test suite runs.

    --
    Robin Hugh Johnson
    Gentoo Linux: Dev, Infra Lead, Foundation President & Treasurer
    E-Mail : robbat2@gentoo.org
    GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
    GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v2
    Comment: Robbat2 @ Orbis-Terrarum Networks - The text below is a digital signature. If it doesn't make any sense to you, ignore it.

    iQKTBAABCgB9FiEEveu2pS8Vb98xaNkRGTlfI8WIJsQFAmW7UylfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEJE RUJCNkE1MkYxNTZGREYzMTY4RDkxMTE5Mzk1RjIzQzU4ODI2QzQACgkQGTlfI8WI JsQw7g/9FbJeBISxF3ylZQ2hDxa2RH5lxYI+Gv75PgRpXaQAadzGkhok/pDqweIb 4B5vUWDGOeIDgXteAbIaFSZYm9sjvrsoU+vaQINtcwLkBfPbDSA+0PZP/9Un44C6 5RCAt93y0gWwhi65ggGqcStGYYmmJ8YO7h9MvE+9rlO5NEC6n6Z7s+w0b5oMv4uO 0zkhPqxzFZSVpu48Bg/5mkB2IeUbRaI3+5XA0G7Tkwcv6d/uvjLKiDqXVKHI6gmX R1YCKXbIky50xj3lVDOLMMRxKlziNLK9pwWslF3EShbgGUeBGVUVKynwjr5j8GrV GWH3pDurqAs20iYf7CkjuNvJpdWlGLnPGw3atajvpjm/9fvTLied21NR07hvq+Cx
    GYWRLDyT
  • From Mike Gilbert@21:1/5 to robbat2@gentoo.org on Thu Feb 1 20:50:01 2024
    On Thu, Feb 1, 2024 at 3:15 AM Robin H. Johnson <robbat2@gentoo.org> wrote:

    TL;DR:
    I'd like to propose a change where packages should NOT install their
    tests to ${D} by default. Such an install may optionally enabled with USE=test, which should be decoupled from FEATURES=test. Or depending on
    the color of the bikeshed, we add something new like USE=install-tests.

    Background:
    Python packages install a number of _test.py files, and related .pyc
    files. The files are generally useful for running tests after the
    package is installed, and may have additional testing dependencies that
    are not installed via RDEPEND.

    It sounds like you want to resurrect bug 531648.

    https://bugs.gentoo.org/531648

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?=@21:1/5 to Robin H. Johnson on Thu Feb 1 22:10:01 2024
    On Thu, 2024-02-01 at 08:15 +0000, Robin H. Johnson wrote:
    TL;DR:
    I'd like to propose a change where packages should NOT install their
    tests to ${D} by default.

    That sounds like reverting a lot of effort that has been cleaned up all
    over the past years.

    Such an install may optionally enabled with
    USE=test, which should be decoupled from FEATURES=test.

    This would prevent us from detecting files accidentally left over by FEATURES=test, as ago does right now. A few times this was actually
    helpful in identifying packages whose test suites wrote temporary files
    into site-packages directory, and they effectively ended up being
    installed.

    Or depending on
    the color of the bikeshed, we add something new like USE=install-tests.

    Background:
    Python packages install a number of _test.py files, and related .pyc
    files. The files are generally useful for running tests after the
    package is installed, and may have additional testing dependencies that
    are not installed via RDEPEND.

    This is not really relevant. This is the case of optional runtime
    dependencies (i.e. you don't have to have them installed when installing
    the test suite in question), and per policy PG-0001 these can be
    expressed using postinst messages or likewise.

    As an example, on the livegui install media, these files take 100MB+
    before squashfs compression.

    Some users MAY wish to verify that a package continues to function
    correctly, and they should have the USE=test dependencies available at runtime, and the tests installed.

    Such post-install testing may also require other files to be present, to configure the test suite runs.

    I suppose you are referring to dev-lang/python here. Unfortunately,
    removing tests from it is a non-trivial problem. As I've mentioned to
    you before, there are packages that actually import modules form the
    test directory.

    Remember that Gentoo has a policy of following upstream, and this policy
    is specifically targeted towards developers who expect Gentoo to
    be a good baseline environment for developing packages. By explicitly diverging from upstream default install by default, we are effectively
    creating an incompatible Python environment and requiring users to go
    through extra steps to restore upstream compatibility.

    What's worse, Python development is often done via virtual environments.
    For regular Python packages, all kinds of Gentoo divergence can be
    easily "reverted" by installing the upstream packages inside the virtual environment. However, this isn't the case for dev-lang/python itself --
    by stripping it down we're effectively creating virtual environments
    that are not fully functional and whose functionality cannot be restored without actually rebuilding dev-lang/python.

    On top of that, as you are also perfectly aware, stripping down tests is
    often non-trivial and either requires complex patches (we are talking of modifying the build system not to install something), or ugly hacks (we
    are talking about having Python install and compile modules, and
    removing them afterwards).

    Finally, "requiring other files to be present" effectively means adding
    USE dependencies. This is going to be a lot of effort, considering that effectively developers would have to run without test suites installed
    to detect missing dependencies, and then rebuild dev-lang/python every
    time they are about to run tests in a package that actually requires
    these files.

    Overall, I don't believe you've provided sufficient rationale to justify diverging from upstream, adding significant complexity to ebuilds
    and adding a lot of additional work to package maintainers who will be responsible for ensuring the correctness of package installations
    and USE dependencies.

    --
    Best regards,
    Michał Górny


    -----BEGIN PGP SIGNATURE-----

    iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAmW8Bz0SHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQOUC4H/3hjghyabfkHkttQVMCunbjV4UWpFPxM SKudqzH4s9ZZucM64j6dzy0+k4pjwqhhTvzLWl6kNbqch/hmiKWiq6l+zH4B8ysV o30vLmyMg2zSY9SqySKUvJ2dFu9dhewOmRfV6eWYA96k/UjWdbA2pKcLW8B7WEbw rLV4sKV+eShSujRA7dAfkMHQaNEHSnFEO6buAnutC7h7V0eN9BLuFEx0MNI232rP tHATlckA4o5mo/gTaxbInDZyebwNXv9jmU/clzMoMYCISgKul+NEBSiPUFsmQZPS tp4JqHw/aNkgUZSIifjccA+yoe5nJmTmrpiWfjgXMems6XD9Cr8s0FY=
    =FfbL
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mike Gilbert@21:1/5 to eschwartz93@gmail.com on Thu Feb 1 23:00:01 2024
    On Thu, Feb 1, 2024 at 4:38 PM Eli Schwartz <eschwartz93@gmail.com> wrote:

    On 2/1/24 4:03 PM, Michał Górny wrote:
    I suppose you are referring to dev-lang/python here. Unfortunately, removing tests from it is a non-trivial problem. As I've mentioned to
    you before, there are packages that actually import modules form the
    test directory.

    Remember that Gentoo has a policy of following upstream, and this policy
    is specifically targeted towards developers who expect Gentoo to
    be a good baseline environment for developing packages. By explicitly diverging from upstream default install by default, we are effectively creating an incompatible Python environment and requiring users to go through extra steps to restore upstream compatibility.


    The "following upstream" argument is extremely weak.

    eschwartz ~/git/cpython $ ./configure --help| grep test
    --disable-test-modules don't build nor install test modules


    Clearly, upstream is giving you loud and clear permission to refrain
    from installing this. They even have a split in their Makefile variables:
    - LIBSUBDIRS: things that are always installed
    - TESTSUBDIRS: test files that you can disable

    Ah, that's a relatively new configure option added in Python 3.10. It
    didn't exist back in 2015 when I was looking into this.

    https://docs.python.org/3/whatsnew/3.10.html#build-changes

    Thanks for pointing it out.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas K. Huettel@21:1/5 to All on Fri Feb 2 00:07:35 2024
    Copy: robbat2@gentoo.org (Robin H. Johnson)

    Am Donnerstag, 1. Februar 2024, 09:15:39 CET schrieb Robin H. Johnson:
    TL;DR:
    I'd like to propose a change where packages should NOT install their
    tests to ${D} by default. Such an install may optionally enabled with USE=test, which should be decoupled from FEATURES=test. Or depending on
    the color of the bikeshed, we add something new like USE=install-tests.

    I see where you come from, but the decision what precisely to install
    (when we do not follow upstream) can be very non-trivial.

    I'm not familiar with Python, but for Perl there is quite some test infrastructure in the main Perl package... Then there are regular
    Perl packages that are extensions to the test suite. These would require
    the test modules from core Perl then? I really wouldnt want to figure
    out what to keep and what to drop, and spend a lot of effort getting
    the dependencies right.

    Also this is an infinite source of upstream "It's Gentoo, we don't
    support that because they do weird stuff" bugs.

    tl;dr: no



    --
    Andreas K. Hüttel
    dilfridge@gentoo.org
    Gentoo Linux developer
    (council, comrel, toolchain, base-system, perl, libreoffice) https://wiki.gentoo.org/wiki/User:Dilfridge
    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCAAdFiEE/Rnm0xsZLuTcY+rT3CsWIV7VQSoFAmW8JDcACgkQ3CsWIV7V QSpJdRAAlTmEOyPxBerd1Fk2UDQGF4Dis+eTD6Zb7aLFMxNFskeYw5MZIZMFsUBb 8bQnE3cLBC3KcbwgVSGiSXhLy3Vnd9/hQMzVgERS7sRt36lupJxjJcu70DwN2BeA 2cF+13PmCT/9pG575KO3nNeEjmUVfe/4v3/KXiktuvyr6lIX6Lb6f1fmgWkUoo5y DjIzb6dKve9JYwo5PyARcPPOhQTF4sHIUZgRkuBFi4vhvyIVajPn748jRjbmQ3A8 3MzBpFCH10SzM5LK4F/Uk0Ns84uZTfizCuZFzIvS/UYXkBOyZ0qxZ+hHHPzm9nyM KpaCRkhVBgwmOmwIfzXTAO0YXnM/a3u87iUTrc83TREX00gBGcLCTKvANWWjC5sM LvjInaK/F47ivuUM/QQR0OqYpITNlYqW/pk1nr9TBvTk/jrmLW6qES8OfQsdtLTW uOhk5gk8b8d3AnRaTXJenkyuslGKqqJSBYPQwcOA+UJ/E2l/42btGCVk9QOxRc2x 9d/wpwD4znxgNlMAxI9vrv7dwrfpDnI7+2UAvghlJguJfhEFFLn5mdLGTrORDVxI cgBIXeLkGFCt73Ke/NlAuTb51l88PmfvBPeUyA1lugUcDSS/JQDgx73Qka2/6zEm RccDPqzYHGkYMm8Z0yyiR42NmZUah8Kf8myD7B5FcqJYIUYD6q8=
    =IhyU
    -----END PGP SIGNATURE----