• Re: Bug#1091197: ITP: python-genson -- user-friendly JSON Schema genera

    From Simon Josefsson@21:1/5 to All on Thu Dec 26 01:30:01 2024
    Hi. I'm struggling with packaging for this package:

    https://salsa.debian.org/python-team/packages/python-genson

    On the 'debian/latest' branch there is a "normal" packaging (except that
    it ignore self-check errors), and there are two related problems:

    https://salsa.debian.org/python-team/packages/python-genson/-/jobs/6818093

    1) Self-checks cannot find the 'genson.schema' module.

    E ModuleNotFoundError: No module named 'genson.schema'

    2) The resulting package doesn't have the genson/schema sub-directory:

    jas@kaka:~/dpkg/python-genson$ curl --silent 'https://salsa.debian.org/python-team/packages/python-genson/-/jobs/6818096/artifacts/raw/aptly/pool/main/p/python-genson/python3-genson_1.3.0-1+salsaci+20241226+4_all.deb'|dpkg -c -|grep dist-packages/genson/
    drwxr-xr-x root/root 0 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/
    -rw-r--r-- root/root 347 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/__init__.py
    -rw-r--r-- root/root 5877 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/__main__.py
    jas@kaka:~/dpkg/python-genson$

    On the 'hack' branch I made a crude attempt at resolving this, and
    indeed self-checks are happy, even autopkgtest, and the package contains
    the sub-directory:

    https://salsa.debian.org/python-team/packages/python-genson/-/pipelines/786628

    jas@kaka:~/dpkg/python-genson$ curl --silent 'https://salsa.debian.org/python-team/packages/python-genson/-/jobs/6818159/artifacts/raw/aptly/pool/main/p/python-genson/python3-genson_1.3.0-1+salsaci+20241226+6_all.deb'|dpkg -c -|grep dist-packages/genson/
    drwxr-xr-x root/root 0 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/
    -rw-r--r-- root/root 347 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/__init__.py
    -rw-r--r-- root/root 5877 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/__main__.py
    drwxr-xr-x root/root 0 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/
    -rw-r--r-- root/root 0 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/__init__.py
    -rw-r--r-- root/root 5423 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/builder.py
    -rw-r--r-- root/root 4741 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/node.py
    drwxr-xr-x root/root 0 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/
    -rw-r--r-- root/root 522 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/__init__.py
    -rw-r--r-- root/root 2109 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/array.py
    -rw-r--r-- root/root 2196 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/base.py
    -rw-r--r-- root/root 3276 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/object.py
    -rw-r--r-- root/root 1891 2024-12-23 12:52 ./usr/lib/python3/dist-packages/genson/schema/strategies/scalar.py
    jas@kaka:~/dpkg/python-genson$

    But the patch is really ugly! See below. It copies files from the
    source direcory into the pybuild internal path, including hard coded
    version numbers. Pybuild should have copied these files by itself.

    Why doesn't pybuild do the right thing by default?

    Any ideas how to resolve?

    Feel free to clone the project and/or push to it to experiment. I'll
    take a little break from this package now...

    jas@kaka:~/dpkg/python-genson$ git diff debian/latest..hack
    diff --git a/debian/rules b/debian/rules
    index a1954a9..1658f7b 100755
    --- a/debian/rules
    +++ b/debian/rules
    @@ -3,6 +3,7 @@
    include /usr/share/dpkg/pkg-info.mk # DEB_VERSION

    #export PYBUILD_NAME = genson
    +export PYBUILD_TEST_ARGS= -k 'not test_no_input'

    %:
    dh $@ --buildsystem=pybuild
    @@ -10,8 +11,9 @@ include /usr/share/dpkg/pkg-info.mk # DEB_VERSION
    B = $(CURDIR)/debian/tmp/usr/bin
    M = $(CURDIR)/debian/tmp/usr/share/man/man1

    -override_dh_auto_test:
    - -dh_auto_test $(DH_BUILD_OPTS)
    +execute_before_dh_auto_test:
    + cp -rv genson/schema .pybuild/cpython3_3.12/build/genson/
    + cp -rv genson/schema .pybuild/cpython3_3.13/build/genson/

    execute_after_dh_auto_install:
    ifeq (,$(filter nodoc,$(DEB_BUILD_PROFILES)))
    jas@kaka:~/dpkg/python-genson$

    /Simon

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZ2yiFxQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFoosvAPkBj8EzymSoTcUztgHPc7tniDbHMcbW 31QEcv+juBBRiwD/c+cJsGtaGAFjVFm5B2/N
  • From Andrey Rakhmatullin@21:1/5 to All on Thu Dec 26 08:10:01 2024
    Wasn't the proposed fix "export PYBUILD_NAME as the docs say"? I see you
    are not doing this.


    --
    WBR, wRAR

    -----BEGIN PGP SIGNATURE-----

    iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmds//gtFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh jE4P/2iOGm2fDwDx/mMTpUV/SByPE7669ZXtR/5/mA7BzzpJAhxP9OqC8PfYP1r8 3PnMxyB2gTrEmW1I8Dk0RzQ5dO6meVX/VYEYIzIK68SfnxnYxlZVzMkdHdZQpWrC GNSDbSsztOttMiTTAfhPNcuwG9wntsQwmhG1Ln/Ck9CwDkcKUzNMyo/H/86WNHvQ 4cZN0MunMEBu3lP8nDSGu5BHD6bI2sv2kTy3UuZ7643uxQg1jQsL8KsbnJgoLg8j zzyBAvyGBQkUhaNaFmn8JwMhrDyZZGFJIR6tnJcQEcyp6zyI4gB2Dlv6aLXY+R3E SRaza1QSrMbDhKOTOpbcehUpszb93O80MqUK259JvD38PbPPcwMezo2ebPYi2dIN XpszY4c3SFT6OOFl9YKKk0a5XvNupI3MxFL+dAIESz02u25+L6v6Ubbxaj97186k OGT76eLL1h7u6LbwU9VBldFcNg7My7+9iQLyCYLb71cyo/wXmbFf5yWEB4fwT9Wz I8ZXPFeOBvPyd9Ijg8eXpg7t6NruS7vp117vF195yuzjJKGOcXbQc2XybJqFdeTy 7pcy9DHWKe0dhB+j5/xmoRZOGNcSCHX7C4MrXGTHAPZpBKcKwWVHURiYbJLrw3zD 9ys5nIYoDQGOkjAuk+2HtURHNGt8QCs3dbujveiVZ8qvBa43
    =W3OT
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carsten Schoenert@21:1/5 to All on Thu Dec 26 08:50:02 2024
    Hello Simon,

    Am 26.12.24 um 02:23 schrieb Simon Josefsson:
    Hi. I'm struggling with packaging for this package:

    https://salsa.debian.org/python-team/packages/python-genson

    ...
    Why doesn't pybuild do the right thing by default?

    on which base pybuild could make the right decision? pybuild is doing
    what it need and should do.

    The missing installation of the subfolder looks to me like an upstream
    issue.
    If the folder is needed for later usage it needs to get installed by setuptools.

    Any ideas how to resolve?

    Feel free to clone the project and/or push to it to experiment. I'll
    take a little break from this package now...

    jas@kaka:~/dpkg/python-genson$ git diff debian/latest..hack
    diff --git a/debian/rules b/debian/rules
    index a1954a9..1658f7b 100755
    --- a/debian/rules
    +++ b/debian/rules
    @@ -3,6 +3,7 @@
    include /usr/share/dpkg/pkg-info.mk # DEB_VERSION

    #export PYBUILD_NAME = genson
    +export PYBUILD_TEST_ARGS= -k 'not test_no_input'

    %:
    dh $@ --buildsystem=pybuild
    @@ -10,8 +11,9 @@ include /usr/share/dpkg/pkg-info.mk # DEB_VERSION
    B = $(CURDIR)/debian/tmp/usr/bin
    M = $(CURDIR)/debian/tmp/usr/share/man/man1

    -override_dh_auto_test:
    - -dh_auto_test $(DH_BUILD_OPTS)
    +execute_before_dh_auto_test:
    + cp -rv genson/schema .pybuild/cpython3_3.12/build/genson/
    + cp -rv genson/schema .pybuild/cpython3_3.13/build/genson/

    execute_after_dh_auto_install:
    ifeq (,$(filter nodoc,$(DEB_BUILD_PROFILES))) jas@kaka:~/dpkg/python-genson$

    You can simplify this all by using the variable PYBUILD_BEFORE_TEST

    $ git diff debian/rules
    diff --git a/debian/rules b/debian/rules
    index a1954a9..77ab492 100755
    --- a/debian/rules
    +++ b/debian/rules
    @@ -2,7 +2,9 @@

    include /usr/share/dpkg/pkg-info.mk # DEB_VERSION

    -#export PYBUILD_NAME = genson
    +export PYBUILD_NAME = genson
    +export PYBUILD_BEFORE_TEST = cp -rv {dir}/genson/schema {build_dir}/$(PYBUILD_NAME)
    +export PYBUILD_TEST_ARGS= -k 'not test_no_input'

    %:
    dh $@ --buildsystem=pybuild
    @@ -10,9 +12,6 @@ include /usr/share/dpkg/pkg-info.mk # DEB_VERSION
    B = $(CURDIR)/debian/tmp/usr/bin
    M = $(CURDIR)/debian/tmp/usr/share/man/man1

    -override_dh_auto_test:
    - -dh_auto_test $(DH_BUILD_OPTS)
    -
    execute_after_dh_auto_install:
    ifeq (,$(filter nodoc,$(DEB_BUILD_PROFILES)))
    mkdir -pv $(M)


    And I personally would introduce another small package with just the
    binary genson. To mee it's totally fine it's serverd by the package python3-genson.
    It's done very often within other Python binary packages.

    And if you define and use own variables in the file d/rules please
    speaking namings. B and M are hard to read and for me using one
    character variable name is a bad habit.

    I would drop them completely, they are used just one time!

    --
    Regards
    Carsten Schönert

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon Josefsson@21:1/5 to Andrey Rakhmatullin on Thu Dec 26 12:10:01 2024
    Andrey Rakhmatullin <wrar@debian.org> writes:

    Wasn't the proposed fix "export PYBUILD_NAME as the docs say"? I see you
    are not doing this.

    Thanks for reply! I added that but commented out since adding it did
    not change the pybuild behaviour. The extra genson/schema/ files are
    not built and installed, only the top-level genson/ directory which is
    the case even without PYBUILD_NAME.

    /Simon

    -----BEGIN PGP SIGNATURE-----

    iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZ204GBQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFomrsAQC5yp2FdYEPQb3cA60UwNxY4pc9l+wl gOej6O+ZCzS39gEA1I3vet1m7YbYz+6h5pWsrLqyMgNywsvJ7pyaASIr6gc=
    =Ppic
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andrey Rakhmatullin@21:1/5 to Simon Josefsson on Thu Dec 26 13:00:01 2024
    On Thu, Dec 26, 2024 at 12:03:51PM +0100, Simon Josefsson wrote:
    Andrey Rakhmatullin <wrar@debian.org> writes:

    Wasn't the proposed fix "export PYBUILD_NAME as the docs say"? I see you are not doing this.

    Thanks for reply! I added that but commented out since adding it did
    not change the pybuild behaviour. The extra genson/schema/ files are
    not built and installed, only the top-level genson/ directory which is
    the case even without PYBUILD_NAME.

    I've rechecked and the proposed fix also included "also, note the big fat warnings about upstream not configuring setuptools correctly".
    As I've just checked, running `python3 -m build` in the upstream repo also produces a wheel without the schema subpackage.



    --
    WBR, wRAR

    -----BEGIN PGP SIGNATURE-----

    iQJhBAABCgBLFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAmdtRQ8tFIAAAAAAFQAP cGthLWFkZHJlc3NAZ251cGcub3Jnd3JhckBkZWJpYW4ub3JnAAoJEDNi9wMaSZLh RrsP/04KXyww6BVXdRhXX/7BY1tUsPoqy1rPvMfrAq8UZeAodraWuMc4E4JlsnHV MLOfnH4efAhqCRXBy1PW0ixsh3pmemQXxE8NRNIDhJH0+8dyCv78e041FTH13QBZ vpsh/dkOqRmUiKFRhgqBb1b7aSDZHeXqL0N06D+4x0wjwR8xwEs1IotnwIcJnJxi eUXHVIauMPDqDIzuQmkSsRkX2lReqTPqeQspHxUGlW8rG4acTvykQR/sPMJGNeVV lWw7F6lknzi/TtIDPNpYJMbyCIA/APjYR5bGA7lePZ+xYjVJSDqXgoEWd0ucCzXV 8kA1pbJ0s1X1ofKmXviWSZZZyvSKioJH7Y+tFtEhWL6SbaC8M3vZirwEdBmPsqu1 FCkToWWvhkjQdCBH/GZi4T55gfgt8+vjZ7vf5aX6W9PWrIDIihR1OZvnPHuML7iC Sn/lis8KITCVVYDhmgWuS5De0dHAssydR8BFcz8dqoXBAP5OtVx+oRM3rynI11bE NRfDD2CkE4UUY1W46zHxCxKxO1Umxq51PYUZlK70BUR2xMQi/PoFPcMFWvXaaFjO da/8ZaBtY9Lz7iiAIL3lIO3hD0PTUruDq6QrZCxsJtcLrW3x2mpEiTU6hN1GYkmh aVh/QIl7SnlgNYgrEfHR/IYke1DIB4SSst2/qDyIhMUqQ9KO
    =hLqY
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon Josefsson@21:1/5 to Carsten Schoenert on Thu Dec 26 12:50:01 2024
    Many thanks for a helping hand!

    Carsten Schoenert <c.schoenert@t-online.de> writes:

    Hello Simon,

    Am 26.12.24 um 02:23 schrieb Simon Josefsson:
    Hi. I'm struggling with packaging for this package:
    https://salsa.debian.org/python-team/packages/python-genson

    ...
    Why doesn't pybuild do the right thing by default?

    on which base pybuild could make the right decision? pybuild is doing
    what it need and should do.

    The missing installation of the subfolder looks to me like an upstream
    issue.
    If the folder is needed for later usage it needs to get installed by setuptools.

    Yes I suspected this may be an upstream issue, but I'm not familiar
    enough with python build systems to understand how to debug or have
    confidence in a upstream bug report. I tried to channel this forward:

    https://github.com/wolverdude/GenSON/issues/80

    +execute_before_dh_auto_test:
    + cp -rv genson/schema .pybuild/cpython3_3.12/build/genson/
    + cp -rv genson/schema .pybuild/cpython3_3.13/build/genson/

    You can simplify this all by using the variable PYBUILD_BEFORE_TEST

    Which is not documented in pybuild(1) but now I found time to read https://wiki.debian.org/Python/Pybuild

    +export PYBUILD_BEFORE_TEST = cp -rv {dir}/genson/schema {build_dir}/$(PYBUILD_NAME)

    Yay thank you! This works and is cleaner than my hack.

    All is not perfect, now autopkgtest fail with a different error. The
    strange thing is that this didn't happen in my 'hack' branch which ought
    to be fairly similar, except for maybe PYTHON_BUILD_NAME and the
    packaging merge. The autopkgtest was successful there. Ideas?

    https://salsa.debian.org/python-team/packages/python-genson/-/jobs/6820031

    ___________________ ERROR collecting test/test_add_multi.py ____________________
    ImportError while importing test module '/tmp/autopkgtest-lxc.394h6eqm/downtmp/autopkgtest_tmp/build/test/test_add_multi.py'.
    Hint: make sure your test modules/packages have valid Python names.
    Traceback:
    /usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level) test/test_add_multi.py:1: in <module>
    from . import base
    test/base.py:4: in <module>
    from genson import SchemaNode, SchemaBuilder
    E ImportError: cannot import name 'SchemaNode' from 'genson' (/tmp/autopkgtest-lxc.394h6eqm/downtmp/autopkgtest_tmp/build/genson/__init__.py)

    And I personally would introduce another small package with just the
    binary genson. To mee it's totally fine it's serverd by the package python3-genson.
    It's done very often within other Python binary packages.

    Done!

    In the Go team the preference is the reverse, I believe, but I suppose
    the key difference for Python is that /usr/bin binaries end up being Architecture:any for Go programs but for Python they are still
    Architecture:all so there is no duplication of /usr/lib Python code for
    all the architectures which there would be for /usr/lib Go code if library+program are put in the same Debian binary package. I hadn't
    realized this difference can influence the packaging style so much.

    This decision makes lintian unhappy:

    X: python3-genson: application-in-library-section python [usr/bin/genson]
    N:
    N: This package contains a binary in $PATH but is in a section just thought N: for libraries. It likely should be in another section like e.g. utils,
    N: text, devel, misc, etc., but not in e.g. perl, ruby or python.
    N:
    N: People tend to skip these package sections when looking for applications N: in the package list and hence wouldn't notice this package.
    N:
    N: In case the program in $PATH is only a helper tool and the package is
    N: primarily a library, please add a Lintian override for this tag.
    N:
    N: Visibility: info
    N: Show-Always: no
    N: Check: application-not-library
    N: This tag is experimental.

    It also makes deciding the section harder for the resulting combined tool+library package:

    X: python3-genson: library-package-name-for-application [usr/bin/genson]
    N:
    N: This package contains a program in $PATH but is named like a library. E.g. N: instead of libfoo-bar-perl it should be named just foo-bar.
    N:
    N: People tend to skip library-like named packages when looking for
    N: applications in the package list and hence wouldn't notice this package. N: See the reference for some (not perl-specific) reasoning.
    N:
    N: In case the program in $PATH is only a helper tool and the package is
    N: primarily a library, please add a Lintian override for this tag.
    N:
    N: Please refer to
    N: https://perl-team.pages.debian.net/policy.html#Package_Naming_Policy for N: details.
    N:
    N: Visibility: info
    N: Show-Always: no
    N: Check: application-not-library
    N: This tag is experimental.

    I don't think 'genson' is a helper tool since it is generally useful for developers, so Section:devel felt appropriate but that triggers another
    lintian warning:

    I: python3-genson: wrong-section-according-to-package-name devel => python
    N:
    N: This package has a name suggesting that it belongs to a section other than N: the one it is currently categorized in.
    N:
    N: Visibility: info
    N: Show-Always: no
    N: Check: fields/section

    Is the pattern to silence such lintian complaints, or just ignore them
    in the Python team?

    And if you define and use own variables in the file d/rules please
    speaking namings. B and M are hard to read and for me using one
    character variable name is a bad habit.

    I would drop them completely, they are used just one time!

    Done.

    /Simon

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZ21B+BQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFoqvKAQCVv/IQ6erxc99K76nuL2KhCw+qkxeD 1Bu8fRvYTrCjxQEAmR6GRTYAHkF4dQ2cpxIci7TvLHEOOPVWGnZDrWaFBQkux
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon Josefsson@21:1/5 to Andrey Rakhmatullin on Thu Dec 26 13:50:01 2024
    Andrey Rakhmatullin <wrar@debian.org> writes:

    On Thu, Dec 26, 2024 at 12:03:51PM +0100, Simon Josefsson wrote:
    Andrey Rakhmatullin <wrar@debian.org> writes:

    Wasn't the proposed fix "export PYBUILD_NAME as the docs say"? I see you >> > are not doing this.

    Thanks for reply! I added that but commented out since adding it did
    not change the pybuild behaviour. The extra genson/schema/ files are
    not built and installed, only the top-level genson/ directory which is
    the case even without PYBUILD_NAME.

    I've rechecked and the proposed fix also included "also, note the big fat warnings about upstream not configuring setuptools correctly".
    As I've just checked, running `python3 -m build` in the upstream repo also produces a wheel without the schema subpackage.

    Thanks for testing. Yes so maybe the best is if upstream make
    genson/schema/ installed by default, and maybe that will solve the
    autopkgtest failure too.

    Or we can look into reverting back something based on my 'hack'
    approach, which is pretty much the same as the PYBUILD_BEFORE_TEST
    command, that made autopkgtest happy.

    I didn't understand which big fat warning this was? I looked in the
    build logs.

    /Simon

    --=-=-Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iIoEARYIADIWIQSjzJyHC50xCrrUzy9RcisI/kdFogUCZ21PRBQcc2ltb25Aam9z ZWZzc29uLm9yZwAKCRBRcisI/kdFomI6AQDnivOz0spPe413EZqWfUZuZAyxG0W7 Js3PwZAjdi6yrgEAuRBA32LVCD0gwJNDRuILL+soxXMyvIaTztj4AQ6OYgElrR
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Simon Josefsson@21:1/5 to Andrey Rakhmatullin on Mon Jan 27 11:20:01 2025
    Hi

    Could someone help me and upstream how to get python setuptools to get
    all files installed for this package?

    Please see this thread and upstream bug report:

    https://github.com/wolverdude/GenSON/issues/80

    I work around the problem in the debian packaging like this:

    export PYBUILD_BEFORE_TEST = \
    cp -rv {dir}/genson/schema {build_dir}/$(PYBUILD_NAME)

    However installing these files should be done by upstream setuptools
    magic somehow. Some hint or ideally an upstream merge request to fix
    this would be appreciated...

    /Simon

    Andrey Rakhmatullin <wrar@debian.org> writes:

    Thanks for reply! I added that but commented out since adding it did
    not change the pybuild behaviour. The extra genson/schema/ files are
    not built and installed, only the top-level genson/ directory which is
    the case even without PYBUILD_NAME.

    I've rechecked and the proposed fix also included "also, note the big fat warnings about upstream not configuring setuptools correctly".
    As I've just checked, running `python3 -m build` in the upstream repo also produces a wheel without the schema subpackage.

    Carsten Schoenert <c.schoenert@t-online.de> writes:

    Why doesn't pybuild do the right thing by default?

    on which base pybuild could make the right decision? pybuild is doing
    what it need and should do.

    The missing installation of the subfolder looks to me like an upstream
    issue.
    If the folder is needed for later usage it needs to get installed by setuptools.

    /Simon

    -----BEGIN PGP SIGNATURE-----

    iQNoBAEWCAMQFiEEo8ychwudMQq61M8vUXIrCP5HRaIFAmeXW+sUHHNpbW9uQGpv c2Vmc3Nvbi5vcmfCHCYAmDMEXJLOtBYJKwYBBAHaRw8BAQdACIcrZIvhrxDBkK9f V+QlTmXxo2naObDuGtw58YaxlOu0JVNpbW9uIEpvc2Vmc3NvbiA8c2ltb25Aam9z ZWZzc29uLm9yZz6IlgQTFggAPgIbAwULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgBYh BLHSvRN1vst4TPT4xNc89jjFPAa+BQJl/YgIBQkLehFUAAoJENc89jjFPAa+CboA +wUa06RD5e5VTCxvSWtPS75Wq2qBeYGZnf0jvUMxa2n4AP4xkUeAPPnNuMsTm2fs FCDIGaEM2Yn6Vb2huzzT1Fw/BLgzBFySz4EWCSsGAQQB2kcPAQEHQOxTCIOaeXAx I2hIX4HK9bQTpNVei708oNr1Klm8qCGKiPUEGBYIACYCGwIWIQSx0r0Tdb7LeEz0 +MTXPPY4xTwGvgUCZf2IKwUJC3oQqgCBdiAEGRYIAB0WIQSjzJyHC50xCrrUzy9R cisI/kdFogUCXJLPgQAKCRBRcisI/kdFoqdMAQCgH45aseZgIrwKOvUOA9QfsmeE 8GZHYNuFHmM9FEQS6AD6A4x5aYvoY6lo98pgtw2HPDhmcCXFItjXCrV4A0GmJA4J ENc89jjFPAa+GcYA/26YQY05bLtnXiIjTiAzrGQrRXxTHPA8Av7TDFHvIetWAP9s HSoU8OfTwmTiEnGwLlsV7QJclZg3YNz/Ypcp9TqQBrg4BFySz2oSCisGAQQBl1UB BQEBB0AxlRumDW6nZY7A+VCfek9VpEx6PJmdJyYPt3lNHMd6HAMBCAeIfgQYFggA JgIbDBYhBLHSvRN1vst4TPT4xNc89jjFPAa+BQJl/YgwBQkLehDGAAoJENc89jjF PAa+phoA/jrDqIrl/55vUMBhIQv+TP635d2iCTEnyFmbUcP9+gh6APoDsXalVd2c OGxQtSC+TF8PkZMn1TLkJKAjVxr+xx40AgAKCRBRcisI/kdFolQOAP0QEGKjA5H/ /LplNwG/rP0Ei8QeCDdl1MM6LoLv+UMtMgEAzIx6NFG7HcAluUKvkNTp+/31ZUq1 WMtXl48k2teaIgo=
    =BV0l
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carsten Schoenert@21:1/5 to All on Mon Jan 27 15:10:01 2025
    Hello Simon,

    Am 27.01.25 um 12:11 schrieb Simon Josefsson:
    Hi

    Could someone help me and upstream how to get python setuptools to get
    all files installed for this package?

    Please see this thread and upstream bug report:

    https://github.com/wolverdude/GenSON/issues/80

    I work around the problem in the debian packaging like this:

    export PYBUILD_BEFORE_TEST = \
    cp -rv {dir}/genson/schema {build_dir}/$(PYBUILD_NAME)

    However installing these files should be done by upstream setuptools
    magic somehow. Some hint or ideally an upstream merge request to fix
    this would be appreciated...

    I'm also not a specialist with setuptools or any other Python build tool.

    I've tried this small modifications and at least the wheel and the sdist tarball is then including all data from the genson/ folder.

    $ git diff
    diff --git a/setup.cfg b/setup.cfg
    index 3c9e75f..244f310 100644
    --- a/setup.cfg
    +++ b/setup.cfg
    @@ -32,10 +32,12 @@ classifiers =
    Topic :: Utilities

    [options]
    -packages = genson
    zip_safe = True
    include_package_data = True

    +[options.packages.find]
    +where=genson
    +
    [options.entry_points]
    console_scripts = genson = genson.__main__:main

    A build log (in the upstream source tree) looks then this:

    $ python3 -m build
    * Creating isolated environment: venv+pip...
    * Installing packages in isolated environment:
    - setuptools>=44
    - wheel
    * Getting build dependencies for sdist...
    No `packages` or `py_modules` configuration, performing automatic discovery. `flat-layout` detected -- analysing .
    discovered packages -- ['genson', 'genson.schema', 'genson.schema.strategies']
    * Building sdist...
    No `packages` or `py_modules` configuration, performing automatic discovery. `flat-layout` detected -- analysing .
    discovered packages -- ['genson', 'genson.schema', 'genson.schema.strategies']
    * Building wheel from sdist
    * Creating isolated environment: venv+pip...
    * Installing packages in isolated environment:
    - setuptools>=44
    - wheel
    * Getting build dependencies for wheel...
    No `packages` or `py_modules` configuration, performing automatic discovery. `flat-layout` detected -- analysing .
    discovered packages -- ['genson', 'genson.schema', 'genson.schema.strategies']
    * Building wheel...
    No `packages` or `py_modules` configuration, performing automatic discovery. `flat-layout` detected -- analysing .
    discovered packages -- ['genson', 'genson.schema', 'genson.schema.strategies']

    --
    Regards
    Carsten

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Dmitry Shachnev@21:1/5 to Simon Josefsson on Mon Jan 27 15:20:01 2025
    Hi Simon!

    On Mon, Jan 27, 2025 at 11:11:54AM +0100, Simon Josefsson wrote:
    Hi

    Could someone help me and upstream how to get python setuptools to get
    all files installed for this package?

    Please see this thread and upstream bug report:

    https://github.com/wolverdude/GenSON/issues/80

    I work around the problem in the debian packaging like this:

    export PYBUILD_BEFORE_TEST = \
    cp -rv {dir}/genson/schema {build_dir}/$(PYBUILD_NAME)

    You can also use debian/pybuild.testfiles for that. See pybuild(1) for
    details.

    However installing these files should be done by upstream setuptools
    magic somehow. Some hint or ideally an upstream merge request to fix
    this would be appreciated...

    Debian runs tests before the install phase (but after the build phase).

    And the tests are run from the build directory, not from the target installation directory. So setuptools does not copy all files there.

    --
    Dmitry Shachnev

    -----BEGIN PGP SIGNATURE-----

    iQIzBAABCgAdFiEEq2sdvrA0LydXHe1qsmYUtFL0RrYFAmeXlXwACgkQsmYUtFL0 RraKPg/9GRP5ZT3qe+vf6gMrKowr6rBxj4m2KXwWwH+dFwe2oX1n83JbfFxvWXeC xsxo71SWr6zUjefGwjpHd9+CTSe/BMOEjyW1aUvuL3KuyEahw4GcRtsV3tLhfQT1 n1GEkCL8RrbSHdVt4i6IYJyEj6A7DAg+IdLoE1y9b18L2geLxuumQte6+BIiuhRM ZERV5+uX/eQdIT+DCcFu8Fmgc29fWbZ7SCuId1NnBvVyngyrV4phslA/zKStekzy +H+rUj3qUQggh/1HN1bhgTjVz5TUv2CBfAzDMnaw/w+31N8tLJO4guLQiRb/i60z ACOgOxm4mmSYkFmGItG0/JbK086avg5YGVPwIWXB2UhmvJGt/5uCZvW+RsXr7oBQ Fs8Dy5vfWenIxAwadz1IgEydMmGKxmUuO2zN90WfwpF3NtWBTVRjAp/z4yGIy1g4 yD09XAtNM3ExSK9g/m0r4A5JS3byjaT1GZR5vsHaAhBoGJVyJdS4tOpHVbVvg0iA Yq6UIBYChQMBUlD3PiHBeV4/75WYqpeamwxaC2VC+TrDylOdH1s+XHeyCoCqrQ3y gRlDy8k4DaYTsQqgUMJUIjABywFhm09iCIo2/7qvgIOlJgiXx9XpOPryXaMWlpi9 tGnxp61BRWce6lvyhpFctTNjpQ6r1LSQV0PFhou0RrPa+O9snYY=
    =Yr4o
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)