Unfortunately, my recent cargo.eclass changes didn't fare so well, despite me testing it on a good handful of packages. I've had to sharply change tack to work around Cargo's shortcomings.

The first commit has the main change. I'd like to merge that very soon to put out the immediate fire.

The second commit tries to address a lingering downside where
project-specific flags may get overridden by our own. This was happening,
even before I changed anything for users or ebuilds setting RUSTFLAGS, but
now it is more likely to happen. I'm unsure of this change because it is a little controverial, using app-misc/yq's tomlq (a jq wrapper) to read flags from the project's config files. On the plus side, it should avoid upstreams complaining that we're not using their flags. See what you think. I've kept
it in a separate commit so that it can easily be backed out.

James Le Cuirot (2):
cargo.eclass: Change RUSTFLAGS approach following recent build
failures
cargo.eclass: Preserve project-specific [build] flags by reading
config

eclass/cargo.eclass | 73 ++++++++++++++++++++++++++++-----------------
1 file changed, 46 insertions(+), 27 deletions(-)

--
2.45.2

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Cargo turned out to be even worse at handling flags than I thought. Target-specific flags set by projects were overriding our own, and Cargo
was bailing out when faced with merging a string of flags with an array
of flags.

After weighing up the poor set of options, I've found that it is better
to always set flags via a target-specific environment variable for
reasons set out in comments added here. This approach was previously
just used for cross-compiling, but now we do it unconditionally.

It has the downside of overriding generic [build] flags set by projects,
but these were already being overridden by users and ebuilds setting
RUSTFLAGS themselves.

Closes: https://bugs.gentoo.org/937453
Closes: https://bugs.gentoo.org/937470
Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 65 +++++++++++++++++++++++++++------------------
1 file changed, 39 insertions(+), 26 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index c360c2a6c419..6d4cf1b425eb 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -259,19 +259,6 @@ cargo_crate_uris() {
cargo_gen_config() {
debug-print-function ${FUNCNAME} "$@"

- # The default linker is "cc" so override by setting linker to CC in the - # RUSTFLAGS. The given linker cannot include any arguments, so split these
- # into link-args along with LDFLAGS. Also include external RUSTFLAGS.
- # Note that as of Rust 1.80, the build host RUSTFLAGS are ignored when - # cross-compiling unless you use the unstable host-config feature available
- # with USE=nightly. There is no simple way around this.
- tc-export_build_env
- local LD_A=( $(tc-getBUILD_CC) ${BUILD_LDFLAGS} )
- local MY_BUILD_RUSTFLAGS="-C strip=none -C linker=${LD_A[0]}"
- [[ ${#LD_A[@]} -gt 1 ]] && MY_BUILD_RUSTFLAGS+="$(p

The flags we set an a target-specific environment variable override any
generic [build] flags set by the project, requiring ebuilds to set these themselves, which is undesirable. Work around this by using tomlq to
read the flags from the config files checked by Cargo and prepending
them to our environment variable.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6d4cf1b425eb..dea8c49e4585 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -36,7 +36,8 @@ esac

inherit flag-o-matic multiprocessing rust-toolchain toolchain-funcs

-[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND}"
+# app-misc/yq is needed for tomlq.
+[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND} app-misc/yq"

IUSE="${IUSE} debug"

@@ -566,21 +567,26 @@ cargo_env() {
# It has been common for users and ebuilds to set RUSTFLAGS, which would
# have overridden whatever a project sets anyway, so the least-worst option
# is to include those RUSTFLAGS in target-specific config here, which will
- # merge with any the project sets. Only flags in generic [build] config set
- # by the project will be lost, and ebuilds will need to add those to
- # RUSTFLAGS themselves if they are important.
-

On Wed, 2024-08-07 at 16:52 +0100, James Le Cuirot wrote:

The flags we set an a target-specific environment variable override any generic [build] flags set by the project, requiring ebuilds to set these themselves, which is undesirable. Work around this by using tomlq to
read the flags from the config files checked by Cargo and prepending
them to our environment variable.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6d4cf1b425eb..dea8c49e4585 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -36,7 +36,8 @@ esac

inherit flag-o-matic multiprocessing rust-toolchain toolchain-funcs

-[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND}"
+# app-misc/yq is needed for tomlq.
+[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND} app-misc/yq"

Doesn't this imply that all ebuilds using CARGO_OPTIONAL will now have
to explicitly depend on yq?

--
Best regards,
Michał Górny


-----BEGIN PGP SIGNATURE-----

iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAmazsWISHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQOAH0H/11/Wu5LQ/RQo3SdkVQTxuXFWY1AX1Sn EGW4WQV3yOHbmW52woPdsfFhcES3Gv0jgW1jmwPtbWC6X7VwPCPtSBWCXPBZXum/ 1FCOLIs2TgXli2MB+M2fBWnldpjRT9CmOgSAuxm99hlytXjEbgW2njsx4BR4lFSB ZB8yL6u+odnp45kSv01VFsDmtMGw516yHaBjq1zsVUfoRx1k7iGwY7Rs/2W3aHWw iaqSqv2ssWW3et9Dt7VUNI4BnlKrhK3yMxRuWG09zd76lUxfD9Y3kh9XVvoJ00Xp NH45wlNalUbQXWhUmVyYcCE1BR7c36gisQjK1VxggkKEQ9kdsHA/+48=
=dRZa
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 2024-08-07 at 19:39 +0200, Michał Górny wrote:

On Wed, 2024-08-07 at 16:52 +0100, James Le Cuirot wrote:

The flags we set an a target-specific environment variable override any generic [build] flags set by the project, requiring ebuilds to set these themselves, which is undesirable. Work around this by using tomlq to
read the flags from the config files checked by Cargo and prepending
them to our environment variable.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6d4cf1b425eb..dea8c49e4585 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -36,7 +36,8 @@ esac

inherit flag-o-matic multiprocessing rust-toolchain toolchain-funcs

-[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND}"
+# app-misc/yq is needed for tomlq.
+[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND} app-misc/yq"

Doesn't this imply that all ebuilds using CARGO_OPTIONAL will now have
to explicitly depend on yq?

Good catch, thanks. I can update the eclass docs accordingly. I count 23 ebuilds across 8 packages, and this can be done without a revbump, so it's not too bad. Even if it were missing, it would just continue without applying
these extra flags.

I have also noticed that yq will need some keywording first.

I considered some inline Python with just tomli instead, but then you get into the mess of micro-managing the Python versions. I don't think there's a way to say "use whatever Python tomli is installed for".

-----BEGIN PGP SIGNATURE-----

iQJFBAABCAAvFiEEPxcZ3tkwcedKm2a8EiZBXQDdMTcFAmaz1tgRHGNoZXdpQGdl bnRvby5vcmcACgkQEiZBXQDdMTdMMxAAhfVPAj0guI4a1Oxyd5XLskwax9q95+UI 4hqvRQuxpTXniP2wdaXTpe+mXjM+R0RWMMz4POdgxFr1wjCDfi0LvgeTs7KxWOAg UQDz3ADG0pRpYHqv+bmjdWw+HKxT8etuqfAr/g/ybAdqsH3O9vzYyLlSriHKNCX4 EwxlPW2VCdFWChA2oUZbFC1pGKlfN+Ip/uCqI410gd+0Ol+Y9O7Dp+TNYUIkglmI mBTTR8uPQO4jvJffcV/j+BD+0trSojzHx7s/AysutpRsWpekg92rvLtrhIULaTgo 4ygIw8+pwozQeHdcLRmlW0p9XcgjAJuB1dyzDePj2LsIxCfgSFrw06ru573L2wkA G/u4GZY3sQzx/YKpQTZQxIlRCM+BVG0Yha5AD6SMvtg7AyDiTv/OvHXFNyTi09Mm tBPBrB/TWaAks1Q/m8NBlcAWPhqqjPWipxC7aLZpem7rlBNoItNpJ6TgoypR4RWz pjza3ErVwWeUUQnN6JwxFZ+RsPGD2Fnelf9bi/fdHKdD2YdD6C8BKnzgojpEMj4G y9umI2U32TQLOrB23wSd9mfLfSyC225GZgYOL3YgOYKs7gxf1kX/7bPsvcW9IQMW AefLBUA5NmNRtls/xJNmEXTuLIUVY1UocPLy5OQ9Cj8Bh5CHi146Wd+PdYXAF6Jr
85zjevFlxVM=
=bex+
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, Aug 07, 2024 at 09:19:36PM +0100, James Le Cuirot wrote:

On Wed, 2024-08-07 at 19:39 +0200, Michał Górny wrote:

On Wed, 2024-08-07 at 16:52 +0100, James Le Cuirot wrote:

The flags we set an a target-specific environment variable override any generic [build] flags set by the project, requiring ebuilds to set these themselves, which is undesirable. Work around this by using tomlq to
read the flags from the config files checked by Cargo and prepending
them to our environment variable.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6d4cf1b425eb..dea8c49e4585 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -36,7 +36,8 @@ esac

inherit flag-o-matic multiprocessing rust-toolchain toolchain-funcs

-[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND}"
+# app-misc/yq is needed for tomlq.
+[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND} app-misc/yq"

Doesn't this imply that all ebuilds using CARGO_OPTIONAL will now have
to explicitly depend on yq?

Good catch, thanks. I can update the eclass docs accordingly. I count 23 ebuilds across 8 packages, and this can be done without a revbump, so it's not
too bad. Even if it were missing, it would just continue without applying these extra flags.

Probably no reason to not just add it to RUST_DEPEND, some ebuilds
already use that variable with OPTIONAL. Albeit yeah, it's not
documented nor required to use it at the moment.

I have also noticed that yq will need some keywording first.

I considered some inline Python with just tomli instead, but then you get into
the mess of micro-managing the Python versions. I don't think there's a way to
say "use whatever Python tomli is installed for".

If want to use python, I think simplest would the equivalent of
meson.eclass' dev-build/meson-format-array. Then the ebuild could
handle all the dependencies rather than the eclass.

Could be handy to pull less dependencies, with >=python3.11 can
use the stdlib's tomllib rather than tomli too, be very low footprint.

But is it really worth it over just using yq? I don't know :)
--
ionen

-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEx3SLh1HBoPy/yLVYskQGsLCsQzQFAmaz+D4ACgkQskQGsLCs QzTVawgAjEndJZqlRXhTPSHk5Ow72t6cNbBRQEfRDJplogUGhHlPUdjBtXPKjVil cb9QzbukuCHwjpKniAft6P2U/ja6iItb41/DKH02hubLNMqy0CGPoUNSi3PbrDUf 1ySjBRTL2wa88Q+Sn82eT8/BJEdKBUtdn1Ue60vKso+P6QZLXcVTAZtx4KyPwuiu 9T1TkJnC8c+8xylCEH1TizwgUGjjvIJ0D/GjvDl+fiBDL64I8AbP0nfSb9P/Izh/ 8bQBHJJpQhBStbxee4m95ECtHesY6dnHDcP5KMnwtqKCULI3JdvuNJ4bcbmsrKyZ gEssm+PoMRB9QHkDS1xZTpVWu5XF6w==
=f3e7
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 2024-08-07 at 21:19 +0100, James Le Cuirot wrote:

On Wed, 2024-08-07 at 19:39 +0200, Michał Górny wrote:

On Wed, 2024-08-07 at 16:52 +0100, James Le Cuirot wrote:

The flags we set an a target-specific environment variable override any generic [build] flags set by the project, requiring ebuilds to set these themselves, which is undesirable. Work around this by using tomlq to
read the flags from the config files checked by Cargo and prepending
them to our environment variable.

Signed-off-by: James Le Cuirot <chewi@gentoo.org>
---
eclass/cargo.eclass | 28 +++++++++++++++++-----------
1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/eclass/cargo.eclass b/eclass/cargo.eclass
index 6d4cf1b425eb..dea8c49e4585 100644
--- a/eclass/cargo.eclass
+++ b/eclass/cargo.eclass
@@ -36,7 +36,8 @@ esac

inherit flag-o-matic multiprocessing rust-toolchain toolchain-funcs

-[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND}"
+# app-misc/yq is needed for tomlq.
+[[ ! ${CARGO_OPTIONAL} ]] && BDEPEND="${RUST_DEPEND} app-misc/yq"

Doesn't this imply that all ebuilds using CARGO_OPTIONAL will now have
to explicitly depend on yq?

Good catch, thanks. I can update the eclass docs accordingly. I count 23 ebuilds across 8 packages, and this can be done without a revbump, so it's not
too bad. Even if it were missing, it would just continue without applying these extra flags.

Still, this sounds like hardcoding implementation details into ebuilds,
and implementation details can change over time.

--
Best regards,
Michał Górny


-----BEGIN PGP SIGNATURE-----

iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAma0KtESHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQO864H/jXn4UVrkWH/aEicc1lB7bojFmZtNOzZ Plp7m1RX76uplsZsbGDUYNs9pxnm+LrE4tlokivu4A3+xKa3EVJfTahcr0JAnFxN 9zI71yEB1qoY+lCd+yoZs6fMc52BodsGkX1zLauNX5rToUdDedRt39PIhpp23k4n 8XsdfdbPfSiT4MpOCV24YrfwB7Nu7aaTjZnISImewXb7h55PH5ydQsQI8bUY87FL T1jE2MGfkjb9y/of/EBVoy91dTGUPeCIHGa1JPxDl0Yinx5R4vJErEg7Amju0V/I vAZMvhsPMVieeUcDpNlNhy4iUWyy28lyxYPDsVdImp1SuZdN0iC7dsA=
=uZDE
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)