The EGO_SUM alternatives
- do not have the same level of trust and therefore have a negative
impact on security (a dubious tarball someone put somewhere, especially
when proxy-maint)
- are not easily verifiable
- require additional effort when developing ebuilds
- hinder the packaging and Gentoo's adoption of Go-based projects, which
is worrisome as Go is very popular
- prevent Go modules from being shared as DISTFILES on the mirrors
across various packages
Last but not least, we have the same situation in the Rust ecosystem,
but we allow the EGO_SUM "equivalent" there.
I want to continue the discussion to re-instate EGO_SUM, potentially
leading to a democratic vote on whether EGO_SUM should be re-instated or deprecated.
For the past months, I tried to find *technical reasons*, e.g., reasons
that affect end-users, that justify the deprecation of EGO_SUM. However,
I was unable to find any. The closest thing I could find was portage
being unable to process an ebuild due to its large environment (bug
830187). However, as this happens while developing an ebuild, it should
never affect users. Obviously this is a situation where EGO_SUM can not
be used. Fortunately, it does not affect most Go packages, as seen in my previous analysis of Go packages in ::gentoo and their EGO_SUM size. Furthermore, newer portage versions, with USE=gentoo-dev, will
proactively warn you if the environment caused by the ebuild becomes large.
All further arguments for the deprecation of EGO_SUM where of cosmetic nature.
However, the deprecation of EGO_SUM is harmful to Gentoo and its users.
To briefly re-iterate the reasons:
The EGO_SUM alternatives
- do not have the same level of trust and therefore have a negative
impact on security (a dubious tarball someone put somewhere, especially
when proxy-maint)
- are not easily verifiable
- require additional effort when developing ebuilds
- hinder the packaging and Gentoo's adoption of Go-based projects, which
is worrisome as Go is very popular
- prevent Go modules from being shared as DISTFILES on the mirrors
across various packages
Last but not least, we have the same situation in the Rust ecosystem,
but we allow the EGO_SUM "equivalent" there.
So with portage checking the environment of ebuilds and warning if it
becomes too large, and with the arguments above, I do not see any reason
we should outlaw EGO_SUM.
- Flow
Previous discussions: https://archives.gentoo.org/gentoo-dev/message/1a64a8e7694c3ee11cd48a58a95f2faa
https://archives.gentoo.org/gentoo-dev/message/d78af7f168cef24bfa302f7f75c3ef11
I like to ask the Gentoo council to vote on whether EGO_SUM should be reinstated ("un-deprecated") or not.
EGO_SUM is a project-comprehensive matter, as it affects not only
Go-lang packaging but also the proxy-maint and GURU
projects. Furthermore, as I have mentioned in my previous emails, the deprecation of EGO_SUM has a significant negative impact on our users
and is, therefore, a global Gentoo issue.
Asking for council involvement should be a last resort and only be
done in essential conflicts. But, unfortunately, I was unable to
convince the relevant maintainer with arguments that the deprecation
of EGO_SUM is harmful. And this matter is significant enough to
proceed with this.
Most voices on the related mailing-list threads expressed support for reinstating EGO_SUM. At least, that is my impression. While the
arguments used to deprecate EGO_SUM were mostly of esthetic nature.
I want to state what should be common sense. Namely, asking for a
democratic vote is not a personal attack against any involved
person.
[...]
On 17/04/2023 09.37, Florian Schmaus wrote:
[original msg snipped]
Florian Schmaus <flow@gentoo.org> writes:
[CCing williamh@ as go-module.eclass & dev-lang/go maintainer.]
I like to ask the Gentoo council to vote on whether EGO_SUM should be reinstated ("un-deprecated") or not.
EGO_SUM is a project-comprehensive matter, as it affects not only
Go-lang packaging but also the proxy-maint and GURU
projects. Furthermore, as I have mentioned in my previous emails, the deprecation of EGO_SUM has a significant negative impact on our users
and is, therefore, a global Gentoo issue.
Asking for council involvement should be a last resort and only be
done in essential conflicts. But, unfortunately, I was unable to
convince the relevant maintainer with arguments that the deprecation
of EGO_SUM is harmful. And this matter is significant enough to
proceed with this.
My feeling on this is that this proposal isn't yet complete enough
for the council to assess. In the various previous discussions, the need
for _some_ limit to be implemented (derived from EGO_SUM) was clear from
the QA team and others.
Voting on the matter now would be reopening the issue which led EGO_SUM
to be deprecated in the first place, with only a partial mitigation
(the Portage warning).
Any such limit should be supported by pkgcheck, allow using EGO_SUM
for most packages, but exclude the pathological cases which we're
unlikely to want in ::gentoo.
(Limit-per-ebuild rather than per-package is one option of many,
too.)
Most voices on the related mailing-list threads expressed support for reinstating EGO_SUM. At least, that is my impression. While the
arguments used to deprecate EGO_SUM were mostly of esthetic nature.
I want to state what should be common sense. Namely, asking for a democratic vote is not a personal attack against any involved
person.
[...]
I agree this is an important issue that affects the practicality
of using Gentoo for some, and for contributing to Gentoo to others.
On 17/04/2023 09.37, Florian Schmaus wrote:
[original msg snipped]
Florian Schmaus <flow@gentoo.org> writes:
[CCing williamh@ as go-module.eclass & dev-lang/go maintainer.]
I like to ask the Gentoo council to vote on whether EGO_SUM should befor _some_ limit to be implemented (derived from EGO_SUM) was clear from
reinstated ("un-deprecated") or not > In the various previous discussions, the need
the QA team and others.
Voting on the matter now would be reopening the issue which led EGO_SUM
to be deprecated in the first place, with only a partial mitigation
(the Portage warning).
Any such limit should be supported by pkgcheck, allow using EGO_SUM
for most packages, but exclude the pathological cases which we're
unlikely to want in ::gentoo.
(Limit-per-ebuild rather than per-package is one option of many,
too.)
Most voices on the related mailing-list threads expressed support for
reinstating EGO_SUM. At least, that is my impression. While the
arguments used to deprecate EGO_SUM were mostly of esthetic nature.
I want to state what should be common sense. Namely, asking for a
democratic vote is not a personal attack against any involved
person.
[...]
I agree this is an important issue that affects the practicality
of using Gentoo for some, and for contributing to Gentoo to others.
The discussion would be more productive if someone who is supporting the EGO_SUM deprecation could rationally summarize the main arguments why we deprecated EGO_SUM.
On Wed, Apr 26, 2023 at 11:31 AM Florian Schmaus <flow@gentoo.org> wrote:
The discussion would be more productive if someone who is supporting the
EGO_SUM deprecation could rationally summarize the main arguments why we
deprecated EGO_SUM.
You're requesting the changes. It's on you to read the previous
threads and try to understand. It's not others' responsibilities to
justify the status quo to you, but tl;dr is Manifest files grew to
insane sizes for golang packages with many dependencies, and the
Manifest size is a cost all Gentoo users pay regardless of whether
they use the package.
On 26/04/2023 18:12, Matt Turner wrote:
On Wed, Apr 26, 2023 at 11:31 AM Florian Schmaus <flow@gentoo.org> wrote:
The discussion would be more productive if someone who is supporting the >> EGO_SUM deprecation could rationally summarize the main arguments why we >> deprecated EGO_SUM.
You're requesting the changes. It's on you to read the previous
threads and try to understand. It's not others' responsibilities to
justify the status quo to you, but tl;dr is Manifest files grew to
insane sizes for golang packages with many dependencies, and the
Manifest size is a cost all Gentoo users pay regardless of whether
they use the package.
This is a valid point and I think it is clear. What is not clear however
is why the EGO_SUM method should be dropped entirely instead of keeping
it as an option for overlays (with an appropriate warning). As I
remember this is where the discussion got 'stuck' last time.
There are other cases where things are possible but prohibited in
::gentoo by policy. E.g. the acct-user eclass allows setting
ACCT_USER_ID to -1 for dynamic assignment, but we do not allow this in ::gentoo. I don't see why we could not do the same for EGO_SUM, keep it
as an option, while disallowing it in ::gentoo.
Asking to impose an artificial limit is based on the same unfounded
belief under which EGO_SUM was deprecated in the first place. I am
worried that if we follow this, then a potential next step is to argue
about adding packages to ::gentoo.
Hi Sam,
thanks for your feedback. I am glad for everyone who engages in this discussion and shares their views and new information.
On 24/04/2023 22.28, Sam James wrote:
Florian Schmaus <flow@gentoo.org> writes:
[CCing williamh@ as go-module.eclass & dev-lang/go maintainer.]
I like to ask the Gentoo council to vote on whether EGO_SUM should befor _some_ limit to be implemented (derived from EGO_SUM) was clear from
reinstated ("un-deprecated") or not > In the various previous discussions, the need
the QA team and others.
Asking to impose an artificial limit is based on the same unfounded
belief under which EGO_SUM was deprecated in the first place. I am
worried that if we follow this, then a potential next step is to argue
about adding packages to ::gentoo.
Voting on the matter now would be reopening the issue which led EGO_SUM
to be deprecated in the first place, with only a partial mitigation
(the Portage warning).
I am sorry, but I do not follow. I think this is partly because it is
not clear "what" (else) to mitigate.
The discussion would be more productive if someone who is supporting
the EGO_SUM deprecation could rationally summarize the main arguments
why we deprecated EGO_SUM.
Any such limit should be supported by pkgcheck, allow using EGO_SUM
for most packages, but exclude the pathological cases which we're
unlikely to want in ::gentoo.
(Limit-per-ebuild rather than per-package is one option of many,
too.)
As you probably noticed, I am not aware why we should impose such a
limit. Especially a per-package limit confines the ability to provide
the user with multiple versions of a package, which sometimes comes in
handy [1].
On Thu, 27 Apr 2023, Florian Schmaus wrote:
Network traffic, while also being cheap, may be more of an issue.
Currently, gentoo-latest.tar.xz is ~41 MiB. So on a conservative approximation ::gentoo compresses to 1/10. So, the 10 Go-packages
cause 200 KiB of additional traffic. Even when using a low-bandwidth connection, say 12 KiB/s, this only adds 17 extra seconds to the
transfer duration.
Disk space is cheap.
Network traffic, while also being cheap, may be more of an issue.
On 2023-04-17 09:37, Florian Schmaus wrote:
The EGO_SUM alternatives
- do not have the same level of trust and therefore have a negative
impact on security (a dubious tarball someone put somewhere, especially when proxy-maint)
That, however, doesn't remove the concern about big ebuilds and manifests. I will look at the remainder of the thread to figure out
what is going on with that.
[[PGP Signed Part:Undecided]]
On 26/04/2023 18.12, Matt Turner wrote:
On Wed, Apr 26, 2023 at 11:31 AM Florian Schmaus <flow@gentoo.org> wrote: >>> The discussion would be more productive if someone who is supporting the >>> EGO_SUM deprecation could rationally summarize the main arguments why we >>> deprecated EGO_SUM.
You're requesting the changes. It's on you to read the previous
threads and try to understand. It's not others' responsibilities to
justify the status quo to you, but tl;dr is Manifest files grew to
insane sizes for golang packages with many dependencies, and the
Manifest size is a cost all Gentoo users pay regardless of whether
they use the package.
I am sorry. I did try to understand the reasoning in the previous
threads. However, I do not conclude that the "cost" users must pay for EGO_SUM justifies EGO_SUM's deprecation. It is the other way around: EGO_SUM's advantages do not explain its deprecation, even if users
have to pay a cost.
You write that the "Manifest sizes grew to insane sizes"?
At which boundary does a package size, the total size of the package's directory, become insane?
Disk space is cheap. Currently, ::gentoo, without metadata, is around
470 MiB. If you add 10 Go packages with a whopping 200 KiB each, then
this adds 2 MiB to that. I need someone to explain how this
constitutes an issue with disk space. Even if we add 100 Go packages, probably roughly the number of Go packages we have in ::gentoo, then
those 20 MiB are not significant. Needless to say that the average
size of a Go package is less than the 200 KiB uses in this
calculation.
On Donnerstag, Apr. 27, 2023 at 14:54, Michał Górny <mgorny@gentoo.org (mailto:mgorny@gentoo.org)> wrote:
On Thu, 2023-04-27 at 09:58 +0200, Florian Schmaus wrote:
Disk space is cheap.
No, it's not. Gentoo supports more hardware than your average PC with
beefy hard drive and/or possibility of installing one. Let's not forget
that you need a ::gentoo checkout even on a system running purely
on binary packages.
Let's not forget that git keeps all history, so every bump of a Go
package with large Manifest has a permanent negative impact on clone
size. A few version bumps of Go packages can easily outweigh complete
history of hundreds of other packages.
Network traffic, while also being cheap, may be more of an issue.
Again, you're making assumption based on living in a well-developed area
and discriminating against users who have shoddy Internet connectivity.
That said, this all was discussed in the past. I really wish you would
humble down and try to find a solution that would work for everyone
instead of showing arrogance and lack of concern for users outside your "majority" view of Gentoo.
--
Best regards,
Michał Górny
Maybe I’m getting this wrong, but didn’t we switch to shallow
checkouts for the systems repository? I remember it was a major
outcry on the mailing list. So at least for end users git keeps no
history and our repository history should not impact clone size of a
shallow copy, should it?
On Donnerstag, Apr. 27, 2023 at 14:54, Michał Górny <
mgorny@gentoo.org> wrote:
On Thu, 2023-04-27 at 09:58 +0200, Florian Schmaus wrote:
Disk space is cheap.
No, it's not. Gentoo supports more hardware than your average PC
with
beefy hard drive and/or possibility of installing one. Let's not
forget
that you need a ::gentoo checkout even on a system running purely
on binary packages.
Let's not forget that git keeps all history, so every bump of a
Go
package with large Manifest has a permanent negative impact on
clone
size. A few version bumps of Go packages can easily outweigh
complete
history of hundreds of other packages.
Network traffic, while also being cheap, may be more of an
issue.
Again, you're making assumption based on living in a
well-developed area
and discriminating against users who have shoddy Internet
connectivity.
That said, this all was discussed in the past. I really wish you
would
humble down and try to find a solution that would work for
everyone
instead of showing arrogance and lack of concern for users
outside your
"majority" view of Gentoo.
--
Best regards,
Michał Górny
Pascal Jäger <Pascal.jaeger@leimstift.de> writes:
Maybe I’m getting this wrong, but didn’t we switch to shallow checkouts for the systems repository? I remember it was a major
outcry on the mailing list. So at least for end users git keeps no
history and our repository history should not impact clone size of a shallow copy, should it?
(Try to avoid top-posting if you can, reply after the message you're
replying to.)
rsync copies of the tree aren't affected by this, nor are full
git clones for development.
On Fri, 2023-04-28 at 01:38 +0100, Sam James wrote:
Pascal Jäger <Pascal.jaeger@leimstift.de> writes:
Maybe I’m getting this wrong, but didn’t we switch to shallow
checkouts for the systems repository? I remember it was a major
outcry on the mailing list. So at least for end users git keeps no
history and our repository history should not impact clone size of a
shallow copy, should it?
(Try to avoid top-posting if you can, reply after the message you're
replying to.)
rsync copies of the tree aren't affected by this, nor are full
git clones for development.
Err, but full gentoo.git clones are definitely affected! After all,
that's where huge ebuilds and their Manifests land first.
On 27/04/2023 14.54, Michał Górny wrote:
On Thu, 2023-04-27 at 09:58 +0200, Florian Schmaus wrote:
Disk space is cheap.
No, it's not. Gentoo supports more hardware than your average PC with beefy hard drive and/or possibility of installing one. Let's not forget that you need a ::gentoo checkout even on a system running purely
on binary packages.
You are right. Gentoo supports a broad range of hardware in many
dimensions, e.g., architecture, release date, and composition.
You seem to suggest that are Gentoo systems that can not handle the additional disk space consumption of EGO_SUM Go-packages?
I can not imagine systems that are able to deal with the ~500 MiB
::gentoo repository, but would break if the same repository would
contain 100 additional Go-packages with 200 KiB each.
Even under a "worst-case" assumption, where we would have 256
Go-packages with each having a 1 MiB package-directory size, any system
that can handle the current state of ::gentoo should be able to take the additional 256 MiB (+ metadata).
I am only pursuing the modest request to legitimize any decision
regarding EGO_SUM by a democratic vote.
As far as I can tell, there was never a democratic vote regarding
EGO_SUM. But please correct me if I am wrong.
And I never said that I believe in representing the majority's opinion.
That said, I prefer to have this voted on by an all-developer vote than
a council vote. Then we would know what the majority voted for. Is that possible?
On 27/04/2023 14.54, Michał Górny wrote:This email ended up more rambling than I intended, but I wanted to get the data out there, and enable us to look deeper at the problems and potential impacts of the solutions.
On Thu, 2023-04-27 at 09:58 +0200, Florian Schmaus wrote:
Disk space is cheap.
No, it's not. Gentoo supports more hardware than your average PC with beefy hard drive and/or possibility of installing one. Let's not forget that you need a ::gentoo checkout even on a system running purely
on binary packages.
You are right. Gentoo supports a broad range of hardware in many
dimensions, e.g., architecture, release date, and composition.
You seem to suggest that are Gentoo systems that can not handle the additional disk space consumption of EGO_SUM Go-packages?
I can not imagine systems that are able to deal with the ~500 MiB
::gentoo repository, but would break if the same repository would
contain 100 additional Go-packages with 200 KiB each.
Even under a "worst-case" assumption, where we would have 256
Go-packages with each having a 1 MiB package-directory size, any system
that can handle the current state of ::gentoo should be able to take the additional 256 MiB (+ metadata).
[[PGP Signed Part:Undecided]]
On 27/04/2023 23.16, Sam James wrote:
Florian Schmaus <flow@gentoo.org> writes:
[[PGP Signed Part:Undecided]]The numbers you've used here suggest you've missed some of the
On 26/04/2023 18.12, Matt Turner wrote:
On Wed, Apr 26, 2023 at 11:31 AM Florian Schmaus <flow@gentoo.org> wrote:
The discussion would be more productive if someone who is supporting the >>>>> EGO_SUM deprecation could rationally summarize the main arguments why we >>>>> deprecated EGO_SUM.You're requesting the changes. It's on you to read the previous
threads and try to understand. It's not others' responsibilities to
justify the status quo to you, but tl;dr is Manifest files grew to
insane sizes for golang packages with many dependencies, and the
Manifest size is a cost all Gentoo users pay regardless of whether
they use the package.
I am sorry. I did try to understand the reasoning in the previous
threads. However, I do not conclude that the "cost" users must pay for
EGO_SUM justifies EGO_SUM's deprecation. It is the other way around:
EGO_SUM's advantages do not explain its deprecation, even if users
have to pay a cost.
You write that the "Manifest sizes grew to insane sizes"?
At which boundary does a package size, the total size of the package's
directory, become insane?
Disk space is cheap. Currently, ::gentoo, without metadata, is around
470 MiB. If you add 10 Go packages with a whopping 200 KiB each, then
this adds 2 MiB to that. I need someone to explain how this
constitutes an issue with disk space. Even if we add 100 Go packages,
probably roughly the number of Go packages we have in ::gentoo, then
those 20 MiB are not significant. Needless to say that the average
size of a Go package is less than the 200 KiB uses in this
calculation.
big problematic cases from the past:
- https://bugs.gentoo.org/833478 (1.1MB manifest)
- https://bugs.gentoo.org/833477 (1.6MB manifest)
Thanks for pointing those bugs out.
But please allow me to clarify that I did not miss those "problematic"
cases from the past.
[[PGP Signed Part:Undecided]]
On 28/04/2023 16.34, Michał Górny wrote:
On Fri, 2023-04-28 at 08:59 +0200, Florian Schmaus wrote:
And I never said that I believe in representing the majority's opinion.There's the General Resolution but it's supposed to be used only to
That said, I prefer to have this voted on by an all-developer vote than
a council vote. Then we would know what the majority voted for. Is that
possible?
override Council decisions, so you should go with a Council vote first.
Could we temporarily re-purpose Gentoo's election infrastructure to
hold an all-developer opinion poll?
I imagine a poll asking for opinions, nothing binding. Furthermore,
since Gentoo's voting infrastructure uses the Condorcet method, we
could have multiple options.
I performed a tree-wide analysis regarding EGO_SUM and IIRC published
the results in my previous post about EGO_SUM last year. https://dev.gentoo.org/~flow/ego_sum-2022-01-01.txt shows the analysis results for ::gentoo as of 2022-01-01 (I've recently updated the file to contain the Manifest-size too).
Minikube (#833478) and k3s (#833477) appear there, too, with package-directory sizes over one MiB. However, those packages are under
the top five of packages using EGO_SUM by package-directory size.
They do not represent the average Go package.
The mean size of a Manifest of a package using EGO_SUM was 186 KiB, and
the median was even lower at 84 KiB. Only a tiny percentage of packages, below 5%, had a Manifest-size above one MiB.
To prevent harm from Gentoo, we should reach an agreement that everyone
can live with. To achieve a consensus, and since I can not rule out that
I missed a post that includes specific numbers, please share your ideas
on how EGO_SUM could be reinstated in ::gentoo by replying to this mail.
I am thankful that the council considered my request to vote on the
topic. However, the council decided not to vote on this in its last
session and to return the issue to the mailing lists.
Some see the requirement of some limitations as necessity it comes to reinstating EGO_SUM. Unfortunately, I could not see specific numbers mentioned since June 2022 in the three EGO_SUM threads [1, 2, 3] I am
aware of.
To prevent harm from Gentoo, we should reach an agreement that everyone
can live with. To achieve a consensus, and since I can not rule out that
I missed a post that includes specific numbers, please share your ideas
on how EGO_SUM could be reinstated in ::gentoo by replying to this mail.
Having EGO_SUM would significantly increase the security of Gentoo's
users (amongst other benefits).
Personally, I do not see that we currently need any form of limitation
to reinstate EGO_SUM. I substantiated this with data based on a two-year history analysis of gentoo.git. The summary is that the
- size increase of ::gentoo is unproblematic for users
- additional sync delta of ::gentoo is unproblematic for users
- higher rate of gentoo.git's increase is unproblematic for developers
when we reinstate EGO_SUM in ::gentoo.
Therefore, we could (and IMHO should) simply un-deprecate EGO_SUM.
However, I would review this decision once the number of Go packages has doubled or in two years (whatever comes first).
Many share the concerns of an EGO_SUM-less world. I know that some seek
a compromise by reinstating EGO_SUM with some limitations. The ::gentoo repository is able to handle packages (at least) up to the range of 2 to
1.5 MiB total package-directory size. Therefore I propose a limit in
that range.
- Flow
1: https://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg95175.html
2: https://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg95279.html
3: https://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg97310.html
On 2023-05-30 17:52, Florian Schmaus wrote:
To prevent harm from Gentoo, we should reach an agreement that everyone can live with. To achieve a consensus, and since I can not rule out that
I missed a post that includes specific numbers, please share your ideas
on how EGO_SUM could be reinstated in ::gentoo by replying to this mail.
Instate a policy to allow EGO_SUM in the gentoo tree:
1) from proxied maintainers
Arthur Zamarin <arthurzam@gentoo.org> hat am 30.05.2023 18:35 CEST geschrieben:
Currently the best solution *per package* is to speak with upstream, to
add a CI workflow which create a source tarball which includes `vendor`
dir. This is the best way, and I'm doing that for multiple upstream of
some random Go packages in ::gentoo. But I know the disadvantage - requirement to speak with upstream, explain why, and add it to the
system. This is best long-run solution, but more hardships.
My solution is as such:
1. Undeprecate EGO_SUM in eclass
2. Forbid it's usage in ::gentoo (done by pkgcheck, error level, will
fail CI and as such we can see the misuse). Overlays are allowed.
3. Maintainer starts talks with upstreams to add release workflow to
create vendored source tarball, in hopes of it succeeding. "Start early,
to future profit". I see this flow similar to the "always try to
upstream patches".
4. Until upstream adds it, in ::gentoo use vendor tarballs.
I also think many devs agree with this solution, but I can't talk for
them, so I'll be happy agreeing devs can at least reply shortly their agreement or disagreement.
在 2023年5月31日，14:20，Andrew Ammerlaan <andrewammerlaan@gentoo.org> 写道：
On 30/05/2023 18:35, Arthur Zamarin wrote:
My solution is as such:
1. Undeprecate EGO_SUM in eclass
2. Forbid it's usage in ::gentoo (done by pkgcheck, error level, will
fail CI and as such we can see the misuse). Overlays are allowed.
3. Maintainer starts talks with upstreams to add release workflow to
create vendored source tarball, in hopes of it succeeding. "Start early,
to future profit". I see this flow similar to the "always try to
upstream patches".
4. Until upstream adds it, in ::gentoo use vendor tarballs.
I also think many devs agree with this solution, but I can't talk for
them, so I'll be happy agreeing devs can at least reply shortly their
agreement or disagreement.
I fully agree with Arthur
With regards to proxy-maintained packages: The proxy can generate and upload the vendor tarball for the proxied, this is not that much extra work.
Best regards,
Andrew
On 30/05/2023 18:35, Arthur Zamarin wrote:
My solution is as such:
1. Undeprecate EGO_SUM in eclass
2. Forbid it's usage in ::gentoo (done by pkgcheck, error level, will
fail CI and as such we can see the misuse). Overlays are allowed.
3. Maintainer starts talks with upstreams to add release workflow to
create vendored source tarball, in hopes of it succeeding. "Start early,
to future profit". I see this flow similar to the "always try to
upstream patches".
4. Until upstream adds it, in ::gentoo use vendor tarballs.
I also think many devs agree with this solution, but I can't talk for
them, so I'll be happy agreeing devs can at least reply shortly their
agreement or disagreement.
I fully agree with Arthur
With regards to proxy-maintained packages: The proxy can generate and upload the vendor tarball for the proxied, this is not that much extra work.
Best regards,
Andrew
Arthur Zamarin <arthurzam@gentoo.org> hat am 30.05.2023 18:35 CEST geschrieben:
Currently the best solution *per package* is to speak with upstream, to
add a CI workflow which create a source tarball which includes `vendor` dir. This is the best way, and I'm doing that for multiple upstream of
some random Go packages in ::gentoo. But I know the disadvantage - requirement to speak with upstream, explain why, and add it to the
system. This is best long-run solution, but more hardships.
I would like to add to this, that even if upstream is not willing to do this, devs could automate the creation of vendor tarballs using GitHub actions. I only did this for an upstream repositories that are also on GitHub and for projects written inRust. Initially I did this for complicated Rust projects with several git submodules and submodules of submodules. But with a little tweaking of the GitHub actions I think it would be possible to use it for Go as well.
https://wiki.gentoo.org/wiki/User:Schievel/autocreate_rust_sources
This is additional initial work, but once you set it up, you don't even have the extra work of creating a new EGO_SUM for every package release. Ideally you just have to change the version in the file name of the ebuild to bump a package.
Security wise I do not see a difference between this and creating the vendor tarball manually and uploading it to GitHub, as many proxy maintainers without devspace do it.
On 30/05/2023 18.35, Arthur Zamarin wrote:
On 30/05/2023 18.52, Florian Schmaus wrote:
To prevent harm from Gentoo, we should reach an agreement that everyone can live with. To achieve a consensus, and since I can not rule out that I missed a post that includes specific numbers, please share your ideas on how EGO_SUM could be reinstated in ::gentoo by replying to this mail.
I still want to ask why in ::gentoo should it be enabled? I'm trying to understand why?
In short: Auditability
Let me try to explain with a simplified example.
Gentoo's ebuilds contain the instructions to transform source code
(input) via a compilation process (transformation) into a binary image (output).
A pseudo-example ebuild may contain the following
foo-1.0.ebuild:
```
# Input
SRC_URI="https://foo-soft.org/foo/1.0/foo-1.0.tar.gz"
# Transformation
src_compile() {
emake foo
}
# Output into imagedir $D
src_install() {
emake DESTDIR="${D}" foo-install
}
```
A Gentoo developer, Gentoo user, or, anyone can look at the ebuild and immediately tell that it will likely not inject malicious code into the resulting binary image. Furthermore, the only input is from upstream,
and while you may not look at every line of source code, you assign a certain trust level to upstream and probably assume that the input is
also likely non-malicious.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 475 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 17:20:56 |
| Calls: | 9,487 |
| Calls today: | 6 |
| Files: | 13,617 |
| Messages: | 6,121,089 |