• Bug#1099632: Treat AARs as zips

    From Matt Corallo@21:1/5 to All on Thu Mar 6 03:00:01 2025
    Package: diffoscope
    Version: 240+deb12u1

    AAR files are identified by file as "Android package (APK), with AndroidManifest.xml", but really
    they're just zips with specific files inside. It would be nice to be able to use diffoscope's diff
    view given that, but currently we just drop to a binary diff.

    Tried to file this on salsa which the website suggests, but sadly, "Your request to join
    https://salsa.debian.org/ has been rejected."

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Lamb@21:1/5 to All on Thu Mar 6 09:00:04 2025
    Hi Matt,

    Package: diffoscope
    Version: 240+deb12u1

    AAR files are identified by file as "Android package (APK), with AndroidManifest.xml", but really
    they're just zips with specific files inside. It would be nice to be
    able to use diffoscope's diff
    view given that, but currently we just drop to a binary diff.

    Ah, thanks for letting us know. Do you happen to have two .aar files
    lying around so I can test that...?

    Tried to file this on salsa which the website suggests, but sadly,
    "Your request to join https://salsa.debian.org/ has been rejected."

    Oh, that's annoying. We don't run salsa.debian.org ourselves, so you
    might have to followup with the admins here:

    https://wiki.debian.org/Salsa/FAQ#My_account_registration_request_was_rejected._What_do_I_do.3F

    (You are likely simply being caught by some anti-spam thing.)


    Best wishes,

    --
    ,''`.
    : :' : Chris Lamb
    `. `'` lamby@debian.org 🍥 chris-lamb.co.uk
    `-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matt Corallo@21:1/5 to Chris Lamb on Thu Mar 6 18:00:01 2025
    On 3/6/25 2:15 AM, Chris Lamb wrote:
    Hi Matt,

    Package: diffoscope
    Version: 240+deb12u1

    AAR files are identified by file as "Android package (APK), with
    AndroidManifest.xml", but really
    they're just zips with specific files inside. It would be nice to be
    able to use diffoscope's diff
    view given that, but currently we just drop to a binary diff.

    Ah, thanks for letting us know. Do you happen to have two .aar files
    lying around so I can test that...?

    Hmm, that have only marginal diff, not handy, but if you just want to look at some AAR files, you
    could for example grab LDK-release.aar from

    https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.1.0;hb=refs/heads/main

    and

    https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.0.0;hb=refs/heads/main

    Tried to file this on salsa which the website suggests, but sadly,
    "Your request to join https://salsa.debian.org/ has been rejected."

    Oh, that's annoying. We don't run salsa.debian.org ourselves, so you
    might have to followup with the admins here:

    https://wiki.debian.org/Salsa/FAQ#My_account_registration_request_was_rejected._What_do_I_do.3F

    (You are likely simply being caught by some anti-spam thing.)

    Doesn't matter much, I just wanted to file this bug, if it got to the right place that's fine.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Lamb@21:1/5 to Matt Corallo on Tue Mar 11 15:30:02 2025
    Matt Corallo wrote:

    https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.1.0;hb=refs/heads/main
    and https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.0.0;hb=refs/heads/main

    Thanks for linking these. So, the proximate cause of this issue is
    that the underlying file(1) utility reports these as Android packages:

    $ file v0.1.0.0_LDK-release.aar v0.1.1.0_LDK-release.aar
    v0.1.0.0_LDK-release.aar: Android package (APK), with AndroidManifest.xml
    v0.1.1.0_LDK-release.aar: Android package (APK), with AndroidManifest.xml

    In general, diffoscope basically has to trust what file(1) reports and
    proceed on that basis. We used to work around its bugs, but as you can
    imagine, that became something of a game of whack-a-mole...

    In the first instance, just to confirm that file(1) is at fault here? As
    in, these are definitely not Android packages. :)


    Regards,

    --
    ,''`.
    : :' : Chris Lamb
    `. `'` lamby@debian.org 🍥 chris-lamb.co.uk
    `-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matt Corallo@21:1/5 to Chris Lamb on Tue Mar 11 16:30:01 2025
    On 3/11/25 10:16 AM, Chris Lamb wrote:
    Matt Corallo wrote:

    https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.1.0;hb=refs/heads/main
    and
    https://git.bitcoin.ninja/?p=ldk-java-bins;a=tree;f=v0.1.0.0;hb=refs/heads/main

    Thanks for linking these. So, the proximate cause of this issue is
    that the underlying file(1) utility reports these as Android packages:

    $ file v0.1.0.0_LDK-release.aar v0.1.1.0_LDK-release.aar
    v0.1.0.0_LDK-release.aar: Android package (APK), with AndroidManifest.xml
    v0.1.1.0_LDK-release.aar: Android package (APK), with AndroidManifest.xml

    In general, diffoscope basically has to trust what file(1) reports and proceed on that basis. We used to work around its bugs, but as you can imagine, that became something of a game of whack-a-mole...

    In the first instance, just to confirm that file(1) is at fault here? As
    in, these are definitely not Android packages. :)

    They're android "packages" in that they're libraries that other APKs can depend on, but certainly
    not runnable apps or anything of the like.

    AFAIU, all "Android package (APK)"s should be treated like zips cause they are just zips with
    specific files in them.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Chris Lamb@21:1/5 to Matt Corallo on Wed Mar 19 15:10:01 2025
    tags 1099632 + pending
    thanks

    Matt Corallo wrote:

    In the first instance, just to confirm that file(1) is at fault here? As
    in, these are definitely not Android packages. :)

    They're android "packages" in that they're libraries that other APKs
    can depend on, but certainly not runnable apps or anything of the like.

    Getcha. I suppose it was kinda understandable that they lumped them
    together as being APK related, even though it's not strictly accurate.

    However, that "packages" comes from file(1) itself rather than
    diffoscope itself. They may be interested in ensuring the accuracy of
    that description: I'm convinced, but you will have to contact the file
    upstream for this.

    AFAIU, all "Android package (APK)"s should be treated like zips cause
    they are just zips with specific files in them.

    They are, yes! But diffoscope has special handling for APK files as it
    can improve the output for those files, even though they are, as you
    say, "just" zip files underneath.

    What was preventing diffoscope handling your .aar files properly,
    though, was that this special handling for APK files was reserved for
    files with the .apk extension only, even if file(1) was reporting them as
    an "APK package".

    Extending this code to consider AAR files as equivalent as APK files
    (at least as far as *diffoscope* is concerned), results in it not
    falling back to a binary diff.

    --- diffoscope/comparators/apk.py
    +++ diffoscope/comparators/apk.py
    @@ -238,7 +238,7 @@ class ApkFile(ZipFileBase):
    FILE_TYPE_HEADER_PREFIX = b"PK\x03\x04"
    FILE_TYPE_RE = re.compile(r"^(Android package|(Java|Zip) archive data)\b")
    - FILE_EXTENSION_SUFFIX = {".apk"}
    + FILE_EXTENSION_SUFFIX = {".apk", ".aar"}
    CONTAINER_CLASSES = [ApkContainer, ZipContainer]


    I've made this change in Git and I plan to release it later this week.


    Best wishes,

    --
    ,''`.
    : :' : Chris Lamb
    `. `'` lamby@debian.org 🍥 chris-lamb.co.uk
    `-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)