• list_del corruption ... kernel BUG at lib/list_debug.c:62!

    From Uwe =?utf-8?Q?Kleine-K=C3=B6nig?=@21:1/5 to All on Wed Apr 23 11:00:01 2025
    XPost: linux.debian.kernel

    Hello Andrew,

    Alexis Huxley reported a bug against the Debian kernel where his x86_64
    system died on a list corruption in rmqueue_bulk running a 6.1.129
    kernel. Find below the output of the first BUG. Further details and logs
    are available at https://bugs.debian.org/1101944.

    The problem isn't reproducible, so Alexis cannot reliably confirm if
    newer kernels are fixed. Alexis ran a memtest on that machine over-night without any reported problems.

    Does this ring a bell for you?

    Best regards
    Uwe

    2025-04-02T11:16:47.035411+02:00 pici kernel: [747711.533523] ------------[ cut here ]------------
    2025-04-02T11:16:47.035411+02:00 pici kernel: [747711.540294] kernel BUG at lib/list_debug.c:62!
    2025-04-02T11:16:47.035412+02:00 pici kernel: [747711.546872] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
    2025-04-02T11:16:47.035412+02:00 pici kernel: [747711.554212] CPU: 10 PID: 6770 Comm: python3 Not tainted 6.1.0-32-amd64 #1 Debian 6.1.129-1
    2025-04-02T11:16:47.035413+02:00 pici kernel: [747711.564696] Hardware name: Gigabyte Technology Co., Ltd. Z790 AERO G/Z790 AERO G, BIOS F13b 12/04/2024
    2025-04-02T11:16:47.035413+02:00 pici kernel: [747711.576153] RIP: 0010:__list_del_entry_valid.cold+0x23/0x6f
    2025-04-02T11:16:47.035413+02:00 pici kernel: [747711.583892] Code: e8 0d 9f fe ff 0f 0b 48 89 fe 48 c7 c7 98 62 fa bc e8 fc 9e fe ff 0f 0b 48 89 d1 48 c7 c7 b8 63 fa bc 48 89 c2 e8 e8 9e fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 68 63 fa bc e8 d4 9e fe
    ff 0f 0b
    2025-04-02T11:16:47.035414+02:00 pici kernel: [747711.604949] RSP: 0000:ffffa9fba3247b60 EFLAGS: 00010046
    2025-04-02T11:16:47.035414+02:00 pici kernel: [747711.612451] RAX: 000000000000006d RBX: ffff9ba2bf6b78e8 RCX: 0000000000000000
    2025-04-02T11:16:47.035414+02:00 pici kernel: [747711.621863] RDX: 0000000000000000 RSI: ffff9ba2bf6a03a0 RDI: ffff9ba2bf6a03a0
    2025-04-02T11:16:47.035414+02:00 pici kernel: [747711.631254] RBP: 0000000000000010 R08: 0000000000000000 R09: ffffa9fba32479f8
    2025-04-02T11:16:47.035415+02:00 pici kernel: [747711.640657] R10: 0000000000000003 R11: ffff9ba2ffb75fe8 R12: 0000000000000000
    2025-04-02T11:16:47.035415+02:00 pici kernel: [747711.650042] R13: ffffd11bca09ac88 R14: ffff9ba2ffbd5c00 R15: 0000000000000000
    2025-04-02T11:16:47.035416+02:00 pici kernel: [747711.659410] FS: 00007f9245392740(0000) GS:ffff9ba2bf680000(0000) knlGS:0000000000000000
    2025-04-02T11:16:47.035416+02:00 pici kernel: [747711.669723] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    2025-04-02T11:16:47.035416+02:00 pici kernel: [747711.677683] CR2: 00007f92449af024 CR3: 000000074b884000 CR4: 0000000000752ee0
    2025-04-02T11:16:47.035416+02:00 pici kernel: [747711.687062] PKRU: 55555554 2025-04-02T11:16:47.035417+02:00 pici kernel: [747711.691986] Call Trace: 2025-04-02T11:16:47.035417+02:00 pici kernel: [747711.696619] <TASK> 2025-04-02T11:16:47.035417+02:00 pici kernel: [747711.700866] ? __die_body.cold+0x1a/0x1f
    2025-04-02T11:16:47.035418+02:00 pici kernel: [747711.706918] ? die+0x2a/0x50 2025-04-02T11:16:47.035418+02:00 pici kernel: [747711.711930] ? do_trap+0xc5/0x110
    2025-04-02T11:16:47.035418+02:00 pici kernel: [747711.717348] ? __list_del_entry_valid.cold+0x23/0x6f
    2025-04-02T11:16:47.035419+02:00 pici kernel: [747711.724402] ? do_error_trap+0x6a/0x90
    2025-04-02T11:16:47.035419+02:00 pici kernel: [747711.730211] ? __list_del_entry_valid.cold+0x23/0x6f
    2025-04-02T11:16:47.035419+02:00 pici kernel: [747711.737229] ? exc_invalid_op+0x4c/0x60
    2025-04-02T11:16:47.035420+02:00 pici kernel: [747711.743122] ? __list_del_entry_valid.cold+0x23/0x6f
    2025-04-02T11:16:47.035420+02:00 pici kernel: [747711.750119] ? asm_exc_invalid_op+0x16/0x20
    2025-04-02T11:16:47.035420+02:00 pici kernel: [747711.756313] ? __list_del_entry_valid.cold+0x23/0x6f
    2025-04-02T11:16:47.035420+02:00 pici kernel: [747711.763278] rmqueue_bulk+0x124/0x5d0
    2025-04-02T11:16:47.035421+02:00 pici kernel: [747711.768917] get_page_from_freelist+0xbae/0x11b0
    2025-04-02T11:16:47.035421+02:00 pici kernel: [747711.775519] ? page_add_file_rmap+0xba/0x2f0
    2025-04-02T11:16:47.035421+02:00 pici kernel: [747711.781757] ? do_set_pte+0x189/0x1d0
    2025-04-02T11:16:47.035421+02:00 pici kernel: [747711.787377] __alloc_pages+0x1dc/0x330
    2025-04-02T11:16:47.035422+02:00 pici kernel: [747711.793095] __folio_alloc+0x17/0x50
    2025-04-02T11:16:47.035422+02:00 pici kernel: [747711.798651] ? policy_node+0x51/0x70
    2025-04-02T11:16:47.035422+02:00 pici kernel: [747711.804195] vma_alloc_folio+0x9c/0x370
    2025-04-02T11:16:47.035423+02:00 pici kernel: [747711.809989] __handle_mm_fault+0x92f/0xfa0
    2025-04-02T11:16:47.035423+02:00 pici kernel: [747711.816041] handle_mm_fault+0xdb/0x2d0
    2025-04-02T11:16:47.035423+02:00 pici kernel: [747711.821822] do_user_addr_fault+0x191/0x550
    2025-04-02T11:16:47.035423+02:00 pici kernel: [747711.827943] exc_page_fault+0x70/0x170
    2025-04-02T11:16:47.035424+02:00 pici kernel: [747711.833619] asm_exc_page_fault+0x22/0x30
    2025-04-02T11:16:47.035424+02:00 pici kernel: [747711.839553] RIP: 0033:0x7f9245143628
    2025-04-02T11:16:47.035424+02:00 pici kernel: [747711.845040] Code: fe ff ff 4c 89 f6 48 2b 35 95 52 20 00 49 8b 46 08 48 b9 ab aa aa aa aa aa aa aa 48 c1 fe 04 48 0f af f1 48 8d b8 00 10 00 00 <c7> 40 24 ff ff 00 00 89 70 20 49 89 7e 08 45 89 56 10 45
    85 d2 0f
    2025-04-02T11:16:47.035424+02:00 pici kernel: [747711.865857] RSP: 002b:00007ffe256b2510 EFLAGS: 00010a07
    2025-04-02T11:16:47.035425+02:00 pici kernel: [747711.873138] RAX: 00007f92449af000 RBX: 0000000000000004 RCX: aaaaaaaaaaaaaaab
    2025-04-02T11:16:47.035425+02:00 pici kernel: [747711.882354] RDX: 000000000000002f RSI: 0000000000000018 RDI: 00007f92449b0000
    2025-04-02T11:16:47.035425+02:00 pici kernel: [747711.891593] RBP: 0000000000000008 R08: 000000000000002f R09: 00007f9245348680
    2025-04-02T11:16:47.035425+02:00 pici kernel: [747711.900839] R10: 000000000000002e R11: 0000000019483a70 R12: 00007f92453215c0
    2025-04-02T11:16:47.035426+02:00 pici kernel: [747711.910096] R13: 0000000000000018 R14: 000000001934aaa0 R15: 000000000000002e
    2025-04-02T11:16:47.035426+02:00 pici kernel: [747711.919372] </TASK> 2025-04-02T11:16:47.035426+02:00 pici kernel: [747711.923698] Modules linked in: sunrpc vhost_net vhost vhost_iotlb tap tun veth ipt_REJECT nf_reject_ipv4 xt_tcpudp nft_compat nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
    nfnetlink bridge stp llc qrtr binfmt_misc snd_sof_pci_intel_tgl snd_sof_intel_hda_common soundwire_intel snd_hda_codec_hdmi soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci nls_ascii snd_sof_xtensa_dsp snd_hda_codec_realtek
    nls_cp437 snd_sof vfat snd_hda_codec_generic ledtrig_audio snd_sof_utils fat snd_soc_hdac_hda btusb snd_hda_ext_core snd_soc_acpi_intel_match intel_rapl_msr intel_rapl_common btrtl snd_soc_acpi btbcm x86_pkg_temp_thermal snd_soc_core intel_powerclamp
    iwlmvm coretemp btintel snd_compress mac80211 soundwire_bus btmtk kvm_intel i915 libarc4 kvm bluetooth snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi irqbypass ghash_clmulni_intel snd_hda_codec jitterentropy_rng iwlwifi drm_buddy sha256_ssse3 sha1_
    ssse3 drm_display_helper sha512_ssse3
    2025-04-02T11:16:47.035427+02:00 pici kernel: [747711.923723] aesni_intel cec snd_hda_core sha512_generic ctr rc_core cfg80211 drbg crypto_simd iTCO_wdt snd_hwdep ttm pl2303 snd_pcm intel_pmc_bxt drm_kms_helper ansi_cprng iTCO_vendor_support mei_hdcp
    usbserial ecdh_generic ecc evdev i2c_algo_bit cryptd snd_timer rapl intel_cstate snd rfkill mei_me watchdog soundcore intel_hid gigabyte_wmi wmi_bmof intel_uncore intel_pmc_core pcspkr sparse_keymap acpi_tad acpi_pad sg mei button ib_ipoib rdma_ucm rdma_
    cm iw_cm ib_cm ib_umad mlx5_ib mlx5_core mlxfw psample pci_hyperv_intf ib_uverbs ib_core drbd lru_cache loop fuse efi_pstore drm configfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 efivarfs raid10 raid456 async_raid6_recov async_memcpy async_pq
    async_xor async_tx xor raid6_pq libcrc32c crc32c_generic raid1 multipath linear hid_generic usbhid hid dm_mod raid0 md_mod uas usb_storage sd_mod t10_pi crc64_rocksoft crc64 crc_t10dif crct10dif_generic xhci_pci ahci xhci_hcd libahci libata crct10dif_
    pclmul crct10dif_common scsi_mod usbcore
    2025-04-02T11:16:47.035427+02:00 pici kernel: [747712.022525] igc e1000e crc32_pclmul crc32c_intel intel_lpss_pci i2c_i801 intel_lpss i2c_smbus video usb_common idma64 scsi_common fan wmi pinctrl_alderlake
    2025-04-02T11:16:47.035428+02:00 pici kernel: [747712.142019] ---[ end trace 0000000000000000 ]---

    -----BEGIN PGP SIGNATURE-----

    iQEzBAABCgAdFiEEP4GsaTp6HlmJrf7Tj4D7WH0S/k4FAmgIqkgACgkQj4D7WH0S /k5EPQf/c58SZC8qiKyQ9QbJp5OOQSOr9ngghnK4axTDZslPksfGcFTo7uuN8Rz6 2acbEFPrBDqGXRUGOAl4eAH6epJwtM0Tz7KCW0sdKllR0YEneppUFLg0hnOqFemj 2nZ7nmKwK39U94zYkZNeJX7s5HpB4+KhzM2MQBPrkPE/+PnVJ5W/itG/vumexhxI WLMVQG1Fm7J5SOr/i7gzP46+lx/IaOge0VlifwD8N3DSA2PqF185L9phWw/j2C2J xhhowHYK8lQ7enEdQ+E8CQqwwB0X3lsHFGQV5XnzAzjGnqDkAZXUigdGrpTI7L5L bkZVsj1OrB2RZMTZYS7R0S6fS3UuSQ==
    =tmJ8
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matthew Wilcox@21:1/5 to All on Wed Apr 23 14:50:01 2025
    XPost: linux.debian.kernel

    On Wed, Apr 23, 2025 at 10:52:27AM +0200, Uwe Kleine-König wrote:

    [trimmed all the irrelevant garbage from the start of each line]

    ------------[ cut here ]------------
    kernel BUG at lib/list_debug.c:62!

    That's:

    CHECK_DATA_CORRUPTION(next->prev != entry,
    "list_del corruption. next->prev should be %px, but was %px. (next=%px)\n",
    entry, next->prev, next))
    return false;

    Unfortunately, I don't see that rather useful information in this
    report.

    invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
    CPU: 10 PID: 6770 Comm: python3 Not tainted 6.1.0-32-amd64 #1 Debian 6.1.129-1
    Hardware name: Gigabyte Technology Co., Ltd. Z790 AERO G/Z790 AERO G, BIOS F13b 12/04/2024
    RIP: 0010:__list_del_entry_valid.cold+0x23/0x6f
    Code: e8 0d 9f fe ff 0f 0b 48 89 fe 48 c7 c7 98 62 fa bc e8 fc 9e fe ff 0f 0b 48 89 d1 48 c7 c7 b8 63 fa bc 48 89 c2 e8 e8 9e fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 68 63 fa bc e8 d4 9e fe ff 0f 0b
    RSP: 0000:ffffa9fba3247b60 EFLAGS: 00010046
    RAX: 000000000000006d RBX: ffff9ba2bf6b78e8 RCX: 0000000000000000
    RDX: 0000000000000000 RSI: ffff9ba2bf6a03a0 RDI: ffff9ba2bf6a03a0
    RBP: 0000000000000010 R08: 0000000000000000 R09: ffffa9fba32479f8
    R10: 0000000000000003 R11: ffff9ba2ffb75fe8 R12: 0000000000000000
    R13: ffffd11bca09ac88 R14: ffff9ba2ffbd5c00 R15: 0000000000000000
    FS: 00007f9245392740(0000) GS:ffff9ba2bf680000(0000) knlGS:0000000000000000
    CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: 00007f92449af024 CR3: 000000074b884000 CR4: 0000000000752ee0
    PKRU: 55555554
    Call Trace:
    <TASK>
    ? __die_body.cold+0x1a/0x1f
    ? die+0x2a/0x50
    ? do_trap+0xc5/0x110
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? do_error_trap+0x6a/0x90
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? exc_invalid_op+0x4c/0x60
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? asm_exc_invalid_op+0x16/0x20
    ? __list_del_entry_valid.cold+0x23/0x6f
    rmqueue_bulk+0x124/0x5d0
    get_page_from_freelist+0xbae/0x11b0

    We're in the page allocator. There's nothing particularly exciting to
    be found in this backtrace, but in future, running it through scripts/decode_stacktrace.sh can be very helpful.

    ? page_add_file_rmap+0xba/0x2f0
    ? do_set_pte+0x189/0x1d0
    __alloc_pages+0x1dc/0x330
    __folio_alloc+0x17/0x50
    ? policy_node+0x51/0x70
    vma_alloc_folio+0x9c/0x370
    __handle_mm_fault+0x92f/0xfa0
    handle_mm_fault+0xdb/0x2d0
    do_user_addr_fault+0x191/0x550
    exc_page_fault+0x70/0x170
    asm_exc_page_fault+0x22/0x30
    RIP: 0033:0x7f9245143628
    Code: fe ff ff 4c 89 f6 48 2b 35 95 52 20 00 49 8b 46 08 48 b9 ab aa aa aa aa aa aa aa 48 c1 fe 04 48 0f af f1 48 8d b8 00 10 00 00 <c7> 40 24 ff ff 00 00 89 70 20 49 89 7e 08 45 89 56 10 45 85 d2 0f
    RSP: 002b:00007ffe256b2510 EFLAGS: 00010a07
    RAX: 00007f92449af000 RBX: 0000000000000004 RCX: aaaaaaaaaaaaaaab
    RDX: 000000000000002f RSI: 0000000000000018 RDI: 00007f92449b0000
    RBP: 0000000000000008 R08: 000000000000002f R09: 00007f9245348680
    R10: 000000000000002e R11: 0000000019483a70 R12: 00007f92453215c0
    R13: 0000000000000018 R14: 000000001934aaa0 R15: 000000000000002e
    </TASK>
    Modules linked in: sunrpc vhost_net vhost vhost_iotlb tap tun veth ipt_REJECT nf_reject_ipv4 xt_tcpudp nft_compat nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables nfnetlink bridge stp llc qrtr binfmt_misc snd_sof_pci_intel_tgl
    snd_sof_intel_hda_common soundwire_intel snd_hda_codec_hdmi soundwire_generic_allocation soundwire_cadence snd_sof_intel_hda snd_sof_pci nls_ascii snd_sof_xtensa_dsp snd_hda_codec_realtek nls_cp437 snd_sof vfat snd_hda_codec_generic ledtrig_audio snd_sof_
    utils fat snd_soc_hdac_hda btusb snd_hda_ext_core snd_soc_acpi_intel_match intel_rapl_msr intel_rapl_common btrtl snd_soc_acpi btbcm x86_pkg_temp_thermal snd_soc_core intel_powerclamp iwlmvm coretemp btintel snd_compress mac80211 soundwire_bus btmtk kvm_
    intel i915 libarc4 kvm bluetooth snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi irqbypass ghash_clmulni_intel snd_hda_codec jitterentropy_rng iwlwifi drm_buddy sha256_ssse3 sha1_ssse3 drm_display_helper sha512_ssse3
    aesni_intel cec snd_hda_core sha512_generic ctr rc_core cfg80211 drbg crypto_simd iTCO_wdt snd_hwdep ttm pl2303 snd_pcm intel_pmc_bxt drm_kms_helper ansi_cprng iTCO_vendor_support mei_hdcp usbserial ecdh_generic ecc evdev i2c_algo_bit cryptd snd_
    timer rapl intel_cstate snd rfkill mei_me watchdog soundcore intel_hid gigabyte_wmi wmi_bmof intel_uncore intel_pmc_core pcspkr sparse_keymap acpi_tad acpi_pad sg mei button ib_ipoib rdma_ucm rdma_cm iw_cm ib_cm ib_umad mlx5_ib mlx5_core mlxfw psample
    pci_hyperv_intf ib_uverbs ib_core drbd lru_cache loop fuse efi_pstore drm configfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 efivarfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c crc32c_generic
    raid1 multipath linear hid_generic usbhid hid dm_mod raid0 md_mod uas usb_storage sd_mod t10_pi crc64_rocksoft crc64 crc_t10dif crct10dif_generic xhci_pci ahci xhci_hcd libahci libata crct10dif_pclmul crct10dif_common scsi_mod usbcore
    igc e1000e crc32_pclmul crc32c_intel intel_lpss_pci i2c_i801 intel_lpss i2c_smbus video usb_common idma64 scsi_common fan wmi pinctrl_alderlake
    ---[ end trace 0000000000000000 ]---

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Uwe =?utf-8?Q?Kleine-K=C3=B6nig?=@21:1/5 to Matthew Wilcox on Wed Apr 30 21:30:01 2025
    XPost: linux.debian.kernel

    Hello Alexis,

    On Wed, Apr 23, 2025 at 01:19:21PM +0100, Matthew Wilcox wrote:
    On Wed, Apr 23, 2025 at 10:52:27AM +0200, Uwe Kleine-König wrote:

    [trimmed all the irrelevant garbage from the start of each line]

    As Matthew pointed out the output of the CHECK_DATA_CORRUPTION would be interesting. Can you please check if you can find this line still in
    your journal?

    ------------[ cut here ]------------
    kernel BUG at lib/list_debug.c:62!

    That's:

    CHECK_DATA_CORRUPTION(next->prev != entry,
    "list_del corruption. next->prev should be %px, but was %px. (next=%px)\n",
    entry, next->prev, next))
    return false;

    Unfortunately, I don't see that rather useful information in this
    report.

    invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
    CPU: 10 PID: 6770 Comm: python3 Not tainted 6.1.0-32-amd64 #1 Debian 6.1.129-1
    Hardware name: Gigabyte Technology Co., Ltd. Z790 AERO G/Z790 AERO G, BIOS F13b 12/04/2024
    RIP: 0010:__list_del_entry_valid.cold+0x23/0x6f
    Code: e8 0d 9f fe ff 0f 0b 48 89 fe 48 c7 c7 98 62 fa bc e8 fc 9e fe ff 0f 0b 48 89 d1 48 c7 c7 b8 63 fa bc 48 89 c2 e8 e8 9e fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 68 63 fa bc e8 d4 9e fe ff 0f 0b
    RSP: 0000:ffffa9fba3247b60 EFLAGS: 00010046
    RAX: 000000000000006d RBX: ffff9ba2bf6b78e8 RCX: 0000000000000000
    RDX: 0000000000000000 RSI: ffff9ba2bf6a03a0 RDI: ffff9ba2bf6a03a0
    RBP: 0000000000000010 R08: 0000000000000000 R09: ffffa9fba32479f8
    R10: 0000000000000003 R11: ffff9ba2ffb75fe8 R12: 0000000000000000
    R13: ffffd11bca09ac88 R14: ffff9ba2ffbd5c00 R15: 0000000000000000
    FS: 00007f9245392740(0000) GS:ffff9ba2bf680000(0000) knlGS:0000000000000000
    CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    CR2: 00007f92449af024 CR3: 000000074b884000 CR4: 0000000000752ee0
    PKRU: 55555554
    Call Trace:
    <TASK>
    ? __die_body.cold+0x1a/0x1f
    ? die+0x2a/0x50
    ? do_trap+0xc5/0x110
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? do_error_trap+0x6a/0x90
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? exc_invalid_op+0x4c/0x60
    ? __list_del_entry_valid.cold+0x23/0x6f
    ? asm_exc_invalid_op+0x16/0x20
    ? __list_del_entry_valid.cold+0x23/0x6f
    rmqueue_bulk+0x124/0x5d0
    get_page_from_freelist+0xbae/0x11b0

    We're in the page allocator. There's nothing particularly exciting to
    be found in this backtrace, but in future, running it through scripts/decode_stacktrace.sh can be very helpful.

    I can do that. Will follow up when I find the time.

    Best regards
    Uwe

    -----BEGIN PGP SIGNATURE-----

    iQEzBAABCgAdFiEEP4GsaTp6HlmJrf7Tj4D7WH0S/k4FAmgSd0wACgkQj4D7WH0S /k5GGgf9Es/KXqGxqLkzqUMJvyPNb5aeQaO0KR49+XRBD/kK/8wEhNK4CMM4WGhg qHkvJwz4jQpBS2qs8nEgs32gDOX8Uxmkdpg0iDyMI6GNIZ+P5vxQJiEKAzB0uLi/ 53FKNJ+gI2EhLmc7ZvERX2PLJsufVXUBPq8WYI+RIcCIZPu2jK0jxBJnoRWFArfM 91txB5o6zXfI2oj6eVuU1UF7SNi7X51Tp2a1KJWPwIbC7wJe1AULY4I3V6WzwQNr NyuJ3HOYSVwMGkkRxfsZyEjswkrrU77qndTY1ZJblEnI0hS6cqoHC33Mr9E4H0iE QJs6pDdtZNVwKLeZCgIeYIgQzIFFZg==
    =r6wr
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)