• charset issues

    From Ted Heise@21:1/5 to All on Thu Oct 19 18:34:46 2023
    Hi all,

    This is likley not a newsreader issue, but I'm not sure where else
    I should post my question.

    I've been having trouble with charset translation of quoted text
    in posts from slrn. It only seems to occur with a few posters and
    only in one group that I've noticed. The posts in question are
    generated from NewsTap/5.5 and G2, and I'm told at least one of
    them is using utf-8.

    I've played around with PuTTY and slrn charset configs and can get
    to something that works for me in terms of display, but when I
    start a reply the problem appears. I call pico as the editor.

    Interestingly, when I post from a second shell account (Nyx) using
    slrn with identical configs, the charset issue does not appear.
    On further digging, I find this on Nyx:

    theise@nyx2:~$ pico --version
    GNU nano, version 4.8
    (C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
    (C) 2014-2020 the contributors to nano

    I've pointed slrn to nano on my panix account, but no joy. I
    don't find a way to configure the charset used by nano (or pine,
    for that matter) and my sense from the scant info on the web is
    that it may be a compiled in option. Another idea that I've had is
    perhaps my locale or env settings are at play--but I can't find
    anything relevant.

    Any tips on what I should try next would be greatly appreciated!

    FWIW, I do prefer pico to nano, because it does automatic soft
    line wrapping as I edit, but I suspect nano would do this too if I
    found the right config parameter.

    --
    Ted Heise <theise@panix.com> West Lafayette, IN, USA
    ~

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From vallor@21:1/5 to All on Fri Oct 20 03:07:35 2023
    On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
    wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

    Hi all,

    This is likley not a newsreader issue, but I'm not sure where else
    I should post my question.

    I've been having trouble with charset translation of quoted text
    in posts from slrn. It only seems to occur with a few posters and
    only in one group that I've noticed. The posts in question are
    generated from NewsTap/5.5 and G2, and I'm told at least one of
    them is using utf-8.

    I've played around with PuTTY and slrn charset configs and can get
    to something that works for me in terms of display, but when I
    start a reply the problem appears. I call pico as the editor.

    Interestingly, when I post from a second shell account (Nyx) using
    slrn with identical configs, the charset issue does not appear.
    On further digging, I find this on Nyx:

    theise@nyx2:~$ pico --version
    GNU nano, version 4.8
    (C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
    (C) 2014-2020 the contributors to nano

    I've pointed slrn to nano on my panix account, but no joy. I
    don't find a way to configure the charset used by nano (or pine,
    for that matter) and my sense from the scant info on the web is
    that it may be a compiled in option. Another idea that I've had is
    perhaps my locale or env settings are at play--but I can't find
    anything relevant.

    Any tips on what I should try next would be greatly appreciated!

    FWIW, I do prefer pico to nano, because it does automatic soft
    line wrapping as I edit, but I suspect nano would do this too if I
    found the right config parameter.

    Hello,

    Does section 2.4 of the following document help? It regards
    using UTF-8 with external editors:

    https://slrn.info/docs/slrn-FAQ.txt

    The instructions in that section regard Windows, but I can't
    see that you couldn't use those settings for invoking an external
    utf-8-capable editor.

    Then I created a file with a utf-8 character.[*] On Linux Mint, both
    joe(1) and nano(1) displayed it properly. (pico is a link to nano).

    _ _ _ _ _ _ _
    $ nano --version
    GNU nano, version 6.2
    (C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
    (C) 2014-2022 the contributors to nano
    Compiled options: --disable-libmagic --enable-utf8
    _ _ _ _ _ _ _

    So it appears nano may need to be compiled with --enable-utf8
    to support that functionality, if I'm not thoroughly mistaken.

    (I don't use slrn, so not sure if this helps.)

    [*] utf-8 file I created was basically my command prompt:

    _[/home/vallor]_(vallor@lm)🐧_
    $

    --
    -v
    🖖️

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ted Heise@21:1/5 to vallor on Fri Oct 20 13:42:47 2023
    On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
    vallor <vallor@cultnix.org> wrote:
    On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
    wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

    Hi all,

    This is likley not a newsreader issue, but I'm not sure where else
    I should post my question.

    I've been having trouble with charset translation of quoted text
    in posts from slrn. It only seems to occur with a few posters and
    only in one group that I've noticed. The posts in question are
    generated from NewsTap/5.5 and G2, and I'm told at least one of
    them is using utf-8.

    I've played around with PuTTY and slrn charset configs and can get
    to something that works for me in terms of display, but when I
    start a reply the problem appears. I call pico as the editor.

    Interestingly, when I post from a second shell account (Nyx) using
    slrn with identical configs, the charset issue does not appear.
    On further digging, I find this on Nyx:

    theise@nyx2:~$ pico --version
    GNU nano, version 4.8
    (C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
    (C) 2014-2020 the contributors to nano

    I've pointed slrn to nano on my panix account, but no joy. I
    don't find a way to configure the charset used by nano (or pine,
    for that matter) and my sense from the scant info on the web is
    that it may be a compiled in option. Another idea that I've had is
    perhaps my locale or env settings are at play--but I can't find
    anything relevant.

    Any tips on what I should try next would be greatly appreciated!

    FWIW, I do prefer pico to nano, because it does automatic soft
    line wrapping as I edit, but I suspect nano would do this too if I
    found the right config parameter.

    Hello,

    Does section 2.4 of the following document help? It regards
    using UTF-8 with external editors:

    https://slrn.info/docs/slrn-FAQ.txt

    The instructions in that section regard Windows, but I can't
    see that you couldn't use those settings for invoking an external
    utf-8-capable editor.

    Then I created a file with a utf-8 character.[*] On Linux Mint, both
    joe(1) and nano(1) displayed it properly. (pico is a link to nano).

    _ _ _ _ _ _ _
    $ nano --version
    GNU nano, version 6.2
    (C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
    (C) 2014-2022 the contributors to nano
    Compiled options: --disable-libmagic --enable-utf8
    _ _ _ _ _ _ _

    So it appears nano may need to be compiled with --enable-utf8
    to support that functionality, if I'm not thoroughly mistaken.

    (I don't use slrn, so not sure if this helps.)

    [*] utf-8 file I created was basically my command prompt:

    _[/home/vallor]_(vallor@lm)🐧_
    $

    Very good suggestions, thank you! I'm not sure that has worked
    (used the charset editor "utf-8" config option with a call to
    nano), do your 8-bit characters show correctly for you?

    At the least, this shows I'm on the right track.

    Now off to see if I can tell whether pico and nano on my shell
    server have utf-8 compiled in.

    Thanks again for the help!

    --
    Ted Heise <theise@panix.com> West Lafayette, IN, USA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From vallor@21:1/5 to All on Sat Oct 21 02:35:02 2023
    On Fri, 20 Oct 2023 13:42:47 -0000 (UTC), Ted Heise <theise@panix.com>
    wrote in <slrnuj50un.5rm.theise@panix2.panix.com>:

    On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
    vallor <vallor@cultnix.org> wrote:
    On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
    wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

    Hi all,

    This is likley not a newsreader issue, but I'm not sure where else
    I should post my question.

    I've been having trouble with charset translation of quoted text
    in posts from slrn. It only seems to occur with a few posters and
    only in one group that I've noticed. The posts in question are
    generated from NewsTap/5.5 and G2, and I'm told at least one of
    them is using utf-8.

    I've played around with PuTTY and slrn charset configs and can get
    to something that works for me in terms of display, but when I
    start a reply the problem appears. I call pico as the editor.

    Interestingly, when I post from a second shell account (Nyx) using
    slrn with identical configs, the charset issue does not appear.
    On further digging, I find this on Nyx:

    theise@nyx2:~$ pico --version
    GNU nano, version 4.8
    (C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
    (C) 2014-2020 the contributors to nano

    I've pointed slrn to nano on my panix account, but no joy. I
    don't find a way to configure the charset used by nano (or pine,
    for that matter) and my sense from the scant info on the web is
    that it may be a compiled in option. Another idea that I've had is
    perhaps my locale or env settings are at play--but I can't find
    anything relevant.

    Any tips on what I should try next would be greatly appreciated!

    FWIW, I do prefer pico to nano, because it does automatic soft
    line wrapping as I edit, but I suspect nano would do this too if I
    found the right config parameter.

    Hello,

    Does section 2.4 of the following document help? It regards
    using UTF-8 with external editors:

    https://slrn.info/docs/slrn-FAQ.txt

    The instructions in that section regard Windows, but I can't
    see that you couldn't use those settings for invoking an external
    utf-8-capable editor.

    Then I created a file with a utf-8 character.[*] On Linux Mint, both
    joe(1) and nano(1) displayed it properly. (pico is a link to nano).

    _ _ _ _ _ _ _
    $ nano --version
    GNU nano, version 6.2
    (C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
    (C) 2014-2022 the contributors to nano
    Compiled options: --disable-libmagic --enable-utf8
    _ _ _ _ _ _ _

    So it appears nano may need to be compiled with --enable-utf8
    to support that functionality, if I'm not thoroughly mistaken.

    (I don't use slrn, so not sure if this helps.)

    [*] utf-8 file I created was basically my command prompt:

    _[/home/vallor]_(vallor@lm)🐧_
    $

    Very good suggestions, thank you! I'm not sure that has worked
    (used the charset editor "utf-8" config option with a call to
    nano), do your 8-bit characters show correctly for you?

    At the least, this shows I'm on the right track.

    Now off to see if I can tell whether pico and nano on my shell
    server have utf-8 compiled in.

    Thanks again for the help!

    Okay, I got an account on panix and checked out the situation.

    Add this to your .bash_profile (or whatever):

    export LC_CTYPE=en_US.UTF-8

    Now both nano and slrn display the UTF-8 properly. :)

    BTW, their system is running NetBSD.

    --
    -v

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ted Heise@21:1/5 to vallor on Sat Oct 21 17:05:57 2023
    On Sat, 21 Oct 2023 02:35:02 -0000 (UTC),
    vallor <vallor@cultnix.org> wrote:
    On Fri, 20 Oct 2023 13:42:47 -0000 (UTC), Ted Heise <theise@panix.com>
    wrote in <slrnuj50un.5rm.theise@panix2.panix.com>:
    On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
    vallor <vallor@cultnix.org> wrote:

    Does section 2.4 of the following document help? It regards
    using UTF-8 with external editors:

    https://slrn.info/docs/slrn-FAQ.txt

    The instructions in that section regard Windows, but I can't
    see that you couldn't use those settings for invoking an external
    utf-8-capable editor.

    Very good suggestions, thank you! I'm not sure that has
    worked (used the charset editor "utf-8" config option with a
    call to nano), do your 8-bit characters show correctly for
    you?

    At the least, this shows I'm on the right track.

    Now off to see if I can tell whether pico and nano on my shell
    server have utf-8 compiled in.

    Okay, I got an account on panix and checked out the situation.

    Add this to your .bash_profile (or whatever):

    export LC_CTYPE=en_US.UTF-8

    Now both nano and slrn display the UTF-8 properly. :)

    I think I had earned that nano did utf-8 and had found ways to
    configure it to my liking. But I'm trying this with my original
    pico config and the environment options you suggested. Seems to
    work as well.

    Not sure which approach I'll stick with, but I sure learned a lot!


    BTW, their system is running NetBSD.

    Yeah, I probably should have mentioned that.

    Thanks so much for all the help, you really went above and beyond.

    --
    Ted Heise <theise@panix.com> West Lafayette, IN, USA

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)