Hi all,

This is likley not a newsreader issue, but I'm not sure where else
I should post my question.

I've been having trouble with charset translation of quoted text
in posts from slrn. It only seems to occur with a few posters and
only in one group that I've noticed. The posts in question are
generated from NewsTap/5.5 and G2, and I'm told at least one of
them is using utf-8.

I've played around with PuTTY and slrn charset configs and can get
to something that works for me in terms of display, but when I
start a reply the problem appears. I call pico as the editor.

Interestingly, when I post from a second shell account (Nyx) using
slrn with identical configs, the charset issue does not appear.
On further digging, I find this on Nyx:

theise@nyx2:~$ pico --version
GNU nano, version 4.8
(C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
(C) 2014-2020 the contributors to nano

I've pointed slrn to nano on my panix account, but no joy. I
don't find a way to configure the charset used by nano (or pine,
for that matter) and my sense from the scant info on the web is
that it may be a compiled in option. Another idea that I've had is
perhaps my locale or env settings are at play--but I can't find
anything relevant.

Any tips on what I should try next would be greatly appreciated!

FWIW, I do prefer pico to nano, because it does automatic soft
line wrapping as I edit, but I suspect nano would do this too if I
found the right config parameter.

--
Ted Heise <theise@panix.com> West Lafayette, IN, USA
~

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

Hi all,

This is likley not a newsreader issue, but I'm not sure where else
I should post my question.

I've been having trouble with charset translation of quoted text
in posts from slrn. It only seems to occur with a few posters and
only in one group that I've noticed. The posts in question are
generated from NewsTap/5.5 and G2, and I'm told at least one of
them is using utf-8.

I've played around with PuTTY and slrn charset configs and can get
to something that works for me in terms of display, but when I
start a reply the problem appears. I call pico as the editor.

Interestingly, when I post from a second shell account (Nyx) using
slrn with identical configs, the charset issue does not appear.
On further digging, I find this on Nyx:

theise@nyx2:~$ pico --version
GNU nano, version 4.8
(C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
(C) 2014-2020 the contributors to nano

I've pointed slrn to nano on my panix account, but no joy. I
don't find a way to configure the charset used by nano (or pine,
for that matter) and my sense from the scant info on the web is
that it may be a compiled in option. Another idea that I've had is
perhaps my locale or env settings are at play--but I can't find
anything relevant.

Any tips on what I should try next would be greatly appreciated!

FWIW, I do prefer pico to nano, because it does automatic soft
line wrapping as I edit, but I suspect nano would do this too if I
found the right config parameter.

Hello,

Does section 2.4 of the following document help? It regards
using UTF-8 with external editors:

https://slrn.info/docs/slrn-FAQ.txt

The instructions in that section regard Windows, but I can't
see that you couldn't use those settings for invoking an external
utf-8-capable editor.

Then I created a file with a utf-8 character.[*] On Linux Mint, both
joe(1) and nano(1) displayed it properly. (pico is a link to nano).

_ _ _ _ _ _ _
$ nano --version
GNU nano, version 6.2
(C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
(C) 2014-2022 the contributors to nano
Compiled options: --disable-libmagic --enable-utf8
_ _ _ _ _ _ _

So it appears nano may need to be compiled with --enable-utf8
to support that functionality, if I'm not thoroughly mistaken.

(I don't use slrn, so not sure if this helps.)

[*] utf-8 file I created was basically my command prompt:

_[/home/vallor]_(vallor@lm)🐧_
$

--
-v
🖖️

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
vallor <vallor@cultnix.org> wrote:

On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

Hi all,

This is likley not a newsreader issue, but I'm not sure where else
I should post my question.

I've been having trouble with charset translation of quoted text
in posts from slrn. It only seems to occur with a few posters and
only in one group that I've noticed. The posts in question are
generated from NewsTap/5.5 and G2, and I'm told at least one of
them is using utf-8.

I've played around with PuTTY and slrn charset configs and can get
to something that works for me in terms of display, but when I
start a reply the problem appears. I call pico as the editor.

Interestingly, when I post from a second shell account (Nyx) using
slrn with identical configs, the charset issue does not appear.
On further digging, I find this on Nyx:

theise@nyx2:~$ pico --version
GNU nano, version 4.8
(C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
(C) 2014-2020 the contributors to nano

I've pointed slrn to nano on my panix account, but no joy. I
don't find a way to configure the charset used by nano (or pine,
for that matter) and my sense from the scant info on the web is
that it may be a compiled in option. Another idea that I've had is
perhaps my locale or env settings are at play--but I can't find
anything relevant.

Any tips on what I should try next would be greatly appreciated!

FWIW, I do prefer pico to nano, because it does automatic soft
line wrapping as I edit, but I suspect nano would do this too if I
found the right config parameter.

Hello,

Does section 2.4 of the following document help? It regards
using UTF-8 with external editors:

https://slrn.info/docs/slrn-FAQ.txt

The instructions in that section regard Windows, but I can't
see that you couldn't use those settings for invoking an external
utf-8-capable editor.

Then I created a file with a utf-8 character.[*] On Linux Mint, both
joe(1) and nano(1) displayed it properly. (pico is a link to nano).

_ _ _ _ _ _ _
$ nano --version
GNU nano, version 6.2
(C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
(C) 2014-2022 the contributors to nano
Compiled options: --disable-libmagic --enable-utf8
_ _ _ _ _ _ _

So it appears nano may need to be compiled with --enable-utf8
to support that functionality, if I'm not thoroughly mistaken.

(I don't use slrn, so not sure if this helps.)

[*] utf-8 file I created was basically my command prompt:

_[/home/vallor]_(vallor@lm)🐧_
$

Very good suggestions, thank you! I'm not sure that has worked
(used the charset editor "utf-8" config option with a call to
nano), do your 8-bit characters show correctly for you?

At the least, this shows I'm on the right track.

Now off to see if I can tell whether pico and nano on my shell
server have utf-8 compiled in.

Thanks again for the help!

--
Ted Heise <theise@panix.com> West Lafayette, IN, USA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 20 Oct 2023 13:42:47 -0000 (UTC), Ted Heise <theise@panix.com>
wrote in <slrnuj50un.5rm.theise@panix2.panix.com>:

On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
vallor <vallor@cultnix.org> wrote:

On Thu, 19 Oct 2023 18:34:46 -0000 (UTC), Ted Heise <theise@panix.com>
wrote in <slrnuj2tm6.a30.theise@panix2.panix.com>:

Hi all,

This is likley not a newsreader issue, but I'm not sure where else
I should post my question.

I've been having trouble with charset translation of quoted text
in posts from slrn. It only seems to occur with a few posters and
only in one group that I've noticed. The posts in question are
generated from NewsTap/5.5 and G2, and I'm told at least one of
them is using utf-8.

I've played around with PuTTY and slrn charset configs and can get
to something that works for me in terms of display, but when I
start a reply the problem appears. I call pico as the editor.

Interestingly, when I post from a second shell account (Nyx) using
slrn with identical configs, the charset issue does not appear.
On further digging, I find this on Nyx:

theise@nyx2:~$ pico --version
GNU nano, version 4.8
(C) 1999-2011, 2013-2020 Free Software Foundation, Inc.
(C) 2014-2020 the contributors to nano

I've pointed slrn to nano on my panix account, but no joy. I
don't find a way to configure the charset used by nano (or pine,
for that matter) and my sense from the scant info on the web is
that it may be a compiled in option. Another idea that I've had is
perhaps my locale or env settings are at play--but I can't find
anything relevant.

Any tips on what I should try next would be greatly appreciated!

FWIW, I do prefer pico to nano, because it does automatic soft
line wrapping as I edit, but I suspect nano would do this too if I
found the right config parameter.

Hello,

Does section 2.4 of the following document help? It regards
using UTF-8 with external editors:

https://slrn.info/docs/slrn-FAQ.txt

The instructions in that section regard Windows, but I can't
see that you couldn't use those settings for invoking an external
utf-8-capable editor.

Then I created a file with a utf-8 character.[*] On Linux Mint, both
joe(1) and nano(1) displayed it properly. (pico is a link to nano).

_ _ _ _ _ _ _
$ nano --version
GNU nano, version 6.2
(C) 1999-2011, 2013-2022 Free Software Foundation, Inc.
(C) 2014-2022 the contributors to nano
Compiled options: --disable-libmagic --enable-utf8
_ _ _ _ _ _ _

So it appears nano may need to be compiled with --enable-utf8
to support that functionality, if I'm not thoroughly mistaken.

(I don't use slrn, so not sure if this helps.)

[*] utf-8 file I created was basically my command prompt:

_[/home/vallor]_(vallor@lm)🐧_
$

Very good suggestions, thank you! I'm not sure that has worked
(used the charset editor "utf-8" config option with a call to
nano), do your 8-bit characters show correctly for you?

At the least, this shows I'm on the right track.

Now off to see if I can tell whether pico and nano on my shell
server have utf-8 compiled in.

Thanks again for the help!

Okay, I got an account on panix and checked out the situation.

Add this to your .bash_profile (or whatever):

export LC_CTYPE=en_US.UTF-8

Now both nano and slrn display the UTF-8 properly. :)

BTW, their system is running NetBSD.

--
-v

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 21 Oct 2023 02:35:02 -0000 (UTC),
vallor <vallor@cultnix.org> wrote:

On Fri, 20 Oct 2023 13:42:47 -0000 (UTC), Ted Heise <theise@panix.com>
wrote in <slrnuj50un.5rm.theise@panix2.panix.com>:

On Fri, 20 Oct 2023 03:07:35 -0000 (UTC),
vallor <vallor@cultnix.org> wrote:

Does section 2.4 of the following document help? It regards
using UTF-8 with external editors:

https://slrn.info/docs/slrn-FAQ.txt

The instructions in that section regard Windows, but I can't
see that you couldn't use those settings for invoking an external
utf-8-capable editor.

Very good suggestions, thank you! I'm not sure that has
worked (used the charset editor "utf-8" config option with a
call to nano), do your 8-bit characters show correctly for
you?

At the least, this shows I'm on the right track.

Now off to see if I can tell whether pico and nano on my shell
server have utf-8 compiled in.

Okay, I got an account on panix and checked out the situation.

Add this to your .bash_profile (or whatever):

export LC_CTYPE=en_US.UTF-8

Now both nano and slrn display the UTF-8 properly. :)

I think I had earned that nano did utf-8 and had found ways to
configure it to my liking. But I'm trying this with my original
pico config and the environment options you suggested. Seems to
work as well.

Not sure which approach I'll stick with, but I sure learned a lot!

BTW, their system is running NetBSD.

Yeah, I probably should have mentioned that.

Thanks so much for all the help, you really went above and beyond.

--
Ted Heise <theise@panix.com> West Lafayette, IN, USA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)