On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:

Hi all,

I have a fairly large killfile. Frequently, Gnus fetches the news and shows me that there are groups with unread messages, but when I try to go into one of those groups, Gnus fetches more information and realizes that all of the unread articles have been scored down below the threshold of being visible.

This is a bit annoying. I'd like to get Gnus to apply my scores in each
group as soon as it has fetched news.

This is my attempt to get that behavior:

(defun enter-all-groups-to-force-scoring ()
(gnus-message 5 "Scoring...")
(catch 'done-looping
(let ((seen-newsgroups '())
(most-recent-newsgroup nil))
(while t
(gnus-group-select-group)
(setq most-recent-newsgroup gnus-newsgroup-name)
(gnus-summary-exit)
(if (seq-contains-p seen-newsgroups most-recent-newsgroup)
(throw 'done-looping nil)
(setq seen-newsgroups (cons most-recent-newsgroup
seen-newsgroups))))))
(gnus-message 5 "Scoring... done"))

(add-hook 'gnus-group-prepare-hook
'enter-all-groups-to-force-scoring)
(add-hook 'gnus-after-getting-new-news-hook
'enter-all-groups-to-force-scoring)

This seems to work *sometimes*--I can watch as groups that had unread articles are reduced to zero unread articles. Other times, though, it seems that scores in a particular group were not applied, and I run into the sequence of events described at the top of this post.

(This solution also opens the last newsgroup twice, because I wasn't sure
how else to determine that we're done looping through the groups!)

Is there a better way of accomplishing this bit of configuration?

Thanks,

Benjamin

(I also asked this question on the Emacs Stack Exchange site at <https://emacs.stackexchange.com/q/77110/24262>.)

I'm not a LISP guy, but are you by chance scoring on article
headers (or the article body) that are not available until the
whole article is retrieved?

News servers only dish out a subset of article headers when
getting a list of articles for perusal and initial scoring(*).
Only when the entire article is retrieved for full viewing can it
be scored on "non-XOVER" headers (or the body text).

(*) To see the list of headers your news server provides
initially, telnet into your news server (usually at port 119, at
least) and give it a "list overview.fmt" command.

Doubtful you're on Windows 1[0-1], but if you are it would look
something like this:

Microsoft Windows [Version 10.0.19044.2965]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>cmd.exe
'cmd.exe' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>command.com
'command.com' is not recognized as an internal or external
command, operable program or batch file.

C:\Users\Sqwertz>fuck you
'fuck' is not recognized as an internal or external command,
operable program or batch file.

So it's a good thing you're not using Windows, eh?

-sw

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Sqwertz <sqwertzme@gmail.compost> wrote:

On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:

[...]

I'm not a LISP guy, but are you by chance scoring on article
headers (or the article body) that are not available until the
whole article is retrieved?

News servers only dish out a subset of article headers when
getting a list of articles for perusal and initial scoring(*).
Only when the entire article is retrieved for full viewing can it
be scored on "non-XOVER" headers (or the body text).

(*) To see the list of headers your news server provides
initially, telnet into your news server (usually at port 119, at
least) and give it a "list overview.fmt" command.

Doubtful you're on Windows 1[0-1], but if you are it would look
something like this:

Microsoft Windows [Version 10.0.19044.2965]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

On Windows - also on 10 and 11 - you can still enable Telnet:

Control Panel -> Programs -> Programs and Features -> Turn Windows
features on and off. Towards the bottom of the list there is 'Telnet
Client' with a tick-box.

C:\Users\Sqwertz>cmd.exe
'cmd.exe' is not recognized as an internal or external command,
operable program or batch file.

Hmmm!? Strange! Apparently C:\WINDOWS\system32 is not in your Path
variable.

Are you using a Windows Terminal window instead of a Command Prompt
window? (I only use the latter.)

C:\Users\Sqwertz>command.com
'command.com' is not recognized as an internal or external
command, operable program or batch file.

IIRC, command.com is from the pre-/non-NT era.

C:\Users\Sqwertz>fuck you
'fuck' is not recognized as an internal or external command,
operable program or batch file.

So it's a good thing you're not using Windows, eh?

Well, I'm running tin and Hamster on Windows, so ... :-)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 12 May 2023 14:32:25 GMT, Frank Slootweg wrote:

Sqwertz <sqwertzme@gmail.compost> wrote:

On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:

[...]

I'm not a LISP guy, but are you by chance scoring on article
headers (or the article body) that are not available until the
whole article is retrieved?

News servers only dish out a subset of article headers when
getting a list of articles for perusal and initial scoring(*).
Only when the entire article is retrieved for full viewing can it
be scored on "non-XOVER" headers (or the body text).

(*) To see the list of headers your news server provides
initially, telnet into your news server (usually at port 119, at
least) and give it a "list overview.fmt" command.

Doubtful you're on Windows 1[0-1], but if you are it would look
something like this:

Microsoft Windows [Version 10.0.19044.2965]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

On Windows - also on 10 and 11 - you can still enable Telnet:

Control Panel -> Programs -> Programs and Features -> Turn Windows
features on and off. Towards the bottom of the list there is 'Telnet
Client' with a tick-box.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

...does the same thing.

C:\Users\Sqwertz>cmd.exe
'cmd.exe' is not recognized as an internal or external command,
operable program or batch file.

Hmmm!? Strange! Apparently C:\WINDOWS\system32 is not in your Path variable.

Are you using a Windows Terminal window instead of a Command Prompt
window? (I only use the latter.)

C:\Users\Sqwertz>command.com
'command.com' is not recognized as an internal or external
command, operable program or batch file.

IIRC, command.com is from the pre-/non-NT era.

I needed to fork a new shell after the pkgadd for telnet to work.

Either one would have sufficed, but I was being lazy and was
demonstrating the irony of not including a standard OS
command/utility that only occupies a measly 132K of disk space in
Windows 10+ ... considering all the other bloated space-wasting
trash including in the OS.

(AND....I forgot the sequence to get into a newserver as a guest
for its overview.fmt and/or active list without posting my USER
and PASS creds <heh>)

-sw

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Sqwertz wrote:

(AND....I forgot the sequence to get into a newserver as a guest
for its overview.fmt and/or active list without posting my USER
and PASS creds <heh>)

i believe that's not inherent in op's case. point is, gnus by default
applies score rules once you enter groups, and there's a logic behind
this; a way to have the actual #'s of unread articles in group (or
topic) buffer is by using the function i pointed out.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

issdr wrote:

Benjamin Esham wrote:

I have a fairly large killfile. Frequently, Gnus fetches the news and
shows me that there are groups with unread messages, but when I try to
go into one of those groups, Gnus fetches more information and
realizes that all of the unread articles have been scored down below
the threshold of being visible.

did you try `M-x g-ba-s' from group buffer?

I had not tried that, thank you! (The full name is gnus-batch-score, for my future reference.) I tried to run that function automatically with

(add-hook 'gnus-started-hook 'gnus-batch-score)
(add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)

but that didn't seem to work: there was a very long delay after I started Gnus--maybe as long as a minute--but still not all of my scores had been applied.

I went into the Gnus source and extracted the "main part" of that function, which is

(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit)))))

and found that if I ran this interactively (at the M-: prompt) after news
had been fetched, then the scores would really, truly be applied. This is promising! Now I just need to put the code back into a function and figure
out exactly when and how to call it in order to make all of this automatic.

if that was not what you had in mind, you could also post on <nntp://news.eternal-september.org/gnu.emacs.gnus>.

I'm pretty sure I've posted in that group before, and yet Gnus is telling me that it doesn't exist on the server. But that will be a separate thing to debug...

Thanks,

Benjamin

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Benjamin Esham wrote:

issdr wrote:

[...]

(add-hook 'gnus-started-hook 'gnus-batch-score)
(add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)

but that didn't seem to work: there was a very long delay after I started Gnus--maybe as long as a minute--but still not all of my scores had been applied.

those vars are labeled as risky in local files. second one generated an
endless loop, as `gnus-batch-score' does get new news itself.

I went into the Gnus source and extracted the "main part" of that function, which is

(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit)))))

and found that if I ran this interactively (at the M-: prompt) after news
had been fetched, then the scores would really, truly be applied.

good. define a new function with it, then *add* it to the after-getting-new-news hook (you were replacing it, and it's non empty
by default), like

(eval-after-load "gnus-start"
'(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))

[...]

<nntp://news.eternal-september.org/gnu.emacs.gnus>.

I'm pretty sure I've posted in that group before, and yet Gnus is
telling me that it doesn't exist on the server. But that will be a
separate thing to debug...

i believe i have an account there, i'll check when i reach my pc

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

issdr wrote:

Benjamin Esham wrote:

issdr wrote:

[...]

(add-hook 'gnus-started-hook 'gnus-batch-score)
(add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)

but that didn't seem to work: there was a very long delay after I started
Gnus--maybe as long as a minute--but still not all of my scores had been
applied.

those vars are labeled as risky in local files. second one generated an endless loop, as `gnus-batch-score' does get new news itself.

Can you tell me where/how they're labeled as risky? I'm pretty new to Emacs Lisp.

I went into the Gnus source and extracted the "main part" of that
function, which is

(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit)))))

and found that if I ran this interactively (at the M-: prompt) after news
had been fetched, then the scores would really, truly be applied.

good. define a new function with it, then *add* it to the after-getting-new-news hook (you were replacing it, and it's non empty
by default), like

(eval-after-load "gnus-start"
'(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))

[...]

Are you sure? The Elisp manual [1] seems to make it pretty clear that
add-hook will *append* the given function to the given hook, not replace any previously-added functions.

<nntp://news.eternal-september.org/gnu.emacs.gnus>.

I'm pretty sure I've posted in that group before, and yet Gnus is
telling me that it doesn't exist on the server. But that will be a
separate thing to debug...

i believe i have an account there, i'll check when i reach my pc

It looks like I misremembered the error I had gotten. Gnus told me "Group gnu.emacs.gnus contains no messages", which is surprising (*no one* has
posted recently enough for Eternal September to retain the message?) but not completely implausible.

Benjamin


[1] https://www.gnu.org/software/emacs/manual/html_node/elisp/Setting-Hooks.html#index-add_002dhook

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Benjamin Esham wrote:

issdr wrote:

[...]

those vars are labeled as risky in local files. second one generated an
endless loop, as `gnus-batch-score' does get new news itself.

Can you tell me where/how they're labeled as risky? I'm pretty new to Emacs Lisp.

go `C-h v RET' while on one


[...]

(you were replacing it, and it's non empty by default), like

(eval-after-load "gnus-start"
'(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))

[...]

Are you sure?

nope, my bad. that must have been me using a setq in the rush.

The Elisp manual [1] seems to make it pretty clear that add-hook will *append* the given function to the given hook, not replace any previously-added functions.

thanks for the refresher, pretty useful function btw.

i tried the piece of code you isolated, it works both at startup and
when checking news again, it just gives warnings of no group found once
it's thru.

[...]

It looks like I misremembered the error I had gotten. Gnus told me "Group gnu.emacs.gnus contains no messages", which is surprising (*no one* has posted recently enough for Eternal September to retain the message?) but not completely implausible.

no high volume indeed, but on my server there's a relatively fresh
post. i asked on es's support group what's going on

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

issdr wrote:

i tried the piece of code you isolated, it works both at startup and
when checking news again, it just gives warnings of no group found once
it's thru.

Yes, the warnings are a bit annoying. Maybe it's possible to suppress them
by setting gnus-verbose locally? Otherwise, though, this works well:

; Apply scores as soon as news is fetched
(defun apply-scores-to-subscribed-groups ()
(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit))))))

(add-hook 'gnus-started-hook
'apply-scores-to-subscribed-groups)
(add-hook 'gnus-after-getting-new-news-hook
'apply-scores-to-subscribed-groups)

Thank you for the assistance!

Benjamin

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Benjamin Esham wrote:

issdr wrote:

i tried the piece of code you isolated, it works both at startup and
when checking news again, it just gives warnings of no group found once
it's thru.

Yes, the warnings are a bit annoying. Maybe it's possible to suppress them
by setting gnus-verbose locally?

i changed this from 3 to 4 and re-byte-compiled, but could have broken something else, i don't know (no bad signs so far):

~ $ rgrep -n "No articles in the group" emacs/ emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
~ $

Otherwise, though, this works well:

; Apply scores as soon as news is fetched
(defun apply-scores-to-subscribed-groups ()
(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit))))))

(add-hook 'gnus-started-hook
'apply-scores-to-subscribed-groups)
(add-hook 'gnus-after-getting-new-news-hook
'apply-scores-to-subscribed-groups)

Thank you for the assistance!

great! no prob.

from e-s.support:

,----
| The groups still exist on E-S, but they are empty, presumably because
| nobody bothered to send rmgroups when the Stanford news server hosting
| the mail2news gateway for the GNU mailing lists shut down in September
| 2020.
|
| Time to remove the gnu.* groups, I reckon.
`----

even though i see an active gnu.emacs.help, where gnus topics are
discussed.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

issdr wrote:

Benjamin Esham wrote:

issdr wrote:

i tried the piece of code you isolated, it works both at startup and
when checking news again, it just gives warnings of no group found once
it's thru.

Yes, the warnings are a bit annoying. Maybe it's possible to suppress
them by setting gnus-verbose locally?

i changed this from 3 to 4 and re-byte-compiled, but could have broken something else, i don't know (no bad signs so far):

~ $ rgrep -n "No articles in the group" emacs/ emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
~ $

It certainly seems like a safe change to me.

[snip]

from e-s.support:

,----
| The groups still exist on E-S, but they are empty, presumably because
| nobody bothered to send rmgroups when the Stanford news server hosting
| the mail2news gateway for the GNU mailing lists shut down in September
| 2020.
|
| Time to remove the gnu.* groups, I reckon.
`----

even though i see an active gnu.emacs.help, where gnus topics are
discussed.

Ah, that's a shame. One more piece of news infrastructure that has crumbled away...

Benjamin

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 13 May 2023 13:32:13 +0200, issdr wrote:

Sqwertz wrote:

(AND....I forgot the sequence to get into a newserver as a guest
for its overview.fmt and/or active list without posting my USER
and PASS creds <heh>)

i believe that's not inherent in op's case. point is, gnus by default
applies score rules once you enter groups, and there's a logic behind
this; a way to have the actual #'s of unread articles in group (or
topic) buffer is by using the function i pointed out.

I think you quoted the wrong paragraph of mine and from the wrong
post. But it's whatever <shrug>

-sw

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Sqwertz wrote:

On Sat, 13 May 2023 13:32:13 +0200, issdr wrote:

Sqwertz wrote:

(AND....I forgot the sequence to get into a newserver as a guest
for its overview.fmt and/or active list without posting my USER
and PASS creds <heh>)

i believe that's not inherent in op's case. point is, gnus by default
applies score rules once you enter groups, and there's a logic behind
this; a way to have the actual #'s of unread articles in group (or
topic) buffer is by using the function i pointed out.

I think you quoted the wrong paragraph of mine and from the wrong
post. But it's whatever <shrug>

were you trying to access a news server to check overview headers? that
wasn't relevant to op's problem.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Benjamin Esham wrote:

~ $ rgrep -n "No articles in the group" emacs/
emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
~ $

It certainly seems like a safe change to me.

it does break the `e' (or `D e') command in the drafts group, dunno why.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)