2022-05-25 19:58:42

by David Woodhouse

[permalink] [raw]
Subject: Re: Getting rid of infradead.org - corrupted subjects

On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> CC dwmw2

Thanks, Geert.

> On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> <[email protected]> wrote:
>
> > The address list is semi-random as I don't know whom to approach.
> > Problem: infradead.org corrupts email subjects by changing:
> > s/,/, /

I found an old thread on mailman-users which appears to be claiming
that it's just standard folding as described in RFC2822 §2.2.3.

https://mail.python.org/pipermail/mailman-users/2007-May/057119.html

2.2.3. Long Header Fields

Each header field is logically a single line of characters comprising
the field name, the colon, and the field body. For convenience
however, and to deal with the 998/78 character limitations per line,
the field body portion of a header field can be split into a multiple
line representation; this is called "folding". The general rule is
that wherever this standard allows for folding white space (not
simply WSP characters), a CRLF may be inserted before any WSP. For
example, the header field:

Subject: This is a test

can be represented as:

Subject: This
is a test

But this is folding at a *comma*, not at whitespace. The original
subject line was (in a single line):

Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties

... and Mailman 'folded' it in the wrong place to:

Subject: [PATCH v2] dt-bindings: mtd: jedec,
spi-nor: remove unneeded properties

That *isn't* proper folding because it didn't happen at a whitespace.

I do need to upgrade to Mailman 3 at some point; I've been
procrastinating on the basis that it "ain't broke". For the time being
I've just disabled folding in MM2 with a trivial hack:

--- Message.py.orig 2022-05-25 02:51:05.917145192 -0700
+++ Message.py 2022-05-25 02:50:44.137142069 -0700
@@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
Headers will by default _not_ be folded in attachments.
"""
def __init__(self, outfp, mangle_from_=True,
- maxheaderlen=78, children_maxheaderlen=0):
+ maxheaderlen=0, children_maxheaderlen=0):
email.Generator.Generator.__init__(self, outfp,
mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
self.__children_maxheaderlen = children_maxheaderlen

Looking at http://lists.infradead.org/pipermail/testlist/2022-May/subject.html
it seems to have had the desired effect. Please let me know if not (or
if there are other side-effects).

Thanks for the report.


Attachments:
smime.p7s (5.83 kB)

2022-05-26 00:48:13

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: Getting rid of infradead.org - corrupted subjects

On Wed, May 25, 2022 at 10:55:18AM +0100, David Woodhouse wrote:
> On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> > CC dwmw2
>
> Thanks, Geert.
>
> > On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> > <[email protected]> wrote:
> >
> > > The address list is semi-random as I don't know whom to approach.
> > > Problem: infradead.org corrupts email subjects by changing:
> > > s/,/, /
>
> I found an old thread on mailman-users which appears to be claiming
> that it's just standard folding as described in RFC2822 ?2.2.3.
>
> https://mail.python.org/pipermail/mailman-users/2007-May/057119.html
>
> 2.2.3. Long Header Fields
>
> Each header field is logically a single line of characters comprising
> the field name, the colon, and the field body. For convenience
> however, and to deal with the 998/78 character limitations per line,
> the field body portion of a header field can be split into a multiple
> line representation; this is called "folding". The general rule is
> that wherever this standard allows for folding white space (not
> simply WSP characters), a CRLF may be inserted before any WSP. For
> example, the header field:
>
> Subject: This is a test
>
> can be represented as:
>
> Subject: This
> is a test
>
> But this is folding at a *comma*, not at whitespace. The original
> subject line was (in a single line):
>
> Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties
>
> ... and Mailman 'folded' it in the wrong place to:
>
> Subject: [PATCH v2] dt-bindings: mtd: jedec,
> spi-nor: remove unneeded properties
>
> That *isn't* proper folding because it didn't happen at a whitespace.
>
> I do need to upgrade to Mailman 3 at some point; I've been
> procrastinating on the basis that it "ain't broke". For the time being
> I've just disabled folding in MM2 with a trivial hack:
>
> --- Message.py.orig 2022-05-25 02:51:05.917145192 -0700
> +++ Message.py 2022-05-25 02:50:44.137142069 -0700
> @@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
> Headers will by default _not_ be folded in attachments.
> """
> def __init__(self, outfp, mangle_from_=True,
> - maxheaderlen=78, children_maxheaderlen=0):
> + maxheaderlen=0, children_maxheaderlen=0):
> email.Generator.Generator.__init__(self, outfp,
> mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
> self.__children_maxheaderlen = children_maxheaderlen

For the record: Some time ago I debugged that myself, too and found an
equivalent change. So this looks good to me.

Best regards and thanks for caring,
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | https://www.pengutronix.de/ |


Attachments:
(No filename) (2.88 kB)
signature.asc (499.00 B)
Download all attachments

2022-06-01 19:46:48

by Uwe Kleine-König

[permalink] [raw]
Subject: Re: Getting rid of infradead.org - corrupted subjects

Hello,

On Wed, May 25, 2022 at 09:23:32PM +0200, Uwe Kleine-K?nig wrote:
> On Wed, May 25, 2022 at 10:55:18AM +0100, David Woodhouse wrote:
> > On Wed, 2022-05-25 at 10:34 +0200, Geert Uytterhoeven wrote:
> > > CC dwmw2
> >
> > Thanks, Geert.
> >
> > > On Wed, May 25, 2022 at 10:07 AM Krzysztof Kozlowski
> > > <[email protected]> wrote:
> > >
> > > > The address list is semi-random as I don't know whom to approach.
> > > > Problem: infradead.org corrupts email subjects by changing:
> > > > s/,/, /
> >
> > I found an old thread on mailman-users which appears to be claiming
> > that it's just standard folding as described in RFC2822 ?2.2.3.
> >
> > https://mail.python.org/pipermail/mailman-users/2007-May/057119.html
> >
> > 2.2.3. Long Header Fields
> >
> > Each header field is logically a single line of characters comprising
> > the field name, the colon, and the field body. For convenience
> > however, and to deal with the 998/78 character limitations per line,
> > the field body portion of a header field can be split into a multiple
> > line representation; this is called "folding". The general rule is
> > that wherever this standard allows for folding white space (not
> > simply WSP characters), a CRLF may be inserted before any WSP. For
> > example, the header field:
> >
> > Subject: This is a test
> >
> > can be represented as:
> >
> > Subject: This
> > is a test
> >
> > But this is folding at a *comma*, not at whitespace. The original
> > subject line was (in a single line):
> >
> > Subject: [PATCH v2] dt-bindings: mtd: jedec,spi-nor: remove unneeded properties
> >
> > ... and Mailman 'folded' it in the wrong place to:
> >
> > Subject: [PATCH v2] dt-bindings: mtd: jedec,
> > spi-nor: remove unneeded properties
> >
> > That *isn't* proper folding because it didn't happen at a whitespace.
> >
> > I do need to upgrade to Mailman 3 at some point; I've been
> > procrastinating on the basis that it "ain't broke". For the time being
> > I've just disabled folding in MM2 with a trivial hack:
> >
> > --- Message.py.orig 2022-05-25 02:51:05.917145192 -0700
> > +++ Message.py 2022-05-25 02:50:44.137142069 -0700
> > @@ -49,7 +49,7 @@ class Generator(email.Generator.Generato
> > Headers will by default _not_ be folded in attachments.
> > """
> > def __init__(self, outfp, mangle_from_=True,
> > - maxheaderlen=78, children_maxheaderlen=0):
> > + maxheaderlen=0, children_maxheaderlen=0):
> > email.Generator.Generator.__init__(self, outfp,
> > mangle_from_=mangle_from_, maxheaderlen=maxheaderlen)
> > self.__children_maxheaderlen = children_maxheaderlen
>
> For the record: Some time ago I debugged that myself, too and found an
> equivalent change. So this looks good to me.

small supplement: I found the change I did back then:

--- /usr/lib/python2.7/email/header.py 2018-09-26 20:42:22.000000000 +0200
+++ /usr/lib/python2.7/email/header.py 2018-09-26 20:42:22.000000000 +0200
@@ -371,7 +371,7 @@
joiner = NL + self._continuation_ws
return joiner.join(chunks)

- def encode(self, splitchars=';, '):
+ def encode(self, splitchars=' '):
"""Encode a message header into an RFC-compliant format.

There are many issues involved in converting a given string for use in

Best regards
Uwe

--
Pengutronix e.K. | Uwe Kleine-K?nig |
Industrial Linux Solutions | https://www.pengutronix.de/ |


Attachments:
(No filename) (3.61 kB)
signature.asc (499.00 B)
Download all attachments