Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
In-Reply-To: <20180514165844.GD27984@xo-6d-61-c0.localdomain>
References: <CAFkk2KRTnRpi5thUGTM2+LZMyitggdJtiOBO7E0pTn93fz6-SQ@mail.gmail.com>
 <d4c3b2ab-385d-fe7a-bdb1-2d497d620de5@infradead.org> <CAFkk2KRozscmd4kQSJKwKmY3WpTody7S5XZZObf8SKLF62k6+w@mail.gmail.com>
 <20180514165844.GD27984@xo-6d-61-c0.localdomain>
From:   Ulf Magnusson <ulfalizer@gmail.com>
Date:   Mon, 14 May 2018 20:12:21 +0200
Message-ID: <CAFkk2KQX=j5_VCSRR1fm31x=yC3zse47fnpJaqpLGOLw-v5QkA@mail.gmail.com>
Subject: Re: [ANNOUNCE] Kconfiglib menuconfig implementation
To:     Pavel Machek <pavel@ucw.cz>
Cc:     Randy Dunlap <rdunlap@infradead.org>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, May 14, 2018 at 6:58 PM, Pavel Machek <pavel@ucw.cz> wrote:
> Hi!
>
>> > config CAN_PEAK_PCIEFD
>> >         depends on PCI
>> >         tristate "PEAK-System PCAN-PCIe FD cards"
>> >         ---help---
>> >           This driver adds support for the PEAK-System PCI Express FD
>> >           CAN-FD cards family.
>> >           These 1x or 2x CAN-FD channels cards offer CAN 2.0 a/b as we=
ll as
>> >           CAN-FD access to the CAN bus. Besides the nominal bitrate of=
 up to
>> >           1 Mbit/s, the data bytes of CAN-FD frames can be transmitted=
 with
>> >           up to 12 Mbit/s. A galvanic isolation of the CAN ports prote=
cts the
>> >           electronics of the card and the respective computer against
>> >           disturbances of up to 500 Volts. The PCAN-PCI Express FD can=
 be
>> >           operated with ambient temperatures in a range of -40 to +85 =
=C2=B0C.
>> >
>> >
>> > kconfiglib.KconfigSyntaxError:
>> > Malformed ascii in drivers/net/can/peak_canfd/Kconfig
>> > Context: b't temperatures in a range of -40 to +85 \xc2\xb0C.\n'
>> > Problematic data: b'\xc2'
>> > Reason: ordinal not in range(128)
>> >
>> >
>> > thanks,
>> > --
>> > ~Randy
>>
>> Thanks for trying it out!
>>
>> You're probably running in the C locale, which implies an ASCII
>> encoding. That has caused enough trouble that the Python devs decided
>> to automatically convert it to UTF-8 in Python 3.7:
>> https://www.python.org/dev/peps/pep-0538/. LC_CTYPE=3DC.UTF-8 should fix
>> it.
>>
>> It's a bit silly to have it crash for something like that though. I
>> could force UTF-8 instead of respecting the locale (though it feels
>> neater to respect settings), or tell Python to ignore decoding errors.
>> Should probably do something at least...
>
> Well, Kconfig files are utf-8, as is the rest of kernel.

Yeah, it's the only sane assumption anyway.

>
> So yes, you should respect $LC_CTYPE for display etc, but you should alwa=
ys
> assume Kconfig files are utf-8... and that does not need to be optional.

Already made it optional, with UTF-8 as the default (via
Kconfig.__init__(..., encoding=3D"utf-8")).

I did a similar thing to what PEP 538 does for the menuconfig:
https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py#L2271.
The C locale is converted to a UTF-8 locale for LC_CTYPE if possible.
Other locales are respected. That'll be the default behavior in Python
3.7, and prevents issues with get_wch().

>
>                                                                         P=
avel

Cheers,
Ulf