From: Randy Dunlap <[email protected]>
kernel-doc parsing uses as ASCII codec, so let people know that
kernel-doc comments should be in ASCII characters only.
WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
Signed-off-by: Randy Dunlap <[email protected]>
---
Documentation/doc-guide/kernel-doc.rst | 3 +++
1 file changed, 3 insertions(+)
--- lnx-413-rc7.orig/Documentation/doc-guide/kernel-doc.rst
+++ lnx-413-rc7/Documentation/doc-guide/kernel-doc.rst
@@ -108,6 +108,9 @@ The function and type kernel-doc comment
function or type being described. The overview kernel-doc comments may be freely
placed at the top indentation level.
+.. attention:: kernel-doc comments should be written **only** in ASCII
+ characters since they are processed as ASCII input.
+
Example kernel-doc function comment::
/**
On Mon, 28 Aug 2017 16:10:09 -0700
Randy Dunlap <[email protected]> wrote:
> kernel-doc parsing uses as ASCII codec, so let people know that
> kernel-doc comments should be in ASCII characters only.
>
> WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
So I don't get this error. What kind of system are you running the docs
build on? I would really rather that the docs system could handle modern
text if possible, so it would be better to figure out what's going on
here...
Thanks,
jon
On 08/30/17 14:23, Jonathan Corbet wrote:
> On Mon, 28 Aug 2017 16:10:09 -0700
> Randy Dunlap <[email protected]> wrote:
>
>> kernel-doc parsing uses as ASCII codec, so let people know that
>> kernel-doc comments should be in ASCII characters only.
>>
>> WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
>
> So I don't get this error. What kind of system are you running the docs
> build on? I would really rather that the docs system could handle modern
> text if possible, so it would be better to figure out what's going on
> here...
I'm OK with that. Source files in general don't need to be ASCII (0-127).
I did this patch based on this (private) comment:
> Yes, using ASCII should fix the problem.
what kind of system? HP laptop.
Linux midway.site 4.4.79-18.26-default #1 SMP Thu Aug 10 20:30:05 UTC 2017 (fa5a935) x86_64 x86_64 x86_64 GNU/Linux
> sphinx-build --version
Sphinx (sphinx-build) 1.3.1
--
~Randy
Em Wed, 30 Aug 2017 15:02:59 -0700
Randy Dunlap <[email protected]> escreveu:
> On 08/30/17 14:23, Jonathan Corbet wrote:
> > On Mon, 28 Aug 2017 16:10:09 -0700
> > Randy Dunlap <[email protected]> wrote:
> >
> >> kernel-doc parsing uses as ASCII codec, so let people know that
> >> kernel-doc comments should be in ASCII characters only.
> >>
> >> WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
> >
> > So I don't get this error. What kind of system are you running the docs
> > build on? I would really rather that the docs system could handle modern
> > text if possible, so it would be better to figure out what's going on
> > here...
>
> I'm OK with that. Source files in general don't need to be ASCII (0-127).
>
> I did this patch based on this (private) comment:
>
> > Yes, using ASCII should fix the problem.
>
> what kind of system? HP laptop.
>
> Linux midway.site 4.4.79-18.26-default #1 SMP Thu Aug 10 20:30:05 UTC 2017 (fa5a935) x86_64 x86_64 x86_64 GNU/Linux
>
> > sphinx-build --version
> Sphinx (sphinx-build) 1.3.1
I suspect that the problem is not related to the version, but to
what you might have set on LANG.
Maybe if we add something like:
LANG=C.utf-8
to the Documentation/Makefile or adding:
.. -*- coding: utf-8; mode: rst -*-
as the first line on the *.rst file that include the kernel-doc
directive would solve the issue.
Regards,
Mauro
On Wed, 30 Aug 2017 19:15:53 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> I suspect that the problem is not related to the version, but to
> what you might have set on LANG.
>
> Maybe if we add something like:
> LANG=C.utf-8
>
> to the Documentation/Makefile
That's worth a try; Randy, can you give it a quick go?
> or adding:
>
> .. -*- coding: utf-8; mode: rst -*-
>
> as the first line on the *.rst file that include the kernel-doc
> directive would solve the issue.
I guess I don't see how that would help, instead. Emacs reads that line,
but it's not involved in the problem.
I wish I could reproduce this, then we could see what in that massive
try..except block in kerneldoc.py is throwing the exception. Putting in
an explicit decode call might be enough to make the problem go away.
Thanks,
jon
On 08/30/17 15:31, Jonathan Corbet wrote:
> On Wed, 30 Aug 2017 19:15:53 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
>> I suspect that the problem is not related to the version, but to
>> what you might have set on LANG.
>>
>> Maybe if we add something like:
>> LANG=C.utf-8
>>
>> to the Documentation/Makefile
>
> That's worth a try; Randy, can you give it a quick go?
Yes, that fixes it for me. Thanks.
>> or adding:
>>
>> .. -*- coding: utf-8; mode: rst -*-
>>
>> as the first line on the *.rst file that include the kernel-doc
>> directive would solve the issue.
>
> I guess I don't see how that would help, instead. Emacs reads that line,
> but it's not involved in the problem.
>
> I wish I could reproduce this, then we could see what in that massive
> try..except block in kerneldoc.py is throwing the exception. Putting in
> an explicit decode call might be enough to make the problem go away.
--
~Randy
On 08/30/17 16:01, Randy Dunlap wrote:
> On 08/30/17 15:31, Jonathan Corbet wrote:
>> On Wed, 30 Aug 2017 19:15:53 -0300
>> Mauro Carvalho Chehab <[email protected]> wrote:
>>
>>> I suspect that the problem is not related to the version, but to
>>> what you might have set on LANG.
>>>
>>> Maybe if we add something like:
>>> LANG=C.utf-8
>>>
>>> to the Documentation/Makefile
>>
>> That's worth a try; Randy, can you give it a quick go?
>
> Yes, that fixes it for me. Thanks.
Wait! I forgot to unpatch demux.h. I'll test again now....
>>> or adding:
>>>
>>> .. -*- coding: utf-8; mode: rst -*-
>>>
>>> as the first line on the *.rst file that include the kernel-doc
>>> directive would solve the issue.
>>
>> I guess I don't see how that would help, instead. Emacs reads that line,
>> but it's not involved in the problem.
>>
>> I wish I could reproduce this, then we could see what in that massive
>> try..except block in kerneldoc.py is throwing the exception. Putting in
>> an explicit decode call might be enough to make the problem go away.
>
>
>
--
~Randy
On 08/30/17 16:04, Randy Dunlap wrote:
> On 08/30/17 16:01, Randy Dunlap wrote:
>> On 08/30/17 15:31, Jonathan Corbet wrote:
>>> On Wed, 30 Aug 2017 19:15:53 -0300
>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>
>>>> I suspect that the problem is not related to the version, but to
>>>> what you might have set on LANG.
>>>>
>>>> Maybe if we add something like:
>>>> LANG=C.utf-8
>>>>
>>>> to the Documentation/Makefile
>>>
>>> That's worth a try; Randy, can you give it a quick go?
>>
>> Yes, that fixes it for me. Thanks.
>
> Wait! I forgot to unpatch demux.h. I'll test again now....
OK, still works for me.
>
>>>> or adding:
>>>>
>>>> .. -*- coding: utf-8; mode: rst -*-
>>>>
>>>> as the first line on the *.rst file that include the kernel-doc
>>>> directive would solve the issue.
>>>
>>> I guess I don't see how that would help, instead. Emacs reads that line,
>>> but it's not involved in the problem.
>>>
>>> I wish I could reproduce this, then we could see what in that massive
>>> try..except block in kerneldoc.py is throwing the exception. Putting in
>>> an explicit decode call might be enough to make the problem go away.
>>
>>
>>
>
>
--
~Randy
Em Wed, 30 Aug 2017 15:02:59 -0700
Randy Dunlap <[email protected]> escreveu:
> On 08/30/17 14:23, Jonathan Corbet wrote:
> > On Mon, 28 Aug 2017 16:10:09 -0700
> > Randy Dunlap <[email protected]> wrote:
> >
> >> kernel-doc parsing uses as ASCII codec, so let people know that
> >> kernel-doc comments should be in ASCII characters only.
> >>
> >> WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
> >
> > So I don't get this error. What kind of system are you running the docs
> > build on? I would really rather that the docs system could handle modern
> > text if possible, so it would be better to figure out what's going on
> > here...
>
> I'm OK with that. Source files in general don't need to be ASCII (0-127).
>
> I did this patch based on this (private) comment:
>
> > Yes, using ASCII should fix the problem.
>
> what kind of system? HP laptop.
>
> Linux midway.site 4.4.79-18.26-default #1 SMP Thu Aug 10 20:30:05 UTC 2017 (fa5a935) x86_64 x86_64 x86_64 GNU/Linux
>
> > sphinx-build --version
> Sphinx (sphinx-build) 1.3.1
I tried hard to reproduce the error here... I even added some Chinese
chars on a kernel-doc markup and changed the language on my system
to LANG=en_US.iso885915.
No luck.
As Documentation/conf.py has:
# -*- coding: utf-8 -*-
on its first line, I suspect that the error you're getting is likely
due to the usage of a python version that doesn't recognize this.
It seems that such dialect was introduced on python version 2.3:
https://docs.python.org/2.3/whatsnew/section-encodings.html
Yet, the documentation there seems to require a line before it,
e. g.:
#!/usr/bin/env python
# -*- coding: UTF-8 -*-
I suspect, however, that, if such line is added, on some systems it
may not work, e. g. if both python 2 and 3 are installed, it could
use the python version that doesn't have Sphinx installed.
So, I suspect that the safest way to fix it is with something like the
enclosed patch. Still, it could be useful to know what's happening,
just in case we get other reports.
Randy,
What's your python version?
Thanks,
Mauro
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 85f7856f0092..94eb98031b56 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -55,6 +55,7 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
cmd_sphinx = $(MAKE) BUILDDIR=$(abspath $(BUILDDIR)) $(build)=Documentation/media $2 && \
PYTHONDONTWRITEBYTECODE=1 \
BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(srctree)/$(src)/$5/$(SPHINX_CONF)) \
+ LANG=C.utf8\
$(SPHINXBUILD) \
-b $2 \
-c $(abspath $(srctree)/$(src)) \
On Thu, 31 Aug 2017, Mauro Carvalho Chehab <[email protected]> wrote:
> As Documentation/conf.py has:
>
> # -*- coding: utf-8 -*-
>
> on its first line, I suspect that the error you're getting is likely
> due to the usage of a python version that doesn't recognize this.
AFAIK that has nothing to do with python I/O, and everything to do with
the encoding of that specific python source file.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
Em Thu, 31 Aug 2017 16:26:44 +0300
Jani Nikula <[email protected]> escreveu:
> On Thu, 31 Aug 2017, Mauro Carvalho Chehab <[email protected]> wrote:
> > As Documentation/conf.py has:
> >
> > # -*- coding: utf-8 -*-
> >
> > on its first line, I suspect that the error you're getting is likely
> > due to the usage of a python version that doesn't recognize this.
>
> AFAIK that has nothing to do with python I/O, and everything to do with
> the encoding of that specific python source file.
Jani,
It should have something to do with python version and/or to some
locale info at the system, as neither I or Jon can reproduce it.
Can you reproduce it on your system?
Thanks,
Mauro
On Thu, 31 Aug 2017 10:56:26 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> It should have something to do with python version and/or to some
> locale info at the system, as neither I or Jon can reproduce it.
I can't reproduce it here, but I have certainly seen situations where
Python 2 wants to run with the ascii codec by default.
Note that the exception happens in our Sphinx extension, not in Sphinx
itself. We've had other non-ascii text in our docs, so I think Sphinx is
doing the right thing. The problem is with our own code. If I could
reproduce it, it shouldn't be too hard to track down - take out that
massive "except anything" block and see where it explodes.
Randy, which distribution are you running, and are you using their version
of Sphinx?
Thanks,
jon
On 08/31/17 07:17, Jonathan Corbet wrote:
> On Thu, 31 Aug 2017 10:56:26 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
>> It should have something to do with python version and/or to some
>> locale info at the system, as neither I or Jon can reproduce it.
>
> I can't reproduce it here, but I have certainly seen situations where
> Python 2 wants to run with the ascii codec by default.
>
> Note that the exception happens in our Sphinx extension, not in Sphinx
> itself. We've had other non-ascii text in our docs, so I think Sphinx is
> doing the right thing. The problem is with our own code. If I could
> reproduce it, it shouldn't be too hard to track down - take out that
> massive "except anything" block and see where it explodes.
>
> Randy, which distribution are you running, and are you using their version
> of Sphinx?
opensuse LEAP 42.2
Yes, their sphinx 1.3.1.
--
~Randy
On 08/31/17 02:49, Mauro Carvalho Chehab wrote:
> Em Wed, 30 Aug 2017 15:02:59 -0700
> Randy Dunlap <[email protected]> escreveu:
>
>> On 08/30/17 14:23, Jonathan Corbet wrote:
>>> On Mon, 28 Aug 2017 16:10:09 -0700
>>> Randy Dunlap <[email protected]> wrote:
>>>
>>>> kernel-doc parsing uses as ASCII codec, so let people know that
>>>> kernel-doc comments should be in ASCII characters only.
>>>>
>>>> WARNING: kernel-doc '../scripts/kernel-doc -rst -enable-lineno ../drivers/media/dvb-core/demux.h' processing failed with: 'ascii' codec can't decode byte 0xe2 in position 6368: ordinal not in range(128)
>>>
>>> So I don't get this error. What kind of system are you running the docs
>>> build on? I would really rather that the docs system could handle modern
>>> text if possible, so it would be better to figure out what's going on
>>> here...
>>
>> I'm OK with that. Source files in general don't need to be ASCII (0-127).
>>
>> I did this patch based on this (private) comment:
>>
>>> Yes, using ASCII should fix the problem.
>>
>> what kind of system? HP laptop.
>>
>> Linux midway.site 4.4.79-18.26-default #1 SMP Thu Aug 10 20:30:05 UTC 2017 (fa5a935) x86_64 x86_64 x86_64 GNU/Linux
>>
>>> sphinx-build --version
>> Sphinx (sphinx-build) 1.3.1
>
> I tried hard to reproduce the error here... I even added some Chinese
> chars on a kernel-doc markup and changed the language on my system
> to LANG=en_US.iso885915.
>
> No luck.
>
> As Documentation/conf.py has:
>
> # -*- coding: utf-8 -*-
>
> on its first line, I suspect that the error you're getting is likely
> due to the usage of a python version that doesn't recognize this.
>
> It seems that such dialect was introduced on python version 2.3:
>
> https://docs.python.org/2.3/whatsnew/section-encodings.html
>
> Yet, the documentation there seems to require a line before it,
> e. g.:
>
> #!/usr/bin/env python
> # -*- coding: UTF-8 -*-
>
> I suspect, however, that, if such line is added, on some systems it
> may not work, e. g. if both python 2 and 3 are installed, it could
> use the python version that doesn't have Sphinx installed.
>
> So, I suspect that the safest way to fix it is with something like the
> enclosed patch. Still, it could be useful to know what's happening,
> just in case we get other reports.
>
> Randy,
>
> What's your python version?
> python --version
Python 2.7.13
--
~Randy
On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
> On 08/31/17 07:17, Jonathan Corbet wrote:
>> On Thu, 31 Aug 2017 10:56:26 -0300
>> Mauro Carvalho Chehab <[email protected]> wrote:
>>
>>> It should have something to do with python version and/or to some
>>> locale info at the system, as neither I or Jon can reproduce it.
>>
>> I can't reproduce it here, but I have certainly seen situations where
>> Python 2 wants to run with the ascii codec by default.
>>
>> Note that the exception happens in our Sphinx extension, not in Sphinx
>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>> doing the right thing. The problem is with our own code. If I could
>> reproduce it, it shouldn't be too hard to track down - take out that
>> massive "except anything" block and see where it explodes.
>>
>> Randy, which distribution are you running, and are you using their version
>> of Sphinx?
>
> opensuse LEAP 42.2
> Yes, their sphinx 1.3.1.
What's your LANG setting? I think that's what it boils down to, and
trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
ways.
The odd thing is that I can reproduce the issue using a small python
snippet, but not through Sphinx.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
On 08/31/17 09:05, Jani Nikula wrote:
> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>
>>>> It should have something to do with python version and/or to some
>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>
>>> I can't reproduce it here, but I have certainly seen situations where
>>> Python 2 wants to run with the ascii codec by default.
>>>
>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>> doing the right thing. The problem is with our own code. If I could
>>> reproduce it, it shouldn't be too hard to track down - take out that
>>> massive "except anything" block and see where it explodes.
>>>
>>> Randy, which distribution are you running, and are you using their version
>>> of Sphinx?
>>
>> opensuse LEAP 42.2
>> Yes, their sphinx 1.3.1.
>
> What's your LANG setting? I think that's what it boils down to, and
> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
> ways.
(default)
LANG=C
until I add the patch:
+LANG=C.utf-8
> The odd thing is that I can reproduce the issue using a small python
> snippet, but not through Sphinx.
--
~Randy
On Thu, 31 Aug 2017, Jani Nikula <[email protected]> wrote:
> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>
>>>> It should have something to do with python version and/or to some
>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>
>>> I can't reproduce it here, but I have certainly seen situations where
>>> Python 2 wants to run with the ascii codec by default.
>>>
>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>> doing the right thing. The problem is with our own code. If I could
>>> reproduce it, it shouldn't be too hard to track down - take out that
>>> massive "except anything" block and see where it explodes.
>>>
>>> Randy, which distribution are you running, and are you using their version
>>> of Sphinx?
>>
>> opensuse LEAP 42.2
>> Yes, their sphinx 1.3.1.
>
> What's your LANG setting? I think that's what it boils down to, and
> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
> ways.
>
> The odd thing is that I can reproduce the issue using a small python
> snippet, but not through Sphinx.
Your original error message suggests your Sphinx actually uses python
3. Can you check that? The clue is that it's the *decode* that fails.
Does the below patch help? It avoids the implicit ascii decoding due to
universal_newlines=True and your LANG setting, and does explicit utf-8
decoding instead.
Fingers crossed.
BR,
Jani.
diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py
index d15e07f36881..39aa9e8697cc 100644
--- a/Documentation/sphinx/kerneldoc.py
+++ b/Documentation/sphinx/kerneldoc.py
@@ -27,6 +27,7 @@
# Please make sure this works on both python2 and python3.
#
+import codecs
import os
import subprocess
import sys
@@ -88,13 +89,10 @@ class KernelDocDirective(Directive):
try:
env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
- p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
+ p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
- # python2 needs conversion to unicode.
- # python3 with universal_newlines=True returns strings.
- if sys.version_info.major < 3:
- out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
+ out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
if p.returncode != 0:
sys.stderr.write(err)
--
Jani Nikula, Intel Open Source Technology Center
On 08/31/17 09:36, Jani Nikula wrote:
> On Thu, 31 Aug 2017, Jani Nikula <[email protected]> wrote:
>> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>>
>>>>> It should have something to do with python version and/or to some
>>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>>
>>>> I can't reproduce it here, but I have certainly seen situations where
>>>> Python 2 wants to run with the ascii codec by default.
>>>>
>>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>>> doing the right thing. The problem is with our own code. If I could
>>>> reproduce it, it shouldn't be too hard to track down - take out that
>>>> massive "except anything" block and see where it explodes.
>>>>
>>>> Randy, which distribution are you running, and are you using their version
>>>> of Sphinx?
>>>
>>> opensuse LEAP 42.2
>>> Yes, their sphinx 1.3.1.
>>
>> What's your LANG setting? I think that's what it boils down to, and
>> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
>> ways.
>>
>> The odd thing is that I can reproduce the issue using a small python
>> snippet, but not through Sphinx.
>
> Your original error message suggests your Sphinx actually uses python
> 3. Can you check that? The clue is that it's the *decode* that fails.
Where do you see that clue?
My /usr/bin/python is linked to python2.7:
> ll /usr/bin/python
lrwxrwxrwx 1 root root 9 Jun 10 19:59 /usr/bin/python -> python2.7*
> Does the below patch help? It avoids the implicit ascii decoding due to
> universal_newlines=True and your LANG setting, and does explicit utf-8
> decoding instead.
>
> Fingers crossed.
testing now.
--
~Randy
On 08/31/17 10:34, Randy Dunlap wrote:
> On 08/31/17 09:36, Jani Nikula wrote:
>> On Thu, 31 Aug 2017, Jani Nikula <[email protected]> wrote:
>>> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>>>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>>>
>>>>>> It should have something to do with python version and/or to some
>>>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>>>
>>>>> I can't reproduce it here, but I have certainly seen situations where
>>>>> Python 2 wants to run with the ascii codec by default.
>>>>>
>>>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>>>> doing the right thing. The problem is with our own code. If I could
>>>>> reproduce it, it shouldn't be too hard to track down - take out that
>>>>> massive "except anything" block and see where it explodes.
>>>>>
>>>>> Randy, which distribution are you running, and are you using their version
>>>>> of Sphinx?
>>>>
>>>> opensuse LEAP 42.2
>>>> Yes, their sphinx 1.3.1.
>>>
>>> What's your LANG setting? I think that's what it boils down to, and
>>> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
>>> ways.
>>>
>>> The odd thing is that I can reproduce the issue using a small python
>>> snippet, but not through Sphinx.
>>
>> Your original error message suggests your Sphinx actually uses python
>> 3. Can you check that? The clue is that it's the *decode* that fails.
>
> Where do you see that clue?
> My /usr/bin/python is linked to python2.7:
>
>> ll /usr/bin/python
> lrwxrwxrwx 1 root root 9 Jun 10 19:59 /usr/bin/python -> python2.7*
>
>> Does the below patch help? It avoids the implicit ascii decoding due to
>> universal_newlines=True and your LANG setting, and does explicit utf-8
>> decoding instead.
>>
>> Fingers crossed.
>
> testing now.
Yes, that works for me. Thanks.
--
~Randy
On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
> On 08/31/17 09:36, Jani Nikula wrote:
>> On Thu, 31 Aug 2017, Jani Nikula <[email protected]> wrote:
>>> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>>>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>>>
>>>>>> It should have something to do with python version and/or to some
>>>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>>>
>>>>> I can't reproduce it here, but I have certainly seen situations where
>>>>> Python 2 wants to run with the ascii codec by default.
>>>>>
>>>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>>>> doing the right thing. The problem is with our own code. If I could
>>>>> reproduce it, it shouldn't be too hard to track down - take out that
>>>>> massive "except anything" block and see where it explodes.
>>>>>
>>>>> Randy, which distribution are you running, and are you using their version
>>>>> of Sphinx?
>>>>
>>>> opensuse LEAP 42.2
>>>> Yes, their sphinx 1.3.1.
>>>
>>> What's your LANG setting? I think that's what it boils down to, and
>>> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
>>> ways.
>>>
>>> The odd thing is that I can reproduce the issue using a small python
>>> snippet, but not through Sphinx.
>>
>> Your original error message suggests your Sphinx actually uses python
>> 3. Can you check that? The clue is that it's the *decode* that fails.
>
> Where do you see that clue?
The message, "'ascii' codec can't decode byte 0xe2 in position 6368:
ordinal not in range(128)". In my testing I could only get that *decode*
error message using python 3.
> My /usr/bin/python is linked to python2.7:
>
>> ll /usr/bin/python
> lrwxrwxrwx 1 root root 9 Jun 10 19:59 /usr/bin/python -> python2.7*
Sure, but how about 'head $(which sphinx-build)'?
I could be completely mistaken too. ;)
>> Does the below patch help? It avoids the implicit ascii decoding due to
>> universal_newlines=True and your LANG setting, and does explicit utf-8
>> decoding instead.
>>
>> Fingers crossed.
>
> testing now.
Thanks.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
On 08/31/17 10:55, Jani Nikula wrote:
> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>> On 08/31/17 09:36, Jani Nikula wrote:
>>> On Thu, 31 Aug 2017, Jani Nikula <[email protected]> wrote:
>>>> On Thu, 31 Aug 2017, Randy Dunlap <[email protected]> wrote:
>>>>> On 08/31/17 07:17, Jonathan Corbet wrote:
>>>>>> On Thu, 31 Aug 2017 10:56:26 -0300
>>>>>> Mauro Carvalho Chehab <[email protected]> wrote:
>>>>>>
>>>>>>> It should have something to do with python version and/or to some
>>>>>>> locale info at the system, as neither I or Jon can reproduce it.
>>>>>>
>>>>>> I can't reproduce it here, but I have certainly seen situations where
>>>>>> Python 2 wants to run with the ascii codec by default.
>>>>>>
>>>>>> Note that the exception happens in our Sphinx extension, not in Sphinx
>>>>>> itself. We've had other non-ascii text in our docs, so I think Sphinx is
>>>>>> doing the right thing. The problem is with our own code. If I could
>>>>>> reproduce it, it shouldn't be too hard to track down - take out that
>>>>>> massive "except anything" block and see where it explodes.
>>>>>>
>>>>>> Randy, which distribution are you running, and are you using their version
>>>>>> of Sphinx?
>>>>>
>>>>> opensuse LEAP 42.2
>>>>> Yes, their sphinx 1.3.1.
>>>>
>>>> What's your LANG setting? I think that's what it boils down to, and
>>>> trying to work around non-UTF-8 LANG in both python 2 and 3 compatible
>>>> ways.
>>>>
>>>> The odd thing is that I can reproduce the issue using a small python
>>>> snippet, but not through Sphinx.
>>>
>>> Your original error message suggests your Sphinx actually uses python
>>> 3. Can you check that? The clue is that it's the *decode* that fails.
>>
>> Where do you see that clue?
>
> The message, "'ascii' codec can't decode byte 0xe2 in position 6368:
> ordinal not in range(128)". In my testing I could only get that *decode*
> error message using python 3.
>
>> My /usr/bin/python is linked to python2.7:
>>
>>> ll /usr/bin/python
>> lrwxrwxrwx 1 root root 9 Jun 10 19:59 /usr/bin/python -> python2.7*
>
> Sure, but how about 'head $(which sphinx-build)'?
You are correct:
#!/usr/bin/python3
> I could be completely mistaken too. ;)
--
~Randy