Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <6e81b372-d010-fb75-cdfe-b76c6b197f07@suse.cz>
Date:   Mon, 9 Jan 2023 15:14:46 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [PATCH] docs: Fix the docs build with Sphinx 6.0
Content-Language: en-US
To:     Mauro Carvalho Chehab <mchehab@kernel.org>,
        Akira Yokosawa <akiyks@gmail.com>
Cc:     Jonathan Corbet <corbet@lwn.net>, linux-doc@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <87wn629ggg.fsf@meer.lwn.net>
 <cb916eae-fdf6-504f-8f38-7928e0fa8344@gmail.com>
 <20230108150135.060b0c7a@coco.lan>
From:   =?UTF-8?Q?Martin_Li=c5=a1ka?= <mliska@suse.cz>
In-Reply-To: <20230108150135.060b0c7a@coco.lan>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 1/8/23 15:01, Mauro Carvalho Chehab wrote:
> Em Sat, 7 Jan 2023 14:17:24 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 
>> On Wed, 04 Jan 2023 13:45:35 -0700, Jonathan Corbet wrote:
>>> Sphinx 6.0 removed the execfile_() function, which we use as part of the
>>> configuration process.  They *did* warn us...  Just open-code the
>>> functionality as is done in Sphinx itself.
>>>
>>> Tested (using SPHINX_CONF, since this code is only executed with an
>>> alternative config file) on various Sphinx versions from 2.5 through 6.0.
>>>
>>> Reported-by: Martin Liška <mliska@suse.cz>
>>> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
>>
>> I have tested full builds of documentation with this change
>> with Sphinx versions 1.7.9, 2.4.5, 3.4.3, 4.5.0, 5.3.0, and 6.0.0.
>>
>> Tested-by: Akira Yokosawa <akiyks@gmail.com>
>>
>> That said, Sphinx 6.0.0 needs much more time and memory than earlier
>> versions.
>>
>> FYI, I needed to limit parallel slot to 2 (make -j2) on a 16GB machine.
>> If you are lucky, -j3 and -j4 might succeed. -j5 or more ended up in
>> OOM situations for me:
>>
>> Comparison of elapsed time and maxresident with -j2:
>>
>>    ============== ============ ===========
>>    Sphinx version elapsed time maxresident
>>    ============== ============ ===========
>>    5.3.0          10:16.81      937660
>>    6.0.0          17:29.07     5292392
>>    ============== ============ ===========

Hi.

I can confirm the regression, I bisected Sphinx revision that caused that
and filled an upstream issues:
https://github.com/sphinx-doc/sphinx/issues/11116

Cheers,
Martin

> 
>  From the changelogs:
> 	https://www.sphinx-doc.org/en/master/changes.html
> 
> It seems that 6.1 came with some performance optimizations, in particular:
> 
>      Cache doctrees in the build environment during the writing phase.
> 
>      Make all writing phase tasks support parallel execution.
> 
>      Cache doctrees between the reading and writing phases.
> 
> It would be nice if you could also test and check elapsed time
> there too, as I suspect that 6.0 will have a very short usage, as
> 6.1 was released just a few days after it.
> 
> Regards,
> Mauro.
> 
> 
> 
>>
>>          Thanks, Akira
>>
>>> ---
>>>   Documentation/sphinx/load_config.py | 6 ++++--
>>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py
>>> index eeb394b39e2c..8b416bfd75ac 100644
>>> --- a/Documentation/sphinx/load_config.py
>>> +++ b/Documentation/sphinx/load_config.py
>>> @@ -3,7 +3,7 @@
>>>   
>>>   import os
>>>   import sys
>>> -from sphinx.util.pycompat import execfile_
>>> +from sphinx.util.osutil import fs_encoding
>>>   
>>>   # ------------------------------------------------------------------------------
>>>   def loadConfig(namespace):
>>> @@ -48,7 +48,9 @@ def loadConfig(namespace):
>>>               sys.stdout.write("load additional sphinx-config: %s\n" % config_file)
>>>               config = namespace.copy()
>>>               config['__file__'] = config_file
>>> -            execfile_(config_file, config)
>>> +            with open(config_file, 'rb') as f:
>>> +                code = compile(f.read(), fs_encoding, 'exec')
>>> +                exec(code, config)
>>>               del config['__file__']
>>>               namespace.update(config)
>>>           else:
>>> -- 
>>> 2.38.1
> 
> 
> 
> Thanks,
> Mauro