Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a037a08c-44c4-24e8-1cba-7e4e8b21ffaa@collabora.com>
Date:   Mon, 12 Jun 2023 15:40:18 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.9.0
Subject: Re: [PATCH v4] Makefile.compiler: replace cc-ifversion with
 compiler-specific macros
Content-Language: en-US
To:     Masahiro Yamada <masahiroy@kernel.org>
Cc:     Greg KH <gregkh@linuxfoundation.org>,
        Maksim Panchenko <maks@meta.com>,
        =?UTF-8?Q?Ricardo_Ca=c3=b1uelo?= <ricardo.canuelo@collabora.com>,
        Michal Marek <michal.lkml@markovi.net>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        clang-built-linux <llvm@lists.linux.dev>,
        Bill Wendling <morbo@google.com>,
        Nathan Chancellor <nathan@kernel.org>,
        regressions@lists.linux.dev,
        "gustavo.padovan@collabora.com" <gustavo.padovan@collabora.com>,
        Guillaume Charles Tucker <guillaume.tucker@collabora.com>,
        denys.f@collabora.com, Nick Desaulniers <ndesaulniers@google.com>,
        kernelci@lists.linux.dev,
        Collabora Kernel ML <kernel@collabora.com>
References: <CAKwvOd=4hBcU4fAkddU0b-GOZc9FzTZoj3PFW6ZZrX0jS8x+bg@mail.gmail.com>
 <17c91d37-7d9c-0df4-2438-2b30ca0b5777@collabora.com>
 <CAKwvOdk4QO8x_bs64fFRCsMu__AjhXd4Ew2KfgzQOb9Q3FMqSA@mail.gmail.com>
 <b5d0cf82-0e42-f6a1-c9f5-c145fdc4c622@collabora.com>
 <CAKwvOdkFxu9hYSL_RCXadpR0dQd1+dZmAUVXdfFiLUfxg4D_Xw@mail.gmail.com>
 <878rdlk9bi.fsf@rcn-XPS-13-9305.i-did-not-set--mail-host-address--so-tickle-me>
 <CAKwvOd=8LVU+iANkFx18Wm1jg7gYiAXovAmo9t5ZZaVdMULn-Q@mail.gmail.com>
 <875y8ok9b5.fsf@rcn-XPS-13-9305.i-did-not-set--mail-host-address--so-tickle-me>
 <CAKwvOdmJJibt6sHSp91v2s7BxUWBC6xG7F7+3C6gUxNMzZ2xRA@mail.gmail.com>
 <87353ok78h.fsf@rcn-XPS-13-9305.i-did-not-set--mail-host-address--so-tickle-me>
 <2023052247-bobtail-factsheet-d104@gregkh>
 <CAKwvOd=2zAV_mizvzLFdyHE_4OzBY5OVu6KLWuQPOMZK37vsmQ@mail.gmail.com>
 <cff33e12-3d80-7e62-1993-55411ccabc01@collabora.com>
 <CAKwvOd=F29-UkNO7FtUWpVV=POOZLb6QgD=mhLMWtRfkRSSi2A@mail.gmail.com>
From:   Shreeya Patel <shreeya.patel@collabora.com>
In-Reply-To: <CAKwvOd=F29-UkNO7FtUWpVV=POOZLb6QgD=mhLMWtRfkRSSi2A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Masahiro,


On 24/05/23 02:57, Nick Desaulniers wrote:
> On Tue, May 23, 2023 at 3:27 AM Shreeya Patel
> <shreeya.patel@collabora.com> wrote:
>> Hi Nick and Masahiro,
>>
>> On 23/05/23 01:22, Nick Desaulniers wrote:
>>> On Mon, May 22, 2023 at 9:52 AM Greg KH <gregkh@linuxfoundation.org> wrote:
>>>> On Mon, May 22, 2023 at 12:09:34PM +0200, Ricardo Cañuelo wrote:
>>>>> On vie, may 19 2023 at 08:57:24, Nick Desaulniers <ndesaulniers@google.com> wrote:
>>>>>> It could be; if the link order was changed, it's possible that this
>>>>>> target may be hitting something along the lines of:
>>>>>> https://isocpp.org/wiki/faq/ctors#static-init-order i.e. the "static
>>>>>> initialization order fiasco"
>>>>>>
>>>>>> I'm struggling to think of how this appears in C codebases, but I
>>>>>> swear years ago I had a discussion with GKH (maybe?) about this. I
>>>>>> think I was playing with converting Kbuild to use Ninja rather than
>>>>>> Make; the resulting kernel image wouldn't boot because I had modified
>>>>>> the order the object files were linked in.  If you were to randomly
>>>>>> shuffle the object files in the kernel, I recall some hazard that may
>>>>>> prevent boot.
>>>>> I thought that was specifically a C++ problem? But then again, the
>>>>> kernel docs explicitly say that the ordering of obj-y goals in kbuild is
>>>>> significant in some instances [1]:
>>>> Yes, it matters, you can not change it.  If you do, systems will break.
>>>> It is the only way we have of properly ordering our init calls within
>>>> the same "level".
>>> Ah, right it was the initcall ordering. Thanks for the reminder.
>>>
>>> (There's a joke in there similar to the use of regexes to solve a
>>> problem resulting in two new problems; initcalls have levels for
>>> ordering, but we still have (unexpressed) dependencies between calls
>>> of the same level; brittle!).
>>>
>>> +Maksim, since that might be relevant info for the BOLT+Kernel work.
>>>
>>> Ricardo,
>>> https://elinux.org/images/e/e8/2020_ELCE_initcalls_myjosserand.pdf
>>> mentions that there's a kernel command line param `initcall_debug`.
>>> Perhaps that can be used to see if
>>> 5750121ae7382ebac8d47ce6d68012d6cd1d7926 somehow changed initcall
>>> ordering, resulting in a config that cannot boot?
>>
>> Here are the links to Lava jobs ran with initcall_debug added to the
>> kernel command line.
>>
>> 1. Where regression happens (5750121ae7382ebac8d47ce6d68012d6cd1d7926)
>> https://lava.collabora.dev/scheduler/job/10417706
>> <https://lava.collabora.dev/scheduler/job/10417706>
>>
>> 2. With a revert of the commit 5750121ae7382ebac8d47ce6d68012d6cd1d7926
>> https://lava.collabora.dev/scheduler/job/10418012
>> <https://lava.collabora.dev/scheduler/job/10418012>
> Thanks!
>
> Yeah, I can see a diff in the initcall ordering as a result of
> commit 5750121ae738 ("kbuild: list sub-directories in ./Kbuild")
>
> https://gist.github.com/nickdesaulniers/c09db256e42ad06b90842a4bb85cc0f4
>
> Not just different orderings, but some initcalls seem unique to the
> before vs. after, which is troubling. (example init_events and
> init_fs_sysctls respectively)
>
> That isn't conclusive evidence that changes to initcall ordering are
> to blame, but I suspect confirming that precisely to be very very time
> consuming.
>
> Masahiro, what are your thoughts on reverting 5750121ae738? There are
> conflicts in Kbuild and Makefile when reverting 5750121ae738 on
> mainline.

I'm not sure if you followed the conversation but we are still seeing 
this regression with the latest kernel builds and would like to know if 
you plan to revert 5750121ae738?


Thanks,
Shreeya Patel

>>
>> Thanks,
>> Shreeya Patel
>>
>