Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Johan Hovold <johan@kernel.org>
Cc:     Johan Hovold <johan+linaro@kernel.org>,
        Marc Zyngier <maz@kernel.org>, x86@kernel.org,
        platform-driver-x86@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3 19/19] irqdomain: Switch to per-domain locking
In-Reply-To: <Y5c6z3t+sV/kIMjF@hovoldconsulting.com>
References: <20221209140150.1453-1-johan+linaro@kernel.org>
 <20221209140150.1453-20-johan+linaro@kernel.org> <87mt7sbtf9.ffs@tglx>
 <Y5c6z3t+sV/kIMjF@hovoldconsulting.com>
Date:   Mon, 12 Dec 2022 17:18:11 +0100
Message-ID: <878rjcbnp8.ffs@tglx>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On Mon, Dec 12 2022 at 15:29, Johan Hovold wrote:
> On Mon, Dec 12, 2022 at 03:14:34PM +0100, Thomas Gleixner wrote:
>> On Fri, Dec 09 2022 at 15:01, Johan Hovold wrote:
>> > The IRQ domain structures are currently protected by the global
>> > irq_domain_mutex. Switch to using more fine-grained per-domain locking,
>> > which may potentially speed up parallel probing somewhat.
>> >
>> > Note that the domain lock of the root domain (innermost domain) must be
>> > used for hierarchical domains. For non-hierarchical domain (as for root
>> > domains), the new root pointer is set to the domain itself so that
>> > domain->root->mutex can be used in shared code paths.
>> >
>> > Also note that hierarchical domains should be constructed using
>> > irq_domain_create_hierarchy() (or irq_domain_add_hierarchy()) to avoid
>> > poking at irqdomain internals.
>> 
>> While I agree in principle, this change really makes me nervous.
>> 
>> Any fail in setting up domain->root correctly, e.g. by not using
>> irq_domain_create_hierarchy(), cannot be detected and creates nasty to
>> debug race conditions.
>> 
>> So we need some debug mechanism which allows to validate that
>> domain->root is set up correctly when domain->parent != NULL.
>
> Lockdep will catch that due to the
>
> 	lockdep_assert_held(&domain->root->mutex);
>
> I added to irq_domain_set_mapping() and which is is called for each
> (inner) domain in a hierarchy when allocating an IRQ.

Hmm. Indeed that should do the trick.

Some comments would be appreciated as the rules around domain->root are
far from obvious.

Thanks,

        tglx