Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <352465f6-394a-6f91-712d-85ed15588822@redhat.com>
Date:   Fri, 4 Aug 2023 13:03:36 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.1
Subject: Re: [PATCH v3] perf/arm-dmc620: Fix
 dmc620_pmu_irqs_lock/cpu_hotplug_lock circular lock dependency
Content-Language: en-US
To:     Will Deacon <will@kernel.org>
Cc:     Mark Rutland <mark.rutland@arm.com>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        Robin Murphy <robin.murphy@arm.com>
References: <20230722031729.3913953-1-longman@redhat.com>
 <20230728150614.GF21718@willie-the-truck>
 <62d4b353-0237-9ec6-a63e-8a7a6764aba5@redhat.com>
 <20230804162812.GC30679@willie-the-truck>
 <458ac4d2-bb8a-0359-f198-dd53f4c84bd3@redhat.com>
 <20230804165943.GG30679@willie-the-truck>
From:   Waiman Long <longman@redhat.com>
In-Reply-To: <20230804165943.GG30679@willie-the-truck>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 8/4/23 12:59, Will Deacon wrote:
> On Fri, Aug 04, 2023 at 12:51:47PM -0400, Waiman Long wrote:
>> On 8/4/23 12:28, Will Deacon wrote:
>>>>>>     struct dmc620_pmu {
>>>>>> @@ -423,9 +424,14 @@ static struct dmc620_pmu_irq *__dmc620_pmu_get_irq(int irq_num)
>>>>>>     	struct dmc620_pmu_irq *irq;
>>>>>>     	int ret;
>>>>>> -	list_for_each_entry(irq, &dmc620_pmu_irqs, irqs_node)
>>>>>> -		if (irq->irq_num == irq_num && refcount_inc_not_zero(&irq->refcount))
>>>>>> +	list_for_each_entry(irq, &dmc620_pmu_irqs, irqs_node) {
>>>>>> +		if (irq->irq_num != irq_num)
>>>>>> +			continue;
>>>>>> +		if (!irq->valid)
>>>>>> +			return ERR_PTR(-EAGAIN);	/* Try again later */
>>>>> It looks like this can bubble up to the probe() routine. Does the driver
>>>>> core handle -EAGAIN coming back from a probe routine?
>>>> Right, I should add code to handle this error condition. I think it can be
>>>> handled in dmc620_pmu_get_irq(). The important thing is to release the
>>>> mutex, wait a few ms and try again. What do you think?
>>> I don't really follow, but waiting a few ms and trying again sounds like
>>> a really nasty hack for something which doesn't appear to be constrained
>>> by broken hardware. In other words, we got ourselves into this mess, so
>>> we should be able to resolve it properly.
>>  From my point of view, the proper way to solve the problem is to reverse the
>> locking order. Since you don't to add a EXPORT statement to the core kernel
>> code, we will have to find a way around it by not holding the
>> dmc620_pmu_irqs_lock when cpuhp_state_add_instance_nocalls() is called.
>> Another alternative that I can think of is to add one more mutex that we
>> will hold just for the entirety of  __dmc620_pmu_get_irq() and take
>> dmc620_pmu_irqs_lock only when the linked list is being modified. That will
>> eliminate the need to introduce arbitrary wait as other caller of
>> __dmc620_pmu_get_irq() will wait in the new mutex. Will this work for you?
> Yes. To be honest, I think we've both spent far too much time trying to
> fix this (and I admire your persistence!), so adding a mutex to make it
> "obviously" correct sounds like the right thing to me. We can look at
> optimisations later if anybody cares.

Sorry to be too persistent sometimes:-) Will send out a new version soon.

Cheers,
Longman