Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH] usb: dwc3: Prevent indefinite sleep in _dwc3_set_mode
 during suspend/resume
To:     Baolin Wang <baolin.wang@linaro.org>
CC:     Felipe Balbi <balbi@kernel.org>, USB <linux-usb@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
References: <1519730526-22274-1-git-send-email-rogerq@ti.com>
 <87sh9l5z4l.fsf@linux.intel.com>
 <94cd6377-1327-2309-8d69-6ab0de2bdfd4@ti.com>
 <87po4i3o1v.fsf@linux.intel.com>
 <a907787a-dea2-5f15-3d25-693680a42b71@ti.com>
 <CAMz4kuKOnAPckX1=nqVPyF7O5u-0YrekTfMddYwe0WC2-0AFDw@mail.gmail.com>
From:   Roger Quadros <rogerq@ti.com>
Message-ID: <6e3a58fb-f1d8-610f-d39c-7d1259833911@ti.com>
Date:   Mon, 5 Mar 2018 13:03:29 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <CAMz4kuKOnAPckX1=nqVPyF7O5u-0YrekTfMddYwe0WC2-0AFDw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 05/03/18 12:41, Baolin Wang wrote:
> Hi Roger,
> 
> On 5 March 2018 at 17:45, Roger Quadros <rogerq@ti.com> wrote:
>> Felipe,
>>
>> On 05/03/18 10:49, Felipe Balbi wrote:
>>>
>>> Hi,
>>>
>>> Roger Quadros <rogerq@ti.com> writes:
>>>>> Roger Quadros <rogerq@ti.com> writes:
>>>>>> In the following test we get stuck by sleeping forever in _dwc3_set_mode()
>>>>>> after which dual-role switching doesn't work.
>>>>>>
>>>>>> On dra7-evm's dual-role port,
>>>>>> - Load g_zero gadget driver and enumerate to host
>>>>>> - suspend to mem
>>>>>> - disconnect USB cable to host and connect otg cable with Pen drive in it.
>>>>>> - resume system
>>>>>> - we sleep indefinitely in _dwc3_set_mode due to.
>>>>>>   dwc3_gadget_exit()->usb_del_gadget_udc()->udc_stop()->
>>>>>>    dwc3_gadget_stop()->wait_event_lock_irq()
>>>>>>
>>>>>> Let's clear the DWC3_EP_END_TRANSFER_PENDING flag on all endpoints
>>>>>> so we don't wait in dwc3_gadget_stop().
>>>>>>
>>>>>> Signed-off-by: Roger Quadros <rogerq@ti.com>
>>>>>> ---
>>>>>>  drivers/usb/dwc3/gadget.c | 14 ++++++++++++++
>>>>>>  1 file changed, 14 insertions(+)
>>>>>>
>>>>>> diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c
>>>>>> index 2bda4eb..0a360da 100644
>>>>>> --- a/drivers/usb/dwc3/gadget.c
>>>>>> +++ b/drivers/usb/dwc3/gadget.c
>>>>>> @@ -3273,6 +3273,20 @@ int dwc3_gadget_init(struct dwc3 *dwc)
>>>>>>
>>>>>>  void dwc3_gadget_exit(struct dwc3 *dwc)
>>>>>>  {
>>>>>> +  int epnum;
>>>>>> +  unsigned long flags;
>>>>>> +
>>>>>> +  spin_lock_irqsave(&dwc->lock, flags);
>>>>>> +  for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
>>>>>> +          struct dwc3_ep  *dep = dwc->eps[epnum];
>>>>>> +
>>>>>> +          if (!dep)
>>>>>> +                  continue;
>>>>>> +
>>>>>> +          dep->flags &= ~DWC3_EP_END_TRANSFER_PENDING;
>>>>>> +  }
>>>>>> +  spin_unlock_irqrestore(&dwc->lock, flags);
>>>>>> +
>>>>>>    usb_del_gadget_udc(&dwc->gadget);
>>>>>>    dwc3_gadget_free_endpoints(dwc);
>>>>>
>>>>> free endpoints is a better place for this. It's already going to free
>>>>> the memory anyway. Might as well clear all flags to 0 there.
>>>>>
>>>>
>>>> But it won't solve the deadlock issue. Since dwc3_gadget_free_endpoints()
>>>> is called after usb_del_gadget_udc() and the deadlock happens when
>>>>
>>>> usb_del_gadget_udc()->udc_stop()->dwc3_gadget_stop()->wait_event_lock_irq()
>>>>
>>>> and DWC3_EP_END_TRANSFER_PENDING flag is set.
>>>
>>> indeed. Iterating twice over the entire endpoint list seems
>>> wasteful. Perhaps we just shouldn't wait when removing the UDC since
>>> that's essentially what this patch will do, right? If you clear the flag
>>> before calling ->udc_stop(), this means the loop in dwc3_gadget_stop()
>>> will do nothing. Might as well remove it.
>>>
>>
>> This means that we will never wait for DWC3_EP_END_TRANSFER_PENDING to clear
>> in dwc3_gadget_stop() like we used to. This is perfectly fine, right?
>>
>> It makes sense to me as dwc3_gadget_stop() calls __dwc3_gadget_stop() which
>> masks all interrupts and nobody will ever clear that flag if it was set.
> 
> I don't think so. It can not mask the endpoint events, please check
> the events which will be masked in DEVTEN register. The reason why we

Correct, endpoint events are not managed by DEVTEN.

> should wait for DWC3_EP_END_TRANSFER_PENDING to clear is that,
> sometimes the DWC3_DEPEVT_EPCMDCMPLT event will be triggered later
> than 100us, but now we may have freed the gadget irq which will cause
> crash.
> 

OK. So what is the right approach here?
In the test case I mentioned in the commit log the endpoint interrupt never
happens and it waits forever in dwc3_gadget_stop().

Since we know we're winding up, can we explicitly disable the endpoint events
here?

-- 
cheers,
-roger

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki