Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH] usb: dwc3: Prevent indefinite sleep in _dwc3_set_mode
 during suspend/resume
To:     Felipe Balbi <balbi@kernel.org>,
        Baolin Wang <baolin.wang@linaro.org>
CC:     USB <linux-usb@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
References: <1519730526-22274-1-git-send-email-rogerq@ti.com>
 <87sh9l5z4l.fsf@linux.intel.com>
 <94cd6377-1327-2309-8d69-6ab0de2bdfd4@ti.com>
 <87po4i3o1v.fsf@linux.intel.com>
 <a907787a-dea2-5f15-3d25-693680a42b71@ti.com>
 <CAMz4kuKOnAPckX1=nqVPyF7O5u-0YrekTfMddYwe0WC2-0AFDw@mail.gmail.com>
 <87k1uq3ho6.fsf@linux.intel.com>
 <8ec0485e-89af-568b-e34a-b0cd490817d0@ti.com>
 <CAMz4kuJpBXY92zhG7s_VRkpcTHeLCE_vBLLSFVxLWjN2GQvyyw@mail.gmail.com>
 <87h8puwyn5.fsf@linux.intel.com>
From:   Roger Quadros <rogerq@ti.com>
Message-ID: <5bc56ef5-66b1-d40c-1639-e748fe18cdbd@ti.com>
Date:   Fri, 9 Mar 2018 11:19:37 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.6.0
MIME-Version: 1.0
In-Reply-To: <87h8puwyn5.fsf@linux.intel.com>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 05/03/18 13:27, Felipe Balbi wrote:
> 
> Hi,
> 
> Baolin Wang <baolin.wang@linaro.org> writes:
>>>>>>>>>>  void dwc3_gadget_exit(struct dwc3 *dwc)
>>>>>>>>>>  {
>>>>>>>>>> +  int epnum;
>>>>>>>>>> +  unsigned long flags;
>>>>>>>>>> +
>>>>>>>>>> +  spin_lock_irqsave(&dwc->lock, flags);
>>>>>>>>>> +  for (epnum = 2; epnum < DWC3_ENDPOINTS_NUM; epnum++) {
>>>>>>>>>> +          struct dwc3_ep  *dep = dwc->eps[epnum];
>>>>>>>>>> +
>>>>>>>>>> +          if (!dep)
>>>>>>>>>> +                  continue;
>>>>>>>>>> +
>>>>>>>>>> +          dep->flags &= ~DWC3_EP_END_TRANSFER_PENDING;
>>>>>>>>>> +  }
>>>>>>>>>> +  spin_unlock_irqrestore(&dwc->lock, flags);
>>>>>>>>>> +
>>>>>>>>>>    usb_del_gadget_udc(&dwc->gadget);
>>>>>>>>>>    dwc3_gadget_free_endpoints(dwc);
>>>>>>>>>
>>>>>>>>> free endpoints is a better place for this. It's already going to free
>>>>>>>>> the memory anyway. Might as well clear all flags to 0 there.
>>>>>>>>>
>>>>>>>>
>>>>>>>> But it won't solve the deadlock issue. Since dwc3_gadget_free_endpoints()
>>>>>>>> is called after usb_del_gadget_udc() and the deadlock happens when
>>>>>>>>
>>>>>>>> usb_del_gadget_udc()->udc_stop()->dwc3_gadget_stop()->wait_event_lock_irq()
>>>>>>>>
>>>>>>>> and DWC3_EP_END_TRANSFER_PENDING flag is set.
>>>>>>>
>>>>>>> indeed. Iterating twice over the entire endpoint list seems
>>>>>>> wasteful. Perhaps we just shouldn't wait when removing the UDC since
>>>>>>> that's essentially what this patch will do, right? If you clear the flag
>>>>>>> before calling ->udc_stop(), this means the loop in dwc3_gadget_stop()
>>>>>>> will do nothing. Might as well remove it.
>>>>>>>
>>>>>>
>>>>>> This means that we will never wait for DWC3_EP_END_TRANSFER_PENDING to clear
>>>>>> in dwc3_gadget_stop() like we used to. This is perfectly fine, right?
>>>>>>
>>>>>> It makes sense to me as dwc3_gadget_stop() calls __dwc3_gadget_stop() which
>>>>>> masks all interrupts and nobody will ever clear that flag if it was set.
>>>>>
>>>>> I don't think so. It can not mask the endpoint events, please check
>>>>> the events which will be masked in DEVTEN register. The reason why we
>>>>> should wait for DWC3_EP_END_TRANSFER_PENDING to clear is that,
>>>>> sometimes the DWC3_DEPEVT_EPCMDCMPLT event will be triggered later
>>>>> than 100us, but now we may have freed the gadget irq which will cause
>>>>> crash.
>>>>
>>>> We could mask command complete events as soon as ->udc_stop() is called,
>>>> right? Hmm, actually, __dwc3_gadget_stop() already clears DEVTEN
>>>> completely.
>>>
>>> But which bit in DEVTEN says Endpoint events are disabled?
>>
>> When we set up the DWC3_DEPCMD_ENDTRANSFER command in
>> dwc3_stop_active_transfer(), we can do not set DWC3_DEPCMD_CMDIOC,
>> then there will no endpoint command complete interrupts I think.
>>
>> cmd |= DWC3_DEPCMD_CMDIOC;
> 
> I remember some part of the databook mandating CMDIOC to be set. We
> could test it out without and see if anything blows up. I would,
> however, require a lengthy comment explaining that we're deviating from
> databook revision x.yya, section foobar because $reasons. :-)
> 

This is what the v3.10 databook says

"When issuing an End Transfer command, software must set the CmdIOC bit (field 8) so that an Endpoint
Command Complete event is generated after the transfer ends. This is necessary to synchronize the
conclusion of system bus traffic before the End Transfer command is completed."

with a note
"If GUCTL2[Rst_actbitlater] is set, Software can poll the completion of the End Transfer
command by polling the command active bit to be cleared to 0."

fyi.
Rst_actbitlater - "Enable clearing of the command active bit for the ENDXFER
command after the command execution is completed.
This bit is valid in device mode only."

So I'd prefer not to clear CMDIOC for all cases.

Could we some how just tackle the dwc3_gadget_exit case like I did in this patch?

-- 
cheers,
-roger

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki