Message-ID: <8c6b145c-6d38-2055-7526-7586edbea45d@quicinc.com>
Date:   Thu, 2 Feb 2023 17:44:04 -0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Subject: Re: [RFC PATCH v2 09/22] ASoC: qcom: qdsp6: Introduce USB AFE port to
 q6dsp
Content-Language: en-US
From:   Wesley Cheng <quic_wcheng@quicinc.com>
To:     Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>,
        <srinivas.kandagatla@linaro.org>, <mathias.nyman@intel.com>,
        <perex@perex.cz>, <lgirdwood@gmail.com>, <andersson@kernel.org>,
        <krzysztof.kozlowski+dt@linaro.org>, <gregkh@linuxfoundation.org>,
        <Thinh.Nguyen@synopsys.com>, <broonie@kernel.org>,
        <bgoswami@quicinc.com>, <tiwai@suse.com>, <robh+dt@kernel.org>,
        <agross@kernel.org>
CC:     <devicetree@vger.kernel.org>, <alsa-devel@alsa-project.org>,
        <linux-arm-msm@vger.kernel.org>, <linux-usb@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <quic_jackp@quicinc.com>,
        <quic_plai@quicinc.com>
References: <20230126031424.14582-1-quic_wcheng@quicinc.com>
 <20230126031424.14582-10-quic_wcheng@quicinc.com>
 <dea77277-6971-fe27-1ae0-ed551e84b6e4@linux.intel.com>
 <5dec443d-9894-2d06-1798-c56b8f2e1e5e@quicinc.com>
 <ebf8ebed-ef79-bf18-4635-360f916877a6@linux.intel.com>
 <fa35124c-682f-8045-6733-c71f9ce8df9e@quicinc.com>
 <de03c6b1-b6d6-0045-ea60-6ae800913e95@linux.intel.com>
 <a494e063-f0a0-2a71-5b7b-cb247efa245c@quicinc.com>
In-Reply-To: <a494e063-f0a0-2a71-5b7b-cb247efa245c@quicinc.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Pierre,

On 2/2/2023 5:23 PM, Wesley Cheng wrote:
> Hi Pierre,
> 
> On 1/31/2023 7:02 PM, Pierre-Louis Bossart wrote:
>>
>>
>> On 1/31/23 20:40, Wesley Cheng wrote:
>>> Hi Pierre,
>>>
>>> On 1/30/2023 3:59 PM, Pierre-Louis Bossart wrote:
>>>>
>>>>
>>>> On 1/30/23 16:54, Wesley Cheng wrote:
>>>>> Hi Pierre,
>>>>>
>>>>> On 1/26/2023 7:38 AM, Pierre-Louis Bossart wrote:
>>>>>>
>>>>>>
>>>>>> On 1/25/23 21:14, Wesley Cheng wrote:
>>>>>>> The QC ADSP is able to support USB playback endpoints, so that the
>>>>>>> main
>>>>>>> application processor can be placed into lower CPU power modes.  
>>>>>>> This
>>>>>>> adds
>>>>>>> the required AFE port configurations and port start command to
>>>>>>> start an
>>>>>>> audio session.
>>>>>>>
>>>>>>> Specifically, the QC ADSP can support all potential endpoints 
>>>>>>> that are
>>>>>>> exposed by the audio data interface.  This includes, feedback
>>>>>>> endpoints
>>>>>>> (both implicit and explicit) as well as the isochronous (data)
>>>>>>> endpoints.
>>>>>>> The size of audio samples sent per USB frame (microframe) will be
>>>>>>> adjusted
>>>>>>> based on information received on the feedback endpoint.
>>>>>>
>>>>>> I think you meant "support all potential endpoint types"
>>>>>>
>>>>>> It's likely that some USB devices have more endpoints than what 
>>>>>> the DSP
>>>>>> can handle, no?
>>>>>>
>>>>>
>>>>> True, as we discussed before, we only handle the endpoints for the 
>>>>> audio
>>>>> interface.  Other endpoints, such as HID, or control is still 
>>>>> handled by
>>>>> the main processor.
>>>>
>>>> The number of isoc/audio endpoints can be larger than 1 per direction,
>>>> it's not uncommon for a USB device to have multiple connectors on the
>>>> front side for instruments, mics, monitor speakers, you name it. Just
>>>> google 'motu' or 'rme usb' and you'll see examples of USB devices that
>>>> are very different from plain vanilla headsets.
>>>>
>>>
>>> Thanks for the reference.
>>>
>>> I tried to do some research on the RME USB audio devices, and they
>>> mentioned that they do have a "class compliant mode," which is for
>>> compatibility w/ Linux hosts.  I didn't see a vendor specific USB SND
>>> driver matching the USB VID/PID either, so I am assuming that it uses
>>> the USB SND driver as is.(and that Linux doesn't currently support their
>>> vendor specific mode)  In that case, the device should conform to the
>>> UAC2.0 spec (same statement seen on UAC3.0), which states in Section
>>> 4.9.1 Standard AS Interface Descriptor Table 4-26:
>>>
>>> "4 bNumEndpoints 1 Number Number of endpoints used by this
>>> interface (excluding endpoint 0). Must be
>>> either 0 (no data endpoint), 1 (data
>>> endpoint) or 2 (data and explicit feedback
>>> endpoint)."
>>>
>>> So each audio streaming interface should only have 1 data and
>>> potentially 1 feedback.  However, this device does expose a large number
>>> of channels (I saw up to 18 channels), which the USB backend won't be
>>> able to support.  I still need to check how ASoC behaves if I pass in a
>>> profile that the backend can't support.

Getting back to passing in a format/profile that the USB BE doesn't 
support.  It looks like ASoC doesn't actually check against the PCM HW 
params received (for components), so the audio playback does still occur 
even though its outside of what we support.

Will need to add changes to specifically check for # of channels, 
format, etc... before we allow the session to proceed.

Thanks
Wesley Cheng