Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <297dff63-e199-d14b-7148-916888030740@intel.com>
Date:   Wed, 19 Oct 2022 10:21:19 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.11.0
Subject: Re: [PATCH v5 4/7] dmaengine: Add provider documentation on cookie
 assignment
Content-Language: en-US
To:     Vinod Koul <vkoul@kernel.org>
CC:     <dmaengine@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20220622193753.3044206-1-benjamin.walker@intel.com>
 <20220829203537.30676-1-benjamin.walker@intel.com>
 <20220829203537.30676-5-benjamin.walker@intel.com> <Y1Am/RpgWv3PAVaU@matsya>
From:   "Walker, Benjamin" <benjamin.walker@intel.com>
In-Reply-To: <Y1Am/RpgWv3PAVaU@matsya>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TGpKRHVLWHluWjVZTTFGMExJYUsybkhVZjBIUEplRU1PY0hnUUdRN2ZZeFJS?=
 =?utf-8?B?ekxSdkkvSkx0bXlkWk9aSi9DNmZ4dnM3UmJsMjRTSG1XSkx2L3AzVFR3WEJ6?=
 =?utf-8?B?N3NMemg3WjJxSnowcnFqNDZFT0ZmRVAvaThDU3VZTWFsZHIxdy9nVm50Q3Z3?=
 =?utf-8?B?VjJpTDJqSTVGRlFkOGZSY3ZObXBJWWZtNThtZ2pPd0s2QVRjcHdtU1JtbitV?=
 =?utf-8?B?bCtyMS9zNFVnZHFRZCtoRjhnUDd1K1dmY2pxVVBkRnV5KzdzdUhOUjRGekQw?=
 =?utf-8?B?aEtyK0xadjIvcmcya3FTVnU5T1hBazE1V2pDVGY5MDFRN2lYNHBnenZwZGdP?=
 =?utf-8?B?clZoblFrcmtsRzVVVjFjczBwcFVuL1BzL2xjOXhEZTBaRUhXRlZnNStIOUxr?=
 =?utf-8?B?RU5Lb1EyN3ZwcjNsSkhIcTZjMThzRUdsQlBHcTUzQ2t2a3JNN0ZqR2NPQjQ1?=
 =?utf-8?B?cTJadzlvbmJjR0YvQmVmUG1JWFdxaVdXUUQzSkRxUWNBemFPRTlNeEw2NGRZ?=
 =?utf-8?B?dDV2NFZoWG1EV2xYNE5jazhqenhVMWo2bFF0bmdCMTc4QWIrakZiWVlaSmc3?=
 =?utf-8?B?dmM0SEg1TkZZZDdVMTY5eUYyWFZJUlhUcmtOMHE0QjFXMVdYZGgwNDZJSXRx?=
 =?utf-8?B?Wm9XbDA3K2pSanNGWkZDUnhpSUgxcDFINGVrZ0l2OE9LSllHUkVqdkhWYnBF?=
 =?utf-8?B?TVBaUzlpWGJhUkwwTjhuSVdmejNJTlRjV2wxUE03aUN2Z2FEUUJJM1R2Mzdh?=
 =?utf-8?B?Rm1RckcwVnJZREp4Q0FaM3dtaHFCaWFCN01Hc2NlTlVPQk1GQ3hOTnhjayta?=
 =?utf-8?B?ZjVzZ1hHWUdVU1oxM3JtQXhqSVhKSTdnbzJIeVc1eStCY1gra0YyQ1BTdGRS?=
 =?utf-8?B?UGMrVHVWMDl6RUJhS3FBQmtGT0xwT3JUMDRXbGNWdWJxd0hGK201cnpzSkFs?=
 =?utf-8?B?ZCt5bUJyVnpqZ3p0aDM0WXVlK3BnNEdwaGN4RlZGVGh2cDlZTlhxWS9seTZy?=
 =?utf-8?B?VlBuYm9nVlNjc1BHRGFKa3Fjei9uOW9zMVVzZzdZeEdQRG40RlIxUy9BUlA3?=
 =?utf-8?B?WWtLRE9tNnhPR0hpL21WYkJkbisvU2dERnEzYWkrOERRSktVYmgvQ0JLS2Ni?=
 =?utf-8?B?MnBSVWFEOUFaVEtPc0tRYWRtUUtNUGlLWnVrcXM3eG0vOHVDSy9VRXZpWFBk?=
 =?utf-8?B?WkFTbUprdHJnTjBHZ1dNSXBxL2RnUWszVHliWnk1azIrMkJwZk1aZy9ONW9n?=
 =?utf-8?B?bWFxd21STjVKZnZKVmZpbEh2Yk10ZVc4YVk5eUtuQnRvaDRJV1Y2WUdsQjBT?=
 =?utf-8?B?ODQzQS9wSS8valpOeURKVG9udjhLeFEwcE44Tk9JbEM4T2tCM1lYZE1JT3Nj?=
 =?utf-8?B?TU0zZEx5Ull6eVpDcHF5eGN2YVpJUm1nZkM5aGt4VTRJOVJPTHQyOXAxUkNG?=
 =?utf-8?B?UFBnV1MwYlViT3JqeDEyUmdQVURpOFRQMlFDSGo2dkJlQ0EzbnlhZml3YlZw?=
 =?utf-8?B?RHNZNmd2V2l0MFphL0I4dmlLVDVSazBKSDNudkdCc25ZU2JyTzZjMjFoQ0c2?=
 =?utf-8?B?NkRGUGhyczZQcDlwdEpSeVFpb3FHK1VJa3B4T21Wd2tSY2JVNFpNQ3hGcmha?=
 =?utf-8?B?Rm41eldzeWVDdFRSUk4xNDBid2pGR2UvZWpxcXk4dGdCYnRRYjluREVsVERi?=
 =?utf-8?B?TE5SblVMOHE0REt5NEJ5UUp6eUVraXdKM1RIOEY4NEhsS0ZJM2RWamlUZlF4?=
 =?utf-8?B?UzRiLzFSVGdHdFpSVFhGV0ltanVlYU9zSzVxMjBXRHQxWjlDV3BjdDd0dVZD?=
 =?utf-8?B?cHFqY3U1MjYxQmxjemVYWlBLMTBSMXdmOUNGNzdES0Z3VHU1TFJ4N2FleE1B?=
 =?utf-8?B?UHdUTmtkZ0hoTWxrZnVaR1AzRnlEcjhSckVHb2ZFRzRyejhlaDVjM3BvQk9Z?=
 =?utf-8?B?VXpNenBhZklta1VKU2pQeW9zeWc5aVdFYkpPODc4d2RXeFRCYkZ2VkhYZ0Iw?=
 =?utf-8?B?WTNjMDk5Z3k4b1RvNFZCNWJVSFVOLzViMkVtdE9XTjU5R3VPRkRsMlBXd0JK?=
 =?utf-8?B?dStuN0xOVE9MdVg0MksxZXNWdlI5bU9pMlVKcFlOVmlXaHhJY1BnaUxsbXJn?=
 =?utf-8?B?c0gya2NFTHpSQmZUZFl0b3Z3cnJmUExDaXRXdGJLREhQWXg1Ti9YaTJrYWVo?=
 =?utf-8?B?dlE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4cf86be4-1ccc-4e1c-e0dc-08dab1f65770
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB2824.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Oct 2022 17:21:21.4526
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2ZG8xwIDMwzHqP64iMaWncd5VbzrH2P+jl5z5OgDk/jpw7k/x/NF6xlbwcI1hfzSn4gU2bZwOuJNUV89PCJ0cUbgk2dmPd/IPBJTE3cTH5o=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4967
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 10/19/2022 9:34 AM, Vinod Koul wrote:
> On 29-08-22, 13:35, Ben Walker wrote:
>> Clarify the rules on assigning cookies to DMA transactions.
>>
>> Signed-off-by: Ben Walker <benjamin.walker@intel.com>
>> ---
>>   .../driver-api/dmaengine/provider.rst         | 45 +++++++++++++++----
>>   1 file changed, 37 insertions(+), 8 deletions(-)
>>
>> diff --git a/Documentation/driver-api/dmaengine/provider.rst b/Documentation/driver-api/dmaengine/provider.rst
>> index 1d0da2777921d..a5539f816d125 100644
>> --- a/Documentation/driver-api/dmaengine/provider.rst
>> +++ b/Documentation/driver-api/dmaengine/provider.rst
>> @@ -417,7 +417,9 @@ supported.
>>   
>>       - tx_submit: A pointer to a function you have to implement,
>>         that is supposed to push the current transaction descriptor to a
>> -      pending queue, waiting for issue_pending to be called.
>> +      pending queue, waiting for issue_pending to be called. Each
>> +      descriptor is given a cookie to identify it. See the section
>> +      "Cookie Management" below.
>>   
>>     - In this structure the function pointer callback_result can be
>>       initialized in order for the submitter to be notified that a
>> @@ -522,6 +524,40 @@ supported.
>>   
>>     - May sleep.
>>   
>> +Cookie Management
>> +------------------
>> +
>> +When a transaction is queued for submission via tx_submit(), the provider
>> +must assign that transaction a cookie (dma_cookie_t) to uniquely identify it.
>> +The provider is allowed to perform this assignment however it wants, but for
> 
> We assumes that we have monotonically increasing cookie and
> if cookie 10 is marked complete cookie 8 is assumed complete too...

That's exactly what this patch series is changing. The earlier patches 
make changes to no longer report to the client the "last" or "used" 
cookie (to compare against) in the client APIs, and it turns out that 
nothing in the kernel actually cares about this behavior. So it's simply 
a documentation change to indicate that the client no longer has any 
visibility into the cookie behavior.

Immediately below here the documentation then says that there's some 
convenience functions that providers can use that do produce 
monotonically increasing cookies. These are now optional for providers 
to use, if they find them useful, rather than the required way to manage 
the cookies.

> 
> Completion is always in order unless we specify DMA_COMPLETION_NO_ORDER

The final patch in this series eliminates DMA_COMPLETION_NO_ORDER 
entirely. It was only used by the IDXD driver, and the reason I'm doing 
these patches is so that we can poll the IDXD driver for completions 
even though it can complete out of order.

> 
>> +convenience the following utility functions are available to create
>> +monotonically increasing cookies
>> +
>> +  .. code-block:: c
>> +
>> +    void dma_cookie_init(struct dma_chan *chan);
>> +
>> +  Called once at channel creation
>> +
>> +  .. code-block:: c
>> +
>> +    dma_cookie_t dma_cookie_assign(struct dma_async_tx_descriptor *tx);
>> +
>> +  Assign a cookie to the given descriptor
>> +
>> +  .. code-block:: c
>> +
>> +    void dma_cookie_complete(struct dma_async_tx_descriptor *tx);
>> +
>> +  Mark the descriptor as complete and invalidate the cookie
>> +
>> +  .. code-block:: c
>> +
>> +    enum dma_status dma_cookie_status(struct dma_chan *chan,
>> +      dma_cookie_t cookie, struct dma_tx_state *state);
>> +
>> +  Report the status of the cookie and filling in state, if not NULL.
>> +
>>   
>>   Misc notes
>>   ==========
>> @@ -537,13 +573,6 @@ where to put them)
>>   - Makes sure that dependent operations are run before marking it
>>     as complete.
>>   
>> -dma_cookie_t
>> -
>> -- it's a DMA transaction ID.
>> -
>> -- The value can be chosen by the provider, or use the helper APIs
>> -  such as dma_cookie_assign() and dma_cookie_complete().
>> -
>>   DMA_CTRL_ACK
>>   
>>   - If clear, the descriptor cannot be reused by provider until the
>> -- 
>> 2.37.1
>