Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [PATCH net-next 6/9] page_pool: avoid calling no-op externals
 when possible
To:     Alexander Lobakin <aleksander.lobakin@intel.com>
CC:     "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
        Larysa Zaremba <larysa.zaremba@intel.com>,
        Alexander Duyck <alexanderduyck@fb.com>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        Ilias Apalodimas <ilias.apalodimas@linaro.org>,
        Simon Horman <simon.horman@corigine.com>,
        <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20230727144336.1646454-1-aleksander.lobakin@intel.com>
 <20230727144336.1646454-7-aleksander.lobakin@intel.com>
 <a79cc7ed-5355-ef7d-8865-0ba9673af5c6@huawei.com>
 <604d4f6c-a6e7-e921-2d9a-45fe46ab9e79@intel.com>
 <799ebbaf-961d-860a-6071-b74e10360e29@huawei.com>
 <1644b9d0-27a5-0c2b-c530-bcaa347f73c2@intel.com>
From:   Yunsheng Lin <linyunsheng@huawei.com>
Message-ID: <00695c43-b376-169d-a62d-c1a373cde90c@huawei.com>
Date:   Wed, 2 Aug 2023 19:37:35 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
 Thunderbird/52.2.0
MIME-Version: 1.0
In-Reply-To: <1644b9d0-27a5-0c2b-c530-bcaa347f73c2@intel.com>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 2023/8/1 21:42, Alexander Lobakin wrote:
...

>>>>
>>>> It seems other subsystem may have the similar problem as page_pool,
>>>> is it possible to implement this kind of trick in the dma subsystem
>>>> instead of every subsystem inventing their own trick?
>>>
>>> In the ladder I described above most of overhead comes from jumping
>>> between Page Pool functions, not the generic DMA ones. Let's say I do
>>> this shortcut in dma_sync_single_range_for_device(), that is too late
>>> already to count on some good CPU saves.
>>
>> We can force inline the page_pool_dma_sync_for_device() function if it
>> is 'the good CPU saves' you mentioned above.
>>
>>> Plus, DMA sync API operates with dma_addr_t, not struct page. IOW it's
>>> not clear to me where to store this "we can shortcut" bit in that case.
>>
>> It seems we only need one bit in 'struct device' to do the 'shortcut',
>> and there seems to have avaliable bit at the end of 'struct device'?
> 
> dma_need_sync() can return different results for two different DMA
> addresses within the same device.

Yes, that's why we need a per device state in order to do the
similar trick like this patch does.

> 
>>
>> Is it possible that we do something like this patch does in
>> dma_sync_single_range_for_device()?
>>
>> One thing to note is that there may be multi concurrent callers to
>> dma_sync_single_range_for_device(), which seems to be different from
>> atomic context for page_pool_dma_map(), so it may need some atomic
>> operation for the state changing if we want to implement it in a 'generic'
>> way.
>>
>>>
>>> >From "other subsystem" I remember only XDP sockets. There, they also
>>> avoid calling their own non-inline functions in the first place, not the
>>> generic DMA ones. So I'd say both cases (PP and XSk) can't be solved via
>>> some "generic" solution.
>>
>> If PP and XSk both have a similar trick, isn't it a more clear sight
>> that it may be solved via some "generic" solution?
> 
> Both shortcut their own functions in the first place, so I don't know
> what generic solution could be to optimize non-generic functions.

If we are able to shortcut the generic functions, for the page_pool
and XSK case，it seems the non-generic functions just need to be
inlined if I understand your concern correctly.

And for that we may be able to shortcut the generic functions for
dma_sync_single_range_for_device() used in driver too?

> 
>>
>> Is there any reason there is no a similar trick for sync for cpu in
>> XSk as below code indicates?
>> https://elixir.free-electrons.com/linux/v6.4-rc6/source/include/net/xsk_buff_pool.h#L152