Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 19 May 2023 12:02:44 +0200
From:   Simon Horman <simon.horman@corigine.com>
To:     Ratheesh Kannoth <rkannoth@marvell.com>
Cc:     netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        sgoutham@marvell.com, davem@davemloft.net, edumazet@google.com,
        kuba@kernel.org, pabeni@redhat.com, linyunsheng@huawei.com,
        sbhatta@marvell.com, gakula@marvell.com, schalla@marvell.com,
        hkelam@marvell.com
Subject: Re: [PATCH net-next v3] octeontx2-pf: Add support for page pool
Message-ID: <ZGdJRMfuXHnvVQy9@corigine.com>
References: <20230519071352.3967986-1-rkannoth@marvell.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230519071352.3967986-1-rkannoth@marvell.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?u9bworcSDLIIwLVr5qSXfMTwTclxsVbth8CymgozkXLIhMAaOtBO/W+S9Ofu?=
 =?us-ascii?Q?3b2uhaz576QjUNaRcCSxxYJKD8HfILQC3y+9hNcZAlwheTxgNchLqyo+09s+?=
 =?us-ascii?Q?vomXxg2nqwEELXbVigthQOaopNcN2JIa0YPim/zYYKJo8DyZEaZMzvqugJ0g?=
 =?us-ascii?Q?MHMqjExW4xhlIyrrjtYpfQDwCuJjxATaFYWSHhWlpngE+uMMX5KkHnXoHAUq?=
 =?us-ascii?Q?8U0CUmCUEbM6KZNFVQPomIkQLwhR8UYJCiYgGncjwdU/LIAoYJzoqQdv52cF?=
 =?us-ascii?Q?AImaReVz6WDnIta0NrQu/ih1mi5kc71/Yd0EXr2Ryd3RlT70oHSfAleRvHTj?=
 =?us-ascii?Q?OsPdhuyqEZyhHPhFyhW56lj/6kxUoDpUArnni+WrSdO8ZFnSeIfRTqI0j9I3?=
 =?us-ascii?Q?UUDYQC5S1WmZQhthXQ1NmE4YrCwFYyYc43TA6X1WBn1jMEeTmkPsOg9OEG6b?=
 =?us-ascii?Q?Aca8qzycIKpIW2qHq9KqLQLF1KZbjeDpQi8QdLIUaYVYXzh6wqkKhtiAkehf?=
 =?us-ascii?Q?jcDYHH+m80mYhps3dXxn22zATucmXxrrGl7p5d9CK1Uwvg/CjKfRx0hgxLm+?=
 =?us-ascii?Q?IFKM9kMoyTVXDpbcftEirZT0KjFII+d0ngWxrUmgx84IuRjlXp9x1pnYsv+e?=
 =?us-ascii?Q?c6WMSJOVNAaWVe2gOQdrcdtNKTBEJHZ9vVfFP8jDh0JTljzH40A1uXJMd82f?=
 =?us-ascii?Q?aDWVh0MY8Mj52BnCAnFyiUH7m3FmkLVEsyGsuUstwOZv4dG+m+nUS45jg9mk?=
 =?us-ascii?Q?47brx95am1UbGzYIHV4p708F4VNNqn2loz6ltMk3Q0875PGjA2l1Ur1O+Cgs?=
 =?us-ascii?Q?d2jBUe2NPES747hlmIp7bZoXZZ/Td3nV4gL8ITWCEJcP+tpQLs492fVU99pi?=
 =?us-ascii?Q?EJWm6y0X7Vf436yWPDuY4r5VMHMXXcCCRFN1n7P1B7bsR3vlngREUfBdjgot?=
 =?us-ascii?Q?3hlP8OAEvljM2xvNpSBlDQKAe8hyc6IaV+cXB4k0k0jBd6jACQSRmmwfekEM?=
 =?us-ascii?Q?vPfcK0U8HlSXW6vX+uGaJtmsH06Dj4E6wjn4eXuNXSswnE9oAl3gEGKnXkq7?=
 =?us-ascii?Q?yeEJ7A1xTN0MbDcryAzPaPADWC5KHVSSz6AeXHRAs+gabsRrKa8cPLr4jkhn?=
 =?us-ascii?Q?q1zLDZ5/ciT1W0iTSvi1njWVhpsRRWKvXHcQsRBZuQHPVl9aqOj0cLYBigXL?=
 =?us-ascii?Q?6kG0G8mB9S4/uUpjvAtlB2AifT1Q4Y5tpTeVx4NYQloEzkRCHwfWjc46xnaO?=
 =?us-ascii?Q?U3ZCAZyU44gOucel6uYiGLf0JDaW/+Wi5ikd2BMVoBUpz248VE4XJHl3zZHR?=
 =?us-ascii?Q?F76vQsw26/P9IvVNwpsjbzLtnnpGR8zA5JgOUosWMd/vORLR6hUqpmV9UVMA?=
 =?us-ascii?Q?s+31+b5UIk1SE77cN6oXLtxfVd+ihd8B02COxoRm0yCiWUQFNACztJiqnI7V?=
 =?us-ascii?Q?/Y8Ja/2kv6zwoxw9g3focnZawud5GeE/olViRagHK/Fz9Uu5pXMq66fJ5/MZ?=
 =?us-ascii?Q?Z/MjLzJ+TEbhmSQisRvpOx5ahwRcdv1EMmtioi0KPEAm+0ipdhMXkXjPd719?=
 =?us-ascii?Q?YUVrnhI/xET94pQSQdbzqa3jj1QYm85/jdLpI6MxZNCsG9hV5prG+F6WGbC7?=
 =?us-ascii?Q?UmokbLBVH6V/pSrAFscYkh7XsgRq8/5aVf6E+aCQ5QxUc6ukIzR/U+vTkV69?=
 =?us-ascii?Q?/5jtRw=3D=3D?=
X-OriginatorOrg: corigine.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f0a1b910-24e0-4236-f7f8-08db5850350a
X-MS-Exchange-CrossTenant-AuthSource: PH0PR13MB4842.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 May 2023 10:02:51.5368
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mIBWxv/3DI9q9JTU1AacDGnn6EJ6leYiqvZN5c+AFXkuEmnCEHF5+1WLyLoS8Jp+VCCra8184CBEwIt58Ju8rHdsukInBcNNaFWRe4OfSPg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR13MB5038
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, May 19, 2023 at 12:43:52PM +0530, Ratheesh Kannoth wrote:
> Page pool for each rx queue enhance rx side performance
> by reclaiming buffers back to each queue specific pool. DMA
> mapping is done only for first allocation of buffers.
> As subsequent buffers allocation avoid DMA mapping,
> it results in performance improvement.
> 
> Image        |  Performance
> ------------ | ------------
> Vannila      |   3Mpps
>              |
> with this    |   42Mpps
> change	     |
> ---------------------------
> 
> Signed-off-by: Ratheesh Kannoth <rkannoth@marvell.com>

...

> @@ -1205,10 +1226,28 @@ void otx2_sq_free_sqbs(struct otx2_nic *pfvf)
>  	}
>  }
>  
> +void otx2_free_bufs(struct otx2_nic *pfvf, struct otx2_pool *pool,
> +		    u64 iova, int size)
> +{
> +	u64 pa = otx2_iova_to_phys(pfvf->iommu_domain, iova);
> +	struct page *page = virt_to_head_page(phys_to_virt(pa));

nit: please arrange local variables in networking code in reverse xmas tree
     order - longest line to shortest.

	u64 pa = otx2_iova_to_phys(pfvf->iommu_domain, iova);
	struct page *page;

	page = virt_to_head_page(phys_to_virt(pa));

     The following tool can check this:

        https://github.com/ecree-solarflare/xmastree

...

> @@ -1186,11 +1185,13 @@ bool otx2_sq_append_skb(struct net_device *netdev, struct otx2_snd_queue *sq,
>  }
>  EXPORT_SYMBOL(otx2_sq_append_skb);
>  
> -void otx2_cleanup_rx_cqes(struct otx2_nic *pfvf, struct otx2_cq_queue *cq)
> +void otx2_cleanup_rx_cqes(struct otx2_nic *pfvf, struct otx2_cq_queue *cq, int qidx)
>  {
>  	struct nix_cqe_rx_s *cqe;
>  	int processed_cqe = 0;
> -	u64 iova, pa;
> +	struct otx2_pool *pool;
> +	u16 pool_id;
> +	u64 iova;

Likewise here.

>  
>  	if (pfvf->xdp_prog)
>  		xdp_rxq_info_unreg(&cq->xdp_rxq);

...