Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chen Wandun <chenwandun@huawei.com>
To:     <joro@8bytes.org>, <will@kernel.org>,
        <iommu@lists.linux-foundation.org>, <linux-kernel@vger.kernel.org>
CC:     <wangkefeng.wang@huawei.com>, <chenwandun@huawei.com>
Subject: [PATCH] iommu/dma: speedup memory allocation in __iommu_dma_alloc_pages
Date:   Tue, 6 Sep 2022 10:39:02 +0800
Message-ID: <20220906023902.133103-1-chenwandun@huawei.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7BIT
Content-Type:   text/plain; charset=US-ASCII
Precedence: bulk

We found kcompactd was excessively running in Android, after
some debug, found some order-9 allocations in iommu/dma.

It splits contiguous page to single page in dma allocation,
that means it is not necessary to alloc contiguous page, what
is more, allocation for high order may cause direct memory
reclaim and compaction, result in poor performance.

In this patch, try to alloc memory by alloc_pages_bulk_array_node
first, speedup memory allocation by saving unnecessary direct
memory reclaim and compaction, fallback to original path when
failed, beside remove __GFP_DIRECT_RECLAIM for costly order.

Signed-off-by: Chen Wandun <chenwandun@huawei.com>
---
 drivers/iommu/dma-iommu.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index f90251572a5d..b8463934d806 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -720,7 +720,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 		unsigned int count, unsigned long order_mask, gfp_t gfp)
 {
 	struct page **pages;
-	unsigned int i = 0, nid = dev_to_node(dev);
+	unsigned int i, nid = dev_to_node(dev);
 
 	order_mask &= (2U << MAX_ORDER) - 1;
 	if (!order_mask)
@@ -736,6 +736,11 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 	/* It makes no sense to muck about with huge pages */
 	gfp &= ~__GFP_COMP;
 
+	i = alloc_pages_bulk_array_node(gfp, nid, count, pages);
+	if (count == i)
+		return pages;
+	count -= i;
+
 	while (count) {
 		struct page *page = NULL;
 		unsigned int order_size;
@@ -753,6 +758,10 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
 			order_size = 1U << order;
 			if (order_mask > order_size)
 				alloc_flags |= __GFP_NORETRY;
+
+			if (order > PAGE_ALLOC_COSTLY_ORDER)
+				alloc_flags &= ~__GFP_DIRECT_RECLAIM;
+
 			page = alloc_pages_node(nid, alloc_flags, order);
 			if (!page)
 				continue;
-- 
2.25.1