We found kcompactd was excessively running in Android, after
some debug, found some order-9 allocations in iommu/dma.
It splits contiguous page to single page in dma allocation,
that means it is not necessary to alloc contiguous page, what
is more, allocation for high order may cause direct memory
reclaim and compaction, result in poor performance.
In this patch, try to alloc memory by alloc_pages_bulk_array_node
first, speedup memory allocation by saving unnecessary direct
memory reclaim and compaction, fallback to original path when
failed, beside remove __GFP_DIRECT_RECLAIM for costly order.
Signed-off-by: Chen Wandun <[email protected]>
---
drivers/iommu/dma-iommu.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index f90251572a5d..b8463934d806 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -720,7 +720,7 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
unsigned int count, unsigned long order_mask, gfp_t gfp)
{
struct page **pages;
- unsigned int i = 0, nid = dev_to_node(dev);
+ unsigned int i, nid = dev_to_node(dev);
order_mask &= (2U << MAX_ORDER) - 1;
if (!order_mask)
@@ -736,6 +736,11 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
/* It makes no sense to muck about with huge pages */
gfp &= ~__GFP_COMP;
+ i = alloc_pages_bulk_array_node(gfp, nid, count, pages);
+ if (count == i)
+ return pages;
+ count -= i;
+
while (count) {
struct page *page = NULL;
unsigned int order_size;
@@ -753,6 +758,10 @@ static struct page **__iommu_dma_alloc_pages(struct device *dev,
order_size = 1U << order;
if (order_mask > order_size)
alloc_flags |= __GFP_NORETRY;
+
+ if (order > PAGE_ALLOC_COSTLY_ORDER)
+ alloc_flags &= ~__GFP_DIRECT_RECLAIM;
+
page = alloc_pages_node(nid, alloc_flags, order);
if (!page)
continue;
--
2.25.1