Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751788AbdIUJAG (ORCPT ); Thu, 21 Sep 2017 05:00:06 -0400 Received: from mail-co1nam03on0052.outbound.protection.outlook.com ([104.47.40.52]:14142 "EHLO NAM03-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751068AbdIUJAE (ORCPT ); Thu, 21 Sep 2017 05:00:04 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Ganapatrao.Kulkarni@cavium.com; From: Ganapatrao Kulkarni To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, iommu@lists.linux-foundation.org, linux-mm@kvack.org Cc: Will.Deacon@arm.com, robin.murphy@arm.com, lorenzo.pieralisi@arm.com, hanjun.guo@linaro.org, joro@8bytes.org, vbabka@suse.cz, akpm@linux-foundation.org, mhocko@suse.com, Tomasz.Nowicki@cavium.com, Robert.Richter@cavium.com, jnair@caviumnetworks.com, gklkml16@gmail.com Subject: [PATCH 0/4] numa, iommu/smmu: IOMMU/SMMU driver optimization for NUMA systems Date: Thu, 21 Sep 2017 14:29:18 +0530 Message-Id: <20170921085922.11659-1-ganapatrao.kulkarni@cavium.com> X-Mailer: git-send-email 2.9.4 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: PN1PR01CA0111.INDPRD01.PROD.OUTLOOK.COM (10.174.144.27) To CO2PR07MB2566.namprd07.prod.outlook.com (10.166.201.17) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 33398301-a61b-4349-d3c0-08d500cf2434 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:CO2PR07MB2566; X-Microsoft-Exchange-Diagnostics: 1;CO2PR07MB2566;3:5u9t6h3CobkHKgQp4lKwRs9Oqq+cV2ELWhlyRywatGMzwJ1VGtQFvACQFaXDvPEG9OvhMay0pBo8hTiD6y1gT75GZD5EgYCfxpfNiG6EGMwJFjtuPWPzDKLZnmcTIwFsfbUhlK0vVZrTYMKzsCTQNtE/BMQKv8tNugA1jtGslE8+ob9sYNHiCSlmjcJDH3DXhoG/foo8LRm0hQDzAOufWoQ3mnSg14pJ0E90QnNOWEeetD0AK4vihwPYompbTk+O;25:63mtswbzQ0H64Ic3JTgr1kblq/xDn0pSkmgi3WrmvmjRZ+c1UntVbG7dawKbzcYxOtQheU2IAFYfCTdyCvg7Mqd7ULzvTLKWfKgDvRlUSQOA701VYVnIQ0rC1TCLdllGX8nYztbHegj5Q5f5aYV25MqzxEW7gsdpK52WBhc6JA1sSTZmCFi/DC3/o2fG8zT9XHck0phOOXye34l3QS8FJyKZuNZlbBiPzNXDT2AqVtZAxtQ17TrRU3P9epb4b274xtEBv5RBO49TAr6KGX81ujwm1gnvEoV3aTyuCKSE4IyQQrrsSu/efPFEp+XB2LjMWmR2pW2AH1xOMfOhPb+aiQ==;31:+1t+YQVtA8zZpfQl8Gd35fw99bqxE0TQ5LBwhCbZhKoFFNNQfTkq0YBRm11TtCO0Iad3Hyr9z6T5mv1OzY676WwdnmyqjrAy5WyyYqsGuaUSJsUq28R8Cw8UTtW6ieeGeu7HZMOSIu3iuRXbQLpoaUVFl7VOCrEGVIz7BkINNvSsMBdG+BLRhOexGZX9gMt6+mAofQogm7XEELHD6YLvDwkX37p0VHiQHvSXbFqxfXQ= X-MS-TrafficTypeDiagnostic: CO2PR07MB2566: X-Microsoft-Exchange-Diagnostics: 1;CO2PR07MB2566;20:TobC7wDIczJy2IehaNBJSsZKCEji8/jfosLlsF7HkxPg1bTBG44WNNR3Ls3iy6O6fNGHz6HV1B0o8xu7tWhuacYInU/fn8ls9Q9Yf3dJQRanzAZREyZHJtsLA8ITs1YRg6yFEJrVjYIXDI5+Sn6sEUSZJPIgb0Y+YrbPanJmr1s7OztPxtnd5ZZ9xoQmFY7O6xXlv/79/4NDVUdcjQ9On8TfpM1/jGN8ztu7KqO3xpnK+QcZ7Oxv4GFpDIpNFPhxlHAlL24fzov9jG3B1MpfSxGIdOqgZvhKX5h3jL23088ecrWJosOpWfusmQeVcyg6OQHVOQFX3XpACo8/60U8UcWpySYUo62uBym9aQ9fPjP/mz4FSMRcK89HYBl1bk5kETwM2MlKDCPaRqI5XyvF7va70lxmlVwthR1e/45rlZZi28DdVAtf4vLLYf8LlfGuzZy1N5wJJVFFsDYh6SFQibb5A6Q9jYU5Y7LO9/SEzRKjZ4G11N5Jx9vcsesaIg4A;4:PxYvq46/DGjg5YV0hYB9H4ueT80p4xuzd2MvCylpsxTat4RnVO/5L7OMaPcy4r/UYGfI/+u/b2rLMGwz17gtnLLsZuaCJzx/AEK65YRP6WQMzzFpdxVfMTdE/ePTpVzir1WV+hYVMo8820q00B35Aq9pixwcUXAPTUcFMe8aQOjMb+0mS/MAMVWLYcIVwdZrHfUgsY16mcQx7YGlGTGoIQsGTPSi8orPfqHxXsOYlcKcCmYPBveXnZcDs8BDEq8k X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123555025)(20161123562025)(20161123560025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:CO2PR07MB2566;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:CO2PR07MB2566; X-Forefront-PRVS: 04371797A5 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6009001)(376002)(346002)(199003)(189002)(1076002)(3846002)(6116002)(6486002)(316002)(68736007)(6506006)(16526017)(25786009)(97736004)(305945005)(7736002)(36756003)(6666003)(16586007)(5003940100001)(4326008)(5009440100003)(86362001)(53936002)(50986999)(189998001)(39060400002)(66066001)(50226002)(478600001)(8936002)(106356001)(105586002)(33646002)(7416002)(2906002)(47776003)(8676002)(101416001)(81166006)(6512007)(81156014)(5660300001)(72206003)(50466002)(48376002);DIR:OUT;SFP:1101;SCL:1;SRVR:CO2PR07MB2566;H:mypc.cavium.com.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;CO2PR07MB2566;23:lpECgCal9Cd4mStEvd0ATEXZjBDfX9zGND0tKLknI?= =?us-ascii?Q?Pg90QuO1qVhT48a0aGiiEXKLXlDN1VRStK7Q264IRtlmxNissrO4EngSXYey?= =?us-ascii?Q?XV2wo8bBFHyymg+pU/xFagrHk/IR1NyRMjYaqCd0ayzCFda5IXCAedxNi90q?= =?us-ascii?Q?BV1Pnu8bX3PWSGTwShTAwxTQl5iXJBpj7S0Ofz9nnOmGozZUNpB6vBtXpEkL?= =?us-ascii?Q?a+VKi4k84UTXI/rMO9DUtvf21uEkt8mdvgv90kcvswTu9/WE6i0L4Ckx/Dte?= =?us-ascii?Q?KsfDtpd+XK85PvV9Ezs0oj10cLXIAJTSDM/nQasH5ysX0CBLbcxF7A0F0LjP?= =?us-ascii?Q?DTvhaTMYID062G1DWS9mDK/PrvDkxe9aiYP6N8wQXJcZ5TVQaat7oLidmCdb?= =?us-ascii?Q?A55ZR5+Dsh33fj/ldx+vmjXVOK9bl4igS5IPZQ/KMLu66rwJQaCBhVmw2zrF?= =?us-ascii?Q?kpvyINd0P9Cb8XlJ5OUHLLJS/kkScC883NnNfQtXB/KFK2W8MBwS6LB2JFoL?= =?us-ascii?Q?YeduBXkCPEBiO03vy/XFY39FQhdd/URIGxEXsfWOHnTonBcvq2T+UDgIuWt3?= =?us-ascii?Q?GOEinMpXxpf5UJqd032wxghUyo3jiwCQeLVLRx3pJy7T/rAVN1LkvX61rTyR?= =?us-ascii?Q?uB1R+H/kA/2Ai/LnkhQhLRDHihkZrrgbsPjDVZFWr5HFiSkQWYCK4E18+o8Q?= =?us-ascii?Q?UsIzIiD3/Kfag9mIVFsB5cvZemiJRiTtn3RV0NEoRjIwAxdmjpI+Tu5l09Et?= =?us-ascii?Q?CixQeutetQzH048YIK05IZ3wAvyr5fRz4C7Cyhi+Y4Hd43s/70G73mbRYA32?= =?us-ascii?Q?JV2prvweZOpKfXjDPdWnfrtrwmuoAqPiUofQLCpW+1VwLlNVsM9B3RYQBUBP?= =?us-ascii?Q?1XRXCVaQGtAJ/gDzVIzV22KoqQIRbZmpJzg/Nh+Bui0mEGx69+9RnrJG8CAI?= =?us-ascii?Q?ePWuTWoe+N4zzkZLJrAcaAGffhiTbZagyOIYP8HAbt7H3k5CVm5202Vy205o?= =?us-ascii?Q?4V0amTGbnxdLKk0tMaD7NIQ8kg3Qzmdv+V3xhiPvT55xaox+KJqUWxckoQRO?= =?us-ascii?Q?TFi1NCQ5976qPC1N6kfWGhyk9Z8SnzvAJ4lGJ8sHM0PIQ6ewzKjfFCYOMkSD?= =?us-ascii?Q?XeyBUJHaF0=3D?= X-Microsoft-Exchange-Diagnostics: 1;CO2PR07MB2566;6:DlPEF6jopmTG+6OyXCB/g0FK1Wrg9p8rn0XL9DGyBg17zI2LnUbZwt+i2nidpjCYHcygc2Rv2+66IPCLXu5w8k4fEaZHPYISFN+SUJu4HQMGHHjE+VW62HC4+rg9Y8CyavcoRhCi1XYO9p342fXfXnI4u6uSRLSKZOREgass6IETG5CrZtSo9x/GEqfiaVUNOYqp4aCzI64mSL4JzR1JnVsnzjsfeVbV3uXbjZms7u6jCqw8eeBmYpCnSM8ZfvLHAOAKzmjL9wjpj70N4qBzrMeHPCUrk1NTPeVlPZSL0DVd1cCAjuJPJOzbsVCY997sa7qFMrR9CcJmrqk7UmGi9Q==;5:VgVwN86YB3NPiGCO5sfbwDJl5QuiXNQmIjSnSVdq3GPlVeQuPRO0S5irBIhKmQ9RHRjMT8yqa73vCKmXL9XnIugQxI/2weI6081rfyPdiQcypiwy9sT+jf9CV7qYeCrkCbHto9lwnRqFQngVs+jB8w==;24:Blvfr+E0UI+YQSVju1RxSGZzcUqQ9PjsiK71KPE8bQ6+Y78KQUdCQdUBTTvHwrLJYErbj4CXW67IaIFJCyjrW5WQTU/aEHeKKPiRrh8GibM=;7:y3NEcmeJcPcT/PIFxiV3zOQRbk2Q8t2MyPmVI1BvdawXVjpwztjiM8rqA92bCN7d0ZU+S09nh//IxgjoHSp2KGjqfBqW79jA9BzT8CwZkoKp8FC/BDirn5+NRLThQ2OHZGghImXyWOnYZCQGitEK+Wq+Kcm4P7Gjh+zEqLgzCxYmsKhB+6axoQR0c6njG0QDOpz+/+eOlOph4VRfTq8TpMXAIRz0Mq8xB2mehCC8iJg= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2017 08:59:56.0964 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO2PR07MB2566 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 956 Lines: 24 Adding numa aware memory allocations used for iommu dma allocation and memory allocated for SMMU stream tables, page walk tables and command queues. With this patch, iperf testing on ThunderX2, with 40G NIC card on NODE 1 PCI shown same performance(around 30% improvement) as NODE 0. Ganapatrao Kulkarni (4): mm: move function alloc_pages_exact_nid out of __meminit numa, iommu/io-pgtable-arm: Use NUMA aware memory allocation for smmu translation tables iommu/arm-smmu-v3: Use NUMA memory allocations for stream tables and comamnd queues iommu/dma, numa: Use NUMA aware memory allocations in __iommu_dma_alloc_pages drivers/iommu/arm-smmu-v3.c | 57 +++++++++++++++++++++++++++++++++++++----- drivers/iommu/dma-iommu.c | 17 +++++++------ drivers/iommu/io-pgtable-arm.c | 4 ++- include/linux/gfp.h | 2 +- mm/page_alloc.c | 3 ++- 5 files changed, 67 insertions(+), 16 deletions(-) -- 2.9.4