Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932234AbcCURj0 (ORCPT ); Mon, 21 Mar 2016 13:39:26 -0400 Received: from mail-bl2on0064.outbound.protection.outlook.com ([65.55.169.64]:57824 "EHLO na01-bl2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932128AbcCURjX (ORCPT ); Mon, 21 Mar 2016 13:39:23 -0400 From: "Chalamarla, Tirumalesh" To: Catalin Marinas , Will Deacon CC: Ganesh Mahendran , "stable@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH] Revert "arm64: Increase the max granular size" Thread-Topic: [PATCH] Revert "arm64: Increase the max granular size" Thread-Index: AQHRf2q07H5kbo3Bx0+pHW/tbsCLGJ9fP6wAgATroICAAAKCgIAAAt6A//+MVQA= Date: Mon, 21 Mar 2016 17:39:20 +0000 Message-ID: References: <1458120743-12145-1-git-send-email-opensource.ganesh@gmail.com> <20160321171403.GE25466@e104818-lin.cambridge.arm.com> <20160321172301.GP23397@arm.com> <20160321173317.GF25466@e104818-lin.cambridge.arm.com> In-Reply-To: <20160321173317.GF25466@e104818-lin.cambridge.arm.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/0.0.0.160212 authentication-results: arm.com; dkim=none (message not signed) header.d=none;arm.com; dmarc=none action=none header.from=caviumnetworks.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [64.2.3.194] x-ms-office365-filtering-correlation-id: d10719fc-e596-4ba5-9624-08d351afbc21 x-microsoft-exchange-diagnostics: 1;BLUPR0701MB1777;5:K11hPSeFpcGURLL8EbtFQh3NK6Q0IvAkt6JzSUS9Hu43rYA39SnuIAJu2RHZPVwvBtg5q9sm+eojRrQlf3YvGmhfkawtADbp0DUpkle4EpPVgGlwrtIrkNxfW2pKldje92mtDrcWxX64N0yJ0Kx3Cg==;24:daN4pAz2Mek4QfLg2zWkEA4BS56goeUZB+JA8LPQQzptHsfdrOSNSpHRaUAaHbGkiwxhcKQ+yT17xd+dM4Ue5eO7AYYn9V0IaIbeL+Qs1+8= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0701MB1777; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001);SRVR:BLUPR0701MB1777;BCL:0;PCL:0;RULEID:;SRVR:BLUPR0701MB1777; x-forefront-prvs: 0888B1D284 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(377454003)(24454002)(83716003)(1096002)(1220700001)(77096005)(6116002)(3846002)(102836003)(586003)(33656002)(81166005)(2900100001)(5002640100001)(5001770100001)(11100500001)(54356999)(87936001)(122556002)(86362001)(76176999)(5004730100002)(50986999)(4001350100001)(10400500002)(82746002)(2950100001)(106116001)(83506001)(66066001)(3280700002)(19580405001)(93886004)(99286002)(19580395003)(3660700001)(2906002)(4326007)(92566002)(5008740100001)(36756003)(189998001)(104396002);DIR:OUT;SFP:1101;SCL:1;SRVR:BLUPR0701MB1777;H:BLUPR0701MB1780.namprd07.prod.outlook.com;FPR:;SPF:None;MLV:sfv;LANG:en; spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: <2B851F2A5EE45A448C53E4A8623A3FF9@namprd07.prod.outlook.com> MIME-Version: 1.0 X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2016 17:39:20.4316 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLUPR0701MB1777 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id u2LHdZVn008395 Content-Length: 2127 Lines: 53 On 3/21/16, 10:33 AM, "Catalin Marinas" wrote: >On Mon, Mar 21, 2016 at 05:23:01PM +0000, Will Deacon wrote: >> On Mon, Mar 21, 2016 at 05:14:03PM +0000, Catalin Marinas wrote: >> > diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h >> > index 5082b30bc2c0..4b5d7b27edaf 100644 >> > --- a/arch/arm64/include/asm/cache.h >> > +++ b/arch/arm64/include/asm/cache.h >> > @@ -18,17 +18,17 @@ >> > >> > #include >> > >> > -#define L1_CACHE_SHIFT 7 >> > +#define L1_CACHE_SHIFT 6 >> > #define L1_CACHE_BYTES (1 << L1_CACHE_SHIFT) >> > >> > /* >> > * Memory returned by kmalloc() may be used for DMA, so we must make >> > - * sure that all such allocations are cache aligned. Otherwise, >> > - * unrelated code may cause parts of the buffer to be read into the >> > - * cache before the transfer is done, causing old data to be seen by >> > - * the CPU. >> > + * sure that all such allocations are aligned to the maximum *known* >> > + * cache line size on ARMv8 systems. Otherwise, unrelated code may cause >> > + * parts of the buffer to be read into the cache before the transfer is >> > + * done, causing old data to be seen by the CPU. >> > */ >> > -#define ARCH_DMA_MINALIGN L1_CACHE_BYTES >> > +#define ARCH_DMA_MINALIGN (128) >> >> Does this actually fix the reported iperf regression? My assumption was >> that ARCH_DMA_MINALIGN is the problem, but I could be wrong. > >I can't tell. But since I haven't seen any better explanation in this >thread yet, I hope that at least someone would try this patch and come >back with numbers. > >For networking, SKB_DATA_ALIGN() uses SMP_CACHE_BYTES (== L1_CACHE_BYTES). >I think (hope) this alignment is not meant for non-coherent DMA, >otherwise using SMP_CACHE_BYTES wouldn't make sense. As I see the problem, may be it is because of fewer number of buffers available because of this alignment requirement. But that should be fixed by making slab alignment a run time thing. I may be totally wrong. We are still coming up with a decent benchmark that shows degradation. > >-- >Catalin