Received: by 2002:a25:8b91:0:0:0:0:0 with SMTP id j17csp1284157ybl; Mon, 2 Dec 2019 00:47:30 -0800 (PST) X-Google-Smtp-Source: APXvYqxtCo16bHcV3RjOO34ZgxBlY//1lM9AlQWMOl8T5t61R664463xTjDJOmksYjuCQzn1P+00 X-Received: by 2002:a50:d0d0:: with SMTP id g16mr55194034edf.75.1575276449942; Mon, 02 Dec 2019 00:47:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1575276449; cv=pass; d=google.com; s=arc-20160816; b=KLRKNSAiJGiwYairWT2U49Q3N3pHs4QqV7xTBLrUyeOriJcAR98nEJDE4iWZT0kxNx N4rEm04vKkMrwyuV7z1y6h52+0hQ+4CzKKs2u0BAWojr67NSPz7dO46Jn8w8ZDhWopQh +k/JRLHB/6eaITxwRoxFhpQWjgYm7YItWrJ50oJo5sD1+03gc8dXXmmVa6MSR2MHv/hO s+8rZ1vtxrpdrN+RW/w3myScw9YmrEnoi6FFcHGXpdEnEN6/OLmFXWATb3Ft/X+gEPyl jih4uhl7CKVBMojrj9APCnyUgWoP8qdLpz2RDYtw5CcY1ZRCBY2WVaxkSeOtiP9IFHy+ 7UzA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:content-transfer-encoding :content-id:content-language:accept-language:in-reply-to:references :message-id:date:thread-index:thread-topic:subject:cc:to:from :dkim-signature; bh=S73SrrV4ml/iFdYstP6labvxSjw9MJNNdi7ic7L1N7Q=; b=L469B/l4H9rxLtHt4oxsBytML7MtfllOeOircEZbAq3EQMD1jAyFz8bB8E2Tptx6ii v04XlTsSgj7mCRLRwBxp2+VMXJShckHcTDe5b5KPtvFO3gynYshgJ9UbUVJxKeyZ5n7Y qCJhJ8R4LJNQKnie3ahTmmaxYqp6jdkvTxdvsrwfdAkHVQrExl19MmOnfiv8XMAesGHz 1tpGXI4ijqcp/n9VIKolFi1E69/c4fvYeK1zOFsW2YzSZUYoKehzt/YHuRZRSo8IAIDC 6Ie+8YV3tLBMJYyXk52oHmYCCNw2t6YLjj2j/szv51f1WCSY1vS4hgPeLal3d5zx++1e tVsA== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@whamcloud.com header.s=selector2 header.b=jYedvnXp; arc=pass (i=1 spf=pass spfdomain=whamcloud.com dkim=pass dkdomain=whamcloud.com dmarc=pass fromdomain=whamcloud.com); spf=pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c1si20106434ejn.250.2019.12.02.00.46.56; Mon, 02 Dec 2019 00:47:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@whamcloud.com header.s=selector2 header.b=jYedvnXp; arc=pass (i=1 spf=pass spfdomain=whamcloud.com dkim=pass dkdomain=whamcloud.com dmarc=pass fromdomain=whamcloud.com); spf=pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726057AbfLBIqv (ORCPT + 99 others); Mon, 2 Dec 2019 03:46:51 -0500 Received: from mail-eopbgr730070.outbound.protection.outlook.com ([40.107.73.70]:13312 "EHLO NAM05-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725977AbfLBIqv (ORCPT ); Mon, 2 Dec 2019 03:46:51 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DV9Ua7uGDjQLrPt/Ix/CtBMUozeOWhB5ba5Yq01w9d9N8wiEil7q0J3LPS0ApgVobEjphSGcsN+ab3vTxYJT1NaLoO0fb9i1EEIxKfFKzyxYOxJpDI/mw6uLe7o0eW90qX174FZRTI+gmRSuIFcXPxy0zB1O+XeHz8v208PM7jITOHH2KYvsxKL7wik6P6bPnp7uqW8WiuZjpzqGlgb6HGzlDgIHg2ZSsBdruDlkQpO9vh9rgOyIoVVpFqTxb3f0b2FG6EEUA/kKK+tfLNH+hpM2ePzN1zq2qWtTaB4PLFYUNqehZNaWPvi85p1iBd3pvrifyXSEqYZJIjS/zqJU8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S73SrrV4ml/iFdYstP6labvxSjw9MJNNdi7ic7L1N7Q=; b=LBNCsAv9mNZQtDW9a72zkjKbbUS8GOw6+0qb8f1gCWj8PERbr46ulV4A5jvNgKSJrLEYJ9JBSqUD3h0hCP4FLYAsgaFJXXSM/Zql5sA5SaPKm7OuFymKFnbix+GcUP80BSrSrhAhEpHQBWSRHDW6Zuu4fLBxxPRHWiY42OfQ6lsygNUqmEmJrWv4kBHALufZdXFMRzpNtxBfA7gbXQjCkdGHWH3hAYFLo4UHRk7OwLYe+n5uAawJ5smPlFP96+z+it28FuuoBT5vGHCXL0yekfd2i8eME8PowfTYdyCUHKIZ6pzd4EBFMcVARIwzGx51m6F43hT/3udMfLtnB0trxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=whamcloud.com; dmarc=pass action=none header.from=whamcloud.com; dkim=pass header.d=whamcloud.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=whamcloud.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S73SrrV4ml/iFdYstP6labvxSjw9MJNNdi7ic7L1N7Q=; b=jYedvnXp2RP/tycYOEXptsgQ9Af3NMs0IwydUExQFtFYZ2prjeEJvSlRU4SSyY0YtFC6PSaIf8eMrSfbM4G3hn9U97/iftwi2L9GgrVcATYyHaguWuhfXD5X6SemAU/uyuLe1WyuuDsgm23Fo8DwP8/Eom7ENUu3KZzhtMtwsvw= Received: from MN2PR19MB2894.namprd19.prod.outlook.com (20.178.254.95) by MN2PR19MB2701.namprd19.prod.outlook.com (20.178.252.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2495.18; Mon, 2 Dec 2019 08:46:47 +0000 Received: from MN2PR19MB2894.namprd19.prod.outlook.com ([fe80::a499:dae8:b1c1:b08e]) by MN2PR19MB2894.namprd19.prod.outlook.com ([fe80::a499:dae8:b1c1:b08e%7]) with mapi id 15.20.2495.014; Mon, 2 Dec 2019 08:46:47 +0000 From: Alex Zhuravlev To: Andreas Dilger CC: Alex Zhuravlev , =?koi8-r?B?4szBx8/EwdLFzsvPIOHS1KPN?= , "linux-ext4@vger.kernel.org" Subject: Re: [RFC] improve malloc for large filesystems Thread-Topic: [RFC] improve malloc for large filesystems Thread-Index: AQHVn45AfmzanBfwskq4XYElVj+Zs6eUXWuAgADXAYCAABhfAIAABgOAgAAHawCAAFpFAIAGviuAgAooSQA= Date: Mon, 2 Dec 2019 08:46:47 +0000 Message-ID: <432DE9DD-34E2-442C-8399-345ECA4B534A@whamcloud.com> References: <8738E8FF-820F-48A5-9150-7FF64219ED42@whamcloud.com> <20191120181353.GG4262@mit.edu> <9E04C147-D878-45CE-8473-EF8C67FE4E86@whamcloud.com> <4EB2303A-01A3-49AC-B713-195126DB621B@gmail.com> <9114E776-B44E-4CA5-BD49-C432A688C24E@whamcloud.com> <43DA6456-AAF9-4225-A79F-CF632AC5241B@gmail.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=azhuravlev@whamcloud.com; x-originating-ip: [128.72.176.24] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3489bb2f-2d97-433f-11fd-08d777042a84 x-ms-traffictypediagnostic: MN2PR19MB2701: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 0239D46DB6 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(376002)(346002)(366004)(39850400004)(396003)(189003)(199004)(26005)(91956017)(76116006)(8936002)(2906002)(71200400001)(71190400001)(33656002)(6116002)(3846002)(478600001)(2616005)(6512007)(5660300002)(36756003)(99286004)(14454004)(305945005)(446003)(76176011)(6506007)(81156014)(66066001)(81166006)(7736002)(6916009)(14444005)(66556008)(66476007)(86362001)(64756008)(66446008)(66946007)(102836004)(11346002)(8676002)(53546011)(256004)(229853002)(186003)(6436002)(25786009)(6486002)(4326008)(316002)(4744005)(54906003)(6246003);DIR:OUT;SFP:1101;SCL:1;SRVR:MN2PR19MB2701;H:MN2PR19MB2894.namprd19.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: whamcloud.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: E6Dop2+5l1esJlJI9CvOnhgk0zB8jboXCODV9pSH7z5A/gEC0UtVpZkUGo/vHtz2hRwiVm0NfwcEy+wQFMDFAfRcltfYxgQTdCP/ZJEPS1K+1u4yF2qFYSrL7yk2UG6/llA0gech0Gyfwf3iUIZ2se9RtueJ02v3UAlySdOFMbhLqRS6ztWK31nHihvWzozIkalotx+YYv/gMftwMpbTWm+855mevn+3E+xDLmdYkPVH6z+linmu1Qp+HNks4SoGqXZySCkY4SuSxpgbvkRP7gvx5UAn5Z+8uOgSmsfuSFFuud1Ub7XZmRU8lFOnaq379xemutwOTJCbAejyt51qG/8GWM37RLU70qY0zkN8RXFZwsm3LTMPHHNojOGuaIvtr/onBw5SYS2BoBYHG/CbBOf8YALyrT4zEeLk2gldVKGm7mu+BDQKbus/ZFSOOKf3 Content-Type: text/plain; charset="koi8-r" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: whamcloud.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3489bb2f-2d97-433f-11fd-08d777042a84 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Dec 2019 08:46:47.1738 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: fP603y+lTtk1HN7DQeC7d39NwQ/UV5G9IGl3HDg7iSaRbLkiD8cNRh/vrdIY7vFNdaaTmQ5YAF6iZ1b4pEDS6g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR19MB2701 Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org > On 26 Nov 2019, at 00:39, Andreas Dilger wrote: >=20 > I think it is important to understand what the actual goal size is at thi= s > point. The filesystems where we are seeing problems are _huge_ (650TiB a= nd > larger) and are relatively full (70% or more) but take tens of minutes to > finish mounting. Lustre does some small writes at mount time, but it sho= uldn't > take so long to find some small allocations for the config log update. >=20 > The filesystems are automatically getting "s_stripe_size =3D 512" from mk= e2fs > (presumably from the underlying RAID), and I _think_ this is causing mbal= loc > to inflate the IO request to 8-16MB prealloc chunks, which would be much > harder to find, and unnecessary for a small allocation. >=20 Yes, I agree. It makes sense to limit group preallocation in cases like thi= s. Thanks, Alex