Received: by 2002:a05:6a10:1287:0:0:0:0 with SMTP id d7csp2384759pxv; Sat, 24 Jul 2021 14:48:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJztF+42YXiTAlt809RuHJFzMiBO9WSuf60eBlq8uZ8vgkcqHXLImcb25eUiI6b/+xAYgejb X-Received: by 2002:aa7:c706:: with SMTP id i6mr12847138edq.191.1627163281736; Sat, 24 Jul 2021 14:48:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627163281; cv=none; d=google.com; s=arc-20160816; b=FbgbHmXvv9bcJtK8GuFEntxS0R3SGEyFCTVHItdnpLSlp74ksGzCY/v4jkFxKoGFl5 KkRUvKLWOrxQIjG7+wS+AsAwEhnbl1jfHsViqNFa+z/sJvtQJt4OX/uwyPb6Oho/3p8/ lC3qQFIYyLtBXyQe3Pnn3EJD55fPQshBEp42zPYdLkaocPvxR9jz5sa2sh2cvKwtXbrF cayNh3sFmwyFNTIimVl0AKmSGQUzD0X6Nc64Iyf7T7zsBfQLYflHuS9zoByKprhmjIa+ DNEvuuewxB93TedjzvzBA7bSRtGSvum9hOlxx07BYCk0VpRJPMmsj2AW3hjHtgeeKFQt sx7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature :dkim-signature; bh=3V7FevWb1p5n5Vec3/jBp3sIx0KQg89281MvPR3XSBw=; b=KYOpAkacNS1ARIJb8HYqVLllPzXlDnU9bA3QHaLJu07uAesYzlvi4ugo6eBp9kIsbn XA+4ZblbHodG7pFBX2/hYvMA05VQapWuZPJXdh9FZbk4dRBjXwDWTR5ip003ZkQlbHn7 oY5E++9LVfVYBBlnfMykjwDINF8KVBSV8CSrCc7qmKoMvI9QbXvz6PBNERPW5Xw1znAD 5volGxMirMLkPg/3R6s7s6WI4QKl1D6YfbDsY+MKdPLlRToh7pamI4E+s8f7kSSqBQ5W Hgti1uyR17LHEuHCRi9jl1YF9XKETaKlw90xnps/DGOEOX8Jb9mRSvmHs8JUVX25yGWX uvgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@anarazel.de header.s=fm2 header.b=MhmoEUW9; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ChNj0gcB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h2si36281856ejx.355.2021.07.24.14.47.38; Sat, 24 Jul 2021 14:48:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@anarazel.de header.s=fm2 header.b=MhmoEUW9; dkim=pass header.i=@messagingengine.com header.s=fm3 header.b=ChNj0gcB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229866AbhGXVDr (ORCPT + 99 others); Sat, 24 Jul 2021 17:03:47 -0400 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:37751 "EHLO wout2-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229510AbhGXVDq (ORCPT ); Sat, 24 Jul 2021 17:03:46 -0400 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 2EBEF320024A; Sat, 24 Jul 2021 17:44:17 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Sat, 24 Jul 2021 17:44:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=anarazel.de; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm2; bh=3V7FevWb1p5n5Vec3/jBp3sIx0K Qg89281MvPR3XSBw=; b=MhmoEUW9N5gQrmySruuN/8u/+OjCR8a1gNKZ/JLHgTD VVIXWTiXhsHcTYjCJ1nJCsTjeVcmXWxau0BiD498s/6ytid/QtblgtDKrR5HJz6e BLwbh/FXOMWObfcQNLgOao1QRGh5WIHEayLnP4NrwWVG5xlWopZAx5Mq7PWvysH5 AkqQ3nNN0/GI6fszm2OiFLLj3sOIQt0FkZ0tdCJpgEvuxlgxXd2pfR77+qrYCD25 cNMNZJSF25q4UMqFG/5ZiwxM87F9J6t95ATeoSOcYcph2tkxTUTV9ip1JF63XD09 ZPXGtlh5ib7eZxUUGqoUmzUqBxBcjQLYnYsWAGQpLMw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=3V7Fev Wb1p5n5Vec3/jBp3sIx0KQg89281MvPR3XSBw=; b=ChNj0gcB73QLDqaNGZC61C QJ3uy13y/8L5kegrtWxbbVjo6fte5IN4bDiB4KAP4cQoWBegXRRDUVksBs6oKOZM YLaEftfiIZ2OYfCrmQRA227N8yR/pqbzsFZZlnB7lCYTxxXB6bBlS3MHmKQ0gc8y o9NeBdlIY278Xm4GCiDsJhLBDudGoB7Z84C/Kdl7CWQAOEshUuZnQyzZmuYhToDT FAYg0NePFPh32+WJycFQ8QdHmeFfgwBoBb22QfcENgoUiFPH69cvVo8wyqKpVLpN CrEN2PwUELB8364UFcbc2dMp0s8Y+PzK3o0u9xxGxAi3hVoJsK/8z4zRr4FjUMwQ == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrgedugddtvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpeffhffvuffkfhggtggujgesthdtredttddtvdenucfhrhhomheptehnughrvghs ucfhrhgvuhhnugcuoegrnhgurhgvshesrghnrghrrgiivghlrdguvgeqnecuggftrfgrth htvghrnhepudeuvefhudevtefhgeehveetjedutdekfeejudehkedttdeuueeikeekteev tedunecuffhomhgrihhnpehgihhthhhusgdrtghomhdpihhnshhtrghllhdrshhhnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheprghnughrvghs segrnhgrrhgriigvlhdruggv X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 24 Jul 2021 17:44:15 -0400 (EDT) Date: Sat, 24 Jul 2021 14:44:13 -0700 From: Andres Freund To: Matthew Wilcox Cc: James Bottomley , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Linus Torvalds , Andrew Morton , "Darrick J. Wong" , Christoph Hellwig , Michael Larabel Subject: Re: Folios give an 80% performance win Message-ID: <20210724214413.fqsbjxhhodfzchs6@alap3.anarazel.de> References: <20210715033704.692967-1-willy@infradead.org> <1e48f7edcb6d9a67e8b78823660939007e14bae1.camel@HansenPartnership.com> <17a9d8bf-cd52-4e6c-9b3e-2fbc1e4592d9@www.fastmail.com> <4c634d08-c658-44cf-ac92-92097eeb8532@www.fastmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4c634d08-c658-44cf-ac92-92097eeb8532@www.fastmail.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, On 2021-07-24 12:12:36 -0700, Andres Freund wrote: > On Sat, Jul 24, 2021, at 12:01, Matthew Wilcox wrote: > > On Sat, Jul 24, 2021 at 11:45:26AM -0700, Andres Freund wrote: > > It's always possible I just broke something. The xfstests aren't > > exhaustive, and no regressions doesn't mean no problems. > > > > Can you guide Michael towards parameters for pgbench that might give > > an indication of performance on a more realistic workload that doesn't > > entirely fit in memory? > > Fitting in memory isn't bad - that's a large post of real workloads. It just makes it hard to believe the performance improvement, given that we expect to be bound by disk sync speed... I just tried to compare folio-14 vs its baseline, testing commit 8096acd7442e against 480552d0322d. In a VM however (but at least with its memory being backed by huge pages and storage being passed through). I got about 7% improvement with just some baseline tuning of postgres applied. I think a 1-2% of that is potentially runtime variance (I saw slightly different timings leading around checkpointing that lead to a bit "unfair" advantage to the folio run). That's a *nice* win! WRT the ~70% improvement: > Michael, where do I find more details about the codification used during the > run? After some digging I found https://github.com/phoronix-test-suite/phoronix-test-suite/blob/94562dd4a808637be526b639d220c7cd937e2aa1/ob-cache/test-profiles/pts/pgbench-1.10.1/install.sh For one the test says its done on ext4, while I used xfs. But I think the bigger thing is the following: The phoronix test uses postgres with only one relevant setting adjusted (increasing the max connection count). That will end up using a buffer pool of 128MB, no huge pages, and importantly is configured to aim for not more than 1GB for postgres' journal, which will lead to constant checkpointing. The test also only runs for 15 seconds, which likely isn't even enough to "warm up" (the creation of the data set here will take longer than the run). Given that the dataset phoronix is using is about ~16GB of data (excluding WAL), and uses 256 concurrent clients running full tilt, using that limited postgres settings doesn't end up measuring something particularly interesting in my opinion. Without changing the filesystem, using a configuration more similar to phoronix', I do get a bigger win. But the run-to-run variance is so high (largely due to the short test duration) that I don't trust those results much. It does look like there's a less slowdown due to checkpoints (i.e. fsyncing all data files postgres modified since the last checkpoints) on the folio branch, which does make some sense to me and would be a welcome improvement. Greetings, Andres Freund