Received: by 2002:a05:6a10:1287:0:0:0:0 with SMTP id d7csp2326138pxv; Sat, 24 Jul 2021 12:23:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyZoiYtMaIQ8PZvWFHi2DHpBJhgPHYPOUbvrJp6IGdMtJ1IRXAPlTeLAfvBpnx8ukXCHO7j X-Received: by 2002:a05:6402:2283:: with SMTP id cw3mr4369729edb.87.1627154594363; Sat, 24 Jul 2021 12:23:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627154594; cv=none; d=google.com; s=arc-20160816; b=lhkdyP62Mj1oq80E0yJlt3Jj5gqG8JBIjpVkwbnws54sWNwnYOy6fBzuSn2FF6c3e9 buHtrf6MLz+cI9IZTczMTS7UOSMYyOJO80KlRrlaRax1rfV5oj25yCDkRReLhE+pxo3A Z4GnqCzHXB9it4BBrxOS2nMnpXlN6v8E0kKGY2b8abXoIhidvT+r34toaQ97G2XAyxep v5NVEeZZ/vy2xyotD+AjP8cuJjHJQTptiywmgQdsogc9y8nSMYk0bqbId1EtpvOKcuIe YuWwzeu1c9yrngFuM8wetpoZutEn6o6xx4VEZ2FMnZRaUdUC+Y4mq6qEqSicIt8f+kZ8 fvqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:date:cc:to:from:subject :message-id:dkim-signature:dkim-signature; bh=GGCQdnqzrZux0IUKpKpqmH1lvjjXkBGvDnHRmuxL3gw=; b=IHPUt8V5JSPh0mb7cxYNU2Qn1TGiX1noi/chlnL0xg7Hdor0U/fMoVzS+zv5y5WDww cdNXaK7QWeeT+egRg2OUyru43mgANNLIorLElhKwidVxH6XEPHTLr/Iz9Jksfs51bseQ MFR/I6P9IhGHeP+E4lCbrxhEcCNZ5QNE1hmaB40Lt5rLm8mqgNBVs5PGBQeYqM2w8X7H KsnRSWegcUKEci3KmYfEbocH1KriCeVsEUbm6aD5UMOVkBs1uFyukSs9oIygj4WY6+PV icL8j+xcIgnNMSJu2Gp6XedGjHRWQuDExSF1ApJNgWPLXnLPWdxu9OCc3u108IvtZg0N SokA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@hansenpartnership.com header.s=20151216 header.b=EsGCwytk; dkim=pass header.i=@hansenpartnership.com header.s=20151216 header.b=UoAYYAcL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=hansenpartnership.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id r17si35712173edv.508.2021.07.24.12.22.49; Sat, 24 Jul 2021 12:23:14 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@hansenpartnership.com header.s=20151216 header.b=EsGCwytk; dkim=pass header.i=@hansenpartnership.com header.s=20151216 header.b=UoAYYAcL; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=hansenpartnership.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229588AbhGXSkr (ORCPT + 99 others); Sat, 24 Jul 2021 14:40:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229476AbhGXSkq (ORCPT ); Sat, 24 Jul 2021 14:40:46 -0400 Received: from bedivere.hansenpartnership.com (bedivere.hansenpartnership.com [IPv6:2607:fcd0:100:8a00::2]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E9FFC061575; Sat, 24 Jul 2021 12:21:18 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by bedivere.hansenpartnership.com (Postfix) with ESMTP id 251C81280541; Sat, 24 Jul 2021 12:21:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hansenpartnership.com; s=20151216; t=1627154478; bh=RwmyDLi8Otc7MLTn5MLWY8ZIvHMRyae2630SrNyNvd4=; h=Message-ID:Subject:From:To:Date:In-Reply-To:References:From; b=EsGCwytkbLX6pUETj7D3+5M3ugtzjkH76d1GnEPLwRl8a7rSwC6FxNxqdQJ2eQqYf rzf5sfyyGcl/n/0aqXBBp2IZZNQr8EErbWOf9SvKVUcjzYqpJNkWOFqmYvEvA9yxJ2 Oadd9mn5PzfgwKOpbJDL/a6gedSqdr+XH7AoOFAA= Received: from bedivere.hansenpartnership.com ([127.0.0.1]) by localhost (bedivere.hansenpartnership.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F2DEa1vXkPjj; Sat, 24 Jul 2021 12:21:18 -0700 (PDT) Received: from jarvis.int.hansenpartnership.com (unknown [IPv6:2601:600:8280:66d1::527]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bedivere.hansenpartnership.com (Postfix) with ESMTPSA id 798DF1280534; Sat, 24 Jul 2021 12:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=hansenpartnership.com; s=20151216; t=1627154477; bh=RwmyDLi8Otc7MLTn5MLWY8ZIvHMRyae2630SrNyNvd4=; h=Message-ID:Subject:From:To:Date:In-Reply-To:References:From; b=UoAYYAcLzUOuZJV9op1vWS6kU+0PyuMTg++SW4Jb/zU4jOJTzWv7u0x6DH8acKDrz 6jhc4biqzwPMeX1IfwKyMNVumkwu3DHv8bhFmInjqG1t5sQmnsT0dVu/EYS4CxYjUn WEU/WBB19lmhYdkyvX6nVEv88/ZJiSMTfhD6BZRE= Message-ID: Subject: Re: Folios give an 80% performance win From: James Bottomley To: Matthew Wilcox Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Linus Torvalds , Andrew Morton , "Darrick J. Wong" , Christoph Hellwig , Andres Freund , Michael Larabel Date: Sat, 24 Jul 2021 12:21:16 -0700 In-Reply-To: References: <20210715033704.692967-1-willy@infradead.org> <1e48f7edcb6d9a67e8b78823660939007e14bae1.camel@HansenPartnership.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.4 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, 2021-07-24 at 19:50 +0100, Matthew Wilcox wrote: > On Sat, Jul 24, 2021 at 11:23:25AM -0700, James Bottomley wrote: > > On Sat, 2021-07-24 at 19:14 +0100, Matthew Wilcox wrote: > > > On Sat, Jul 24, 2021 at 11:09:02AM -0700, James Bottomley wrote: > > > > On Sat, 2021-07-24 at 18:27 +0100, Matthew Wilcox wrote: > > > > > What blows me away is the 80% performance improvement for > > > > > PostgreSQL. I know they use the page cache extensively, so > > > > > it's > > > > > plausibly real. I'm a bit surprised that it has such good > > > > > locality, and the size of the win far exceeds my > > > > > expectations. We should probably dive into it and figure out > > > > > exactly what's going on. > > > > > > > > Since none of the other tested databases showed more than a 3% > > > > improvement, this looks like an anomalous result specific to > > > > something in postgres ... although the next biggest db: mariadb > > > > wasn't part of the tests so I'm not sure that's > > > > definitive. Perhaps the next step should be to t > > > > est mariadb? Since they're fairly similar in domain (both full > > > > SQL) if mariadb shows this type of improvement, you can > > > > safely assume it's something in the way SQL databases handle > > > > paging and if it doesn't, it's likely fixing a postgres > > > > inefficiency. > > > > > > I think the thing that's specific to PostgreSQL is that it's a > > > heavy user of the page cache. My understanding is that most > > > databases use direct IO and manage their own page cache, while > > > PostgreSQL trusts the kernel to get it right. > > > > That's testable with mariadb, at least for the innodb engine since > > the flush_method is settable. > > We're still not communicating well. I'm not talking about writes, > I'm talking about reads. Postgres uses the page cache for reads. > InnoDB uses O_DIRECT (afaict). See articles like this one: > https://www.percona.com/blog/2018/02/08/fsync-performance-storage-devices/ If it were all about reads, wouldn't the Phoronix pgbench read only test have shown a better improvement than 7%? I think the Phoronix data shows that whatever it is it's to do with writes ... that does imply something in the way the log syncs data. James