Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp1010584pxb; Fri, 22 Jan 2021 05:11:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJxvqlAJPswO9JuSC92HN9KjsABW7w1e2RoTQXigngtcqqKsSp5mlm6l9/kqxZGC+k7OITNc X-Received: by 2002:a17:906:e107:: with SMTP id gj7mr2758015ejb.298.1611321069756; Fri, 22 Jan 2021 05:11:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611321069; cv=none; d=google.com; s=arc-20160816; b=I6lxu6CKM/7Ewj4J0SJ9w6nqJHLfa2UcqmI2EpkOXEqgKuBXN+QQJDjg2IQPwT9rQO 6TiXbuJBL+CaiPY7dIqMrHD0idXbMXtdotXaBIUlyC+0LD2hDfa43kB5/1/9rqSP5u0C yx7t9Ty7pBODq+SSTJdTlIV+1Cn3S1xdGI1oR1N2Zk1NM/VG4NEgYEAM+oIyBTsAmfnY AYhTvzy3Lo/JnQyaL981zmckTZ0f8WWXvSfOq+dUBP7bfgfv3nVes7yGG4tEN/c/IRzA JjXila2jFjhB+UMoKmLgztg4kJ8TImcjc3up2ceUFS7TUCkvudhngAnA7En8A59yF0wI OcFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=hdjcX1r0hcP0ziPjwGYoacie9+kzbK27+8zHrMy8Ek0=; b=QGhfBso4J9XlPEVMbMzGyFrQBkaQWXnqnDcXlIc7pJN9wuaLknxecRBfkRzdX537vb /KUOWde9oJKFw5jjulCCYaaS4d6JcB2s/lJR0yGQXCZlws7ZaXY0udLtCQM6CQ3V+mkI WoHIPDoKzHpRfEPMkkeqO9wHiX8wPbU+nMuPtZQVarJRs0XKOB1ZicpC0w1W+3cWcqvD i/vPt3c2LyuRt+pYASvke6lcpuYWhrE2IylDT+QahKZn1Klfv1fUgNM3ZTWE5BKJp6Ah PJkm5FCCJj7oEfFeuplYnwrxJnZzmg3fxbSuwKQqcm2dII5Z0oQAUz4cyohgxpctsA+a 0+rQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uy7LbRH2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id qh15si2929920ejb.612.2021.01.22.05.10.45; Fri, 22 Jan 2021 05:11:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=uy7LbRH2; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727931AbhAVNIy (ORCPT + 99 others); Fri, 22 Jan 2021 08:08:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34274 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727928AbhAVNHI (ORCPT ); Fri, 22 Jan 2021 08:07:08 -0500 Received: from mail-lf1-x132.google.com (mail-lf1-x132.google.com [IPv6:2a00:1450:4864:20::132]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2E90C06174A for ; Fri, 22 Jan 2021 05:06:15 -0800 (PST) Received: by mail-lf1-x132.google.com with SMTP id m22so7424663lfg.5 for ; Fri, 22 Jan 2021 05:06:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hdjcX1r0hcP0ziPjwGYoacie9+kzbK27+8zHrMy8Ek0=; b=uy7LbRH2a+1REAcuKTsefeoWzLSAk8lD9cg8t0qFzoBoRZ8P9cscIE2Ez8S826Ssbj oMgNkvLij+BJWNVphXcP2grz7sz5MmhvWX8s8QapQ7Zp7THp4qoDs9RX7XZJsf2PcsY4 uW508nwG14h2e9L+j5tziHYVrWo/G3+6vzgcxNylnWYTLG5kK/XaXj1ogJam7olKIa6w hJh/17mcPKTgLSeAog8TjZgFNuH1x1j0SVpt2z5DnL94M6kimZfip4lMPLYWsWKr+ZtP OhaJjeHvl0Ka0pUYtjLcUAAZ00B+wAXwK28VR9FJ+prxtNRVwZLA/fbeaJWV4sBp6Jup LlNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hdjcX1r0hcP0ziPjwGYoacie9+kzbK27+8zHrMy8Ek0=; b=O3663rKm/JtmrvuLOLxi7ku4DKnRbqs7HzEFwg+TTYbN7OLibnQ2gYpV9QBMHs5kiS snshUuA+SFDMtqE1Df2Tewvyq/npUAzCEMuLr2K821eNrv/d9Qx7W0QWO75b9x/d0ZFs LcjqdVS45kLRcMmowKJqYdb2d/mYQ9UEAD4c4bwRvUP3R6HRoXZ6krmJd3jyIsy7tfMU Sx0V84Bezi5MP9wFwa1wQFx+jgmNNQ588vUxTpqOx+BuBU6P4I6PZjz3v6uW7nDuppgt krgKVIk6BVm+QwrLn8Fulc35yRTkl9H+CX7FEThjBjd3Unz0gAkq9ULUrM+WuGTlBvBf F0nA== X-Gm-Message-State: AOAM530WTIZ0KGUbR4oCLM2elIxF9e5QIM3Gn5KOYKbSgwIIHr3oM+dq M9OGflXGLGjEI2OxdTS6haXcxMXQA7bE4puBLaRWk8cZrpkYdw== X-Received: by 2002:a19:197:: with SMTP id 145mr123836lfb.352.1611320774090; Fri, 22 Jan 2021 05:06:14 -0800 (PST) MIME-Version: 1.0 References: <20201118082759.1413056-1-bharata@linux.ibm.com> <20210121053003.GB2587010@in.ibm.com> In-Reply-To: From: Jann Horn Date: Fri, 22 Jan 2021 14:05:47 +0100 Message-ID: Subject: Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order To: Vlastimil Babka Cc: Christoph Lameter , Bharata B Rao , Vincent Guittot , linux-kernel , Linux-MM , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Shakeel Butt , Johannes Weiner , aneesh.kumar@linux.ibm.com, Michal Hocko Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jan 21, 2021 at 7:19 PM Vlastimil Babka wrote: > On 1/21/21 11:01 AM, Christoph Lameter wrote: > > On Thu, 21 Jan 2021, Bharata B Rao wrote: > > > >> > The problem is that calculate_order() is called a number of times > >> > before secondaries CPUs are booted and it returns 1 instead of 224. > >> > This makes the use of num_online_cpus() irrelevant for those cases > >> > > >> > After adding in my command line "slub_min_objects=36" which equals to > >> > 4 * (fls(num_online_cpus()) + 1) with a correct num_online_cpus == 224 > >> > , the regression diseapears: > >> > > >> > 9 iterations of hackbench -l 16000 -g 16: 3.201sec (+/- 0.90%) > > I'm surprised that hackbench is that sensitive to slab performance, anyway. It's > supposed to be a scheduler benchmark? What exactly is going on? Uuuh, I think powerpc doesn't have cmpxchg_double? "vgrep cmpxchg_double arch/" just spits out arm64, s390 and x86? And says under "POWERPC": "no DW LL/SC" So powerpc is probably hitting the page-bitlock-based implementation all the time for stuff like __slub_free()? Do you have detailed profiling results from "perf top" or something like that? (I actually have some WIP patches and a design document for getting rid of cmpxchg_double in struct page that I hacked together in the last couple days; I'm currently in the process of sending them over to some other folks in the company who hopefully have cycles to review/polish/benchmark them so that they can be upstreamed, assuming that those folks think they're important enough. I don't have the cycles for it...)