Received: by 2002:a05:6358:45e:b0:b5:b6eb:e1f9 with SMTP id 30csp1294005rwe; Thu, 1 Sep 2022 16:14:43 -0700 (PDT) X-Google-Smtp-Source: AA6agR6IRkmWXEH+o/oQw0Ljg20VDhQ00aRjWED93i7I7gwdNDD67nqzZgcbo0dUJISin39KC57Y X-Received: by 2002:aa7:c956:0:b0:43b:206d:c283 with SMTP id h22-20020aa7c956000000b0043b206dc283mr30516816edt.381.1662074082870; Thu, 01 Sep 2022 16:14:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1662074082; cv=none; d=google.com; s=arc-20160816; b=aghbgz3LjW9dSLVjM2RtehU0PMxKEl1vHh6UbWOo8wPZ34/K3suFYeBuJzmSxTtrox JI6zVMx9VMt153Gt8axx/9VMXxPiQ0bcr+dCgcikFl6lZgD71L5vRYyqs9h41u1Wk+Kc K5Rw23YhpIP03yHkTDhM9FTlij9f+6EmpzX/itXChCAW4XK1JDv+x8R9c0IWd8ctVI7E x7Mb7kC7LxsYj77KCVONbffqKvmEaTP1z2SGY0c11FsWM5oaYhBLMH1CHMK+1XMIUdSm SU5endxfny2vJCdWDDi3BiFVyrweH0bepVvkOTU4/lmV26YzNL0PXHHClAu40u76E5R6 K+KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:dkim-signature:date; bh=HsNTuFQ7XbCclBZlUbN3U5MkNcD4kydlxiKj9ag/mqM=; b=sLDORrcxlr4ELKL9QXVC0lEfl6RF7k3SItITr62BE31riWUm8iouTy1D5LPPLBu4Si JUtPqSAZxjOL46W4Y/D/jgJmMS5zYgXXSzVLzehNRFGSUVDHB/Exy4dFRpWkzoFnzPl+ DUBivOgKg4srUCuUSowx9r5Gi+RjmfcK/sXdDFFQHqMwN+1g64VUa3XCQ6qZTvPUeDuo z0KZpErzvC0hh2abvf9SRNWxvv+lo2u3le2LKbINKPyHu8+z7P0qY9EBBVG+E88DRR6A ToIP/FzNB2HWlksK8ifw9NZiqGaNzgEjFNn5rrJ7/yDyrGz1VYGuvBkfYP+xzkE1ygc5 qbIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oPzrWlMu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hb19-20020a170907161300b00730a1d46e98si438220ejc.734.2022.09.01.16.14.17; Thu, 01 Sep 2022 16:14:42 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux.dev header.s=key1 header.b=oPzrWlMu; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linux.dev Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234801AbiIAWy3 (ORCPT + 99 others); Thu, 1 Sep 2022 18:54:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234373AbiIAWy0 (ORCPT ); Thu, 1 Sep 2022 18:54:26 -0400 Received: from out1.migadu.com (out1.migadu.com [91.121.223.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BDBF275FEE; Thu, 1 Sep 2022 15:54:24 -0700 (PDT) Date: Thu, 1 Sep 2022 15:53:57 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1662072863; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=HsNTuFQ7XbCclBZlUbN3U5MkNcD4kydlxiKj9ag/mqM=; b=oPzrWlMu1zttBKXqMooMVKLQBDEeBWlyjX9vsDJkAXG+i91talZG3CnOLGZqJ/j7NVGRKB wZTWLVRasLoef5Won/PgGguSqmh0FC7JUNFBw6OgxkY16iM6YRuGaquB3T/TIBb4G8gtzw fhM8xA0vhhBp50IAflanG0eJ5LKzxK0= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Kent Overstreet Cc: Yosry Ahmed , Michal Hocko , Mel Gorman , Peter Zijlstra , Suren Baghdasaryan , Andrew Morton , Vlastimil Babka , Johannes Weiner , dave@stgolabs.net, Matthew Wilcox , liam.howlett@oracle.com, void@manifault.com, juri.lelli@redhat.com, ldufour@linux.ibm.com, Peter Xu , David Hildenbrand , axboe@kernel.dk, mcgrof@kernel.org, masahiroy@kernel.org, nathan@kernel.org, changbin.du@intel.com, ytcoode@gmail.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, Steven Rostedt , bsegall@google.com, bristot@redhat.com, vschneid@redhat.com, Christoph Lameter , Pekka Enberg , Joonsoo Kim , 42.hyeyoo@gmail.com, glider@google.com, elver@google.com, dvyukov@google.com, Shakeel Butt , Muchun Song , arnd@arndb.de, jbaron@akamai.com, David Rientjes , minchan@google.com, kaleshsingh@google.com, kernel-team@android.com, Linux-MM , iommu@lists.linux.dev, kasan-dev@googlegroups.com, io-uring@vger.kernel.org, linux-arch@vger.kernel.org, xen-devel@lists.xenproject.org, linux-bcache@vger.kernel.org, linux-modules@vger.kernel.org, Linux Kernel Mailing List Subject: Re: [RFC PATCH 00/30] Code tagging framework and applications Message-ID: References: <20220830214919.53220-1-surenb@google.com> <20220831084230.3ti3vitrzhzsu3fs@moria.home.lan> <20220831101948.f3etturccmp5ovkl@suse.de> <20220831190154.qdlsxfamans3ya5j@moria.home.lan> <20220901223720.e4gudprscjtwltif@moria.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220901223720.e4gudprscjtwltif@moria.home.lan> X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 01, 2022 at 06:37:20PM -0400, Kent Overstreet wrote: > On Thu, Sep 01, 2022 at 03:27:27PM -0700, Roman Gushchin wrote: > > On Wed, Aug 31, 2022 at 01:56:08PM -0700, Yosry Ahmed wrote: > > > This is very interesting work! Do you have any data about the overhead > > > this introduces, especially in a production environment? I am > > > especially interested in memory allocations tracking and detecting > > > leaks. > > > > +1 > > > > I think the question whether it indeed can be always turned on in the production > > or not is the main one. If not, the advantage over ftrace/bpf/... is not that > > obvious. Otherwise it will be indeed a VERY useful thing. > > Low enough overhead to run in production was my primary design goal. > > Stats are kept in a struct that's defined at the callsite. So this adds _no_ > pointer chasing to the allocation path, unless we've switch to percpu counters > at that callsite (see the lazy percpu counters patch), where we need to deref > one percpu pointer to save an atomic. > > Then we need to stash a pointer to the alloc_tag, so that kfree() can find it. > For slab allocations this uses the same storage area as memcg, so for > allocations that are using that we won't be touching any additional cachelines. > (I wanted the pointer to the alloc_tag to be stored inline with the allocation, > but that would've caused alignment difficulties). > > Then there's a pointer deref introduced to the kfree() path, to get back to the > original alloc_tag and subtract the allocation from that callsite. That one > won't be free, and with percpu counters we've got another dependent load too - > hmm, it might be worth benchmarking with just atomics, skipping the percpu > counters. > > So the overhead won't be zero, I expect it'll show up in some synthetic > benchmarks, but yes I do definitely expect this to be worth enabling in > production in many scenarios. I'm somewhat sceptical, but I usually am. And in this case I'll be really happy to be wrong. On a bright side, maybe most of the overhead will come from few allocations, so an option to explicitly exclude them will do the trick. I'd suggest to run something like iperf on a fast hardware. And maybe some io_uring stuff too. These are two places which were historically most sensitive to the (kernel) memory accounting speed. Thanks!