Received-SPF: pass (google.com: domain of linux-kernel+bounces-17093-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Thu, 4 Jan 2024 10:29:03 -0800
From: Dan Williams <dan.j.williams@intel.com>
To: Lukas Wunner <lukas@wunner.de>, Peter Zijlstra <peterz@infradead.org>
CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>, Dan Williams
	<dan.j.williams@intel.com>, <linux-coco@lists.linux.dev>, Andrew Morton
	<akpm@linux-foundation.org>, <x86@kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2 4/5] mm/slab: Add __free() support for kvfree
Message-ID: <6596f8ef5fafe_8dc682949e@dwillia2-xfh.jf.intel.com.notmuch>
References: <169199898909.1782217.10899362240465838600.stgit@dwillia2-xfh.jf.intel.com>
 <169199901230.1782217.9803098171993981037.stgit@dwillia2-xfh.jf.intel.com>
 <2023081449-blurry-bath-248e@gregkh>
 <20230814161731.GN776869@hirez.programming.kicks-ass.net>
 <20240104065744.GA6055@wunner.de>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240104065744.GA6055@wunner.de>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?V+Sa7fZQczkAzlDeiwiyJNryI6EFSuvRvrF1ipWMzda1enRYIu4E5I/sHZY/?=
 =?us-ascii?Q?AaLqyDGMyuFYRsMgooAcctZT7rEnYtmvK+hUcYeMFdHlNIYsDdPjvxvDYFkm?=
 =?us-ascii?Q?EShZjv8xrk/wQSJPxYSvxFyY7oGtj33zW+rEErBlGw9fYRCbJlMdd2FSL0xE?=
 =?us-ascii?Q?BB9xhqJB3E2lFCaXrPPYO13PFv70xxhBTiZ5lNdlmwDri4QGPYJmtou7do53?=
 =?us-ascii?Q?Aus6O2bMwmrfvMW5uJJHANRwcgfeaVZnW94xpYd6QMwpdEmIhctygYH93clz?=
 =?us-ascii?Q?arhWXI6gnTaQfsRR/qrDNZE2pkoeJJukvXaaMY9V0RWa+9a+ejo1T4mXSgz8?=
 =?us-ascii?Q?l8xQgQwRh2FPDN0LarxwPz5dODoFnj7felMFwAHCk2mKgFdWHBy9OI8n6IhM?=
 =?us-ascii?Q?mHGNoGKTsB7T6AB+rizfLem8fkuTWhGjNyzjrehmKXpNIOsrGULnmxuDYHqi?=
 =?us-ascii?Q?W75JsPmHYCU07mahQIdLh0MncHMBSDjgV+k8l3Nxxt3ixeUZa48cB6PkdbhZ?=
 =?us-ascii?Q?MlYCr9+LYwCDTfbg/p8i0jb/LwDzZ9duC03b0w1ZR8N1Jp16rHlKjzEZZD4d?=
 =?us-ascii?Q?ijHaW31SbKa22sQtXjeKo5XTOM2wdNk87h23W7fU9epQ/eEyQ40X3ucFJQSJ?=
 =?us-ascii?Q?4DeaLYxXaRwDpIxEaY9Xsykub4agf8EgrtA9iavyKQLEMJCaJLDy/JpE/vM1?=
 =?us-ascii?Q?ROk6Sy4204OjiYCNNOk7xREJmZ9/V9txZAM2s1V5kJqIUsuYf3TLWqn+il6X?=
 =?us-ascii?Q?EV3HJ6RcFzAp8EKQlQXepbbg3o+yEWJe/HMOKD22l3ird5HXiWVabd4BzbrU?=
 =?us-ascii?Q?dwg8T4r0x2bWwf7id6Fo5XcWk0U5qaDcYW/WO3wU/qdqqLGSV0x7+HW+agK4?=
 =?us-ascii?Q?1/G3A1K0RoB79fXLipBJCH2aFWSc3D/q7L+j8mrMcJKHsYaW+NHADt8DcUrd?=
 =?us-ascii?Q?y4Q24bbh0SF3DfdxktkzKlVttmrrOgwqBBQM+dKeycbd6Nlz9+v18XmueG5S?=
 =?us-ascii?Q?WERcDDLE3L3zRppmU7J9UxEwSDq+T4HzRQOVg3lCMChJL/0YOBrL92Nne2JI?=
 =?us-ascii?Q?WakKlG8Ev/ibb5Q5wMA2G8YL++Ms599Oi1GYoBtwHlHs3Jwtp4yBelrUeErL?=
 =?us-ascii?Q?XFl14KJldWgqmBdJ+fivAJrUOG+FAlQFWofRb3vmjE4Dp7MppvbDPjY9bMYx?=
 =?us-ascii?Q?kB+AOE9AMpr3Ij4UCNGRQIdkAeugBD0bKFoYiHF0dThAFAFxYYdUZtdvxTTE?=
 =?us-ascii?Q?bkwQbx0CrNdjl5DWisWdd+aG8BgIM8+43aOe1ETAO9RodXpcNb7FNZr9dCXp?=
 =?us-ascii?Q?qQr8BGgLW432RYwYnrubPRDneK+3fR2QfUmTbgMN4cJeVsLP/Nq0Q3SDpSOE?=
 =?us-ascii?Q?LdulqY3WaVqV3utHKCCz+EwUwRIecuyeZGn0ScgavSvuni2BnELfDXk4htK4?=
 =?us-ascii?Q?YRQpdXIf11p3fWy0UiiV0CrrguqmnUGR1mlUfJM9y642oWNsqHOhgsxrAh0X?=
 =?us-ascii?Q?JsnacAlerF2CEih5EtBgNogbXW3iiPfJ8wzJZB/N0lS+UIS50UTaeDI3PwLK?=
 =?us-ascii?Q?kqKsyemgOpUgYUSh2Uu8s9ws1uO6k8ypObP4ZO1WEW2TjTzs3qcBO5W2OT8T?=
 =?us-ascii?Q?KQ=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: d466f859-cf7d-4ba5-4b3c-08dc0d5308d4
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jan 2024 18:29:06.2928
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ZwKKQ412J5vzNVahjPO28kLQmvXbAktQGnr1ZhswAWV4rO/c/GEWDcKD7+IysDjzrt33WT5K+4TauRjSuJYtBKgfmHkpCLby2Lf5c/zipKU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB5773
X-OriginatorOrg: intel.com

Lukas Wunner wrote:
> On Mon, Aug 14, 2023 at 06:17:31PM +0200, Peter Zijlstra wrote:
> > On Mon, Aug 14, 2023 at 05:31:27PM +0200, Greg Kroah-Hartman wrote:
> > > On Mon, Aug 14, 2023 at 12:43:32AM -0700, Dan Williams wrote:
> > > > Allow for the declaration of variables that trigger kvfree() when they
> > > > go out of scope.
> > > > 
> > > > Cc: Andrew Morton <akpm@linux-foundation.org>
> > > > Cc: Peter Zijlstra <peterz@infradead.org>
> > > > Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> > > > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> > > > ---
> > > >  include/linux/slab.h |    2 ++
> > > >  1 file changed, 2 insertions(+)
> > > > 
> > > > diff --git a/include/linux/slab.h b/include/linux/slab.h
> > > > index 848c7c82ad5a..241025367943 100644
> > > > --- a/include/linux/slab.h
> > > > +++ b/include/linux/slab.h
> > > > @@ -746,6 +746,8 @@ static inline __alloc_size(1, 2) void *kvcalloc(size_t n, size_t size, gfp_t fla
> > > >  extern void *kvrealloc(const void *p, size_t oldsize, size_t newsize, gfp_t flags)
> > > >  		      __realloc_size(3);
> > > >  extern void kvfree(const void *addr);
> > > > +DEFINE_FREE(kvfree, void *, if (_T) kvfree(_T))
> > > 
> > > No need to check _T before calling this, right (as was also pointed out
> > > earlier).
> > 
> > Well, that does mean you get an unconditional call to kvfree() in the
> > success case. Linus argued against this.
> > 
> > This way the compiler sees:
> > 
> > 	buf = NULL;
> > 	if (buf)
> > 		kvfree(buf);
> > 
> > and goes: 'let me clean that up for you'. And all is well.
> 
> Have you actually verified that assumption in the generated Assembler code?
> 
> The kernel is compiled with -fno-delete-null-pointer-checks since commit
> a3ca86aea507 ("Add '-fno-delete-null-pointer-checks' to gcc CFLAGS").
> 
> So NULL pointer checks are *not* optimized away even if the compiler
> knows that a pointer is NULL.

Interesting, I am not sure how -fno-delete-null-pointer-checks plays
into this, but I can confirm that Peter's expectations are being met in
a routine with:

DEFINE_FREE(pci_dev_put, struct pci_dev *, if (_T) pci_dev_put(_T))

...without that conditional the assembly is:

   0xffffffff819ad129 <+41>:	call   0xffffffff81800840 <pci_get_domain_bus_and_slot>
   0xffffffff819ad12e <+46>:	mov    %rax,%r12
   0xffffffff819ad131 <+49>:	test   %rax,%rax
   0xffffffff819ad134 <+52>:	je     0xffffffff819ad154 <cxl_cper_event_call+84>
   0xffffffff819ad136 <+54>:	mov    %rax,%rdi
   0xffffffff819ad139 <+57>:	call   0xffffffff817f5f10 <pci_dev_lock>
   0xffffffff819ad13e <+62>:	cmpq   $0xffffffff82c681c0,0x80(%r12)
   0xffffffff819ad14a <+74>:	je     0xffffffff819ad160 <cxl_cper_event_call+96>
   0xffffffff819ad14c <+76>:	mov    %r12,%rdi
   0xffffffff819ad14f <+79>:	call   0xffffffff817f5fa0 <pci_dev_unlock>
   0xffffffff819ad154 <+84>:	pop    %rbx
   0xffffffff819ad155 <+85>:	mov    %r12,%rdi
   0xffffffff819ad158 <+88>:	pop    %rbp
   0xffffffff819ad159 <+89>:	pop    %r12
   0xffffffff819ad15b <+91>:	jmp    0xffffffff817fe1e0 <pci_dev_put>

...i.e. the check for NULL at 0xffffffff819ad134 jumps to do an
unnecessary pci_dev_put(). With the conditional in the macro the
sequence is:

   0xffffffff819ad129 <+41>:	call   0xffffffff81800840 <pci_get_domain_bus_and_slot>
   0xffffffff819ad12e <+46>:	test   %rax,%rax
   0xffffffff819ad131 <+49>:	je     0xffffffff819ad18c <cxl_cper_event_call+140>
   0xffffffff819ad133 <+51>:	mov    %rax,%r12
   0xffffffff819ad136 <+54>:	mov    %rax,%rdi
   0xffffffff819ad139 <+57>:	call   0xffffffff817f5f10 <pci_dev_lock>
   0xffffffff819ad13e <+62>:	cmpq   $0xffffffff82c681c0,0x80(%r12)
   0xffffffff819ad14a <+74>:	je     0xffffffff819ad160 <cxl_cper_event_call+96>
   0xffffffff819ad14c <+76>:	mov    %r12,%rdi
   0xffffffff819ad14f <+79>:	call   0xffffffff817f5fa0 <pci_dev_unlock>
   ...
   0xffffffff819ad18c <+140>:	pop    %rbx
   0xffffffff819ad18d <+141>:	pop    %rbp
   0xffffffff819ad18e <+142>:	pop    %r12
   0xffffffff819ad190 <+144>:	jmp    0xffffffff81efc6a0 <__x86_return_thunk>

...i.e. optimize away the pci_dev_put() and return directly when @pdev
is already known to be NULL. So empirically
-fno-delete-null-pointer-checks still allows for redundant NULL checks
to be optimized.