Received: by 2002:a05:6a10:f3d0:0:0:0:0 with SMTP id a16csp565179pxv; Thu, 8 Jul 2021 08:48:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzQmc2OS7+tEuaSp9VBRxsRjpN/09SPQQUEKv2yp1ZI6GxPa+XoS47I0Zki7q1fniIzXC57 X-Received: by 2002:a05:6e02:ef4:: with SMTP id j20mr22744184ilk.141.1625759338055; Thu, 08 Jul 2021 08:48:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625759338; cv=none; d=google.com; s=arc-20160816; b=fRZkNcOI1azfpn55ZJdySJ/9T7ohEFmUDsPoN/CQOa01RzWYpFq5M3t1tfEabLah7S U0edmaW88kwUeWMlKrw9pKtSVihsCoR8EaAay1rst1UA48+PG98m0MsTQDU/Nrq1oqga s5e0tePcmcfilsbIxRXR3atRk3vAiok+0YLyddK4iPhvD4lMum1jlagFXCPfOYHo8UEg 4zduKoDYZTbvxWBY6rGF7nv1PY8FSz5bYFpkPF7YJoUfWqPJ2EldO+OeN4Kj5mH3bun5 InuQMxL8wU3fkAwVO4aePhK3xApRAqRJ/w/d1oxQBg9GtFiCT3xJlkPf/Jas2BIWOuKl GDdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=WJZpdcS1MW7Mj0T2TQvKXriF3MrXgDJAl+1qoxtsXRg=; b=yE8Dx6OEeieSCO5pCm6kptQ0RafjoqbiBagVZypzptMP/GWpaenfizkE3ifaB1oBo9 gAfHfG6PF3ZIfCvfqcrfviouhEU9b3hKBXqg0+D8IrOL0YwBLvEa6pB+etsntttiWely WZlATYMqocbe4M3BdAJxIjbaLOlrKC3jW2Uy3DPvbeJ5qQulYZUhZAi5Rx1IkCJauSux U7MPLNeTNUmJBOyRPcrjGMUM7KiMCMB73JXFVzy21SfuWXNQfQlOOwJ2BlOTpQJKCuaE i5EKRS1EwtIDhZOwp7fC97F+iUVAuijvjc8BFevga+S8ncQhZqNTdO5hSlcveHXbR8eN tX7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mHfMB0eA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id y7si2557585jap.7.2021.07.08.08.48.46; Thu, 08 Jul 2021 08:48:58 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=mHfMB0eA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232048AbhGHPud (ORCPT + 99 others); Thu, 8 Jul 2021 11:50:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231139AbhGHPuc (ORCPT ); Thu, 8 Jul 2021 11:50:32 -0400 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11173C061760 for ; Thu, 8 Jul 2021 08:47:49 -0700 (PDT) Received: by mail-wr1-x42f.google.com with SMTP id k4so1816958wrc.8 for ; Thu, 08 Jul 2021 08:47:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=WJZpdcS1MW7Mj0T2TQvKXriF3MrXgDJAl+1qoxtsXRg=; b=mHfMB0eAj0cSxvZ60HKwN3+EhKCHtLHT+8luhQ4eY8fRDYpK9CaMIv1ifSd1fYpdy/ XS6M0+kAJNKbfnba459XmxOqX4Y7EN6z71Q9go4+f1zU5We4FEgis8U+Vn/Ct+RSOPqb fkzW7m9HokblG4zTQoDzp8XQoge2BZKTB9UjRlvCtT2BZs2BybA9GaFYRRr2ipw3Vxs8 thuRsKHak4/x7lfKkC/+0gaSfKTOOT6/RIo/cZXRLmUmKihtR9SFdGJL8N+I2ObgBUO1 O+qr9cpRMZ0Fx53Le7z3jODSLuCCWE5QhAmp0c4j7kK37z4Axp1aQ2p3pXft/mLpryc7 bc4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=WJZpdcS1MW7Mj0T2TQvKXriF3MrXgDJAl+1qoxtsXRg=; b=HFaiyrJs61y0e+5KmdZfaqJg8EyjHBbD3RegdxDV7/0c29BUNxDSeNLCOvdIrFHV2j ej6kgodnoHOjykPmu+mxhX3FUP4um4cSWNAQK3W9PaQZjMng4ugaYQ9pE11aXUc8U06x fhGIv81oAufYuuDIokeOtt6I3ZxkQ+wAp5zCZYd87XncrDr8+Z/b0egKsNES7YrEtjAW t98ztigjKXjMkCKmjJpK2+Yh1KRt72mA2XnSljNse2gdYcV0kkN3WQIEeNOuY2r7BCvV QTQBS4un6/oDuFzHwX6QzP+dBK+l/PGKEcy4uVjChI0NejikCP9+d53kZltxOIZ4VrzP VVIA== X-Gm-Message-State: AOAM532cbpOkUV7JdwGxJXLvCTM17k515FeUfPlOs3z5TXhBHrhUyfaJ dX3Vxn/5ZaoDBJz5bVtHDByAgA== X-Received: by 2002:a5d:47a7:: with SMTP id 7mr3990891wrb.150.1625759267621; Thu, 08 Jul 2021 08:47:47 -0700 (PDT) Received: from enceladus (ppp-94-66-242-227.home.otenet.gr. [94.66.242.227]) by smtp.gmail.com with ESMTPSA id x1sm9535888wmc.0.2021.07.08.08.47.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 08:47:47 -0700 (PDT) Date: Thu, 8 Jul 2021 18:47:43 +0300 From: Ilias Apalodimas To: Alexander Duyck Cc: Yunsheng Lin , David Miller , Jakub Kicinski , linuxarm@openeuler.org, yisen.zhuang@huawei.com, Salil Mehta , thomas.petazzoni@bootlin.com, Marcin Wojtas , Russell King - ARM Linux , hawk@kernel.org, Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrew Morton , Peter Zijlstra , Will Deacon , Matthew Wilcox , Vlastimil Babka , fenghua.yu@intel.com, guro@fb.com, peterx@redhat.com, Feng Tang , Jason Gunthorpe , mcroce@microsoft.com, Hugh Dickins , Jonathan Lemon , Alexander Lobakin , Willem de Bruijn , wenxu@ucloud.cn, cong.wang@bytedance.com, Kevin Hao , nogikh@google.com, Marco Elver , Netdev , LKML , bpf Subject: Re: [PATCH net-next RFC 1/2] page_pool: add page recycling support based on elevated refcnt Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 08, 2021 at 08:41:08AM -0700, Alexander Duyck wrote: > On Thu, Jul 8, 2021 at 8:36 AM Ilias Apalodimas > wrote: > > > > On Thu, Jul 08, 2021 at 08:29:56AM -0700, Alexander Duyck wrote: > > > On Thu, Jul 8, 2021 at 8:17 AM Ilias Apalodimas > > > wrote: > > > > > > > What do you think about resetting pp_recycle bit on pskb_expand_head()? > > > > > > I assume you mean specifically in the cloned case? > > > > > > > Yes. Even if we do it unconditionally we'll just loose non-cloned buffers from > > the recycling. > > I'll send a patch later today. > > If you do it unconditionally you could leak DMA mappings since in the > non-cloned case we don't bother with releasing the shared info since > we just did a memcpy of it without the reference count tweaks. We have > to be really careful here. The idea is that we have to make exactly > one call to the __page_pool_put_page function for this page. > > > > > If my memory serves me right Eric wanted that from the beginning. Then the > > > > cloned/expanded SKB won't trigger the recycling. If that skb hits the free > > > > path first, we'll end up recycling the fragments eventually. If the > > > > original one goes first, we'll just unmap the page(s) and freeing the cloned > > > > one will free all the remaining buffers. > > > > > > I *think* that should be fine. Effectively what we are doing is making > > > it so that if the original skb is freed first the pages are released, > > > and if it is released after the clone/expended skb then it can be > > > recycled. > > > > Exactly > > > > > > > > The issue is we have to maintain it so that there will be exactly one > > > caller of the recycling function for the pages. So any spot where we > > > are updating skb->head we will have to see if there is a clone and if > > > so we have to clear the pp_recycle flag on our skb so that it doesn't > > > try to recycle the page frags as well. > > > > Correct. I'll keep looking around in case there's something less fragile we > > can do > > That is the risk to this kind of thing. We have to make the call once > and only once and if we either miss it or call it too many times we > can introduce some serious issues. And I fully agree. Let me fix the obvious one now and I'll have a closer look on the recycling function it self. I can probably pick up the "changed head"/expanded SKB in the generic recycling code and refuse to recycle these packets. Then we'll just accept the fact that if those kind of packets are freed last, we won't recycle. Thanks, that was a very nice catch /Ilias > > Thanks. > > - Alex