Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <601c0203-ee5f-03a3-e9dd-fdb241f3bcdc@intel.com>
Date:   Tue, 8 Aug 2023 15:16:01 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: [PATCH net-next v4 5/6] page_pool: add a lockdep check for
 recycling in hardirq
Content-Language: en-US
To:     Alexander H Duyck <alexander.duyck@gmail.com>
CC:     "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
        Larysa Zaremba <larysa.zaremba@intel.com>,
        Yunsheng Lin <linyunsheng@huawei.com>,
        Alexander Duyck <alexanderduyck@fb.com>,
        "Jesper Dangaard Brouer" <hawk@kernel.org>,
        Ilias Apalodimas <ilias.apalodimas@linaro.org>,
        Simon Horman <simon.horman@corigine.com>,
        <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20230804180529.2483231-1-aleksander.lobakin@intel.com>
 <20230804180529.2483231-6-aleksander.lobakin@intel.com>
 <692d71dc8068b3d27aba39d7141c811755965786.camel@gmail.com>
From:   Alexander Lobakin <aleksander.lobakin@intel.com>
In-Reply-To: <692d71dc8068b3d27aba39d7141c811755965786.camel@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S2dCVlhTSmN3L2NuTUJWVXJXU0ludU51SDRJNTFqUm44UG5NMS9nMTVIZ0hG?=
 =?utf-8?B?Q2VQc3YxamVnRjFidkVqNy9nL1RFNUxyOXRwQnJUUXdQanM1dHVsblFVTVBQ?=
 =?utf-8?B?WnBiQTJwS2owU2xRSXdhZkdHemxKZHUvSXZqazRTcnBMR2g0K21BTnI5THBF?=
 =?utf-8?B?eVFwakkrTWs2UW5tRXFzMUNYaWFocy9jNmFMZkNEcUJTWW9DMGozNEdIbysx?=
 =?utf-8?B?bnRVdUFSamRlV2xwSmdTUnVBUTh3UUhMdWFHQ2hsWWVUOGprTmJBSE9rZDEy?=
 =?utf-8?B?UDVGblJ4ZkRaQ0QrVjg3OGVrbWdoZzV5R1IwbVN1T2lKQU90M2RoUnZ5Qk9y?=
 =?utf-8?B?RTJ6K2lXdkhDTThCbDhpZGlTTVBodmJWaHRTVzBCNHdZMk1xa1hIN0FOZWNk?=
 =?utf-8?B?TVFHRW42TEp6bVRXT0VhK29oTVljZG9OVGpqeHVRNW83aUlubnpNdGdnNXEy?=
 =?utf-8?B?bG51eEtOZnNwdWNpZ3NzdUNVTkNBa1pjb2NzclRiUWNoMk96RmdneTVLQ0lW?=
 =?utf-8?B?SlJadnpMYSt3SkQ5Ni9ncVlmamlwQW9RQXRSOXNSLzR6eEFRUWtCMlJuZEVm?=
 =?utf-8?B?OU9xUGRwdUVkZXJScnpKS25LemUvVjZOa0E4ZUpqK2p0YVkwTlNiV2FsSnBJ?=
 =?utf-8?B?a09URENWWVp3bEd1b2sxY0JYcnRRaE00S0M3WDNucHNOdE1rcENqOGhFTGd1?=
 =?utf-8?B?SGp0U01xWHdCVFdEdVlzRzJEMlZYbWQxbkk4QTBXUGhzeDdueXlMS3ZMaEJi?=
 =?utf-8?B?eVZTT2tVRG1rNUVCcVJqVmpncWRvR082cmpBZmRmTkNFeXM3dGZvUkdEMkN1?=
 =?utf-8?B?RlVnOHQyZEJuUVhQcVRUbERBMExhRlRPcEtyMXZNcWY5RVdIa1BJa3VoQ20w?=
 =?utf-8?B?NWo4dE9PVVlKWW45c3k0OUExamMwNytwcjlqc1ZLelVsUWcvWS8xSHBwSWt4?=
 =?utf-8?B?SGZLS2xsYkJwVmNCelJMWGZtNDZQTWNQUzJ0WGZvam4ybFJTaThuVjBzYm5Q?=
 =?utf-8?B?SCtFbHFYMGppN2lXYXRRL2hnRGlIWlJJTjNLWHJqWjB3eUEyVkQvQ3ExQm41?=
 =?utf-8?B?VEVGeFhPRFFkZU4zMVllMVNmcGpsUE9NODZWeHZ0RGo3UFViSlFHV09LUGhO?=
 =?utf-8?B?MjRaanRmb2NiekF3SGROeFBySWlUaTdlbmJTKzBQSjNMQ2krbXc4TFRjME1Z?=
 =?utf-8?B?SWJhbHNkcmEwTFFyOGI3b1RRbmpoTUc3RXUrNWZYNnc1b3p4aHljWXUwU3Fj?=
 =?utf-8?B?MUsvR0FINnZHR3dDMmU2TWlVcUZYMlhTb2k0cWltNk5Wd09LOHdDZTJUNGxD?=
 =?utf-8?B?dTZ1MnpTYTJ0K2tpSWh3aWRmMmNHTjZ0SURreFdxdjhFQUgvN1E2ZEdyUUM2?=
 =?utf-8?B?NzI3MEpTdGc4U1RFbXZTenJkTWRaeWdVdnpOUmNUZmFiZTVuRXU5SGp1R2Yw?=
 =?utf-8?B?S3E1Nk52dzhCWHZSdUhFN0d1U2Q5dlhGdTBTNldTOVRGeWx6YnI4SlhVVklD?=
 =?utf-8?B?T2NsWmJSZW96NjBSb1VQb3R1ajZrdG15czR0Y3RJclQ0RDRPejN0dUlVZHZY?=
 =?utf-8?B?cXQrM2paMllVcnExMk10aWlQOWxkZGovTjE3cUt6YnRhZVlJNzRYcFNZRFpF?=
 =?utf-8?B?ZmN0YzNRMXlwWHQyRXdzak1CcDNxMWx0a2FvYmwzTWlZaXRXMU56Qm9PUm9z?=
 =?utf-8?B?TXczZzQwSVJYd0dMazd1SzhIUlNTNE1XQW5lZGNoNWhIdHRNU2VKZWErTGpl?=
 =?utf-8?B?RXFKSWFmeDZmY0d1SERGYkZscThCdmNOZncwaWdEMmdHa0VGZktyRStreXha?=
 =?utf-8?B?dldVd08wT1o0eXFLajNrdmlqbmo1WDR4Z1FiY3NwRitsTkI5WlFYSEVxT2M1?=
 =?utf-8?B?RzZoQ09GV2w1emtuVEdocmY1RExtNGIyYWpJOTAza0xRaXQwd2Y2azNFenFm?=
 =?utf-8?B?WGp1bzRpSExVM2lmQ0NLa2tQRCtENnNPcHRCNVJMd2F4Y3JFZXhLU2xYbXkv?=
 =?utf-8?B?WGFEbXBFekQ4blU1UzE4U0NFUjVzNUpjQ3h1TjBZWFdnaEpmN0NTRVJid2xO?=
 =?utf-8?B?UStoSFN2dGR6Y05mMkxHU2NueG95enZRWGR1TGF2NERKYnd2WDQrS2h2Wngv?=
 =?utf-8?B?dlpRSFh5d0w0REtsaVUvb2NBOVZtbXhJTkRBNjJHNG9uSHdBOGQrUHBNbWd4?=
 =?utf-8?B?c0E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e457b603-2dfd-43de-29f7-08db9811b40f
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3625.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2023 13:16:40.7813
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MzaGlm+dpv1eYaGjJP3jSmoJ1n7KzTrzgkAW6ggfcrxdjLAuJr4qEFHlQcYVGXW3aftTJ08sE0Ll4lXoigTmmPgf8Q/HC7XZuy/gkkHWjlI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB8259
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Alexander H Duyck <alexander.duyck@gmail.com>
Date: Mon, 07 Aug 2023 07:48:54 -0700

> On Fri, 2023-08-04 at 20:05 +0200, Alexander Lobakin wrote:
>> From: Jakub Kicinski <kuba@kernel.org>
>>
>> Page pool use in hardirq is prohibited, add debug checks
>> to catch misuses. IIRC we previously discussed using
>> DEBUG_NET_WARN_ON_ONCE() for this, but there were concerns
>> that people will have DEBUG_NET enabled in perf testing.
>> I don't think anyone enables lockdep in perf testing,
>> so use lockdep to avoid pushback and arguing :)
>>
>> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
>> Acked-by: Jesper Dangaard Brouer <hawk@kernel.org>
>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
>> ---
>>  include/linux/lockdep.h | 7 +++++++
>>  net/core/page_pool.c    | 2 ++
>>  2 files changed, 9 insertions(+)
>>
>> diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h
>> index 310f85903c91..dc2844b071c2 100644
>> --- a/include/linux/lockdep.h
>> +++ b/include/linux/lockdep.h
>> @@ -625,6 +625,12 @@ do {									\
>>  	WARN_ON_ONCE(__lockdep_enabled && !this_cpu_read(hardirq_context)); \
>>  } while (0)
>>  
>> +#define lockdep_assert_no_hardirq()					\
>> +do {									\
>> +	WARN_ON_ONCE(__lockdep_enabled && (this_cpu_read(hardirq_context) || \
>> +					   !this_cpu_read(hardirqs_enabled))); \
>> +} while (0)
>> +
>>  #define lockdep_assert_preemption_enabled()				\
>>  do {									\
>>  	WARN_ON_ONCE(IS_ENABLED(CONFIG_PREEMPT_COUNT)	&&		\
>> @@ -659,6 +665,7 @@ do {									\
>>  # define lockdep_assert_irqs_enabled() do { } while (0)
>>  # define lockdep_assert_irqs_disabled() do { } while (0)
>>  # define lockdep_assert_in_irq() do { } while (0)
>> +# define lockdep_assert_no_hardirq() do { } while (0)
>>  
>>  # define lockdep_assert_preemption_enabled() do { } while (0)
>>  # define lockdep_assert_preemption_disabled() do { } while (0)
>> diff --git a/net/core/page_pool.c b/net/core/page_pool.c
>> index 03ad74d25959..77cb75e63aca 100644
>> --- a/net/core/page_pool.c
>> +++ b/net/core/page_pool.c
>> @@ -587,6 +587,8 @@ static __always_inline struct page *
>>  __page_pool_put_page(struct page_pool *pool, struct page *page,
>>  		     unsigned int dma_sync_size, bool allow_direct)
>>  {
>> +	lockdep_assert_no_hardirq();
>> +
>>  	/* This allocator is optimized for the XDP mode that uses
>>  	 * one-frame-per-page, but have fallbacks that act like the
>>  	 * regular page allocator APIs.
> 
> So two points.
> 
> First could we look at moving this inside the if statement just before
> we return the page, as there isn't a risk until we get into that path
> of needing a lock.
> 
> Secondly rather than returning an error is there any reason why we
> couldn't just look at not returning page and instead just drop into the
> release path which wouldn't take the locks in the first place? Either

That is exception path to quickly catch broken drivers and fix them, why
bother? It's not something we have to live with.

> that or I would even be good with some combination of the two where we
> threw a warning, but still just dropped the page so we reduce our risk
> further of actually locking things up.

Thanks,
Olek