Received: by 2002:a05:6500:2018:b0:1fb:9675:f89d with SMTP id t24csp41378lqh; Thu, 30 May 2024 13:20:54 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCWpf5RFYLHIVx+rY3RjE3woZhVs8gMmgzBl77l5KlH5AvNdHeHT9y0BsTOoc2HpJH69u898rN9W6LCwVew0xafnU+mxxqV07dm4NU/3sQ== X-Google-Smtp-Source: AGHT+IGN2QE3mG5PYBSv5asbJ9K6zmPVkIFhHKoYEFa4pEb/1++Zd/05HNoNWWubegiS7TiYpclw X-Received: by 2002:a50:f687:0:b0:57a:2e7c:1d88 with SMTP id 4fb4d7f45d1cf-57a3638fb05mr23356a12.20.1717100454307; Thu, 30 May 2024 13:20:54 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1717100454; cv=pass; d=google.com; s=arc-20160816; b=I+mmlSzuaoL0ytS6TLLItOd02KYDx4lLIumDZtvKzUfd6lLKhvZka+F5FiZhXzNnaP YBDXiFvB5UzuyoEZAlnULz/UDRDDRrVlI+5KmxwdNCWhoNP3Om/dCbl4dLJUlu3WGa+P T6yZnNfYBuNCIXmakfAtfji3P8ylVUw+Rb7pyO8IolC8Q/jjGFu2IHtO2YPXODTUmxBB /3atjnkRw4JahwMiBhxMMxeAx/K5aVp8dlQLDUnQsdRjXh56PMOv/AwvDgC7OCyu0fU5 GFZF2uVCKaiNqF4TM6v3mhYKqM6aiooEs0pMhnR9AO9F8dB12cFNxaEhtW8Z9e5PSa3z 0eDQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=7JDXDL+KykbN51qW6GKMUxsCjcMIQ2866w2vsxUfz10=; fh=zn7RcBkYlb8rgP3RasvC8/AJaraK80shaS2abNXb3gs=; b=quvy8Smf0oEoIjwlghJmd65SMnB+OoQCVY7Yd/E0hgkpo3UvzvjFHj+W2PZAmXKCpT KAuhYQrX1bcuM93yePSLAdBZCbYXUTCEZSaJ41prIIbqGfKK8SSG/TNiNioxzZEXXWSH 4llh3EFsJgdeEjLeMMWNLWbtSuIWu6wb1K64Q1nHLpxoijrpne+lH6zf7xPqQ2sOq99j +SaH0FqkEg9dTD7+9o2ofcyoctaoOs4hE3YTXOfW5KP9gXHqqtY25sbtkHa3uShYN69x 5is8Qaoo8PV5WqP76r3zpaibtKTr5F9159tpUC2UjF66bKPcHDr101R/ykGJJeyAVlfq HW2A==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=v2XlomQn; arc=pass (i=1 spf=pass spfdomain=flex--almasrymina.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-195870-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-195870-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id 4fb4d7f45d1cf-57a31c6320csi163661a12.242.2024.05.30.13.20.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 May 2024 13:20:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-195870-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=v2XlomQn; arc=pass (i=1 spf=pass spfdomain=flex--almasrymina.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-195870-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-195870-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id D5D601F22379 for ; Thu, 30 May 2024 20:20:53 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5D7201862BC; Thu, 30 May 2024 20:16:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="v2XlomQn" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8563218308B for ; Thu, 30 May 2024 20:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717100210; cv=none; b=q91DKm0Y4a4OjjAKBYUSdGl7AoMPOdogTd+Pj6C3YauTFv9JZy8lGL8uowGSSTov7Z81CEiw5GvDvIskhNh5O6WLnkvGo64FKxFOTr5At/qF4J3kCgkaT+GBk1IHe41SfXfZylpGeA+B2QTXeFqhnDfg+jRMIJOeSpFo5WGfg6o= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717100210; c=relaxed/simple; bh=wEgQUTxKepJR2zfOGMhG3HAnzjRVc98hRtm+5uHFTqM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=sdOd1AMIKLdD7Et7Tb2z2syEx+5ykPTWFGZOVyH6zwNo/fOjfD0G8POtGL1wHQMrSjlYGEOsobC8OqEc4ibuWxY+LjXCTrGs3ifqAHhTO84Jm+BksJnQXtLxXMnSmwhAe5fbZw3hHfe2f8Dhbp+qeqTZlJVrYnW2OIRmjD8VXXg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=v2XlomQn; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--almasrymina.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-df4dc93d0d3so2067019276.2 for ; Thu, 30 May 2024 13:16:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717100201; x=1717705001; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7JDXDL+KykbN51qW6GKMUxsCjcMIQ2866w2vsxUfz10=; b=v2XlomQnh9flKLNM9EXwCHXoZ1QUgMrpPewWepM90zTbGge7C4XQtF9HD5lBNqFXtW I5psUf97NtD6vuTJx9//03gYj5P5bOY6XDT+4CFoLFN2Gntsi9120WpR79I8XnnJeDRi lgMRx9QkIOxqqPjv/miofENHmwxlYGIdhwnlPxJElhNlIxZqLTgeWXfBsKyGZ1WCZ9CI EgC/Id978MmGqWXHKZ3u22/OOD8fbU36VAStwao5WqNQRO6Zit75p90WAukl9+qV6skj auCzH5pggOuubkcGnTPRIGHADN9kwzlQmqW5MG9UYGEY9bU5Jd/9UA4CCX0aXwynclO8 naIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717100201; x=1717705001; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7JDXDL+KykbN51qW6GKMUxsCjcMIQ2866w2vsxUfz10=; b=XvR2xm0ZfyIB9DudRqGGd9OX8lJLKBPKQIO2lrOGBU9bOf0b8G0zOp+PXRbxDV5r8t 52ItDHh5mCjI6JMPd1PBrMPTs81mrh7idtPFav9I/BOq5hSN2fRflaOlJ9WWj9P/3yJq FpSKyJWQcYoVogo/k4oZzTHbZhAfR0HHpFonoaVXMsoHZ/cHB+kdi9BlT6Tf4ErmRS9+ ITZ+C/3jouQ2ULJmQSiBuVU0uVS9GIFls7XnYDdPoG8rt2PJuDBGOzkid4nekKaXgxcZ DXQwbM+m5/OENiyfjBTbz9CXEvw95hUTDNXJQCjhe7H8wRxF86Rn2Volveo7rAUWRchw TSKA== X-Forwarded-Encrypted: i=1; AJvYcCX4a5fEVI0EjR4T0bONW2JW+WHcu1Ug8NCc3IHpBl07E1m00jLXIG/mPJ6rim1Tm4EbNhCaJyMG0Q35u/eKlMBxTS1xs0yuasMj/n6F X-Gm-Message-State: AOJu0YyexUB+jtMEddS+hNfCU/Ueqr1b5aHcObLvEAz6dlbCrvtBaJrG rCou4SDDu6CRc6XPAOwC2KhXESbpRscVSu+qwsE4aGD1ZYGq1WBwg9gXA6Fp3iDUhD1xIiQjkYu PoWzxQ8eMmHGFoa2Gsu6YUQ== X-Received: from almasrymina.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:4bc5]) (user=almasrymina job=sendgmr) by 2002:a05:6902:154a:b0:df7:b717:10c2 with SMTP id 3f1490d57ef6-dfa5a5ab4c7mr748332276.2.1717100200760; Thu, 30 May 2024 13:16:40 -0700 (PDT) Date: Thu, 30 May 2024 20:16:11 +0000 In-Reply-To: <20240530201616.1316526-1-almasrymina@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240530201616.1316526-1-almasrymina@google.com> X-Mailer: git-send-email 2.45.1.288.g0e0cd299f1-goog Message-ID: <20240530201616.1316526-13-almasrymina@google.com> Subject: [PATCH net-next v10 12/14] net: add SO_DEVMEM_DONTNEED setsockopt to release RX frags From: Mina Almasry To: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-arch@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Mina Almasry , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Donald Hunter , Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Andreas Larsson , Jesper Dangaard Brouer , Ilias Apalodimas , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Arnd Bergmann , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Steffen Klassert , Herbert Xu , David Ahern , Willem de Bruijn , Shuah Khan , Sumit Semwal , "=?UTF-8?q?Christian=20K=C3=B6nig?=" , Pavel Begunkov , David Wei , Jason Gunthorpe , Yunsheng Lin , Shailend Chand , Harshitha Ramamurthy , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi , Willem de Bruijn , Kaiyuan Zhang Content-Type: text/plain; charset="UTF-8" Add an interface for the user to notify the kernel that it is done reading the devmem dmabuf frags returned as cmsg. The kernel will drop the reference on the frags to make them available for reuse. Signed-off-by: Willem de Bruijn Signed-off-by: Kaiyuan Zhang Signed-off-by: Mina Almasry --- v10: - Fix leak of tokens (Nikolay). v7: - Updated SO_DEVMEM_* uapi to use the next available entry (Arnd). v6: - Squash in locking optimizations from edumazet@google.com. With his changes we lock the xarray once per sock_devmem_dontneed operation rather than once per frag. Changes in v1: - devmemtoken -> dmabuf_token (David). - Use napi_pp_put_page() for refcounting (Yunsheng). - Fix build error with missing socket options on other asms. --- arch/alpha/include/uapi/asm/socket.h | 1 + arch/mips/include/uapi/asm/socket.h | 1 + arch/parisc/include/uapi/asm/socket.h | 1 + arch/sparc/include/uapi/asm/socket.h | 1 + include/uapi/asm-generic/socket.h | 1 + include/uapi/linux/uio.h | 4 ++ net/core/sock.c | 61 +++++++++++++++++++++++++++ 7 files changed, 70 insertions(+) diff --git a/arch/alpha/include/uapi/asm/socket.h b/arch/alpha/include/uapi/asm/socket.h index ef4656a41058a..251b73c5481ea 100644 --- a/arch/alpha/include/uapi/asm/socket.h +++ b/arch/alpha/include/uapi/asm/socket.h @@ -144,6 +144,7 @@ #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 79 #define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF +#define SO_DEVMEM_DONTNEED 80 #if !defined(__KERNEL__) diff --git a/arch/mips/include/uapi/asm/socket.h b/arch/mips/include/uapi/asm/socket.h index 414807d55e33f..8ab7582291abf 100644 --- a/arch/mips/include/uapi/asm/socket.h +++ b/arch/mips/include/uapi/asm/socket.h @@ -155,6 +155,7 @@ #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 79 #define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF +#define SO_DEVMEM_DONTNEED 80 #if !defined(__KERNEL__) diff --git a/arch/parisc/include/uapi/asm/socket.h b/arch/parisc/include/uapi/asm/socket.h index 2b817efd45444..38fc0b188e084 100644 --- a/arch/parisc/include/uapi/asm/socket.h +++ b/arch/parisc/include/uapi/asm/socket.h @@ -136,6 +136,7 @@ #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 79 #define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF +#define SO_DEVMEM_DONTNEED 80 #if !defined(__KERNEL__) diff --git a/arch/sparc/include/uapi/asm/socket.h b/arch/sparc/include/uapi/asm/socket.h index 00248fc689773..57084ed2f3c4e 100644 --- a/arch/sparc/include/uapi/asm/socket.h +++ b/arch/sparc/include/uapi/asm/socket.h @@ -137,6 +137,7 @@ #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 0x0058 #define SCM_DEVMEM_DMABUF SO_DEVMEM_DMABUF +#define SO_DEVMEM_DONTNEED 0x0059 #if !defined(__KERNEL__) diff --git a/include/uapi/asm-generic/socket.h b/include/uapi/asm-generic/socket.h index 25a2f5255f523..1acb77780f103 100644 --- a/include/uapi/asm-generic/socket.h +++ b/include/uapi/asm-generic/socket.h @@ -135,6 +135,7 @@ #define SO_PASSPIDFD 76 #define SO_PEERPIDFD 77 +#define SO_DEVMEM_DONTNEED 97 #define SO_DEVMEM_LINEAR 98 #define SCM_DEVMEM_LINEAR SO_DEVMEM_LINEAR #define SO_DEVMEM_DMABUF 99 diff --git a/include/uapi/linux/uio.h b/include/uapi/linux/uio.h index 3a22ddae376a2..d17f8fcd93ec9 100644 --- a/include/uapi/linux/uio.h +++ b/include/uapi/linux/uio.h @@ -33,6 +33,10 @@ struct dmabuf_cmsg { */ }; +struct dmabuf_token { + __u32 token_start; + __u32 token_count; +}; /* * UIO_MAXIOV shall be at least 16 1003.1g (5.4.1.1) */ diff --git a/net/core/sock.c b/net/core/sock.c index 521e6373d4f73..128d6a57a7af0 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -124,6 +124,7 @@ #include #include #include +#include #include #include #include @@ -1049,6 +1050,62 @@ static int sock_reserve_memory(struct sock *sk, int bytes) return 0; } +#ifdef CONFIG_PAGE_POOL +static noinline_for_stack int +sock_devmem_dontneed(struct sock *sk, sockptr_t optval, unsigned int optlen) +{ + unsigned int num_tokens, i, j, k, netmem_num = 0; + struct dmabuf_token *tokens; + netmem_ref netmems[16]; + int ret = 0; + + if (sk->sk_type != SOCK_STREAM || sk->sk_protocol != IPPROTO_TCP) + return -EBADF; + + if (optlen % sizeof(struct dmabuf_token) || + optlen > sizeof(*tokens) * 128) + return -EINVAL; + + tokens = kvmalloc_array(128, sizeof(*tokens), GFP_KERNEL); + if (!tokens) + return -ENOMEM; + + num_tokens = optlen / sizeof(struct dmabuf_token); + if (copy_from_sockptr(tokens, optval, optlen)) { + kvfree(tokens); + return -EFAULT; + } + + xa_lock_bh(&sk->sk_user_frags); + for (i = 0; i < num_tokens; i++) { + for (j = 0; j < tokens[i].token_count; j++) { + netmem_ref netmem = (__force netmem_ref)__xa_erase( + &sk->sk_user_frags, tokens[i].token_start + j); + + if (netmem && + !WARN_ON_ONCE(!netmem_is_net_iov(netmem))) { + netmems[netmem_num++] = netmem; + if (netmem_num == ARRAY_SIZE(netmems)) { + xa_unlock_bh(&sk->sk_user_frags); + for (k = 0; k < netmem_num; k++) + WARN_ON_ONCE(!napi_pp_put_page(netmems[k])); + netmem_num = 0; + xa_lock_bh(&sk->sk_user_frags); + } + ret++; + } + } + } + + xa_unlock_bh(&sk->sk_user_frags); + for (k = 0; k < netmem_num; k++) + WARN_ON_ONCE(!napi_pp_put_page(netmems[k])); + + kvfree(tokens); + return ret; +} +#endif + void sockopt_lock_sock(struct sock *sk) { /* When current->bpf_ctx is set, the setsockopt is called from @@ -1200,6 +1257,10 @@ int sk_setsockopt(struct sock *sk, int level, int optname, ret = -EOPNOTSUPP; return ret; } +#ifdef CONFIG_PAGE_POOL + case SO_DEVMEM_DONTNEED: + return sock_devmem_dontneed(sk, optval, optlen); +#endif } sockopt_lock_sock(sk); -- 2.45.1.288.g0e0cd299f1-goog