2023-06-14 23:19:01

by Dmitry Safonov

Subject: [PATCH v7 00/22] net/tcp: Add TCP-AO support

Hi,

This is version 7 of TCP-AO support. I based it on master and there's
trivial conflict with commit c0a8966e2bc7 ("net: ipv4: use consistent
txhash in TIME_WAIT and SYN_RECV") in linux-next.

Big thing is that I've managed to drop per-CPU ahash request allocations
in tcp_sigpool. The only 2 reasons the pool yet exists are:
- scratch_area pre-allocation
- allocation of tfms on setsockopt(), those will be cloned on RX/TX

There's a side patch set to fix cmac(aes128) cloning:
https://lore.kernel.org/all/[email protected]/T/#u

Another big thing was TCP_AO_REPAIR UAPI, that allowed me to add yet
even more selftests, as it provides a handy way to get socket in a
needed state and hack its TCP-AO state. So, here comes RST segments,
Sequence Number Extension (SNE) and Initial Sequence Numbers (ISN)
selftests. Which also helped to refactor tcp_v{4,6}_send_reset()
refactoring.

The only thing left from my previous cover-letter-TODO lists is to write
a Documentation/ page about kernel TCP-AO support and its UAPIs.
And I have a lot of ideas how-to selftest/stress-test/benchmark TCP-AO.
Those seem like they can wait and I'd prioritize fixing review comments
over them, so please provide your reviews.

One more thing is verifying segments on TCP_TIME_WAIT sockets. At this
version, TCP-AO does the same as TCP-MD5: doesn't bother verifying
inbound segments (except SYN, that may revive the connection).
There's independent of TCP-AO patch set for TCP-MD5 to verify segments
on twsk:
https://lore.kernel.org/all/[email protected]/T/#u

This is also available as a git branch for pull:
https://github.com/0x7f454c46/linux/tree/tcp-ao-v7
And another branch with selftests, that will be sent later separately:
https://github.com/0x7f454c46/linux/tree/tcp-ao-v7-with-selftests

Thanks for your time and reviews,
Dmitry

--- Changelog ---

Changes from v6:
- Some more trivial build warnings fixups (kernel test robot <[email protected]>)
- Added TCP_AO_REPAIR setsockopt(), getsockopt()
- Allowed TCP_AO_* setsockopts if (tp->repair) is on
- Added selftests for TCP_AO_REPAIR, that also check incorrect
ISNs/SNEs, which result in a broken TCP-AO connection - that verifies
that both Initial Sequence Numbers and Sequence Number Extension are
part of MAC generation
- Using TCP_AO_REPAIR added a selftest for SEQ numbers rollover,
checking that SNE was incremented, connection is alive post-rolloever
and no TCP segments with a wrong signature arrived
- Wrote a selftest for RST segments: both active reset (goes through
transmit_skb()) and passive reset (goes through tcp_v{4,6}_send_reset()).
- Refactored and made readable tcp_v{4,6}_send_reset(), also adding
support for TCP_LISTEN/TCP_NEW_SYN_RECV
- Dropped per-CPU ahash requests allocations in favor of Herbert's
clone-tfm crypto API
- Added Donald Cassidy to Cc as he's interested in getting it into RHEL.

Version 6: https://lore.kernel.org/all/[email protected]/T/#u

iperf[3] benchmarks for version 6:
v6.4-rc1 TCP-AO-v6
TCP 43.9 Gbits/sec 43.5 Gbits/sec
TCP-MD5 2.20 Gbits/sec 2.25 Gbits/sec
TCP-AO(hmac(sha1)) 2.53 Gbits/sec
TCP-AO(hmac(sha512)) 1.67 Gbits/sec
TCP-AO(hmac(sha384)) 1.77 Gbits/sec
TCP-AO(hmac(sha224)) 1.29 Gbits/sec
TCP-AO(hmac(sha3-512)) 481 Mbits/sec
TCP-AO(hmac(md5)) 2.07 Gbits/sec
TCP-AO(hmac(rmd160)) 1.01 Gbits/sec
TCP-AO(cmac(aes128)) 2.11 Gbits/sec

Changes from v5:
- removed check for TCP_AO_KEYF_IFINDEX in delete command:
VRF might have been destroyed, there still needs to be a way to delete
keys that were bound to that l3intf (should tcp_v{4,6}_parse_md5_keys()
avoid the same check as well?)
- corrected copy'n'paste typo in tcp_ao_info_cmd() (assign ao_info->rnext_key)
- simplified a bit tcp_ao_copy_mkts_to_user(); added more UAPI checks
for getsockopt(TCP_AO_GET_KEYS)
- More UAPI selftests in setsockopt-closed: 29 => 120
- ported TCP-AO patches on Herbert's clone-tfm changes
- adjusted iperf patch for TCP-AO UAPI changes from version 5
- added measures for TCP-AO with tcp_sigpool & clone_tfm backends

Version 5: https://lore.kernel.org/all/[email protected]/T/#u

Changes from v4:
- Renamed tcp_ao_matched_key() => tcp_ao_established_key()
- Missed `static` in function definitions
(kernel test robot <[email protected]>)
- Fixed CONFIG_IPV6=m build
- Unexported tcp_md5_*_sigpool() functions
- Cleaned up tcp_ao.h: undeclared tcp_ao_cache_traffic_keys(),
tcp_v4_ao_calc_key_skb(); removed tcp_v4_inbound_ao_hash()
- Marked "net/tcp: Prepare tcp_md5sig_pool for TCP-AO" as a [draft] patch
- getsockopt() now returns TCP-AO per-key counters
- Another getsockopt() now returns per-ao_info stats: counters
and accept_icmps flag state
- Wired up getsockopt() returning counters to selftests
- Fixed a porting mistake: TCP-AO hash in some cases was written in TCP
header without accounting for MAC length of the key, rewritting skb
shared info
- Fail adding a key with L3 ifindex when !TCP_AO_KEYF_IFINDEX, instead
of ignoring tcpa_ifindex (stricter UAPI check)
- Added more test-cases to setsockopt-closed.c selftest
- tcp_ao_hash_skb_data() was a copy'n'paste of tcp_md5_hash_skb_data()
share it now under tcp_sigpool_hash_skb_data()
- tcp_ao_mkt_overlap_v{4,6}() deleted as they just re-invented
tcp_ao_do_lookup(). That fixes an issue with multiple IPv4-mapped-IPv6
keys for different peers on a listening socket.
- getsockopt() now is tested to return correct VRF number for a key
- TCP-AO and TCP-MD5 interraction in non/default VRFs: added +19 selftests
made them SKIP when CONFIG_VRF=n
- unsigned-md5 selftests now checks both scenarios:
(1) adding TCP-AO key _after_ TCP-MD5 key
(2) adding TCP-MD5 key _after_ TCP-AO key
- Added a ratelimited warning if TCP-AO key.ifindex doesn't match
sk->sk_bound_dev_if - that will warn a user for potential VRF issues
- tcp_v{4,6}_parse_md5_keys() now allows adding TCP-MD5 key with
ifindex=0 and TCP_MD5SIG_FLAG_IFINDEX together with TCP-AO key from
another VRF
- Add TCP_AO_CMDF_AO_REQUIRED, which makes a socket TCP-AO only,
rejecting TCP-MD5 keys or any unsigned TCP segments
- Remove `tcpa_' prefix for UAPI structure members
- UAPI cleanup: I've separated & renamed per-socket settings
(such as ao_info flags + current/rnext set) from per-key changes:
TCP_AO => TCP_AO_ADD_KEY
TCP_AO_DEL => TCP_AO_DEL_KEY
TCP_AO_GET => TCP_AO_GET_KEYS
TCP_AO_MOD => TCP_AO_INFO, the structure is now valid for both
getsockopt() and setsockopt().
- tcp_ao_current_rnext() was split up in order to fail earlier when
sndid/rcvid specified can't be set, before anything was changed in ao_info
- fetch current_key before dumping TCP-AO keys in getsockopt(TCP_AO_GET_KEYS):
it may race with changing current_key by RX, which in result might
produce a dump with no current_key for userspace.
- instead of TCP_AO_CMDF_* flags, used bitfileds: the flags weren't
shared between all TCP_AO_{ADD,GET,DEL}_KEY{,S}, so bitfields are more
descriptive here
- use READ_ONCE()/WRITE_ONCE() for current_key and rnext_key more
consistently; document in comment the rules for accessing them
- selftests: check all setsockopts()/getsockopts() support extending
option structs

Version 4: https://lore.kernel.org/all/[email protected]/T/#u

Changes from v3:
- TCP_MD5 dynamic static key enable/disable patches merged separately [4]
- crypto_pool patches were nacked [5], so instead this patch set extends
TCP-MD5-sigpool to be used for TCP-AO as well as for TCP-MD5
- Added missing `static' for tcp_v6_ao_calc_key()
(kernel test robot <[email protected]>)
- Removed CONFIG_TCP_AO default=y and added "If unsure, say N."
- Don't leak ao_info and don't create an unsigned TCP socket if there was
a TCP-AO key during handshake, but it was removed from listening socket
while the connection was being established
- Migrate to use static_key_fast_inc_not_disabled() and check return
code of static_branch_inc()
- Change some return codes to EAFNOSUPPORT for error-pathes where
family is neither AF_INET nor AF_INET6
- setsockopt()s on a closed/listen socket might have created stray ao_info,
remove it if connect() is called with a correct TCP-MD5 key, the same
for the reverse situation: remove md5sig_info straight away from the
socket if it's going to be TCP-AO connection
- IPv4-mapped-IPv6 addresses + selftest in fcnal-test.sh (by Salam)
- fix using uninitialized sisn/disn from stack - it would only make
non-SYN packets fail verification on a listen socket, which are not
expected anyway (kernel test robot <[email protected]>)
- implicit padding in UAPI TCP-AO structures converted to explicit
(spotted-by David Laight)
- Some selftests missed zero-initializers for uapi structs on stack
- Removed tcp_ao_do_lookup_rcvid() and tcp_ao_do_lookup_sndid() in
favor of unified tcp_ao_matched_key()
- Disallowed setting current/rnext keys on listen sockets - that wasn't
supported and didn't affect anything, cleanup for the UAPI
- VRFs support for TCP-AO

Version 3: https://lore.kernel.org/all/[email protected]/T/#u

Changes from v2:
- Added more missing `static' declarations for local functions
(kernel test robot <[email protected]>)
- Building now with CONFIG_TCP_AO=n and CONFIG_TCP_MD5SIG=n
(kernel test robot <[email protected]>)
- Now setsockopt(TCP_AO) is allowed when it's TCP_LISTEN or TCP_CLOSE
state OR the key added is not the first key on a socket (by Salam)
- CONFIG_TCP_AO does not depend on CONFIG_TCP_MD5SIG anymore
- Don't leak tcp_md5_needed static branch counter when TCP-MD5 key
is modified/changed
- TCP-AO lookups are dynamically enabled/disabled with static key when
there is ao_info in the system (and when it is destroyed)
- Wired SYN cookies up to TCP-AO (by Salam)
- Fix verification for possible re-transmitted SYN packets (by Salam)
- use sockopt_lock_sock() instead of lock_sock()
(from v6.1 rebase, commit d51bbff2aba7)
- use sockptr_t in getsockopt(TCP_AO_GET)
(from v6.1 rebase, commit 34704ef024ae)
- Fixed reallocating crypto_pool's scratch area by IPI while
crypto_pool_get() was get by another CPU
- selftests on older kernels (or with CONFIG_TCP_AO=n) should exit with
SKIP, not FAIL (Shuah Khan <[email protected]>)
- selftests that check interaction between TCP-AO and TCP-MD5 now
SKIP when CONFIG_TCP_MD5SIG=n
- Measured the performance of different hashing algorithms for TCP-AO
and compare with TCP-MD5 performance. This is done with hacky patches
to iperf (see [3]). At this moment I've done it in qemu/KVM with CPU
affinities set on Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz.
No performance degradation was noticed before/after patches, but given
the measures were done in a VM, without measuring it on a physical dut
it only gives a hint of relative speed for different hash algorithms
with TCP-AO. Here are results, averaging on 30 measures each:
TCP: 3.51Gbits/sec
TCP-MD5: 1.12Gbits/sec
TCP-AO(HMAC(SHA1)): 1.53Gbits/sec
TCP-AO(CMAC(AES128)): 621Mbits/sec
TCP-AO(HMAC(SHA512)): 1.21Gbits/sec
TCP-AO(HMAC(SHA384)): 1.20Gbits/sec
TCP-AO(HMAC(SHA224)): 961Mbits/sec
TCP-AO(HMAC(SHA3-512)): 157Mbits/sec
TCP-AO(HMAC(RMD160)): 659Mbits/sec
TCP-AO(HMAC(MD5): 1.12Gbits/sec
(the last one is just for fun, but may make sense as it provides
the same security as TCP-MD5, but allows multiple keys and a mechanism
to change them from RFC5925)

Version 2: https://lore.kernel.org/all/[email protected]/T/#u

Changes from v1:
- Building now with CONFIG_IPV6=n (kernel test robot <[email protected]>)
- Added missing static declarations for local functions
(kernel test robot <[email protected]>)
- Addressed static analyzer and review comments by Dan Carpenter
(thanks, they were very useful!)
- Fix elif without defined() for !CONFIG_TCP_AO
- Recursively build selftests/net/tcp_ao (Shuah Khan), patches in:
https://lore.kernel.org/all/[email protected]/T/#u
- Don't leak crypto_pool reference when TCP-MD5 key is modified/changed
- Add TCP-AO support for nettest.c and fcnal-test.sh
(will be used for VRF testing in later versions)

Comparison between Leonard proposal and this (overview):
https://lore.kernel.org/all/[email protected]/T/#u

Version 1: https://lore.kernel.org/all/[email protected]/T/#u

This patchset implements the TCP-AO option as described in RFC5925. There
is a request from industry to move away from TCP-MD5SIG and it seems the time
is right to have a TCP-AO upstreamed. This TCP option is meant to replace
the TCP MD5 option and address its shortcomings. Specifically, it provides
more secure hashing, key rotation and support for long-lived connections
(see the summary of TCP-AO advantages over TCP-MD5 in (1.3) of RFC5925).
The patch series starts with six patches that are not specific to TCP-AO
but implement a general crypto facility that we thought is useful
to eliminate code duplication between TCP-MD5SIG and TCP-AO as well as other
crypto users. These six patches are being submitted separately in
a different patchset [1]. Including them here will show better the gain
in code sharing. Next are 18 patches that implement the actual TCP-AO option,
followed by patches implementing selftests.

The patch set was written as a collaboration of three authors (in alphabetical
order): Dmitry Safonov, Francesco Ruggeri and Salam Noureddine. Additional
credits should be given to Prasad Koya, who was involved in early prototyping
a few years back. There is also a separate submission done by Leonard Crestez
whom we thank for his efforts getting an implementation of RFC5925 submitted
for review upstream [2]. This is an independent implementation that makes
different design decisions.

For example, we chose a similar design to the TCP-MD5SIG implementation and
used setsockopts to program per-socket keys, avoiding the extra complexity
of managing a centralized key database in the kernel. A centralized database
in the kernel has dubious benefits since it doesn’t eliminate per-socket
setsockopts needed to specify which sockets need TCP-AO and what are the
currently preferred keys. It also complicates traffic key caching and
preventing deletion of in-use keys.

In this implementation, a centralized database of keys can be thought of
as living in user space and user applications would have to program those
keys on matching sockets. On the server side, the user application programs
keys (MKTS in TCP-AO nomenclature) on the listening socket for all peers that
are expected to connect. Prefix matching on the peer address is supported.
When a peer issues a successful connect, all the MKTs matching the IP address
of the peer are copied to the newly created socket. On the active side,
when a connect() is issued all MKTs that do not match the peer are deleted
from the socket since they will never match the peer. This implementation
uses three setsockopt()s for adding, deleting and modifying keys on a socket.
All three setsockopt()s have extensive sanity checks that prevent
inconsistencies in the keys on a given socket. A getsockopt() is provided
to get key information from any given socket.

Few things to note about this implementation:
- Traffic keys are cached for established connections avoiding the cost of
such calculation for each packet received or sent.
- Great care has been taken to avoid deleting in-use MKTs
as required by the RFC.
- Any crypto algorithm supported by the Linux kernel can be used
to calculate packet hashes.
- Fastopen works with TCP-AO but hasn’t been tested extensively.
- Tested for interop with other major networking vendors (on linux-4.19),
including testing for key rotation and long lived connections.

[1]: https://lore.kernel.org/all/[email protected]/
[2]: https://lore.kernel.org/all/[email protected]/
[3]: https://github.com/0x7f454c46/iperf/tree/tcp-md5-ao
[4]: https://lore.kernel.org/all/166995421700.16716.17446147162780881407.git-patchwork-notify@kernel.org/T/#u
[5]: https://lore.kernel.org/all/[email protected]/T/#u
[6]: https://lore.kernel.org/all/[email protected]/T/#u

Cc: Andy Lutomirski <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Bob Gilligan <[email protected]>
Cc: Dan Carpenter <[email protected]>
Cc: David Ahern <[email protected]>
Cc: David Laight <[email protected]>
Cc: "David S. Miller" <[email protected]>
Cc: Dmitry Safonov <[email protected]>
Cc: Donald Cassidy <[email protected]>
Cc: Eric Biggers <[email protected]>
Cc: Eric Dumazet <[email protected]>
Cc: "Eric W. Biederman" <[email protected]>
Cc: Francesco Ruggeri <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Hideaki YOSHIFUJI <[email protected]>
Cc: Ivan Delalande <[email protected]>
Cc: Jakub Kicinski <[email protected]>
Cc: Leonard Crestez <[email protected]>
Cc: Paolo Abeni <[email protected]>
Cc: Salam Noureddine <[email protected]>
Cc: [email protected]
Cc: [email protected]

Dmitry Safonov (22):
net/tcp: Prepare tcp_md5sig_pool for TCP-AO
net/tcp: Add TCP-AO config and structures
net/tcp: Introduce TCP_AO setsockopt()s
net/tcp: Prevent TCP-MD5 with TCP-AO being set
net/tcp: Calculate TCP-AO traffic keys
net/tcp: Add TCP-AO sign to outgoing packets
net/tcp: Add tcp_parse_auth_options()
net/tcp: Add AO sign to RST packets
net/tcp: Add TCP-AO sign to twsk
net/tcp: Wire TCP-AO to request sockets
net/tcp: Sign SYN-ACK segments with TCP-AO
net/tcp: Verify inbound TCP-AO signed segments
net/tcp: Add TCP-AO segments counters
net/tcp: Add TCP-AO SNE support
net/tcp: Add tcp_hash_fail() ratelimited logs
net/tcp: Ignore specific ICMPs for TCP-AO connections
net/tcp: Add option for TCP-AO to (not) hash header
net/tcp: Add TCP-AO getsockopt()s
net/tcp: Allow asynchronous delete for TCP-AO keys (MKTs)
net/tcp: Add static_key for TCP-AO
net/tcp: Wire up l3index to TCP-AO
net/tcp: Add TCP_AO_REPAIR

include/linux/sockptr.h | 23 +
include/linux/tcp.h | 30 +-
include/net/dropreason-core.h | 30 +
include/net/tcp.h | 224 +++-
include/net/tcp_ao.h | 335 +++++
include/uapi/linux/snmp.h | 5 +
include/uapi/linux/tcp.h | 107 ++
net/ipv4/Kconfig | 17 +
net/ipv4/Makefile | 2 +
net/ipv4/proc.c | 5 +
net/ipv4/syncookies.c | 4 +
net/ipv4/tcp.c | 236 ++--
net/ipv4/tcp_ao.c | 2309 +++++++++++++++++++++++++++++++++
net/ipv4/tcp_input.c | 104 +-
net/ipv4/tcp_ipv4.c | 344 +++--
net/ipv4/tcp_minisocks.c | 35 +-
net/ipv4/tcp_output.c | 227 +++-
net/ipv4/tcp_sigpool.c | 357 +++++
net/ipv6/Makefile | 1 +
net/ipv6/syncookies.c | 5 +
net/ipv6/tcp_ao.c | 156 +++
net/ipv6/tcp_ipv6.c | 336 +++--
22 files changed, 4517 insertions(+), 375 deletions(-)
create mode 100644 include/net/tcp_ao.h
create mode 100644 net/ipv4/tcp_ao.c
create mode 100644 net/ipv4/tcp_sigpool.c
create mode 100644 net/ipv6/tcp_ao.c


base-commit: b6dad5178ceaf23f369c3711062ce1f2afc33644
--
2.40.0

2023-06-14 23:19:03

by Dmitry Safonov

Subject: [PATCH v7 16/22] net/tcp: Ignore specific ICMPs for TCP-AO connections

Similarly to IPsec, RFC5925 prescribes:
">> A TCP-AO implementation MUST default to ignore incoming ICMPv4
messages of Type 3 (destination unreachable), Codes 2-4 (protocol
unreachable, port unreachable, and fragmentation needed -- ’hard
errors’), and ICMPv6 Type 1 (destination unreachable), Code 1
(administratively prohibited) and Code 4 (port unreachable) intended
for connections in synchronized states (ESTABLISHED, FIN-WAIT-1, FIN-
WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT) that match MKTs."

A selftest (later in patch series) verifies that this attack is not
possible in this TCP-AO implementation.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp_ao.h | 10 ++++++-
include/uapi/linux/snmp.h | 1 +
include/uapi/linux/tcp.h | 4 ++-
net/ipv4/proc.c | 1 +
net/ipv4/tcp_ao.c | 62 +++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_ipv4.c | 5 ++++
net/ipv6/tcp_ipv6.c | 4 +++
7 files changed, 85 insertions(+), 2 deletions(-)

diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 5397ba6f84b6..b97e297fe080 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -25,6 +25,7 @@ struct tcp_ao_counters {
atomic64_t pkt_bad;
atomic64_t key_not_found;
atomic64_t ao_required;
+ atomic64_t dropped_icmp;
};

struct tcp_ao_key {
@@ -94,7 +95,8 @@ struct tcp_ao_info {
struct tcp_ao_key *rnext_key;
struct tcp_ao_counters counters;
u32 ao_required :1,
- __unused :31;
+ accept_icmps :1,
+ __unused :30;
__be32 lisn;
__be32 risn;
u32 snd_sne;
@@ -174,6 +176,7 @@ int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
void tcp_ao_destroy_sock(struct sock *sk, bool twsk);
u32 tcp_ao_compute_sne(u32 sne, u32 seq, u32 new_seq);
void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp);
+bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code);
enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
const struct request_sock *req,
@@ -252,6 +255,11 @@ static inline void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
{
}

+static inline bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code)
+{
+ return false;
+}
+
static inline enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
const struct request_sock *req, const struct tcp_ao_hdr *aoh)
diff --git a/include/uapi/linux/snmp.h b/include/uapi/linux/snmp.h
index 06ddf4cd295c..47a6b47da66f 100644
--- a/include/uapi/linux/snmp.h
+++ b/include/uapi/linux/snmp.h
@@ -300,6 +300,7 @@ enum
LINUX_MIB_TCPAOBAD, /* TCPAOBad */
LINUX_MIB_TCPAOKEYNOTFOUND, /* TCPAOKeyNotFound */
LINUX_MIB_TCPAOGOOD, /* TCPAOGood */
+ LINUX_MIB_TCPAODROPPEDICMPS, /* TCPAODroppedIcmps */
__LINUX_MIB_MAX
};

diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 3fe0612ec59a..ca7ed18ce67b 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -392,7 +392,8 @@ struct tcp_ao_info_opt { /* setsockopt(TCP_AO_INFO) */
set_rnext :1, /* corresponding ::rnext */
ao_required :1, /* don't accept non-AO connects */
set_counters :1, /* set/clear ::pkt_* counters */
- reserved :28; /* must be 0 */
+ accept_icmps :1, /* accept incoming ICMPs */
+ reserved :27; /* must be 0 */
__u16 reserved2; /* padding, must be 0 */
__u8 current_key; /* KeyID to set as Current_key */
__u8 rnext; /* KeyID to set as Rnext_key */
@@ -400,6 +401,7 @@ struct tcp_ao_info_opt { /* setsockopt(TCP_AO_INFO) */
__u64 pkt_bad; /* failed verification */
__u64 pkt_key_not_found; /* could not find a key to verify */
__u64 pkt_ao_required; /* segments missing TCP-AO sign */
+ __u64 pkt_dropped_icmp; /* ICMPs that were ignored */
} __attribute__((aligned(8)));

/* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */
diff --git a/net/ipv4/proc.c b/net/ipv4/proc.c
index 3f643cd29cfe..5d3c9c96773e 100644
--- a/net/ipv4/proc.c
+++ b/net/ipv4/proc.c
@@ -302,6 +302,7 @@ static const struct snmp_mib snmp4_net_list[] = {
SNMP_MIB_ITEM("TCPAOBad", LINUX_MIB_TCPAOBAD),
SNMP_MIB_ITEM("TCPAOKeyNotFound", LINUX_MIB_TCPAOKEYNOTFOUND),
SNMP_MIB_ITEM("TCPAOGood", LINUX_MIB_TCPAOGOOD),
+ SNMP_MIB_ITEM("TCPAODroppedIcmps", LINUX_MIB_TCPAODROPPEDICMPS),
SNMP_MIB_SENTINEL
};

diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 1c358889ac10..af4934cb23a5 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -15,6 +15,7 @@

#include <net/tcp.h>
#include <net/ipv6.h>
+#include <net/icmp.h>

int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
unsigned int len)
@@ -52,6 +53,64 @@ int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
return 1;
}

+bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code)
+{
+ struct tcp_ao_info *ao;
+ bool ignore_icmp = false;
+
+ /* RFC5925, 7.8:
+ * >> A TCP-AO implementation MUST default to ignore incoming ICMPv4
+ * messages of Type 3 (destination unreachable), Codes 2-4 (protocol
+ * unreachable, port unreachable, and fragmentation needed -- ’hard
+ * errors’), and ICMPv6 Type 1 (destination unreachable), Code 1
+ * (administratively prohibited) and Code 4 (port unreachable) intended
+ * for connections in synchronized states (ESTABLISHED, FIN-WAIT-1, FIN-
+ * WAIT-2, CLOSE-WAIT, CLOSING, LAST-ACK, TIME-WAIT) that match MKTs.
+ */
+ if (sk->sk_family == AF_INET) {
+ if (type != ICMP_DEST_UNREACH)
+ return false;
+ if (code < ICMP_PROT_UNREACH || code > ICMP_FRAG_NEEDED)
+ return false;
+ } else if (sk->sk_family == AF_INET6) {
+ if (type != ICMPV6_DEST_UNREACH)
+ return false;
+ if (code != ICMPV6_ADM_PROHIBITED && code != ICMPV6_PORT_UNREACH)
+ return false;
+ } else {
+ WARN_ON_ONCE(1);
+ return false;
+ }
+
+ rcu_read_lock();
+ switch (sk->sk_state) {
+ case TCP_TIME_WAIT:
+ ao = rcu_dereference(tcp_twsk(sk)->ao_info);
+ break;
+ case TCP_SYN_SENT:
+ case TCP_SYN_RECV:
+ case TCP_LISTEN:
+ case TCP_NEW_SYN_RECV:
+ /* RFC5925 specifies to ignore ICMPs *only* on connections
+ * in synchronized states.
+ */
+ rcu_read_unlock();
+ return false;
+ default:
+ ao = rcu_dereference(tcp_sk(sk)->ao_info);
+ }
+
+ if (ao && !ao->accept_icmps) {
+ ignore_icmp = true;
+ __NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAODROPPEDICMPS);
+ atomic64_inc(&ao->counters.dropped_icmp);
+ }
+ rcu_read_unlock();
+
+ return ignore_icmp;
+}
+EXPORT_SYMBOL_GPL(tcp_ao_ignore_icmp);
+
/* Optimized version of tcp_ao_do_lookup(): only for sockets for which
* it's known that the keys in ao_info are matching peer's
* family/address/port/VRF/etc.
@@ -1008,6 +1067,7 @@ int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
new_ao->lisn = htonl(tcp_rsk(req)->snt_isn);
new_ao->risn = htonl(tcp_rsk(req)->rcv_isn);
new_ao->ao_required = ao->ao_required;
+ new_ao->accept_icmps = ao->accept_icmps;

if (family == AF_INET) {
addr = (union tcp_ao_addr *)&newsk->sk_daddr;
@@ -1701,9 +1761,11 @@ static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
atomic64_set(&ao_info->counters.pkt_bad, cmd.pkt_bad);
atomic64_set(&ao_info->counters.key_not_found, cmd.pkt_key_not_found);
atomic64_set(&ao_info->counters.ao_required, cmd.pkt_ao_required);
+ atomic64_set(&ao_info->counters.dropped_icmp, cmd.pkt_dropped_icmp);
}

ao_info->ao_required = cmd.ao_required;
+ ao_info->accept_icmps = cmd.accept_icmps;
if (new_current)
WRITE_ONCE(ao_info->current_key, new_current);
if (new_rnext)
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 9799ab8e7e00..ee055d133e76 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -492,6 +492,8 @@ int tcp_v4_err(struct sk_buff *skb, u32 info)
return -ENOENT;
}
if (sk->sk_state == TCP_TIME_WAIT) {
+ /* To increase the counter of ignored icmps for TCP-AO */
+ tcp_ao_ignore_icmp(sk, type, code);
inet_twsk_put(inet_twsk(sk));
return 0;
}
@@ -506,6 +508,9 @@ int tcp_v4_err(struct sk_buff *skb, u32 info)
}

bh_lock_sock(sk);
+ if (tcp_ao_ignore_icmp(sk, type, code))
+ goto out;
+
/* If too many ICMPs get dropped on busy
* servers this needs to be solved differently.
* We do take care of PMTU discovery (RFC1191) special case :
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 0c45c0020ff9..8831254e1b8c 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -399,6 +399,8 @@ static int tcp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
}

if (sk->sk_state == TCP_TIME_WAIT) {
+ /* To increase the counter of ignored icmps for TCP-AO */
+ tcp_ao_ignore_icmp(sk, type, code);
inet_twsk_put(inet_twsk(sk));
return 0;
}
@@ -410,6 +412,8 @@ static int tcp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
}

bh_lock_sock(sk);
+ if (tcp_ao_ignore_icmp(sk, type, code))
+ goto out;
if (sock_owned_by_user(sk) && type != ICMPV6_PKT_TOOBIG)
__NET_INC_STATS(net, LINUX_MIB_LOCKDROPPEDICMPS);

--
2.40.0

2023-06-14 23:19:03

by Dmitry Safonov

Subject: [PATCH v7 03/22] net/tcp: Introduce TCP_AO setsockopt()s

Add 3 setsockopt()s:
1. TCP_AO_ADD_KEY to add a new Master Key Tuple (MKT) on a socket
2. TCP_AO_DEL_KEY to delete present MKT from a socket
3. TCP_AO_INFO to change flags, Current_key/RNext_key on a TCP-AO sk

Userspace has to introduce keys on every socket it wants to use TCP-AO
option on, similarly to TCP_MD5SIG/TCP_MD5SIG_EXT.
RFC5925 prohibits definition of MKTs that would match the same peer,
so do sanity checks on the data provided by userspace. Be as
conservative as possible, including refusal of defining MKT on
an established connection with no AO, removing the key in-use and etc.

(1) and (2) are to be used by userspace key manager to add/remove keys.
(3) main purpose is to set RNext_key, which (as prescribed by RFC5925)
is the KeyID that will be requested in TCP-AO header from the peer to
sign their segments with.

At this moment the life of ao_info ends in tcp_v4_destroy_sock().

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/linux/sockptr.h | 23 ++
include/net/tcp.h | 6 +
include/net/tcp_ao.h | 18 +-
include/uapi/linux/tcp.h | 46 +++
net/ipv4/Makefile | 1 +
net/ipv4/tcp.c | 17 +
net/ipv4/tcp_ao.c | 786 +++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_ipv4.c | 10 +-
net/ipv6/Makefile | 1 +
net/ipv6/tcp_ao.c | 20 +
net/ipv6/tcp_ipv6.c | 39 +-
11 files changed, 949 insertions(+), 18 deletions(-)
create mode 100644 net/ipv4/tcp_ao.c
create mode 100644 net/ipv6/tcp_ao.c

diff --git a/include/linux/sockptr.h b/include/linux/sockptr.h
index bae5e2369b4f..307961b41541 100644
--- a/include/linux/sockptr.h
+++ b/include/linux/sockptr.h
@@ -55,6 +55,29 @@ static inline int copy_from_sockptr(void *dst, sockptr_t src, size_t size)
return copy_from_sockptr_offset(dst, src, 0, size);
}

+static inline int copy_struct_from_sockptr(void *dst, size_t ksize,
+ sockptr_t src, size_t usize)
+{
+ size_t size = min(ksize, usize);
+ size_t rest = max(ksize, usize) - size;
+
+ if (!sockptr_is_kernel(src))
+ return copy_struct_from_user(dst, ksize, src.user, size);
+
+ if (usize < ksize) {
+ memset(dst + size, 0, rest);
+ } else if (usize > ksize) {
+ char *p = src.kernel;
+
+ while (rest--) {
+ if (*p++)
+ return -E2BIG;
+ }
+ }
+ memcpy(dst, src.kernel, size);
+ return 0;
+}
+
static inline int copy_to_sockptr_offset(sockptr_t dst, size_t offset,
const void *src, size_t size)
{
diff --git a/include/net/tcp.h b/include/net/tcp.h
index c3f1eb550cf1..511514e59772 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2096,6 +2096,12 @@ struct tcp_sock_af_ops {
sockptr_t optval,
int optlen);
#endif
+#ifdef CONFIG_TCP_AO
+ int (*ao_parse)(struct sock *sk,
+ int optname,
+ sockptr_t optval,
+ int optlen);
+#endif
};

struct tcp_request_sock_ops {
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 234ae371043b..916df79450e0 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -83,7 +83,8 @@ struct tcp_ao_info {
*/
struct tcp_ao_key *current_key;
struct tcp_ao_key *rnext_key;
- u32 flags;
+ u32 ao_required :1,
+ __unused :31;
__be32 lisn;
__be32 risn;
u32 snd_sne;
@@ -93,4 +94,19 @@ struct tcp_ao_info {
struct rcu_head rcu;
};

+#ifdef CONFIG_TCP_AO
+int tcp_parse_ao(struct sock *sk, int cmd, unsigned short int family,
+ sockptr_t optval, int optlen);
+void tcp_ao_destroy_sock(struct sock *sk);
+/* ipv4 specific functions */
+int tcp_v4_parse_ao(struct sock *sk, int optname, sockptr_t optval, int optlen);
+/* ipv6 specific functions */
+int tcp_v6_parse_ao(struct sock *sk, int cmd,
+ sockptr_t optval, int optlen);
+#else
+static inline void tcp_ao_destroy_sock(struct sock *sk)
+{
+}
+#endif
+
#endif /* _TCP_AO_H */
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 5655bfe28b8d..250e0ce2cc38 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -129,6 +129,9 @@ enum {

#define TCP_TX_DELAY 37 /* delay outgoing packets by XX usec */

+#define TCP_AO_ADD_KEY 38 /* Add/Set MKT */
+#define TCP_AO_DEL_KEY 39 /* Delete MKT */
+#define TCP_AO_INFO 40 /* Modify TCP-AO per-socket options */

#define TCP_REPAIR_ON 1
#define TCP_REPAIR_OFF 0
@@ -350,6 +353,49 @@ struct tcp_diag_md5sig {

#define TCP_AO_MAXKEYLEN 80

+#define TCP_AO_KEYF_IFINDEX (1 << 0) /* L3 ifindex for VRF */
+
+struct tcp_ao_add { /* setsockopt(TCP_AO_ADD_KEY) */
+ struct __kernel_sockaddr_storage addr; /* peer's address for the key */
+ char alg_name[64]; /* crypto hash algorithm to use */
+ __s32 ifindex; /* L3 dev index for VRF */
+ __u32 set_current :1, /* set key as Current_key at once */
+ set_rnext :1, /* request it from peer with RNext_key */
+ reserved :30; /* must be 0 */
+ __u16 reserved2; /* padding, must be 0 */
+ __u8 prefix; /* peer's address prefix */
+ __u8 sndid; /* SendID for outgoing segments */
+ __u8 rcvid; /* RecvID to match for incoming seg */
+ __u8 maclen; /* length of authentication code (hash) */
+ __u8 keyflags; /* see TCP_AO_KEYF_ */
+ __u8 keylen; /* length of ::key */
+ __u8 key[TCP_AO_MAXKEYLEN];
+} __attribute__((aligned(8)));
+
+struct tcp_ao_del { /* setsockopt(TCP_AO_DEL_KEY) */
+ struct __kernel_sockaddr_storage addr; /* peer's address for the key */
+ __s32 ifindex; /* L3 dev index for VRF */
+ __u32 set_current :1, /* corresponding ::current_key */
+ set_rnext :1, /* corresponding ::rnext */
+ reserved :30; /* must be 0 */
+ __u16 reserved2; /* padding, must be 0 */
+ __u8 prefix; /* peer's address prefix */
+ __u8 sndid; /* SendID for outgoing segments */
+ __u8 rcvid; /* RecvID to match for incoming seg */
+ __u8 current_key; /* KeyID to set as Current_key */
+ __u8 rnext; /* KeyID to set as Rnext_key */
+ __u8 keyflags; /* see TCP_AO_KEYF_ */
+} __attribute__((aligned(8)));
+
+struct tcp_ao_info_opt { /* setsockopt(TCP_AO_INFO) */
+ __u32 set_current :1, /* corresponding ::current_key */
+ set_rnext :1, /* corresponding ::rnext */
+ ao_required :1, /* don't accept non-AO connects */
+ reserved :29; /* must be 0 */
+ __u8 current_key; /* KeyID to set as Current_key */
+ __u8 rnext; /* KeyID to set as Rnext_key */
+} __attribute__((aligned(8)));
+
/* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */

#define TCP_RECEIVE_ZEROCOPY_FLAG_TLB_CLEAN_HINT 0x1
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index cd760793cfcb..e144a02a6a61 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -69,6 +69,7 @@ obj-$(CONFIG_NETLABEL) += cipso_ipv4.o

obj-$(CONFIG_XFRM) += xfrm4_policy.o xfrm4_state.o xfrm4_input.o \
xfrm4_output.o xfrm4_protocol.o
+obj-$(CONFIG_TCP_AO) += tcp_ao.o

ifeq ($(CONFIG_BPF_JIT),y)
obj-$(CONFIG_BPF_SYSCALL) += bpf_tcp_ca.o
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 2417a8549142..029e7946ff33 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -3714,6 +3714,23 @@ int do_tcp_setsockopt(struct sock *sk, int level, int optname,
__tcp_sock_set_quickack(sk, val);
break;

+#ifdef CONFIG_TCP_AO
+ case TCP_AO_ADD_KEY:
+ case TCP_AO_DEL_KEY:
+ case TCP_AO_INFO: {
+ /* If this is the first TCP-AO setsockopt() on the socket,
+ * sk_state has to be LISTEN or CLOSE
+ */
+ if (((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)) ||
+ rcu_dereference_protected(tcp_sk(sk)->ao_info,
+ lockdep_sock_is_held(sk)))
+ err = tp->af_specific->ao_parse(sk, optname, optval,
+ optlen);
+ else
+ err = -EISCONN;
+ break;
+ }
+#endif
#ifdef CONFIG_TCP_MD5SIG
case TCP_MD5SIG:
case TCP_MD5SIG_EXT:
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
new file mode 100644
index 000000000000..356747d064e8
--- /dev/null
+++ b/net/ipv4/tcp_ao.c
@@ -0,0 +1,786 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * INET An implementation of the TCP Authentication Option (TCP-AO).
+ * See RFC5925.
+ *
+ * Authors: Dmitry Safonov <[email protected]>
+ * Francesco Ruggeri <[email protected]>
+ * Salam Noureddine <[email protected]>
+ */
+#define pr_fmt(fmt) "TCP: " fmt
+
+#include <crypto/hash.h>
+#include <linux/inetdevice.h>
+#include <linux/tcp.h>
+
+#include <net/tcp.h>
+#include <net/ipv6.h>
+
+/* Optimized version of tcp_ao_do_lookup(): only for sockets for which
+ * it's known that the keys in ao_info are matching peer's
+ * family/address/port/VRF/etc.
+ */
+static struct tcp_ao_key *tcp_ao_established_key(struct tcp_ao_info *ao,
+ int sndid, int rcvid)
+{
+ struct tcp_ao_key *key;
+
+ hlist_for_each_entry_rcu(key, &ao->head, node) {
+ if ((sndid >= 0 && key->sndid != sndid) ||
+ (rcvid >= 0 && key->rcvid != rcvid))
+ continue;
+ return key;
+ }
+
+ return NULL;
+}
+
+static inline int ipv4_prefix_cmp(const struct in_addr *addr1,
+ const struct in_addr *addr2,
+ unsigned int prefixlen)
+{
+ __be32 mask = inet_make_mask(prefixlen);
+
+ if ((addr1->s_addr & mask) == (addr2->s_addr & mask))
+ return 0;
+ return ((addr1->s_addr & mask) > (addr2->s_addr & mask)) ? 1 : -1;
+}
+
+static int __tcp_ao_key_cmp(const struct tcp_ao_key *key,
+ const union tcp_ao_addr *addr, u8 prefixlen,
+ int family, int sndid, int rcvid, u16 port)
+{
+ if (sndid >= 0 && key->sndid != sndid)
+ return (key->sndid > sndid) ? 1 : -1;
+ if (rcvid >= 0 && key->rcvid != rcvid)
+ return (key->rcvid > rcvid) ? 1 : -1;
+ if (port != 0 && key->port != 0 && port != key->port)
+ return (key->port > port) ? 1 : -1;
+
+ if (family == AF_UNSPEC)
+ return 0;
+ if (key->family != family)
+ return (key->family > family) ? 1 : -1;
+
+ if (family == AF_INET) {
+ if (key->addr.a4.s_addr == INADDR_ANY)
+ return 0;
+ if (addr->a4.s_addr == INADDR_ANY)
+ return 0;
+ return ipv4_prefix_cmp(&key->addr.a4, &addr->a4, prefixlen);
+#if IS_ENABLED(CONFIG_IPV6)
+ } else {
+ if (ipv6_addr_any(&key->addr.a6) || ipv6_addr_any(&addr->a6))
+ return 0;
+ if (ipv6_prefix_equal(&key->addr.a6, &addr->a6, prefixlen))
+ return 0;
+ return memcmp(&key->addr.a6, &addr->a6, prefixlen);
+#endif
+ }
+ return -1;
+}
+
+static int tcp_ao_key_cmp(const struct tcp_ao_key *key,
+ const union tcp_ao_addr *addr, u8 prefixlen,
+ int family, int sndid, int rcvid, u16 port)
+{
+#if IS_ENABLED(CONFIG_IPV6)
+ if (family == AF_INET6 && ipv6_addr_v4mapped(&addr->a6)) {
+ __be32 addr4 = addr->a6.s6_addr32[3];
+
+ return __tcp_ao_key_cmp(key, (union tcp_ao_addr *)&addr4,
+ prefixlen, AF_INET, sndid, rcvid, port);
+ }
+#endif
+ return __tcp_ao_key_cmp(key, addr, prefixlen, family, sndid, rcvid, port);
+}
+
+static struct tcp_ao_key *__tcp_ao_do_lookup(const struct sock *sk,
+ const union tcp_ao_addr *addr, int family, u8 prefix,
+ int sndid, int rcvid, u16 port)
+{
+ struct tcp_ao_key *key;
+ struct tcp_ao_info *ao;
+
+ ao = rcu_dereference_check(tcp_sk(sk)->ao_info,
+ lockdep_sock_is_held(sk));
+ if (!ao)
+ return NULL;
+
+ hlist_for_each_entry_rcu(key, &ao->head, node) {
+ u8 prefixlen = min(prefix, key->prefixlen);
+
+ if (!tcp_ao_key_cmp(key, addr, prefixlen,
+ family, sndid, rcvid, port))
+ return key;
+ }
+ return NULL;
+}
+
+struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
+ const union tcp_ao_addr *addr,
+ int family, int sndid, int rcvid, u16 port)
+{
+ return __tcp_ao_do_lookup(sk, addr, family, U8_MAX,
+ sndid, rcvid, port);
+}
+EXPORT_SYMBOL_GPL(tcp_ao_do_lookup);
+
+static struct tcp_ao_info *tcp_ao_alloc_info(gfp_t flags)
+{
+ struct tcp_ao_info *ao;
+
+ ao = kzalloc(sizeof(*ao), flags);
+ if (!ao)
+ return NULL;
+ INIT_HLIST_HEAD(&ao->head);
+
+ return ao;
+}
+
+static void tcp_ao_link_mkt(struct tcp_ao_info *ao, struct tcp_ao_key *mkt)
+{
+ hlist_add_head_rcu(&mkt->node, &ao->head);
+}
+
+static void tcp_ao_key_free_rcu(struct rcu_head *head)
+{
+ struct tcp_ao_key *key = container_of(head, struct tcp_ao_key, rcu);
+
+ tcp_sigpool_release(key->tcp_sigpool_id);
+ kfree(key);
+}
+
+void tcp_ao_destroy_sock(struct sock *sk)
+{
+ struct tcp_ao_info *ao;
+ struct tcp_ao_key *key;
+ struct hlist_node *n;
+
+ ao = rcu_dereference_protected(tcp_sk(sk)->ao_info, 1);
+ tcp_sk(sk)->ao_info = NULL;
+
+ if (!ao)
+ return;
+
+ hlist_for_each_entry_safe(key, n, &ao->head, node) {
+ hlist_del_rcu(&key->node);
+ atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
+ call_rcu(&key->rcu, tcp_ao_key_free_rcu);
+ }
+
+ kfree_rcu(ao, rcu);
+}
+
+static bool tcp_ao_can_set_current_rnext(struct sock *sk)
+{
+ /* There aren't current/rnext keys on TCP_LISTEN sockets */
+ if (sk->sk_state == TCP_LISTEN)
+ return false;
+ return true;
+}
+
+static int tcp_ao_verify_port(struct sock *sk, u16 port)
+{
+ struct inet_sock *inet = inet_sk(sk);
+
+ if (port != 0) /* FIXME */
+ return -EINVAL;
+
+ /* Check that MKT port is consistent with socket */
+ if (port != 0 && inet->inet_dport != 0 && port != inet->inet_dport)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int tcp_ao_verify_ipv4(struct sock *sk, struct tcp_ao_add *cmd,
+ union tcp_ao_addr **addr, u16 *port)
+{
+ struct sockaddr_in *sin = (struct sockaddr_in *)&cmd->addr;
+ struct inet_sock *inet = inet_sk(sk);
+
+ if (sin->sin_family != AF_INET)
+ return -EINVAL;
+
+ if (tcp_ao_verify_port(sk, ntohs(sin->sin_port)))
+ return -EINVAL;
+
+ /* Check prefix and trailing 0's in addr */
+ if (cmd->prefix != 0) {
+ __be32 mask;
+
+ if (sin->sin_addr.s_addr == INADDR_ANY)
+ return -EINVAL;
+ if (cmd->prefix > 32)
+ return -EINVAL;
+
+ mask = inet_make_mask(cmd->prefix);
+ if (sin->sin_addr.s_addr & ~mask)
+ return -EINVAL;
+
+ /* Check that MKT address is consistent with socket */
+ if (inet->inet_daddr != INADDR_ANY &&
+ (inet->inet_daddr & mask) != sin->sin_addr.s_addr)
+ return -EINVAL;
+ } else {
+ if (sin->sin_addr.s_addr != INADDR_ANY)
+ return -EINVAL;
+ }
+
+ *addr = (union tcp_ao_addr *)&sin->sin_addr;
+ *port = ntohs(sin->sin_port);
+ return 0;
+}
+
+static int tcp_ao_parse_crypto(struct tcp_ao_add *cmd, struct tcp_ao_key *key)
+{
+ unsigned int syn_tcp_option_space;
+ bool is_kdf_aes_128_cmac = false;
+ struct tcp_sigpool hp;
+ struct crypto_ahash *tfm;
+ int err, pool_id;
+
+ /* Force null-termination of alg_name */
+ cmd->alg_name[ARRAY_SIZE(cmd->alg_name) - 1] = '\0';
+
+ /* RFC5926, 3.1.1.2. KDF_AES_128_CMAC */
+ if (!strcmp("cmac(aes128)", cmd->alg_name)) {
+ strcpy(cmd->alg_name, "cmac(aes)");
+ is_kdf_aes_128_cmac = (cmd->keylen != 16);
+ }
+
+ key->maclen = cmd->maclen ?: 12; /* 12 is the default in RFC5925 */
+
+ /* Check: maclen + tcp-ao header <= (MAX_TCP_OPTION_SPACE - mss
+ * - tstamp - wscale - sackperm),
+ * see tcp_syn_options(), tcp_synack_options(), commit 33ad798c924b.
+ *
+ * In order to allow D-SACK with TCP-AO, the header size should be:
+ * (MAX_TCP_OPTION_SPACE - TCPOLEN_TSTAMP_ALIGNED
+ * - TCPOLEN_SACK_BASE_ALIGNED
+ * - 2 * TCPOLEN_SACK_PERBLOCK) = 8 (maclen = 4),
+ * see tcp_established_options().
+ *
+ * RFC5925, 2.2:
+ * Typical MACs are 96-128 bits (12-16 bytes), but any length
+ * that fits in the header of the segment being authenticated
+ * is allowed.
+ *
+ * RFC5925, 7.6:
+ * TCP-AO continues to consume 16 bytes in non-SYN segments,
+ * leaving a total of 24 bytes for other options, of which
+ * the timestamp consumes 10. This leaves 14 bytes, of which 10
+ * are used for a single SACK block. When two SACK blocks are used,
+ * such as to handle D-SACK, a smaller TCP-AO MAC would be required
+ * to make room for the additional SACK block (i.e., to leave 18
+ * bytes for the D-SACK variant of the SACK option) [RFC2883].
+ * Note that D-SACK is not supportable in TCP MD5 in the presence
+ * of timestamps, because TCP MD5’s MAC length is fixed and too
+ * large to leave sufficient option space.
+ */
+ syn_tcp_option_space = MAX_TCP_OPTION_SPACE;
+ syn_tcp_option_space -= TCPOLEN_TSTAMP_ALIGNED;
+ syn_tcp_option_space -= TCPOLEN_WSCALE_ALIGNED;
+ syn_tcp_option_space -= TCPOLEN_SACKPERM_ALIGNED;
+ if (tcp_ao_len(key) > syn_tcp_option_space)
+ return -EMSGSIZE;
+
+ key->keylen = cmd->keylen;
+ memcpy(key->key, cmd->key, cmd->keylen);
+
+ /* Full TCP header (th->doff << 2) should fit into scratch area,
+ * see tcp_ao_hash_header().
+ */
+ pool_id = tcp_sigpool_alloc_ahash(cmd->alg_name, 60);
+ if (pool_id < 0)
+ return pool_id;
+
+ err = tcp_sigpool_start(pool_id, &hp);
+ if (err)
+ goto err_free_pool;
+
+ tfm = crypto_ahash_reqtfm(hp.req);
+ if (crypto_ahash_alignmask(tfm) > TCP_AO_KEY_ALIGN) {
+ err = -EOPNOTSUPP;
+ goto err_pool_end;
+ }
+
+ if (is_kdf_aes_128_cmac) {
+ void *scratch = hp.scratch;
+ struct scatterlist sg;
+
+ /* Using zero-key of 16 bytes as described in RFC5926 */
+ memset(scratch, 0, 16);
+ sg_init_one(&sg, cmd->key, cmd->keylen);
+
+ err = crypto_ahash_setkey(tfm, scratch, 16);
+ if (err)
+ goto err_pool_end;
+
+ err = crypto_ahash_init(hp.req);
+ if (err)
+ goto err_pool_end;
+
+ ahash_request_set_crypt(hp.req, &sg, key->key, cmd->keylen);
+ err = crypto_ahash_update(hp.req);
+ if (err)
+ goto err_pool_end;
+
+ err |= crypto_ahash_final(hp.req);
+ if (err)
+ goto err_pool_end;
+ key->keylen = 16;
+ }
+
+ err = crypto_ahash_setkey(tfm, key->key, key->keylen);
+ if (err)
+ goto err_pool_end;
+
+ key->digest_size = crypto_ahash_digestsize(tfm);
+ tcp_sigpool_end(&hp);
+
+ /* TODO: remove TCP_AO_MAX_HASH_SIZE in favor of dynamically
+ * allocated buffer.
+ */
+ if (key->digest_size > TCP_AO_MAX_HASH_SIZE) {
+ err = -ENOBUFS;
+ goto err_free_pool;
+ }
+ if (tcp_ao_maclen(key) > key->digest_size) {
+ err = -EINVAL;
+ goto err_free_pool;
+ }
+
+ key->tcp_sigpool_id = pool_id;
+ return 0;
+
+err_pool_end:
+ tcp_sigpool_end(&hp);
+err_free_pool:
+ tcp_sigpool_release(pool_id);
+ return err;
+}
+
+#if IS_ENABLED(CONFIG_IPV6)
+static int tcp_ao_verify_ipv6(struct sock *sk, struct tcp_ao_add *cmd,
+ union tcp_ao_addr **paddr, u16 *port,
+ unsigned short int *family)
+{
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd->addr;
+ struct in6_addr *addr = &sin6->sin6_addr;
+ u8 prefix = cmd->prefix;
+
+ if (sin6->sin6_family != AF_INET6)
+ return -EINVAL;
+
+ if (tcp_ao_verify_port(sk, ntohs(sin6->sin6_port)))
+ return -EINVAL;
+
+ /* Check prefix and trailing 0's in addr */
+ if (cmd->prefix != 0 && ipv6_addr_v4mapped(addr)) {
+ __be32 addr4 = addr->s6_addr32[3];
+ __be32 mask;
+
+ if (prefix > 32 || addr4 == INADDR_ANY)
+ return -EINVAL;
+
+ mask = inet_make_mask(prefix);
+ if (addr4 & ~mask)
+ return -EINVAL;
+
+ /* Check that MKT address is consistent with socket */
+ if (!ipv6_addr_any(&sk->sk_v6_daddr)) {
+ __be32 daddr4 = sk->sk_v6_daddr.s6_addr32[3];
+
+ if (!ipv6_addr_v4mapped(&sk->sk_v6_daddr))
+ return -EINVAL;
+ if ((daddr4 & mask) != addr4)
+ return -EINVAL;
+ }
+
+ *paddr = (union tcp_ao_addr *)&addr->s6_addr32[3];
+ *family = AF_INET;
+ *port = ntohs(sin6->sin6_port);
+ return 0;
+ } else if (cmd->prefix != 0) {
+ struct in6_addr pfx;
+
+ if (ipv6_addr_any(addr) || prefix > 128)
+ return -EINVAL;
+
+ ipv6_addr_prefix(&pfx, addr, prefix);
+ if (ipv6_addr_cmp(&pfx, addr))
+ return -EINVAL;
+
+ /* Check that MKT address is consistent with socket */
+ if (!ipv6_addr_any(&sk->sk_v6_daddr) &&
+ !ipv6_prefix_equal(&sk->sk_v6_daddr, addr, prefix))
+
+ return -EINVAL;
+ } else {
+ if (!ipv6_addr_any(addr))
+ return -EINVAL;
+ }
+
+ *paddr = (union tcp_ao_addr *)addr;
+ *port = ntohs(sin6->sin6_port);
+ return 0;
+}
+#else
+static inline int tcp_ao_verify_ipv6(struct sock *sk, struct tcp_ao_add *cmd,
+ union tcp_ao_addr **paddr, u16 *port,
+ unsigned short int *family)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+
+static struct tcp_ao_info *setsockopt_ao_info(struct sock *sk)
+{
+ if (sk_fullsock(sk)) {
+ return rcu_dereference_protected(tcp_sk(sk)->ao_info,
+ lockdep_sock_is_held(sk));
+ }
+ return ERR_PTR(-ESOCKTNOSUPPORT);
+}
+
+#define TCP_AO_KEYF_ALL (0)
+
+static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
+ sockptr_t optval, int optlen)
+{
+ struct tcp_ao_info *ao_info;
+ union tcp_ao_addr *addr;
+ struct tcp_ao_key *key;
+ struct tcp_ao_add cmd;
+ bool first = false;
+ int ret, size;
+ u16 port;
+
+ if (optlen < sizeof(cmd))
+ return -EINVAL;
+
+ ret = copy_struct_from_sockptr(&cmd, sizeof(cmd), optval, optlen);
+ if (ret)
+ return ret;
+
+ if (cmd.keylen > TCP_AO_MAXKEYLEN)
+ return -EINVAL;
+
+ if (cmd.reserved != 0 || cmd.reserved2 != 0)
+ return -EINVAL;
+
+ if (family == AF_INET)
+ ret = tcp_ao_verify_ipv4(sk, &cmd, &addr, &port);
+ else
+ ret = tcp_ao_verify_ipv6(sk, &cmd, &addr, &port, &family);
+ if (ret)
+ return ret;
+
+ if (cmd.keyflags & ~TCP_AO_KEYF_ALL)
+ return -EINVAL;
+
+ if (cmd.set_current || cmd.set_rnext) {
+ if (!tcp_ao_can_set_current_rnext(sk))
+ return -EINVAL;
+ }
+
+ ao_info = setsockopt_ao_info(sk);
+ if (IS_ERR(ao_info))
+ return PTR_ERR(ao_info);
+
+ if (!ao_info) {
+ ao_info = tcp_ao_alloc_info(GFP_KERNEL);
+ if (!ao_info)
+ return -ENOMEM;
+ first = true;
+ } else {
+ /* Check that neither RecvID nor SendID match any
+ * existing key for the peer, RFC5925 3.1:
+ * > The IDs of MKTs MUST NOT overlap where their
+ * > TCP connection identifiers overlap.
+ */
+ if (__tcp_ao_do_lookup(sk, addr, family,
+ cmd.prefix, -1, cmd.rcvid, port))
+ return -EEXIST;
+ if (__tcp_ao_do_lookup(sk, addr, family,
+ cmd.prefix, cmd.sndid, -1, port))
+ return -EEXIST;
+ }
+
+ /* TODO: We should add twice the key->diget_size instead of the max
+ * so rework this in a way to know the digest_size before allocating
+ * the tcp_ao_key struct.
+ */
+ size = sizeof(struct tcp_ao_key) + (TCP_AO_MAX_HASH_SIZE << 1);
+ key = sock_kmalloc(sk, size, GFP_KERNEL);
+ if (!key) {
+ ret = -ENOMEM;
+ goto err_free_ao;
+ }
+
+ INIT_HLIST_NODE(&key->node);
+ memcpy(&key->addr, addr, (family == AF_INET) ? sizeof(struct in_addr) :
+ sizeof(struct in6_addr));
+ key->port = port;
+ key->prefixlen = cmd.prefix;
+ key->family = family;
+ key->keyflags = cmd.keyflags;
+ key->sndid = cmd.sndid;
+ key->rcvid = cmd.rcvid;
+
+ ret = tcp_ao_parse_crypto(&cmd, key);
+ if (ret < 0)
+ goto err_free_sock;
+
+ tcp_ao_link_mkt(ao_info, key);
+ if (first) {
+ sk_gso_disable(sk);
+ rcu_assign_pointer(tcp_sk(sk)->ao_info, ao_info);
+ }
+
+ if (cmd.set_current)
+ WRITE_ONCE(ao_info->current_key, key);
+ if (cmd.set_rnext)
+ WRITE_ONCE(ao_info->rnext_key, key);
+ return 0;
+
+err_free_sock:
+ atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
+ kfree(key);
+err_free_ao:
+ if (first)
+ kfree(ao_info);
+ return ret;
+}
+
+static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info,
+ struct tcp_ao_key *key,
+ struct tcp_ao_key *new_current,
+ struct tcp_ao_key *new_rnext)
+{
+ int err;
+
+ hlist_del_rcu(&key->node);
+
+ /* At this moment another CPU could have looked this key up
+ * while it was unlinked from the list. Wait for RCU grace period,
+ * after which the key is off-list and can't be looked up again;
+ * the rx path [just before RCU came] might have used it and set it
+ * as current_key (very unlikely).
+ */
+ synchronize_rcu();
+ if (new_current)
+ WRITE_ONCE(ao_info->current_key, new_current);
+ if (new_rnext)
+ WRITE_ONCE(ao_info->rnext_key, new_rnext);
+
+ if (unlikely(READ_ONCE(ao_info->current_key) == key ||
+ READ_ONCE(ao_info->rnext_key) == key)) {
+ err = -EBUSY;
+ goto add_key;
+ }
+
+ atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
+ call_rcu(&key->rcu, tcp_ao_key_free_rcu);
+
+ return 0;
+add_key:
+ hlist_add_head_rcu(&key->node, &ao_info->head);
+ return err;
+}
+
+static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
+ sockptr_t optval, int optlen)
+{
+ struct tcp_ao_key *key, *new_current = NULL, *new_rnext = NULL;
+ struct tcp_ao_info *ao_info;
+ union tcp_ao_addr *addr;
+ struct tcp_ao_del cmd;
+ int err;
+ __u8 prefix;
+ __be16 port;
+ int addr_len;
+
+ if (optlen < sizeof(cmd))
+ return -EINVAL;
+
+ err = copy_struct_from_sockptr(&cmd, sizeof(cmd), optval, optlen);
+ if (err)
+ return err;
+
+ if (cmd.reserved != 0 || cmd.reserved2 != 0)
+ return -EINVAL;
+
+ if (cmd.set_current || cmd.set_rnext) {
+ if (!tcp_ao_can_set_current_rnext(sk))
+ return -EINVAL;
+ }
+
+
+ ao_info = setsockopt_ao_info(sk);
+ if (IS_ERR(ao_info))
+ return PTR_ERR(ao_info);
+ if (!ao_info)
+ return -ENOENT;
+
+ /* For sockets in TCP_CLOSED it's possible set keys that aren't
+ * matching the future peer (address/port/VRF/etc),
+ * tcp_ao_connect_init() will choose a correct matching MKT
+ * if there's any.
+ */
+ if (cmd.set_current) {
+ new_current = tcp_ao_established_key(ao_info, cmd.current_key, -1);
+ if (!new_current)
+ return -ENOENT;
+ }
+ if (cmd.set_rnext) {
+ new_rnext = tcp_ao_established_key(ao_info, -1, cmd.rnext);
+ if (!new_rnext)
+ return -ENOENT;
+ }
+
+ if (family == AF_INET) {
+ struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.addr;
+
+ addr = (union tcp_ao_addr *)&sin->sin_addr;
+ addr_len = sizeof(struct in_addr);
+ port = ntohs(sin->sin_port);
+ } else {
+ struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.addr;
+ struct in6_addr *addr6 = &sin6->sin6_addr;
+
+ if (ipv6_addr_v4mapped(addr6)) {
+ addr = (union tcp_ao_addr *)&addr6->s6_addr32[3];
+ addr_len = sizeof(struct in_addr);
+ family = AF_INET;
+ } else {
+ addr = (union tcp_ao_addr *)addr6;
+ addr_len = sizeof(struct in6_addr);
+ }
+ port = ntohs(sin6->sin6_port);
+ }
+ prefix = cmd.prefix;
+
+ /* We could choose random present key here for current/rnext
+ * but that's less predictable. Let's be strict and don't
+ * allow removing a key that's in use. RFC5925 doesn't
+ * specify how-to coordinate key removal, but says:
+ * "It is presumed that an MKT affecting a particular
+ * connection cannot be destroyed during an active connection"
+ */
+ hlist_for_each_entry_rcu(key, &ao_info->head, node) {
+ if (cmd.sndid != key->sndid ||
+ cmd.rcvid != key->rcvid)
+ continue;
+
+ if (family != key->family ||
+ prefix != key->prefixlen ||
+ port != key->port ||
+ memcmp(addr, &key->addr, addr_len))
+ continue;
+
+ if (key == new_current || key == new_rnext)
+ continue;
+
+ return tcp_ao_delete_key(sk, ao_info, key,
+ new_current, new_rnext);
+ }
+ return -ENOENT;
+}
+
+static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
+ sockptr_t optval, int optlen)
+{
+ struct tcp_ao_info *ao_info;
+ struct tcp_ao_key *new_current = NULL, *new_rnext = NULL;
+ struct tcp_ao_info_opt cmd;
+ bool first = false;
+ int err;
+
+ if (optlen < sizeof(cmd))
+ return -EINVAL;
+
+ err = copy_struct_from_sockptr(&cmd, sizeof(cmd), optval, optlen);
+ if (err)
+ return err;
+
+ if (cmd.set_current || cmd.set_rnext) {
+ if (!tcp_ao_can_set_current_rnext(sk))
+ return -EINVAL;
+ }
+
+ if (cmd.reserved != 0)
+ return -EINVAL;
+
+ ao_info = setsockopt_ao_info(sk);
+ if (IS_ERR(ao_info))
+ return PTR_ERR(ao_info);
+ if (!ao_info) {
+ ao_info = tcp_ao_alloc_info(GFP_KERNEL);
+ if (!ao_info)
+ return -ENOMEM;
+ first = true;
+ }
+
+ /* For sockets in TCP_CLOSED it's possible set keys that aren't
+ * matching the future peer (address/port/VRF/etc),
+ * tcp_ao_connect_init() will choose a correct matching MKT
+ * if there's any.
+ */
+ if (cmd.set_current) {
+ new_current = tcp_ao_established_key(ao_info, cmd.current_key, -1);
+ if (!new_current) {
+ err = -ENOENT;
+ goto out;
+ }
+ }
+ if (cmd.set_rnext) {
+ new_rnext = tcp_ao_established_key(ao_info, -1, cmd.rnext);
+ if (!new_rnext) {
+ err = -ENOENT;
+ goto out;
+ }
+ }
+
+ ao_info->ao_required = cmd.ao_required;
+ if (new_current)
+ WRITE_ONCE(ao_info->current_key, new_current);
+ if (new_rnext)
+ WRITE_ONCE(ao_info->rnext_key, new_rnext);
+ if (first) {
+ sk_gso_disable(sk);
+ rcu_assign_pointer(tcp_sk(sk)->ao_info, ao_info);
+ }
+ return 0;
+out:
+ if (first)
+ kfree(ao_info);
+ return err;
+}
+
+int tcp_parse_ao(struct sock *sk, int cmd, unsigned short int family,
+ sockptr_t optval, int optlen)
+{
+ if (WARN_ON_ONCE(family != AF_INET && family != AF_INET6))
+ return -EAFNOSUPPORT;
+
+ switch (cmd) {
+ case TCP_AO_ADD_KEY:
+ return tcp_ao_add_cmd(sk, family, optval, optlen);
+ case TCP_AO_DEL_KEY:
+ return tcp_ao_del_cmd(sk, family, optval, optlen);
+ case TCP_AO_INFO:
+ return tcp_ao_info_cmd(sk, family, optval, optlen);
+ default:
+ WARN_ON_ONCE(1);
+ return -EINVAL;
+ }
+}
+
+int tcp_v4_parse_ao(struct sock *sk, int cmd, sockptr_t optval, int optlen)
+{
+ return tcp_parse_ao(sk, cmd, AF_INET, optval, optlen);
+}
+
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index f9423f5dd4e9..6248af483b67 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -2262,11 +2262,16 @@ const struct inet_connection_sock_af_ops ipv4_specific = {
};
EXPORT_SYMBOL(ipv4_specific);

-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
static const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
+#ifdef CONFIG_TCP_MD5SIG
.md5_lookup = tcp_v4_md5_lookup,
.calc_md5_hash = tcp_v4_md5_hash_skb,
.md5_parse = tcp_v4_parse_md5_keys,
+#endif
+#ifdef CONFIG_TCP_AO
+ .ao_parse = tcp_v4_parse_ao,
+#endif
};
#endif

@@ -2281,7 +2286,7 @@ static int tcp_v4_init_sock(struct sock *sk)

icsk->icsk_af_ops = &ipv4_specific;

-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
tcp_sk(sk)->af_specific = &tcp_sock_ipv4_specific;
#endif

@@ -2332,6 +2337,7 @@ void tcp_v4_destroy_sock(struct sock *sk)
rcu_assign_pointer(tp->md5sig_info, NULL);
}
#endif
+ tcp_ao_destroy_sock(sk);

/* Clean up a referenced TCP bind bucket. */
if (inet_csk(sk)->icsk_bind_hash)
diff --git a/net/ipv6/Makefile b/net/ipv6/Makefile
index 3036a45e8a1e..d283c59df4c1 100644
--- a/net/ipv6/Makefile
+++ b/net/ipv6/Makefile
@@ -52,4 +52,5 @@ obj-$(subst m,y,$(CONFIG_IPV6)) += inet6_hashtables.o
ifneq ($(CONFIG_IPV6),)
obj-$(CONFIG_NET_UDP_TUNNEL) += ip6_udp_tunnel.o
obj-y += mcast_snoop.o
+obj-$(CONFIG_TCP_AO) += tcp_ao.o
endif
diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
new file mode 100644
index 000000000000..d0aa6414b7d9
--- /dev/null
+++ b/net/ipv6/tcp_ao.c
@@ -0,0 +1,20 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * INET An implementation of the TCP Authentication Option (TCP-AO).
+ * See RFC5925.
+ *
+ * Authors: Dmitry Safonov <[email protected]>
+ * Francesco Ruggeri <[email protected]>
+ * Salam Noureddine <[email protected]>
+ */
+#include <linux/tcp.h>
+
+#include <net/tcp.h>
+#include <net/ipv6.h>
+
+int tcp_v6_parse_ao(struct sock *sk, int cmd,
+ sockptr_t optval, int optlen)
+{
+ return tcp_parse_ao(sk, cmd, AF_INET6, optval, optlen);
+}
+EXPORT_SYMBOL_GPL(tcp_v6_parse_ao);
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 7e0c43cb3fd8..84e69d5f2f81 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -76,16 +76,9 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_do_rcv(struct sock *sk, struct sk_buff *skb);

static const struct inet_connection_sock_af_ops ipv6_mapped;
const struct inet_connection_sock_af_ops ipv6_specific;
-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
static const struct tcp_sock_af_ops tcp_sock_ipv6_specific;
static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific;
-#else
-static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
- const struct in6_addr *addr,
- int l3index)
-{
- return NULL;
-}
#endif

/* Helper returning the inet6 address from a given tcp socket.
@@ -243,7 +236,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr,
if (sk_is_mptcp(sk))
mptcpv6_handle_mapped(sk, true);
sk->sk_backlog_rcv = tcp_v4_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
tp->af_specific = &tcp_sock_ipv6_mapped_specific;
#endif

@@ -256,7 +249,7 @@ static int tcp_v6_connect(struct sock *sk, struct sockaddr *uaddr,
if (sk_is_mptcp(sk))
mptcpv6_handle_mapped(sk, false);
sk->sk_backlog_rcv = tcp_v6_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
tp->af_specific = &tcp_sock_ipv6_specific;
#endif
goto failure;
@@ -772,7 +765,13 @@ static int tcp_v6_md5_hash_skb(char *md5_hash,
memset(md5_hash, 0, 16);
return 1;
}
-
+#else /* CONFIG_TCP_MD5SIG */
+static struct tcp_md5sig_key *tcp_v6_md5_do_lookup(const struct sock *sk,
+ const struct in6_addr *addr,
+ int l3index)
+{
+ return NULL;
+}
#endif

static void tcp_v6_init_req(struct request_sock *req,
@@ -1232,7 +1231,7 @@ static struct sock *tcp_v6_syn_recv_sock(const struct sock *sk, struct sk_buff *
if (sk_is_mptcp(newsk))
mptcpv6_handle_mapped(newsk, true);
newsk->sk_backlog_rcv = tcp_v4_do_rcv;
-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
newtp->af_specific = &tcp_sock_ipv6_mapped_specific;
#endif

@@ -1896,11 +1895,16 @@ const struct inet_connection_sock_af_ops ipv6_specific = {
.mtu_reduced = tcp_v6_mtu_reduced,
};

-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
+#ifdef CONFIG_TCP_MD5SIG
.md5_lookup = tcp_v6_md5_lookup,
.calc_md5_hash = tcp_v6_md5_hash_skb,
.md5_parse = tcp_v6_parse_md5_keys,
+#endif
+#ifdef CONFIG_TCP_AO
+ .ao_parse = tcp_v6_parse_ao,
+#endif
};
#endif

@@ -1922,11 +1926,16 @@ static const struct inet_connection_sock_af_ops ipv6_mapped = {
.mtu_reduced = tcp_v4_mtu_reduced,
};

-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
+#ifdef CONFIG_TCP_MD5SIG
.md5_lookup = tcp_v4_md5_lookup,
.calc_md5_hash = tcp_v4_md5_hash_skb,
.md5_parse = tcp_v6_parse_md5_keys,
+#endif
+#ifdef CONFIG_TCP_AO
+ .ao_parse = tcp_v6_parse_ao,
+#endif
};
#endif

@@ -1941,7 +1950,7 @@ static int tcp_v6_init_sock(struct sock *sk)

icsk->icsk_af_ops = &ipv6_specific;

-#ifdef CONFIG_TCP_MD5SIG
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
tcp_sk(sk)->af_specific = &tcp_sock_ipv6_specific;
#endif

--
2.40.0

2023-06-14 23:19:17

by Dmitry Safonov

Subject: [PATCH v7 04/22] net/tcp: Prevent TCP-MD5 with TCP-AO being set

Be as conservative as possible: if there is TCP-MD5 key for a given peer
regardless of L3 interface - don't allow setting TCP-AO key for the same
peer. According to RFC5925, TCP-AO is supposed to replace TCP-MD5 and
there can't be any switch between both on any connected tuple.
Later it can be relaxed, if there's a use, but in the beginning restrict
any intersection.

Note: it's still should be possible to set both TCP-MD5 and TCP-AO keys
on a listening socket for *different* peers.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp.h | 43 +++++++++++++++++++++++++++++++++++++--
include/net/tcp_ao.h | 15 ++++++++++++++
net/ipv4/tcp_ao.c | 40 ++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_ipv4.c | 14 ++++++++++---
net/ipv4/tcp_output.c | 47 +++++++++++++++++++++++++++++++++++++++++++
net/ipv6/tcp_ao.c | 18 +++++++++++++++++
net/ipv6/tcp_ipv6.c | 26 ++++++++++++++++++++----
7 files changed, 194 insertions(+), 9 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 511514e59772..219604c8ae6e 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1707,6 +1707,7 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr,

int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr,
int family, u8 prefixlen, int l3index, u8 flags);
+void tcp_clear_md5_list(struct sock *sk);
struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
const struct sock *addr_sk);

@@ -1715,14 +1716,23 @@ struct tcp_md5sig_key *tcp_v4_md5_lookup(const struct sock *sk,
extern struct static_key_false_deferred tcp_md5_needed;
struct tcp_md5sig_key *__tcp_md5_do_lookup(const struct sock *sk, int l3index,
const union tcp_md5_addr *addr,
- int family);
+ int family, bool any_l3index);
static inline struct tcp_md5sig_key *
tcp_md5_do_lookup(const struct sock *sk, int l3index,
const union tcp_md5_addr *addr, int family)
{
if (!static_branch_unlikely(&tcp_md5_needed.key))
return NULL;
- return __tcp_md5_do_lookup(sk, l3index, addr, family);
+ return __tcp_md5_do_lookup(sk, l3index, addr, family, false);
+}
+
+static inline struct tcp_md5sig_key *
+tcp_md5_do_lookup_any_l3index(const struct sock *sk,
+ const union tcp_md5_addr *addr, int family)
+{
+ if (!static_branch_unlikely(&tcp_md5_needed.key))
+ return NULL;
+ return __tcp_md5_do_lookup(sk, 0, addr, family, true);
}

enum skb_drop_reason
@@ -1740,6 +1750,13 @@ tcp_md5_do_lookup(const struct sock *sk, int l3index,
return NULL;
}

+static inline struct tcp_md5sig_key *
+tcp_md5_do_lookup_any_l3index(const struct sock *sk,
+ const union tcp_md5_addr *addr, int family)
+{
+ return NULL;
+}
+
static inline enum skb_drop_reason
tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
const void *saddr, const void *daddr,
@@ -2101,6 +2118,9 @@ struct tcp_sock_af_ops {
int optname,
sockptr_t optval,
int optlen);
+ struct tcp_ao_key *(*ao_lookup)(const struct sock *sk,
+ struct sock *addr_sk,
+ int sndid, int rcvid);
#endif
};

@@ -2513,4 +2533,23 @@ static inline u64 tcp_transmit_time(const struct sock *sk)
return 0;
}

+static inline bool tcp_ao_required(struct sock *sk, const void *saddr,
+ int family)
+{
+#ifdef CONFIG_TCP_AO
+ struct tcp_ao_info *ao_info;
+ struct tcp_ao_key *ao_key;
+
+ ao_info = rcu_dereference_check(tcp_sk(sk)->ao_info,
+ lockdep_sock_is_held(sk));
+ if (!ao_info)
+ return false;
+
+ ao_key = tcp_ao_do_lookup(sk, saddr, family, -1, -1, 0);
+ if (ao_info->ao_required || ao_key)
+ return true;
+#endif
+ return false;
+}
+
#endif /* _TCP_H */
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 916df79450e0..73f584b499f6 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -98,12 +98,27 @@ struct tcp_ao_info {
int tcp_parse_ao(struct sock *sk, int cmd, unsigned short int family,
sockptr_t optval, int optlen);
void tcp_ao_destroy_sock(struct sock *sk);
+struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
+ const union tcp_ao_addr *addr,
+ int family, int sndid, int rcvid, u16 port);
/* ipv4 specific functions */
int tcp_v4_parse_ao(struct sock *sk, int optname, sockptr_t optval, int optlen);
+struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
+ int sndid, int rcvid);
/* ipv6 specific functions */
+struct tcp_ao_key *tcp_v6_ao_lookup(const struct sock *sk,
+ struct sock *addr_sk,
+ int sndid, int rcvid);
int tcp_v6_parse_ao(struct sock *sk, int cmd,
sockptr_t optval, int optlen);
#else
+static inline struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
+ const union tcp_ao_addr *addr,
+ int family, int sndid, int rcvid, u16 port)
+{
+ return NULL;
+}
+
static inline void tcp_ao_destroy_sock(struct sock *sk)
{
}
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 356747d064e8..b4d0b9f9e10a 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -172,6 +172,14 @@ void tcp_ao_destroy_sock(struct sock *sk)
kfree_rcu(ao, rcu);
}

+struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
+ int sndid, int rcvid)
+{
+ union tcp_ao_addr *addr = (union tcp_ao_addr *)&addr_sk->sk_daddr;
+
+ return tcp_ao_do_lookup(sk, addr, AF_INET, sndid, rcvid, 0);
+}
+
static bool tcp_ao_can_set_current_rnext(struct sock *sk)
{
/* There aren't current/rnext keys on TCP_LISTEN sockets */
@@ -486,6 +494,10 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
return -EINVAL;
}

+ /* Don't allow keys for peers that have a matching TCP-MD5 key */
+ if (tcp_md5_do_lookup_any_l3index(sk, addr, family))
+ return -EKEYREJECTED;
+
ao_info = setsockopt_ao_info(sk);
if (IS_ERR(ao_info))
return PTR_ERR(ao_info);
@@ -690,6 +702,31 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
return -ENOENT;
}

+/* cmd.ao_required makes a socket TCP-AO only.
+ * Don't allow any md5 keys for any l3intf on the socket together with it.
+ * Restricting it early in setsockopt() removes a check for
+ * ao_info->ao_required on inbound tcp segment fast-path.
+ */
+static inline int tcp_ao_required_verify(struct sock *sk)
+{
+#ifdef CONFIG_TCP_MD5SIG
+ const struct tcp_md5sig_info *md5sig;
+
+ if (!static_branch_unlikely(&tcp_md5_needed.key))
+ return 0;
+
+ md5sig = rcu_dereference_check(tcp_sk(sk)->md5sig_info,
+ lockdep_sock_is_held(sk));
+ if (!md5sig)
+ return 0;
+
+ if (rcu_dereference_check(hlist_first_rcu(&md5sig->head),
+ lockdep_sock_is_held(sk)))
+ return 1;
+#endif
+ return 0;
+}
+
static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
sockptr_t optval, int optlen)
{
@@ -724,6 +761,9 @@ static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
first = true;
}

+ if (cmd.ao_required && tcp_ao_required_verify(sk))
+ return -EKEYREJECTED;
+
/* For sockets in TCP_CLOSED it's possible set keys that aren't
* matching the future peer (address/port/VRF/etc),
* tcp_ao_connect_init() will choose a correct matching MKT
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 6248af483b67..b06b286f9ec2 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1074,7 +1074,7 @@ static bool better_md5_match(struct tcp_md5sig_key *old, struct tcp_md5sig_key *
/* Find the Key structure for an address. */
struct tcp_md5sig_key *__tcp_md5_do_lookup(const struct sock *sk, int l3index,
const union tcp_md5_addr *addr,
- int family)
+ int family, bool any_l3index)
{
const struct tcp_sock *tp = tcp_sk(sk);
struct tcp_md5sig_key *key;
@@ -1093,7 +1093,8 @@ struct tcp_md5sig_key *__tcp_md5_do_lookup(const struct sock *sk, int l3index,
lockdep_sock_is_held(sk)) {
if (key->family != family)
continue;
- if (key->flags & TCP_MD5SIG_FLAG_IFINDEX && key->l3index != l3index)
+ if (!any_l3index && key->flags & TCP_MD5SIG_FLAG_IFINDEX &&
+ key->l3index != l3index)
continue;
if (family == AF_INET) {
mask = inet_make_mask(key->prefixlen);
@@ -1305,7 +1306,7 @@ int tcp_md5_do_del(struct sock *sk, const union tcp_md5_addr *addr, int family,
}
EXPORT_SYMBOL(tcp_md5_do_del);

-static void tcp_clear_md5_list(struct sock *sk)
+void tcp_clear_md5_list(struct sock *sk)
{
struct tcp_sock *tp = tcp_sk(sk);
struct tcp_md5sig_key *key;
@@ -1375,6 +1376,12 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
return -EINVAL;

+ /* Don't allow keys for peers that have a matching TCP-AO key.
+ * See the comment in tcp_ao_add_cmd()
+ */
+ if (tcp_ao_required(sk, addr, AF_INET))
+ return -EKEYREJECTED;
+
return tcp_md5_do_add(sk, addr, AF_INET, prefixlen, l3index, flags,
cmd.tcpm_key, cmd.tcpm_keylen);
}
@@ -2270,6 +2277,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
.md5_parse = tcp_v4_parse_md5_keys,
#endif
#ifdef CONFIG_TCP_AO
+ .ao_lookup = tcp_v4_ao_lookup,
.ao_parse = tcp_v4_parse_ao,
#endif
};
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index cfe128b81a01..324b241e95bc 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3839,6 +3839,53 @@ int tcp_connect(struct sock *sk)

tcp_call_bpf(sk, BPF_SOCK_OPS_TCP_CONNECT_CB, 0, NULL);

+#if defined(CONFIG_TCP_MD5SIG) && defined(CONFIG_TCP_AO)
+ /* Has to be checked late, after setting daddr/saddr/ops.
+ * Return error if the peer has both a md5 and a tcp-ao key
+ * configured as this is ambiguous.
+ */
+ if (unlikely(rcu_dereference_protected(tp->md5sig_info,
+ lockdep_sock_is_held(sk)))) {
+ bool needs_md5 = !!tp->af_specific->md5_lookup(sk, sk);
+ bool needs_ao = !!tp->af_specific->ao_lookup(sk, sk, -1, -1);
+ struct tcp_ao_info *ao_info;
+
+ ao_info = rcu_dereference_check(tp->ao_info,
+ lockdep_sock_is_held(sk));
+ if (ao_info) {
+ /* This is an extra check: tcp_ao_required() in
+ * tcp_v{4,6}_parse_md5_keys() should prevent adding
+ * md5 keys on ao_required socket.
+ */
+ needs_ao |= ao_info->ao_required;
+ WARN_ON_ONCE(ao_info->ao_required && needs_md5);
+ }
+ if (needs_md5 && needs_ao)
+ return -EKEYREJECTED;
+
+ /* If we have a matching md5 key and no matching tcp-ao key
+ * then free up ao_info if allocated.
+ */
+ if (needs_md5) {
+ tcp_ao_destroy_sock(sk);
+ } else if (needs_ao) {
+ tcp_clear_md5_list(sk);
+ kfree(rcu_replace_pointer(tp->md5sig_info, NULL,
+ lockdep_sock_is_held(sk)));
+ }
+ }
+#endif
+#ifdef CONFIG_TCP_AO
+ if (unlikely(rcu_dereference_protected(tp->ao_info,
+ lockdep_sock_is_held(sk)))) {
+ /* Don't allow connecting if ao is configured but no
+ * matching key is found.
+ */
+ if (!tp->af_specific->ao_lookup(sk, sk, -1, -1))
+ return -EKEYREJECTED;
+ }
+#endif
+
if (inet_csk(sk)->icsk_af_ops->rebuild_header(sk))
return -EHOSTUNREACH; /* Routing failure or similar. */

diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index d0aa6414b7d9..3d2be5f73cf0 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -12,6 +12,24 @@
#include <net/tcp.h>
#include <net/ipv6.h>

+struct tcp_ao_key *tcp_v6_ao_do_lookup(const struct sock *sk,
+ const struct in6_addr *addr,
+ int sndid, int rcvid)
+{
+ return tcp_ao_do_lookup(sk, (union tcp_ao_addr *)addr, AF_INET6,
+ sndid, rcvid, 0);
+}
+
+struct tcp_ao_key *tcp_v6_ao_lookup(const struct sock *sk,
+ struct sock *addr_sk,
+ int sndid, int rcvid)
+{
+ struct in6_addr *addr = &addr_sk->sk_v6_daddr;
+
+ return tcp_v6_ao_do_lookup(sk, addr, sndid, rcvid);
+}
+EXPORT_SYMBOL_GPL(tcp_v6_ao_lookup);
+
int tcp_v6_parse_ao(struct sock *sk, int cmd,
sockptr_t optval, int optlen)
{
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 84e69d5f2f81..a946a3a66a92 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -603,6 +603,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
{
struct tcp_md5sig cmd;
struct sockaddr_in6 *sin6 = (struct sockaddr_in6 *)&cmd.tcpm_addr;
+ union tcp_ao_addr *addr;
int l3index = 0;
u8 prefixlen;
u8 flags;
@@ -657,13 +658,28 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
if (cmd.tcpm_keylen > TCP_MD5SIG_MAXKEYLEN)
return -EINVAL;

- if (ipv6_addr_v4mapped(&sin6->sin6_addr))
- return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3],
+ if (ipv6_addr_v4mapped(&sin6->sin6_addr)) {
+ addr = (union tcp_md5_addr *)&sin6->sin6_addr.s6_addr32[3];
+
+ /* Don't allow keys for peers that have a matching TCP-AO key.
+ * See the comment in tcp_ao_add_cmd()
+ */
+ if (tcp_ao_required(sk, addr, AF_INET))
+ return -EKEYREJECTED;
+ return tcp_md5_do_add(sk, addr,
AF_INET, prefixlen, l3index, flags,
cmd.tcpm_key, cmd.tcpm_keylen);
+ }

- return tcp_md5_do_add(sk, (union tcp_md5_addr *)&sin6->sin6_addr,
- AF_INET6, prefixlen, l3index, flags,
+ addr = (union tcp_md5_addr *)&sin6->sin6_addr;
+
+ /* Don't allow keys for peers that have a matching TCP-AO key.
+ * See the comment in tcp_ao_add_cmd()
+ */
+ if (tcp_ao_required(sk, addr, AF_INET6))
+ return -EKEYREJECTED;
+
+ return tcp_md5_do_add(sk, addr, AF_INET6, prefixlen, l3index, flags,
cmd.tcpm_key, cmd.tcpm_keylen);
}

@@ -1903,6 +1919,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
.md5_parse = tcp_v6_parse_md5_keys,
#endif
#ifdef CONFIG_TCP_AO
+ .ao_lookup = tcp_v6_ao_lookup,
.ao_parse = tcp_v6_parse_ao,
#endif
};
@@ -1934,6 +1951,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
.md5_parse = tcp_v6_parse_md5_keys,
#endif
#ifdef CONFIG_TCP_AO
+ .ao_lookup = tcp_v6_ao_lookup,
.ao_parse = tcp_v6_parse_ao,
#endif
};
--
2.40.0

2023-06-14 23:19:24

by Dmitry Safonov

Subject: [PATCH v7 21/22] net/tcp: Wire up l3index to TCP-AO

Similarly how TCP_MD5SIG_FLAG_IFINDEX works for TCP-MD5,
TCP_AO_KEYF_IFINDEX is an AO-key flag that binds that MKT to a specified
by L3 ifinndex. Similarly, without this flag the key will work in
the default VRF l3index = 0 for connections.
To prevent AO-keys from overlapping, it's restricted to add key B for a
socket that has key A, which have the same sndid/rcvid and one of
the following is true:
- !(A.keyflags & TCP_AO_KEYF_IFINDEX) or !(B.keyflags & TCP_AO_KEYF_IFINDEX)
so that any key is non-bound to a VRF
- A.l3index == B.l3index
both want to work for the same VRF

Additionally, it's restricted to match TCP-MD5 keys for the same peer
the following way:
|--------------|--------------------|----------------|---------------|
| | MD5 key without | MD5 key | MD5 key |
| | l3index | l3index=0 | l3index=N |
|--------------|--------------------|----------------|---------------|
| TCP-AO key | | | |
| without | reject | reject | reject |
| l3index | | | |
|--------------|--------------------|----------------|---------------|
| TCP-AO key | | | |
| l3index=0 | reject | reject | allow |
|--------------|--------------------|----------------|---------------|
| TCP-AO key | | | |
| l3index=N | reject | allow | reject |
|--------------|--------------------|----------------|---------------|

This is done with the help of tcp_md5_do_lookup_any_l3index() to reject
adding AO key without TCP_AO_KEYF_IFINDEX if there's TCP-MD5 in any VRF.
This is important for case where sysctl_tcp_l3mdev_accept = 1
Similarly, for TCP-AO lookups tcp_ao_do_lookup() may be used with
l3index < 0, so that __tcp_ao_key_cmp() will match TCP-AO key in any VRF.

Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp.h | 11 +--
include/net/tcp_ao.h | 17 +++--
net/ipv4/syncookies.c | 6 +-
net/ipv4/tcp_ao.c | 163 +++++++++++++++++++++++++++++++-----------
net/ipv4/tcp_ipv4.c | 13 ++--
net/ipv6/syncookies.c | 5 +-
net/ipv6/tcp_ao.c | 20 +++---
net/ipv6/tcp_ipv6.c | 15 ++--
8 files changed, 170 insertions(+), 80 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 6060513ab83d..e6c02517254a 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2579,7 +2579,7 @@ static inline int tcp_parse_auth_options(const struct tcphdr *th,
}

static inline bool tcp_ao_required(struct sock *sk, const void *saddr,
- int family, bool stat_inc)
+ int family, int l3index, bool stat_inc)
{
#ifdef CONFIG_TCP_AO
struct tcp_ao_info *ao_info;
@@ -2593,7 +2593,7 @@ static inline bool tcp_ao_required(struct sock *sk, const void *saddr,
if (!ao_info)
return false;

- ao_key = tcp_ao_do_lookup(sk, saddr, family, -1, -1, 0);
+ ao_key = tcp_ao_do_lookup(sk, l3index, saddr, family, -1, -1, 0);
if (ao_info->ao_required || ao_key) {
if (stat_inc) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOREQUIRED);
@@ -2646,21 +2646,22 @@ tcp_inbound_hash(struct sock *sk, const struct request_sock *req,
* the last key is impossible to remove, so there's
* always at least one current_key.
*/
- if (tcp_ao_required(sk, saddr, family, true)) {
+ if (tcp_ao_required(sk, saddr, family, l3index, true)) {
tcp_hash_fail("AO hash is required, but not found",
family, skb, "L3 index %d", l3index);
return SKB_DROP_REASON_TCP_AONOTFOUND;
}
if (unlikely(tcp_md5_do_lookup(sk, l3index, saddr, family))) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
- tcp_hash_fail("MD5 Hash not found", family, skb, "");
+ tcp_hash_fail("MD5 Hash not found",
+ family, skb, "L3 index %d", l3index);
return SKB_DROP_REASON_TCP_MD5NOTFOUND;
}
return SKB_NOT_DROPPED_YET;
}

if (aoh)
- return tcp_inbound_ao_hash(sk, skb, family, req, aoh);
+ return tcp_inbound_ao_hash(sk, skb, family, req, l3index, aoh);

return tcp_inbound_md5_hash(sk, skb, saddr, daddr, family,
l3index, md5_location);
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 714a46e30f3f..3449d093143e 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -33,6 +33,7 @@ struct tcp_ao_key {
union tcp_ao_addr addr;
u8 key[TCP_AO_MAXKEYLEN] __tcp_ao_key_align;
unsigned int tcp_sigpool_id;
+ int l3index;
u16 port;
u8 prefixlen;
u8 family;
@@ -183,9 +184,9 @@ int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen);
int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen);
enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
- const struct request_sock *req,
+ const struct request_sock *req, int l3index,
const struct tcp_ao_hdr *aoh);
-struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
+struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk, int l3index,
const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port);
int tcp_ao_hash_hdr(unsigned short family, char *ao_hash,
@@ -229,9 +230,6 @@ int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
__be32 disn, bool send);
int tcp_v6_ao_calc_key_rsk(struct tcp_ao_key *mkt, u8 *key,
struct request_sock *req);
-struct tcp_ao_key *tcp_v6_ao_do_lookup(const struct sock *sk,
- const struct in6_addr *addr,
- int sndid, int rcvid);
struct tcp_ao_key *tcp_v6_ao_lookup(const struct sock *sk,
struct sock *addr_sk,
int sndid, int rcvid);
@@ -250,12 +248,12 @@ void tcp_ao_finish_connect(struct sock *sk, struct sk_buff *skb);
void tcp_ao_connect_init(struct sock *sk);
void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
struct tcp_request_sock *treq,
- unsigned short int family);
+ unsigned short int family, int l3index);
#else /* CONFIG_TCP_AO */

static inline void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
struct tcp_request_sock *treq,
- unsigned short int family)
+ unsigned short int family, int l3index)
{
}

@@ -266,13 +264,14 @@ static inline bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code)

static inline enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
- const struct request_sock *req, const struct tcp_ao_hdr *aoh)
+ const struct request_sock *req, int l3index,
+ const struct tcp_ao_hdr *aoh)
{
return SKB_NOT_DROPPED_YET;
}

static inline struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
- const union tcp_ao_addr *addr,
+ int l3index, const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port)
{
return NULL;
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 9e3d7083af7d..a994a3405a67 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -337,6 +337,7 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
struct rtable *rt;
__u8 rcv_wscale;
struct flowi4 fl4;
+ int l3index;
u32 tsoff = 0;

if (!READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_syncookies) ||
@@ -394,13 +395,14 @@ struct sock *cookie_v4_check(struct sock *sk, struct sk_buff *skb)
treq->snt_synack = 0;
treq->tfo_listener = false;

- tcp_ao_syncookie(sk, skb, treq, AF_INET);
-
if (IS_ENABLED(CONFIG_SMC))
ireq->smc_ok = 0;

ireq->ir_iif = inet_request_bound_dev_if(sk, skb);

+ l3index = l3mdev_master_ifindex_by_index(sock_net(sk), ireq->ir_iif);
+ tcp_ao_syncookie(sk, skb, treq, AF_INET, l3index);
+
/* We throwed the options of the initial SYN away, so we hope
* the ACK carries the same options again (see RFC1122 4.2.3.8)
*/
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 2c4a31d8f177..d8a02cf0ba3c 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -148,7 +148,7 @@ static inline int ipv4_prefix_cmp(const struct in_addr *addr1,
return ((addr1->s_addr & mask) > (addr2->s_addr & mask)) ? 1 : -1;
}

-static int __tcp_ao_key_cmp(const struct tcp_ao_key *key,
+static int __tcp_ao_key_cmp(const struct tcp_ao_key *key, int l3index,
const union tcp_ao_addr *addr, u8 prefixlen,
int family, int sndid, int rcvid, u16 port)
{
@@ -158,6 +158,10 @@ static int __tcp_ao_key_cmp(const struct tcp_ao_key *key,
return (key->rcvid > rcvid) ? 1 : -1;
if (port != 0 && key->port != 0 && port != key->port)
return (key->port > port) ? 1 : -1;
+ if (l3index >= 0 && (key->keyflags & TCP_AO_KEYF_IFINDEX)) {
+ if (key->l3index != l3index)
+ return (key->l3index > l3index) ? 1 : -1;
+ }

if (family == AF_UNSPEC)
return 0;
@@ -182,7 +186,7 @@ static int __tcp_ao_key_cmp(const struct tcp_ao_key *key,
return -1;
}

-static int tcp_ao_key_cmp(const struct tcp_ao_key *key,
+static int tcp_ao_key_cmp(const struct tcp_ao_key *key, int l3index,
const union tcp_ao_addr *addr, u8 prefixlen,
int family, int sndid, int rcvid, u16 port)
{
@@ -190,14 +194,16 @@ static int tcp_ao_key_cmp(const struct tcp_ao_key *key,
if (family == AF_INET6 && ipv6_addr_v4mapped(&addr->a6)) {
__be32 addr4 = addr->a6.s6_addr32[3];

- return __tcp_ao_key_cmp(key, (union tcp_ao_addr *)&addr4,
+ return __tcp_ao_key_cmp(key, l3index,
+ (union tcp_ao_addr *)&addr4,
prefixlen, AF_INET, sndid, rcvid, port);
}
#endif
- return __tcp_ao_key_cmp(key, addr, prefixlen, family, sndid, rcvid, port);
+ return __tcp_ao_key_cmp(key, l3index, addr,
+ prefixlen, family, sndid, rcvid, port);
}

-static struct tcp_ao_key *__tcp_ao_do_lookup(const struct sock *sk,
+static struct tcp_ao_key *__tcp_ao_do_lookup(const struct sock *sk, int l3index,
const union tcp_ao_addr *addr, int family, u8 prefix,
int sndid, int rcvid, u16 port)
{
@@ -215,18 +221,18 @@ static struct tcp_ao_key *__tcp_ao_do_lookup(const struct sock *sk,
hlist_for_each_entry_rcu(key, &ao->head, node) {
u8 prefixlen = min(prefix, key->prefixlen);

- if (!tcp_ao_key_cmp(key, addr, prefixlen,
+ if (!tcp_ao_key_cmp(key, l3index, addr, prefixlen,
family, sndid, rcvid, port))
return key;
}
return NULL;
}

-struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
+struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk, int l3index,
const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port)
{
- return __tcp_ao_do_lookup(sk, addr, family, U8_MAX,
+ return __tcp_ao_do_lookup(sk, l3index, addr, family, U8_MAX,
sndid, rcvid, port);
}
EXPORT_SYMBOL_GPL(tcp_ao_do_lookup);
@@ -669,18 +675,21 @@ struct tcp_ao_key *tcp_v4_ao_lookup_rsk(const struct sock *sk,
struct request_sock *req,
int sndid, int rcvid)
{
- union tcp_ao_addr *addr =
- (union tcp_ao_addr *)&inet_rsk(req)->ir_rmt_addr;
+ struct inet_request_sock *ireq = inet_rsk(req);
+ union tcp_ao_addr *addr = (union tcp_ao_addr *)&ireq->ir_rmt_addr;
+ int l3index = l3mdev_master_ifindex_by_index(sock_net(sk), ireq->ir_iif);

- return tcp_ao_do_lookup(sk, addr, AF_INET, sndid, rcvid, 0);
+ return tcp_ao_do_lookup(sk, l3index, addr, AF_INET, sndid, rcvid, 0);
}

struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
int sndid, int rcvid)
{
union tcp_ao_addr *addr = (union tcp_ao_addr *)&addr_sk->sk_daddr;
+ int l3index = l3mdev_master_ifindex_by_index(sock_net(sk),
+ addr_sk->sk_bound_dev_if);

- return tcp_ao_do_lookup(sk, addr, AF_INET, sndid, rcvid, 0);
+ return tcp_ao_do_lookup(sk, l3index, addr, AF_INET, sndid, rcvid, 0);
}

int tcp_ao_prepare_reset(const struct sock *sk, struct sk_buff *skb,
@@ -755,24 +764,26 @@ int tcp_ao_prepare_reset(const struct sock *sk, struct sk_buff *skb,

static struct tcp_ao_key *tcp_ao_inbound_lookup(unsigned short int family,
const struct sock *sk, const struct sk_buff *skb,
- int sndid, int rcvid)
+ int sndid, int rcvid, int l3index)
{
if (family == AF_INET) {
const struct iphdr *iph = ip_hdr(skb);

- return tcp_ao_do_lookup(sk, (union tcp_ao_addr *)&iph->saddr,
- AF_INET, sndid, rcvid, 0);
+ return tcp_ao_do_lookup(sk, l3index,
+ (union tcp_ao_addr *)&iph->saddr,
+ AF_INET, sndid, rcvid, 0);
} else {
const struct ipv6hdr *iph = ipv6_hdr(skb);

- return tcp_ao_do_lookup(sk, (union tcp_ao_addr *)&iph->saddr,
- AF_INET6, sndid, rcvid, 0);
+ return tcp_ao_do_lookup(sk, l3index,
+ (union tcp_ao_addr *)&iph->saddr,
+ AF_INET6, sndid, rcvid, 0);
}
}

void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
struct tcp_request_sock *treq,
- unsigned short int family)
+ unsigned short int family, int l3index)
{
const struct tcphdr *th = tcp_hdr(skb);
const struct tcp_ao_hdr *aoh;
@@ -787,7 +798,7 @@ void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
if (!aoh)
return;

- key = tcp_ao_inbound_lookup(family, sk, skb, -1, aoh->keyid);
+ key = tcp_ao_inbound_lookup(family, sk, skb, -1, aoh->keyid, l3index);
if (!key)
/* Key not found, continue without TCP-AO */
return;
@@ -802,7 +813,7 @@ static enum skb_drop_reason
tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
unsigned short int family, struct tcp_ao_info *info,
const struct tcp_ao_hdr *aoh, struct tcp_ao_key *key,
- u8 *traffic_key, u8 *phash, u32 sne)
+ u8 *traffic_key, u8 *phash, u32 sne, int l3index)
{
unsigned char newhash[TCP_AO_MAX_HASH_SIZE] __tcp_ao_key_align;
u8 maclen = aoh->length - sizeof(struct tcp_ao_hdr);
@@ -813,7 +824,8 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
atomic64_inc(&info->counters.pkt_bad);
atomic64_inc(&key->pkt_bad);
tcp_hash_fail("AO hash wrong length", family, skb,
- "%u != %d", maclen, tcp_ao_maclen(key));
+ "%u != %d L3index: %d", maclen,
+ tcp_ao_maclen(key), l3index);
return SKB_DROP_REASON_TCP_AOFAILURE;
}

@@ -824,7 +836,8 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOBAD);
atomic64_inc(&info->counters.pkt_bad);
atomic64_inc(&key->pkt_bad);
- tcp_hash_fail("AO hash mismatch", family, skb, "");
+ tcp_hash_fail("AO hash mismatch", family, skb,
+ "L3index: %d", l3index);
return SKB_DROP_REASON_TCP_AOFAILURE;
}
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOGOOD);
@@ -836,7 +849,7 @@ tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
enum skb_drop_reason
tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
unsigned short int family, const struct request_sock *req,
- const struct tcp_ao_hdr *aoh)
+ int l3index, const struct tcp_ao_hdr *aoh)
{
u8 key_buf[TCP_AO_MAX_HASH_SIZE] __tcp_ao_key_align;
const struct tcphdr *th = tcp_hdr(skb);
@@ -851,7 +864,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
if (!info) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOKEYNOTFOUND);
tcp_hash_fail("AO key not found", family, skb,
- "keyid: %u", aoh->keyid);
+ "keyid: %u L3index: %d", aoh->keyid, l3index);
return SKB_DROP_REASON_TCP_AOUNEXPECTED;
}

@@ -886,7 +899,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
/* Established socket, traffic key are cached */
traffic_key = rcv_other_key(key);
err = tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
- traffic_key, phash, sne);
+ traffic_key, phash, sne, l3index);
if (err)
return err;
current_key = READ_ONCE(info->current_key);
@@ -907,7 +920,7 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
* - request sockets would race on those key pointers
* - tcp_ao_del_cmd() allows async key removal
*/
- key = tcp_ao_inbound_lookup(family, sk, skb, -1, aoh->keyid);
+ key = tcp_ao_inbound_lookup(family, sk, skb, -1, aoh->keyid, l3index);
if (!key)
goto key_not_found;

@@ -945,13 +958,13 @@ tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
traffic_key = key_buf;
tcp_ao_calc_key_skb(key, traffic_key, skb, sisn, disn, family);
return tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
- traffic_key, phash, sne);
+ traffic_key, phash, sne, l3index);

key_not_found:
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPAOKEYNOTFOUND);
atomic64_inc(&info->counters.key_not_found);
tcp_hash_fail("Requested by the peer AO key id not found",
- family, skb, "");
+ family, skb, "L3index: %d", l3index);
return SKB_DROP_REASON_TCP_AOKEYNOTFOUND;
}
EXPORT_SYMBOL_GPL(tcp_inbound_ao_hash);
@@ -980,7 +993,7 @@ void tcp_ao_connect_init(struct sock *sk)
struct tcp_ao_key *key;
struct tcp_sock *tp = tcp_sk(sk);
union tcp_ao_addr *addr;
- int family;
+ int family, l3index;

ao_info = rcu_dereference_protected(tp->ao_info,
lockdep_sock_is_held(sk));
@@ -997,9 +1010,11 @@ void tcp_ao_connect_init(struct sock *sk)
#endif
else
return;
+ l3index = l3mdev_master_ifindex_by_index(sock_net(sk),
+ sk->sk_bound_dev_if);

hlist_for_each_entry_rcu(key, &ao_info->head, node) {
- if (tcp_ao_key_cmp(key, addr, key->prefixlen, family,
+ if (tcp_ao_key_cmp(key, l3index, addr, key->prefixlen, family,
-1, -1, sk->sk_dport) == 0)
continue;

@@ -1063,7 +1078,7 @@ int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
struct hlist_node *key_head;
union tcp_ao_addr *addr;
bool match = false;
- int ret = -ENOMEM;
+ int l3index, ret = -ENOMEM;

ao = rcu_dereference(tcp_sk(sk)->ao_info);
if (!ao)
@@ -1091,9 +1106,11 @@ int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
ret = -EAFNOSUPPORT;
goto free_ao;
}
+ l3index = l3mdev_master_ifindex_by_index(sock_net(newsk),
+ newsk->sk_bound_dev_if);

hlist_for_each_entry_rcu(key, &ao->head, node) {
- if (tcp_ao_key_cmp(key, addr, key->prefixlen, family,
+ if (tcp_ao_key_cmp(key, l3index, addr, key->prefixlen, family,
-1, -1, 0))
continue;

@@ -1435,17 +1452,18 @@ static struct tcp_ao_info *setsockopt_ao_info(struct sock *sk)
return ERR_PTR(-ESOCKTNOSUPPORT);
}

-#define TCP_AO_KEYF_ALL (TCP_AO_KEYF_EXCLUDE_OPT)
+#define TCP_AO_KEYF_ALL (TCP_AO_KEYF_IFINDEX | TCP_AO_KEYF_EXCLUDE_OPT)
+#define TCP_AO_GET_KEYF_VALID (TCP_AO_KEYF_IFINDEX)

static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
sockptr_t optval, int optlen)
{
struct tcp_ao_info *ao_info;
+ int ret, size, l3index = 0;
union tcp_ao_addr *addr;
struct tcp_ao_key *key;
struct tcp_ao_add cmd;
bool first = false;
- int ret, size;
u16 port;

if (optlen < sizeof(cmd))
@@ -1476,9 +1494,46 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
return -EINVAL;
}

+ if (cmd.ifindex && !(cmd.keyflags & TCP_AO_KEYF_IFINDEX))
+ return -EINVAL;
+
+ /* For cmd.tcp_ifindex = 0 the key will apply to the default VRF */
+ if (cmd.keyflags & TCP_AO_KEYF_IFINDEX && cmd.ifindex) {
+ int bound_dev_if = READ_ONCE(sk->sk_bound_dev_if);
+ struct net_device *dev;
+
+ rcu_read_lock();
+ dev = dev_get_by_index_rcu(sock_net(sk), cmd.ifindex);
+ if (dev && netif_is_l3_master(dev))
+ l3index = dev->ifindex;
+ rcu_read_unlock();
+
+ if (!dev || !l3index)
+ return -EINVAL;
+
+ /* It's still possible to bind after adding keys or even
+ * re-bind to a different dev (with CAP_NET_RAW).
+ * So, no reason to return error here, rather try to be
+ * nice and warn the user.
+ */
+ if (bound_dev_if && bound_dev_if != cmd.ifindex)
+ net_warn_ratelimited("AO key ifindex %d != sk bound ifindex %d\n",
+ cmd.ifindex, bound_dev_if);
+ }
+
/* Don't allow keys for peers that have a matching TCP-MD5 key */
- if (tcp_md5_do_lookup_any_l3index(sk, addr, family))
- return -EKEYREJECTED;
+ if (cmd.keyflags & TCP_AO_KEYF_IFINDEX) {
+ /* Non-_exact version of tcp_md5_do_lookup() will
+ * as well match keys that aren't bound to a specific VRF
+ * (that will make them match AO key with
+ * sysctl_tcp_l3dev_accept = 1
+ */
+ if (tcp_md5_do_lookup(sk, l3index, addr, family))
+ return -EKEYREJECTED;
+ } else {
+ if (tcp_md5_do_lookup_any_l3index(sk, addr, family))
+ return -EKEYREJECTED;
+ }

ao_info = setsockopt_ao_info(sk);
if (IS_ERR(ao_info))
@@ -1495,10 +1550,10 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
* > The IDs of MKTs MUST NOT overlap where their
* > TCP connection identifiers overlap.
*/
- if (__tcp_ao_do_lookup(sk, addr, family,
+ if (__tcp_ao_do_lookup(sk, l3index, addr, family,
cmd.prefix, -1, cmd.rcvid, port))
return -EEXIST;
- if (__tcp_ao_do_lookup(sk, addr, family,
+ if (__tcp_ao_do_lookup(sk, l3index, addr, family,
cmd.prefix, cmd.sndid, -1, port))
return -EEXIST;
}
@@ -1523,6 +1578,7 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
key->keyflags = cmd.keyflags;
key->sndid = cmd.sndid;
key->rcvid = cmd.rcvid;
+ key->l3index = l3index;
atomic64_set(&key->pkt_good, 0);
atomic64_set(&key->pkt_bad, 0);

@@ -1610,17 +1666,17 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info,
return err;
}

+#define TCP_AO_DEL_KEYF_ALL (TCP_AO_KEYF_IFINDEX)
static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
sockptr_t optval, int optlen)
{
struct tcp_ao_key *key, *new_current = NULL, *new_rnext = NULL;
+ int err, addr_len, l3index = 0;
struct tcp_ao_info *ao_info;
union tcp_ao_addr *addr;
struct tcp_ao_del cmd;
- int err;
__u8 prefix;
__be16 port;
- int addr_len;

if (optlen < sizeof(cmd))
return -EINVAL;
@@ -1637,6 +1693,16 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
return -EINVAL;
}

+ if (cmd.keyflags & ~TCP_AO_DEL_KEYF_ALL)
+ return -EINVAL;
+
+ /* No sanity check for TCP_AO_KEYF_IFINDEX as if a VRF
+ * was destroyed, there still should be a way to delete keys,
+ * that were bound to that l3intf. So, fail late at lookup stage
+ * if there is no key for that ifindex.
+ */
+ if (cmd.ifindex && !(cmd.keyflags & TCP_AO_KEYF_IFINDEX))
+ return -EINVAL;

ao_info = setsockopt_ao_info(sk);
if (IS_ERR(ao_info))
@@ -1702,6 +1768,13 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
memcmp(addr, &key->addr, addr_len))
continue;

+ if ((cmd.keyflags & TCP_AO_KEYF_IFINDEX) !=
+ (key->keyflags & TCP_AO_KEYF_IFINDEX))
+ continue;
+
+ if (key->l3index != l3index)
+ continue;
+
if (key == new_current || key == new_rnext)
continue;

@@ -1887,10 +1960,10 @@ static int tcp_ao_copy_mkts_to_user(struct tcp_ao_info *ao_info,
struct tcp_ao_key *key, *current_key;
bool do_address_matching = true;
union tcp_ao_addr *addr = NULL;
+ int err, l3index, user_len;
unsigned int max_keys; /* maximum number of keys to copy to user */
size_t out_offset = 0;
size_t bytes_to_write; /* number of bytes to write to user level */
- int err, user_len;
u32 matched_keys; /* keys from ao_info matched so far */
int optlen_out;
u16 port = 0;
@@ -1909,11 +1982,16 @@ static int tcp_ao_copy_mkts_to_user(struct tcp_ao_info *ao_info,

if (opt_in.pkt_good || opt_in.pkt_bad)
return -EINVAL;
+ if (opt_in.keyflags & ~TCP_AO_GET_KEYF_VALID)
+ return -EINVAL;
+ if (opt_in.ifindex && !(opt_in.keyflags & TCP_AO_KEYF_IFINDEX))
+ return -EINVAL;

if (opt_in.reserved != 0)
return -EINVAL;

max_keys = opt_in.nkeys;
+ l3index = (opt_in.keyflags & TCP_AO_KEYF_IFINDEX) ? opt_in.ifindex : -1;

if (opt_in.get_all || opt_in.is_current || opt_in.is_rnext) {
if (opt_in.get_all && (opt_in.is_current || opt_in.is_rnext))
@@ -2015,7 +2093,7 @@ static int tcp_ao_copy_mkts_to_user(struct tcp_ao_info *ao_info,
continue;
}

- if (tcp_ao_key_cmp(key, addr, opt_in.prefix,
+ if (tcp_ao_key_cmp(key, l3index, addr, opt_in.prefix,
opt_in.addr.ss_family,
opt_in.sndid, opt_in.rcvid, port) != 0)
continue;
@@ -2048,6 +2126,7 @@ static int tcp_ao_copy_mkts_to_user(struct tcp_ao_info *ao_info,
opt_out.nkeys = 0;
opt_out.maclen = key->maclen;
opt_out.keylen = key->keylen;
+ opt_out.ifindex = key->l3index;
opt_out.pkt_good = atomic64_read(&key->pkt_good);
opt_out.pkt_bad = atomic64_read(&key->pkt_bad);
memcpy(&opt_out.key, key->key, key->keylen);
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index ee055d133e76..211d0edd6eea 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1073,6 +1073,7 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
struct tcp_ao_key *ao_key = NULL;
const union tcp_md5_addr *addr;
u8 keyid = 0;
+ int l3index;
#ifdef CONFIG_TCP_AO
u8 traffic_key[TCP_AO_MAX_HASH_SIZE] __tcp_ao_key_align;
const struct tcp_ao_hdr *aoh;
@@ -1086,6 +1087,7 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
u32 seq = (sk->sk_state == TCP_LISTEN) ? tcp_rsk(req)->snt_isn + 1 :
tcp_sk(sk)->snd_nxt;
addr = (union tcp_md5_addr *)&ip_hdr(skb)->saddr;
+ l3index = tcp_v4_sdif(skb) ? inet_iif(skb) : 0;

if (tcp_rsk_used_ao(req)) {
#ifdef CONFIG_TCP_AO
@@ -1096,11 +1098,11 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
if (!aoh)
return;

- ao_key = tcp_ao_do_lookup(sk, addr, AF_INET,
+ ao_key = tcp_ao_do_lookup(sk, l3index, addr, AF_INET,
aoh->rnext_keyid, -1, 0);
if (unlikely(!ao_key)) {
/* Send ACK with any matching MKT for the peer */
- ao_key = tcp_ao_do_lookup(sk, addr,
+ ao_key = tcp_ao_do_lookup(sk, l3index, addr,
AF_INET, -1, -1, 0);
/* Matching key disappeared (user removed the key?)
* let the handshake timeout.
@@ -1119,9 +1121,6 @@ static void tcp_v4_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
tcp_v4_ao_calc_key_rsk(ao_key, traffic_key, req);
#endif
} else {
- int l3index;
-
- l3index = tcp_v4_sdif(skb) ? inet_iif(skb) : 0;
md5_key = tcp_md5_do_lookup(sk, l3index, addr, AF_INET);
}
/* RFC 7323 2.3
@@ -1479,6 +1478,7 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
const union tcp_md5_addr *addr;
u8 prefixlen = 32;
int l3index = 0;
+ bool l3flag;
u8 flags;

if (optlen < sizeof(cmd))
@@ -1491,6 +1491,7 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
return -EINVAL;

flags = cmd.tcpm_flags & TCP_MD5SIG_FLAG_IFINDEX;
+ l3flag = cmd.tcpm_flags & TCP_MD5SIG_FLAG_IFINDEX;

if (optname == TCP_MD5SIG_EXT &&
cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
@@ -1528,7 +1529,7 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
/* Don't allow keys for peers that have a matching TCP-AO key.
* See the comment in tcp_ao_add_cmd()
*/
- if (tcp_ao_required(sk, addr, AF_INET, false))
+ if (tcp_ao_required(sk, addr, AF_INET, l3flag ? l3index : -1, false))
return -EKEYREJECTED;

return tcp_md5_do_add(sk, addr, AF_INET, prefixlen, l3index, flags,
diff --git a/net/ipv6/syncookies.c b/net/ipv6/syncookies.c
index ad7a8caa7b2a..500f6ed3b8cf 100644
--- a/net/ipv6/syncookies.c
+++ b/net/ipv6/syncookies.c
@@ -140,6 +140,7 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
struct dst_entry *dst;
__u8 rcv_wscale;
u32 tsoff = 0;
+ int l3index;

if (!READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_syncookies) ||
!th->ack || th->rst)
@@ -214,7 +215,9 @@ struct sock *cookie_v6_check(struct sock *sk, struct sk_buff *skb)
treq->snt_isn = cookie;
treq->ts_off = 0;
treq->txhash = net_tx_rndhash();
- tcp_ao_syncookie(sk, skb, treq, AF_INET6);
+
+ l3index = l3mdev_master_ifindex_by_index(sock_net(sk), ireq->ir_iif);
+ tcp_ao_syncookie(sk, skb, treq, AF_INET6, l3index);

if (IS_ENABLED(CONFIG_SMC))
ireq->smc_ok = 0;
diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index 32e19b2dbda8..16e0fbc08c9f 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -79,21 +79,16 @@ int tcp_v6_ao_calc_key_rsk(struct tcp_ao_key *mkt, u8 *key,
}
EXPORT_SYMBOL_GPL(tcp_v6_ao_calc_key_rsk);

-struct tcp_ao_key *tcp_v6_ao_do_lookup(const struct sock *sk,
- const struct in6_addr *addr,
- int sndid, int rcvid)
-{
- return tcp_ao_do_lookup(sk, (union tcp_ao_addr *)addr, AF_INET6,
- sndid, rcvid, 0);
-}
-
struct tcp_ao_key *tcp_v6_ao_lookup(const struct sock *sk,
struct sock *addr_sk,
int sndid, int rcvid)
{
struct in6_addr *addr = &addr_sk->sk_v6_daddr;
+ int l3index = l3mdev_master_ifindex_by_index(sock_net(sk),
+ addr_sk->sk_bound_dev_if);

- return tcp_v6_ao_do_lookup(sk, addr, sndid, rcvid);
+ return tcp_ao_do_lookup(sk, l3index, (union tcp_ao_addr *)addr,
+ AF_INET6, sndid, rcvid, 0);
}
EXPORT_SYMBOL_GPL(tcp_v6_ao_lookup);

@@ -101,9 +96,12 @@ struct tcp_ao_key *tcp_v6_ao_lookup_rsk(const struct sock *sk,
struct request_sock *req,
int sndid, int rcvid)
{
- struct in6_addr *addr = &inet_rsk(req)->ir_v6_rmt_addr;
+ struct inet_request_sock *ireq = inet_rsk(req);
+ struct in6_addr *addr = &ireq->ir_v6_rmt_addr;
+ int l3index = l3mdev_master_ifindex_by_index(sock_net(sk), ireq->ir_iif);

- return tcp_v6_ao_do_lookup(sk, addr, sndid, rcvid);
+ return tcp_ao_do_lookup(sk, l3index, (union tcp_ao_addr *)addr,
+ AF_INET6, sndid, rcvid, 0);
}
EXPORT_SYMBOL_GPL(tcp_v6_ao_lookup_rsk);

diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 8831254e1b8c..6b39da6c477e 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -610,6 +610,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
union tcp_ao_addr *addr;
int l3index = 0;
u8 prefixlen;
+ bool l3flag;
u8 flags;

if (optlen < sizeof(cmd))
@@ -622,6 +623,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
return -EINVAL;

flags = cmd.tcpm_flags & TCP_MD5SIG_FLAG_IFINDEX;
+ l3flag = cmd.tcpm_flags & TCP_MD5SIG_FLAG_IFINDEX;

if (optname == TCP_MD5SIG_EXT &&
cmd.tcpm_flags & TCP_MD5SIG_FLAG_PREFIX) {
@@ -668,7 +670,8 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
/* Don't allow keys for peers that have a matching TCP-AO key.
* See the comment in tcp_ao_add_cmd()
*/
- if (tcp_ao_required(sk, addr, AF_INET, false))
+ if (tcp_ao_required(sk, addr, AF_INET,
+ l3flag ? l3index : -1, false))
return -EKEYREJECTED;
return tcp_md5_do_add(sk, addr,
AF_INET, prefixlen, l3index, flags,
@@ -680,7 +683,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
/* Don't allow keys for peers that have a matching TCP-AO key.
* See the comment in tcp_ao_add_cmd()
*/
- if (tcp_ao_required(sk, addr, AF_INET6, false))
+ if (tcp_ao_required(sk, addr, AF_INET6, l3flag ? l3index : -1, false))
return -EKEYREJECTED;

return tcp_md5_do_add(sk, addr, AF_INET6, prefixlen, l3index, flags,
@@ -1231,10 +1234,14 @@ static void tcp_v6_reqsk_send_ack(const struct sock *sk, struct sk_buff *skb,
return;
if (!aoh)
return;
- ao_key = tcp_v6_ao_do_lookup(sk, addr, aoh->rnext_keyid, -1);
+ ao_key = tcp_ao_do_lookup(sk, l3index,
+ (union tcp_ao_addr *)addr, AF_INET6,
+ aoh->rnext_keyid, -1, 0);
if (unlikely(!ao_key)) {
/* Send ACK with any matching MKT for the peer */
- ao_key = tcp_v6_ao_do_lookup(sk, addr, -1, -1);
+ ao_key = tcp_ao_do_lookup(sk, l3index,
+ (union tcp_ao_addr *)addr,
+ AF_INET6, -1, -1, 0);
/* Matching key disappeared (user removed the key?)
* let the handshake timeout.
*/
--
2.40.0

2023-06-14 23:19:26

by Dmitry Safonov

Subject: [PATCH v7 02/22] net/tcp: Add TCP-AO config and structures

Introduce new kernel config option and common structures as well as
helpers to be used by TCP-AO code.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/linux/tcp.h | 9 +++-
include/net/tcp.h | 8 +---
include/net/tcp_ao.h | 96 ++++++++++++++++++++++++++++++++++++++++
include/uapi/linux/tcp.h | 2 +
net/ipv4/Kconfig | 13 ++++++
5 files changed, 120 insertions(+), 8 deletions(-)
create mode 100644 include/net/tcp_ao.h

diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index b4c08ac86983..673b32e47c30 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -435,13 +435,18 @@ struct tcp_sock {
bool syn_smc; /* SYN includes SMC */
#endif

-#ifdef CONFIG_TCP_MD5SIG
-/* TCP AF-Specific parts; only used by MD5 Signature support so far */
+#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
+/* TCP AF-Specific parts; only used by TCP-AO/MD5 Signature support so far */
const struct tcp_sock_af_ops *af_specific;

+#ifdef CONFIG_TCP_MD5SIG
/* TCP MD5 Signature Option information */
struct tcp_md5sig_info __rcu *md5sig_info;
#endif
+#ifdef CONFIG_TCP_AO
+ struct tcp_ao_info __rcu *ao_info;
+#endif
+#endif

/* TCP fastopen related information */
struct tcp_fastopen_request *fastopen_req;
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 81134a70c9ba..c3f1eb550cf1 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -37,6 +37,7 @@
#include <net/snmp.h>
#include <net/ip.h>
#include <net/tcp_states.h>
+#include <net/tcp_ao.h>
#include <net/inet_ecn.h>
#include <net/dst.h>
#include <net/mptcp.h>
@@ -1621,12 +1622,7 @@ static inline void tcp_clear_all_retrans_hints(struct tcp_sock *tp)
tp->retransmit_skb_hint = NULL;
}

-union tcp_md5_addr {
- struct in_addr a4;
-#if IS_ENABLED(CONFIG_IPV6)
- struct in6_addr a6;
-#endif
-};
+#define tcp_md5_addr tcp_ao_addr

/* - key database */
struct tcp_md5sig_key {
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
new file mode 100644
index 000000000000..234ae371043b
--- /dev/null
+++ b/include/net/tcp_ao.h
@@ -0,0 +1,96 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+#ifndef _TCP_AO_H
+#define _TCP_AO_H
+
+#define TCP_AO_MAX_HASH_SIZE 64
+#define TCP_AO_KEY_ALIGN 1
+#define __tcp_ao_key_align __aligned(TCP_AO_KEY_ALIGN)
+
+union tcp_ao_addr {
+ struct in_addr a4;
+#if IS_ENABLED(CONFIG_IPV6)
+ struct in6_addr a6;
+#endif
+};
+
+struct tcp_ao_hdr {
+ u8 kind;
+ u8 length;
+ u8 keyid;
+ u8 rnext_keyid;
+};
+
+struct tcp_ao_key {
+ struct hlist_node node;
+ union tcp_ao_addr addr;
+ u8 key[TCP_AO_MAXKEYLEN] __tcp_ao_key_align;
+ unsigned int tcp_sigpool_id;
+ u16 port;
+ u8 prefixlen;
+ u8 family;
+ u8 keylen;
+ u8 keyflags;
+ u8 sndid;
+ u8 rcvid;
+ u8 maclen;
+ u8 digest_size;
+ struct rcu_head rcu;
+ u8 traffic_keys[];
+};
+
+static inline u8 *rcv_other_key(struct tcp_ao_key *key)
+{
+ return key->traffic_keys;
+}
+
+static inline u8 *snd_other_key(struct tcp_ao_key *key)
+{
+ return key->traffic_keys + key->digest_size;
+}
+
+static inline int tcp_ao_maclen(const struct tcp_ao_key *key)
+{
+ return key->maclen;
+}
+
+static inline int tcp_ao_sizeof_key(const struct tcp_ao_key *key)
+{
+ return sizeof(struct tcp_ao_key) + (TCP_AO_MAX_HASH_SIZE << 1);
+}
+
+static inline int tcp_ao_len(const struct tcp_ao_key *key)
+{
+ return tcp_ao_maclen(key) + sizeof(struct tcp_ao_hdr);
+}
+
+static inline unsigned int tcp_ao_digest_size(struct tcp_ao_key *key)
+{
+ return key->digest_size;
+}
+
+struct tcp_ao_info {
+ /* List of tcp_ao_key's */
+ struct hlist_head head;
+ /* current_key and rnext_key aren't maintained on listen sockets.
+ * Their purpose is to cache keys on established connections,
+ * saving needless lookups. Never dereference any of them from
+ * listen sockets.
+ * ::current_key may change in RX to the key that was requested by
+ * the peer, please use READ_ONCE()/WRITE_ONCE() in order to avoid
+ * load/store tearing.
+ * Do the same for ::rnext_key, if you don't hold socket lock
+ * (it's changed only by userspace request in setsockopt()).
+ */
+ struct tcp_ao_key *current_key;
+ struct tcp_ao_key *rnext_key;
+ u32 flags;
+ __be32 lisn;
+ __be32 risn;
+ u32 snd_sne;
+ u32 snd_sne_seq;
+ u32 rcv_sne;
+ u32 rcv_sne_seq;
+ struct rcu_head rcu;
+};
+
+#endif /* _TCP_AO_H */
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 879eeb0a084b..5655bfe28b8d 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -348,6 +348,8 @@ struct tcp_diag_md5sig {
__u8 tcpm_key[TCP_MD5SIG_MAXKEYLEN];
};

+#define TCP_AO_MAXKEYLEN 80
+
/* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */

#define TCP_RECEIVE_ZEROCOPY_FLAG_TLB_CLEAN_HINT 0x1
diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
index 89e2ab023272..2bcda91462c8 100644
--- a/net/ipv4/Kconfig
+++ b/net/ipv4/Kconfig
@@ -744,6 +744,19 @@ config DEFAULT_TCP_CONG
config TCP_SIGPOOL
tristate

+config TCP_AO
+ bool "TCP: Authentication Option (RFC5925)"
+ select CRYPTO
+ select TCP_SIGPOOL
+ depends on 64BIT # seq-number extension needs WRITE_ONCE(u64)
+ help
+ TCP-AO specifies the use of stronger Message Authentication Codes (MACs),
+ protects against replays for long-lived TCP connections, and
+ provides more details on the association of security with TCP
+ connections than TCP MD5 (See RFC5925)
+
+ If unsure, say N.
+
config TCP_MD5SIG
bool "TCP: MD5 Signature Option support (RFC2385)"
select CRYPTO
--
2.40.0

2023-06-14 23:19:38

by Dmitry Safonov

Subject: [PATCH v7 05/22] net/tcp: Calculate TCP-AO traffic keys

Add traffic key calculation the way it's described in RFC5926.
Wire it up to tcp_finish_connect() and cache the new keys straight away
on already established TCP connections.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp.h | 5 ++
include/net/tcp_ao.h | 42 ++++++++-
net/ipv4/tcp_ao.c | 196 ++++++++++++++++++++++++++++++++++++++++++
net/ipv4/tcp_input.c | 1 +
net/ipv4/tcp_ipv4.c | 1 +
net/ipv4/tcp_output.c | 1 +
net/ipv6/tcp_ao.c | 40 +++++++++
net/ipv6/tcp_ipv6.c | 1 +
8 files changed, 286 insertions(+), 1 deletion(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 219604c8ae6e..799829adbb5a 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2121,6 +2121,11 @@ struct tcp_sock_af_ops {
struct tcp_ao_key *(*ao_lookup)(const struct sock *sk,
struct sock *addr_sk,
int sndid, int rcvid);
+ int (*ao_calc_key_sk)(struct tcp_ao_key *mkt,
+ u8 *key,
+ const struct sock *sk,
+ __be32 sisn, __be32 disn,
+ bool send);
#endif
};

diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 73f584b499f6..1172d9d9517a 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -95,8 +95,30 @@ struct tcp_ao_info {
};

#ifdef CONFIG_TCP_AO
+/* TCP-AO structures and functions */
+
+struct tcp4_ao_context {
+ __be32 saddr;
+ __be32 daddr;
+ __be16 sport;
+ __be16 dport;
+ __be32 sisn;
+ __be32 disn;
+};
+
+struct tcp6_ao_context {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ __be16 sport;
+ __be16 dport;
+ __be32 sisn;
+ __be32 disn;
+};
+
int tcp_parse_ao(struct sock *sk, int cmd, unsigned short int family,
sockptr_t optval, int optlen);
+int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
+ unsigned int len);
void tcp_ao_destroy_sock(struct sock *sk);
struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
const union tcp_ao_addr *addr,
@@ -105,13 +127,23 @@ struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
int tcp_v4_parse_ao(struct sock *sk, int optname, sockptr_t optval, int optlen);
struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
int sndid, int rcvid);
+int tcp_v4_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
+ const struct sock *sk,
+ __be32 sisn, __be32 disn, bool send);
/* ipv6 specific functions */
+int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
+ const struct sock *sk, __be32 sisn,
+ __be32 disn, bool send);
struct tcp_ao_key *tcp_v6_ao_lookup(const struct sock *sk,
struct sock *addr_sk,
int sndid, int rcvid);
int tcp_v6_parse_ao(struct sock *sk, int cmd,
sockptr_t optval, int optlen);
-#else
+void tcp_ao_finish_connect(struct sock *sk, struct sk_buff *skb);
+void tcp_ao_connect_init(struct sock *sk);
+
+#else /* CONFIG_TCP_AO */
+
static inline struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port)
@@ -122,6 +154,14 @@ static inline struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
static inline void tcp_ao_destroy_sock(struct sock *sk)
{
}
+
+static inline void tcp_ao_finish_connect(struct sock *sk, struct sk_buff *skb)
+{
+}
+
+static inline void tcp_ao_connect_init(struct sock *sk)
+{
+}
#endif

#endif /* _TCP_AO_H */
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index b4d0b9f9e10a..24f27c790a51 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -16,6 +16,42 @@
#include <net/tcp.h>
#include <net/ipv6.h>

+int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
+ unsigned int len)
+{
+ struct tcp_sigpool hp;
+ struct scatterlist sg;
+ int ret;
+
+ if (tcp_sigpool_start(mkt->tcp_sigpool_id, &hp))
+ goto clear_hash_noput;
+
+ if (crypto_ahash_setkey(crypto_ahash_reqtfm(hp.req),
+ mkt->key, mkt->keylen))
+ goto clear_hash;
+
+ ret = crypto_ahash_init(hp.req);
+ if (ret)
+ goto clear_hash;
+
+ sg_init_one(&sg, ctx, len);
+ ahash_request_set_crypt(hp.req, &sg, key, len);
+ crypto_ahash_update(hp.req);
+
+ /* TODO: Revisit on how to get different output length */
+ ret = crypto_ahash_final(hp.req);
+ if (ret)
+ goto clear_hash;
+
+ tcp_sigpool_end(&hp);
+ return 0;
+clear_hash:
+ tcp_sigpool_end(&hp);
+clear_hash_noput:
+ memset(key, 0, tcp_ao_digest_size(mkt));
+ return 1;
+}
+
/* Optimized version of tcp_ao_do_lookup(): only for sockets for which
* it's known that the keys in ao_info are matching peer's
* family/address/port/VRF/etc.
@@ -172,6 +208,62 @@ void tcp_ao_destroy_sock(struct sock *sk)
kfree_rcu(ao, rcu);
}

+/* 4 tuple and ISNs are expected in NBO */
+static int tcp_v4_ao_calc_key(struct tcp_ao_key *mkt, u8 *key,
+ __be32 saddr, __be32 daddr,
+ __be16 sport, __be16 dport,
+ __be32 sisn, __be32 disn)
+{
+ /* See RFC5926 3.1.1 */
+ struct kdf_input_block {
+ u8 counter;
+ u8 label[6];
+ struct tcp4_ao_context ctx;
+ __be16 outlen;
+ } __packed tmp;
+
+ tmp.counter = 1;
+ memcpy(tmp.label, "TCP-AO", 6);
+ tmp.ctx.saddr = saddr;
+ tmp.ctx.daddr = daddr;
+ tmp.ctx.sport = sport;
+ tmp.ctx.dport = dport;
+ tmp.ctx.sisn = sisn;
+ tmp.ctx.disn = disn;
+ tmp.outlen = htons(tcp_ao_digest_size(mkt) * 8); /* in bits */
+
+ return tcp_ao_calc_traffic_key(mkt, key, &tmp, sizeof(tmp));
+}
+
+int tcp_v4_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
+ const struct sock *sk,
+ __be32 sisn, __be32 disn, bool send)
+{
+ if (send)
+ return tcp_v4_ao_calc_key(mkt, key, sk->sk_rcv_saddr,
+ sk->sk_daddr, htons(sk->sk_num),
+ sk->sk_dport, sisn, disn);
+ else
+ return tcp_v4_ao_calc_key(mkt, key, sk->sk_daddr,
+ sk->sk_rcv_saddr, sk->sk_dport,
+ htons(sk->sk_num), disn, sisn);
+}
+EXPORT_SYMBOL_GPL(tcp_v4_ao_calc_key_sk);
+
+static int tcp_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
+ const struct sock *sk,
+ __be32 sisn, __be32 disn, bool send)
+{
+ if (mkt->family == AF_INET)
+ return tcp_v4_ao_calc_key_sk(mkt, key, sk, sisn, disn, send);
+#if IS_ENABLED(CONFIG_IPV6)
+ else if (mkt->family == AF_INET6)
+ return tcp_v6_ao_calc_key_sk(mkt, key, sk, sisn, disn, send);
+#endif
+ else
+ return -EOPNOTSUPP;
+}
+
struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
int sndid, int rcvid)
{
@@ -180,6 +272,104 @@ struct tcp_ao_key *tcp_v4_ao_lookup(const struct sock *sk, struct sock *addr_sk,
return tcp_ao_do_lookup(sk, addr, AF_INET, sndid, rcvid, 0);
}

+static int tcp_ao_cache_traffic_keys(const struct sock *sk,
+ struct tcp_ao_info *ao,
+ struct tcp_ao_key *ao_key)
+{
+ u8 *traffic_key = snd_other_key(ao_key);
+ int ret;
+
+ ret = tcp_ao_calc_key_sk(ao_key, traffic_key, sk,
+ ao->lisn, ao->risn, true);
+ if (ret)
+ return ret;
+
+ traffic_key = rcv_other_key(ao_key);
+ ret = tcp_ao_calc_key_sk(ao_key, traffic_key, sk,
+ ao->lisn, ao->risn, false);
+ return ret;
+}
+
+void tcp_ao_connect_init(struct sock *sk)
+{
+ struct tcp_ao_info *ao_info;
+ struct tcp_ao_key *key;
+ struct tcp_sock *tp = tcp_sk(sk);
+ union tcp_ao_addr *addr;
+ int family;
+
+ ao_info = rcu_dereference_protected(tp->ao_info,
+ lockdep_sock_is_held(sk));
+ if (!ao_info)
+ return;
+
+ /* Remove all keys that don't match the peer */
+ family = sk->sk_family;
+ if (family == AF_INET)
+ addr = (union tcp_ao_addr *)&sk->sk_daddr;
+#if IS_ENABLED(CONFIG_IPV6)
+ else if (family == AF_INET6)
+ addr = (union tcp_ao_addr *)&sk->sk_v6_daddr;
+#endif
+ else
+ return;
+
+ hlist_for_each_entry_rcu(key, &ao_info->head, node) {
+ if (tcp_ao_key_cmp(key, addr, key->prefixlen, family,
+ -1, -1, sk->sk_dport) == 0)
+ continue;
+
+ if (key == ao_info->current_key)
+ ao_info->current_key = NULL;
+ if (key == ao_info->rnext_key)
+ ao_info->rnext_key = NULL;
+ hlist_del_rcu(&key->node);
+ tcp_sigpool_release(key->tcp_sigpool_id);
+ atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
+ kfree_rcu(key, rcu);
+ }
+
+ key = tp->af_specific->ao_lookup(sk, sk, -1, -1);
+ if (key) {
+ /* if current_key or rnext_key were not provided,
+ * use the first key matching the peer
+ */
+ if (!ao_info->current_key)
+ ao_info->current_key = key;
+ if (!ao_info->rnext_key)
+ ao_info->rnext_key = key;
+ tp->tcp_header_len += tcp_ao_len(key);
+
+ ao_info->lisn = htonl(tp->write_seq);
+ ao_info->snd_sne = 0;
+ ao_info->snd_sne_seq = tp->write_seq;
+ } else {
+ /* TODO: probably, it should fail to connect() here */
+ rcu_assign_pointer(tp->ao_info, NULL);
+ kfree(ao_info);
+ }
+}
+
+void tcp_ao_finish_connect(struct sock *sk, struct sk_buff *skb)
+{
+ struct tcp_ao_info *ao;
+ struct tcp_ao_key *key;
+
+ ao = rcu_dereference_protected(tcp_sk(sk)->ao_info,
+ lockdep_sock_is_held(sk));
+ if (!ao)
+ return;
+
+ ao->risn = tcp_hdr(skb)->seq;
+
+ ao->rcv_sne = 0;
+ ao->rcv_sne_seq = ntohl(tcp_hdr(skb)->seq);
+
+ hlist_for_each_entry_rcu(key, &ao->head, node) {
+ tcp_ao_cache_traffic_keys(sk, ao, key);
+ }
+}
+
static bool tcp_ao_can_set_current_rnext(struct sock *sk)
{
/* There aren't current/rnext keys on TCP_LISTEN sockets */
@@ -546,6 +736,12 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
if (ret < 0)
goto err_free_sock;

+ /* Change this condition if we allow adding keys in states
+ * like close_wait, syn_sent or fin_wait...
+ */
+ if (sk->sk_state == TCP_ESTABLISHED)
+ tcp_ao_cache_traffic_keys(sk, ao_info, key);
+
tcp_ao_link_mkt(ao_info, key);
if (first) {
sk_gso_disable(sk);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index bf8b22218dd4..d959ec746d2d 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -6066,6 +6066,7 @@ void tcp_finish_connect(struct sock *sk, struct sk_buff *skb)
struct tcp_sock *tp = tcp_sk(sk);
struct inet_connection_sock *icsk = inet_csk(sk);

+ tcp_ao_finish_connect(sk, skb);
tcp_set_state(sk, TCP_ESTABLISHED);
icsk->icsk_ack.lrcvtime = tcp_jiffies32;

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index b06b286f9ec2..a6a4240609b5 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -2279,6 +2279,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv4_specific = {
#ifdef CONFIG_TCP_AO
.ao_lookup = tcp_v4_ao_lookup,
.ao_parse = tcp_v4_parse_ao,
+ .ao_calc_key_sk = tcp_v4_ao_calc_key_sk,
#endif
};
#endif
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 324b241e95bc..5e680e69fc0a 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3663,6 +3663,7 @@ static void tcp_connect_init(struct sock *sk)
if (tp->af_specific->md5_lookup(sk, sk))
tp->tcp_header_len += TCPOLEN_MD5SIG_ALIGNED;
#endif
+ tcp_ao_connect_init(sk);

/* If user gave his TCP_MAXSEG, record it to clamp */
if (tp->rx_opt.user_mss)
diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index 3d2be5f73cf0..2be0103fc4f8 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -12,6 +12,46 @@
#include <net/tcp.h>
#include <net/ipv6.h>

+static int tcp_v6_ao_calc_key(struct tcp_ao_key *mkt, u8 *key,
+ const struct in6_addr *saddr,
+ const struct in6_addr *daddr,
+ __be16 sport, __be16 dport,
+ __be32 sisn, __be32 disn)
+{
+ struct kdf_input_block {
+ u8 counter;
+ u8 label[6];
+ struct tcp6_ao_context ctx;
+ __be16 outlen;
+ } __packed tmp;
+
+ tmp.counter = 1;
+ memcpy(tmp.label, "TCP-AO", 6);
+ tmp.ctx.saddr = *saddr;
+ tmp.ctx.daddr = *daddr;
+ tmp.ctx.sport = sport;
+ tmp.ctx.dport = dport;
+ tmp.ctx.sisn = sisn;
+ tmp.ctx.disn = disn;
+ tmp.outlen = htons(tcp_ao_digest_size(mkt) * 8); /* in bits */
+
+ return tcp_ao_calc_traffic_key(mkt, key, &tmp, sizeof(tmp));
+}
+
+int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
+ const struct sock *sk, __be32 sisn,
+ __be32 disn, bool send)
+{
+ if (send)
+ return tcp_v6_ao_calc_key(mkt, key, &sk->sk_v6_rcv_saddr,
+ &sk->sk_v6_daddr, htons(sk->sk_num),
+ sk->sk_dport, sisn, disn);
+ else
+ return tcp_v6_ao_calc_key(mkt, key, &sk->sk_v6_daddr,
+ &sk->sk_v6_rcv_saddr, sk->sk_dport,
+ htons(sk->sk_num), disn, sisn);
+}
+
struct tcp_ao_key *tcp_v6_ao_do_lookup(const struct sock *sk,
const struct in6_addr *addr,
int sndid, int rcvid)
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index a946a3a66a92..e47151af9d42 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1921,6 +1921,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv6_specific = {
#ifdef CONFIG_TCP_AO
.ao_lookup = tcp_v6_ao_lookup,
.ao_parse = tcp_v6_parse_ao,
+ .ao_calc_key_sk = tcp_v6_ao_calc_key_sk,
#endif
};
#endif
--
2.40.0

2023-06-14 23:20:56

by Dmitry Safonov

Subject: [PATCH v7 12/22] net/tcp: Verify inbound TCP-AO signed segments

Now there is a common function to verify signature on TCP segments:
tcp_inbound_hash(). It has checks for all possible cross-interactions
with MD5 signs as well as with unsigned segments.

The rules from RFC5925 are:
(1) Any TCP segment can have at max only one signature.
(2) TCP connections can't switch between using TCP-MD5 and TCP-AO.
(3) TCP-AO connections can't stop using AO, as well as unsigned
connections can't suddenly start using AO.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/dropreason-core.h | 17 ++++
include/net/tcp.h | 53 ++++++++++-
include/net/tcp_ao.h | 17 ++++
net/ipv4/tcp.c | 39 ++-------
net/ipv4/tcp_ao.c | 160 ++++++++++++++++++++++++++++++++++
net/ipv4/tcp_ipv4.c | 10 +--
net/ipv6/tcp_ao.c | 13 +++
net/ipv6/tcp_ipv6.c | 11 +--
8 files changed, 277 insertions(+), 43 deletions(-)

diff --git a/include/net/dropreason-core.h b/include/net/dropreason-core.h
index 383ac5215284..0ff272d3b680 100644
--- a/include/net/dropreason-core.h
+++ b/include/net/dropreason-core.h
@@ -24,6 +24,10 @@
FN(TCP_MD5NOTFOUND) \
FN(TCP_MD5UNEXPECTED) \
FN(TCP_MD5FAILURE) \
+ FN(TCP_AONOTFOUND) \
+ FN(TCP_AOUNEXPECTED) \
+ FN(TCP_AOKEYNOTFOUND) \
+ FN(TCP_AOFAILURE) \
FN(SOCKET_BACKLOG) \
FN(TCP_FLAGS) \
FN(TCP_ZEROWINDOW) \
@@ -160,6 +164,19 @@ enum skb_drop_reason {
* to LINUX_MIB_TCPMD5FAILURE
*/
SKB_DROP_REASON_TCP_MD5FAILURE,
+ /**
+ * @SKB_DROP_REASON_TCP_AONOTFOUND: no TCP-AO hash and one was expected
+ */
+ SKB_DROP_REASON_TCP_AONOTFOUND,
+ /**
+ * @SKB_DROP_REASON_TCP_AOUNEXPECTED: TCP-AO hash is present and it
+ * was not expected.
+ */
+ SKB_DROP_REASON_TCP_AOUNEXPECTED,
+ /** @SKB_DROP_REASON_TCP_AOKEYNOTFOUND: TCP-AO key is unknown */
+ SKB_DROP_REASON_TCP_AOKEYNOTFOUND,
+ /** @SKB_DROP_REASON_TCP_AOFAILURE: TCP-AO hash is wrong */
+ SKB_DROP_REASON_TCP_AOFAILURE,
/**
* @SKB_DROP_REASON_SOCKET_BACKLOG: failed to add skb to socket backlog (
* see LINUX_MIB_TCPBACKLOGDROP)
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 1d38ecac9b00..b3fa6aada5c7 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1738,7 +1738,7 @@ tcp_md5_do_lookup_any_l3index(const struct sock *sk,
enum skb_drop_reason
tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
const void *saddr, const void *daddr,
- int family, int dif, int sdif);
+ int family, int l3index, const __u8 *hash_location);


#define tcp_twsk_md5_key(twsk) ((twsk)->tw_md5_key)
@@ -1760,7 +1760,7 @@ tcp_md5_do_lookup_any_l3index(const struct sock *sk,
static inline enum skb_drop_reason
tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
const void *saddr, const void *daddr,
- int family, int dif, int sdif)
+ int family, int l3index, const __u8 *hash_location)
{
return SKB_NOT_DROPPED_YET;
}
@@ -2597,4 +2597,53 @@ static inline bool tcp_ao_required(struct sock *sk, const void *saddr,
return false;
}

+/* Called with rcu_read_lock() */
+static inline enum skb_drop_reason
+tcp_inbound_hash(struct sock *sk, const struct request_sock *req,
+ const struct sk_buff *skb,
+ const void *saddr, const void *daddr,
+ int family, int dif, int sdif)
+{
+ const struct tcphdr *th = tcp_hdr(skb);
+ const struct tcp_ao_hdr *aoh;
+ const __u8 *md5_location;
+ int l3index;
+
+ /* Invalid option or two times meet any of auth options */
+ if (tcp_parse_auth_options(th, &md5_location, &aoh))
+ return SKB_DROP_REASON_TCP_AUTH_HDR;
+
+ if (req) {
+ if (tcp_rsk_used_ao(req) != !!aoh)
+ return SKB_DROP_REASON_TCP_AOFAILURE;
+ }
+
+ /* sdif set, means packet ingressed via a device
+ * in an L3 domain and dif is set to the l3mdev
+ */
+ l3index = sdif ? dif : 0;
+
+ /* Fast path: unsigned segments */
+ if (likely(!md5_location && !aoh)) {
+ /* Drop if there's TCP-MD5 or TCP-AO key with any rcvid/sndid
+ * for the remote peer. On TCP-AO established connection
+ * the last key is impossible to remove, so there's
+ * always at least one current_key.
+ */
+ if (tcp_ao_required(sk, saddr, family))
+ return SKB_DROP_REASON_TCP_AONOTFOUND;
+ if (unlikely(tcp_md5_do_lookup(sk, l3index, saddr, family))) {
+ NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
+ return SKB_DROP_REASON_TCP_MD5NOTFOUND;
+ }
+ return SKB_NOT_DROPPED_YET;
+ }
+
+ if (aoh)
+ return tcp_inbound_ao_hash(sk, skb, family, req, aoh);
+
+ return tcp_inbound_md5_hash(sk, skb, saddr, daddr, family,
+ l3index, md5_location);
+}
+
#endif /* _TCP_H */
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 933b39445575..880eeee4a6b9 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -116,6 +116,9 @@ struct tcp6_ao_context {
__be32 disn;
};

+#define TCP_AO_ESTABLISHED (TCPF_ESTABLISHED|TCPF_FIN_WAIT1|TCPF_FIN_WAIT2|\
+ TCPF_CLOSE|TCPF_CLOSE_WAIT|TCPF_LAST_ACK|TCPF_CLOSING)
+
int tcp_ao_hash_skb(unsigned short int family,
char *ao_hash, struct tcp_ao_key *key,
const struct sock *sk, const struct sk_buff *skb,
@@ -132,6 +135,10 @@ int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
void tcp_ao_destroy_sock(struct sock *sk, bool twsk);
u32 tcp_ao_compute_sne(u32 sne, u32 seq, u32 new_seq);
void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp);
+enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
+ const struct sk_buff *skb, unsigned short int family,
+ const struct request_sock *req,
+ const struct tcp_ao_hdr *aoh);
struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port);
@@ -168,6 +175,9 @@ struct tcp_sigpool;
int tcp_v6_ao_hash_pseudoheader(struct tcp_sigpool *hp,
const struct in6_addr *daddr,
const struct in6_addr *saddr, int nbytes);
+int tcp_v6_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
+ const struct sk_buff *skb, __be32 sisn,
+ __be32 disn);
int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
const struct sock *sk, __be32 sisn,
__be32 disn, bool send);
@@ -203,6 +213,13 @@ static inline void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
{
}

+static inline enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
+ const struct sk_buff *skb, unsigned short int family,
+ const struct request_sock *req, const struct tcp_ao_hdr *aoh)
+{
+ return SKB_NOT_DROPPED_YET;
+}
+
static inline struct tcp_ao_key *tcp_ao_do_lookup(const struct sock *sk,
const union tcp_ao_addr *addr,
int family, int sndid, int rcvid, u16 port)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 66e324b0364c..20e16ada8d5d 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4477,42 +4477,23 @@ EXPORT_SYMBOL(tcp_md5_hash_key);
enum skb_drop_reason
tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
const void *saddr, const void *daddr,
- int family, int dif, int sdif)
+ int family, int l3index, const __u8 *hash_location)
{
- /*
- * This gets called for each TCP segment that arrives
- * so we want to be efficient.
+ /* This gets called for each TCP segment that has TCP-MD5 option.
* We have 3 drop cases:
* o No MD5 hash and one expected.
* o MD5 hash and we're not expecting one.
* o MD5 hash and its wrong.
*/
- const __u8 *hash_location = NULL;
- struct tcp_md5sig_key *hash_expected;
const struct tcphdr *th = tcp_hdr(skb);
const struct tcp_sock *tp = tcp_sk(sk);
- int genhash, l3index;
+ struct tcp_md5sig_key *key;
+ int genhash;
u8 newhash[16];

- /* sdif set, means packet ingressed via a device
- * in an L3 domain and dif is set to the l3mdev
- */
- l3index = sdif ? dif : 0;
+ key = tcp_md5_do_lookup(sk, l3index, saddr, family);

- hash_expected = tcp_md5_do_lookup(sk, l3index, saddr, family);
- if (tcp_parse_auth_options(th, &hash_location, NULL))
- return SKB_DROP_REASON_TCP_AUTH_HDR;
-
- /* We've parsed the options - do we have a hash? */
- if (!hash_expected && !hash_location)
- return SKB_NOT_DROPPED_YET;
-
- if (hash_expected && !hash_location) {
- NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5NOTFOUND);
- return SKB_DROP_REASON_TCP_MD5NOTFOUND;
- }
-
- if (!hash_expected && hash_location) {
+ if (!key && hash_location) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5UNEXPECTED);
return SKB_DROP_REASON_TCP_MD5UNEXPECTED;
}
@@ -4522,14 +4503,10 @@ tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
* IPv4-mapped case.
*/
if (family == AF_INET)
- genhash = tcp_v4_md5_hash_skb(newhash,
- hash_expected,
- NULL, skb);
+ genhash = tcp_v4_md5_hash_skb(newhash, key, NULL, skb);
else
- genhash = tp->af_specific->calc_md5_hash(newhash,
- hash_expected,
+ genhash = tp->af_specific->calc_md5_hash(newhash, key,
NULL, skb);
-
if (genhash || memcmp(hash_location, newhash, 16) != 0) {
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPMD5FAILURE);
if (family == AF_INET) {
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index a22d6321bc81..3f405bf1fd82 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -322,6 +322,30 @@ int tcp_v4_ao_calc_key_rsk(struct tcp_ao_key *mkt, u8 *key,
htonl(tcp_rsk(req)->rcv_isn));
}

+static int tcp_v4_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
+ const struct sk_buff *skb,
+ __be32 sisn, __be32 disn)
+{
+ const struct iphdr *iph = ip_hdr(skb);
+ const struct tcphdr *th = tcp_hdr(skb);
+
+ return tcp_v4_ao_calc_key(mkt, key, iph->saddr, iph->daddr,
+ th->source, th->dest, sisn, disn);
+}
+
+static int tcp_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
+ const struct sk_buff *skb, __be32 sisn,
+ __be32 disn, int family)
+{
+ if (family == AF_INET)
+ return tcp_v4_ao_calc_key_skb(mkt, key, skb, sisn, disn);
+#if IS_ENABLED(CONFIG_IPV6)
+ else if (family == AF_INET6)
+ return tcp_v6_ao_calc_key_skb(mkt, key, skb, sisn, disn);
+#endif
+ return -EAFNOSUPPORT;
+}
+
static int tcp_v4_ao_hash_pseudoheader(struct tcp_sigpool *hp,
__be32 daddr, __be32 saddr,
int nbytes)
@@ -701,6 +725,142 @@ void tcp_ao_syncookie(struct sock *sk, const struct sk_buff *skb,
}
EXPORT_SYMBOL_GPL(tcp_ao_syncookie);

+static enum skb_drop_reason
+tcp_ao_verify_hash(const struct sock *sk, const struct sk_buff *skb,
+ unsigned short int family, struct tcp_ao_info *info,
+ const struct tcp_ao_hdr *aoh, struct tcp_ao_key *key,
+ u8 *traffic_key, u8 *phash, u32 sne)
+{
+ unsigned char newhash[TCP_AO_MAX_HASH_SIZE] __tcp_ao_key_align;
+ u8 maclen = aoh->length - sizeof(struct tcp_ao_hdr);
+ const struct tcphdr *th = tcp_hdr(skb);
+
+ if (maclen != tcp_ao_maclen(key))
+ return SKB_DROP_REASON_TCP_AOFAILURE;
+
+ /* XXX: make it per-AF callback? */
+ tcp_ao_hash_skb(family, newhash, key, sk, skb, traffic_key,
+ (phash - (u8 *)th), sne);
+ if (memcmp(phash, newhash, maclen))
+ return SKB_DROP_REASON_TCP_AOFAILURE;
+ return SKB_NOT_DROPPED_YET;
+}
+
+enum skb_drop_reason
+tcp_inbound_ao_hash(struct sock *sk, const struct sk_buff *skb,
+ unsigned short int family, const struct request_sock *req,
+ const struct tcp_ao_hdr *aoh)
+{
+ u8 key_buf[TCP_AO_MAX_HASH_SIZE] __tcp_ao_key_align;
+ const struct tcphdr *th = tcp_hdr(skb);
+ u8 *phash = (u8 *)(aoh + 1); /* hash goes just after the header */
+ struct tcp_ao_info *info;
+ struct tcp_ao_key *key;
+ __be32 sisn, disn;
+ u8 *traffic_key;
+ u32 sne = 0;
+
+ info = rcu_dereference(tcp_sk(sk)->ao_info);
+ if (!info)
+ return SKB_DROP_REASON_TCP_AOUNEXPECTED;
+
+ if (unlikely(th->syn)) {
+ sisn = th->seq;
+ disn = 0;
+ }
+
+ /* Fast-path */
+ /* TODO: fix fastopen and simultaneous open (TCPF_SYN_RECV) */
+ if (likely((1 << sk->sk_state) & (TCP_AO_ESTABLISHED | TCPF_SYN_RECV))) {
+ enum skb_drop_reason err;
+ struct tcp_ao_key *current_key;
+
+ /* Check if this socket's rnext_key matches the keyid in the
+ * packet. If not we lookup the key based on the keyid
+ * matching the rcvid in the mkt.
+ */
+ key = READ_ONCE(info->rnext_key);
+ if (key->rcvid != aoh->keyid) {
+ key = tcp_ao_established_key(info, -1, aoh->keyid);
+ if (!key)
+ goto key_not_found;
+ }
+
+ /* Delayed retransmitted SYN */
+ if (unlikely(th->syn && !th->ack))
+ goto verify_hash;
+
+ sne = tcp_ao_compute_sne(info->rcv_sne, info->rcv_sne_seq,
+ ntohl(th->seq));
+ /* Established socket, traffic key are cached */
+ traffic_key = rcv_other_key(key);
+ err = tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
+ traffic_key, phash, sne);
+ if (err)
+ return err;
+ current_key = READ_ONCE(info->current_key);
+ /* Key rotation: the peer asks us to use new key (RNext) */
+ if (unlikely(aoh->rnext_keyid != current_key->sndid)) {
+ /* If the key is not found we do nothing. */
+ key = tcp_ao_established_key(info, aoh->rnext_keyid, -1);
+ if (key)
+ /* pairs with tcp_ao_del_cmd */
+ WRITE_ONCE(info->current_key, key);
+ }
+ return SKB_NOT_DROPPED_YET;
+ }
+
+ /* Lookup key based on peer address and keyid.
+ * current_key and rnext_key must not be used on tcp listen
+ * sockets as otherwise:
+ * - request sockets would race on those key pointers
+ * - tcp_ao_del_cmd() allows async key removal
+ */
+ key = tcp_ao_inbound_lookup(family, sk, skb, -1, aoh->keyid);
+ if (!key)
+ goto key_not_found;
+
+ if (th->syn && !th->ack)
+ goto verify_hash;
+
+ if ((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_NEW_SYN_RECV)) {
+ /* Make the initial syn the likely case here */
+ if (unlikely(req)) {
+ sne = tcp_ao_compute_sne(0, tcp_rsk(req)->rcv_isn,
+ ntohl(th->seq));
+ sisn = htonl(tcp_rsk(req)->rcv_isn);
+ disn = htonl(tcp_rsk(req)->snt_isn);
+ } else if (unlikely(th->ack && !th->syn)) {
+ /* Possible syncookie packet */
+ sisn = htonl(ntohl(th->seq) - 1);
+ disn = htonl(ntohl(th->ack_seq) - 1);
+ sne = tcp_ao_compute_sne(0, ntohl(sisn),
+ ntohl(th->seq));
+ } else if (unlikely(!th->syn)) {
+ /* no way to figure out initial sisn/disn - drop */
+ return SKB_DROP_REASON_TCP_FLAGS;
+ }
+ } else if (sk->sk_state == TCP_SYN_SENT) {
+ disn = info->lisn;
+ if (th->syn || th->rst)
+ sisn = th->seq;
+ else
+ sisn = info->risn;
+ } else {
+ WARN_ONCE(1, "TCP-AO: Unexpected sk_state %d", sk->sk_state);
+ return SKB_DROP_REASON_TCP_AOFAILURE;
+ }
+verify_hash:
+ traffic_key = key_buf;
+ tcp_ao_calc_key_skb(key, traffic_key, skb, sisn, disn, family);
+ return tcp_ao_verify_hash(sk, skb, family, info, aoh, key,
+ traffic_key, phash, sne);
+
+key_not_found:
+ return SKB_DROP_REASON_TCP_AOKEYNOTFOUND;
+}
+EXPORT_SYMBOL_GPL(tcp_inbound_ao_hash);
+
static int tcp_ao_cache_traffic_keys(const struct sock *sk,
struct tcp_ao_info *ao,
struct tcp_ao_key *ao_key)
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 97712b954e2d..1a0c2f1485fa 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -2195,9 +2195,9 @@ int tcp_v4_rcv(struct sk_buff *skb)
if (!xfrm4_policy_check(sk, XFRM_POLICY_IN, skb))
drop_reason = SKB_DROP_REASON_XFRM_POLICY;
else
- drop_reason = tcp_inbound_md5_hash(sk, skb,
- &iph->saddr, &iph->daddr,
- AF_INET, dif, sdif);
+ drop_reason = tcp_inbound_hash(sk, req, skb,
+ &iph->saddr, &iph->daddr,
+ AF_INET, dif, sdif);
if (unlikely(drop_reason)) {
sk_drops_add(sk, skb);
reqsk_put(req);
@@ -2274,8 +2274,8 @@ int tcp_v4_rcv(struct sk_buff *skb)
goto discard_and_relse;
}

- drop_reason = tcp_inbound_md5_hash(sk, skb, &iph->saddr,
- &iph->daddr, AF_INET, dif, sdif);
+ drop_reason = tcp_inbound_hash(sk, NULL, skb, &iph->saddr, &iph->daddr,
+ AF_INET, dif, sdif);
if (drop_reason)
goto discard_and_relse;

diff --git a/net/ipv6/tcp_ao.c b/net/ipv6/tcp_ao.c
index bc032e441ef8..32e19b2dbda8 100644
--- a/net/ipv6/tcp_ao.c
+++ b/net/ipv6/tcp_ao.c
@@ -39,6 +39,18 @@ static int tcp_v6_ao_calc_key(struct tcp_ao_key *mkt, u8 *key,
return tcp_ao_calc_traffic_key(mkt, key, &tmp, sizeof(tmp));
}

+int tcp_v6_ao_calc_key_skb(struct tcp_ao_key *mkt, u8 *key,
+ const struct sk_buff *skb,
+ __be32 sisn, __be32 disn)
+{
+ const struct ipv6hdr *iph = ipv6_hdr(skb);
+ const struct tcphdr *th = tcp_hdr(skb);
+
+ return tcp_v6_ao_calc_key(mkt, key, &iph->saddr,
+ &iph->daddr, th->source,
+ th->dest, sisn, disn);
+}
+
int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
const struct sock *sk, __be32 sisn,
__be32 disn, bool send)
@@ -52,6 +64,7 @@ int tcp_v6_ao_calc_key_sk(struct tcp_ao_key *mkt, u8 *key,
&sk->sk_v6_rcv_saddr, sk->sk_dport,
htons(sk->sk_num), disn, sisn);
}
+EXPORT_SYMBOL_GPL(tcp_v6_ao_calc_key_sk);

int tcp_v6_ao_calc_key_rsk(struct tcp_ao_key *mkt, u8 *key,
struct request_sock *req)
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 19f644970196..cd9b3714d8a6 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1784,9 +1784,9 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
struct sock *nsk;

sk = req->rsk_listener;
- drop_reason = tcp_inbound_md5_hash(sk, skb,
- &hdr->saddr, &hdr->daddr,
- AF_INET6, dif, sdif);
+ drop_reason = tcp_inbound_hash(sk, req, skb,
+ &hdr->saddr, &hdr->daddr,
+ AF_INET6, dif, sdif);
if (drop_reason) {
sk_drops_add(sk, skb);
reqsk_put(req);
@@ -1859,8 +1859,8 @@ INDIRECT_CALLABLE_SCOPE int tcp_v6_rcv(struct sk_buff *skb)
goto discard_and_relse;
}

- drop_reason = tcp_inbound_md5_hash(sk, skb, &hdr->saddr, &hdr->daddr,
- AF_INET6, dif, sdif);
+ drop_reason = tcp_inbound_hash(sk, NULL, skb, &hdr->saddr, &hdr->daddr,
+ AF_INET6, dif, sdif);
if (drop_reason)
goto discard_and_relse;

@@ -2088,6 +2088,7 @@ static const struct tcp_sock_af_ops tcp_sock_ipv6_mapped_specific = {
.ao_lookup = tcp_v6_ao_lookup,
.calc_ao_hash = tcp_v4_ao_hash_skb,
.ao_parse = tcp_v6_parse_ao,
+ .ao_calc_key_sk = tcp_v4_ao_calc_key_sk,
#endif
};
#endif
--
2.40.0

2023-06-14 23:21:00

by Dmitry Safonov

Subject: [PATCH v7 01/22] net/tcp: Prepare tcp_md5sig_pool for TCP-AO

TCP-AO, similarly to TCP-MD5, needs to allocate tfms on a slow-path,
which is setsockopt() and use crypto ahash requests on fast paths,
which are RX/TX softirqs. Also, it needs a temporary/scratch buffer
for preparing the hash.

Rework tcp_md5sig_pool in order to support other hashing algorithms
than MD5. It will make it possible to share pre-allocated crypto_ahash
descriptors and scratch area between all TCP hash users.

Internally tcp_sigpool calls crypto_clone_ahash() API over pre-allocated
crypto ahash tfm. Kudos to Herbert, who provided this new crypto API.

I was a little concerned over GFP_ATOMIC allocations of ahash and
crypto_request in RX/TX (see tcp_sigpool_start()), so I benchmarked both
"backends" with different algorithms, using patched version of iperf3[2].
On my laptop with i7-7600U @ 2.80GHz:

clone-tfm per-CPU-requests
TCP-MD5 2.25 Gbits/sec 2.30 Gbits/sec
TCP-AO(hmac(sha1)) 2.53 Gbits/sec 2.54 Gbits/sec
TCP-AO(hmac(sha512)) 1.67 Gbits/sec 1.64 Gbits/sec
TCP-AO(hmac(sha384)) 1.77 Gbits/sec 1.80 Gbits/sec
TCP-AO(hmac(sha224)) 1.29 Gbits/sec 1.30 Gbits/sec
TCP-AO(hmac(sha3-512)) 481 Mbits/sec 480 Mbits/sec
TCP-AO(hmac(md5)) 2.07 Gbits/sec 2.12 Gbits/sec
TCP-AO(hmac(rmd160)) 1.01 Gbits/sec 995 Mbits/sec
TCP-AO(cmac(aes128)) [not supporetd yet] 2.11 Gbits/sec

So, it seems that my concerns don't have strong grounds and per-CPU
crypto_request allocation can be dropped/removed from tcp_sigpool once
ciphers get crypto_clone_ahash() support.

[1]: https://lore.kernel.org/all/[email protected]/T/#u
[2]: https://github.com/0x7f454c46/iperf/tree/tcp-md5-ao
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp.h | 46 +++--
net/ipv4/Kconfig | 4 +
net/ipv4/Makefile | 1 +
net/ipv4/tcp.c | 135 +++------------
net/ipv4/tcp_ipv4.c | 97 ++++++-----
net/ipv4/tcp_minisocks.c | 21 ++-
net/ipv4/tcp_sigpool.c | 357 +++++++++++++++++++++++++++++++++++++++
net/ipv6/tcp_ipv6.c | 58 +++----
8 files changed, 510 insertions(+), 209 deletions(-)
create mode 100644 net/ipv4/tcp_sigpool.c

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 5066e4586cf0..81134a70c9ba 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -1670,12 +1670,35 @@ union tcp_md5sum_block {
#endif
};

-/* - pool: digest algorithm, hash description and scratch buffer */
-struct tcp_md5sig_pool {
- struct ahash_request *md5_req;
- void *scratch;
+/*
+ * struct tcp_sigpool - per-CPU pool of ahash_requests
+ * @scratch: per-CPU temporary area, that can be used between
+ * tcp_sigpool_start() and tcp_sigpool_end() to perform
+ * crypto request
+ * @req: pre-allocated ahash request
+ */
+struct tcp_sigpool {
+ void *scratch;
+ struct ahash_request *req;
};
+int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size);
+void tcp_sigpool_get(unsigned int id);
+void tcp_sigpool_release(unsigned int id);
+int tcp_sigpool_hash_skb_data(struct tcp_sigpool *hp,
+ const struct sk_buff *skb,
+ unsigned int header_len);

+/**
+ * tcp_sigpool_start - disable bh and start using tcp_sigpool_ahash
+ * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash()
+ * @c: returned tcp_sigpool for usage (uninitialized on failure)
+ */
+int tcp_sigpool_start(unsigned int id, struct tcp_sigpool *c);
+/**
+ * tcp_sigpool_end - enable bh and stop using tcp_sigpool
+ */
+void tcp_sigpool_end(struct tcp_sigpool *c);
+size_t tcp_sigpool_algo(unsigned int id, char *buf, size_t buf_len);
/* - functions */
int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
const struct sock *sk, const struct sk_buff *skb);
@@ -1731,17 +1754,12 @@ tcp_inbound_md5_hash(const struct sock *sk, const struct sk_buff *skb,
#define tcp_twsk_md5_key(twsk) NULL
#endif

-bool tcp_alloc_md5sig_pool(void);
+int tcp_md5_alloc_sigpool(void);
+void tcp_md5_release_sigpool(void);
+void tcp_md5_add_sigpool(void);
+extern int tcp_md5_sigpool_id;

-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void);
-static inline void tcp_put_md5sig_pool(void)
-{
- local_bh_enable();
-}
-
-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *, const struct sk_buff *,
- unsigned int header_len);
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp,
+int tcp_md5_hash_key(struct tcp_sigpool *hp,
const struct tcp_md5sig_key *key);

/* From tcp_fastopen.c */
diff --git a/net/ipv4/Kconfig b/net/ipv4/Kconfig
index 2dfb12230f08..89e2ab023272 100644
--- a/net/ipv4/Kconfig
+++ b/net/ipv4/Kconfig
@@ -741,10 +741,14 @@ config DEFAULT_TCP_CONG
default "bbr" if DEFAULT_BBR
default "cubic"

+config TCP_SIGPOOL
+ tristate
+
config TCP_MD5SIG
bool "TCP: MD5 Signature Option support (RFC2385)"
select CRYPTO
select CRYPTO_MD5
+ select TCP_SIGPOOL
help
RFC2385 specifies a method of giving MD5 protection to TCP sessions.
Its main (only?) use is to protect BGP sessions between core routers
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index b18ba8ef93ad..cd760793cfcb 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -62,6 +62,7 @@ obj-$(CONFIG_TCP_CONG_SCALABLE) += tcp_scalable.o
obj-$(CONFIG_TCP_CONG_LP) += tcp_lp.o
obj-$(CONFIG_TCP_CONG_YEAH) += tcp_yeah.o
obj-$(CONFIG_TCP_CONG_ILLINOIS) += tcp_illinois.o
+obj-$(CONFIG_TCP_SIGPOOL) += tcp_sigpool.o
obj-$(CONFIG_NET_SOCK_MSG) += tcp_bpf.o
obj-$(CONFIG_BPF_SYSCALL) += udp_bpf.o
obj-$(CONFIG_NETLABEL) += cipso_ipv4.o
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 8d20d9221238..2417a8549142 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4415,141 +4415,44 @@ int tcp_getsockopt(struct sock *sk, int level, int optname, char __user *optval,
EXPORT_SYMBOL(tcp_getsockopt);

#ifdef CONFIG_TCP_MD5SIG
-static DEFINE_PER_CPU(struct tcp_md5sig_pool, tcp_md5sig_pool);
-static DEFINE_MUTEX(tcp_md5sig_mutex);
-static bool tcp_md5sig_pool_populated = false;
+int tcp_md5_sigpool_id = -1;
+EXPORT_SYMBOL_GPL(tcp_md5_sigpool_id);

-static void __tcp_alloc_md5sig_pool(void)
+int tcp_md5_alloc_sigpool(void)
{
- struct crypto_ahash *hash;
- int cpu;
+ size_t scratch_size;
+ int ret;

- hash = crypto_alloc_ahash("md5", 0, CRYPTO_ALG_ASYNC);
- if (IS_ERR(hash))
- return;
-
- for_each_possible_cpu(cpu) {
- void *scratch = per_cpu(tcp_md5sig_pool, cpu).scratch;
- struct ahash_request *req;
-
- if (!scratch) {
- scratch = kmalloc_node(sizeof(union tcp_md5sum_block) +
- sizeof(struct tcphdr),
- GFP_KERNEL,
- cpu_to_node(cpu));
- if (!scratch)
- return;
- per_cpu(tcp_md5sig_pool, cpu).scratch = scratch;
- }
- if (per_cpu(tcp_md5sig_pool, cpu).md5_req)
- continue;
-
- req = ahash_request_alloc(hash, GFP_KERNEL);
- if (!req)
- return;
-
- ahash_request_set_callback(req, 0, NULL, NULL);
-
- per_cpu(tcp_md5sig_pool, cpu).md5_req = req;
+ scratch_size = sizeof(union tcp_md5sum_block) + sizeof(struct tcphdr);
+ ret = tcp_sigpool_alloc_ahash("md5", scratch_size);
+ if (ret >= 0) {
+ tcp_md5_sigpool_id = ret;
+ return 0;
}
- /* before setting tcp_md5sig_pool_populated, we must commit all writes
- * to memory. See smp_rmb() in tcp_get_md5sig_pool()
- */
- smp_wmb();
- /* Paired with READ_ONCE() from tcp_alloc_md5sig_pool()
- * and tcp_get_md5sig_pool().
- */
- WRITE_ONCE(tcp_md5sig_pool_populated, true);
+ return ret;
}

-bool tcp_alloc_md5sig_pool(void)
+void tcp_md5_release_sigpool(void)
{
- /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */
- if (unlikely(!READ_ONCE(tcp_md5sig_pool_populated))) {
- mutex_lock(&tcp_md5sig_mutex);
-
- if (!tcp_md5sig_pool_populated)
- __tcp_alloc_md5sig_pool();
-
- mutex_unlock(&tcp_md5sig_mutex);
- }
- /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */
- return READ_ONCE(tcp_md5sig_pool_populated);
+ tcp_sigpool_release(tcp_md5_sigpool_id);
}
-EXPORT_SYMBOL(tcp_alloc_md5sig_pool);

-
-/**
- * tcp_get_md5sig_pool - get md5sig_pool for this user
- *
- * We use percpu structure, so if we succeed, we exit with preemption
- * and BH disabled, to make sure another thread or softirq handling
- * wont try to get same context.
- */
-struct tcp_md5sig_pool *tcp_get_md5sig_pool(void)
+void tcp_md5_add_sigpool(void)
{
- local_bh_disable();
-
- /* Paired with WRITE_ONCE() from __tcp_alloc_md5sig_pool() */
- if (READ_ONCE(tcp_md5sig_pool_populated)) {
- /* coupled with smp_wmb() in __tcp_alloc_md5sig_pool() */
- smp_rmb();
- return this_cpu_ptr(&tcp_md5sig_pool);
- }
- local_bh_enable();
- return NULL;
+ tcp_sigpool_get(tcp_md5_sigpool_id);
}
-EXPORT_SYMBOL(tcp_get_md5sig_pool);

-int tcp_md5_hash_skb_data(struct tcp_md5sig_pool *hp,
- const struct sk_buff *skb, unsigned int header_len)
-{
- struct scatterlist sg;
- const struct tcphdr *tp = tcp_hdr(skb);
- struct ahash_request *req = hp->md5_req;
- unsigned int i;
- const unsigned int head_data_len = skb_headlen(skb) > header_len ?
- skb_headlen(skb) - header_len : 0;
- const struct skb_shared_info *shi = skb_shinfo(skb);
- struct sk_buff *frag_iter;
-
- sg_init_table(&sg, 1);
-
- sg_set_buf(&sg, ((u8 *) tp) + header_len, head_data_len);
- ahash_request_set_crypt(req, &sg, NULL, head_data_len);
- if (crypto_ahash_update(req))
- return 1;
-
- for (i = 0; i < shi->nr_frags; ++i) {
- const skb_frag_t *f = &shi->frags[i];
- unsigned int offset = skb_frag_off(f);
- struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
-
- sg_set_page(&sg, page, skb_frag_size(f),
- offset_in_page(offset));
- ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
- if (crypto_ahash_update(req))
- return 1;
- }
-
- skb_walk_frags(skb, frag_iter)
- if (tcp_md5_hash_skb_data(hp, frag_iter, 0))
- return 1;
-
- return 0;
-}
-EXPORT_SYMBOL(tcp_md5_hash_skb_data);
-
-int tcp_md5_hash_key(struct tcp_md5sig_pool *hp, const struct tcp_md5sig_key *key)
+int tcp_md5_hash_key(struct tcp_sigpool *hp,
+ const struct tcp_md5sig_key *key)
{
u8 keylen = READ_ONCE(key->keylen); /* paired with WRITE_ONCE() in tcp_md5_do_add */
struct scatterlist sg;

sg_init_one(&sg, key->key, keylen);
- ahash_request_set_crypt(hp->md5_req, &sg, NULL, keylen);
+ ahash_request_set_crypt(hp->req, &sg, NULL, keylen);

/* We use data_race() because tcp_md5_do_add() might change key->key under us */
- return data_race(crypto_ahash_update(hp->md5_req));
+ return data_race(crypto_ahash_update(hp->req));
}
EXPORT_SYMBOL(tcp_md5_hash_key);

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 06d2573685ca..f9423f5dd4e9 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1213,10 +1213,6 @@ static int __tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
key = sock_kmalloc(sk, sizeof(*key), gfp | __GFP_ZERO);
if (!key)
return -ENOMEM;
- if (!tcp_alloc_md5sig_pool()) {
- sock_kfree_s(sk, key, sizeof(*key));
- return -ENOMEM;
- }

memcpy(key->key, newkey, newkeylen);
key->keylen = newkeylen;
@@ -1238,15 +1234,21 @@ int tcp_md5_do_add(struct sock *sk, const union tcp_md5_addr *addr,
struct tcp_sock *tp = tcp_sk(sk);

if (!rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk))) {
- if (tcp_md5sig_info_add(sk, GFP_KERNEL))
+ if (tcp_md5_alloc_sigpool())
return -ENOMEM;

+ if (tcp_md5sig_info_add(sk, GFP_KERNEL)) {
+ tcp_md5_release_sigpool();
+ return -ENOMEM;
+ }
+
if (!static_branch_inc(&tcp_md5_needed.key)) {
struct tcp_md5sig_info *md5sig;

md5sig = rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk));
rcu_assign_pointer(tp->md5sig_info, NULL);
kfree_rcu(md5sig, rcu);
+ tcp_md5_release_sigpool();
return -EUSERS;
}
}
@@ -1263,8 +1265,12 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr,
struct tcp_sock *tp = tcp_sk(sk);

if (!rcu_dereference_protected(tp->md5sig_info, lockdep_sock_is_held(sk))) {
- if (tcp_md5sig_info_add(sk, sk_gfp_mask(sk, GFP_ATOMIC)))
+ tcp_md5_add_sigpool();
+
+ if (tcp_md5sig_info_add(sk, sk_gfp_mask(sk, GFP_ATOMIC))) {
+ tcp_md5_release_sigpool();
return -ENOMEM;
+ }

if (!static_key_fast_inc_not_disabled(&tcp_md5_needed.key.key)) {
struct tcp_md5sig_info *md5sig;
@@ -1273,6 +1279,7 @@ int tcp_md5_key_copy(struct sock *sk, const union tcp_md5_addr *addr,
net_warn_ratelimited("Too many TCP-MD5 keys in the system\n");
rcu_assign_pointer(tp->md5sig_info, NULL);
kfree_rcu(md5sig, rcu);
+ tcp_md5_release_sigpool();
return -EUSERS;
}
}
@@ -1372,7 +1379,7 @@ static int tcp_v4_parse_md5_keys(struct sock *sk, int optname,
cmd.tcpm_key, cmd.tcpm_keylen);
}

-static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
+static int tcp_v4_md5_hash_headers(struct tcp_sigpool *hp,
__be32 daddr, __be32 saddr,
const struct tcphdr *th, int nbytes)
{
@@ -1392,38 +1399,35 @@ static int tcp_v4_md5_hash_headers(struct tcp_md5sig_pool *hp,
_th->check = 0;

sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
- ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+ ahash_request_set_crypt(hp->req, &sg, NULL,
sizeof(*bp) + sizeof(*th));
- return crypto_ahash_update(hp->md5_req);
+ return crypto_ahash_update(hp->req);
}

static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
__be32 daddr, __be32 saddr, const struct tcphdr *th)
{
- struct tcp_md5sig_pool *hp;
- struct ahash_request *req;
+ struct tcp_sigpool hp;

- hp = tcp_get_md5sig_pool();
- if (!hp)
- goto clear_hash_noput;
- req = hp->md5_req;
+ if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp))
+ goto clear_hash_nostart;

- if (crypto_ahash_init(req))
+ if (crypto_ahash_init(hp.req))
goto clear_hash;
- if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+ if (tcp_v4_md5_hash_headers(&hp, daddr, saddr, th, th->doff << 2))
goto clear_hash;
- if (tcp_md5_hash_key(hp, key))
+ if (tcp_md5_hash_key(&hp, key))
goto clear_hash;
- ahash_request_set_crypt(req, NULL, md5_hash, 0);
- if (crypto_ahash_final(req))
+ ahash_request_set_crypt(hp.req, NULL, md5_hash, 0);
+ if (crypto_ahash_final(hp.req))
goto clear_hash;

- tcp_put_md5sig_pool();
+ tcp_sigpool_end(&hp);
return 0;

clear_hash:
- tcp_put_md5sig_pool();
-clear_hash_noput:
+ tcp_sigpool_end(&hp);
+clear_hash_nostart:
memset(md5_hash, 0, 16);
return 1;
}
@@ -1432,8 +1436,7 @@ int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
const struct sock *sk,
const struct sk_buff *skb)
{
- struct tcp_md5sig_pool *hp;
- struct ahash_request *req;
+ struct tcp_sigpool hp;
const struct tcphdr *th = tcp_hdr(skb);
__be32 saddr, daddr;

@@ -1446,30 +1449,28 @@ int tcp_v4_md5_hash_skb(char *md5_hash, const struct tcp_md5sig_key *key,
daddr = iph->daddr;
}

- hp = tcp_get_md5sig_pool();
- if (!hp)
- goto clear_hash_noput;
- req = hp->md5_req;
+ if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp))
+ goto clear_hash_nostart;

- if (crypto_ahash_init(req))
+ if (crypto_ahash_init(hp.req))
goto clear_hash;

- if (tcp_v4_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+ if (tcp_v4_md5_hash_headers(&hp, daddr, saddr, th, skb->len))
goto clear_hash;
- if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+ if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2))
goto clear_hash;
- if (tcp_md5_hash_key(hp, key))
+ if (tcp_md5_hash_key(&hp, key))
goto clear_hash;
- ahash_request_set_crypt(req, NULL, md5_hash, 0);
- if (crypto_ahash_final(req))
+ ahash_request_set_crypt(hp.req, NULL, md5_hash, 0);
+ if (crypto_ahash_final(hp.req))
goto clear_hash;

- tcp_put_md5sig_pool();
+ tcp_sigpool_end(&hp);
return 0;

clear_hash:
- tcp_put_md5sig_pool();
-clear_hash_noput:
+ tcp_sigpool_end(&hp);
+clear_hash_nostart:
memset(md5_hash, 0, 16);
return 1;
}
@@ -2287,6 +2288,18 @@ static int tcp_v4_init_sock(struct sock *sk)
return 0;
}

+#ifdef CONFIG_TCP_MD5SIG
+static void tcp_md5sig_info_free_rcu(struct rcu_head *head)
+{
+ struct tcp_md5sig_info *md5sig;
+
+ md5sig = container_of(head, struct tcp_md5sig_info, rcu);
+ kfree(md5sig);
+ static_branch_slow_dec_deferred(&tcp_md5_needed);
+ tcp_md5_release_sigpool();
+}
+#endif
+
void tcp_v4_destroy_sock(struct sock *sk)
{
struct tcp_sock *tp = tcp_sk(sk);
@@ -2311,10 +2324,12 @@ void tcp_v4_destroy_sock(struct sock *sk)
#ifdef CONFIG_TCP_MD5SIG
/* Clean up the MD5 key list, if any */
if (tp->md5sig_info) {
+ struct tcp_md5sig_info *md5sig;
+
+ md5sig = rcu_dereference_protected(tp->md5sig_info, 1);
tcp_clear_md5_list(sk);
- kfree_rcu(rcu_dereference_protected(tp->md5sig_info, 1), rcu);
- tp->md5sig_info = NULL;
- static_branch_slow_dec_deferred(&tcp_md5_needed);
+ call_rcu(&md5sig->rcu, tcp_md5sig_info_free_rcu);
+ rcu_assign_pointer(tp->md5sig_info, NULL);
}
#endif

diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c
index dac0d62120e6..63b5c5f42a87 100644
--- a/net/ipv4/tcp_minisocks.c
+++ b/net/ipv4/tcp_minisocks.c
@@ -261,10 +261,9 @@ static void tcp_time_wait_init(struct sock *sk, struct tcp_timewait_sock *tcptw)
tcptw->tw_md5_key = kmemdup(key, sizeof(*key), GFP_ATOMIC);
if (!tcptw->tw_md5_key)
return;
- if (!tcp_alloc_md5sig_pool())
- goto out_free;
if (!static_key_fast_inc_not_disabled(&tcp_md5_needed.key.key))
goto out_free;
+ tcp_md5_add_sigpool();
}
return;
out_free:
@@ -349,16 +348,26 @@ void tcp_time_wait(struct sock *sk, int state, int timeo)
}
EXPORT_SYMBOL(tcp_time_wait);

+#ifdef CONFIG_TCP_MD5SIG
+static void tcp_md5_twsk_free_rcu(struct rcu_head *head)
+{
+ struct tcp_md5sig_key *key;
+
+ key = container_of(head, struct tcp_md5sig_key, rcu);
+ kfree(key);
+ static_branch_slow_dec_deferred(&tcp_md5_needed);
+ tcp_md5_release_sigpool();
+}
+#endif
+
void tcp_twsk_destructor(struct sock *sk)
{
#ifdef CONFIG_TCP_MD5SIG
if (static_branch_unlikely(&tcp_md5_needed.key)) {
struct tcp_timewait_sock *twsk = tcp_twsk(sk);

- if (twsk->tw_md5_key) {
- kfree_rcu(twsk->tw_md5_key, rcu);
- static_branch_slow_dec_deferred(&tcp_md5_needed);
- }
+ if (twsk->tw_md5_key)
+ call_rcu(&twsk->tw_md5_key->rcu, tcp_md5_twsk_free_rcu);
}
#endif
}
diff --git a/net/ipv4/tcp_sigpool.c b/net/ipv4/tcp_sigpool.c
new file mode 100644
index 000000000000..93940761c4ec
--- /dev/null
+++ b/net/ipv4/tcp_sigpool.c
@@ -0,0 +1,357 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+
+#include <crypto/hash.h>
+#include <linux/cpu.h>
+#include <linux/kref.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/percpu.h>
+#include <linux/workqueue.h>
+#include <net/tcp.h>
+
+static size_t __scratch_size;
+static DEFINE_PER_CPU(void __rcu *, sigpool_scratch);
+
+struct sigpool_entry {
+ struct crypto_ahash *hash;
+ const char *alg;
+ struct kref kref;
+ uint16_t needs_key:1,
+ reserved:15;
+};
+
+#define CPOOL_SIZE (PAGE_SIZE / sizeof(struct sigpool_entry))
+static struct sigpool_entry cpool[CPOOL_SIZE];
+static unsigned int cpool_populated;
+static DEFINE_MUTEX(cpool_mutex);
+
+/* Slow-path */
+struct scratches_to_free {
+ struct rcu_head rcu;
+ unsigned int cnt;
+ void *scratches[];
+};
+
+static void free_old_scratches(struct rcu_head *head)
+{
+ struct scratches_to_free *stf;
+
+ stf = container_of(head, struct scratches_to_free, rcu);
+ while (stf->cnt--)
+ kfree(stf->scratches[stf->cnt]);
+ kfree(stf);
+}
+
+/**
+ * sigpool_reserve_scratch - re-allocates scratch buffer, slow-path
+ * @size: request size for the scratch/temp buffer
+ */
+static int sigpool_reserve_scratch(size_t size)
+{
+ struct scratches_to_free *stf;
+ size_t stf_sz = struct_size(stf, scratches, num_possible_cpus());
+ int cpu, err = 0;
+
+ lockdep_assert_held(&cpool_mutex);
+ if (__scratch_size >= size)
+ return 0;
+
+ stf = kmalloc(stf_sz, GFP_KERNEL);
+ if (!stf)
+ return -ENOMEM;
+ stf->cnt = 0;
+
+ size = max(size, __scratch_size);
+ cpus_read_lock();
+ for_each_possible_cpu(cpu) {
+ void *scratch, *old_scratch;
+
+ scratch = kmalloc_node(size, GFP_KERNEL, cpu_to_node(cpu));
+ if (!scratch) {
+ err = -ENOMEM;
+ break;
+ }
+
+ old_scratch = rcu_replace_pointer(per_cpu(sigpool_scratch, cpu), scratch, lockdep_is_held(&cpool_mutex));
+ if (!cpu_online(cpu) || !old_scratch) {
+ kfree(old_scratch);
+ continue;
+ }
+ stf->scratches[stf->cnt++] = old_scratch;
+ }
+ cpus_read_unlock();
+ if (!err)
+ __scratch_size = size;
+
+ call_rcu(&stf->rcu, free_old_scratches);
+ return err;
+}
+
+static void sigpool_scratch_free(void)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ kfree(rcu_replace_pointer(per_cpu(sigpool_scratch, cpu),
+ NULL, lockdep_is_held(&cpool_mutex)));
+ __scratch_size = 0;
+}
+
+static int __cpool_try_clone(struct crypto_ahash *hash)
+{
+ struct crypto_ahash *tmp;
+
+ tmp = crypto_clone_ahash(hash);
+ if (IS_ERR(tmp))
+ return PTR_ERR(tmp);
+
+ crypto_free_ahash(tmp);
+ return 0;
+}
+
+static int __cpool_alloc_ahash(struct sigpool_entry *e, const char *alg)
+{
+ struct crypto_ahash *cpu0_hash;
+ int ret;
+
+ e->alg = kstrdup(alg, GFP_KERNEL);
+ if (!e->alg)
+ return -ENOMEM;
+
+ cpu0_hash = crypto_alloc_ahash(alg, 0, CRYPTO_ALG_ASYNC);
+ if (IS_ERR(cpu0_hash)) {
+ ret = PTR_ERR(cpu0_hash);
+ goto out_free_alg;
+ }
+
+ e->needs_key = crypto_ahash_get_flags(cpu0_hash) & CRYPTO_TFM_NEED_KEY;
+
+ ret = __cpool_try_clone(cpu0_hash);
+ if (ret)
+ goto out_free_cpu0_hash;
+ e->hash = cpu0_hash;
+ kref_init(&e->kref);
+ return 0;
+
+out_free_cpu0_hash:
+ crypto_free_ahash(cpu0_hash);
+out_free_alg:
+ kfree(e->alg);
+ e->alg = NULL;
+ return ret;
+}
+
+/**
+ * tcp_sigpool_alloc_ahash - allocates pool for ahash requests
+ * @alg: name of async hash algorithm
+ * @scratch_size: reserve a tcp_sigpool::scratch buffer of this size
+ */
+int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size)
+{
+ int i, ret;
+
+ /* slow-path */
+ mutex_lock(&cpool_mutex);
+ ret = sigpool_reserve_scratch(scratch_size);
+ if (ret)
+ goto out;
+ for (i = 0; i < cpool_populated; i++) {
+ if (!cpool[i].alg)
+ continue;
+ if (strcmp(cpool[i].alg, alg))
+ continue;
+
+ if (kref_read(&cpool[i].kref) > 0)
+ kref_get(&cpool[i].kref);
+ else
+ kref_init(&cpool[i].kref);
+ ret = i;
+ goto out;
+ }
+
+ for (i = 0; i < cpool_populated; i++) {
+ if (!cpool[i].alg)
+ break;
+ }
+ if (i >= CPOOL_SIZE) {
+ ret = -ENOSPC;
+ goto out;
+ }
+
+ ret = __cpool_alloc_ahash(&cpool[i], alg);
+ if (!ret) {
+ ret = i;
+ if (i == cpool_populated)
+ cpool_populated++;
+ }
+out:
+ mutex_unlock(&cpool_mutex);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_alloc_ahash);
+
+static void __cpool_free_entry(struct sigpool_entry *e)
+{
+ crypto_free_ahash(e->hash);
+ kfree(e->alg);
+ memset(e, 0, sizeof(*e));
+}
+
+static void cpool_cleanup_work_cb(struct work_struct *work)
+{
+ unsigned int i;
+ bool free_scratch = true;
+
+ mutex_lock(&cpool_mutex);
+ for (i = 0; i < cpool_populated; i++) {
+ if (kref_read(&cpool[i].kref) > 0) {
+ free_scratch = false;
+ continue;
+ }
+ if (!cpool[i].alg)
+ continue;
+ __cpool_free_entry(&cpool[i]);
+ }
+ if (free_scratch)
+ sigpool_scratch_free();
+ mutex_unlock(&cpool_mutex);
+}
+
+static DECLARE_WORK(cpool_cleanup_work, cpool_cleanup_work_cb);
+static void cpool_schedule_cleanup(struct kref *kref)
+{
+ schedule_work(&cpool_cleanup_work);
+}
+
+/**
+ * tcp_sigpool_release - decreases number of users for a pool. If it was
+ * the last user of the pool, releases any memory that was consumed.
+ * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash()
+ */
+void tcp_sigpool_release(unsigned int id)
+{
+ if (WARN_ON_ONCE(id > cpool_populated || !cpool[id].alg))
+ return;
+
+ /* slow-path */
+ kref_put(&cpool[id].kref, cpool_schedule_cleanup);
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_release);
+
+/**
+ * tcp_sigpool_get - increases number of users (refcounter) for a pool
+ * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash()
+ */
+void tcp_sigpool_get(unsigned int id)
+{
+ if (WARN_ON_ONCE(id > cpool_populated || !cpool[id].alg))
+ return;
+ kref_get(&cpool[id].kref);
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_get);
+
+int tcp_sigpool_start(unsigned int id, struct tcp_sigpool *c)
+{
+ struct crypto_ahash *hash;
+
+ rcu_read_lock_bh();
+ if (WARN_ON_ONCE(id > cpool_populated || !cpool[id].alg)) {
+ rcu_read_unlock_bh();
+ return -EINVAL;
+ }
+
+ hash = crypto_clone_ahash(cpool[id].hash);
+ if (IS_ERR(hash)) {
+ rcu_read_unlock_bh();
+ return PTR_ERR(hash);
+ }
+
+ c->req = ahash_request_alloc(hash, GFP_ATOMIC);
+ if (!c->req) {
+ crypto_free_ahash(hash);
+ rcu_read_unlock_bh();
+ return -ENOMEM;
+ }
+ ahash_request_set_callback(c->req, 0, NULL, NULL);
+
+ /* Pairs with tcp_sigpool_reserve_scratch(), scratch area is
+ * valid (allocated) until tcp_sigpool_end().
+ */
+ c->scratch = rcu_dereference_bh(*this_cpu_ptr(&sigpool_scratch));
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_start);
+
+void tcp_sigpool_end(struct tcp_sigpool *c)
+{
+ struct crypto_ahash *hash = crypto_ahash_reqtfm(c->req);
+
+ rcu_read_unlock_bh();
+ ahash_request_free(c->req);
+ crypto_free_ahash(hash);
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_end);
+
+/**
+ * tcp_sigpool_algo - return algorithm of tcp_sigpool
+ * @id: tcp_sigpool that was previously allocated by tcp_sigpool_alloc_ahash()
+ * @buf: buffer to return name of algorithm
+ * @buf_len: size of @buf
+ */
+size_t tcp_sigpool_algo(unsigned int id, char *buf, size_t buf_len)
+{
+ if (WARN_ON_ONCE(id > cpool_populated || !cpool[id].alg))
+ return -EINVAL;
+
+ return strscpy(buf, cpool[id].alg, buf_len);
+}
+EXPORT_SYMBOL_GPL(tcp_sigpool_algo);
+
+/**
+ * tcp_sigpool_hash_skb_data - hash data in skb with initialized tcp_sigpool
+ * @hp: tcp_sigpool pointer
+ * @skb: buffer to add sign for
+ * @header_len: TCP header length for this segment
+ */
+int tcp_sigpool_hash_skb_data(struct tcp_sigpool *hp,
+ const struct sk_buff *skb,
+ unsigned int header_len)
+{
+ struct scatterlist sg;
+ const struct tcphdr *tp = tcp_hdr(skb);
+ struct ahash_request *req = hp->req;
+ unsigned int i;
+ const unsigned int head_data_len = skb_headlen(skb) > header_len ?
+ skb_headlen(skb) - header_len : 0;
+ const struct skb_shared_info *shi = skb_shinfo(skb);
+ struct sk_buff *frag_iter;
+
+ sg_init_table(&sg, 1);
+
+ sg_set_buf(&sg, ((u8 *)tp) + header_len, head_data_len);
+ ahash_request_set_crypt(req, &sg, NULL, head_data_len);
+ if (crypto_ahash_update(req))
+ return 1;
+
+ for (i = 0; i < shi->nr_frags; ++i) {
+ const skb_frag_t *f = &shi->frags[i];
+ unsigned int offset = skb_frag_off(f);
+ struct page *page = skb_frag_page(f) + (offset >> PAGE_SHIFT);
+
+ sg_set_page(&sg, page, skb_frag_size(f),
+ offset_in_page(offset));
+ ahash_request_set_crypt(req, &sg, NULL, skb_frag_size(f));
+ if (crypto_ahash_update(req))
+ return 1;
+ }
+
+ skb_walk_frags(skb, frag_iter)
+ if (tcp_sigpool_hash_skb_data(hp, frag_iter, 0))
+ return 1;
+
+ return 0;
+}
+EXPORT_SYMBOL(tcp_sigpool_hash_skb_data);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Per-CPU pool of crypto requests");
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 7132eb213a7a..7e0c43cb3fd8 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -674,7 +674,7 @@ static int tcp_v6_parse_md5_keys(struct sock *sk, int optname,
cmd.tcpm_key, cmd.tcpm_keylen);
}

-static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
+static int tcp_v6_md5_hash_headers(struct tcp_sigpool *hp,
const struct in6_addr *daddr,
const struct in6_addr *saddr,
const struct tcphdr *th, int nbytes)
@@ -695,39 +695,36 @@ static int tcp_v6_md5_hash_headers(struct tcp_md5sig_pool *hp,
_th->check = 0;

sg_init_one(&sg, bp, sizeof(*bp) + sizeof(*th));
- ahash_request_set_crypt(hp->md5_req, &sg, NULL,
+ ahash_request_set_crypt(hp->req, &sg, NULL,
sizeof(*bp) + sizeof(*th));
- return crypto_ahash_update(hp->md5_req);
+ return crypto_ahash_update(hp->req);
}

static int tcp_v6_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key,
const struct in6_addr *daddr, struct in6_addr *saddr,
const struct tcphdr *th)
{
- struct tcp_md5sig_pool *hp;
- struct ahash_request *req;
+ struct tcp_sigpool hp;

- hp = tcp_get_md5sig_pool();
- if (!hp)
- goto clear_hash_noput;
- req = hp->md5_req;
+ if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp))
+ goto clear_hash_nostart;

- if (crypto_ahash_init(req))
+ if (crypto_ahash_init(hp.req))
goto clear_hash;
- if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, th->doff << 2))
+ if (tcp_v6_md5_hash_headers(&hp, daddr, saddr, th, th->doff << 2))
goto clear_hash;
- if (tcp_md5_hash_key(hp, key))
+ if (tcp_md5_hash_key(&hp, key))
goto clear_hash;
- ahash_request_set_crypt(req, NULL, md5_hash, 0);
- if (crypto_ahash_final(req))
+ ahash_request_set_crypt(hp.req, NULL, md5_hash, 0);
+ if (crypto_ahash_final(hp.req))
goto clear_hash;

- tcp_put_md5sig_pool();
+ tcp_sigpool_end(&hp);
return 0;

clear_hash:
- tcp_put_md5sig_pool();
-clear_hash_noput:
+ tcp_sigpool_end(&hp);
+clear_hash_nostart:
memset(md5_hash, 0, 16);
return 1;
}
@@ -738,8 +735,7 @@ static int tcp_v6_md5_hash_skb(char *md5_hash,
const struct sk_buff *skb)
{
const struct in6_addr *saddr, *daddr;
- struct tcp_md5sig_pool *hp;
- struct ahash_request *req;
+ struct tcp_sigpool hp;
const struct tcphdr *th = tcp_hdr(skb);

if (sk) { /* valid for establish/request sockets */
@@ -751,30 +747,28 @@ static int tcp_v6_md5_hash_skb(char *md5_hash,
daddr = &ip6h->daddr;
}

- hp = tcp_get_md5sig_pool();
- if (!hp)
- goto clear_hash_noput;
- req = hp->md5_req;
+ if (tcp_sigpool_start(tcp_md5_sigpool_id, &hp))
+ goto clear_hash_nostart;

- if (crypto_ahash_init(req))
+ if (crypto_ahash_init(hp.req))
goto clear_hash;

- if (tcp_v6_md5_hash_headers(hp, daddr, saddr, th, skb->len))
+ if (tcp_v6_md5_hash_headers(&hp, daddr, saddr, th, skb->len))
goto clear_hash;
- if (tcp_md5_hash_skb_data(hp, skb, th->doff << 2))
+ if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2))
goto clear_hash;
- if (tcp_md5_hash_key(hp, key))
+ if (tcp_md5_hash_key(&hp, key))
goto clear_hash;
- ahash_request_set_crypt(req, NULL, md5_hash, 0);
- if (crypto_ahash_final(req))
+ ahash_request_set_crypt(hp.req, NULL, md5_hash, 0);
+ if (crypto_ahash_final(hp.req))
goto clear_hash;

- tcp_put_md5sig_pool();
+ tcp_sigpool_end(&hp);
return 0;

clear_hash:
- tcp_put_md5sig_pool();
-clear_hash_noput:
+ tcp_sigpool_end(&hp);
+clear_hash_nostart:
memset(md5_hash, 0, 16);
return 1;
}
--
2.40.0

2023-06-14 23:21:59

by Dmitry Safonov

Subject: [PATCH v7 22/22] net/tcp: Add TCP_AO_REPAIR

Add TCP_AO_REPAIR setsockopt(), getsockopt(). They let a user to repair
TCP-AO ISNs/SNEs. Also let the user hack around when (tp->repair) is on
and add ao_info on a socket in any supported state.
As SNEs now can be read/written at any moment, use
WRITE_ONCE()/READ_ONCE() to set/read them.

Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp_ao.h | 14 ++++++
include/uapi/linux/tcp.h | 10 ++++
net/ipv4/tcp.c | 24 +++++++---
net/ipv4/tcp_ao.c | 100 ++++++++++++++++++++++++++++++++++++---
net/ipv4/tcp_input.c | 12 ++---
net/ipv4/tcp_ipv4.c | 4 +-
net/ipv4/tcp_output.c | 3 +-
net/ipv6/tcp_ipv6.c | 4 +-
8 files changed, 147 insertions(+), 24 deletions(-)

diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 3449d093143e..54eec4859b35 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -182,6 +182,8 @@ void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp);
bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code);
int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen);
int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen);
+int tcp_ao_get_repair(struct sock *sk, sockptr_t optval, sockptr_t optlen);
+int tcp_ao_set_repair(struct sock *sk, sockptr_t optval, unsigned int optlen);
enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
const struct request_sock *req, int l3index,
@@ -303,6 +305,18 @@ static inline int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockpt
{
return -ENOPROTOOPT;
}
+
+static inline int tcp_ao_get_repair(struct sock *sk,
+ sockptr_t optval, sockptr_t optlen)
+{
+ return -ENOPROTOOPT;
+}
+
+static inline int tcp_ao_set_repair(struct sock *sk,
+ sockptr_t optval, unsigned int optlen)
+{
+ return -ENOPROTOOPT;
+}
#endif

#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 979ff960fddb..a0dd4612a37c 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -133,6 +133,7 @@ enum {
#define TCP_AO_DEL_KEY 39 /* Delete MKT */
#define TCP_AO_INFO 40 /* Set/list TCP-AO per-socket options */
#define TCP_AO_GET_KEYS 41 /* List MKT(s) */
+#define TCP_AO_REPAIR 42 /* Get/Set SNEs and ISNs */

#define TCP_REPAIR_ON 1
#define TCP_REPAIR_OFF 0
@@ -445,6 +446,15 @@ struct tcp_ao_getsockopt { /* getsockopt(TCP_AO_GET_KEYS) */
__u64 pkt_bad; /* out: segments that failed verification */
} __attribute__((aligned(8)));

+struct tcp_ao_repair { /* {s,g}etsockopt(TCP_AO_REPAIR) */
+ __be32 snt_isn;
+ __be32 rcv_isn;
+ __u32 snd_sne;
+ __u32 snd_sne_seq;
+ __u32 rcv_sne;
+ __u32 rcv_sne_seq;
+} __attribute__((aligned(8)));
+
/* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */

#define TCP_RECEIVE_ZEROCOPY_FLAG_TLB_CLEAN_HINT 0x1
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index c4143b67a67b..b814a3ce9e69 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -3714,20 +3714,28 @@ int do_tcp_setsockopt(struct sock *sk, int level, int optname,
__tcp_sock_set_quickack(sk, val);
break;

+ case TCP_AO_REPAIR:
+ err = tcp_ao_set_repair(sk, optval, optlen);
+ break;
#ifdef CONFIG_TCP_AO
case TCP_AO_ADD_KEY:
case TCP_AO_DEL_KEY:
case TCP_AO_INFO: {
/* If this is the first TCP-AO setsockopt() on the socket,
- * sk_state has to be LISTEN or CLOSE
+ * sk_state has to be LISTEN or CLOSE. Allow TCP_REPAIR
+ * in any state.
*/
- if (((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)) ||
- rcu_dereference_protected(tcp_sk(sk)->ao_info,
+ if ((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE))
+ goto ao_parse;
+ if (rcu_dereference_protected(tcp_sk(sk)->ao_info,
lockdep_sock_is_held(sk)))
- err = tp->af_specific->ao_parse(sk, optname, optval,
- optlen);
- else
- err = -EISCONN;
+ goto ao_parse;
+ if (tp->repair)
+ goto ao_parse;
+ err = -EISCONN;
+ break;
+ao_parse:
+ err = tp->af_specific->ao_parse(sk, optname, optval, optlen);
break;
}
#endif
@@ -4394,6 +4402,8 @@ int do_tcp_getsockopt(struct sock *sk, int level,
return err;
}
#endif
+ case TCP_AO_REPAIR:
+ return tcp_ao_get_repair(sk, optval, optlen);
case TCP_AO_GET_KEYS:
case TCP_AO_INFO: {
int err;
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index d8a02cf0ba3c..6fe745e0b04c 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -1061,8 +1061,8 @@ void tcp_ao_finish_connect(struct sock *sk, struct sk_buff *skb)

ao->risn = tcp_hdr(skb)->seq;

- ao->rcv_sne = 0;
- ao->rcv_sne_seq = ntohl(tcp_hdr(skb)->seq);
+ WRITE_ONCE(ao->rcv_sne, 0);
+ WRITE_ONCE(ao->rcv_sne_seq, ntohl(tcp_hdr(skb)->seq));

hlist_for_each_entry_rcu(key, &ao->head, node) {
tcp_ao_cache_traffic_keys(sk, ao, key);
@@ -1452,6 +1452,16 @@ static struct tcp_ao_info *setsockopt_ao_info(struct sock *sk)
return ERR_PTR(-ESOCKTNOSUPPORT);
}

+static struct tcp_ao_info *getsockopt_ao_info(struct sock *sk)
+{
+ if (sk_fullsock(sk))
+ return rcu_dereference(tcp_sk(sk)->ao_info);
+ else if (sk->sk_state == TCP_TIME_WAIT)
+ return rcu_dereference(tcp_twsk(sk)->ao_info);
+
+ return ERR_PTR(-ESOCKTNOSUPPORT);
+}
+
#define TCP_AO_KEYF_ALL (TCP_AO_KEYF_IFINDEX | TCP_AO_KEYF_EXCLUDE_OPT)
#define TCP_AO_GET_KEYF_VALID (TCP_AO_KEYF_IFINDEX)

@@ -1586,11 +1596,13 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
if (ret < 0)
goto err_free_sock;

- /* Change this condition if we allow adding keys in states
- * like close_wait, syn_sent or fin_wait...
- */
- if (sk->sk_state == TCP_ESTABLISHED)
+ if (!((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE))) {
tcp_ao_cache_traffic_keys(sk, ao_info, key);
+ if (first) {
+ ao_info->current_key = key;
+ ao_info->rnext_key = key;
+ }
+ }

tcp_ao_link_mkt(ao_info, key);
if (first) {
@@ -1837,6 +1849,8 @@ static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
if (IS_ERR(ao_info))
return PTR_ERR(ao_info);
if (!ao_info) {
+ if (!((1 << sk->sk_state) & (TCPF_LISTEN | TCPF_CLOSE)))
+ return -EINVAL;
ao_info = tcp_ao_alloc_info(GFP_KERNEL);
if (!ao_info)
return -ENOMEM;
@@ -2219,3 +2233,77 @@ int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen)
return 0;
}

+int tcp_ao_set_repair(struct sock *sk, sockptr_t optval, unsigned int optlen)
+{
+ struct tcp_sock *tp = tcp_sk(sk);
+ struct tcp_ao_repair cmd;
+ struct tcp_ao_key *key;
+ struct tcp_ao_info *ao;
+ int err;
+
+ if (optlen < sizeof(cmd))
+ return -EINVAL;
+
+ err = copy_struct_from_sockptr(&cmd, sizeof(cmd), optval, optlen);
+ if (err)
+ return err;
+
+ if (!tp->repair)
+ return -EPERM;
+
+ ao = setsockopt_ao_info(sk);
+ if (IS_ERR(ao))
+ return PTR_ERR(ao);
+ if (!ao)
+ return -ENOENT;
+
+ WRITE_ONCE(ao->lisn, cmd.snt_isn);
+ WRITE_ONCE(ao->risn, cmd.rcv_isn);
+ WRITE_ONCE(ao->snd_sne, cmd.snd_sne);
+ WRITE_ONCE(ao->snd_sne_seq, cmd.snd_sne_seq);
+ WRITE_ONCE(ao->rcv_sne, cmd.rcv_sne);
+ WRITE_ONCE(ao->rcv_sne_seq, cmd.rcv_sne_seq);
+
+ hlist_for_each_entry_rcu(key, &ao->head, node)
+ tcp_ao_cache_traffic_keys(sk, ao, key);
+
+ return 0;
+}
+
+int tcp_ao_get_repair(struct sock *sk, sockptr_t optval, sockptr_t optlen)
+{
+ struct tcp_sock *tp = tcp_sk(sk);
+ struct tcp_ao_repair opt;
+ struct tcp_ao_info *ao;
+ int len;
+
+ if (copy_from_sockptr(&len, optlen, sizeof(int)))
+ return -EFAULT;
+
+ if (len <= 0)
+ return -EINVAL;
+
+ if (!tp->repair)
+ return -EPERM;
+
+ rcu_read_lock();
+ ao = getsockopt_ao_info(sk);
+ if (IS_ERR(ao))
+ return PTR_ERR(ao);
+ if (!ao) {
+ rcu_read_unlock();
+ return -ENOENT;
+ }
+
+ opt.snt_isn = ao->lisn;
+ opt.rcv_isn = ao->risn;
+ opt.snd_sne = READ_ONCE(ao->snd_sne);
+ opt.snd_sne_seq = READ_ONCE(ao->snd_sne_seq);
+ opt.rcv_sne = READ_ONCE(ao->rcv_sne);
+ opt.rcv_sne_seq = READ_ONCE(ao->rcv_sne_seq);
+ rcu_read_unlock();
+
+ if (copy_to_sockptr(optval, &opt, len))
+ return -EFAULT;
+ return 0;
+}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index c0c18b05fd1c..1bd47cb8fb9a 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -3536,8 +3536,8 @@ static void tcp_snd_sne_update(struct tcp_sock *tp, u32 ack)
lockdep_sock_is_held((struct sock *)tp));
if (ao) {
if (ack < ao->snd_sne_seq)
- ao->snd_sne++;
- ao->snd_sne_seq = ack;
+ WRITE_ONCE(ao->snd_sne, ao->snd_sne + 1);
+ WRITE_ONCE(ao->snd_sne_seq, ack);
}
#endif
}
@@ -3565,8 +3565,8 @@ static void tcp_rcv_sne_update(struct tcp_sock *tp, u32 seq)
lockdep_sock_is_held((struct sock *)tp));
if (ao) {
if (seq < ao->rcv_sne_seq)
- ao->rcv_sne++;
- ao->rcv_sne_seq = seq;
+ WRITE_ONCE(ao->rcv_sne, ao->rcv_sne + 1);
+ WRITE_ONCE(ao->rcv_sne_seq, seq);
}
#endif
}
@@ -6416,8 +6416,8 @@ static int tcp_rcv_synsent_state_process(struct sock *sk, struct sk_buff *skb,
lockdep_sock_is_held(sk));
if (ao) {
ao->risn = th->seq;
- ao->rcv_sne = 0;
- ao->rcv_sne_seq = ntohl(th->seq);
+ WRITE_ONCE(ao->rcv_sne, 0);
+ WRITE_ONCE(ao->rcv_sne_seq, ntohl(th->seq));
}
#endif
tcp_set_state(sk, TCP_SYN_RECV);
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 211d0edd6eea..229317e38a5e 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1040,8 +1040,8 @@ static void tcp_v4_timewait_ack(struct sock *sk, struct sk_buff *skb)
* below since sne probably doesn't change once we are
* in timewait state.
*/
- ao_sne = tcp_ao_compute_sne(ao_info->snd_sne,
- ao_info->snd_sne_seq,
+ ao_sne = tcp_ao_compute_sne(READ_ONCE(ao_info->snd_sne),
+ READ_ONCE(ao_info->snd_sne_seq),
tcptw->tw_snd_nxt);
rnext_key = READ_ONCE(ao_info->rnext_key);
rcv_next = rnext_key->rcvid;
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index f055ef225c37..e68ae7b896b7 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -1455,7 +1455,8 @@ static int __tcp_transmit_skb(struct sock *sk, struct sk_buff *skb,
} else {
traffic_key = snd_other_key(ao_key);
}
- sne = tcp_ao_compute_sne(ao->snd_sne, ao->snd_sne_seq,
+ sne = tcp_ao_compute_sne(READ_ONCE(ao->snd_sne),
+ READ_ONCE(ao->snd_sne_seq),
ntohl(th->seq));
tp->af_specific->calc_ao_hash(opts.hash_location, ao_key, sk, skb,
traffic_key,
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 6b39da6c477e..ba9d4571b265 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -1190,8 +1190,8 @@ static void tcp_v6_timewait_ack(struct sock *sk, struct sk_buff *skb)
/* rcv_next switches to our rcv_next */
rnext_key = READ_ONCE(ao_info->rnext_key);
rcv_next = rnext_key->rcvid;
- ao_sne = tcp_ao_compute_sne(ao_info->snd_sne,
- ao_info->snd_sne_seq,
+ ao_sne = tcp_ao_compute_sne(READ_ONCE(ao_info->snd_sne),
+ READ_ONCE(ao_info->snd_sne_seq),
tcptw->tw_snd_nxt);
}
#endif
--
2.40.0

2023-06-14 23:27:44

by Dmitry Safonov

Subject: [PATCH v7 18/22] net/tcp: Add TCP-AO getsockopt()s

Introduce getsockopt(TCP_AO_GET_KEYS) that lets a user get TCP-AO keys
and their properties from a socket. The user can provide a filter
to match the specific key to be dumped or ::get_all = 1 may be
used to dump all keys in one syscall.

Add another getsockopt(TCP_AO_INFO) for providing per-socket/per-ao_info
stats: packet counters, Current_key/RNext_key and flags like
::ao_required and ::accept_icmps.

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp_ao.h | 12 ++
include/uapi/linux/tcp.h | 63 +++++++--
net/ipv4/tcp.c | 13 ++
net/ipv4/tcp_ao.c | 295 +++++++++++++++++++++++++++++++++++++++
4 files changed, 369 insertions(+), 14 deletions(-)

diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index b97e297fe080..49402458b69d 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -177,6 +177,8 @@ void tcp_ao_destroy_sock(struct sock *sk, bool twsk);
u32 tcp_ao_compute_sne(u32 sne, u32 seq, u32 new_seq);
void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp);
bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code);
+int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen);
+int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen);
enum skb_drop_reason tcp_inbound_ao_hash(struct sock *sk,
const struct sk_buff *skb, unsigned short int family,
const struct request_sock *req,
@@ -290,6 +292,16 @@ static inline void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw,
static inline void tcp_ao_connect_init(struct sock *sk)
{
}
+
+static inline int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen)
+{
+ return -ENOPROTOOPT;
+}
+
+static inline int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen)
+{
+ return -ENOPROTOOPT;
+}
#endif

#if defined(CONFIG_TCP_MD5SIG) || defined(CONFIG_TCP_AO)
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 3275ade3293a..1109093bbb24 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -131,7 +131,8 @@ enum {

#define TCP_AO_ADD_KEY 38 /* Add/Set MKT */
#define TCP_AO_DEL_KEY 39 /* Delete MKT */
-#define TCP_AO_INFO 40 /* Modify TCP-AO per-socket options */
+#define TCP_AO_INFO 40 /* Set/list TCP-AO per-socket options */
+#define TCP_AO_GET_KEYS 41 /* List MKT(s) */

#define TCP_REPAIR_ON 1
#define TCP_REPAIR_OFF 0
@@ -392,21 +393,55 @@ struct tcp_ao_del { /* setsockopt(TCP_AO_DEL_KEY) */
__u8 keyflags; /* see TCP_AO_KEYF_ */
} __attribute__((aligned(8)));

-struct tcp_ao_info_opt { /* setsockopt(TCP_AO_INFO) */
- __u32 set_current :1, /* corresponding ::current_key */
- set_rnext :1, /* corresponding ::rnext */
- ao_required :1, /* don't accept non-AO connects */
- set_counters :1, /* set/clear ::pkt_* counters */
- accept_icmps :1, /* accept incoming ICMPs */
+struct tcp_ao_info_opt { /* setsockopt(TCP_AO_INFO), getsockopt(TCP_AO_INFO) */
+ /* Here 'in' is for setsockopt(), 'out' is for getsockopt() */
+ __u32 set_current :1, /* in/out: corresponding ::current_key */
+ set_rnext :1, /* in/out: corresponding ::rnext */
+ ao_required :1, /* in/out: don't accept non-AO connects */
+ set_counters :1, /* in: set/clear ::pkt_* counters */
+ accept_icmps :1, /* in/out: accept incoming ICMPs */
reserved :27; /* must be 0 */
__u16 reserved2; /* padding, must be 0 */
- __u8 current_key; /* KeyID to set as Current_key */
- __u8 rnext; /* KeyID to set as Rnext_key */
- __u64 pkt_good; /* verified segments */
- __u64 pkt_bad; /* failed verification */
- __u64 pkt_key_not_found; /* could not find a key to verify */
- __u64 pkt_ao_required; /* segments missing TCP-AO sign */
- __u64 pkt_dropped_icmp; /* ICMPs that were ignored */
+ __u8 current_key; /* in/out: KeyID of Current_key */
+ __u8 rnext; /* in/out: keyid of RNext_key */
+ __u64 pkt_good; /* in/out: verified segments */
+ __u64 pkt_bad; /* in/out: failed verification */
+ __u64 pkt_key_not_found; /* in/out: could not find a key to verify */
+ __u64 pkt_ao_required; /* in/out: segments missing TCP-AO sign */
+ __u64 pkt_dropped_icmp; /* in/out: ICMPs that were ignored */
+} __attribute__((aligned(8)));
+
+struct tcp_ao_getsockopt { /* getsockopt(TCP_AO_GET_KEYS) */
+ struct __kernel_sockaddr_storage addr; /* in/out: dump keys for peer
+ * with this address/prefix
+ */
+ char alg_name[64]; /* out: crypto hash algorithm */
+ __u8 key[TCP_AO_MAXKEYLEN];
+ __u32 nkeys; /* in: size of the userspace buffer
+ * @optval, measured in @optlen - the
+ * sizeof(struct tcp_ao_getsockopt)
+ * out: number of keys that matched
+ */
+ __u16 is_current :1, /* in: match and dump Current_key,
+ * out: the dumped key is Current_key
+ */
+
+ is_rnext :1, /* in: match and dump RNext_key,
+ * out: the dumped key is RNext_key
+ */
+ get_all :1, /* in: dump all keys */
+ reserved :13; /* padding, must be 0 */
+ __u8 sndid; /* in/out: dump keys with SendID */
+ __u8 rcvid; /* in/out: dump keys with RecvID */
+ __u8 prefix; /* in/out: dump keys with address/prefix */
+ __u8 maclen; /* out: key's length of authentication
+ * code (hash)
+ */
+ __u8 keyflags; /* in/out: see TCP_AO_KEYF_ */
+ __u8 keylen; /* out: length of ::key */
+ __s32 ifindex; /* in/out: L3 dev index for VRF */
+ __u64 pkt_good; /* out: verified segments */
+ __u64 pkt_bad; /* out: segments that failed verification */
} __attribute__((aligned(8)));

/* setsockopt(fd, IPPROTO_TCP, TCP_ZEROCOPY_RECEIVE, ...) */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index dd1885d7a81f..c4143b67a67b 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -4394,6 +4394,19 @@ int do_tcp_getsockopt(struct sock *sk, int level,
return err;
}
#endif
+ case TCP_AO_GET_KEYS:
+ case TCP_AO_INFO: {
+ int err;
+
+ sockopt_lock_sock(sk);
+ if (optname == TCP_AO_GET_KEYS)
+ err = tcp_ao_get_mkts(sk, optval, optlen);
+ else
+ err = tcp_ao_get_sock_info(sk, optval, optlen);
+ sockopt_release_sock(sk);
+
+ return err;
+ }
default:
return -ENOPROTOOPT;
}
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 1a5a6b38b3b4..6e0c0b2fd011 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -1807,3 +1807,298 @@ int tcp_v4_parse_ao(struct sock *sk, int cmd, sockptr_t optval, int optlen)
return tcp_parse_ao(sk, cmd, AF_INET, optval, optlen);
}

+/* tcp_ao_copy_mkts_to_user(ao_info, optval, optlen)
+ *
+ * @ao_info: struct tcp_ao_info on the socket that
+ * socket getsockopt(TCP_AO_GET_KEYS) is executed on
+ * @optval: pointer to array of tcp_ao_getsockopt structures in user space.
+ * Must be != NULL.
+ * @optlen: pointer to size of tcp_ao_getsockopt structure.
+ * Must be != NULL.
+ *
+ * Return value: 0 on success, a negative error number otherwise.
+ *
+ * optval points to an array of tcp_ao_getsockopt structures in user space.
+ * optval[0] is used as both input and output to getsockopt. It determines
+ * which keys are returned by the kernel.
+ * optval[0].nkeys is the size of the array in user space. On return it contains
+ * the number of keys matching the search criteria.
+ * If tcp_ao_getsockopt::get_all is set, then all keys in the socket are
+ * returned, otherwise only keys matching <addr, prefix, sndid, rcvid>
+ * in optval[0] are returned.
+ * optlen is also used as both input and output. The user provides the size
+ * of struct tcp_ao_getsockopt in user space, and the kernel returns the size
+ * of the structure in kernel space.
+ * The size of struct tcp_ao_getsockopt may differ between user and kernel.
+ * There are three cases to consider:
+ * * If usize == ksize, then keys are copied verbatim.
+ * * If usize < ksize, then the userspace has passed an old struct to a
+ * newer kernel. The rest of the trailing bytes in optval[0]
+ * (ksize - usize) are interpreted as 0 by the kernel.
+ * * If usize > ksize, then the userspace has passed a new struct to an
+ * older kernel. The trailing bytes unknown to the kernel (usize - ksize)
+ * are checked to ensure they are zeroed, otherwise -E2BIG is returned.
+ * On return the kernel fills in min(usize, ksize) in each entry of the array.
+ * The layout of the fields in the user and kernel structures is expected to
+ * be the same (including in the 32bit vs 64bit case).
+ */
+static int tcp_ao_copy_mkts_to_user(struct tcp_ao_info *ao_info,
+ sockptr_t optval, sockptr_t optlen)
+{
+ struct tcp_ao_getsockopt opt_in, opt_out;
+ struct tcp_ao_key *key, *current_key;
+ bool do_address_matching = true;
+ union tcp_ao_addr *addr = NULL;
+ unsigned int max_keys; /* maximum number of keys to copy to user */
+ size_t out_offset = 0;
+ size_t bytes_to_write; /* number of bytes to write to user level */
+ int err, user_len;
+ u32 matched_keys; /* keys from ao_info matched so far */
+ int optlen_out;
+ u16 port = 0;
+
+ if (copy_from_sockptr(&user_len, optlen, sizeof(int)))
+ return -EFAULT;
+
+ if (user_len <= 0)
+ return -EINVAL;
+
+ memset(&opt_in, 0, sizeof(struct tcp_ao_getsockopt));
+ err = copy_struct_from_sockptr(&opt_in, sizeof(opt_in),
+ optval, user_len);
+ if (err < 0)
+ return err;
+
+ if (opt_in.pkt_good || opt_in.pkt_bad)
+ return -EINVAL;
+
+ if (opt_in.reserved != 0)
+ return -EINVAL;
+
+ max_keys = opt_in.nkeys;
+
+ if (opt_in.get_all || opt_in.is_current || opt_in.is_rnext) {
+ if (opt_in.get_all && (opt_in.is_current || opt_in.is_rnext))
+ return -EINVAL;
+ do_address_matching = false;
+ }
+
+ switch (opt_in.addr.ss_family) {
+ case AF_INET: {
+ struct sockaddr_in *sin;
+ __be32 mask;
+
+ sin = (struct sockaddr_in *)&opt_in.addr;
+ port = sin->sin_port;
+ addr = (union tcp_ao_addr *)&sin->sin_addr;
+
+ if (opt_in.prefix > 32)
+ return -EINVAL;
+
+ if (sin->sin_addr.s_addr == INADDR_ANY &&
+ opt_in.prefix != 0)
+ return -EINVAL;
+
+ mask = inet_make_mask(opt_in.prefix);
+ if (sin->sin_addr.s_addr & ~mask)
+ return -EINVAL;
+
+ break;
+ }
+ case AF_INET6: {
+ struct sockaddr_in6 *sin6;
+ struct in6_addr *addr6;
+
+ sin6 = (struct sockaddr_in6 *)&opt_in.addr;
+ addr = (union tcp_ao_addr *)&sin6->sin6_addr;
+ addr6 = &sin6->sin6_addr;
+ port = sin6->sin6_port;
+
+ /* We don't have to change family and @addr here if
+ * ipv6_addr_v4mapped() like in key adding:
+ * tcp_ao_key_cmp() does it. Do the sanity checks though.
+ */
+ if (opt_in.prefix != 0) {
+ if (ipv6_addr_v4mapped(addr6)) {
+ __be32 mask, addr4 = addr6->s6_addr32[3];
+
+ if (opt_in.prefix > 32 ||
+ addr4 == INADDR_ANY)
+ return -EINVAL;
+ mask = inet_make_mask(opt_in.prefix);
+ if (addr4 & ~mask)
+ return -EINVAL;
+ } else {
+ struct in6_addr pfx;
+
+ if (ipv6_addr_any(addr6) ||
+ opt_in.prefix > 128)
+ return -EINVAL;
+
+ ipv6_addr_prefix(&pfx, addr6, opt_in.prefix);
+ if (ipv6_addr_cmp(&pfx, addr6))
+ return -EINVAL;
+ }
+ } else if (!ipv6_addr_any(addr6)) {
+ return -EINVAL;
+ }
+ break;
+ }
+ case 0:
+ if (!do_address_matching)
+ break;
+ fallthrough;
+ default:
+ return -EAFNOSUPPORT;
+ }
+
+ if (!do_address_matching) {
+ /* We could just ignore those, but let's do stricter checks */
+ if (addr || port)
+ return -EINVAL;
+ if (opt_in.prefix || opt_in.sndid || opt_in.rcvid)
+ return -EINVAL;
+ }
+
+ bytes_to_write = min_t(int, user_len, sizeof(struct tcp_ao_getsockopt));
+ matched_keys = 0;
+ /* May change in RX, while we're dumping, pre-fetch it */
+ current_key = READ_ONCE(ao_info->current_key);
+
+ hlist_for_each_entry_rcu(key, &ao_info->head, node) {
+ if (opt_in.get_all)
+ goto match;
+
+ if (opt_in.is_current || opt_in.is_rnext) {
+ if (opt_in.is_current && key == current_key)
+ goto match;
+ if (opt_in.is_rnext && key == ao_info->rnext_key)
+ goto match;
+ continue;
+ }
+
+ if (tcp_ao_key_cmp(key, addr, opt_in.prefix,
+ opt_in.addr.ss_family,
+ opt_in.sndid, opt_in.rcvid, port) != 0)
+ continue;
+match:
+ matched_keys++;
+ if (matched_keys > max_keys)
+ continue;
+
+ memset(&opt_out, 0, sizeof(struct tcp_ao_getsockopt));
+
+ if (key->family == AF_INET) {
+ struct sockaddr_in *sin_out = (struct sockaddr_in *)&opt_out.addr;
+
+ sin_out->sin_family = key->family;
+ sin_out->sin_port = ntohs(key->port);
+ memcpy(&sin_out->sin_addr, &key->addr, sizeof(struct in_addr));
+ } else {
+ struct sockaddr_in6 *sin6_out = (struct sockaddr_in6 *)&opt_out.addr;
+
+ sin6_out->sin6_family = key->family;
+ sin6_out->sin6_port = ntohs(key->port);
+ memcpy(&sin6_out->sin6_addr, &key->addr, sizeof(struct in6_addr));
+ }
+ opt_out.sndid = key->sndid;
+ opt_out.rcvid = key->rcvid;
+ opt_out.prefix = key->prefixlen;
+ opt_out.keyflags = key->keyflags;
+ opt_out.is_current = (key == current_key);
+ opt_out.is_rnext = (key == ao_info->rnext_key);
+ opt_out.nkeys = 0;
+ opt_out.maclen = key->maclen;
+ opt_out.keylen = key->keylen;
+ opt_out.pkt_good = atomic64_read(&key->pkt_good);
+ opt_out.pkt_bad = atomic64_read(&key->pkt_bad);
+ memcpy(&opt_out.key, key->key, key->keylen);
+ tcp_sigpool_algo(key->tcp_sigpool_id, opt_out.alg_name, 64);
+
+ /* Copy key to user */
+ if (copy_to_sockptr_offset(optval, out_offset,
+ &opt_out, bytes_to_write))
+ return -EFAULT;
+ out_offset += user_len;
+ }
+
+ optlen_out = (int)sizeof(struct tcp_ao_getsockopt);
+ if (copy_to_sockptr(optlen, &optlen_out, sizeof(int)))
+ return -EFAULT;
+
+ out_offset = offsetof(struct tcp_ao_getsockopt, nkeys);
+ if (copy_to_sockptr_offset(optval, out_offset,
+ &matched_keys, sizeof(u32)))
+ return -EFAULT;
+
+ return 0;
+}
+
+int tcp_ao_get_mkts(struct sock *sk, sockptr_t optval, sockptr_t optlen)
+{
+ struct tcp_ao_info *ao_info;
+
+ ao_info = setsockopt_ao_info(sk);
+ if (IS_ERR(ao_info))
+ return PTR_ERR(ao_info);
+ if (!ao_info)
+ return -ENOENT;
+
+ return tcp_ao_copy_mkts_to_user(ao_info, optval, optlen);
+}
+
+int tcp_ao_get_sock_info(struct sock *sk, sockptr_t optval, sockptr_t optlen)
+{
+ struct tcp_ao_info_opt out, in = {};
+ struct tcp_ao_key *current_key;
+ struct tcp_ao_info *ao;
+ int err, len;
+
+ if (copy_from_sockptr(&len, optlen, sizeof(int)))
+ return -EFAULT;
+
+ if (len <= 0)
+ return -EINVAL;
+
+ /* Copying this "in" only to check ::reserved, ::reserved2,
+ * that may be needed to extend (struct tcp_ao_info_opt) and
+ * what getsockopt() provides in future.
+ */
+ err = copy_struct_from_sockptr(&in, sizeof(in), optval, len);
+ if (err)
+ return err;
+
+ if (in.reserved != 0 || in.reserved2 != 0)
+ return -EINVAL;
+
+ ao = setsockopt_ao_info(sk);
+ if (IS_ERR(ao))
+ return PTR_ERR(ao);
+ if (!ao)
+ return -ENOENT;
+
+ memset(&out, 0, sizeof(out));
+ out.ao_required = ao->ao_required;
+ out.accept_icmps = ao->accept_icmps;
+ out.pkt_good = atomic64_read(&ao->counters.pkt_good);
+ out.pkt_bad = atomic64_read(&ao->counters.pkt_bad);
+ out.pkt_key_not_found = atomic64_read(&ao->counters.key_not_found);
+ out.pkt_ao_required = atomic64_read(&ao->counters.ao_required);
+ out.pkt_dropped_icmp = atomic64_read(&ao->counters.dropped_icmp);
+
+ current_key = READ_ONCE(ao->current_key);
+ if (current_key) {
+ out.set_current = 1;
+ out.current_key = current_key->sndid;
+ }
+ if (ao->rnext_key) {
+ out.set_rnext = 1;
+ out.rnext = ao->rnext_key->rcvid;
+ }
+
+ if (copy_to_sockptr(optval, &out, min_t(int, len, sizeof(out))))
+ return -EFAULT;
+
+ return 0;
+}
+
--
2.40.0

2023-06-14 23:29:21

by Dmitry Safonov

Subject: [PATCH v7 19/22] net/tcp: Allow asynchronous delete for TCP-AO keys (MKTs)

Delete becomes very, very fast - almost free, but after setsockopt()
syscall returns, the key is still alive until next RCU grace period.
Which is fine for listen sockets as userspace needs to be aware of
setsockopt(TCP_AO) and accept() race and resolve it with verification
by getsockopt() after TCP connection was accepted.

The benchmark results (on non-loaded box, worse with more RCU work pending):
> ok 33 Worst case delete 16384 keys: min=5ms max=10ms mean=6.93904ms stddev=0.263421
> ok 34 Add a new key 16384 keys: min=1ms max=4ms mean=2.17751ms stddev=0.147564
> ok 35 Remove random-search 16384 keys: min=5ms max=10ms mean=6.50243ms stddev=0.254999
> ok 36 Remove async 16384 keys: min=0ms max=0ms mean=0.0296107ms stddev=0.0172078

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/uapi/linux/tcp.h | 3 ++-
net/ipv4/tcp_ao.c | 21 ++++++++++++++++++---
2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 1109093bbb24..979ff960fddb 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -383,7 +383,8 @@ struct tcp_ao_del { /* setsockopt(TCP_AO_DEL_KEY) */
__s32 ifindex; /* L3 dev index for VRF */
__u32 set_current :1, /* corresponding ::current_key */
set_rnext :1, /* corresponding ::rnext */
- reserved :30; /* must be 0 */
+ del_async :1, /* only valid for listen sockets */
+ reserved :29; /* must be 0 */
__u16 reserved2; /* padding, must be 0 */
__u8 prefix; /* peer's address prefix */
__u8 sndid; /* SendID for outgoing segments */
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 6e0c0b2fd011..3799432a386a 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -1543,7 +1543,7 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
}

static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info,
- struct tcp_ao_key *key,
+ bool del_async, struct tcp_ao_key *key,
struct tcp_ao_key *new_current,
struct tcp_ao_key *new_rnext)
{
@@ -1551,11 +1551,24 @@ static int tcp_ao_delete_key(struct sock *sk, struct tcp_ao_info *ao_info,

hlist_del_rcu(&key->node);

+ /* Support for async delete on listening sockets: as they don't
+ * need current_key/rnext_key maintaining, we don't need to check
+ * them and we can just free all resources in RCU fashion.
+ */
+ if (del_async) {
+ atomic_sub(tcp_ao_sizeof_key(key), &sk->sk_omem_alloc);
+ call_rcu(&key->rcu, tcp_ao_key_free_rcu);
+ return 0;
+ }
+
/* At this moment another CPU could have looked this key up
* while it was unlinked from the list. Wait for RCU grace period,
* after which the key is off-list and can't be looked up again;
* the rx path [just before RCU came] might have used it and set it
* as current_key (very unlikely).
+ * Free the key with next RCU grace period (in case it was
+ * current_key before tcp_ao_current_rnext() might have
+ * changed it in forced-delete).
*/
synchronize_rcu();
if (new_current)
@@ -1627,6 +1640,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
if (!new_rnext)
return -ENOENT;
}
+ if (cmd.del_async && sk->sk_state != TCP_LISTEN)
+ return -EINVAL;

if (family == AF_INET) {
struct sockaddr_in *sin = (struct sockaddr_in *)&cmd.addr;
@@ -1671,8 +1686,8 @@ static int tcp_ao_del_cmd(struct sock *sk, unsigned short int family,
if (key == new_current || key == new_rnext)
continue;

- return tcp_ao_delete_key(sk, ao_info, key,
- new_current, new_rnext);
+ return tcp_ao_delete_key(sk, ao_info, cmd.del_async, key,
+ new_current, new_rnext);
}
return -ENOENT;
}
--
2.40.0

2023-06-14 23:35:24

by Dmitry Safonov

Subject: [PATCH v7 20/22] net/tcp: Add static_key for TCP-AO

Similarly to TCP-MD5, add a static key to TCP-AO that is patched out
when there are no keys on a machine and dynamically enabled with the
first setsockopt(TCP_AO) adds a key on any socket. The static key is as
well dynamically disabled later when the socket is destructed.

The lifetime of enabled static key here is the same as ao_info: it is
enabled on allocation, passed over from full socket to twsk and
destructed when ao_info is scheduled for destruction.

Signed-off-by: Dmitry Safonov <[email protected]>
---
include/net/tcp.h | 3 +++
include/net/tcp_ao.h | 2 ++
net/ipv4/tcp_ao.c | 23 +++++++++++++++++++++++
net/ipv4/tcp_input.c | 42 ++++++++++++++++++++++++++++--------------
4 files changed, 56 insertions(+), 14 deletions(-)

diff --git a/include/net/tcp.h b/include/net/tcp.h
index 1d3cf13ae66b..6060513ab83d 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -2585,6 +2585,9 @@ static inline bool tcp_ao_required(struct sock *sk, const void *saddr,
struct tcp_ao_info *ao_info;
struct tcp_ao_key *ao_key;

+ if (!static_branch_unlikely(&tcp_ao_needed.key))
+ return false;
+
ao_info = rcu_dereference_check(tcp_sk(sk)->ao_info,
lockdep_sock_is_held(sk));
if (!ao_info)
diff --git a/include/net/tcp_ao.h b/include/net/tcp_ao.h
index 49402458b69d..714a46e30f3f 100644
--- a/include/net/tcp_ao.h
+++ b/include/net/tcp_ao.h
@@ -138,6 +138,8 @@ do { \

#ifdef CONFIG_TCP_AO
/* TCP-AO structures and functions */
+#include <linux/jump_label.h>
+extern struct static_key_false_deferred tcp_ao_needed;

struct tcp4_ao_context {
__be32 saddr;
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index 3799432a386a..2c4a31d8f177 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -17,6 +17,9 @@
#include <net/ipv6.h>
#include <net/icmp.h>

+DEFINE_STATIC_KEY_DEFERRED_FALSE(tcp_ao_needed, HZ);
+EXPORT_SYMBOL_GPL(tcp_ao_needed);
+
int tcp_ao_calc_traffic_key(struct tcp_ao_key *mkt, u8 *key, void *ctx,
unsigned int len)
{
@@ -58,6 +61,9 @@ bool tcp_ao_ignore_icmp(struct sock *sk, int type, int code)
struct tcp_ao_info *ao;
bool ignore_icmp = false;

+ if (!static_branch_unlikely(&tcp_ao_needed.key))
+ return false;
+
/* RFC5925, 7.8:
* >> A TCP-AO implementation MUST default to ignore incoming ICMPv4
* messages of Type 3 (destination unreachable), Codes 2-4 (protocol
@@ -198,6 +204,9 @@ static struct tcp_ao_key *__tcp_ao_do_lookup(const struct sock *sk,
struct tcp_ao_key *key;
struct tcp_ao_info *ao;

+ if (!static_branch_unlikely(&tcp_ao_needed.key))
+ return NULL;
+
ao = rcu_dereference_check(tcp_sk(sk)->ao_info,
lockdep_sock_is_held(sk));
if (!ao)
@@ -292,6 +301,7 @@ void tcp_ao_destroy_sock(struct sock *sk, bool twsk)
}

kfree_rcu(ao, rcu);
+ static_branch_slow_dec_deferred(&tcp_ao_needed);
}

void tcp_ao_time_wait(struct tcp_timewait_sock *tcptw, struct tcp_sock *tp)
@@ -1107,6 +1117,11 @@ int tcp_ao_copy_all_matching(const struct sock *sk, struct sock *newsk,
goto free_and_exit;
}

+ if (!static_key_fast_inc_not_disabled(&tcp_ao_needed.key.key)) {
+ ret = -EUSERS;
+ goto free_and_exit;
+ }
+
key_head = rcu_dereference(hlist_first_rcu(&new_ao->head));
first_key = hlist_entry_safe(key_head, struct tcp_ao_key, node);

@@ -1523,6 +1538,10 @@ static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,

tcp_ao_link_mkt(ao_info, key);
if (first) {
+ if (!static_branch_inc(&tcp_ao_needed.key)) {
+ ret = -EUSERS;
+ goto err_free_sock;
+ }
sk_gso_disable(sk);
rcu_assign_pointer(tcp_sk(sk)->ao_info, ao_info);
}
@@ -1788,6 +1807,10 @@ static int tcp_ao_info_cmd(struct sock *sk, unsigned short int family,
if (new_rnext)
WRITE_ONCE(ao_info->rnext_key, new_rnext);
if (first) {
+ if (!static_branch_inc(&tcp_ao_needed.key)) {
+ err = -EUSERS;
+ goto out;
+ }
sk_gso_disable(sk);
rcu_assign_pointer(tcp_sk(sk)->ao_info, ao_info);
}
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index eed3f7631b4b..c0c18b05fd1c 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -3524,17 +3524,14 @@ static inline bool tcp_may_update_window(const struct tcp_sock *tp,
(ack_seq == tp->snd_wl1 && nwin > tp->snd_wnd);
}

-/* If we update tp->snd_una, also update tp->bytes_acked */
-static void tcp_snd_una_update(struct tcp_sock *tp, u32 ack)
+static void tcp_snd_sne_update(struct tcp_sock *tp, u32 ack)
{
- u32 delta = ack - tp->snd_una;
#ifdef CONFIG_TCP_AO
struct tcp_ao_info *ao;
-#endif

- sock_owned_by_me((struct sock *)tp);
- tp->bytes_acked += delta;
-#ifdef CONFIG_TCP_AO
+ if (!static_branch_unlikely(&tcp_ao_needed.key))
+ return;
+
ao = rcu_dereference_protected(tp->ao_info,
lockdep_sock_is_held((struct sock *)tp));
if (ao) {
@@ -3543,20 +3540,27 @@ static void tcp_snd_una_update(struct tcp_sock *tp, u32 ack)
ao->snd_sne_seq = ack;
}
#endif
+}
+
+/* If we update tp->snd_una, also update tp->bytes_acked */
+static void tcp_snd_una_update(struct tcp_sock *tp, u32 ack)
+{
+ u32 delta = ack - tp->snd_una;
+
+ sock_owned_by_me((struct sock *)tp);
+ tp->bytes_acked += delta;
+ tcp_snd_sne_update(tp, ack);
tp->snd_una = ack;
}

-/* If we update tp->rcv_nxt, also update tp->bytes_received */
-static void tcp_rcv_nxt_update(struct tcp_sock *tp, u32 seq)
+static void tcp_rcv_sne_update(struct tcp_sock *tp, u32 seq)
{
- u32 delta = seq - tp->rcv_nxt;
#ifdef CONFIG_TCP_AO
struct tcp_ao_info *ao;
-#endif

- sock_owned_by_me((struct sock *)tp);
- tp->bytes_received += delta;
-#ifdef CONFIG_TCP_AO
+ if (!static_branch_unlikely(&tcp_ao_needed.key))
+ return;
+
ao = rcu_dereference_protected(tp->ao_info,
lockdep_sock_is_held((struct sock *)tp));
if (ao) {
@@ -3565,6 +3569,16 @@ static void tcp_rcv_nxt_update(struct tcp_sock *tp, u32 seq)
ao->rcv_sne_seq = seq;
}
#endif
+}
+
+/* If we update tp->rcv_nxt, also update tp->bytes_received */
+static void tcp_rcv_nxt_update(struct tcp_sock *tp, u32 seq)
+{
+ u32 delta = seq - tp->rcv_nxt;
+
+ sock_owned_by_me((struct sock *)tp);
+ tp->bytes_received += delta;
+ tcp_rcv_sne_update(tp, seq);
WRITE_ONCE(tp->rcv_nxt, seq);
}

--
2.40.0

2023-06-14 23:38:31

by Dmitry Safonov

Subject: [PATCH v7 17/22] net/tcp: Add option for TCP-AO to (not) hash header

Provide setsockopt() key flag that makes TCP-AO exclude hashing TCP
header for peers that match the key. This is needed for interraction
with middleboxes that may change TCP options, see RFC5925 (9.2).

Co-developed-by: Francesco Ruggeri <[email protected]>
Signed-off-by: Francesco Ruggeri <[email protected]>
Co-developed-by: Salam Noureddine <[email protected]>
Signed-off-by: Salam Noureddine <[email protected]>
Signed-off-by: Dmitry Safonov <[email protected]>
---
include/uapi/linux/tcp.h | 5 +++++
net/ipv4/tcp_ao.c | 8 +++++---
2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index ca7ed18ce67b..3275ade3293a 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -354,6 +354,11 @@ struct tcp_diag_md5sig {
#define TCP_AO_MAXKEYLEN 80

#define TCP_AO_KEYF_IFINDEX (1 << 0) /* L3 ifindex for VRF */
+#define TCP_AO_KEYF_EXCLUDE_OPT (1 << 1) /* "Indicates whether TCP
+ * options other than TCP-AO
+ * are included in the MAC
+ * calculation"
+ */

struct tcp_ao_add { /* setsockopt(TCP_AO_ADD_KEY) */
struct __kernel_sockaddr_storage addr; /* peer's address for the key */
diff --git a/net/ipv4/tcp_ao.c b/net/ipv4/tcp_ao.c
index af4934cb23a5..1a5a6b38b3b4 100644
--- a/net/ipv4/tcp_ao.c
+++ b/net/ipv4/tcp_ao.c
@@ -566,7 +566,8 @@ int tcp_ao_hash_hdr(unsigned short int family, char *ao_hash,
WARN_ON_ONCE(1);
goto clear_hash;
}
- if (tcp_ao_hash_header(&hp, th, false,
+ if (tcp_ao_hash_header(&hp, th,
+ !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
ao_hash, hash_offset, tcp_ao_maclen(key)))
goto clear_hash;
ahash_request_set_crypt(hp.req, NULL, tmp_hash, 0);
@@ -609,7 +610,8 @@ int tcp_ao_hash_skb(unsigned short int family,
goto clear_hash;
if (tcp_ao_hash_pseudoheader(family, sk, skb, &hp, skb->len))
goto clear_hash;
- if (tcp_ao_hash_header(&hp, th, false,
+ if (tcp_ao_hash_header(&hp, th,
+ !!(key->keyflags & TCP_AO_KEYF_EXCLUDE_OPT),
ao_hash, hash_offset, tcp_ao_maclen(key)))
goto clear_hash;
if (tcp_sigpool_hash_skb_data(&hp, skb, th->doff << 2))
@@ -1418,7 +1420,7 @@ static struct tcp_ao_info *setsockopt_ao_info(struct sock *sk)
return ERR_PTR(-ESOCKTNOSUPPORT);
}

-#define TCP_AO_KEYF_ALL (0)
+#define TCP_AO_KEYF_ALL (TCP_AO_KEYF_EXCLUDE_OPT)

static int tcp_ao_add_cmd(struct sock *sk, unsigned short int family,
sockptr_t optval, int optlen)
--
2.40.0

2023-06-15 11:03:33

by Steen Hegelund

Subject: Re: [PATCH v7 01/22] net/tcp: Prepare tcp_md5sig_pool for TCP-AO

Hi Dmitry,

On Thu, 2023-06-15 at 00:09 +0100, Dmitry Safonov wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the
> content is safe
>
> TCP-AO, similarly to TCP-MD5, needs to allocate tfms on a slow-path,
> which is setsockopt() and use crypto ahash requests on fast paths,
> which are RX/TX softirqs. Also, it needs a temporary/scratch buffer
> for preparing the hash.
>
> Rework tcp_md5sig_pool in order to support other hashing algorithms
> than MD5. It will make it possible to share pre-allocated crypto_ahash
> descriptors and scratch area between all TCP hash users.
>
> Internally tcp_sigpool calls crypto_clone_ahash() API over pre-allocated
> crypto ahash tfm. Kudos to Herbert, who provided this new crypto API.
>
> I was a little concerned over GFP_ATOMIC allocations of ahash and
> crypto_request in RX/TX (see tcp_sigpool_start()), so I benchmarked both
> "backends" with different algorithms, using patched version of iperf3[2].
> On my laptop with i7-7600U @ 2.80GHz:
>

... snip ...

> +/**
> + * tcp_sigpool_alloc_ahash - allocates pool for ahash requests
> + * @alg: name of async hash algorithm
> + * @scratch_size: reserve a tcp_sigpool::scratch buffer of this size
> + */
> +int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size)
> +{
> +       int i, ret;
> +
> +       /* slow-path */
> +       mutex_lock(&cpool_mutex);
> +       ret = sigpool_reserve_scratch(scratch_size);
> +       if (ret)
> +               goto out;
> +       for (i = 0; i < cpool_populated; i++) {
> +               if (!cpool[i].alg)
> +                       continue;
> +               if (strcmp(cpool[i].alg, alg))
> +                       continue;
> +
> +               if (kref_read(&cpool[i].kref) > 0)
> +                       kref_get(&cpool[i].kref);
> +               else
> +                       kref_init(&cpool[i].kref);
> +               ret = i;
> +               goto out;
> +       }

Here it looks to me like you will never get to this part of the code since you
always end up going to the out label in the previous loop.

> +
> +       for (i = 0; i < cpool_populated; i++) {
> +               if (!cpool[i].alg)
> +                       break;
> +       }
> +       if (i >= CPOOL_SIZE) {
> +               ret = -ENOSPC;
> +               goto out;
> +       }
> +
> +       ret = __cpool_alloc_ahash(&cpool[i], alg);
> +       if (!ret) {
> +               ret = i;
> +               if (i == cpool_populated)
> +                       cpool_populated++;
> +       }
> +out:
> +       mutex_unlock(&cpool_mutex);
> +       return ret;
> +}
> +EXPORT_SYMBOL_GPL(tcp_sigpool_alloc_ahash);
> +

... snip ...


>  clear_hash:
> -       tcp_put_md5sig_pool();
> -clear_hash_noput:
> +       tcp_sigpool_end(&hp);
> +clear_hash_nostart:
>         memset(md5_hash, 0, 16);
>         return 1;
>  }
> --
> 2.40.0
>
>

BR
Steen

2023-06-15 16:49:49

by Dmitry Safonov

Subject: Re: [PATCH v7 01/22] net/tcp: Prepare tcp_md5sig_pool for TCP-AO

Hi Steen,

On 6/15/23 11:45, Steen Hegelund wrote:
> Hi Dmitry,
>
> On Thu, 2023-06-15 at 00:09 +0100, Dmitry Safonov wrote:
[..]
>> +/**
>> + * tcp_sigpool_alloc_ahash - allocates pool for ahash requests
>> + * @alg: name of async hash algorithm
>> + * @scratch_size: reserve a tcp_sigpool::scratch buffer of this size
>> + */
>> +int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size)
>> +{
>> +       int i, ret;
>> +
>> +       /* slow-path */
>> +       mutex_lock(&cpool_mutex);
>> +       ret = sigpool_reserve_scratch(scratch_size);
>> +       if (ret)
>> +               goto out;
>> +       for (i = 0; i < cpool_populated; i++) {
>> +               if (!cpool[i].alg)
>> +                       continue;
>> +               if (strcmp(cpool[i].alg, alg))
>> +                       continue;
>> +
>> +               if (kref_read(&cpool[i].kref) > 0)
>> +                       kref_get(&cpool[i].kref);
>> +               else
>> +                       kref_init(&cpool[i].kref);
>> +               ret = i;
>> +               goto out;
>> +       }
>
> Here it looks to me like you will never get to this part of the code since you
> always end up going to the out label in the previous loop.

Well, not exactly: this part is looking if the crypto algorithm is
already in this pool, so that it can increment refcounter rather than
initialize a new tfm. In case strcmp(cpool[i].alg, alg) fails, this loop
will never goto out.

I.e., you issued previously setsockopt()s for TCP-MD5 and TCP-AO with
HMAC-SHA1, so in this pool there'll be two algorithms: "md5" and
"hmac(sha1)". Now if you want to use TCP-AO with "cmac(aes128)" or
"hmac(sha256)", you won't find them in the pool yet.

>
>> +
>> +       for (i = 0; i < cpool_populated; i++) {
>> +               if (!cpool[i].alg)
>> +                       break;
>> +       }
>> +       if (i >= CPOOL_SIZE) {
>> +               ret = -ENOSPC;
>> +               goto out;
>> +       }
>> +
>> +       ret = __cpool_alloc_ahash(&cpool[i], alg);
>> +       if (!ret) {
>> +               ret = i;
>> +               if (i == cpool_populated)
>> +                       cpool_populated++;
>> +       }
>> +out:
>> +       mutex_unlock(&cpool_mutex);
>> +       return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(tcp_sigpool_alloc_ahash);
>> +
>
> ... snip ...
>
>
>>  clear_hash:
>> -       tcp_put_md5sig_pool();
>> -clear_hash_noput:
>> +       tcp_sigpool_end(&hp);
>> +clear_hash_nostart:
>>         memset(md5_hash, 0, 16);
>>         return 1;
>>  }
Thanks,
Dmitry

2023-06-16 07:08:02

by Steen Hegelund

Subject: Re: [PATCH v7 01/22] net/tcp: Prepare tcp_md5sig_pool for TCP-AO

Hi Dmitry,

On Thu, 2023-06-15 at 17:44 +0100, Dmitry Safonov wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the
> content is safe
>
> Hi Steen,
>
> On 6/15/23 11:45, Steen Hegelund wrote:
> > Hi Dmitry,
> >
> > On Thu, 2023-06-15 at 00:09 +0100, Dmitry Safonov wrote:
> [..]
> > > +/**
> > > + * tcp_sigpool_alloc_ahash - allocates pool for ahash requests
> > > + * @alg: name of async hash algorithm
> > > + * @scratch_size: reserve a tcp_sigpool::scratch buffer of this size
> > > + */
> > > +int tcp_sigpool_alloc_ahash(const char *alg, size_t scratch_size)
> > > +{
> > > +       int i, ret;
> > > +
> > > +       /* slow-path */
> > > +       mutex_lock(&cpool_mutex);
> > > +       ret = sigpool_reserve_scratch(scratch_size);
> > > +       if (ret)
> > > +               goto out;
> > > +       for (i = 0; i < cpool_populated; i++) {
> > > +               if (!cpool[i].alg)
> > > +                       continue;
> > > +               if (strcmp(cpool[i].alg, alg))
> > > +                       continue;
> > > +
> > > +               if (kref_read(&cpool[i].kref) > 0)
> > > +                       kref_get(&cpool[i].kref);
> > > +               else
> > > +                       kref_init(&cpool[i].kref);
> > > +               ret = i;
> > > +               goto out;
> > > +       }
> >
> > Here it looks to me like you will never get to this part of the code since
> > you
> > always end up going to the out label in the previous loop.
>
> Well, not exactly: this part is looking if the crypto algorithm is
> already in this pool, so that it can increment refcounter rather than
> initialize a new tfm. In case strcmp(cpool[i].alg, alg) fails, this loop
> will never goto out.

Ah, right, you never find any algo and then get out at the end of the list.

>
> I.e., you issued previously setsockopt()s for TCP-MD5 and TCP-AO with
> HMAC-SHA1, so in this pool there'll be two algorithms: "md5" and
> "hmac(sha1)". Now if you want to use TCP-AO with "cmac(aes128)" or
> "hmac(sha256)", you won't find them in the pool yet.
>
> >
> > > +
> > > +       for (i = 0; i < cpool_populated; i++) {
> > > +               if (!cpool[i].alg)
> > > +                       break;
> > > +       }
> > > +       if (i >= CPOOL_SIZE) {
> > > +               ret = -ENOSPC;
> > > +               goto out;
> > > +       }
> > > +
> > > +       ret = __cpool_alloc_ahash(&cpool[i], alg);
> > > +       if (!ret) {
> > > +               ret = i;
> > > +               if (i == cpool_populated)
> > > +                       cpool_populated++;
> > > +       }
> > > +out:
> > > +       mutex_unlock(&cpool_mutex);
> > > +       return ret;
> > > +}
> > > +EXPORT_SYMBOL_GPL(tcp_sigpool_alloc_ahash);
> > > +
> >
> > ... snip ...
> >
> >
> > >  clear_hash:
> > > -       tcp_put_md5sig_pool();
> > > -clear_hash_noput:
> > > +       tcp_sigpool_end(&hp);
> > > +clear_hash_nostart:
> > >         memset(md5_hash, 0, 16);
> > >         return 1;
> > >  }
> Thanks,
>             Dmitry
>

Reviewed-by: Steen Hegelund <[email protected]>

BR
Steen

2023-06-18 18:09:59

by David Ahern

Subject: Re: [PATCH v7 04/22] net/tcp: Prevent TCP-MD5 with TCP-AO being set

On 6/14/23 4:09 PM, Dmitry Safonov wrote:
> Be as conservative as possible: if there is TCP-MD5 key for a given peer
> regardless of L3 interface - don't allow setting TCP-AO key for the same
> peer. According to RFC5925, TCP-AO is supposed to replace TCP-MD5 and
> there can't be any switch between both on any connected tuple.
> Later it can be relaxed, if there's a use, but in the beginning restrict
> any intersection.
>
> Note: it's still should be possible to set both TCP-MD5 and TCP-AO keys
> on a listening socket for *different* peers.

Does the testsuite cover use of both MD5 and AO for a single listening
socket with different peers and then other tests covering attempts to
use both for a same peer?

2023-06-19 16:55:21

by Dmitry Safonov

Subject: Re: [PATCH v7 04/22] net/tcp: Prevent TCP-MD5 with TCP-AO being set

Hi David,

On 6/18/23 18:50, David Ahern wrote:
> On 6/14/23 4:09 PM, Dmitry Safonov wrote:
>> Be as conservative as possible: if there is TCP-MD5 key for a given peer
>> regardless of L3 interface - don't allow setting TCP-AO key for the same
>> peer. According to RFC5925, TCP-AO is supposed to replace TCP-MD5 and
>> there can't be any switch between both on any connected tuple.
>> Later it can be relaxed, if there's a use, but in the beginning restrict
>> any intersection.
>>
>> Note: it's still should be possible to set both TCP-MD5 and TCP-AO keys
>> on a listening socket for *different* peers.
>
> Does the testsuite cover use of both MD5 and AO for a single listening
> socket with different peers and then other tests covering attempts to
> use both for a same peer?

Thanks for the question, I have written the following tests for
AO/MD5/unsigned listening socket [1]:

1. Listener with TCP-AO key, which has addr = INADDR_ANY
2. Listener with TCP-MD5 key, which has tcpm_addr = INADDR_ANY
3. Listener without any key

Then there's AO_REQUIRED thing, which BGP folks asked to introduce,
which is (7.3) from RFC5925, an option that is per-ao_info, which makes
such socket accepting only TCP-AO enabled segments.

So, 4. Listener with TCP-AO, AO_REQUIRED flag.

And then, going to non-INADDR_ANY:
5. Listener with TCP-AO and TCP-MD5 keys for different peers.

Here again, for each of AO/MD5/unsigned methods, attempt to connect:
6. outside of both key peers
7. inside correct key: i.e. TCP-MD5 client to TCP-MD5 matching key
8. to a wrong key: i.e. TCP-AO client to TCP-MD5 matching key

And another type of checks are the ones expecting *setsockopt()* to fail:
9. Adding TCP-AO key that matches the same peer as TCP-MD5 key
10. The reverse situation
11. Adding TCP-MD5 key to AO_REQUIRED socket
12. Setting AO_REQUIRED on a socket with TCP-MD5 key
13. Adding TCP-AO key on already established connection without any key

And then another bunch of tests that check TCP-AO/TCP-MD5/unsigned
interaction in non/default VRFs.
I think the output of selftest [1] is more-or-less self-descriptive,
correct me if I could improve that.

[1]
https://github.com/0x7f454c46/linux/commit/d7b321f2b5a481e5ff0e80e2e0b3503b1ddb9817

Thanks,
Dmitry

2023-06-19 16:56:21

by Dmitry Safonov

Subject: Re: [PATCH v7 04/22] net/tcp: Prevent TCP-MD5 with TCP-AO being set

On 6/19/23 17:31, Dmitry Safonov wrote:
> Hi David,
>
> On 6/18/23 18:50, David Ahern wrote:
>> On 6/14/23 4:09 PM, Dmitry Safonov wrote:
>>> Be as conservative as possible: if there is TCP-MD5 key for a given peer
>>> regardless of L3 interface - don't allow setting TCP-AO key for the same
>>> peer. According to RFC5925, TCP-AO is supposed to replace TCP-MD5 and
>>> there can't be any switch between both on any connected tuple.
>>> Later it can be relaxed, if there's a use, but in the beginning restrict
>>> any intersection.
>>>
>>> Note: it's still should be possible to set both TCP-MD5 and TCP-AO keys
>>> on a listening socket for *different* peers.
>>
>> Does the testsuite cover use of both MD5 and AO for a single listening
>> socket with different peers and then other tests covering attempts to
>> use both for a same peer?
>
> Thanks for the question, I have written the following tests for
> AO/MD5/unsigned listening socket [1]:
>
> 1. Listener with TCP-AO key, which has addr = INADDR_ANY
> 2. Listener with TCP-MD5 key, which has tcpm_addr = INADDR_ANY
> 3. Listener without any key
>
> Then there's AO_REQUIRED thing, which BGP folks asked to introduce,
> which is (7.3) from RFC5925, an option that is per-ao_info, which makes
> such socket accepting only TCP-AO enabled segments.
>
> So, 4. Listener with TCP-AO, AO_REQUIRED flag.
>
> And then, going to non-INADDR_ANY:
> 5. Listener with TCP-AO and TCP-MD5 keys for different peers.
>
> Here again, for each of AO/MD5/unsigned methods, attempt to connect:
> 6. outside of both key peers
> 7. inside correct key: i.e. TCP-MD5 client to TCP-MD5 matching key
> 8. to a wrong key: i.e. TCP-AO client to TCP-MD5 matching key
>
> And another type of checks are the ones expecting *setsockopt()* to fail:
> 9. Adding TCP-AO key that matches the same peer as TCP-MD5 key
> 10. The reverse situation
> 11. Adding TCP-MD5 key to AO_REQUIRED socket
> 12. Setting AO_REQUIRED on a socket with TCP-MD5 key
> 13. Adding TCP-AO key on already established connection without any key

Oh, yeah, forgot to mention, there are another 2 tests for TCP_CLOSE
socket (just a new one), that has both TCP-AO and TCP-MD5 keys and tries
to call connect(). In discussion with the team, it seems really
unexpected situation and better to force userspace to remove either AO
or MD5 key before calling connect(). Those from the output in [1] are:

> ok 39 AO+MD5 server: client with both [TCP-MD5] and TCP-AO keys:
connect() was prevented
> ok 40 AO+MD5 server: client with both TCP-MD5 and [TCP-AO] keys:
connect() was prevented

>
> And then another bunch of tests that check TCP-AO/TCP-MD5/unsigned
> interaction in non/default VRFs.
> I think the output of selftest [1] is more-or-less self-descriptive,
> correct me if I could improve that.
>
> [1]
> https://github.com/0x7f454c46/linux/commit/d7b321f2b5a481e5ff0e80e2e0b3503b1ddb9817

Thanks,
Dmitry

2023-06-19 17:07:47

by Dmitry Safonov

Subject: Re: [PATCH v7 04/22] net/tcp: Prevent TCP-MD5 with TCP-AO being set

On 6/19/23 17:41, Dmitry Safonov wrote:
> On 6/19/23 17:31, Dmitry Safonov wrote:
>> Hi David,
>>
>> On 6/18/23 18:50, David Ahern wrote:
>>> On 6/14/23 4:09 PM, Dmitry Safonov wrote:
>>>> Be as conservative as possible: if there is TCP-MD5 key for a given peer
>>>> regardless of L3 interface - don't allow setting TCP-AO key for the same
>>>> peer. According to RFC5925, TCP-AO is supposed to replace TCP-MD5 and
>>>> there can't be any switch between both on any connected tuple.
>>>> Later it can be relaxed, if there's a use, but in the beginning restrict
>>>> any intersection.
>>>>
>>>> Note: it's still should be possible to set both TCP-MD5 and TCP-AO keys
>>>> on a listening socket for *different* peers.
>>>
>>> Does the testsuite cover use of both MD5 and AO for a single listening
>>> socket with different peers and then other tests covering attempts to
>>> use both for a same peer?
>>
>> Thanks for the question, I have written the following tests for
>> AO/MD5/unsigned listening socket [1]:
>>
>> 1. Listener with TCP-AO key, which has addr = INADDR_ANY
>> 2. Listener with TCP-MD5 key, which has tcpm_addr = INADDR_ANY
>> 3. Listener without any key
>>
>> Then there's AO_REQUIRED thing, which BGP folks asked to introduce,
>> which is (7.3) from RFC5925, an option that is per-ao_info, which makes
>> such socket accepting only TCP-AO enabled segments.
>>
>> So, 4. Listener with TCP-AO, AO_REQUIRED flag.
>>
>> And then, going to non-INADDR_ANY:
>> 5. Listener with TCP-AO and TCP-MD5 keys for different peers.
>>
>> Here again, for each of AO/MD5/unsigned methods, attempt to connect:
>> 6. outside of both key peers
>> 7. inside correct key: i.e. TCP-MD5 client to TCP-MD5 matching key
>> 8. to a wrong key: i.e. TCP-AO client to TCP-MD5 matching key
>>
>> And another type of checks are the ones expecting *setsockopt()* to fail:
>> 9. Adding TCP-AO key that matches the same peer as TCP-MD5 key
>> 10. The reverse situation
>> 11. Adding TCP-MD5 key to AO_REQUIRED socket
>> 12. Setting AO_REQUIRED on a socket with TCP-MD5 key
>> 13. Adding TCP-AO key on already established connection without any key
>
> Oh, yeah, forgot to mention, there are another 2 tests for TCP_CLOSE
> socket (just a new one), that has both TCP-AO and TCP-MD5 keys and tries
> to call connect(). In discussion with the team, it seems really
> unexpected situation and better to force userspace to remove either AO
> or MD5 key before calling connect(). Those from the output in [1] are:
>
>> ok 39 AO+MD5 server: client with both [TCP-MD5] and TCP-AO keys:
> connect() was prevented
>> ok 40 AO+MD5 server: client with both TCP-MD5 and [TCP-AO] keys:
> connect() was prevented

And while starring at the selftest results, I noticed in the output
sample a copy-n-paste typo for VRFs, this:
> ok 60 VRF: TCP-AO key (l3index=0) + TCP-MD5 key (no l3index)
> ok 61 VRF: TCP-MD5 key (no l3index) + TCP-AO key (l3index=0)

Should be read as
> ok 60 VRF: TCP-AO key (l3index=0) + TCP-MD5 key (l3index=N)
> ok 61 VRF: TCP-MD5 key (l3index=N) + TCP-AO key (l3index=0)

(those checks are corresponding to the table in VRF-support commit [2])


>> And then another bunch of tests that check TCP-AO/TCP-MD5/unsigned
>> interaction in non/default VRFs.
>> I think the output of selftest [1] is more-or-less self-descriptive,
>> correct me if I could improve that.
>>
>> [1]
>> https://github.com/0x7f454c46/linux/commit/d7b321f2b5a481e5ff0e80e2e0b3503b1ddb9817

[2]
https://lore.kernel.org/all/[email protected]/T/#u

Thanks,
Dmitry