On long-running enterprise production servers, high-order contiguous
memory pages are usually very rare and in most cases we can only get
fragmented pages.
When replacing TCP with SMC-R in such production scenarios, attempting
to allocate high-order physically contiguous sndbufs and RMBs may result
in frequent memory compaction, which will cause unexpected hung issue
and further stability risks.
So this patch set is aimed to allow SMC-R link group to use virtually
contiguous sndbufs and RMBs to avoid potential issues mentioned above.
Whether to use physically or virtually contiguous buffers can be set
by sysctl smcr_buf_type.
Note that using virtually contiguous buffers will bring an acceptable
performance regression, which can be mainly divided into two parts:
1) regression in data path, which is brought by additional address
translation of sndbuf by RNIC in Tx. But in general, translating
address through MTT is fast. According to qperf test, this part
regression is basically less than 10% in latency and bandwidth.
(see patch 5/6 for details)
2) regression in buffer initialization and destruction path, which is
brought by additional MR operations of sndbufs. But thanks to link
group buffer reuse mechanism, the impact of this kind of regression
decreases as times of buffer reuse increases.
Patch set overview:
- Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
operation, which will reduce overhead on the data path, especially
when using virtually contiguous buffers;
- Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
of buffers in new created link group;
- Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
including buffer creation, destruction, MR operation and access;
- patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
v1->v2:
- Patch 5/6 fixes build issue on 32bit;
- Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
Guangguan Wang (2):
net/smc: remove redundant dma sync ops
net/smc: optimize for smc_sndbuf_sync_sg_for_device and
smc_rmb_sync_sg_for_cpu
Wen Gu (4):
net/smc: Introduce a sysctl for setting SMC-R buffer type
net/smc: Use sysctl-specified types of buffers in new link group
net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
net/smc: Extend SMC-R link group netlink attribute
Documentation/networking/smc-sysctl.rst | 13 ++
include/net/netns/smc.h | 1 +
include/uapi/linux/smc.h | 1 +
net/smc/af_smc.c | 68 +++++++--
net/smc/smc_clc.c | 8 +-
net/smc/smc_clc.h | 2 +-
net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
net/smc/smc_core.h | 20 ++-
net/smc/smc_ib.c | 44 +++++-
net/smc/smc_ib.h | 2 +
net/smc/smc_llc.c | 33 +++--
net/smc/smc_rx.c | 92 +++++++++---
net/smc/smc_sysctl.c | 11 ++
net/smc/smc_tx.c | 10 +-
14 files changed, 404 insertions(+), 147 deletions(-)
--
1.8.3.1
From: Guangguan Wang <[email protected]>
smc_ib_sync_sg_for_cpu/device are the ops used for dma memory cache
consistency. Smc sndbufs are dma buffers, where CPU writes data to
it and PCIE device reads data from it. So for sndbufs,
smc_ib_sync_sg_for_device is needed and smc_ib_sync_sg_for_cpu is
redundant as PCIE device will not write the buffers. Smc rmbs
are dma buffers, where PCIE device write data to it and CPU read
data from it. So for rmbs, smc_ib_sync_sg_for_cpu is needed and
smc_ib_sync_sg_for_device is redundant as CPU will not write the buffers.
Signed-off-by: Guangguan Wang <[email protected]>
---
net/smc/af_smc.c | 2 --
net/smc/smc_core.c | 22 ----------------------
net/smc/smc_core.h | 2 --
net/smc/smc_rx.c | 2 --
net/smc/smc_tx.c | 1 -
5 files changed, 29 deletions(-)
diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c
index 433bb5a7..9497a3b 100644
--- a/net/smc/af_smc.c
+++ b/net/smc/af_smc.c
@@ -1226,7 +1226,6 @@ static int smc_connect_rdma(struct smc_sock *smc,
goto connect_abort;
}
}
- smc_rmb_sync_sg_for_device(&smc->conn);
if (aclc->hdr.version > SMC_V1) {
struct smc_clc_msg_accept_confirm_v2 *clc_v2 =
@@ -2113,7 +2112,6 @@ static int smc_listen_rdma_reg(struct smc_sock *new_smc, bool local_first)
if (smcr_lgr_reg_rmbs(conn->lnk, conn->rmb_desc))
return SMC_CLC_DECL_ERR_REGRMB;
}
- smc_rmb_sync_sg_for_device(&new_smc->conn);
return 0;
}
diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index f40f6ed..1faa0cb 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -2290,14 +2290,6 @@ static int __smc_buf_create(struct smc_sock *smc, bool is_smcd, bool is_rmb)
return 0;
}
-void smc_sndbuf_sync_sg_for_cpu(struct smc_connection *conn)
-{
- if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd ||
- !smc_link_active(conn->lnk))
- return;
- smc_ib_sync_sg_for_cpu(conn->lnk, conn->sndbuf_desc, DMA_TO_DEVICE);
-}
-
void smc_sndbuf_sync_sg_for_device(struct smc_connection *conn)
{
if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd ||
@@ -2320,20 +2312,6 @@ void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn)
}
}
-void smc_rmb_sync_sg_for_device(struct smc_connection *conn)
-{
- int i;
-
- if (!smc_conn_lgr_valid(conn) || conn->lgr->is_smcd)
- return;
- for (i = 0; i < SMC_LINKS_PER_LGR_MAX; i++) {
- if (!smc_link_active(&conn->lgr->lnk[i]))
- continue;
- smc_ib_sync_sg_for_device(&conn->lgr->lnk[i], conn->rmb_desc,
- DMA_FROM_DEVICE);
- }
-}
-
/* create the send and receive buffer for an SMC socket;
* receive buffers are called RMBs;
* (even though the SMC protocol allows more than one RMB-element per RMB,
diff --git a/net/smc/smc_core.h b/net/smc/smc_core.h
index 4cb03e9..c441dfe 100644
--- a/net/smc/smc_core.h
+++ b/net/smc/smc_core.h
@@ -513,10 +513,8 @@ void smc_rtoken_set(struct smc_link_group *lgr, int link_idx, int link_idx_new,
__be32 nw_rkey_known, __be64 nw_vaddr, __be32 nw_rkey);
void smc_rtoken_set2(struct smc_link_group *lgr, int rtok_idx, int link_id,
__be64 nw_vaddr, __be32 nw_rkey);
-void smc_sndbuf_sync_sg_for_cpu(struct smc_connection *conn);
void smc_sndbuf_sync_sg_for_device(struct smc_connection *conn);
void smc_rmb_sync_sg_for_cpu(struct smc_connection *conn);
-void smc_rmb_sync_sg_for_device(struct smc_connection *conn);
int smc_vlan_by_tcpsk(struct socket *clcsock, struct smc_init_info *ini);
void smc_conn_free(struct smc_connection *conn);
diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c
index 338b9ef..00ad004 100644
--- a/net/smc/smc_rx.c
+++ b/net/smc/smc_rx.c
@@ -413,7 +413,6 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
if (rc < 0) {
if (!read_done)
read_done = -EFAULT;
- smc_rmb_sync_sg_for_device(conn);
goto out;
}
}
@@ -427,7 +426,6 @@ int smc_rx_recvmsg(struct smc_sock *smc, struct msghdr *msg,
chunk_len_sum += chunk_len;
chunk_off = 0; /* modulo offset in recv ring buffer */
}
- smc_rmb_sync_sg_for_device(conn);
/* update cursors */
if (!(flags & MSG_PEEK)) {
diff --git a/net/smc/smc_tx.c b/net/smc/smc_tx.c
index 805a546..ca0d5f5 100644
--- a/net/smc/smc_tx.c
+++ b/net/smc/smc_tx.c
@@ -246,7 +246,6 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len)
tx_cnt_prep);
chunk_len_sum = chunk_len;
chunk_off = tx_cnt_prep;
- smc_sndbuf_sync_sg_for_cpu(conn);
for (chunk = 0; chunk < 2; chunk++) {
rc = memcpy_from_msg(sndbuf_base + chunk_off,
msg, chunk_len);
--
1.8.3.1
On 14.07.22 11:43, Wen Gu wrote:
> On long-running enterprise production servers, high-order contiguous
> memory pages are usually very rare and in most cases we can only get
> fragmented pages.
>
> When replacing TCP with SMC-R in such production scenarios, attempting
> to allocate high-order physically contiguous sndbufs and RMBs may result
> in frequent memory compaction, which will cause unexpected hung issue
> and further stability risks.
>
> So this patch set is aimed to allow SMC-R link group to use virtually
> contiguous sndbufs and RMBs to avoid potential issues mentioned above.
> Whether to use physically or virtually contiguous buffers can be set
> by sysctl smcr_buf_type.
>
> Note that using virtually contiguous buffers will bring an acceptable
> performance regression, which can be mainly divided into two parts:
>
> 1) regression in data path, which is brought by additional address
> translation of sndbuf by RNIC in Tx. But in general, translating
> address through MTT is fast. According to qperf test, this part
> regression is basically less than 10% in latency and bandwidth.
> (see patch 5/6 for details)
>
> 2) regression in buffer initialization and destruction path, which is
> brought by additional MR operations of sndbufs. But thanks to link
> group buffer reuse mechanism, the impact of this kind of regression
> decreases as times of buffer reuse increases.
>
> Patch set overview:
> - Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
> operation, which will reduce overhead on the data path, especially
> when using virtually contiguous buffers;
> - Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
> of buffers in new created link group;
> - Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
> including buffer creation, destruction, MR operation and access;
> - patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
>
> v1->v2:
> - Patch 5/6 fixes build issue on 32bit;
> - Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
>
> Guangguan Wang (2):
> net/smc: remove redundant dma sync ops
> net/smc: optimize for smc_sndbuf_sync_sg_for_device and
> smc_rmb_sync_sg_for_cpu
>
> Wen Gu (4):
> net/smc: Introduce a sysctl for setting SMC-R buffer type
> net/smc: Use sysctl-specified types of buffers in new link group
> net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
> net/smc: Extend SMC-R link group netlink attribute
>
> Documentation/networking/smc-sysctl.rst | 13 ++
> include/net/netns/smc.h | 1 +
> include/uapi/linux/smc.h | 1 +
> net/smc/af_smc.c | 68 +++++++--
> net/smc/smc_clc.c | 8 +-
> net/smc/smc_clc.h | 2 +-
> net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
> net/smc/smc_core.h | 20 ++-
> net/smc/smc_ib.c | 44 +++++-
> net/smc/smc_ib.h | 2 +
> net/smc/smc_llc.c | 33 +++--
> net/smc/smc_rx.c | 92 +++++++++---
> net/smc/smc_sysctl.c | 11 ++
> net/smc/smc_tx.c | 10 +-
> 14 files changed, 404 insertions(+), 147 deletions(-)
>
This idea is very cool! Thank you for your effort! But we still need to
verify if this solution can run well on our system. I'll come to you soon.
Hello:
This series was applied to netdev/net-next.git (master)
by David S. Miller <[email protected]>:
On Thu, 14 Jul 2022 17:43:59 +0800 you wrote:
> On long-running enterprise production servers, high-order contiguous
> memory pages are usually very rare and in most cases we can only get
> fragmented pages.
>
> When replacing TCP with SMC-R in such production scenarios, attempting
> to allocate high-order physically contiguous sndbufs and RMBs may result
> in frequent memory compaction, which will cause unexpected hung issue
> and further stability risks.
>
> [...]
Here is the summary with links:
- [net-next,v2,1/6] net/smc: remove redundant dma sync ops
https://git.kernel.org/netdev/net-next/c/6d52e2de6415
- [net-next,v2,2/6] net/smc: optimize for smc_sndbuf_sync_sg_for_device and smc_rmb_sync_sg_for_cpu
https://git.kernel.org/netdev/net-next/c/0ef69e788411
- [net-next,v2,3/6] net/smc: Introduce a sysctl for setting SMC-R buffer type
https://git.kernel.org/netdev/net-next/c/4bc5008e4387
- [net-next,v2,4/6] net/smc: Use sysctl-specified types of buffers in new link group
https://git.kernel.org/netdev/net-next/c/b984f370ed51
- [net-next,v2,5/6] net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
https://git.kernel.org/netdev/net-next/c/b8d199451c99
- [net-next,v2,6/6] net/smc: Extend SMC-R link group netlink attribute
https://git.kernel.org/netdev/net-next/c/ddefb2d20553
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
On Thu, Jul 14, 2022 at 05:16:47PM +0200, Wenjia Zhang wrote:
>
>
> On 14.07.22 11:43, Wen Gu wrote:
> > On long-running enterprise production servers, high-order contiguous
> > memory pages are usually very rare and in most cases we can only get
> > fragmented pages.
> >
> > When replacing TCP with SMC-R in such production scenarios, attempting
> > to allocate high-order physically contiguous sndbufs and RMBs may result
> > in frequent memory compaction, which will cause unexpected hung issue
> > and further stability risks.
> >
> > So this patch set is aimed to allow SMC-R link group to use virtually
> > contiguous sndbufs and RMBs to avoid potential issues mentioned above.
> > Whether to use physically or virtually contiguous buffers can be set
> > by sysctl smcr_buf_type.
> >
> > Note that using virtually contiguous buffers will bring an acceptable
> > performance regression, which can be mainly divided into two parts:
> >
> > 1) regression in data path, which is brought by additional address
> > translation of sndbuf by RNIC in Tx. But in general, translating
> > address through MTT is fast. According to qperf test, this part
> > regression is basically less than 10% in latency and bandwidth.
> > (see patch 5/6 for details)
> >
> > 2) regression in buffer initialization and destruction path, which is
> > brought by additional MR operations of sndbufs. But thanks to link
> > group buffer reuse mechanism, the impact of this kind of regression
> > decreases as times of buffer reuse increases.
> >
> > Patch set overview:
> > - Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
> > operation, which will reduce overhead on the data path, especially
> > when using virtually contiguous buffers;
> > - Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
> > of buffers in new created link group;
> > - Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
> > including buffer creation, destruction, MR operation and access;
> > - patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
> >
> > v1->v2:
> > - Patch 5/6 fixes build issue on 32bit;
> > - Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
> >
> > Guangguan Wang (2):
> > net/smc: remove redundant dma sync ops
> > net/smc: optimize for smc_sndbuf_sync_sg_for_device and
> > smc_rmb_sync_sg_for_cpu
> >
> > Wen Gu (4):
> > net/smc: Introduce a sysctl for setting SMC-R buffer type
> > net/smc: Use sysctl-specified types of buffers in new link group
> > net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
> > net/smc: Extend SMC-R link group netlink attribute
> >
> > Documentation/networking/smc-sysctl.rst | 13 ++
> > include/net/netns/smc.h | 1 +
> > include/uapi/linux/smc.h | 1 +
> > net/smc/af_smc.c | 68 +++++++--
> > net/smc/smc_clc.c | 8 +-
> > net/smc/smc_clc.h | 2 +-
> > net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
> > net/smc/smc_core.h | 20 ++-
> > net/smc/smc_ib.c | 44 +++++-
> > net/smc/smc_ib.h | 2 +
> > net/smc/smc_llc.c | 33 +++--
> > net/smc/smc_rx.c | 92 +++++++++---
> > net/smc/smc_sysctl.c | 11 ++
> > net/smc/smc_tx.c | 10 +-
> > 14 files changed, 404 insertions(+), 147 deletions(-)
> >
> This idea is very cool! Thank you for your effort! But we still need to
> verify if this solution can run well on our system. I'll come to you soon.
Hi Wenjia,
We have noticed that SMC community is becoming more active recently.
More and more companies have shown their interests in SMC.
Correspondingly, patches are also increasing. We (Alibaba) are trying to
apply SMC into cloud production environment, extending its abilities and
enhancing the performance. We also contributed some work to community in
the past period of time. So we are more than happy to help review SMC
patches together. If you need, we are very glad to be reviewers to share
the review work.
Hope to hear from you, thank you.
Best wishes,
Tony Lu
On 18.07.22 14:45, Tony Lu wrote:
> On Thu, Jul 14, 2022 at 05:16:47PM +0200, Wenjia Zhang wrote:
>>
>>
>> On 14.07.22 11:43, Wen Gu wrote:
>>> On long-running enterprise production servers, high-order contiguous
>>> memory pages are usually very rare and in most cases we can only get
>>> fragmented pages.
>>>
>>> When replacing TCP with SMC-R in such production scenarios, attempting
>>> to allocate high-order physically contiguous sndbufs and RMBs may result
>>> in frequent memory compaction, which will cause unexpected hung issue
>>> and further stability risks.
>>>
>>> So this patch set is aimed to allow SMC-R link group to use virtually
>>> contiguous sndbufs and RMBs to avoid potential issues mentioned above.
>>> Whether to use physically or virtually contiguous buffers can be set
>>> by sysctl smcr_buf_type.
>>>
>>> Note that using virtually contiguous buffers will bring an acceptable
>>> performance regression, which can be mainly divided into two parts:
>>>
>>> 1) regression in data path, which is brought by additional address
>>> translation of sndbuf by RNIC in Tx. But in general, translating
>>> address through MTT is fast. According to qperf test, this part
>>> regression is basically less than 10% in latency and bandwidth.
>>> (see patch 5/6 for details)
>>>
>>> 2) regression in buffer initialization and destruction path, which is
>>> brought by additional MR operations of sndbufs. But thanks to link
>>> group buffer reuse mechanism, the impact of this kind of regression
>>> decreases as times of buffer reuse increases.
>>>
>>> Patch set overview:
>>> - Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
>>> operation, which will reduce overhead on the data path, especially
>>> when using virtually contiguous buffers;
>>> - Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
>>> of buffers in new created link group;
>>> - Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
>>> including buffer creation, destruction, MR operation and access;
>>> - patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
>>>
>>> v1->v2:
>>> - Patch 5/6 fixes build issue on 32bit;
>>> - Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
>>>
>>> Guangguan Wang (2):
>>> net/smc: remove redundant dma sync ops
>>> net/smc: optimize for smc_sndbuf_sync_sg_for_device and
>>> smc_rmb_sync_sg_for_cpu
>>>
>>> Wen Gu (4):
>>> net/smc: Introduce a sysctl for setting SMC-R buffer type
>>> net/smc: Use sysctl-specified types of buffers in new link group
>>> net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
>>> net/smc: Extend SMC-R link group netlink attribute
>>>
>>> Documentation/networking/smc-sysctl.rst | 13 ++
>>> include/net/netns/smc.h | 1 +
>>> include/uapi/linux/smc.h | 1 +
>>> net/smc/af_smc.c | 68 +++++++--
>>> net/smc/smc_clc.c | 8 +-
>>> net/smc/smc_clc.h | 2 +-
>>> net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
>>> net/smc/smc_core.h | 20 ++-
>>> net/smc/smc_ib.c | 44 +++++-
>>> net/smc/smc_ib.h | 2 +
>>> net/smc/smc_llc.c | 33 +++--
>>> net/smc/smc_rx.c | 92 +++++++++---
>>> net/smc/smc_sysctl.c | 11 ++
>>> net/smc/smc_tx.c | 10 +-
>>> 14 files changed, 404 insertions(+), 147 deletions(-)
>>>
>> This idea is very cool! Thank you for your effort! But we still need to
>> verify if this solution can run well on our system. I'll come to you soon.
>
> Hi Wenjia,
>
> We have noticed that SMC community is becoming more active recently.
> More and more companies have shown their interests in SMC.
> Correspondingly, patches are also increasing. We (Alibaba) are trying to
> apply SMC into cloud production environment, extending its abilities and
> enhancing the performance. We also contributed some work to community in
> the past period of time. So we are more than happy to help review SMC
> patches together. If you need, we are very glad to be reviewers to share
> the review work.
>
> Hope to hear from you, thank you.
>
> Best wishes,
> Tony Lu
Hi Tony,
That is very nice to hear that from you. It would be great for us. If
you like, feel free to add your sign after the review.
Thank you!
Best regards
Wenjia Zhang
On 14.07.22 11:43, Wen Gu wrote:
> On long-running enterprise production servers, high-order contiguous
> memory pages are usually very rare and in most cases we can only get
> fragmented pages.
>
> When replacing TCP with SMC-R in such production scenarios, attempting
> to allocate high-order physically contiguous sndbufs and RMBs may result
> in frequent memory compaction, which will cause unexpected hung issue
> and further stability risks.
>
> So this patch set is aimed to allow SMC-R link group to use virtually
> contiguous sndbufs and RMBs to avoid potential issues mentioned above.
> Whether to use physically or virtually contiguous buffers can be set
> by sysctl smcr_buf_type.
>
> Note that using virtually contiguous buffers will bring an acceptable
> performance regression, which can be mainly divided into two parts:
>
> 1) regression in data path, which is brought by additional address
> translation of sndbuf by RNIC in Tx. But in general, translating
> address through MTT is fast. According to qperf test, this part
> regression is basically less than 10% in latency and bandwidth.
> (see patch 5/6 for details)
>
> 2) regression in buffer initialization and destruction path, which is
> brought by additional MR operations of sndbufs. But thanks to link
> group buffer reuse mechanism, the impact of this kind of regression
> decreases as times of buffer reuse increases.
>
> Patch set overview:
> - Patch 1/6 and 2/6 mainly about simplifying and optimizing DMA sync
> operation, which will reduce overhead on the data path, especially
> when using virtually contiguous buffers;
> - Patch 3/6 and 4/6 introduce a sysctl smcr_buf_type to set the type
> of buffers in new created link group;
> - Patch 5/6 allows SMC-R to use virtually contiguous sndbufs and RMBs,
> including buffer creation, destruction, MR operation and access;
> - patch 6/6 extends netlink attribute for buffer type of SMC-R link group;
>
> v1->v2:
> - Patch 5/6 fixes build issue on 32bit;
> - Patch 3/6 adds description of new sysctl in smc-sysctl.rst;
>
> Guangguan Wang (2):
> net/smc: remove redundant dma sync ops
> net/smc: optimize for smc_sndbuf_sync_sg_for_device and
> smc_rmb_sync_sg_for_cpu
>
> Wen Gu (4):
> net/smc: Introduce a sysctl for setting SMC-R buffer type
> net/smc: Use sysctl-specified types of buffers in new link group
> net/smc: Allow virtually contiguous sndbufs or RMBs for SMC-R
> net/smc: Extend SMC-R link group netlink attribute
>
> Documentation/networking/smc-sysctl.rst | 13 ++
> include/net/netns/smc.h | 1 +
> include/uapi/linux/smc.h | 1 +
> net/smc/af_smc.c | 68 +++++++--
> net/smc/smc_clc.c | 8 +-
> net/smc/smc_clc.h | 2 +-
> net/smc/smc_core.c | 246 +++++++++++++++++++++-----------
> net/smc/smc_core.h | 20 ++-
> net/smc/smc_ib.c | 44 +++++-
> net/smc/smc_ib.h | 2 +
> net/smc/smc_llc.c | 33 +++--
> net/smc/smc_rx.c | 92 +++++++++---
> net/smc/smc_sysctl.c | 11 ++
> net/smc/smc_tx.c | 10 +-
> 14 files changed, 404 insertions(+), 147 deletions(-)
>
It looks good for us. Thank you!
Acked-by: Wenjia Zhang <[email protected]>