2023-11-18 06:02:46

by Cedric Blancher

[permalink] [raw]
Subject: TCP_KEEPALIVE for Linux NFS client?

Good morning!

Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
connections? What are the pro and cons of using that for NFS TCP
connections?

Ced
--
Cedric Blancher <[email protected]>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur


2023-11-20 02:57:32

by Olga Kornievskaia

[permalink] [raw]
Subject: Re: TCP_KEEPALIVE for Linux NFS client?

Hi Ced,

Why do you think it doesn't use it? Have you looked at a network trace
of an idle connection? I seem to recall seeing keep-alive being used.

On Fri, Nov 17, 2023 at 8:02 PM Cedric Blancher
<[email protected]> wrote:
>
> Good morning!
>
> Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
> connections? What are the pro and cons of using that for NFS TCP
> connections?
>
> Ced
> --
> Cedric Blancher <[email protected]>
> [https://plus.google.com/u/0/+CedricBlancher/]
> Institute Pasteur

2023-11-22 22:49:30

by Cedric Blancher

[permalink] [raw]
Subject: Re: TCP_KEEPALIVE for Linux NFS client?

On Mon, 20 Nov 2023 at 03:57, Olga Kornievskaia <[email protected]> wrote:
>
> Hi Ced,
>
> Why do you think it doesn't use it? Have you looked at a network trace
> of an idle connection? I seem to recall seeing keep-alive being used.

Well, I don't see a setsockopt(TCP_KEEPALIVE) in the libtirpc code?

Ced

>
> On Fri, Nov 17, 2023 at 8:02 PM Cedric Blancher
> <[email protected]> wrote:
> >
> > Good morning!
> >
> > Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
> > connections? What are the pro and cons of using that for NFS TCP
> > connections?
> >
> > Ced
> > --
> > Cedric Blancher <[email protected]>
> > [https://plus.google.com/u/0/+CedricBlancher/]
> > Institute Pasteur



--
Cedric Blancher <[email protected]>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

2023-11-22 23:04:05

by Calum Mackay

[permalink] [raw]
Subject: Re: TCP_KEEPALIVE for Linux NFS client?

hi Ced,

On 22/11/2023 10:48 pm, Cedric Blancher wrote:
> On Mon, 20 Nov 2023 at 03:57, Olga Kornievskaia <[email protected]> wrote:
>>
>> Hi Ced,
>>
>> Why do you think it doesn't use it? Have you looked at a network trace
>> of an idle connection? I seem to recall seeing keep-alive being used.
>
> Well, I don't see a setsockopt(TCP_KEEPALIVE) in the libtirpc code?

We have this, in the kernel RPC code:

https://elixir.bootlin.com/linux/latest/source/net/sunrpc/xprtsock.c#L2257

which might be it.

cheers,
calum.

>
> Ced
>
>>
>> On Fri, Nov 17, 2023 at 8:02 PM Cedric Blancher
>> <[email protected]> wrote:
>>>
>>> Good morning!
>>>
>>> Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
>>> connections? What are the pro and cons of using that for NFS TCP
>>> connections?
>>>
>>> Ced
>>> --
>>> Cedric Blancher <[email protected]>
>>> [https://plus.google.com/u/0/+CedricBlancher/]
>>> Institute Pasteur
>
>
>

--
Calum Mackay
Linux Kernel Engineering
Oracle Linux and Virtualisation


Attachments:
OpenPGP_signature.asc (855.00 B)
OpenPGP digital signature

2023-11-22 23:11:05

by Cedric Blancher

[permalink] [raw]
Subject: Re: TCP_KEEPALIVE for Linux NFS client?

On Thu, 23 Nov 2023 at 00:03, Calum Mackay <[email protected]> wrote:
>
> hi Ced,
>
> On 22/11/2023 10:48 pm, Cedric Blancher wrote:
> > On Mon, 20 Nov 2023 at 03:57, Olga Kornievskaia <[email protected]> wrote:
> >>
> >> Hi Ced,
> >>
> >> Why do you think it doesn't use it? Have you looked at a network trace
> >> of an idle connection? I seem to recall seeing keep-alive being used.
> >
> > Well, I don't see a setsockopt(TCP_KEEPALIVE) in the libtirpc code?
>
> We have this, in the kernel RPC code:
>
> https://elixir.bootlin.com/linux/latest/source/net/sunrpc/xprtsock.c#L2257
>
> which might be it.

Yay, But how does the kernel get the fd from libtirpc?

Also, how can I turn this:

/* TCP Keepalive options */
sock_set_keepalive(sock->sk);
tcp_sock_set_keepidle(sock->sk, keepidle);
tcp_sock_set_keepintvl(sock->sk, keepidle);
tcp_sock_set_keepcnt(sock->sk, keepcnt);

/* TCP user timeout (see RFC5482) */
tcp_sock_set_user_timeout(sock->sk, timeo);

into setsockopt() from userland code? Does the BSD socket library have
such options?

Ced

>
> cheers,
> calum.
>
> >
> > Ced
> >
> >>
> >> On Fri, Nov 17, 2023 at 8:02 PM Cedric Blancher
> >> <[email protected]> wrote:
> >>>
> >>> Good morning!
> >>>
> >>> Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
> >>> connections? What are the pro and cons of using that for NFS TCP
> >>> connections?
> >>>
> >>> Ced
> >>> --
> >>> Cedric Blancher <[email protected]>
> >>> [https://plus.google.com/u/0/+CedricBlancher/]
> >>> Institute Pasteur
> >
> >
> >
>
> --
> Calum Mackay
> Linux Kernel Engineering
> Oracle Linux and Virtualisation
>


--
Cedric Blancher <[email protected]>
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur

2023-11-22 23:23:04

by Calum Mackay

[permalink] [raw]
Subject: Re: TCP_KEEPALIVE for Linux NFS client?

On 22/11/2023 11:10 pm, Cedric Blancher wrote:
> On Thu, 23 Nov 2023 at 00:03, Calum Mackay <[email protected]> wrote:
>>
>> hi Ced,
>>
>> On 22/11/2023 10:48 pm, Cedric Blancher wrote:
>>> On Mon, 20 Nov 2023 at 03:57, Olga Kornievskaia <[email protected]> wrote:
>>>>
>>>> Hi Ced,
>>>>
>>>> Why do you think it doesn't use it? Have you looked at a network trace
>>>> of an idle connection? I seem to recall seeing keep-alive being used.
>>>
>>> Well, I don't see a setsockopt(TCP_KEEPALIVE) in the libtirpc code?
>>
>> We have this, in the kernel RPC code:
>>
>> https://elixir.bootlin.com/linux/latest/source/net/sunrpc/xprtsock.c#L2257
>>
>> which might be it.
>
> Yay, But how does the kernel get the fd from libtirpc?
>
> Also, how can I turn this:
>
> /* TCP Keepalive options */
> sock_set_keepalive(sock->sk);
> tcp_sock_set_keepidle(sock->sk, keepidle);
> tcp_sock_set_keepintvl(sock->sk, keepidle);
> tcp_sock_set_keepcnt(sock->sk, keepcnt);
>
> /* TCP user timeout (see RFC5482) */
> tcp_sock_set_user_timeout(sock->sk, timeo);
>
> into setsockopt() from userland code? Does the BSD socket library have
> such options?

What's the userland aspect here? For NFS, the kernel RPC code owns the
socket, and sets the sock options.

You can see whether TCP keepalive is enabled, with the final column of
netstat output:

getz $ netstat -neopa | grep :2049

tcp 0 0 192.168.254.2:49802 192.168.254.7:2049
TIME_WAIT 0 0 - timewait (37.52/0/0)
tcp 0 0 192.168.254.2:722 192.168.254.7:2049
TIME_WAIT 0 0 - timewait (37.52/0/0)
tcp 0 0 192.168.254.2:979 192.168.254.7:2049
ESTABLISHED 0 35214 - keepalive (8.47/0/0)

cheers,
calum.

>
> Ced
>
>>
>> cheers,
>> calum.
>>
>>>
>>> Ced
>>>
>>>>
>>>> On Fri, Nov 17, 2023 at 8:02 PM Cedric Blancher
>>>> <[email protected]> wrote:
>>>>>
>>>>> Good morning!
>>>>>
>>>>> Why does the Linux NFS client not use TCP_KEEPALIVE for its TCP
>>>>> connections? What are the pro and cons of using that for NFS TCP
>>>>> connections?
>>>>>
>>>>> Ced
>>>>> --
>>>>> Cedric Blancher <[email protected]>
>>>>> [https://plus.google.com/u/0/+CedricBlancher/]
>>>>> Institute Pasteur
>>>
>>>
>>>
>>
>> --
>> Calum Mackay
>> Linux Kernel Engineering
>> Oracle Linux and Virtualisation
>>
>
>

--
Calum Mackay
Linux Kernel Engineering
Oracle Linux and Virtualisation


Attachments:
OpenPGP_signature.asc (855.00 B)
OpenPGP digital signature