2022-02-05 19:00:09

by Trond Myklebust

[permalink] [raw]
Subject: Re: v4 clientid uniquifiers in containers/namespaces

On Sat, 2022-02-05 at 10:03 -0500, Benjamin Coddington wrote:
> Hi all,
>
> Is anyone using a udev(-like) implementation with
> NETLINK_LISTEN_ALL_NSID?
> It looks like that is at least necessary to allow the init namespaced
> udev
> to receive notifications on /sys/fs/nfs/net/nfs_client/identifier,
> which
> would be a pre-req to automatically uniquify in containers.
>
> I'md interested since it will inform whether I need to send patches
> to
> systemd's udev, and potentially open the can of worms over there. 
> Yet its
> not yet clear to me how an init namespaced udev process can write to
> a netns
> sysfs path.
>
> Another option might be to create yet another daemon/tool that would
> listen
> specifically for these notifications.  Ugh.
>
> Ben
>

I don't understand. Why do you need a new daemon/tool?

I have the following entry in /etc/udev/rules.d:

[trondmy@leira ~]$ cat /etc/udev/rules.d/50-nfs4.rules
ACTION=="add" KERNEL=="nfs_client" ATTR{identifier}=="(null)" PROGRAM="/usr/sbin/nfs4_uuid" ATTR{identifier}="%c"


...and a very simple script /usr/sbin/nfs4_uuid that reads as follows:

#!/bin/bash
#
if [ ! -f /etc/nfs4_uuid ]
then
uuid="$(uuidgen -r)"
echo -n ${uuid} > /etc/nfs4_uuid
else
uuid="$(cat /etc/nfs4_uuid)"
fi
echo ${uuid}


--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
[email protected]



2022-02-07 12:00:12

by Benjamin Coddington

[permalink] [raw]
Subject: Re: v4 clientid uniquifiers in containers/namespaces

On 5 Feb 2022, at 13:24, Trond Myklebust wrote:

> On Sat, 2022-02-05 at 10:03 -0500, Benjamin Coddington wrote:
>> Hi all,
>>
>> Is anyone using a udev(-like) implementation with
>> NETLINK_LISTEN_ALL_NSID?
>> It looks like that is at least necessary to allow the init namespaced
>> udev
>> to receive notifications on /sys/fs/nfs/net/nfs_client/identifier,
>> which
>> would be a pre-req to automatically uniquify in containers.
>>
>> I'md interested since it will inform whether I need to send patches
>> to
>> systemd's udev, and potentially open the can of worms over there. 
>> Yet its
>> not yet clear to me how an init namespaced udev process can write to
>> a netns
>> sysfs path.
>>
>> Another option might be to create yet another daemon/tool that would
>> listen
>> specifically for these notifications.  Ugh.
>>
>> Ben
>>
>
> I don't understand. Why do you need a new daemon/tool?
>
> I have the following entry in /etc/udev/rules.d:
>
> [trondmy@leira ~]$ cat /etc/udev/rules.d/50-nfs4.rules
> ACTION=="add" KERNEL=="nfs_client" ATTR{identifier}=="(null)"
> PROGRAM="/usr/sbin/nfs4_uuid" ATTR{identifier}="%c"
>
>
> ...and a very simple script /usr/sbin/nfs4_uuid that reads as follows:
>
> #!/bin/bash
> #
> if [ ! -f /etc/nfs4_uuid ]
> then
> uuid="$(uuidgen -r)"
> echo -n ${uuid} > /etc/nfs4_uuid
> else
> uuid="$(cat /etc/nfs4_uuid)"
> fi
> echo ${uuid}


We're in the same place, but what I see is that when I create a new
network
namespace with:

ip netns add testnamespace

Everything in the kernel works up to the point where the userspace udevd
never gets a notification. I suspect thats because it hasn't used
NETLINK_LISTEN_ALL_NSID, so the kernel's skipping the notification in
do_one_broadcast().

If your udev is getting notified of new network namespaces and firing
that
rule each time, something's different between our setups, and I'd like
to
figure out what it might be.

Ben


2022-02-09 06:37:39

by Benjamin Coddington

[permalink] [raw]
Subject: Re: v4 clientid uniquifiers in containers/namespaces

On 5 Feb 2022, at 14:50, Benjamin Coddington wrote:

> On 5 Feb 2022, at 13:24, Trond Myklebust wrote:
>
>> On Sat, 2022-02-05 at 10:03 -0500, Benjamin Coddington wrote:
>>> Hi all,
>>>
>>> Is anyone using a udev(-like) implementation with
>>> NETLINK_LISTEN_ALL_NSID?
>>> It looks like that is at least necessary to allow the init namespaced
>>> udev
>>> to receive notifications on /sys/fs/nfs/net/nfs_client/identifier,
>>> which
>>> would be a pre-req to automatically uniquify in containers.
>>>
>>> I'md interested since it will inform whether I need to send patches
>>> to
>>> systemd's udev, and potentially open the can of worms over there. 
>>> Yet its
>>> not yet clear to me how an init namespaced udev process can write to
>>> a netns
>>> sysfs path.
>>>
>>> Another option might be to create yet another daemon/tool that would
>>> listen
>>> specifically for these notifications.  Ugh.
>>>
>>> Ben
>>>
>>
>> I don't understand. Why do you need a new daemon/tool?

Because what we've got only works for the init namespace.

Udev won't get kobject notifications because its not using
NETLINK_LISTEN_ALL_NSIDs.

We need to figure out if we want:

1) the init namespace udevd to handle all client_id uniquifiers
2) we expect network namespaces to run their own udevd
3) or both.

I think 2 violates "least surprise", and 3 might not be something anyone
ever wants. If they do, we can fix it at that point.

So to make 1 work, we can try to change udevd, or maybe just hacking about
with nfs_netns_object_child_ns_type will be sufficient.

Ben