Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <3001688cf29f03df88bee59e41e6f041fe59357f.camel@huaweicloud.com>
Subject: Re: NFS mount fail
From:   Roberto Sassu <roberto.sassu@huaweicloud.com>
To:     Casey Schaufler <casey@schaufler-ca.com>
Cc:     linux-security-module@vger.kernel.org, linux-nfs@vger.kernel.org,
        chuck.lever@oracle.com, jlayton@kernel.org
Date:   Mon, 08 May 2023 11:16:07 +0200
In-Reply-To: <8361cbdc-5eed-b7c5-2ba8-d6be12c8da88@schaufler-ca.com>
References: <1923bc2f330f576cd246856f976af448c035d02e.camel@huaweicloud.com>
         <d34c30ba-55cc-8662-3587-bb66e234b714@schaufler-ca.com>
         <6b5f941a4dd57f357f942df4051ccc9995b4ba15.camel@huaweicloud.com>
         <8361cbdc-5eed-b7c5-2ba8-d6be12c8da88@schaufler-ca.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.36.5-0ubuntu1 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Fri, 2023-05-05 at 10:06 -0700, Casey Schaufler wrote:
> On 5/4/2023 11:53 PM, Roberto Sassu wrote:
> > On Thu, 2023-05-04 at 17:59 -0700, Casey Schaufler wrote:
> > > On 5/4/2023 9:11 AM, Roberto Sassu wrote:
> > > > Hi Casey
> > > > 
> > > > while developing the fix for overlayfs, I tried first to address the
> > > > issue of a NFS filesystem failing to mount.
> > > > 
> > > > The NFS server does not like the packets sent by the client:
> > > > 
> > > > 14:52:20.827208 IP (tos 0x0, ttl 64, id 60628, offset 0, flags [DF], proto TCP (6), length 72, options (unknown 134,EOL))
> > > >     localhost.localdomain.omginitialrefs > _gateway.nfs: Flags [S], cksum 0x7618 (incorrect -> 0xa18c), seq 455337903, win 64240, options [mss 1460,sackOK,TS val 2178524519 ecr 0,nop,wscale 7], length 0
> > > > 14:52:20.827376 IP (tos 0xc0, ttl 64, id 5906, offset 0, flags [none], proto ICMP (1), length 112, options (unknown 134,EOL))
> > > >     _gateway > localhost.localdomain: ICMP parameter problem - octet 22, length 80
> > > > 
> > > > I looked at the possible causes. SELinux works properly.
> > > SELinux was the reference LSM implementation for labeled networking.
> > > 
> > > > What it seems to happen is that there is a default netlabel mapping,
> > > > that is used to send the packets out.
> > > Correct. SELinux only uses CIPSO options for MLS. Smack uses CIPSO for
> > > almost all packets.
> > > 
> > > > We are in this part of the code:
> > > > 
> > > > Thread 1 hit Breakpoint 2, netlbl_sock_setattr (sk=sk@entry=0xffff888025178000, family=family@entry=2, secattr=0xffff88802504b200) at net/netlabel/netlabel_kapi.c:980
> > > > 980	{
> > > > (gdb) n
> > > > 771		__rcu_read_lock();
> > > > (gdb) 
> > > > 985		dom_entry = netlbl_domhsh_getentry(secattr->domain, family);
> > > > (gdb) 
> > > > 986		if (dom_entry == NULL) {
> > > > (gdb) 
> > > > 990		switch (family) {
> > > > (gdb) 
> > > > 992			switch (dom_entry->def.type) {
> > > > 
> > > > Here is the difference between Smack and SELinux.
> > > > 
> > > > Smack:
> > > > 
> > > > (gdb) p *dom_entry
> > > > $2 = {domain = 0x0 <fixed_percpu_data>, family = 2, def = {type = 3, {addrsel = 0xffff888006bbef40, cipso = 0xffff888006bbef40, calipso = 0xffff888006bbef40}}, valid = 1, list = {next = 0xffff88800767f6e8, prev = 0xffff88800767f6e8}, rcu = {next = 0x0 <fixed_percpu_data>, 
> > > >     func = 0x0 <fixed_percpu_data>}}
> > > > 
> > > > SELinux:
> > > > 
> > > > (gdb) p *dom_entry
> > > > $5 = {domain = 0x0 <fixed_percpu_data>, family = 2, def = {type = 5, {addrsel = 0x0 <fixed_percpu_data>, cipso = 0x0 <fixed_percpu_data>, calipso = 0x0 <fixed_percpu_data>}}, valid = 1, list = {next = 0xffff888006012c88, prev = 0xffff888006012c88}, rcu = {
> > > >     next = 0x0 <fixed_percpu_data>, func = 0x0 <fixed_percpu_data>}}
> > > > 
> > > > 
> > > > type = 3 (for Smack) is NETLBL_NLTYPE_CIPSOV4.
> > > > type = 5 (for SELinux) is NETLBL_NLTYPE_UNLABELED.
> > > > 
> > > > This is why SELinux works (no incompatible options are sent).
> > > SELinux "works" because that's the use case that was verified.
> > > 
> > > > The netlabel mapping is added here:
> > > > 
> > > > static void smk_cipso_doi(void)
> > > > {
> > > > 
> > > > [...]
> > > > 
> > > > 	rc = netlbl_cfg_cipsov4_map_add(doip->doi, NULL, NULL, NULL, &nai);
> > > > 
> > > > 
> > > > Not sure exactly how we can solve this issue. Just checked that
> > > > commenting the call to smk_cipso_doi() in init_smk_fs() allows the NFS
> > > > filesystem to be mounted.
> > > Are both the server and client using Smack? Are they on a network that can
> > > propagate labeled packets? What are you using for a Smack rule configuration?
> > Only the client (Fedora 38).
> 
> Does the client run processes with Smack labels other than floor ("_")?

No, everything with _.

> Are you using any of the Smack mount options?

Just: defaults.

> What value is in /sys/fs/smackfs/ambient?

_


And actually, this seems to be the problem.

In smack_socket_post_create(), kernel threads are given the @ label,
which causes the packets to be processed according to the default
mapping (labeled).

If I do:

echo @ > /sys/fs/smackfs/ambient

NFS can be mounted again.

>  The server is Ubuntu 20.04.06 LTS and uses
> > Apparmor.
> 
> Because the AppArmor server doesn't speak CIPSO you will need to identify
> it as an unlabeled host. This effectively labels all communication with
> the host as having a specific label. See Documentation/admin-guide/LSM/Smack.rst
> for details.
> 
> >  The client is a VM created with libvirt. The connection is
> > the classic tap attached to a bridge.
> 
> OK, does TAP on a bridge support IPv4 options on packets?

Sorry, don't know.

Thanks

Roberto

> > Thanks
> > 
> > Roberto
> >