Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8361cbdc-5eed-b7c5-2ba8-d6be12c8da88@schaufler-ca.com>
Date:   Fri, 5 May 2023 10:06:24 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.0
Subject: Re: NFS mount fail
Content-Language: en-US
To:     Roberto Sassu <roberto.sassu@huaweicloud.com>
Cc:     linux-security-module@vger.kernel.org, linux-nfs@vger.kernel.org,
        chuck.lever@oracle.com, jlayton@kernel.org,
        Casey Schaufler <casey@schaufler-ca.com>
References: <1923bc2f330f576cd246856f976af448c035d02e.camel@huaweicloud.com>
 <d34c30ba-55cc-8662-3587-bb66e234b714@schaufler-ca.com>
 <6b5f941a4dd57f357f942df4051ccc9995b4ba15.camel@huaweicloud.com>
From:   Casey Schaufler <casey@schaufler-ca.com>
In-Reply-To: <6b5f941a4dd57f357f942df4051ccc9995b4ba15.camel@huaweicloud.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 5/4/2023 11:53 PM, Roberto Sassu wrote:
> On Thu, 2023-05-04 at 17:59 -0700, Casey Schaufler wrote:
>> On 5/4/2023 9:11 AM, Roberto Sassu wrote:
>>> Hi Casey
>>>
>>> while developing the fix for overlayfs, I tried first to address the
>>> issue of a NFS filesystem failing to mount.
>>>
>>> The NFS server does not like the packets sent by the client:
>>>
>>> 14:52:20.827208 IP (tos 0x0, ttl 64, id 60628, offset 0, flags [DF], proto TCP (6), length 72, options (unknown 134,EOL))
>>>     localhost.localdomain.omginitialrefs > _gateway.nfs: Flags [S], cksum 0x7618 (incorrect -> 0xa18c), seq 455337903, win 64240, options [mss 1460,sackOK,TS val 2178524519 ecr 0,nop,wscale 7], length 0
>>> 14:52:20.827376 IP (tos 0xc0, ttl 64, id 5906, offset 0, flags [none], proto ICMP (1), length 112, options (unknown 134,EOL))
>>>     _gateway > localhost.localdomain: ICMP parameter problem - octet 22, length 80
>>>
>>> I looked at the possible causes. SELinux works properly.
>> SELinux was the reference LSM implementation for labeled networking.
>>
>>> What it seems to happen is that there is a default netlabel mapping,
>>> that is used to send the packets out.
>> Correct. SELinux only uses CIPSO options for MLS. Smack uses CIPSO for
>> almost all packets.
>>
>>> We are in this part of the code:
>>>
>>> Thread 1 hit Breakpoint 2, netlbl_sock_setattr (sk=sk@entry=0xffff888025178000, family=family@entry=2, secattr=0xffff88802504b200) at net/netlabel/netlabel_kapi.c:980
>>> 980	{
>>> (gdb) n
>>> 771		__rcu_read_lock();
>>> (gdb) 
>>> 985		dom_entry = netlbl_domhsh_getentry(secattr->domain, family);
>>> (gdb) 
>>> 986		if (dom_entry == NULL) {
>>> (gdb) 
>>> 990		switch (family) {
>>> (gdb) 
>>> 992			switch (dom_entry->def.type) {
>>>
>>> Here is the difference between Smack and SELinux.
>>>
>>> Smack:
>>>
>>> (gdb) p *dom_entry
>>> $2 = {domain = 0x0 <fixed_percpu_data>, family = 2, def = {type = 3, {addrsel = 0xffff888006bbef40, cipso = 0xffff888006bbef40, calipso = 0xffff888006bbef40}}, valid = 1, list = {next = 0xffff88800767f6e8, prev = 0xffff88800767f6e8}, rcu = {next = 0x0 <fixed_percpu_data>, 
>>>     func = 0x0 <fixed_percpu_data>}}
>>>
>>> SELinux:
>>>
>>> (gdb) p *dom_entry
>>> $5 = {domain = 0x0 <fixed_percpu_data>, family = 2, def = {type = 5, {addrsel = 0x0 <fixed_percpu_data>, cipso = 0x0 <fixed_percpu_data>, calipso = 0x0 <fixed_percpu_data>}}, valid = 1, list = {next = 0xffff888006012c88, prev = 0xffff888006012c88}, rcu = {
>>>     next = 0x0 <fixed_percpu_data>, func = 0x0 <fixed_percpu_data>}}
>>>
>>>
>>> type = 3 (for Smack) is NETLBL_NLTYPE_CIPSOV4.
>>> type = 5 (for SELinux) is NETLBL_NLTYPE_UNLABELED.
>>>
>>> This is why SELinux works (no incompatible options are sent).
>> SELinux "works" because that's the use case that was verified.
>>
>>> The netlabel mapping is added here:
>>>
>>> static void smk_cipso_doi(void)
>>> {
>>>
>>> [...]
>>>
>>> 	rc = netlbl_cfg_cipsov4_map_add(doip->doi, NULL, NULL, NULL, &nai);
>>>
>>>
>>> Not sure exactly how we can solve this issue. Just checked that
>>> commenting the call to smk_cipso_doi() in init_smk_fs() allows the NFS
>>> filesystem to be mounted.
>> Are both the server and client using Smack? Are they on a network that can
>> propagate labeled packets? What are you using for a Smack rule configuration?
> Only the client (Fedora 38).

Does the client run processes with Smack labels other than floor ("_")?
Are you using any of the Smack mount options?
What value is in /sys/fs/smackfs/ambient?

>  The server is Ubuntu 20.04.06 LTS and uses
> Apparmor.

Because the AppArmor server doesn't speak CIPSO you will need to identify
it as an unlabeled host. This effectively labels all communication with
the host as having a specific label. See Documentation/admin-guide/LSM/Smack.rst
for details.

>  The client is a VM created with libvirt. The connection is
> the classic tap attached to a bridge.

OK, does TAP on a bridge support IPv4 options on packets?

>
> Thanks
>
> Roberto
>