Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
From:   Benjamin Coddington <bcodding@redhat.com>
To:     Charles Hedrick <hedrick@rutgers.edu>
Cc:     linux-nfs@vger.kernel.org
Subject: Re: bad info in NFS context
Date:   Thu, 21 Sep 2023 10:38:22 -0400
Message-ID: <7B541D75-296B-4264-92BC-6154B84F2557@redhat.com>
In-Reply-To: <PH0PR14MB5493A8657F42289DBC301CC2AAF8A@PH0PR14MB5493.namprd14.prod.outlook.com>
References: <PH0PR14MB5493ED33985C95FEBE4DA808AAF9A@PH0PR14MB5493.namprd14.prod.outlook.com>
 <B7D023CD-2810-4BD6-8570-AB0C0EE95287@redhat.com>
 <PH0PR14MB5493AB9814249EC5D66E635DAAF8A@PH0PR14MB5493.namprd14.prod.outlook.com>
 <650954F9-F67D-4F62-AD7B-4D16DF45E168@redhat.com>
 <PH0PR14MB5493A8657F42289DBC301CC2AAF8A@PH0PR14MB5493.namprd14.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On 21 Sep 2023, at 9:56, Charles Hedrick wrote:

> thanks. I can work with that info. Restarting the server isn't practica=
l. This is a large-scale system serving hundreds of students. We generall=
y keep it up uninterrupted for a whole semester.

By web server, I mean the process not the system.  Though, it must have a=
 lot of local state if you don't have it load-balanced and redundant, so =
maybe even restarting the process is impractical.

Without trying this, we're still guessing that it's the ACCESS cache.  Yo=
u should be able to do something like "sudo su - webserveruser" and that =
/should/ count as a login for that process, and so that process /should/ =
gain the access you need from the new membership.  Its worth a test to ma=
ke sure we're not actually dealing with a different problem.

>> =C2=A0So, the NFS client will keep caching the result of previous call=
s to unchanged inodes until it notices that the process' oldest parent wi=
th the same user/credential has a task start_time that is older than the =
currently cached entries.
>
> I trust you mean newer. This is jupyterhub, which likes to keep user pr=
ocesses around after logout and reattach when they login. But as long as =
we know what's going on, there's a way for a user to kill their processes=
 manually.

There's been some attempts to add an "fasc" or "nofasc" mount option to u=
pstream NFS client, which would modify the behavior of the client.  That'=
s not had a lot of traction (I think because the patch wants to change th=
e default behavior again).

It's possible to submit work to add a sysfs knob to flush the access cach=
e.. that could look like a full cache flush for everyone, or maybe upon w=
riting a uid to a sysfs file, a flush for cached entries.

Have to tried talking to your NFS client vendor about this problem?

Ben