Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190808201848.36640-1-olga.kornievskaia@gmail.com>
 <20190808201848.36640-6-olga.kornievskaia@gmail.com> <CAN-5tyF6ZMf_jiKycK1rk9v7xATDb=j_e5d0QBp9LOgdvn-utA@mail.gmail.com>
 <CAN-5tyEyvjJB+4_bKZmEYhv2KrVvk7dDvF27i6mx4naDt33Nww@mail.gmail.com> <20190812200019.GB29812@parsley.fieldses.org>
In-Reply-To: <20190812200019.GB29812@parsley.fieldses.org>
From:   Olga Kornievskaia <olga.kornievskaia@gmail.com>
Date:   Tue, 13 Aug 2019 13:57:39 -0400
Message-ID: <CAN-5tyFpHHsH3n8u+qGyp7POdSRHesiKgd3-YQoE9jJSPBVYRw@mail.gmail.com>
Subject: Re: [PATCH v5 5/9] NFSD add COPY_NOTIFY operation
To:     "J. Bruce Fields" <bfields@redhat.com>
Cc:     linux-nfs <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

On Mon, Aug 12, 2019 at 4:00 PM J. Bruce Fields <bfields@redhat.com> wrote:
>
> On Mon, Aug 12, 2019 at 03:16:47PM -0400, Olga Kornievskaia wrote:
> > On Mon, Aug 12, 2019 at 12:19 PM Olga Kornievskaia
> > <olga.kornievskaia@gmail.com> wrote:
> > > While this passes my testing, in theory this allows for the race that
> > > we get the copy notify size but then offload_cancel arrive and change
> > > the value. Then refcount_sub_and test_check would have an incorrect
> > > value (can subtract larger than an actual reference count). I have no
> > > solution for that as there is no refcount_sub_and_lock() that will
> > > allow to decrement by a multiple under a lock. Thoughts?
> >
> > I tried not to use the client's cl_lock but instead use a specific
> > lock to protect the copy notifies stateid on the stateid list. But
> > since stateid's reference counter (sc_count) is protected by it, I
> > think by getting rid of the special lock and using cl_lock will solve
> > the problem of coordinating access between the sc_count and the
> > copy_notify stateid list. Are the any problems with using such a big
> > lock?
>
> Probably not.  But it can be confusing when a single lock is used for
> several different things.  A comment explaining why you need it might
> help.

While holding the client's cl_lock to manipulate the list of copy
notify stateids solves the refcount problem. It generates a different
problem for the laundromat thread. There, client list is traversed
already holding the cl_lock, so I can't call routines to free
copy_notify stateid because in turn it calls nfs4_put_stid() which
wants to take the cl_lock. Putting the copy_notify stateid on the
reaplist and then I lose a pointer to the client structure that I need
to take the lock. Then it seems the nfs4_cpntf_state structure would
need to keep a pointer to the client structure but then I get a
problem of making sure the nfs4_client structure isn't going away and
because it even a bigger mess.

I think I need to remove the code in the laundromat that looks for the
not referenced copy_notifies stateid and just rely on cleaning on the
removal of the stateid (basically what I originally had). Or I need to
rely on the client to always send FREE_STATEID. I don't see other
options, do you?

>
> --b.