Received-SPF: pass (google.com: best guess record for domain of linux-nfs-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190808201848.36640-1-olga.kornievskaia@gmail.com>
 <20190808201848.36640-6-olga.kornievskaia@gmail.com> <CAN-5tyF6ZMf_jiKycK1rk9v7xATDb=j_e5d0QBp9LOgdvn-utA@mail.gmail.com>
 <CAN-5tyEyvjJB+4_bKZmEYhv2KrVvk7dDvF27i6mx4naDt33Nww@mail.gmail.com>
 <20190812200019.GB29812@parsley.fieldses.org> <CAN-5tyFpHHsH3n8u+qGyp7POdSRHesiKgd3-YQoE9jJSPBVYRw@mail.gmail.com>
In-Reply-To: <CAN-5tyFpHHsH3n8u+qGyp7POdSRHesiKgd3-YQoE9jJSPBVYRw@mail.gmail.com>
From:   Olga Kornievskaia <olga.kornievskaia@gmail.com>
Date:   Wed, 14 Aug 2019 11:05:23 -0400
Message-ID: <CAN-5tyEX=4pfntZudpfkN9Pr9OpZfpvKj+NyT-bBD9YJ5fsg7Q@mail.gmail.com>
Subject: Re: [PATCH v5 5/9] NFSD add COPY_NOTIFY operation
To:     "J. Bruce Fields" <bfields@redhat.com>
Cc:     linux-nfs <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

On Tue, Aug 13, 2019 at 1:57 PM Olga Kornievskaia
<olga.kornievskaia@gmail.com> wrote:
>
> On Mon, Aug 12, 2019 at 4:00 PM J. Bruce Fields <bfields@redhat.com> wrote:
> >
> > On Mon, Aug 12, 2019 at 03:16:47PM -0400, Olga Kornievskaia wrote:
> > > On Mon, Aug 12, 2019 at 12:19 PM Olga Kornievskaia
> > > <olga.kornievskaia@gmail.com> wrote:
> > > > While this passes my testing, in theory this allows for the race that
> > > > we get the copy notify size but then offload_cancel arrive and change
> > > > the value. Then refcount_sub_and test_check would have an incorrect
> > > > value (can subtract larger than an actual reference count). I have no
> > > > solution for that as there is no refcount_sub_and_lock() that will
> > > > allow to decrement by a multiple under a lock. Thoughts?
> > >
> > > I tried not to use the client's cl_lock but instead use a specific
> > > lock to protect the copy notifies stateid on the stateid list. But
> > > since stateid's reference counter (sc_count) is protected by it, I
> > > think by getting rid of the special lock and using cl_lock will solve
> > > the problem of coordinating access between the sc_count and the
> > > copy_notify stateid list. Are the any problems with using such a big
> > > lock?
> >
> > Probably not.  But it can be confusing when a single lock is used for
> > several different things.  A comment explaining why you need it might
> > help.
>
> While holding the client's cl_lock to manipulate the list of copy
> notify stateids solves the refcount problem. It generates a different
> problem for the laundromat thread. There, client list is traversed
> already holding the cl_lock, so I can't call routines to free
> copy_notify stateid because in turn it calls nfs4_put_stid() which
> wants to take the cl_lock. Putting the copy_notify stateid on the
> reaplist and then I lose a pointer to the client structure that I need
> to take the lock. Then it seems the nfs4_cpntf_state structure would
> need to keep a pointer to the client structure but then I get a
> problem of making sure the nfs4_client structure isn't going away and
> because it even a bigger mess.
>
> I think I need to remove the code in the laundromat that looks for the
> not referenced copy_notifies stateid and just rely on cleaning on the
> removal of the stateid (basically what I originally had). Or I need to
> rely on the client to always send FREE_STATEID. I don't see other
> options, do you?

Ignore this Bruce. Trond gave me a good idea and gets me unstuck.

>
> >
> > --b.