Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\))
Subject: Re: [PATCH v2 00/27] NFSD operation monitoring tracepoints
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <BEF0E50C-A658-4AAA-BCBD-49F442A338B5@oracle.com>
Date:   Fri, 25 Sep 2020 13:04:55 -0400
Cc:     Bill Baker <Bill.Baker@oracle.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <551339D6-2109-487D-8279-746BCA106893@oracle.com>
References: <160071167664.1468.1365570508917640511.stgit@klimt.1015granger.net>
 <20200924213617.GA12407@fieldses.org>
 <945A7DE6-909D-4177-852F-F80EF7DFE6B3@oracle.com>
 <20200925143218.GD1096@fieldses.org>
 <23DF63F3-44AC-4DDE-AAB9-E178F4B68103@oracle.com>
 <20200925150038.GF1096@fieldses.org>
 <BEF0E50C-A658-4AAA-BCBD-49F442A338B5@oracle.com>
To:     Bruce Fields <bfields@fieldses.org>
Precedence: bulk


> On Sep 25, 2020, at 11:05 AM, Chuck Lever <chuck.lever@oracle.com> =
wrote:
>=20
>=20
>=20
>> On Sep 25, 2020, at 11:00 AM, Bruce Fields <bfields@fieldses.org> =
wrote:
>>=20
>> On Fri, Sep 25, 2020 at 10:36:42AM -0400, Chuck Lever wrote:
>>>=20
>>>=20
>>>> On Sep 25, 2020, at 10:32 AM, Bruce Fields <bfields@fieldses.org> =
wrote:
>>>>=20
>>>> On Fri, Sep 25, 2020 at 09:59:51AM -0400, Chuck Lever wrote:
>>>>> Thanks Bruce, for your time, attention, and comments!
>>>>>=20
>>>>>> On Sep 24, 2020, at 5:36 PM, J. Bruce Fields =
<bfields@fieldses.org> wrote:
>>>>>>=20
>>>>>> On Mon, Sep 21, 2020 at 02:10:49PM -0400, Chuck Lever wrote:
>>>>>>> As I've been working on various server bugs, I've been adding
>>>>>>> tracepoints that record NFS operation arguments. Here's an =
updated
>>>>>>> snapshot of this work for your review and comment.
>>>>>>>=20
>>>>>>> The idea here is to provide a degree of NFS traffic =
observability
>>>>>>> without needing network capture. Tracepoints are generally =
lighter-
>>>>>>> weight than full network capture, allowing effective =
capture-time
>>>>>>> data reduction:
>>>>>>=20
>>>>>> I do wonder when tracepoints seem to duplicate information you =
could get
>>>>>> from network traces, so thanks for taking the time to explain =
this.  It
>>>>>> makes sense to me.
>>>>>>=20
>>>>>> The patches look fine.  The only one I'm I'm on the fence about =
is the
>>>>>> last with the split up of the dispatch functions.  I'll ask some
>>>>>> questions there....
>>>>>=20
>>>>> To be clear to everyone, this series is still "preview". I expect
>>>>> more churn in these patches, thus I don't consider the series =
ready
>>>>> to be merged by any stretch.
>>>>=20
>>>> OK!
>>>>=20
>>>> One thing I was wondering about: how would you limit tracing to a =
single
>>>> client, say if you wanted to see all DELEGRETURNs from a single =
client?
>>>> I guess you'd probably turn on a tracepoint in the receive code, =
look
>>>> for your client's IP address, then mask the task id to match later
>>>> nfs-level tracepoints.  Is there enough information in those =
tracepoints
>>>> (including network namespace) to uniquely identify a client?
>>>=20
>>> Client IP address information is in the RPC layer trace data. The
>>> DELEGRETURN trace record includes client ID. So maybe not as
>>> straightforward as it could be.
>>=20
>> I guess what I meant was "limit tracing to a single network =
endpoint",
>> not exactly limt to a single NFSv4 client....  So, we can do that as
>> long as all the relevant information is in rpc-layer tracepoints, and =
as
>> long as task id is a reliable way to match up trace points.
>>=20
>> Is the network namespace in there anywhere?  It looks like there'd be =
no
>> way to distinguish clients in different namespaces if they had the =
same
>> address.
>=20
> The client ID has the boot verifier for the net namespace.
>=20
> None of this helps NFSv3, though.

It probably wouldn't be difficult to stuff the client IP address
and the boot verifier in the trace record for each procedure.

Do you think that would be sufficient?


--
Chuck Lever