Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.1\))
Subject: Re: [PATCH v2 00/27] NFSD operation monitoring tracepoints
From:   Chuck Lever <chuck.lever@oracle.com>
In-Reply-To: <20200925183717.GG1096@fieldses.org>
Date:   Fri, 25 Sep 2020 14:41:57 -0400
Cc:     Bill Baker <Bill.Baker@oracle.com>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <80E7EC1D-A226-4765-89AA-2E7F81225671@oracle.com>
References: <160071167664.1468.1365570508917640511.stgit@klimt.1015granger.net>
 <20200924213617.GA12407@fieldses.org>
 <945A7DE6-909D-4177-852F-F80EF7DFE6B3@oracle.com>
 <20200925143218.GD1096@fieldses.org>
 <23DF63F3-44AC-4DDE-AAB9-E178F4B68103@oracle.com>
 <20200925150038.GF1096@fieldses.org>
 <BEF0E50C-A658-4AAA-BCBD-49F442A338B5@oracle.com>
 <551339D6-2109-487D-8279-746BCA106893@oracle.com>
 <20200925183717.GG1096@fieldses.org>
To:     Bruce Fields <bfields@fieldses.org>
Precedence: bulk


> On Sep 25, 2020, at 2:37 PM, Bruce Fields <bfields@fieldses.org> =
wrote:
>=20
> On Fri, Sep 25, 2020 at 01:04:55PM -0400, Chuck Lever wrote:
>>> On Sep 25, 2020, at 11:05 AM, Chuck Lever <chuck.lever@oracle.com> =
wrote:
>>>> On Sep 25, 2020, at 11:00 AM, Bruce Fields <bfields@fieldses.org> =
wrote:
>>>> On Fri, Sep 25, 2020 at 10:36:42AM -0400, Chuck Lever wrote:
>>>>>> One thing I was wondering about: how would you limit tracing to a =
single
>>>>>> client, say if you wanted to see all DELEGRETURNs from a single =
client?
>>>>>> I guess you'd probably turn on a tracepoint in the receive code, =
look
>>>>>> for your client's IP address, then mask the task id to match =
later
>>>>>> nfs-level tracepoints.  Is there enough information in those =
tracepoints
>>>>>> (including network namespace) to uniquely identify a client?
>>>>>=20
>>>>> Client IP address information is in the RPC layer trace data. The
>>>>> DELEGRETURN trace record includes client ID. So maybe not as
>>>>> straightforward as it could be.
>>>>=20
>>>> I guess what I meant was "limit tracing to a single network =
endpoint",
>>>> not exactly limt to a single NFSv4 client....  So, we can do that =
as
>>>> long as all the relevant information is in rpc-layer tracepoints, =
and as
>>>> long as task id is a reliable way to match up trace points.
>>>>=20
>>>> Is the network namespace in there anywhere?  It looks like there'd =
be no
>>>> way to distinguish clients in different namespaces if they had the =
same
>>>> address.
>>>=20
>>> The client ID has the boot verifier for the net namespace.
>>>=20
>>> None of this helps NFSv3, though.
>>=20
>> It probably wouldn't be difficult to stuff the client IP address
>> and the boot verifier in the trace record for each procedure.
>>=20
>> Do you think that would be sufficient?
>=20
> Despite using 64 bits the boot verifier isn't even guaranteed to
> uniquely identify a network namespace.  There should be something
> better.  Digging around....  ns->inum, I think?

Could use that, but NFSD already uses the boot verifier everywhere.
I'll have a look.


> I don't know if it (or ports or addresses) needs to be in every trace
> record.  Maybe just once somewhere in one of the rpc layer =
tracepoints,
> and then we can use the nfsd task id to connect it with other
> tracepoints if necessary.  Does that make sense?

If you want to do filtering during capture, you want to have this
kind of information available in each record so the generic trace
utilities can find it.

We don't have to display it in every record that is output by
trace-cmd report, though.


--
Chuck Lever