2009-03-12 14:00:49

by Chuck Lever III

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

Hi Kevin-

man watch(1)

On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:

> I'd really like to have a way to output the nfsstat counters at
> regular intervals (every 3 seconds) where the output is the
> difference between 3 seconds ago and now. Frequently I'll run a
> test and want to watch the nfs call profile throughout the course of
> the test.
>
> Does something like this already exist?
> Are there objections to seeing a feature like this?
>
> I'm thinking something like:
> nfsstat --sleep=1
>
> nfs v3 call: Server Client
> total: 0 3476
> null: 0 0
> getattr: 0 1679
> setattr: 0 0
> lookup: 0 839
> access: 0 839
> readlink: 0 0
> read: 0 0
> write: 0 0
> create: 0 0
> mkdir: 0 0
> symlink: 0 0
> mknod: 0 0
> remove: 0 0
> rmdir: 0 0
> rename: 0 0
> link: 0 0
> readdir: 0 0
> readdirplus: 0 0
> fsstat: 0 119
> fsinfo: 0 0
> pathconf: 0 0
> commit: 0 0
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com






2009-03-12 16:15:24

by Greg Banks

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

Chuck Lever wrote:
> Hi Kevin-
>
> man watch(1)
Watch + today's nfsstat isn't smart enough to tell you deltas, or do
rate conversion. However...

http://oss.sgi.com/projects/pcp/

PCP includes a gui tool which gives you rate conversion and graphical
presentation with history and automatic scaling of any metrics. Plus
you can do archive recording and playback, and remote access. Supported
metrics include the existing server stats (but not the client-side
per-server stats).

In the absence of a useful tool like PCP, some time ago I did write a
patch add some of those features (rate conversion in particular) to
nfsstat. The patch gives nfsstat a mode where it behaves like "vmstat
1" or "iostat 1", i.e. textually reports deltas or rates continuously in
a loop with a controllable interval. I dug it up earlier today but
unfortunately it won't apply anymore (due to changes for the
implementation of the --sleep mode). I won't have time to work on it
just now, but I can post it if anyone cares to mine it for ideas.

--
Greg Banks, P.Engineer, SGI Australian Software Group.
the brightly coloured sporks of revolution.
I don't speak for SGI.


2009-03-12 15:50:57

by J. Bruce Fields

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
> Hi Kevin-
>
> man watch(1)

What would you watch?

--b.

>
> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>
>> I'd really like to have a way to output the nfsstat counters at
>> regular intervals (every 3 seconds) where the output is the difference
>> between 3 seconds ago and now. Frequently I'll run a test and want to
>> watch the nfs call profile throughout the course of the test.
>>
>> Does something like this already exist?
>> Are there objections to seeing a feature like this?
>>
>> I'm thinking something like:
>> nfsstat --sleep=1
>>
>> nfs v3 call: Server Client
>> total: 0 3476
>> null: 0 0
>> getattr: 0 1679
>> setattr: 0 0
>> lookup: 0 839
>> access: 0 839
>> readlink: 0 0
>> read: 0 0
>> write: 0 0
>> create: 0 0
>> mkdir: 0 0
>> symlink: 0 0
>> mknod: 0 0
>> remove: 0 0
>> rmdir: 0 0
>> rename: 0 0
>> link: 0 0
>> readdir: 0 0
>> readdirplus: 0 0
>> fsstat: 0 119
>> fsinfo: 0 0
>> pathconf: 0 0
>> commit: 0 0
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>> in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2009-03-12 16:09:14

by Chuck Lever III

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>> Hi Kevin-
>>
>> man watch(1)
>
> What would you watch?

For example:

watch -n3 nfsstat -c

You can also use the "-d" option to highlight the differences between
the current sample and the previous sample.

> --b.
>
>>
>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>
>>> I'd really like to have a way to output the nfsstat counters at
>>> regular intervals (every 3 seconds) where the output is the
>>> difference
>>> between 3 seconds ago and now. Frequently I'll run a test and
>>> want to
>>> watch the nfs call profile throughout the course of the test.
>>>
>>> Does something like this already exist?
>>> Are there objections to seeing a feature like this?
>>>
>>> I'm thinking something like:
>>> nfsstat --sleep=1
>>>
>>> nfs v3 call: Server Client
>>> total: 0 3476
>>> null: 0 0
>>> getattr: 0 1679
>>> setattr: 0 0
>>> lookup: 0 839
>>> access: 0 839
>>> readlink: 0 0
>>> read: 0 0
>>> write: 0 0
>>> create: 0 0
>>> mkdir: 0 0
>>> symlink: 0 0
>>> mknod: 0 0
>>> remove: 0 0
>>> rmdir: 0 0
>>> rename: 0 0
>>> link: 0 0
>>> readdir: 0 0
>>> readdirplus: 0 0
>>> fsstat: 0 119
>>> fsinfo: 0 0
>>> pathconf: 0 0
>>> commit: 0 0
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>> in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> --
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-
>> nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com





2009-03-12 16:25:03

by J. Bruce Fields

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>> Hi Kevin-
>>>
>>> man watch(1)
>>
>> What would you watch?
>
> For example:
>
> watch -n3 nfsstat -c
>
> You can also use the "-d" option to highlight the differences between
> the current sample and the previous sample.

He was asking for deltas; the above only gives cumulative totals.

There's no accurate one-line solution using the existing nfsstat
commandline, but it should be easy to add.

--b.

>
>> --b.
>>
>>>
>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>
>>>> I'd really like to have a way to output the nfsstat counters at
>>>> regular intervals (every 3 seconds) where the output is the
>>>> difference
>>>> between 3 seconds ago and now. Frequently I'll run a test and
>>>> want to
>>>> watch the nfs call profile throughout the course of the test.
>>>>
>>>> Does something like this already exist?
>>>> Are there objections to seeing a feature like this?
>>>>
>>>> I'm thinking something like:
>>>> nfsstat --sleep=1
>>>>
>>>> nfs v3 call: Server Client
>>>> total: 0 3476
>>>> null: 0 0
>>>> getattr: 0 1679
>>>> setattr: 0 0
>>>> lookup: 0 839
>>>> access: 0 839
>>>> readlink: 0 0
>>>> read: 0 0
>>>> write: 0 0
>>>> create: 0 0
>>>> mkdir: 0 0
>>>> symlink: 0 0
>>>> mknod: 0 0
>>>> remove: 0 0
>>>> rmdir: 0 0
>>>> rename: 0 0
>>>> link: 0 0
>>>> readdir: 0 0
>>>> readdirplus: 0 0
>>>> fsstat: 0 119
>>>> fsinfo: 0 0
>>>> pathconf: 0 0
>>>> commit: 0 0
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>> in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>> --
>>> Chuck Lever
>>> chuck[dot]lever[at]oracle[dot]com
>>>
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>> in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> Chuck Lever
> chuck[dot]lever[at]oracle[dot]com
>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2009-03-12 16:34:51

by Chuck Lever III

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

On Mar 12, 2009, at Mar 12, 2009, 12:24 PM, J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>> Hi Kevin-
>>>>
>>>> man watch(1)
>>>
>>> What would you watch?
>>
>> For example:
>>
>> watch -n3 nfsstat -c
>>
>> You can also use the "-d" option to highlight the differences between
>> the current sample and the previous sample.
>
> He was asking for deltas; the above only gives cumulative totals.

In that case, he might try one of the Python scripts I wrote that
report NFS and RPC metrics. mountstats and nfs-iostat are included in
the latest nfs-utils tarball. If they don't do exactly what is
needed, it should be easy to hack them.

> There's no accurate one-line solution using the existing nfsstat
> commandline, but it should be easy to add.
>
> --b.
>
>>
>>> --b.
>>>
>>>>
>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>
>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>> regular intervals (every 3 seconds) where the output is the
>>>>> difference
>>>>> between 3 seconds ago and now. Frequently I'll run a test and
>>>>> want to
>>>>> watch the nfs call profile throughout the course of the test.
>>>>>
>>>>> Does something like this already exist?
>>>>> Are there objections to seeing a feature like this?
>>>>>
>>>>> I'm thinking something like:
>>>>> nfsstat --sleep=1
>>>>>
>>>>> nfs v3 call: Server Client
>>>>> total: 0 3476
>>>>> null: 0 0
>>>>> getattr: 0 1679
>>>>> setattr: 0 0
>>>>> lookup: 0 839
>>>>> access: 0 839
>>>>> readlink: 0 0
>>>>> read: 0 0
>>>>> write: 0 0
>>>>> create: 0 0
>>>>> mkdir: 0 0
>>>>> symlink: 0 0
>>>>> mknod: 0 0
>>>>> remove: 0 0
>>>>> rmdir: 0 0
>>>>> rename: 0 0
>>>>> link: 0 0
>>>>> readdir: 0 0
>>>>> readdirplus: 0 0
>>>>> fsstat: 0 119
>>>>> fsinfo: 0 0
>>>>> pathconf: 0 0
>>>>> commit: 0 0
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>>>> nfs"
>>>>> in
>>>>> the body of a message to [email protected]
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>
>>>> --
>>>> Chuck Lever
>>>> chuck[dot]lever[at]oracle[dot]com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>>> nfs"
>>>> in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> --
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-
>> nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com





2009-03-12 16:45:06

by Kevin Constantine

[permalink] [raw]
Subject: Re: nfsstat --sleep=#

J. Bruce Fields wrote:
> On Thu, Mar 12, 2009 at 12:08:57PM -0400, Chuck Lever wrote:
>> On Mar 12, 2009, at Mar 12, 2009, 11:50 AM, J. Bruce Fields wrote:
>>> On Thu, Mar 12, 2009 at 10:00:34AM -0400, Chuck Lever wrote:
>>>> Hi Kevin-
>>>>
>>>> man watch(1)
>>> What would you watch?
>> For example:
>>
>> watch -n3 nfsstat -c
>>
>> You can also use the "-d" option to highlight the differences between
>> the current sample and the previous sample.
>
> He was asking for deltas; the above only gives cumulative totals.
>
> There's no accurate one-line solution using the existing nfsstat
> commandline, but it should be easy to add.
>

Something like this sort of works:
watch -n 1 'nfsstat --since /tmp/stats; nfsstat >/tmp/stats', but it
feels more like a workaround than a feature.

Using watch doesn't allow you to see what happened in the past either.

Moving to a listed output format instead of the traditional nfsstat
output (as seen below) makes it trivial with a simple grep to watch the
stats that you really care about and ignore the rest.

-kevin


> --b.
>
>>> --b.
>>>
>>>> On Mar 11, 2009, at Mar 11, 2009, 9:37 PM, Kevin Constantine wrote:
>>>>
>>>>> I'd really like to have a way to output the nfsstat counters at
>>>>> regular intervals (every 3 seconds) where the output is the
>>>>> difference
>>>>> between 3 seconds ago and now. Frequently I'll run a test and
>>>>> want to
>>>>> watch the nfs call profile throughout the course of the test.
>>>>>
>>>>> Does something like this already exist?
>>>>> Are there objections to seeing a feature like this?
>>>>>
>>>>> I'm thinking something like:
>>>>> nfsstat --sleep=1
>>>>>
>>>>> nfs v3 call: Server Client
>>>>> total: 0 3476
>>>>> null: 0 0
>>>>> getattr: 0 1679
>>>>> setattr: 0 0
>>>>> lookup: 0 839
>>>>> access: 0 839
>>>>> readlink: 0 0
>>>>> read: 0 0
>>>>> write: 0 0
>>>>> create: 0 0
>>>>> mkdir: 0 0
>>>>> symlink: 0 0
>>>>> mknod: 0 0
>>>>> remove: 0 0
>>>>> rmdir: 0 0
>>>>> rename: 0 0
>>>>> link: 0 0
>>>>> readdir: 0 0
>>>>> readdirplus: 0 0
>>>>> fsstat: 0 119
>>>>> fsinfo: 0 0
>>>>> pathconf: 0 0
>>>>> commit: 0 0
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>>> in
>>>>> the body of a message to [email protected]
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>> --
>>>> Chuck Lever
>>>> chuck[dot]lever[at]oracle[dot]com
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
>>>> in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> Chuck Lever
>> chuck[dot]lever[at]oracle[dot]com
>>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html