These are also available in the 'mountstats-improvements-v4' branch at
https://github.com/scottmayhew/nfs-utils.git
v4 changes:
- SteveD was not a fan of the ms-iostat and ms-nfsstat command names.
Rather than think up new names, I decided to make these all sub-commands.
If no sub-command is given by the user then the 'mountstats' sub-command
will run by default (so any existing scripts that run older versions of
the mountstats command should still work).
- Rather than have 3 tiny man pages, I documented the main command and the
three sub-commands on a single man page.
v3 changes:
- Rebased on top of the lastest mountstats version
- Added exception processing to fail more gracefully when the argparse
module is not installed
- Corrected some issues with the man pages (filenames not italicized,
unmatched braces in the synopses)
v2 changes:
- Changed the parsing to use the argparse module instead of optparse
- Added per-mountpoint headers to the output of 'mountstats --rpc'
- Revamped the ms-nfsstat command to take a variable list of mountpoints
(so now mountstats, ms-iostat, and ms-nfsstat all behave in the same
manner). Added -3 and -4 options which behave the same way they do in
nfsstat.c. The output doesn't still doesn't match up 100% with that
of nfsstat.c though (I'm just taking the first 12 characters of the
operation name and converting them to lowercase to use as labels, while
nfsstat.c defines its own labels, but also it looks to me like nfsstat.c
is missing some operations altoegether).
- Updated the man page for mountstats and added man pages for ms-iostat
and ms-nfsstat.
Original cover letter:
-------------------8<------------------
The following patches add a couple of enhancements to mountstats.py. I
also fixed a few bugs I encountered along the way. Highlights include:
- added support for -f/--file to allow stats to be parsed from an
aritrary input file instead of /proc/self/mountstats
- added support for -S/--since to show just the changes that have
occurred between the current and a previous set of statisics (works
with and without the -f option)
- added support for -R/--raw to generate 'raw' statistics (i.e. in the
same format as /proc/self/mountstats). It's intended to be used with
the -f and -S options.
- implemented the ms-nfsstat command to generate client-side
nfsstat-like statisics (only works with a single mountpoint)
My motivation for these changes was so that I could take various copies
of /proc/self/mountstats and massage them into data that I could feed
into the 'report' option of Dros's nfsometer tool for scenarios where
it's not feasible to run nfsometer itself (e.g. systems where we can't
start with an 'idle' state (i.e. no NFS filesystems initially
mounted), systems with multiple NFS filesystems mounted, and workloads
that can't easily be boiled down into an nfsometer workload file or run
via the custom workload environment variables).
Scott Mayhew (14):
mountstats: Fix up NFS event counters
mountstats: Add lists of various counters
mountstats: Refactor __parse_nfs_line and __parse_rpc_line
mountstats: Refactor compare_iostats
mountstats: Convert existing option parsing to use the argparse module
mountstats: Make the iostat sub-command output match that of
nfs-iostat.py
mountstats: Make print_iostat_summary handle newly appearing mounts
mountstats: Add support for -f/--file
mountstats: Add support for -S/--since
mountstats: Fix IndexError in __parse_nfs_line
mountstats: Allow mountstats_command to take a variable number of
mountpoints
mountstats: Add support for -R/--raw to mountstats_command
mountstats: Implement the nfsstat sub-command
mountstats: Updated the mountstats(8) man page.
tools/mountstats/mountstats.man | 142 +++++-
tools/mountstats/mountstats.py | 927 ++++++++++++++++++++++++++++------------
2 files changed, 775 insertions(+), 294 deletions(-)
--
1.9.3
The event counters in the mountstats program aren't in sync with the
event counters in the kernel. Removed syncinodes and added
vfsupdatepage, vfssetattr, congestionwait, pnfsreads, and pnfswrites.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 33 ++++++++++++++++++---------------
1 file changed, 18 insertions(+), 15 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 247a64a..5fc93f4 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -74,25 +74,29 @@ class DeviceData:
self.__nfs_data['dentryrevalidates'] = int(words[2])
self.__nfs_data['datainvalidates'] = int(words[3])
self.__nfs_data['attrinvalidates'] = int(words[4])
- self.__nfs_data['syncinodes'] = int(words[5])
- self.__nfs_data['vfsopen'] = int(words[6])
- self.__nfs_data['vfslookup'] = int(words[7])
- self.__nfs_data['vfspermission'] = int(words[8])
+ self.__nfs_data['vfsopen'] = int(words[5])
+ self.__nfs_data['vfslookup'] = int(words[6])
+ self.__nfs_data['vfspermission'] = int(words[7])
+ self.__nfs_data['vfsupdatepage'] = int(words[8])
self.__nfs_data['vfsreadpage'] = int(words[9])
self.__nfs_data['vfsreadpages'] = int(words[10])
self.__nfs_data['vfswritepage'] = int(words[11])
self.__nfs_data['vfswritepages'] = int(words[12])
self.__nfs_data['vfsreaddir'] = int(words[13])
- self.__nfs_data['vfsflush'] = int(words[14])
- self.__nfs_data['vfsfsync'] = int(words[15])
- self.__nfs_data['vfslock'] = int(words[16])
- self.__nfs_data['vfsrelease'] = int(words[17])
- self.__nfs_data['setattrtrunc'] = int(words[18])
- self.__nfs_data['extendwrite'] = int(words[19])
- self.__nfs_data['sillyrenames'] = int(words[20])
- self.__nfs_data['shortreads'] = int(words[21])
- self.__nfs_data['shortwrites'] = int(words[22])
- self.__nfs_data['delay'] = int(words[23])
+ self.__nfs_data['vfssetattr'] = int(words[14])
+ self.__nfs_data['vfsflush'] = int(words[15])
+ self.__nfs_data['vfsfsync'] = int(words[16])
+ self.__nfs_data['vfslock'] = int(words[17])
+ self.__nfs_data['vfsrelease'] = int(words[18])
+ self.__nfs_data['congestionwait'] = int(words[19])
+ self.__nfs_data['setattrtrunc'] = int(words[20])
+ self.__nfs_data['extendwrite'] = int(words[21])
+ self.__nfs_data['sillyrenames'] = int(words[22])
+ self.__nfs_data['shortreads'] = int(words[23])
+ self.__nfs_data['shortwrites'] = int(words[24])
+ self.__nfs_data['delay'] = int(words[25])
+ self.__nfs_data['pnfsreads'] = int(words[26])
+ self.__nfs_data['pnfswrites'] = int(words[27])
elif words[0] == 'bytes:':
self.__nfs_data['normalreadbytes'] = int(words[1])
self.__nfs_data['normalwritebytes'] = int(words[2])
@@ -202,7 +206,6 @@ class DeviceData:
print('Cache events:')
print(' data cache invalidated %d times' % self.__nfs_data['datainvalidates'])
print(' attribute cache invalidated %d times' % self.__nfs_data['attrinvalidates'])
- print(' inodes synced %d times' % self.__nfs_data['syncinodes'])
print()
print('VFS calls:')
print(' VFS requested %d inode revalidations' % self.__nfs_data['inoderevalidates'])
--
1.9.3
The NfsEventCounters and NfsByteCounters were lifted straight from
nfs-iostat.py. Also added counters for the xprt line (for udp, tcp, and
rdma) as well as for v3 and v4 NFS ops. These will allow for more
compact code in a couple of places.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 168 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 168 insertions(+)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 5fc93f4..bf98813 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -33,6 +33,174 @@ def difference(x, y):
"""
return x - y
+NfsEventCounters = [
+ 'inoderevalidates',
+ 'dentryrevalidates',
+ 'datainvalidates',
+ 'attrinvalidates',
+ 'vfsopen',
+ 'vfslookup',
+ 'vfspermission',
+ 'vfsupdatepage',
+ 'vfsreadpage',
+ 'vfsreadpages',
+ 'vfswritepage',
+ 'vfswritepages',
+ 'vfsreaddir',
+ 'vfssetattr',
+ 'vfsflush',
+ 'vfsfsync',
+ 'vfslock',
+ 'vfsrelease',
+ 'congestionwait',
+ 'setattrtrunc',
+ 'extendwrite',
+ 'sillyrenames',
+ 'shortreads',
+ 'shortwrites',
+ 'delay',
+ 'pnfsreads',
+ 'pnfswrites'
+]
+
+NfsByteCounters = [
+ 'normalreadbytes',
+ 'normalwritebytes',
+ 'directreadbytes',
+ 'directwritebytes',
+ 'serverreadbytes',
+ 'serverwritebytes',
+ 'readpages',
+ 'writepages'
+]
+
+XprtUdpCounters = [
+ 'port',
+ 'bind_count',
+ 'rpcsends',
+ 'rpcreceives',
+ 'badxids',
+ 'inflightsends',
+ 'backlogutil'
+]
+
+XprtTcpCounters = [
+ 'port',
+ 'bind_count',
+ 'connect_count',
+ 'connect_time',
+ 'idle_time',
+ 'rpcsends',
+ 'rpcreceives',
+ 'badxids',
+ 'inflightsends',
+ 'backlogutil'
+]
+
+XprtRdmaCounters = [
+ 'port',
+ 'bind_count',
+ 'connect_count',
+ 'connect_time',
+ 'idle_time',
+ 'rpcsends',
+ 'rpcreceives',
+ 'badxids',
+ 'backlogutil',
+ 'read_chunks',
+ 'write_chunks',
+ 'reply_chunks',
+ 'total_rdma_req',
+ 'total_rdma_rep',
+ 'pullup',
+ 'fixup',
+ 'hardway',
+ 'failed_marshal',
+ 'bad_reply'
+]
+
+Nfsv3ops = [
+ 'NULL',
+ 'GETATTR',
+ 'SETATTR',
+ 'LOOKUP',
+ 'ACCESS',
+ 'READLINK',
+ 'READ',
+ 'WRITE',
+ 'CREATE',
+ 'MKDIR',
+ 'SYMLINK',
+ 'MKNOD',
+ 'REMOVE',
+ 'RMDIR',
+ 'RENAME',
+ 'LINK',
+ 'READDIR',
+ 'READDIRPLUS',
+ 'FSSTAT',
+ 'FSINFO',
+ 'PATHCONF',
+ 'COMMIT'
+]
+
+Nfsv4ops = [
+ 'NULL',
+ 'READ',
+ 'WRITE',
+ 'COMMIT',
+ 'OPEN',
+ 'OPEN_CONFIRM',
+ 'OPEN_NOATTR',
+ 'OPEN_DOWNGRADE',
+ 'CLOSE',
+ 'SETATTR',
+ 'FSINFO',
+ 'RENEW',
+ 'SETCLIENTID',
+ 'SETCLIENTID_CONFIRM',
+ 'LOCK',
+ 'LOCKT',
+ 'LOCKU',
+ 'ACCESS',
+ 'GETATTR',
+ 'LOOKUP',
+ 'LOOKUP_ROOT',
+ 'REMOVE',
+ 'RENAME',
+ 'LINK',
+ 'SYMLINK',
+ 'CREATE',
+ 'PATHCONF',
+ 'STATFS',
+ 'READLINK',
+ 'READDIR',
+ 'SERVER_CAPS',
+ 'DELEGRETURN',
+ 'GETACL',
+ 'SETACL',
+ 'FS_LOCATIONS',
+ 'RELEASE_LOCKOWNER',
+ 'SECINFO',
+ 'FSID_PRESENT',
+ 'EXCHANGE_ID',
+ 'CREATE_SESSION',
+ 'DESTROY_SESSION',
+ 'SEQUENCE',
+ 'GET_LEASE_TIME',
+ 'RECLAIM_COMPLETE',
+ 'LAYOUTGET',
+ 'GETDEVICEINFO',
+ 'LAYOUTCOMMIT',
+ 'LAYOUTRETURN',
+ 'SECINFO_NO_NAME',
+ 'TEST_STATEID',
+ 'FREE_STATEID',
+ 'GETDEVICELIST',
+ 'BIND_CONN_TO_SESSION',
+ 'DESTROY_CLIENTID'
+]
+
class DeviceData:
"""DeviceData objects provide methods for parsing and displaying
data for a single mount grabbed from /proc/self/mountstats
--
1.9.3
Iterate over the newly added counter lists instead of having a ton of
assignment statements.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 92 +++++++++++-------------------------------
1 file changed, 23 insertions(+), 69 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index bf98813..a129d61 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -238,40 +238,18 @@ class DeviceData:
if self.__nfs_data['flavor'] == 6:
self.__nfs_data['pseudoflavor'] = int(keys[1].split('=')[1])
elif words[0] == 'events:':
- self.__nfs_data['inoderevalidates'] = int(words[1])
- self.__nfs_data['dentryrevalidates'] = int(words[2])
- self.__nfs_data['datainvalidates'] = int(words[3])
- self.__nfs_data['attrinvalidates'] = int(words[4])
- self.__nfs_data['vfsopen'] = int(words[5])
- self.__nfs_data['vfslookup'] = int(words[6])
- self.__nfs_data['vfspermission'] = int(words[7])
- self.__nfs_data['vfsupdatepage'] = int(words[8])
- self.__nfs_data['vfsreadpage'] = int(words[9])
- self.__nfs_data['vfsreadpages'] = int(words[10])
- self.__nfs_data['vfswritepage'] = int(words[11])
- self.__nfs_data['vfswritepages'] = int(words[12])
- self.__nfs_data['vfsreaddir'] = int(words[13])
- self.__nfs_data['vfssetattr'] = int(words[14])
- self.__nfs_data['vfsflush'] = int(words[15])
- self.__nfs_data['vfsfsync'] = int(words[16])
- self.__nfs_data['vfslock'] = int(words[17])
- self.__nfs_data['vfsrelease'] = int(words[18])
- self.__nfs_data['congestionwait'] = int(words[19])
- self.__nfs_data['setattrtrunc'] = int(words[20])
- self.__nfs_data['extendwrite'] = int(words[21])
- self.__nfs_data['sillyrenames'] = int(words[22])
- self.__nfs_data['shortreads'] = int(words[23])
- self.__nfs_data['shortwrites'] = int(words[24])
- self.__nfs_data['delay'] = int(words[25])
- self.__nfs_data['pnfsreads'] = int(words[26])
- self.__nfs_data['pnfswrites'] = int(words[27])
+ i = 1
+ for key in NfsEventCounters:
+ try:
+ self.__nfs_data[key] = int(words[i])
+ except IndexError as err:
+ self.__nfs_data[key] = 0
+ i += 1
elif words[0] == 'bytes:':
- self.__nfs_data['normalreadbytes'] = int(words[1])
- self.__nfs_data['normalwritebytes'] = int(words[2])
- self.__nfs_data['directreadbytes'] = int(words[3])
- self.__nfs_data['directwritebytes'] = int(words[4])
- self.__nfs_data['serverreadbytes'] = int(words[5])
- self.__nfs_data['serverwritebytes'] = int(words[6])
+ i = 1
+ for key in NfsByteCounters:
+ self.__nfs_data[key] = int(words[i])
+ i += 1
def __parse_rpc_line(self, words):
if words[0] == 'RPC':
@@ -280,44 +258,20 @@ class DeviceData:
elif words[0] == 'xprt:':
self.__rpc_data['protocol'] = words[1]
if words[1] == 'udp':
- self.__rpc_data['port'] = int(words[2])
- self.__rpc_data['bind_count'] = int(words[3])
- self.__rpc_data['rpcsends'] = int(words[4])
- self.__rpc_data['rpcreceives'] = int(words[5])
- self.__rpc_data['badxids'] = int(words[6])
- self.__rpc_data['inflightsends'] = int(words[7])
- self.__rpc_data['backlogutil'] = int(words[8])
+ i = 2
+ for key in XprtUdpCounters:
+ self.__rpc_data[key] = int(words[i])
+ i += 1
elif words[1] == 'tcp':
- self.__rpc_data['port'] = words[2]
- self.__rpc_data['bind_count'] = int(words[3])
- self.__rpc_data['connect_count'] = int(words[4])
- self.__rpc_data['connect_time'] = int(words[5])
- self.__rpc_data['idle_time'] = int(words[6])
- self.__rpc_data['rpcsends'] = int(words[7])
- self.__rpc_data['rpcreceives'] = int(words[8])
- self.__rpc_data['badxids'] = int(words[9])
- self.__rpc_data['inflightsends'] = int(words[10])
- self.__rpc_data['backlogutil'] = int(words[11])
+ i = 2
+ for key in XprtTcpCounters:
+ self.__rpc_data[key] = int(words[i])
+ i += 1
elif words[1] == 'rdma':
- self.__rpc_data['port'] = words[2]
- self.__rpc_data['bind_count'] = int(words[3])
- self.__rpc_data['connect_count'] = int(words[4])
- self.__rpc_data['connect_time'] = int(words[5])
- self.__rpc_data['idle_time'] = int(words[6])
- self.__rpc_data['rpcsends'] = int(words[7])
- self.__rpc_data['rpcreceives'] = int(words[8])
- self.__rpc_data['badxids'] = int(words[9])
- self.__rpc_data['backlogutil'] = int(words[10])
- self.__rpc_data['read_chunks'] = int(words[11])
- self.__rpc_data['write_chunks'] = int(words[12])
- self.__rpc_data['reply_chunks'] = int(words[13])
- self.__rpc_data['total_rdma_req'] = int(words[14])
- self.__rpc_data['total_rdma_rep'] = int(words[15])
- self.__rpc_data['pullup'] = int(words[16])
- self.__rpc_data['fixup'] = int(words[17])
- self.__rpc_data['hardway'] = int(words[18])
- self.__rpc_data['failed_marshal'] = int(words[19])
- self.__rpc_data['bad_reply'] = int(words[20])
+ i = 2
+ for key in XprtRdmaCounters:
+ self.__rpc_data[key] = int(words[i])
+ i += 1
elif words[0] == 'per-op':
self.__rpc_data['per-op'] = words
else:
--
1.9.3
Iterate over the newly added counters instead of using repeated
assignment statements. Also compute the difference of every counter
where it makes sense -- this will allow support for -S/--since to be
implemented in the future.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index a129d61..912d31a 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -415,7 +415,11 @@ class DeviceData:
def compare_iostats(self, old_stats):
"""Return the difference between two sets of stats
"""
+ if old_stats.__nfs_data['age'] > self.__nfs_data['age']:
+ return self
+
result = DeviceData()
+ protocol = self.__rpc_data['protocol']
# copy self into result
for key, value in self.__nfs_data.items():
@@ -430,12 +434,21 @@ class DeviceData:
for op in result.__rpc_data['ops']:
result.__rpc_data[op] = list(map(difference, self.__rpc_data[op], old_stats.__rpc_data[op]))
- # update the remaining keys we care about
- result.__rpc_data['rpcsends'] -= old_stats.__rpc_data['rpcsends']
- result.__rpc_data['backlogutil'] -= old_stats.__rpc_data['backlogutil']
- result.__nfs_data['serverreadbytes'] -= old_stats.__nfs_data['serverreadbytes']
- result.__nfs_data['serverwritebytes'] -= old_stats.__nfs_data['serverwritebytes']
-
+ # update the remaining keys
+ if protocol == 'udp':
+ for key in XprtUdpCounters:
+ result.__rpc_data[key] -= old_stats.__rpc_data[key]
+ elif protocol == 'tcp':
+ for key in XprtTcpCounters:
+ result.__rpc_data[key] -= old_stats.__rpc_data[key]
+ elif protocol == 'rdma':
+ for key in XprtRdmaCounters:
+ result.__rpc_data[key] -= old_stats.__rpc_data[key]
+ result.__nfs_data['age'] -= old_stats.__nfs_data['age']
+ for key in NfsEventCounters:
+ result.__nfs_data[key] -= old_stats.__nfs_data[key]
+ for key in NfsByteCounters:
+ result.__nfs_data[key] -= old_stats.__nfs_data[key]
return result
def display_iostats(self, sample_time):
--
1.9.3
Made mountstats, nfsstat, and iostat all subcommands (note that the
nfsstat function is still unimplemented here). If no sub-command is
given, then the mountstats sub-command will run by default (so any
scripts that run older versions of the mountstats command will still
work).
Also removed the --start and --end options since all they do is throw
exceptions.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 254 ++++++++++++++++++-----------------------
1 file changed, 110 insertions(+), 144 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 912d31a..b6f573d 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -25,6 +25,12 @@ MA 02110-1301 USA
import sys, os, time
from operator import itemgetter
+try:
+ import argparse
+except ImportError:
+ print('%s: Failed to import argparse - make sure argparse is installed!'
+ % sys.argv[0])
+ sys.exit(1)
Mountstats_version = '0.2'
@@ -533,66 +539,12 @@ def parse_stats_file(filename):
return ms_dict
-def print_mountstats_help(name):
- print('usage: %s [ options ] <mount point>' % name)
- print()
- print(' Version %s' % Mountstats_version)
- print()
- print(' Display NFS client per-mount statistics.')
- print()
- print(' --version display the version of this command')
- print(' --nfs display only the NFS statistics')
- print(' --rpc display only the RPC statistics')
- print(' --start sample and save statistics')
- print(' --end resample statistics and compare them with saved')
- print()
-
-def mountstats_command():
+def mountstats_command(args):
"""Mountstats command
"""
- mountpoints = []
- nfs_only = False
- rpc_only = False
-
- for arg in sys.argv:
- if arg in ['-h', '--help', 'help', 'usage']:
- print_mountstats_help(prog)
- return
-
- if arg in ['-v', '--version', 'version']:
- print('%s version %s' % (sys.argv[0], Mountstats_version))
- sys.exit(0)
-
- if arg in ['-n', '--nfs']:
- nfs_only = True
- continue
-
- if arg in ['-r', '--rpc']:
- rpc_only = True
- continue
-
- if arg in ['-s', '--start']:
- raise Exception('Sampling is not yet implemented')
-
- if arg in ['-e', '--end']:
- raise Exception('Sampling is not yet implemented')
-
- if arg == sys.argv[0]:
- continue
-
- mountpoints += [arg]
-
- if mountpoints == []:
- print_mountstats_help(prog)
- return
-
- if rpc_only == True and nfs_only == True:
- print_mountstats_help(prog)
- return
-
mountstats = parse_stats_file('/proc/self/mountstats')
- for mp in mountpoints:
+ for mp in args.mountpoints:
if mp not in mountstats:
print('Statistics for mount point %s not found' % mp)
continue
@@ -604,11 +556,11 @@ def mountstats_command():
print('Mount point %s exists but is not an NFS mount' % mp)
continue
- if nfs_only:
+ if args.nfs_only:
stats.display_nfs_options()
stats.display_nfs_events()
stats.display_nfs_bytes()
- elif rpc_only:
+ elif args.rpc_only:
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
else:
@@ -617,38 +569,8 @@ def mountstats_command():
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
-def print_nfsstat_help(name):
- print('usage: %s [ options ]' % name)
- print()
- print(' Version %s' % Mountstats_version)
- print()
- print(' nfsstat-like program that uses NFS client per-mount statistics.')
- print()
-
-def nfsstat_command():
- print_nfsstat_help(prog)
-
-def print_iostat_help(name):
- print('usage: %s [ <interval> [ <count> ] ] [ <mount point> ] ' % name)
- print()
- print(' Version %s' % Mountstats_version)
- print()
- print(' iostat-like program to display NFS client per-mount statistics.')
- print()
- print(' The <interval> parameter specifies the amount of time in seconds between')
- print(' each report. The first report contains statistics for the time since each')
- print(' file system was mounted. Each subsequent report contains statistics')
- print(' collected during the interval since the previous report.')
- print()
- print(' If the <count> parameter is specified, the value of <count> determines the')
- print(' number of reports generated at <interval> seconds apart. If the interval')
- print(' parameter is specified without the <count> parameter, the command generates')
- print(' reports continuously.')
- print()
- print(' If one or more <mount point> names are specified, statistics for only these')
- print(' mount points will be displayed. Otherwise, all NFS mount points on the')
- print(' client are listed.')
- print()
+def nfsstat_command(args):
+ return
def print_iostat_summary(old, new, devices, time):
for device in devices:
@@ -662,42 +584,11 @@ def print_iostat_summary(old, new, devices, time):
diff_stats = stats.compare_iostats(old_stats)
diff_stats.display_iostats(time)
-def iostat_command():
+def iostat_command(args):
"""iostat-like command for NFS mount points
"""
mountstats = parse_stats_file('/proc/self/mountstats')
- devices = []
- interval_seen = False
- count_seen = False
-
- for arg in sys.argv:
- if arg in ['-h', '--help', 'help', 'usage']:
- print_iostat_help(prog)
- return
-
- if arg in ['-v', '--version', 'version']:
- print('%s version %s' % (sys.argv[0], Mountstats_version))
- return
-
- if arg == sys.argv[0]:
- continue
-
- if arg in mountstats:
- devices += [arg]
- elif not interval_seen:
- interval = int(arg)
- if interval > 0:
- interval_seen = True
- else:
- print('Illegal <interval> value')
- return
- elif not count_seen:
- count = int(arg)
- if count > 0:
- count_seen = True
- else:
- print('Illegal <count> value')
- return
+ devices = args.mountpoints
# make certain devices contains only NFS mount points
if len(devices) > 0:
@@ -721,44 +612,119 @@ def iostat_command():
old_mountstats = None
sample_time = 0
- if not interval_seen:
+ if args.interval is None:
print_iostat_summary(old_mountstats, mountstats, devices, sample_time)
return
- if count_seen:
+ if args.count is not None:
+ count = args.count
while count != 0:
print_iostat_summary(old_mountstats, mountstats, devices, sample_time)
old_mountstats = mountstats
- time.sleep(interval)
- sample_time = interval
+ time.sleep(args.interval)
+ sample_time = args.interval
mountstats = parse_stats_file('/proc/self/mountstats')
count -= 1
else:
while True:
print_iostat_summary(old_mountstats, mountstats, devices, sample_time)
old_mountstats = mountstats
- time.sleep(interval)
- sample_time = interval
+ time.sleep(args.interval)
+ sample_time = args.interval
mountstats = parse_stats_file('/proc/self/mountstats')
-#
-# Main
-#
-prog = os.path.basename(sys.argv[0])
+class ICMAction(argparse.Action):
+ """Custom action to deal with interval, count, and mountpoints.
+ """
+ def __call__(self, parser, namespace, values, option_string=None):
+ if namespace.mountpoints is None:
+ namespace.mountpoints = []
+ if values is None:
+ return
+ elif (type(values) == type([])):
+ for value in values:
+ self._handle_one(namespace, value)
+ else:
+ self._handle_one(namespace, values)
+
+ def _handle_one(self, namespace, value):
+ try:
+ intval = int(value)
+ self._handle_int(namespace, intval)
+ except ValueError:
+ namespace.mountpoints.append(value)
+
+ def _handle_int(self, namespace, value):
+ if namespace.interval is None:
+ namespace.interval = value
+ elif namespace.count is None:
+ namespace.count = value
+ else:
+ raise argparse.ArgumentError(self, "too many integer arguments")
+
+def main():
+ parser = argparse.ArgumentParser(epilog='For specific sub-command help, '
+ 'run \'mountstats SUB-COMMAND -h|--help\'')
+ subparsers = parser.add_subparsers(help='sub-command help')
+
+ common_parser = argparse.ArgumentParser(add_help=False)
+ common_parser.add_argument('-v', '--version', action='version',
+ version='mountstats ' + Mountstats_version)
+
+ mountstats_parser = subparsers.add_parser('mountstats',
+ parents=[common_parser],
+ help='Display a combination of per-op RPC statistics, NFS event counts, and NFS byte counts. '
+ 'This is the default sub-command if no sub-command is given.')
+ group = mountstats_parser.add_mutually_exclusive_group()
+ group.add_argument('-n', '--nfs', action='store_true', dest='nfs_only',
+ help='Display only the NFS statistics')
+ group.add_argument('-r', '--rpc', action='store_true', dest='rpc_only',
+ help='Display only the RPC statistics')
+ # The mountpoints argument cannot be moved into the common_parser because
+ # it will screw up the parsing of the iostat arguments (interval and count)
+ mountstats_parser.add_argument('mountpoints', nargs='+', metavar='mountpoint',
+ help='Display statistics for this mountpoint. More than one may be specified.')
+ mountstats_parser.set_defaults(func=mountstats_command)
+
+ nfsstat_parser = subparsers.add_parser('nfsstat',
+ parents=[common_parser],
+ help='Display nfsstat-like statistics.')
+ nfsstat_parser.set_defaults(func=nfsstat_command)
+
+ iostat_parser = subparsers.add_parser('iostat',
+ parents=[common_parser],
+ help='Display iostat-like statistics.')
+ iostat_parser.add_argument('interval', nargs='?', action=ICMAction,
+ help='Number of seconds between reports. If absent, only one report will '
+ 'be generated.')
+ iostat_parser.add_argument('count', nargs='?', action=ICMAction,
+ help='Number of reports generated at <interval> seconds apart. If absent, '
+ 'reports will be generated continuously.')
+ # The mountpoints argument cannot be moved into the common_parser because
+ # it will screw up the parsing of the iostat arguments (interval and count)
+ iostat_parser.add_argument('mountpoints', nargs='*', action=ICMAction, metavar='mountpoint',
+ help='Display statsistics for this mountpoint. More than one may be specified. '
+ 'If absent, statistics for all NFS mountpoints will be generated.')
+ iostat_parser.set_defaults(func=iostat_command)
+
+ args = parser.parse_args()
+ return args.func(args)
try:
- if prog == 'mountstats':
- mountstats_command()
- elif prog == 'ms-nfsstat':
- nfsstat_command()
- elif prog == 'ms-iostat':
- iostat_command()
- sys.stdout.close()
- sys.stderr.close()
-except KeyboardInterrupt:
- print('Caught ^C... exiting')
+ if __name__ == '__main__':
+ # Run the mounstats sub-command if no sub-command (or the help flag)
+ # is given. If the argparse module ever gets support for optional
+ # (default) sub-commands, then this can be changed.
+ if len(sys.argv) == 1:
+ sys.argv.insert(1, 'mountstats')
+ elif sys.argv[1] not in ['-h', '--help', 'mountstats', 'iostat', 'nfsstat']:
+ sys.argv.insert(1, 'mountstats')
+ res = main()
+ sys.stdout.close()
+ sys.stderr.close()
+ sys.exit(res)
+except (SystemExit, KeyboardInterrupt, RuntimeError):
sys.exit(1)
except IOError:
pass
-sys.exit(0)
--
1.9.3
Changes mostly lifted straight from nfs-iostat.py.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 104 ++++++++++++++++++++++++-----------------
1 file changed, 61 insertions(+), 43 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index b6f573d..c8cb718 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -457,60 +457,78 @@ class DeviceData:
result.__nfs_data[key] -= old_stats.__nfs_data[key]
return result
+ def __print_rpc_op_stats(self, op, sample_time):
+ """Print generic stats for one RPC op
+ """
+ if op not in self.__rpc_data:
+ return
+
+ rpc_stats = self.__rpc_data[op]
+ ops = float(rpc_stats[0])
+ retrans = float(rpc_stats[1] - rpc_stats[0])
+ kilobytes = float(rpc_stats[3] + rpc_stats[4]) / 1024
+ rtt = float(rpc_stats[6])
+ exe = float(rpc_stats[7])
+
+ # prevent floating point exceptions
+ if ops != 0:
+ kb_per_op = kilobytes / ops
+ retrans_percent = (retrans * 100) / ops
+ rtt_per_op = rtt / ops
+ exe_per_op = exe / ops
+ else:
+ kb_per_op = 0.0
+ retrans_percent = 0.0
+ rtt_per_op = 0.0
+ exe_per_op = 0.0
+
+ op += ':'
+ print(format(op.lower(), '<16s'), end='')
+ print(format('ops/s', '>8s'), end='')
+ print(format('kB/s', '>16s'), end='')
+ print(format('kB/op', '>16s'), end='')
+ print(format('retrans', '>16s'), end='')
+ print(format('avg RTT (ms)', '>16s'), end='')
+ print(format('avg exe (ms)', '>16s'))
+
+ print(format((ops / sample_time), '>24.3f'), end='')
+ print(format((kilobytes / sample_time), '>16.3f'), end='')
+ print(format(kb_per_op, '>16.3f'), end='')
+ retransmits = '{0:>10.0f} ({1:>3.1f}%)'.format(retrans, retrans_percent).strip()
+ print(format(retransmits, '>16'), end='')
+ print(format(rtt_per_op, '>16.3f'), end='')
+ print(format(exe_per_op, '>16.3f'))
+
def display_iostats(self, sample_time):
"""Display NFS and RPC stats in an iostat-like way
"""
sends = float(self.__rpc_data['rpcsends'])
if sample_time == 0:
sample_time = float(self.__nfs_data['age'])
+ # sample_time could still be zero if the export was just mounted.
+ # Set it to 1 to avoid divide by zero errors in this case since we'll
+ # likely still have relevant mount statistics to show.
+ #
+ if sample_time == 0:
+ sample_time = 1;
+ if sends != 0:
+ backlog = (float(self.__rpc_data['backlogutil']) / sends) / sample_time
+ else:
+ backlog = 0.0
print()
print('%s mounted on %s:' % \
(self.__nfs_data['export'], self.__nfs_data['mountpoint']))
+ print()
- print('\top/s\trpc bklog')
- print('\t%.2f' % (sends / sample_time), end=' ')
- if sends != 0:
- print('\t%.2f' % \
- ((float(self.__rpc_data['backlogutil']) / sends) / sample_time))
- else:
- print('\t0.00')
-
- # reads: ops/s, kB/s, avg rtt, and avg exe
- # XXX: include avg xfer size and retransmits?
- read_rpc_stats = self.__rpc_data['READ']
- ops = float(read_rpc_stats[0])
- kilobytes = float(self.__nfs_data['serverreadbytes']) / 1024
- rtt = float(read_rpc_stats[6])
- exe = float(read_rpc_stats[7])
-
- print('\treads:\tops/s\t\tkB/s\t\tavg RTT (ms)\tavg exe (ms)')
- print('\t\t%.2f' % (ops / sample_time), end=' ')
- print('\t\t%.2f' % (kilobytes / sample_time), end=' ')
- if ops != 0:
- print('\t\t%.2f' % (rtt / ops), end=' ')
- print('\t\t%.2f' % (exe / ops))
- else:
- print('\t\t0.00', end=' ')
- print('\t\t0.00')
-
- # writes: ops/s, kB/s, avg rtt, and avg exe
- # XXX: include avg xfer size and retransmits?
- write_rpc_stats = self.__rpc_data['WRITE']
- ops = float(write_rpc_stats[0])
- kilobytes = float(self.__nfs_data['serverwritebytes']) / 1024
- rtt = float(write_rpc_stats[6])
- exe = float(write_rpc_stats[7])
-
- print('\twrites:\tops/s\t\tkB/s\t\tavg RTT (ms)\tavg exe (ms)')
- print('\t\t%.2f' % (ops / sample_time), end=' ')
- print('\t\t%.2f' % (kilobytes / sample_time), end=' ')
- if ops != 0:
- print('\t\t%.2f' % (rtt / ops), end=' ')
- print('\t\t%.2f' % (exe / ops))
- else:
- print('\t\t0.00', end=' ')
- print('\t\t0.00')
+ print(format('ops/s', '>16') + format('rpc bklog', '>16'))
+ print(format((sends / sample_time), '>16.3f'), end='')
+ print(format(backlog, '>16.3f'))
+ print()
+
+ self.__print_rpc_op_stats('READ', sample_time)
+ self.__print_rpc_op_stats('WRITE', sample_time)
+ sys.stdout.flush()
def parse_stats_file(filename):
"""pop the contents of a mountstats file into a dictionary,
--
1.9.3
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index c8cb718..51b4c76 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -594,7 +594,7 @@ def print_iostat_summary(old, new, devices, time):
for device in devices:
stats = DeviceData()
stats.parse_stats(new[device])
- if not old:
+ if not old or device not in old:
stats.display_iostats(time)
else:
old_stats = DeviceData()
--
1.9.3
Add support for the -f/--file option to allow parsing of data from an
arbitrary file instead of /proc/self/mountstats.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 31 +++++++++++++++++++++----------
1 file changed, 21 insertions(+), 10 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 51b4c76..032d8d6 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -530,7 +530,7 @@ class DeviceData:
self.__print_rpc_op_stats('WRITE', sample_time)
sys.stdout.flush()
-def parse_stats_file(filename):
+def parse_stats_file(f):
"""pop the contents of a mountstats file into a dictionary,
keyed by mount point. each value object is a list of the
lines in the mountstats file corresponding to the mount
@@ -539,7 +539,7 @@ def parse_stats_file(filename):
ms_dict = dict()
key = ''
- f = open(filename)
+ f.seek(0)
for line in f.readlines():
words = line.split()
if len(words) == 0:
@@ -553,14 +553,13 @@ def parse_stats_file(filename):
else:
new += [ line.strip() ]
ms_dict[key] = new
- f.close
return ms_dict
def mountstats_command(args):
"""Mountstats command
"""
- mountstats = parse_stats_file('/proc/self/mountstats')
+ mountstats = parse_stats_file(args.infile)
for mp in args.mountpoints:
if mp not in mountstats:
@@ -587,6 +586,8 @@ def mountstats_command(args):
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
+ args.infile.close()
+
def nfsstat_command(args):
return
@@ -605,7 +606,7 @@ def print_iostat_summary(old, new, devices, time):
def iostat_command(args):
"""iostat-like command for NFS mount points
"""
- mountstats = parse_stats_file('/proc/self/mountstats')
+ mountstats = parse_stats_file(args.infile)
devices = args.mountpoints
# make certain devices contains only NFS mount points
@@ -613,9 +614,12 @@ def iostat_command(args):
check = []
for device in devices:
stats = DeviceData()
- stats.parse_stats(mountstats[device])
- if stats.is_nfs_mountpoint():
- check += [device]
+ try:
+ stats.parse_stats(mountstats[device])
+ if stats.is_nfs_mountpoint():
+ check += [device]
+ except KeyError:
+ continue
devices = check
else:
for device, descr in mountstats.items():
@@ -641,7 +645,7 @@ def iostat_command(args):
old_mountstats = mountstats
time.sleep(args.interval)
sample_time = args.interval
- mountstats = parse_stats_file('/proc/self/mountstats')
+ mountstats = parse_stats_file(args.infile)
count -= 1
else:
while True:
@@ -649,7 +653,9 @@ def iostat_command(args):
old_mountstats = mountstats
time.sleep(args.interval)
sample_time = args.interval
- mountstats = parse_stats_file('/proc/self/mountstats')
+ mountstats = parse_stats_file(args.infile)
+
+ args.infile.close()
class ICMAction(argparse.Action):
"""Custom action to deal with interval, count, and mountpoints.
@@ -668,6 +674,8 @@ class ICMAction(argparse.Action):
def _handle_one(self, namespace, value):
try:
intval = int(value)
+ if namespace.infile.name != '/proc/self/mountstats':
+ raise argparse.ArgumentError(self, "not allowed with argument -f/--file")
self._handle_int(namespace, intval)
except ValueError:
namespace.mountpoints.append(value)
@@ -688,6 +696,9 @@ def main():
common_parser = argparse.ArgumentParser(add_help=False)
common_parser.add_argument('-v', '--version', action='version',
version='mountstats ' + Mountstats_version)
+ common_parser.add_argument('-f', '--file', default=open('/proc/self/mountstats', 'r'),
+ type=argparse.FileType('r'), dest='infile',
+ help='Read stats from %(dest)s instead of /proc/self/mountstats')
mountstats_parser = subparsers.add_parser('mountstats',
parents=[common_parser],
--
1.9.3
Add support for -S/--since to display the delta between a previous
copy of /proc/self/mountstats and the current /proc/self/mountstats
file. Can be combined with the -f option to show the statistics
between two different points in time.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 51 +++++++++++++++++++++++++++++++-----------
1 file changed, 38 insertions(+), 13 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 032d8d6..18d9514 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -556,11 +556,28 @@ def parse_stats_file(f):
return ms_dict
+def print_mountstats(stats, nfs_only, rpc_only):
+ if nfs_only:
+ stats.display_nfs_options()
+ stats.display_nfs_events()
+ stats.display_nfs_bytes()
+ elif rpc_only:
+ stats.display_rpc_generic_stats()
+ stats.display_rpc_op_stats()
+ else:
+ stats.display_nfs_options()
+ stats.display_nfs_bytes()
+ stats.display_rpc_generic_stats()
+ stats.display_rpc_op_stats()
+
def mountstats_command(args):
"""Mountstats command
"""
mountstats = parse_stats_file(args.infile)
+ if args.since:
+ old_mountstats = parse_stats_file(args.since)
+
for mp in args.mountpoints:
if mp not in mountstats:
print('Statistics for mount point %s not found' % mp)
@@ -573,20 +590,19 @@ def mountstats_command(args):
print('Mount point %s exists but is not an NFS mount' % mp)
continue
- if args.nfs_only:
- stats.display_nfs_options()
- stats.display_nfs_events()
- stats.display_nfs_bytes()
- elif args.rpc_only:
- stats.display_rpc_generic_stats()
- stats.display_rpc_op_stats()
+ if not args.since:
+ print_mountstats(stats, args.nfs_only, args.rpc_only)
+ elif args.since and mp not in old_mountstats:
+ print_mountstats(stats, args.nfs_only, args.rpc_only)
else:
- stats.display_nfs_options()
- stats.display_nfs_bytes()
- stats.display_rpc_generic_stats()
- stats.display_rpc_op_stats()
+ old_stats = DeviceData()
+ old_stats.parse_stats(old_mountstats[mp])
+ diff_stats = stats.compare_iostats(old_stats)
+ print_mountstats(diff_stats, args.nfs_only, args.rpc_only)
args.infile.close()
+ if args.since:
+ args.since.close()
def nfsstat_command(args):
return
@@ -609,6 +625,11 @@ def iostat_command(args):
mountstats = parse_stats_file(args.infile)
devices = args.mountpoints
+ if args.since:
+ old_mountstats = parse_stats_file(args.since)
+ else:
+ old_mountstats = None
+
# make certain devices contains only NFS mount points
if len(devices) > 0:
check = []
@@ -631,7 +652,6 @@ def iostat_command(args):
print('No NFS mount points were found')
return
- old_mountstats = None
sample_time = 0
if args.interval is None:
@@ -656,6 +676,8 @@ def iostat_command(args):
mountstats = parse_stats_file(args.infile)
args.infile.close()
+ if args.since:
+ args.since.close()
class ICMAction(argparse.Action):
"""Custom action to deal with interval, count, and mountpoints.
@@ -675,7 +697,7 @@ class ICMAction(argparse.Action):
try:
intval = int(value)
if namespace.infile.name != '/proc/self/mountstats':
- raise argparse.ArgumentError(self, "not allowed with argument -f/--file")
+ raise argparse.ArgumentError(self, "not allowed with argument -f/--file or -S/--since")
self._handle_int(namespace, intval)
except ValueError:
namespace.mountpoints.append(value)
@@ -699,6 +721,9 @@ def main():
common_parser.add_argument('-f', '--file', default=open('/proc/self/mountstats', 'r'),
type=argparse.FileType('r'), dest='infile',
help='Read stats from %(dest)s instead of /proc/self/mountstats')
+ common_parser.add_argument('-S', '--since', type=argparse.FileType('r'),
+ metavar='SINCEFILE',
+ help='Show difference between current stats and those in SINCEFILE')
mountstats_parser = subparsers.add_parser('mountstats',
parents=[common_parser],
--
1.9.3
If __parse_nfs_line is is called on a line that has 'fstype nfsd', it'll
raise an IndexError trying to parse a nonexistent statvers entry.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 18d9514..769c458 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -221,13 +221,13 @@ class DeviceData:
self.__nfs_data['export'] = words[1]
self.__nfs_data['mountpoint'] = words[4]
self.__nfs_data['fstype'] = words[7]
- if words[7].find('nfs') != -1:
+ if words[7].find('nfs') != -1 and words[7] != 'nfsd':
self.__nfs_data['statvers'] = words[8]
elif 'nfs' in words or 'nfs4' in words:
self.__nfs_data['export'] = words[0]
self.__nfs_data['mountpoint'] = words[3]
self.__nfs_data['fstype'] = words[6]
- if words[6].find('nfs') != -1:
+ if words[6].find('nfs') != -1 and words[6] != 'nfsd':
self.__nfs_data['statvers'] = words[7]
elif words[0] == 'age:':
self.__nfs_data['age'] = int(words[1])
--
1.9.3
Allow the mountstats command to take a variable number of mountpoints
(including none, in which case it will print stats for all NFS
mountpoints it finds).
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 48 ++++++++++++++++++++++++++++++------------
1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 769c458..4a6dc44 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -311,11 +311,14 @@ class DeviceData:
return True
return False
+ def display_stats_header(self):
+ print('Stats for %s mounted on %s:' % \
+ (self.__nfs_data['export'], self.__nfs_data['mountpoint']))
+
def display_nfs_options(self):
"""Pretty-print the NFS options
"""
- print('Stats for %s mounted on %s:' % \
- (self.__nfs_data['export'], self.__nfs_data['mountpoint']))
+ self.display_stats_header()
print(' NFS mount options: %s' % ','.join(self.__nfs_data['mountoptions']))
print(' NFS server capabilities: %s' % ','.join(self.__nfs_data['servercapabilities']))
@@ -562,6 +565,7 @@ def print_mountstats(stats, nfs_only, rpc_only):
stats.display_nfs_events()
stats.display_nfs_bytes()
elif rpc_only:
+ stats.display_stats_header()
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
else:
@@ -569,27 +573,42 @@ def print_mountstats(stats, nfs_only, rpc_only):
stats.display_nfs_bytes()
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
+ print()
def mountstats_command(args):
"""Mountstats command
"""
mountstats = parse_stats_file(args.infile)
+ mountpoints = args.mountpoints
+
+ # make certain devices contains only NFS mount points
+ if len(mountpoints) > 0:
+ check = []
+ for device in mountpoints:
+ stats = DeviceData()
+ try:
+ stats.parse_stats(mountstats[device])
+ if stats.is_nfs_mountpoint():
+ check += [device]
+ except KeyError:
+ continue
+ mountpoints = check
+ else:
+ for device, descr in mountstats.items():
+ stats = DeviceData()
+ stats.parse_stats(descr)
+ if stats.is_nfs_mountpoint():
+ mountpoints += [device]
+ if len(mountpoints) == 0:
+ print('No NFS mount points were found')
+ return
if args.since:
old_mountstats = parse_stats_file(args.since)
- for mp in args.mountpoints:
- if mp not in mountstats:
- print('Statistics for mount point %s not found' % mp)
- continue
-
+ for mp in mountpoints:
stats = DeviceData()
stats.parse_stats(mountstats[mp])
-
- if not stats.is_nfs_mountpoint():
- print('Mount point %s exists but is not an NFS mount' % mp)
- continue
-
if not args.since:
print_mountstats(stats, args.nfs_only, args.rpc_only)
elif args.since and mp not in old_mountstats:
@@ -736,8 +755,9 @@ def main():
help='Display only the RPC statistics')
# The mountpoints argument cannot be moved into the common_parser because
# it will screw up the parsing of the iostat arguments (interval and count)
- mountstats_parser.add_argument('mountpoints', nargs='+', metavar='mountpoint',
- help='Display statistics for this mountpoint. More than one may be specified.')
+ mountstats_parser.add_argument('mountpoints', nargs='*', metavar='mountpoint',
+ help='Display statistics for this mountpoint. More than one may be specified. '
+ 'If absent, statistics for all NFS mountpoints will be generated.')
mountstats_parser.set_defaults(func=mountstats_command)
nfsstat_parser = subparsers.add_parser('nfsstat',
--
1.9.3
This option displays the mountstats in raw format (i.e. in the same
format as /proc/self/mountstats). It's intended to be used mainly with
the -S/--since option.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 47 ++++++++++++++++++++++++++++++++++++++----
1 file changed, 43 insertions(+), 4 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 4a6dc44..194d246 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -311,6 +311,41 @@ class DeviceData:
return True
return False
+ def display_raw_stats(self):
+ """Prints out stats in the same format as /proc/self/mountstats
+ """
+ print('device %s mounted on %s with fstype %s %s' % \
+ (self.__nfs_data['export'], self.__nfs_data['mountpoint'], \
+ self.__nfs_data['fstype'], self.__nfs_data['statvers']))
+ print('\topts:\t%s' % ','.join(self.__nfs_data['mountoptions']))
+ print('\tage:\t%d' % self.__nfs_data['age'])
+ print('\tcaps:\t%s' % ','.join(self.__nfs_data['servercapabilities']))
+ print('\tsec:\tflavor=%d,pseudoflavor=%d' % (self.__nfs_data['flavor'], \
+ self.__nfs_data['pseudoflavor']))
+ print('\tevents:\t%s' % " ".join([str(self.__nfs_data[key]) for key in NfsEventCounters]))
+ print('\tbytes:\t%s' % " ".join([str(self.__nfs_data[key]) for key in NfsByteCounters]))
+ print('\tRPC iostats version: %1.1f p/v: %s (nfs)' % (self.__rpc_data['statsvers'], \
+ self.__rpc_data['programversion']))
+ if self.__rpc_data['protocol'] == 'udp':
+ print('\txprt:\tudp %s' % " ".join([str(self.__rpc_data[key]) for key in XprtUdpCounters]))
+ elif self.__rpc_data['protocol'] == 'tcp':
+ print('\txprt:\ttcp %s' % " ".join([str(self.__rpc_data[key]) for key in XprtTcpCounters]))
+ elif self.__rpc_data['protocol'] == 'rdma':
+ print('\txprt:\trdma %s' % " ".join([str(self.__rpc_data[key]) for key in XprtRdmaCounters]))
+ else:
+ raise Exception('Unknown RPC transport protocol %s' % self.__rpc_data['protocol'])
+ print('\tper-op statistics')
+ prog, vers = self.__rpc_data['programversion'].split('/')
+ if vers == '3':
+ for op in Nfsv3ops:
+ print('\t%12s: %s' % (op, " ".join(str(x) for x in self.__rpc_data[op])))
+ elif vers == '4':
+ for op in Nfsv4ops:
+ print('\t%12s: %s' % (op, " ".join(str(x) for x in self.__rpc_data[op])))
+ else:
+ print('\tnot implemented for version %d' % vers)
+ print()
+
def display_stats_header(self):
print('Stats for %s mounted on %s:' % \
(self.__nfs_data['export'], self.__nfs_data['mountpoint']))
@@ -559,7 +594,7 @@ def parse_stats_file(f):
return ms_dict
-def print_mountstats(stats, nfs_only, rpc_only):
+def print_mountstats(stats, nfs_only, rpc_only, raw):
if nfs_only:
stats.display_nfs_options()
stats.display_nfs_events()
@@ -568,6 +603,8 @@ def print_mountstats(stats, nfs_only, rpc_only):
stats.display_stats_header()
stats.display_rpc_generic_stats()
stats.display_rpc_op_stats()
+ elif raw:
+ stats.display_raw_stats()
else:
stats.display_nfs_options()
stats.display_nfs_bytes()
@@ -610,14 +647,14 @@ def mountstats_command(args):
stats = DeviceData()
stats.parse_stats(mountstats[mp])
if not args.since:
- print_mountstats(stats, args.nfs_only, args.rpc_only)
+ print_mountstats(stats, args.nfs_only, args.rpc_only, args.raw)
elif args.since and mp not in old_mountstats:
- print_mountstats(stats, args.nfs_only, args.rpc_only)
+ print_mountstats(stats, args.nfs_only, args.rpc_only, args.raw)
else:
old_stats = DeviceData()
old_stats.parse_stats(old_mountstats[mp])
diff_stats = stats.compare_iostats(old_stats)
- print_mountstats(diff_stats, args.nfs_only, args.rpc_only)
+ print_mountstats(diff_stats, args.nfs_only, args.rpc_only, args.raw)
args.infile.close()
if args.since:
@@ -753,6 +790,8 @@ def main():
help='Display only the NFS statistics')
group.add_argument('-r', '--rpc', action='store_true', dest='rpc_only',
help='Display only the RPC statistics')
+ group.add_argument('-R', '--raw', action='store_true',
+ help='Display only the raw statistics')
# The mountpoints argument cannot be moved into the common_parser because
# it will screw up the parsing of the iostat arguments (interval and count)
mountstats_parser.add_argument('mountpoints', nargs='*', metavar='mountpoint',
--
1.9.3
Displays nfssstat-like statistics (client statistics only).
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.py | 156 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 153 insertions(+), 3 deletions(-)
diff --git a/tools/mountstats/mountstats.py b/tools/mountstats/mountstats.py
index 194d246..fd73feb 100644
--- a/tools/mountstats/mountstats.py
+++ b/tools/mountstats/mountstats.py
@@ -24,7 +24,7 @@ MA 02110-1301 USA
"""
import sys, os, time
-from operator import itemgetter
+from operator import itemgetter, add
try:
import argparse
except ImportError:
@@ -32,7 +32,7 @@ except ImportError:
% sys.argv[0])
sys.exit(1)
-Mountstats_version = '0.2'
+Mountstats_version = '0.3'
def difference(x, y):
"""Used for a map() function
@@ -311,6 +311,11 @@ class DeviceData:
return True
return False
+ def nfs_version(self):
+ if self.is_nfs_mountpoint():
+ prog, vers = self.__rpc_data['programversion'].split('/')
+ return int(vers)
+
def display_raw_stats(self):
"""Prints out stats in the same format as /proc/self/mountstats
"""
@@ -456,6 +461,50 @@ class DeviceData:
print('\ttotal execute time: %f (milliseconds)' % \
(float(stats[8]) / count))
+ def client_rpc_stats(self):
+ """Tally high-level rpc stats for the nfsstat command
+ """
+ sends = 0
+ trans = 0
+ authrefrsh = 0
+ for op in self.__rpc_data['ops']:
+ sends += self.__rpc_data[op][0]
+ trans += self.__rpc_data[op][1]
+ retrans = trans - sends
+ # authrefresh stats don't actually get captured in
+ # /proc/self/mountstats, so we fudge it here
+ authrefrsh = sends
+ return (sends, trans, authrefrsh)
+
+ def display_nfsstat_stats(self):
+ """Pretty-print nfsstat-style stats
+ """
+ sends = 0
+ for op in self.__rpc_data['ops']:
+ sends += self.__rpc_data[op][0]
+ if sends == 0:
+ return
+ print()
+ vers = self.nfs_version()
+ print('Client nfs v%d' % vers)
+ info = []
+ for op in self.__rpc_data['ops']:
+ print('%-13s' % str.lower(op)[:12], end='')
+ count = self.__rpc_data[op][0]
+ pct = (count * 100) / sends
+ info.append((count, pct))
+ if (self.__rpc_data['ops'].index(op) + 1) % 6 == 0:
+ print()
+ for (count, pct) in info:
+ print('%-8u%3u%% ' % (count, pct), end='')
+ print()
+ info = []
+ print()
+ if len(info) > 0:
+ for (count, pct) in info:
+ print('%-8u%3u%% ' % (count, pct), end='')
+ print()
+
def compare_iostats(self, old_stats):
"""Return the difference between two sets of stats
"""
@@ -495,6 +544,27 @@ class DeviceData:
result.__nfs_data[key] -= old_stats.__nfs_data[key]
return result
+ def setup_accumulator(self, ops):
+ """Initialize a DeviceData instance to tally stats for all mountpoints
+ with the same major version. This is for the nfsstat command.
+ """
+ if ops == Nfsv3ops:
+ self.__rpc_data['programversion'] = '100003/3'
+ self.__nfs_data['fstype'] = 'nfs'
+ elif ops == Nfsv4ops:
+ self.__rpc_data['programversion'] = '100003/4'
+ self.__nfs_data['fstype'] = 'nfs4'
+ self.__rpc_data['ops'] = ops
+ for op in ops:
+ self.__rpc_data[op] = [0 for i in range(8)]
+
+ def accumulate_iostats(self, new_stats):
+ """Accumulate counters from all RPC op buckets in new_stats. This is
+ for the nfsstat command.
+ """
+ for op in new_stats.__rpc_data['ops']:
+ self.__rpc_data[op] = list(map(add, self.__rpc_data[op], new_stats.__rpc_data[op]))
+
def __print_rpc_op_stats(self, op, sample_time):
"""Print generic stats for one RPC op
"""
@@ -661,7 +731,78 @@ def mountstats_command(args):
args.since.close()
def nfsstat_command(args):
- return
+ """nfsstat-like command for NFS mount points
+ """
+ mountstats = parse_stats_file(args.infile)
+ mountpoints = args.mountpoints
+ v3stats = DeviceData()
+ v3stats.setup_accumulator(Nfsv3ops)
+ v4stats = DeviceData()
+ v4stats.setup_accumulator(Nfsv4ops)
+
+ # ensure stats get printed if neither v3 nor v4 was specified
+ if args.show_v3 or args.show_v4:
+ show_both = False
+ else:
+ show_both = True
+
+ # make certain devices contains only NFS mount points
+ if len(mountpoints) > 0:
+ check = []
+ for device in mountpoints:
+ stats = DeviceData()
+ try:
+ stats.parse_stats(mountstats[device])
+ if stats.is_nfs_mountpoint():
+ check += [device]
+ except KeyError:
+ continue
+ mountpoints = check
+ else:
+ for device, descr in mountstats.items():
+ stats = DeviceData()
+ stats.parse_stats(descr)
+ if stats.is_nfs_mountpoint():
+ mountpoints += [device]
+ if len(mountpoints) == 0:
+ print('No NFS mount points were found')
+ return
+
+ if args.since:
+ old_mountstats = parse_stats_file(args.since)
+
+ for mp in mountpoints:
+ stats = DeviceData()
+ stats.parse_stats(mountstats[mp])
+ vers = stats.nfs_version()
+
+ if not args.since:
+ acc_stats = stats
+ elif args.since and mp not in old_mountstats:
+ acc_stats = stats
+ else:
+ old_stats = DeviceData()
+ old_stats.parse_stats(old_mountstats[mp])
+ acc_stats = stats.compare_iostats(old_stats)
+
+ if vers == 3 and (show_both or args.show_v3):
+ v3stats.accumulate_iostats(acc_stats)
+ elif vers == 4 and (show_both or args.show_v4):
+ v4stats.accumulate_iostats(acc_stats)
+
+ sends, retrans, authrefrsh = map(add, v3stats.client_rpc_stats(), v4stats.client_rpc_stats())
+ print('Client rpc stats:')
+ print('calls retrans authrefrsh')
+ print('%-11u%-11u%-11u' % (sends, retrans, authrefrsh))
+
+ if show_both or args.show_v3:
+ v3stats.display_nfsstat_stats()
+ if show_both or args.show_v4:
+ v4stats.display_nfsstat_stats()
+
+ args.infile.close()
+ if args.since:
+ args.since.close()
def print_iostat_summary(old, new, devices, time):
for device in devices:
@@ -802,6 +943,15 @@ def main():
nfsstat_parser = subparsers.add_parser('nfsstat',
parents=[common_parser],
help='Display nfsstat-like statistics.')
+ nfsstat_parser.add_argument('-3', action='store_true', dest='show_v3',
+ help='Show NFS version 3 statistics')
+ nfsstat_parser.add_argument('-4', action='store_true', dest='show_v4',
+ help='Show NFS version 4 statistics')
+ # The mountpoints argument cannot be moved into the common_parser because
+ # it will screw up the parsing of the iostat arguments (interval and count)
+ nfsstat_parser.add_argument('mountpoints', nargs='*', metavar='mountpoint',
+ help='Display statistics for this mountpoint. More than one may be specified. '
+ 'If absent, statistics for all NFS mountpoints will be generated.')
nfsstat_parser.set_defaults(func=nfsstat_command)
iostat_parser = subparsers.add_parser('iostat',
--
1.9.3
Added documentation for the iostat and nfsstat sub-commands. Added
documentation for all of the options that have recently been added.
Signed-off-by: Scott Mayhew <[email protected]>
---
tools/mountstats/mountstats.man | 142 ++++++++++++++++++++++++++++++++++++----
1 file changed, 128 insertions(+), 14 deletions(-)
diff --git a/tools/mountstats/mountstats.man b/tools/mountstats/mountstats.man
index 0de31b7..bee3f86 100644
--- a/tools/mountstats/mountstats.man
+++ b/tools/mountstats/mountstats.man
@@ -1,32 +1,146 @@
.\"
.\" mountstats(8)
.\"
-.TH mountstats 8 "15 Apr 2010"
+.TH mountstats 8 "12 Dec 2014"
.SH NAME
-mountstats \- Displays NFS client per-mount statistics
+mountstats \- Displays various NFS client per-mount statistics
.SH SYNOPSIS
-.BI "mountstats ["<options> "] " <mount_point> " [ " <mount_point> "]"
-.SH DESCRIPTION
-The
.B mountstats
-command displays NFS client statisitics on each given
-.I <mount_point>
+.RB [ \-h | \-\-help ]
+.RB [ \-v | \-\-version ]
+.RB [ \-f | \-\-file
+.IR infile ]
+.RB [ \-s | \-\-since
+.IR sincefile ]
+.\" .RB [ \-n | \-\-nfs | \-r | \-\-rpc | \-R | \-\-raw ]
+.R [
+.RB [ \-n | \-\-nfs ]
+.R |
+.RB [ \-r | \-\-rpc ]
+.R |
+.RB [ \-R | \-\-raw ]
+.R ]
+.RI [ mountpoint ] ...
+.P
+.B mountstats iostat
+.RB [ \-h | \-\-help ]
+.RB [ \-v | \-\-version ]
+.RB [ \-f | \-\-file
+.IR infile ]
+.RB [ \-s | \-\-since
+.IR sincefile ]
+.RI [ interval ]
+.RI [ count ]
+.RI [ mountpoint ] ...
+.P
+.B mounstats nfsstat
+.RB [ \-h | \-\-help ]
+.RB [ \-v | \-\-version ]
+.RB [ \-f | \-\-file
+.IR infile ]
+.RB [ \-s | \-\-since
+.IR sincefile ]
+.RB [ \-3 ]
+.RB [ \-4 ]
+.RI [ mountpoint ] ...
+.P
+.SH DESCRIPTION
+.RB "The " mountstats " command displays various NFS client statisitics for each given"
+.IR mountpoint .
+.P
+.RI "If no " mountpoint " is given, statistics will be displayed for all NFS mountpoints on the client."
+.SS Sub-commands
+Valid
+.BR mountstats (8)
+subcommands are:
+.IP "\fBmountstats\fP"
+Display a combination of per-op RPC statistics, NFS event counts, and NFS byte counts. This is the default sub-command that will be executed if no sub-command is given.
+.IP "\fBiostat\fP"
+Display iostat-like statistics.
+.IP "\fBnfsstat\fP"
+Display nfsstat-like statistics.
.SH OPTIONS
+.SS Options valid for all sub-commands
+.TP
+.B \-h, \-\-help
+show the help message and exit
+.TP
+.B \-v, \-\-version
+show program's version number and exit
+.TP
+\fB\-f \fIinfile\fR, \fB\-\-file \fIinfile
+Read stats from
+.I infile
+instead of
+.IR /proc/self/mountstats ". " infile
+must be in the same format as
+.IR /proc/self/mountstats .
+This may be used with the
+.BR \-S | \-\-since
+options to display the delta between two different points in time.
+This may not be used with the
+.IR interval " or " count
+options of the
+.B iostat
+sub-command.
.TP
-.B " \-\-nfs
-display only the NFS statistics
+\fB\-S \fIsincefile\fR, \fB\-\-since \fIsincefile
+Show difference between current stats and those in
+.IR sincefile ". " sincefile
+must be in the same format as
+.IR /proc/self/mountstats .
+This may be used with the
+.BR \-f | \-\-file
+options to display the delta between two different points in time.
+This may not be used with the
+.IR interval " or " count
+options of the
+.B iostat
+sub-command.
+.SS Options specific to the mountstats sub-command
+.B \-n, \-\-nfs
+Display only the NFS statistics
.TP
-.B " \-\-rpc
-display only the RPC statistics
+.B \-r, \-\-rpc
+Display only the RPC statistics
.TP
-.B " \-\-version
-display the version of this command
+.B \-R, \-\-raw
+Display only the raw statistics. This is intended for use with the
+.BR \-f | \-\-file
+and
+.BR \-S | \-\-since
+options.
+.SS Options specific to the iostat sub-command
+.IP "\fIinterval\fP"
+Specifies the amount of time in seconds between each report. The first report contains statistics for the time since each file system was mounted. Each subsequent report contains statistics collected during the interval since the previous report. This may not be used with the
+.BR \-f | \-\-file
+or
+.BR \-s | \-\-since
+options.
+.P
+.IP "\fIcount\fP"
+Determines the number of reports generated at
+.I interval
+seconds apart. If the
+.I interval
+parameter is specified without the
+.I count
+parameter, the command generates reports continuously. This may not be used with the
+.BR \-f | \-\-file
+or
+.BR \-s | \-\-since
+options.
+.SS Options specific to the nfsstat sub-command
+.IP "\fB\-3\fP"
+Show only NFS version 3 statistics. The default is to show both version 3 and version 4 statistics.
+.IP "\fB\-4\fP"
+Show only NFS version 4 statistics. The default is to show both version 3 and version 4 statistics.
.SH FILES
.TP
.B /proc/self/mountstats
.SH SEE ALSO
.BR iostat (8),
.BR nfsiostat (8),
-.BR nfsstat(8)
+.BR nfsstat (8)
.SH AUTHOR
Chuck Lever <[email protected]>
--
1.9.3
On 12/12/2014 02:14 PM, Scott Mayhew wrote:
> These are also available in the 'mountstats-improvements-v4' branch at
> https://github.com/scottmayhew/nfs-utils.git
>
> v4 changes:
>
> - SteveD was not a fan of the ms-iostat and ms-nfsstat command names.
> Rather than think up new names, I decided to make these all sub-commands.
> If no sub-command is given by the user then the 'mountstats' sub-command
> will run by default (so any existing scripts that run older versions of
> the mountstats command should still work).
>
> - Rather than have 3 tiny man pages, I documented the main command and the
> three sub-commands on a single man page.
>
> v3 changes:
>
> - Rebased on top of the lastest mountstats version
>
> - Added exception processing to fail more gracefully when the argparse
> module is not installed
>
> - Corrected some issues with the man pages (filenames not italicized,
> unmatched braces in the synopses)
>
> v2 changes:
>
> - Changed the parsing to use the argparse module instead of optparse
>
> - Added per-mountpoint headers to the output of 'mountstats --rpc'
>
> - Revamped the ms-nfsstat command to take a variable list of mountpoints
> (so now mountstats, ms-iostat, and ms-nfsstat all behave in the same
> manner). Added -3 and -4 options which behave the same way they do in
> nfsstat.c. The output doesn't still doesn't match up 100% with that
> of nfsstat.c though (I'm just taking the first 12 characters of the
> operation name and converting them to lowercase to use as labels, while
> nfsstat.c defines its own labels, but also it looks to me like nfsstat.c
> is missing some operations altoegether).
>
> - Updated the man page for mountstats and added man pages for ms-iostat
> and ms-nfsstat.
>
> Original cover letter:
>
> -------------------8<------------------
>
> The following patches add a couple of enhancements to mountstats.py. I
> also fixed a few bugs I encountered along the way. Highlights include:
>
> - added support for -f/--file to allow stats to be parsed from an
> aritrary input file instead of /proc/self/mountstats
>
> - added support for -S/--since to show just the changes that have
> occurred between the current and a previous set of statisics (works
> with and without the -f option)
>
> - added support for -R/--raw to generate 'raw' statistics (i.e. in the
> same format as /proc/self/mountstats). It's intended to be used with
> the -f and -S options.
>
> - implemented the ms-nfsstat command to generate client-side
> nfsstat-like statisics (only works with a single mountpoint)
>
> My motivation for these changes was so that I could take various copies
> of /proc/self/mountstats and massage them into data that I could feed
> into the 'report' option of Dros's nfsometer tool for scenarios where
> it's not feasible to run nfsometer itself (e.g. systems where we can't
> start with an 'idle' state (i.e. no NFS filesystems initially
> mounted), systems with multiple NFS filesystems mounted, and workloads
> that can't easily be boiled down into an nfsometer workload file or run
> via the custom workload environment variables).
>
> Scott Mayhew (14):
> mountstats: Fix up NFS event counters
> mountstats: Add lists of various counters
> mountstats: Refactor __parse_nfs_line and __parse_rpc_line
> mountstats: Refactor compare_iostats
> mountstats: Convert existing option parsing to use the argparse module
> mountstats: Make the iostat sub-command output match that of
> nfs-iostat.py
> mountstats: Make print_iostat_summary handle newly appearing mounts
> mountstats: Add support for -f/--file
> mountstats: Add support for -S/--since
> mountstats: Fix IndexError in __parse_nfs_line
> mountstats: Allow mountstats_command to take a variable number of
> mountpoints
> mountstats: Add support for -R/--raw to mountstats_command
> mountstats: Implement the nfsstat sub-command
> mountstats: Updated the mountstats(8) man page.
>
> tools/mountstats/mountstats.man | 142 +++++-
> tools/mountstats/mountstats.py | 927 ++++++++++++++++++++++++++++------------
> 2 files changed, 775 insertions(+), 294 deletions(-)
>
Committed...
steved.