Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
From:   Chris Chilvers <chilversc@gmail.com>
Date:   Thu, 12 May 2022 15:40:31 +0100
Message-ID: <CAAmbk-cVYfEig+HG6H7apcF==H0g4pwSXfEAGHw4JcjSGkddHA@mail.gmail.com>
Subject: Calculating total server read/write bytes
To:     linux-nfs@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

I want to display the total server read, and server write bytes from the
io_stats. However, this poses a challenge as the only way I can find to retrieve
these stats is from /proc/self/mountstats.

The issue I run into is that io_stats is per nfs_server record, while the output
is reported per mount. Multiple mounts can share the same nfs_server record.

This means while parsing the mountstats I need a way to reliably determine if
two mounts share the same nfs_server record.

From reading the source it appears that nfs_server and super_block are
correlated 1:1, and s_dev for NFS is allocated a unique value via
set_anon_super.

My plan is to run stat on the mount point to fetch the st_dev value. Thus is it
safe to assume (for NFS) if two mount points have the same st_dev value they
share the same io_stats, and if the st_dev value is different they have separate
io_stats, or is there some scenario that I've missed?

Longer term it would be nice to make this process simpler, I can see several
options on how to handle this:

1. Add a unique identifier in the mountstats (e.g. s_dev) that can be used as a
   unique key.

   The concept here is that anything scraping the stats does not need to
   rely on the assumption that io_stats has a 1:1 correlation with super_block
   or that s_dev is unique. If the implementation were to change in future the
   key can be updated to match.

   The only issue is if this uses an existing value such as s_dev consumers
   might ignore the fact that the key should be treated as an opaque value and
   only used to deduplicate io_stats and start relying on the key matching the
   s_dev value. As such it might be necessary to allocate
   a separate unique key to the io_stats.

2. Two separate files in procfs that only output each unique stat once.
   * io_stats for each nfs_server excluding xprt: lines (multiple nfs_server
     entries can share the same RPC transport).
   * stats for RPC transports (xprt lines). These should be unique so that each
     RPC connection is only displayed once.

3. Add a new file to procfs that outputs aggregated io_stats (including xprt
   lines). These stats should be aggregated at the source server level. It is
   still useful when a client has mounted exports from multiple servers to have
   separate stats per server.

--
Chris