Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
From:   "NeilBrown" <neilb@suse.de>
To:     "Chuck Lever" <cel@kernel.org>
Cc:     linux-nfs@vger.kernel.org, "Chuck Lever" <chuck.lever@oracle.com>,
        lorenzo@kernel.org, jlayton@redhat.com, david@fromorbit.com
Subject: Re: [PATCH v2 4/9] SUNRPC: Count ingress RPC messages per svc_pool
In-reply-to: <ZKN6GZ8q9NVLywOJ@manet.1015granger.net>
References: <168842897573.139194.15893960758088950748.stgit@manet.1015granger.net>,
 <168842927591.139194.16920372497489479670.stgit@manet.1015granger.net>,
 <168843152047.8939.5788535164524204707@noble.neil.brown.name>,
 <ZKN6GZ8q9NVLywOJ@manet.1015granger.net>
Date:   Tue, 04 Jul 2023 12:30:57 +1000
Message-id: <168843785749.8939.9913705917013899427@noble.neil.brown.name>
Precedence: bulk

On Tue, 04 Jul 2023, Chuck Lever wrote:
> On Tue, Jul 04, 2023 at 10:45:20AM +1000, NeilBrown wrote:
> > On Tue, 04 Jul 2023, Chuck Lever wrote:
> > > From: Chuck Lever <chuck.lever@oracle.com>
> > > 
> > > To get a sense of the average number of transport enqueue operations
> > > needed to process an incoming RPC message, re-use the "packets" pool
> > > stat. Track the number of complete RPC messages processed by each
> > > thread pool.
> > 
> > If I understand this correctly, then I would say it differently.
> > 
> > I think there are either zero or one transport enqueue operations for
> > each incoming RPC message.  Is that correct?  So the average would be in
> > (0,1].
> > Wouldn't it be more natural to talk about the average number of incoming
> > RPC messages processed per enqueue operation?  This would be typically
> > be around 1 on a lightly loaded server and would climb up as things get
> > busy. 
> > 
> > Was there a reason you wrote it the way around that you did?
> 
> Yes: more than one enqueue is done per incoming RPC. For example,
> svc_data_ready() enqueues, and so does svc_xprt_receive().
> 
> If the RPC requires more than one call to ->recvfrom() to complete
> the receive operation, each one of those calls does an enqueue.
> 

Ahhhh - that makes sense.  Thanks.
So its really that a number of transport enqueue operations are needed
to *receive* the message.  Once it is received, it is then processed
with no more enqueuing.

I was partly thrown by the fact that the series is mostly about the
queue of threads, but this is about the queue of transports.
I guess the more times a transport if queued, the more times a thread
needs to be woken?

Thanks,
NeilBrown