From: "J. Bruce Fields" Subject: Re: [PATCH 0/3] NFSD EOS deferral Date: Fri, 17 Oct 2008 13:44:54 -0400 Message-ID: <20081017174454.GB11884@fieldses.org> References: <1224104426-12293-1-git-send-email-andros@netapp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: linux-nfs@vger.kernel.org To: andros@netapp.com Return-path: Received: from mail.fieldses.org ([66.93.2.214]:41897 "EHLO fieldses.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755251AbYJQRo4 (ORCPT ); Fri, 17 Oct 2008 13:44:56 -0400 In-Reply-To: <1224104426-12293-1-git-send-email-andros@netapp.com> Sender: linux-nfs-owner@vger.kernel.org List-ID: On Wed, Oct 15, 2008 at 05:00:23PM -0400, andros@netapp.com wrote: > Here's a patch set for review - it compiles and seems to work, but I haven't > done stress testing, nor testing of all of the combinations of deferral cases. > > A deferral occurs when NFSD needs information from an rpc cache, and an upcall > is required. Instead of NFSD waiting for the cache to be filled by the upcall, > the RPC request is inserted back into the receive stream for processing at a > later time. > > Exactly once semantics require that NFSD compound RPC deferral processing > restart at the operation that caused the deferral, instead of reprocessing the > full compound RPC from the start possibly repeating operation processing. > These patches add three callbacks, a data pointer, and page pointer storage > to the sunrpc svc deferral architecture that NFSD uses to accomplish this goal. > > Deferrals that do not define the callbacks act as before. Care has been taken > to ensure that combinations of deferrals - those from the NFSv4 server with > the callbacks defined, and those from the RPC layer without the callbacks > defined work together correctly. > > Thoughts, comments and suggestions are really appreciated... Requests longer than a page are still not deferred, so large writes that trigger upcalls still get an ERR_DELAY. OK, probably no big deal. I don't think we can apply this until we have some way to track the number and size of deferred requests outstanding and fall back on ERR_DELAY if it's too much. I do sometimes wonder whether continuing with the current deferred-request approach is best, though: - If we're saving out large parts of the request anyway (the response pages), then maybe we should just keep rqstp's on the deferred request queue instead of copying to a separate deferred_request structure. - Then as long as we're saving all that request data, is there really significant savings from not keeping a thread around too? So I wonder if it'd be better just to let threads sleep (and be more aggressive about starting up new threads if appropriate, and add some other heuristics to avoid a situation where the whole server stalls on a temporarily wedged userspace daemon). --b.