Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 26 Aug 2022 12:35:09 -0600
From:   Keith Busch <kbusch@kernel.org>
To:     "Fabio M. De Francesco" <fmdefrancesco@gmail.com>
Cc:     Al Viro <viro@zeniv.linux.org.uk>, Christoph Hellwig <hch@lst.de>,
        Sagi Grimberg <sagi@grimberg.me>,
        Chaitanya Kulkarni <kch@nvidia.com>,
        James Smart <james.smart@broadcom.com>,
        Ira Weiny <ira.weiny@intel.com>,
        "Venkataramanan, Anirudh" <anirudh.venkataramanan@intel.com>,
        linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org,
        Chaitanya Kulkarni <chaitanyak@nvidia.com>
Subject: Re: [PATCH v3 1/1] nvmet-tcp: Don't kmap() pages which can't come
 from HIGHMEM
Message-ID: <YwkSXZtO+i+36pXI@kbusch-mbp.dhcp.thefacebook.com>
References: <20220822142438.5954-1-fmdefrancesco@gmail.com>
 <20220822142438.5954-2-fmdefrancesco@gmail.com>
 <YwVjpTW6vWvdVQTK@ZenIV>
 <2887364.VdNmn5OnKV@opensuse>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2887364.VdNmn5OnKV@opensuse>
Precedence: bulk

On Fri, Aug 26, 2022 at 08:16:59PM +0200, Fabio M. De Francesco wrote:
> On mercoled? 24 agosto 2022 01:32:53 CEST Al Viro wrote:
> > On Mon, Aug 22, 2022 at 04:24:38PM +0200, Fabio M. De Francesco wrote:
> > > Therefore, replace the kmap() of sg_page(sg) with a page_address() and
> > > delete the "nr_mapped" field from "nvmet_tcp_cmd" and instead pass a
> > > local variable to iov_iter_kvec() from the call site in
> > > nvmet_tcp_map_pdu_iovec().
> > 
> > I'd suggest looking into switching to ITER_BVEC...
> 
> Hello Al,
> 
> It's interesting that, for the second time in a few days, I've been suggested 
> to switch to the use of BVEC. Christoph asked for first[1]
> 
> I responded to Christoph and, in the same email, I also asked Sagi
> (the author of nvmet_tcp_map_pdu_iovec()) if he wanted to add something to 
> that discussion.
> 
> As you may have already read, I'm so new to kernel development that I still
> know very little about many subsystems and drivers. I am not currently
> able to tell the difference between BVEC and KVEC. I could probably try to 
> switch from one to the other (after learning from other code), however I won't 
> be able to explain in the commit message why users should better use BVEC in 
> this case.
> 
> My task is to remove as many kmap() call sites I can from the entire kernel, 
> and this is all I did in nvmet-tcp. After doing many tens of call sites 
> removals or conversions to kmap_local_page(), I learned to not go deep into 
> those details which aren't needed to accomplish the task I've been given.
> 
> However, since you and Christoph suggested doing more (I suppose in a separate
> patch), I did some research which seems to confirm that iov_iter_kvec() may be 
> good for what it is supposed to do.
> 
> Since you are talking again about BVEC, soon after Christoph did the same, I'd
> like to understand better which improvements do we expect from BVEC and why
> and when should developers prefer this or KVEC.
> 
> Can you please provide further information or links to documentation?

This kvec/bvec talk is a digression from your original goal: reduce unnecessary
kmap() usage. If anyone wants to take on the bvec optimization in this code
path, then by all means lets do it in a separate patch.