Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 15 Oct 2019 02:55:22 -0700
From:   Christoph Hellwig <hch@infradead.org>
To:     Alan Mikhak <alan.mikhak@sifive.com>
Cc:     Christoph Hellwig <hch@infradead.org>,
        linux-kernel@vger.kernel.org, martin.petersen@oracle.com,
        alexios.zavras@intel.com, ming.lei@redhat.com,
        gregkh@linuxfoundation.org, tglx@linutronix.de,
        Jason Gunthorpe <jgg@ziepe.ca>, christophe.leroy@c-s.fr,
        Palmer Dabbelt <palmer@sifive.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Logan Gunthorpe <logang@deltatee.com>
Subject: Re: [PATCH] scatterlist: Validate page before calling PageSlab()
Message-ID: <20191015095522.GA22551@infradead.org>
References: <1569885755-10947-1-git-send-email-alan.mikhak@sifive.com>
 <20191007061324.GB17978@infradead.org>
 <CABEDWGyovfKuXsNpfhsSCJ0sryg3EpAsaqRTHxBGC9LFM+=dww@mail.gmail.com>
 <CABEDWGxZaJp17jhd-CPqc+n9ZjYzvp63PymfMo0JVd=jzQEizQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CABEDWGxZaJp17jhd-CPqc+n9ZjYzvp63PymfMo0JVd=jzQEizQ@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Oct 07, 2019 at 02:13:51PM -0700, Alan Mikhak wrote:
> > My goal is to not modify the Linux NVMe target code at all. The NVMe
> > endpoint function driver currently does the work that is required.

You will have to do some modifications, as for example in PCIe you can
have a n:1 relationship between SQs and CQs.  And you need to handle
the Create/Delete SQ/CQ commands, but not the fabrics commands.  And
modifying subsystems in Linux is perfectly acceptable, that is how they
improve.

Do you have a pointer to your code?

> > In my current platform, there are no page struct backing for the PCIe
> > memory address space.

In Linux there aren't struct pages for physical memory remapped using
ioremap().  But if you want to feed them to the I/O subsystem you have
to use devm_memremap_pages to create a page backing.  Assuming you are
on a RISC-V platform given your affiliation you'll need to ensure your
kernel allows for ZONE_DEVICE pages, which Logan (added to Cc) has been
working on.  I don't remember what the current status is.

> Please consider the following information and cost estimate in
> bytes for requiring page structs for PCI memory if used with
> scatterlists. For example, a 128GB PCI memory address space
> could require as much as 256MB of system memory just for
> page struct backing. In a 1GB 64-bit system with flat memory
> model, that consumes 25% of available memory. However,
> not all of the 128GB PCI memory may be mapped for use at
> a given time depending on the application. The cost of PCI
> page structs is an upfront cost to be paid at system start.

I know the pages are costly.  But once you want to feed them through
subsystems that do expect pages you'll have to do that.  And anything
using scatterlists currently does.  A little hack here and there isn't
going to solve that.