From: "Kani, Toshimitsu" <toshi.kani@hpe.com>
To: "dan.j.williams@intel.com" <dan.j.williams@intel.com>,
        "linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>
CC: "dm-devel@redhat.com" <dm-devel@redhat.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "viro@zeniv.linux.org.uk" <viro@zeniv.linux.org.uk>,
        "hch@lst.de" <hch@lst.de>, "x86@kernel.org" <x86@kernel.org>,
        "snitzer@redhat.com" <snitzer@redhat.com>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>
Subject: Re: [PATCH v3 02/14] dm: add ->copy_from_iter() dax operation support
Thread-Topic: [PATCH v3 02/14] dm: add ->copy_from_iter() dax operation
 support
Thread-Index: AQHS4V9NuqSfayrQ902LcK7VpNo0cqIlHq6AgAAJwIA=
Date: Thu, 15 Jun 2017 01:21:08 +0000
Message-ID: <1497489635.9288.18.camel@hpe.com>
References: <149703982465.20620.14881139332926778446.stgit@dwillia2-desk3.amr.corp.intel.com>
         <149703983692.20620.3787021839815275819.stgit@dwillia2-desk3.amr.corp.intel.com>
         <1497487541.9288.16.camel@hpe.com>
In-Reply-To: <1497487541.9288.16.camel@hpe.com>
Accept-Language: en-US
Content-Language: en-US
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <E1E5D9D30646AA43A340A60F7CD7EFE9@NAMPRD84.PROD.OUTLOOK.COM>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2017 01:21:08.0801
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DF4PR84MB0106
X-OriginatorOrg: hpe.com
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from base64 to 8bit by mail.home.local id v5F1LIDc021589
Content-Length: 2086
Lines: 42

On Wed, 2017-06-14 at 18:45 -0600, Toshi Kani wrote:
> On Fri, 2017-06-09 at 13:23 -0700, Dan Williams wrote:
> > Allow device-mapper to route copy_from_iter operations to the
> > per-target implementation. In order for the device stacking to work
> > we need a dax_dev and a pgoff relative to that device. This gives
> > each layer of the stack the information it needs to look up the
> > operation pointer for the next level.
> > 
> > This conceptually allows for an array of mixed device drivers with
> > varying copy_from_iter implementations.
> > 
> > Cc: Toshi Kani <toshi.kani@hpe.com>
> > Reviewed-by: Mike Snitzer <snitzer@redhat.com>
> > Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> 
> I was worried about possible overhead with additional stub calls, but
> it looks fine with a single thread fio write test with direct=1.
> 
>  92.62%  [kernel.kallsyms]   [k] __copy_user_nocache
>   0.04%  [kernel.kallsyms]   [k] entry_SYSCALL_64_fastpath
>   0.08%  libpthread-2.22.so  [.] __GI___libc_write
>   0.01%  [kernel.kallsyms]   [k] sys_write
>   0.02%  [kernel.kallsyms]   [k] vfs_write
>   0.02%  [kernel.kallsyms]   [k] __vfs_write
>   0.02%  [kernel.kallsyms]   [k] ext4_file_write_iter
>   0.02%  [kernel.kallsyms]   [k] dax_iomap_rw
>   0.03%  [kernel.kallsyms]   [k] iomap_apply
>   0.04%  [kernel.kallsyms]   [k] dax_iomap_actor
>   0.01%  [kernel.kallsyms]   [k] dax_copy_from_iter
>   0.01%  [kernel.kallsyms]   [k] dm_dax_copy_from_iter
>   0.01%  [kernel.kallsyms]   [k] linear_dax_copy_from_iter
>   0.03%  [kernel.kallsyms]   [k] copy_from_iter_flushcache
>   0.00%  [kernel.kallsyms]   [k] pmem_copy_from_iter

I had bs=256k, which was too big for this test.  bs=4k result is not
this pretty at all, only 23% in __copy_user_nocache.  This change
accounts for approx. 1% with 4k.  Given we have larger overheads in
many other functions in the path, the change looks acceptable (I keep
my review-by).  I'd prefer to reduce code in the path, though.

Thanks,
-Toshi