2007-05-02 00:26:54

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On Mon, 30 Apr 2007 00:04:31 -0700 (PDT)
Christoph Lameter <[email protected]> wrote:

>
> Replace open-coded kmap_atomic() and kunmap_atomic()
> surrounding two memory clear operations with zero_user_page(), as both
> memory operations act on the same page.
>
> Cc: Nate Diller <[email protected]>
> Signed-off-by: Christoph Lameter <[email protected]>
>
> ---
> fs/buffer.c | 29 ++++++++++-------------------
> fs/libfs.c | 10 +++++-----
> 2 files changed, 15 insertions(+), 24 deletions(-)
>
> Index: linux-2.6.21-rc7-mm2/fs/buffer.c
> ===================================================================
> --- linux-2.6.21-rc7-mm2.orig/fs/buffer.c 2007-04-27 22:51:27.000000000 -0700
> +++ linux-2.6.21-rc7-mm2/fs/buffer.c 2007-04-29 23:58:48.000000000 -0700
> @@ -1796,19 +1796,12 @@ static int __block_prepare_write(struct
> set_buffer_uptodate(bh);
> continue;
> }
> - if (block_end > to || block_start < from) {
> - void *kaddr;
> -
> - kaddr = kmap_atomic(page, KM_USER0);
> - if (block_end > to)
> - memset(kaddr+to, 0,
> - block_end-to);
> - if (block_start < from)
> - memset(kaddr+block_start,
> - 0, from-block_start);
> - flush_dcache_page(page);
> - kunmap_atomic(kaddr, KM_USER0);
> - }
> + if (block_end > to)
> + zero_user_page(page, to,
> + block_end - to, KM_USER0);
> + if (block_start < from)
> + zero_user_page(page, block_start,
> + from - block_start, KM_USER0);
> continue;
> }
> }
> @@ -2224,7 +2217,6 @@ int nobh_prepare_write(struct page *page
> unsigned block_in_page;
> unsigned block_start;
> sector_t block_in_file;
> - char *kaddr;
> int nr_reads = 0;
> int i;
> int ret = 0;
> @@ -2264,13 +2256,12 @@ int nobh_prepare_write(struct page *page
> if (PageUptodate(page))
> continue;
> if (buffer_new(&map_bh) || !buffer_mapped(&map_bh)) {
> - kaddr = kmap_atomic(page, KM_USER0);
> if (block_start < from)
> - memset(kaddr+block_start, 0, from-block_start);
> + zero_user_page(page, block_start,
> + from - block_start, KM_USER0);
> if (block_end > to)
> - memset(kaddr + to, 0, block_end - to);
> - flush_dcache_page(page);
> - kunmap_atomic(kaddr, KM_USER0);
> + zero_user_page(page, to,
> + block_end - to, KM_USER0);
> continue;
> }
> if (buffer_uptodate(&map_bh))
> Index: linux-2.6.21-rc7-mm2/fs/libfs.c
> ===================================================================
> --- linux-2.6.21-rc7-mm2.orig/fs/libfs.c 2007-04-27 22:51:27.000000000 -0700
> +++ linux-2.6.21-rc7-mm2/fs/libfs.c 2007-04-27 22:55:12.000000000 -0700
> @@ -338,11 +338,11 @@ int simple_prepare_write(struct file *fi
> {
> if (!PageUptodate(page)) {
> if (to - from != PAGE_CACHE_SIZE) {
> - void *kaddr = kmap_atomic(page, KM_USER0);
> - memset(kaddr, 0, from);
> - memset(kaddr + to, 0, PAGE_CACHE_SIZE - to);
> - flush_dcache_page(page);
> - kunmap_atomic(kaddr, KM_USER0);
> + if (from)
> + zero_user_page(page, 0, from, KM_USER0);
> + if (to < PAGE_CACHE_SIZE)
> + zero_user_page(page, to,
> + PAGE_CACHE_SIZE - to, KM_USER0);
> }
> }
> return 0;

As Satyam said, this will sometimes cause us to map and unmap the page
twice, and to run flush_dcache_page() twice. In not-terribly-uncommon
circumstances in very frequently called functions.

Doesn't seem worth it to me.


2007-05-02 01:09:36

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On Tue, 1 May 2007, Andrew Morton wrote:

> As Satyam said, this will sometimes cause us to map and unmap the page
> twice, and to run flush_dcache_page() twice. In not-terribly-uncommon
> circumstances in very frequently called functions.
>
> Doesn't seem worth it to me.

Ok but we have that code three times. Should I add a variant of
zero_user_page that zeroes everything but the section specified?

zero_user_page_allbut() ?

2007-05-02 02:16:34

by Nate Diller

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On 5/1/07, Christoph Lameter <[email protected]> wrote:
> On Tue, 1 May 2007, Andrew Morton wrote:
>
> > As Satyam said, this will sometimes cause us to map and unmap the page
> > twice, and to run flush_dcache_page() twice. In not-terribly-uncommon
> > circumstances in very frequently called functions.
> >
> > Doesn't seem worth it to me.
>
> Ok but we have that code three times. Should I add a variant of
> zero_user_page that zeroes everything but the section specified?
>
> zero_user_page_allbut() ?

the function already exists, it's called simple_prepare_write(), and i
thought there were patches to convert those callsites in -mm ...
although it looks like i let a review comment on that slide by. I
need to redo a bunch of patches for re-submission anyway, so i guess
i'll deal with that tomorrow.

NATE

2007-05-02 02:27:18

by Nate Diller

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On 5/1/07, Nate Diller <[email protected]> wrote:
> On 5/1/07, Christoph Lameter <[email protected]> wrote:
> > On Tue, 1 May 2007, Andrew Morton wrote:
> >
> > > As Satyam said, this will sometimes cause us to map and unmap the page
> > > twice, and to run flush_dcache_page() twice. In not-terribly-uncommon
> > > circumstances in very frequently called functions.
> > >
> > > Doesn't seem worth it to me.
> >
> > Ok but we have that code three times. Should I add a variant of
> > zero_user_page that zeroes everything but the section specified?
> >
> > zero_user_page_allbut() ?
>
> the function already exists, it's called simple_prepare_write(), and i
> thought there were patches to convert those callsites in -mm ...
> although it looks like i let a review comment on that slide by. I
> need to redo a bunch of patches for re-submission anyway, so i guess
> i'll deal with that tomorrow.
>
> NATE
>

well, leave it to me to reply too quickly, sorry. i think we should
leave simple_prepare_write() the way it is, since it's a library
function itself. the other two callsites in your patch are buffers,
which may themselves be smaller than a page so you would need a
special function for just those two uses, there's no other way to
avoid making two calls to flush_dcache_page(). if it's tremendously
important to you to eliminate open coding of these, maybe make a
'static int buffer_prepare_write()' or some such in fs/buffer.c

NATE

2007-05-02 03:58:08

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On Tue, 1 May 2007, Nate Diller wrote:

> well, leave it to me to reply too quickly, sorry. i think we should
> leave simple_prepare_write() the way it is, since it's a library
> function itself. the other two callsites in your patch are buffers,
> which may themselves be smaller than a page so you would need a
> special function for just those two uses, there's no other way to
> avoid making two calls to flush_dcache_page(). if it's tremendously
> important to you to eliminate open coding of these, maybe make a
> 'static int buffer_prepare_write()' or some such in fs/buffer.c

All three sites zap two parts of a page. If we had a
zero_user_page2 like this

zero_user_page2(page, start1, end1, start2, end2, kmap)

then all 3 sites could use the same funtion.

libfs.c:

zero_user_page_segments(page, 0, from, to, PAGE_CACHE_SIZE, KM_USER0);

buffer.c:

zero_user_page_segments(page, from, block_start, to, block_end, KM_USER0)

zero_user_page_segments(page, blockstart, from, to, block_end, KM_USER0)


I did not look through the whole kernel but this zapping segments is
likely frequent given the nature of the blocklayer.

The 3 call sites pretty ugly on their own. I think it would be good
to have one clearly commented version of this somewhere. Call sites
will be much clearer since you do not have the kmap_ obfuscation nor
the calculation of the length of each segment.

2007-05-02 04:10:06

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] zero_user_page uses in fs/buffer.c and fs/libfs.c

On Tue, 1 May 2007 20:58:02 -0700 (PDT) Christoph Lameter <[email protected]> wrote:

> On Tue, 1 May 2007, Nate Diller wrote:
>
> > well, leave it to me to reply too quickly, sorry. i think we should
> > leave simple_prepare_write() the way it is, since it's a library
> > function itself. the other two callsites in your patch are buffers,
> > which may themselves be smaller than a page so you would need a
> > special function for just those two uses, there's no other way to
> > avoid making two calls to flush_dcache_page(). if it's tremendously
> > important to you to eliminate open coding of these, maybe make a
> > 'static int buffer_prepare_write()' or some such in fs/buffer.c
>
> All three sites zap two parts of a page. If we had a
> zero_user_page2 like this
>
> zero_user_page2(page, start1, end1, start2, end2, kmap)
>
> then all 3 sites could use the same funtion.
>
> libfs.c:
>
> zero_user_page_segments(page, 0, from, to, PAGE_CACHE_SIZE, KM_USER0);
>
> buffer.c:
>
> zero_user_page_segments(page, from, block_start, to, block_end, KM_USER0)
>
> zero_user_page_segments(page, blockstart, from, to, block_end, KM_USER0)
>

yup. And perhaps zero_user_page() becomes a caller to
zero_user_page_segments() if we're sure that the compiler will dtrt.

>
> I did not look through the whole kernel but this zapping segments is
> likely frequent given the nature of the blocklayer.
>
> The 3 call sites pretty ugly on their own. I think it would be good
> to have one clearly commented version of this somewhere. Call sites
> will be much clearer since you do not have the kmap_ obfuscation nor
> the calculation of the length of each segment.

Sure.