On Tue, Jan 31, 2012 at 05:23:59PM +0100, Marek Szyprowski wrote:
> > > >>+ page = pfn_to_page(pfn);
> > > >>+ if (PageBuddy(page)) {
> > > >>+ pfn += 1 << page_order(page);
> > > >>+ } else if (page_count(page) == 0) {
> > > >>+ set_page_private(page, MIGRATE_ISOLATE);
> > > >>+ ++pfn;
> > > >
> > > >This is dangerous for two reasons. If the page_count is 0, it could
> > > >be because the page is in the process of being freed and is not
> > > >necessarily on the per-cpu lists yet and you cannot be sure if the
> > > >contents of page->private are important. Second, there is nothing to
> > > >prevent another CPU allocating this page from its per-cpu list while
> > > >the private field is getting updated from here which might lead to
> > > >some interesting races.
> > > >
> > > >I recognise that what you are trying to do is respond to Gilad's
> > > >request that you really check if an IPI here is necessary. I think what
> > > >you need to do is check if a page with a count of 0 is encountered
> > > >and if it is, then a draining of the per-cpu lists is necessary. To
> > > >address Gilad's concerns, be sure to only this this once per attempt at
> > > >CMA rather than for every page encountered with a count of 0 to avoid a
> > > >storm of IPIs.
> > >
> > > It's actually more then that.
> > >
> > > This is the same issue that I first fixed with a change to free_pcppages_bulk()
> > > function[1]. At the time of positing, you said you'd like me to try and find
> > > a different solution which would not involve paying the price of calling
> > > get_pageblock_migratetype(). Later I also realised that this solution is
> > > not enough.
> > >
> > > [1] http://article.gmane.org/gmane.linux.kernel.mm/70314
> > >
> >
> > Yes. I had forgotten the history but looking at that patch again,
> > I would reach the conclusion that this was adding a new call to
> > get_pageblock_migratetype() in the bulk free path. That would affect
> > everybody whether they were using CMA or not.
>
> This will be a bit ugly, but we can also use that code and compile it conditionally
> when CMA has been enabled.
That would also be very unfortunate because it means enabling CMA incurs
a performance cost to everyone whether they use CMA or not. For ARM,
this may not be a problem but it would be for other arches if they
wanted to use CMA or if it ever became part of a distro contig.
> Pages, which have incorrect migrate type on free finally
> causes pageblock migration type change from MIGRATE_CMA to MIGRATE_MOVABLE.
I'm not quite seeing this. In free_hot_cold_page(), the pageblock
type is checked so the page private should be set to MIGRATE_CMA or
MIGRATE_ISOLATE for the CMA area. It's not clear how this can change a
pageblock to MIGRATE_MOVABLE in error. If it turns out that you
absolutely have to call get_pageblock_migratetype() from
free_pcppages_bulk() and my alternative suggestion did not work out then
document all these issues in a comment when putting the call under
CONFIG_CMA so that it is not forgotten.
--
Mel Gorman
SUSE Labs
> On Tue, Jan 31, 2012 at 05:23:59PM +0100, Marek Szyprowski wrote:
>> Pages, which have incorrect migrate type on free finally
>> causes pageblock migration type change from MIGRATE_CMA to MIGRATE_MOVABLE.
On Thu, 02 Feb 2012 13:47:29 +0100, Mel Gorman <[email protected]> wrote:
> I'm not quite seeing this. In free_hot_cold_page(), the pageblock
> type is checked so the page private should be set to MIGRATE_CMA or
> MIGRATE_ISOLATE for the CMA area. It's not clear how this can change a
> pageblock to MIGRATE_MOVABLE in error.
Here's what I think may happen:
When drain_all_pages() is called, __free_one_page() is called for each page on
pcp list with migrate type deducted from page_private() which is MIGRATE_CMA.
This result in the page being put on MIGRATE_CMA freelist even though its
pageblock's migrate type is MIGRATE_ISOLATE.
When allocation happens and pcp list is empty, rmqueue_bulk() will get executed
with migratetype argument set to MIGRATE_MOVABLE. It calls __rmqueue() to grab
some pages and because the page described above is on MIGRATE_CMA freelist it
may be returned back to rmqueue_bulk().
But, pageblock's migrate type is not MIGRATE_CMA but MIGRATE_ISOLATE, so the
following code:
#ifdef CONFIG_CMA
if (is_pageblock_cma(page))
set_page_private(page, MIGRATE_CMA);
else
#endif
set_page_private(page, migratetype);
will set it's private to MIGRATE_MOVABLE and in the end the page lands back
on MIGRATE_MOVABLE pcp list but this time with page_private == MIGRATE_MOVABLE
and not MIGRATE_CMA.
One more drain_all_pages() (which may happen since alloc_contig_range() calls
set_migratetype_isolate() for each block) and next __rmqueue_fallback() may
convert the whole pageblock to MIGRATE_MOVABLE.
I know, this sounds crazy and improbable, but I couldn't find an easier path
to destruction. As you pointed, once the page is allocated, free_hot_cold_page()
will do the right thing by reading pageblock's migrate type.
Marek is currently experimenting with various patches including the following
change:
#ifdef CONFIG_CMA
int mt = get_pageblock_migratetype(page);
if (is_migrate_cma(mt) || mt == MIGRATE_ISOLATE)
set_page_private(page, mt);
else
#endif
set_page_private(page, migratetype);
As a matter of fact, if __rmqueue() was changed to return migrate type of the
freelist it took page from, we could avoid this get_pageblock_migratetype() all
together. For now, however, I'd rather not go that way just yet -- I'll be happy
to dig into it once CMA gets merged.
--
Best regards, _ _
.o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o
..o | Computer Science, Michał “mina86” Nazarewicz (o o)
ooo +----<email/xmpp: [email protected]>--------------ooO--(_)--Ooo--
Hello,
On Thursday, February 02, 2012 8:53 PM Michał Nazarewicz wrote:
> > On Tue, Jan 31, 2012 at 05:23:59PM +0100, Marek Szyprowski wrote:
> >> Pages, which have incorrect migrate type on free finally
> >> causes pageblock migration type change from MIGRATE_CMA to MIGRATE_MOVABLE.
>
> On Thu, 02 Feb 2012 13:47:29 +0100, Mel Gorman <[email protected]> wrote:
> > I'm not quite seeing this. In free_hot_cold_page(), the pageblock
> > type is checked so the page private should be set to MIGRATE_CMA or
> > MIGRATE_ISOLATE for the CMA area. It's not clear how this can change a
> > pageblock to MIGRATE_MOVABLE in error.
>
> Here's what I think may happen:
>
> When drain_all_pages() is called, __free_one_page() is called for each page on
> pcp list with migrate type deducted from page_private() which is MIGRATE_CMA.
> This result in the page being put on MIGRATE_CMA freelist even though its
> pageblock's migrate type is MIGRATE_ISOLATE.
>
> When allocation happens and pcp list is empty, rmqueue_bulk() will get executed
> with migratetype argument set to MIGRATE_MOVABLE. It calls __rmqueue() to grab
> some pages and because the page described above is on MIGRATE_CMA freelist it
> may be returned back to rmqueue_bulk().
>
> But, pageblock's migrate type is not MIGRATE_CMA but MIGRATE_ISOLATE, so the
> following code:
>
> #ifdef CONFIG_CMA
> if (is_pageblock_cma(page))
> set_page_private(page, MIGRATE_CMA);
> else
> #endif
> set_page_private(page, migratetype);
>
> will set it's private to MIGRATE_MOVABLE and in the end the page lands back
> on MIGRATE_MOVABLE pcp list but this time with page_private == MIGRATE_MOVABLE
> and not MIGRATE_CMA.
>
> One more drain_all_pages() (which may happen since alloc_contig_range() calls
> set_migratetype_isolate() for each block) and next __rmqueue_fallback() may
> convert the whole pageblock to MIGRATE_MOVABLE.
>
> I know, this sounds crazy and improbable, but I couldn't find an easier path
> to destruction. As you pointed, once the page is allocated, free_hot_cold_page()
> will do the right thing by reading pageblock's migrate type.
>
> Marek is currently experimenting with various patches including the following
> change:
>
> #ifdef CONFIG_CMA
> int mt = get_pageblock_migratetype(page);
> if (is_migrate_cma(mt) || mt == MIGRATE_ISOLATE)
> set_page_private(page, mt);
> else
> #endif
> set_page_private(page, migratetype);
>
> As a matter of fact, if __rmqueue() was changed to return migrate type of the
> freelist it took page from, we could avoid this get_pageblock_migratetype() all
> together. For now, however, I'd rather not go that way just yet -- I'll be happy
> to dig into it once CMA gets merged.
After this and some other changes I'm unable to reproduce that issue. I did a whole
night tests and it still works fine, so it looks that it has been finally solved.
I will post v20 patchset soon :)
Best regards
--
Marek Szyprowski
Samsung Poland R&D Center
On Thu, Feb 02, 2012 at 08:53:25PM +0100, Michal Nazarewicz wrote:
> >On Tue, Jan 31, 2012 at 05:23:59PM +0100, Marek Szyprowski wrote:
> >>Pages, which have incorrect migrate type on free finally
> >>causes pageblock migration type change from MIGRATE_CMA to MIGRATE_MOVABLE.
>
> On Thu, 02 Feb 2012 13:47:29 +0100, Mel Gorman <[email protected]> wrote:
> >I'm not quite seeing this. In free_hot_cold_page(), the pageblock
> >type is checked so the page private should be set to MIGRATE_CMA or
> >MIGRATE_ISOLATE for the CMA area. It's not clear how this can change a
> >pageblock to MIGRATE_MOVABLE in error.
>
> Here's what I think may happen:
>
> When drain_all_pages() is called, __free_one_page() is called for each page on
> pcp list with migrate type deducted from page_private() which is MIGRATE_CMA.
> This result in the page being put on MIGRATE_CMA freelist even though its
> pageblock's migrate type is MIGRATE_ISOLATE.
>
Ok, although it will only be allocated for MIGRATE_CMA-compatible
requests so it is not a disaster.
> When allocation happens and pcp list is empty, rmqueue_bulk() will get executed
> with migratetype argument set to MIGRATE_MOVABLE. It calls __rmqueue() to grab
> some pages and because the page described above is on MIGRATE_CMA freelist it
> may be returned back to rmqueue_bulk().
>
This will allocate the page from a pageblock we are trying to isolate
pages from, but only for a movable page that can still be migrated. It
does mean that CMA is doing more work than it should of course and
the problem also impacts memory hot-remove. It's worse for memory
hot-remove because potentially an UNMOVABLE page was allocated from
a MIGRATE_ISOLATE pageblock.
> But, pageblock's migrate type is not MIGRATE_CMA but MIGRATE_ISOLATE, so the
> following code:
>
> #ifdef CONFIG_CMA
> if (is_pageblock_cma(page))
> set_page_private(page, MIGRATE_CMA);
> else
> #endif
> set_page_private(page, migratetype);
>
> will set it's private to MIGRATE_MOVABLE and in the end the page lands back
> on MIGRATE_MOVABLE pcp list but this time with page_private == MIGRATE_MOVABLE
> and not MIGRATE_CMA.
>
> One more drain_all_pages() (which may happen since alloc_contig_range() calls
> set_migratetype_isolate() for each block) and next __rmqueue_fallback() may
> convert the whole pageblock to MIGRATE_MOVABLE.
>
> I know, this sounds crazy and improbable, but I couldn't find an easier path
> to destruction. As you pointed, once the page is allocated, free_hot_cold_page()
> will do the right thing by reading pageblock's migrate type.
>
Ok, it's crazy but the problem is there.
> Marek is currently experimenting with various patches including the following
> change:
>
> #ifdef CONFIG_CMA
> int mt = get_pageblock_migratetype(page);
> if (is_migrate_cma(mt) || mt == MIGRATE_ISOLATE)
> set_page_private(page, mt);
> else
> #endif
> set_page_private(page, migratetype);
>
> As a matter of fact, if __rmqueue() was changed to return migrate type of the
> freelist it took page from, we could avoid this get_pageblock_migratetype() all
> together. For now, however, I'd rather not go that way just yet -- I'll be happy
> to dig into it once CMA gets merged.
>
Ok, thanks for persisting with this.
--
Mel Gorman
SUSE Labs