From: Frank Mehnert <frank.mehnert@oracle.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: Re: Re: Re: PG_reserved and compound pages
Date: Tue, 19 Apr 2016 12:34:09 +0200
Message-ID: <23955157.iZGXo4h7Qs@noys2>
Organization: Oracle Corporation
User-Agent: KMail/4.14.10 (Linux/4.5.1; KDE/4.14.14; x86_64; ; )
In-Reply-To: <20160407152234.GE32755@dhcp22.suse.cz>
References: <4482994.u2S3pScRyb@noys2> <20567553.kUaGmfXpqH@noys2> <20160407152234.GE32755@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Transfer-Encoding: 8BIT
Content-Type: text/plain; charset="utf-8"
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2651
Lines: 68

Hi Michal,

On Thursday 07 April 2016 17:22:35 Michal Hocko wrote:
> On Thu 07-04-16 15:45:02, Frank Mehnert wrote:
> > On Wednesday 06 April 2016 17:33:43 Michal Hocko wrote:
> [...]
> 
> > > Do you map your pages to the userspace? If yes then vma with VM_IO or
> > > VM_PFNMAP should keep any attempt away from those pages.
> > 
> > Yes, such memory objects are also mapped to userland. Do you think that
> > VM_IO or VM_PFNMAP would guard against NUMA page migration?
> 
> Both auto numa and manual numa migration checks vma_migratable and that
> excludes both VM flags.
> 
> > Because when
> > NUMA page migration was introduced (I believe with Linux 3.8) I tested
> > both flags and saw that they didn't prevent the migration on such VM
> > areas. Maybe this changed in the meantime, do you have more information
> > about that?
> 
> I haven't checked the history much but vma_migratable should be there
> for quite some time. Maybe it wasn't used in the past. Dunno

I did some further tests and indeed, with Linux 3.8 ... Linux 3.12 I was
able to reproduce NUMA page faults while with Linux 3.14 (3.13 didn't run
for some reason on my hardware) I'm no longer able to reproduce NUMA page
faults. The important point is that with Linux 3.8, all pages are unmapped
from time to time and in the page fault handler a decision is made if the
page should be migrated to another NUMA node or not. So even if
vma_migratable() returns FALSE it wouldn't help us as the page has already
faulted.

But as I said, the behaviour with Linux >= 3.14 is different which helps
us a lot!

> > The drawback of at least VM_IO is that such memory is not part of a core
> > dump.
> 
> that seems to be correct as per vma_dump_size
> 
> > Actually currently we use vm_insert_page() for userland mapping
> > and mark the VM areas as
> > 
> >   VM_DONTEXPAND | VM_DONTDUMP
> 
> but that means that it won't end up in the dump either. Or am I missing
> your point.

I guess you are right and we probably don't get these pages into core dumps
either. We can live with that.

Thank you for your suggestions and explanations, it was very helpful!

Frank
-- 
Dr.-Ing. Frank Mehnert | Software Development Director, VirtualBox
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt, Germany

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher