Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
From:   Maxim Levitsky <maximlevitsky@gmail.com>
Date:   Sat, 4 Nov 2017 11:55:14 +0200
Message-ID: <CACAwPwY0owut+314c5sy7jNViZqfrKy3sSf1hjLTocXefrz3xA@mail.gmail.com>
Subject: Guaranteed allocation of huge pages (1G) using movablecore=N doesn't
 seem to work at all
To:     LKML <linux-kernel@vger.kernel.org>
Cc:     linux-mm@kvack.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi!

My system has 64G of ram and I want to create 32 1G huge pages to use
in KVM virtualization,
on demand, only when VM is running.

So I booted the kernel with
'hugepagesz=1G hugepages=0 default_hugepagesz=1G movablecore=40G'

However I still can't allocate the pages reliably.
For instance this simple script is enough to make it not possible to
even allocate one 1G huge page after few dozens of iterations:

while true ; do
    sudo hugeadm  --enable-zone-movable  --pool-pages-min 1G:0G
    sudo hugeadm  --enable-zone-movable  --pool-pages-min 1G:60G
done


I disabled mlock systemwide (now ulimit -l shows 0), I still see 8
pages mlocked in  zone 'Movable' but this is not enough to explain
this
nr_mlock     8

I do have around 64GB of swap too, but I see no even an attempt to use it.

# free
              total        used        free      shared  buff/cache   available
Mem:       65887928     1748344    62640276       61688     1499308    62053832
Swap:      67108860           0    67108860

Any idea about what is going on?

This was tested on 4.14.0-rc5 (my custom compiled) and on several
older kernels (4.10,4.12,4.13) from ubuntu repositories.

Disabling/enabling transparent huge pages in the kernel config didn't
make a difference.

VT-d was enabled during the tests (intel_iommu=on,igfx_off) if that
would make any difference, but no VM was started when I run the above
script, in fact I run it just after the system booted.

Best regards,
          Maxim Levitsky

From 1583358629196455983@xxx Mon Nov 06 22:47:38 +0000 2017
X-GM-THRID: 1581728527965537224
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread