Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Michael Ellerman <mpe@ellerman.id.au>
To:     David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc:     linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org,
        David Hildenbrand <david@redhat.com>,
        Nathan Fontenot <nfont@linux.vnet.ibm.com>,
        Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        Paul Mackerras <paulus@samba.org>,
        Michal Hocko <mhocko@suse.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Oscar Salvador <osalvador@suse.de>,
        Baoquan He <bhe@redhat.com>,
        Wei Yang <richard.weiyang@gmail.com>
Subject: Re: [PATCH v1 1/2] powerpc/pseries/hotplug-memory: stop checking is_mem_section_removable()
In-Reply-To: <20200407135416.24093-2-david@redhat.com>
References: <20200407135416.24093-1-david@redhat.com> <20200407135416.24093-2-david@redhat.com>
Date:   Thu, 09 Apr 2020 17:26:01 +1000
Message-ID: <87sghdjf1y.fsf@mpe.ellerman.id.au>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

David Hildenbrand <david@redhat.com> writes:

> In commit 53cdc1cb29e8 ("drivers/base/memory.c: indicate all memory
> blocks as removable"), the user space interface to compute whether a memory
> block can be offlined (exposed via
> /sys/devices/system/memory/memoryX/removable) has effectively been
> deprecated. We want to remove the leftovers of the kernel implementation.
>
> When offlining a memory block (mm/memory_hotplug.c:__offline_pages()),
> we'll start by:
> 1. Testing if it contains any holes, and reject if so
> 2. Testing if pages belong to different zones, and reject if so
> 3. Isolating the page range, checking if it contains any unmovable pages
>
> Using is_mem_section_removable() before trying to offline is not only racy,
> it can easily result in false positives/negatives. Let's stop manually
> checking is_mem_section_removable(), and let device_offline() handle it
> completely instead. We can remove the racy is_mem_section_removable()
> implementation next.
>
> We now take more locks (e.g., memory hotplug lock when offlining and the
> zone lock when isolating), but maybe we should optimize that
> implementation instead if this ever becomes a real problem (after all,
> memory unplug is already an expensive operation). We started using
> is_mem_section_removable() in commit 51925fb3c5c9 ("powerpc/pseries:
> Implement memory hotplug remove in the kernel"), with the initial
> hotremove support of lmbs.

It's also not very pretty in dmesg.

Before:

  pseries-hotplug-mem: Attempting to hot-add 10 LMB(s)
  pseries-hotplug-mem: Memory hot-add failed, removing any added LMBs
  dlpar: Could not handle DLPAR request "memory add count 10"

After:

  pseries-hotplug-mem: Attempting to hot-remove 10 LMB(s)
  page:c00c000001ca8200 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc00000072a080180
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 c00c000001cffd48 c000000781c51410 c000000793327580
  raw: c00000072a080180 0000000001550001 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001cc4a80 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc00000079b110080
  flags: 0x7ffff000000000()
  raw: 007ffff000000000 5deadbeef0000100 5deadbeef0000122 0000000000000000
  raw: c00000079b110080 0000000000000000 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001d08200 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc00000074208ff00
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 c000000781c5f150 c00c000001d37f88 c000000798a24880
  raw: c00000074208ff00 0000000001550002 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001d40140 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc000000750059c00
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 c00c000001dfcfc8 c00c000001d3c288 c0000007851c2d00
  raw: c000000750059c00 0000000001000003 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001d9c000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc000000002370080
  flags: 0x7ffff000000000()
  raw: 007ffff000000000 5deadbeef0000100 5deadbeef0000122 0000000000000000
  raw: c000000002370080 0000000000000000 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001dc0200 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc000000002370080
  flags: 0x7ffff000000000()
  raw: 007ffff000000000 5deadbeef0000100 5deadbeef0000122 0000000000000000
  raw: c000000002370080 0000000000000000 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001e00000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0x0
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 5deadbeef0000100 5deadbeef0000122 c0000007a801f500
  raw: 0000000000000000 0000000008000800 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001e40440 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0x0
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 5deadbeef0000100 5deadbeef0000122 c0000007a801e380
  raw: 0000000000000000 0000000000400040 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000001e80000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc0000007a0000640
  flags: 0x7ffff000000200(slab)
  raw: 007ffff000000200 c00c000001e5af48 c00c000001e80408 c000000f42d00a00
  raw: c0000007a0000640 00000000066600a2 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000003c89d40 refcount:2 mapcount:1 mapping:0000000018c4a547 index:0x10a41
  anon flags: 0x17ffff000080024(uptodate|active|swapbacked)
  raw: 017ffff000080024 5deadbeef0000100 5deadbeef0000122 c0000007986b03c9
  raw: 0000000000010a41 0000000000000000 0000000200000000 c00000000340b000
  page dumped because: unmovable page
  page->mem_cgroup:c00000000340b000
  page:c00c000003cc0000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc000000f3000fd38
  flags: 0x17ffff000000200(slab)
  raw: 017ffff000000200 c000000f3c911890 c000000f3c911890 c00000079fffd980
  raw: c000000f3000fd38 0000000000700003 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000003d00000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc0000007a2ec0000
  flags: 0x17ffff000000000()
  raw: 017ffff000000000 5deadbeef0000100 5deadbeef0000122 0000000000000000
  raw: c0000007a2ec0000 0000000000000000 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000003e2c000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0xc000000f8b008400
  flags: 0x27ffff000000200(slab)
  raw: 027ffff000000200 c000000f8e000190 c000000f8e000190 c0000007a801e380
  raw: c000000f8b008400 0000000000400038 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  page:c00c000003fec000 refcount:1 mapcount:0 mapping:0000000018c4a547 index:0x0
  flags: 0x37ffff000000000()
  raw: 037ffff000000000 5deadbeef0000100 5deadbeef0000122 0000000000000000
  raw: 0000000000000000 0000000000000000 00000001ffffffff 0000000000000000
  page dumped because: unmovable page
  pseries-hotplug-mem: Memory hot-remove failed, adding LMB's back
  dlpar: Could not handle DLPAR request "memory remove count 10"


It looks like set_migratetype_isolate() and start_isolate_page_range()
can be told not to report those warnings, but we're just calling
device_offline() which doesn't let us specify that.

cheers