LinuxLists.cc - [PATCH] mm: don't defer struct page initialization for Xen pv guests

2018-02-16 19:10:56

Subject: [PATCH] mm: don't defer struct page initialization for Xen pv guests

Commit f7f99100d8d95dbcf09e0216a143211e79418b9f ("mm: stop zeroing
memory during allocation in vmemmap") broke Xen pv domains in some
configurations, as the "Pinned" information in struct page of early
page tables could get lost.

Avoid this problem by not deferring struct page initialization when
running as Xen pv guest.

Cc: <[email protected]> #4.15
Signed-off-by: Juergen Gross <[email protected]>
---
mm/page_alloc.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 81e18ceef579..681d504b9a40 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -347,6 +347,9 @@ static inline bool update_defer_init(pg_data_t *pgdat,
/* Always populate low zones for address-constrained allocations */
if (zone_end < pgdat_end_pfn(pgdat))
return true;
+ /* Xen PV domains need page structures early */
+ if (xen_pv_domain())
+ return true;
(*nr_initialised)++;
if ((*nr_initialised > pgdat->static_init_pgcnt) &&
(pfn & (PAGES_PER_SECTION - 1)) == 0) {
--
2.13.6

2018-02-16 19:11:07

by Michal Hocko

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

[CC Pavel]

On Fri 16-02-18 14:37:26, Juergen Gross wrote:
> Commit f7f99100d8d95dbcf09e0216a143211e79418b9f ("mm: stop zeroing
> memory during allocation in vmemmap") broke Xen pv domains in some
> configurations, as the "Pinned" information in struct page of early
> page tables could get lost.

Could you be more specific please?

> Avoid this problem by not deferring struct page initialization when
> running as Xen pv guest.
>
> Cc: <[email protected]> #4.15
Fixes: f7f99100d8d9 ("mm: stop zeroing memory during allocation in vmemmap")

please

> Signed-off-by: Juergen Gross <[email protected]>
> ---
> mm/page_alloc.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 81e18ceef579..681d504b9a40 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -347,6 +347,9 @@ static inline bool update_defer_init(pg_data_t *pgdat,
> /* Always populate low zones for address-constrained allocations */
> if (zone_end < pgdat_end_pfn(pgdat))
> return true;
> + /* Xen PV domains need page structures early */
> + if (xen_pv_domain())
> + return true;
> (*nr_initialised)++;
> if ((*nr_initialised > pgdat->static_init_pgcnt) &&
> (pfn & (PAGES_PER_SECTION - 1)) == 0) {
> --
> 2.13.6

--
Michal Hocko
SUSE Labs

2018-02-16 19:11:23

by Jürgen Groß

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

On 16/02/18 14:59, Michal Hocko wrote:
> [CC Pavel]
>
> On Fri 16-02-18 14:37:26, Juergen Gross wrote:
>> Commit f7f99100d8d95dbcf09e0216a143211e79418b9f ("mm: stop zeroing
>> memory during allocation in vmemmap") broke Xen pv domains in some
>> configurations, as the "Pinned" information in struct page of early
>> page tables could get lost.
>
> Could you be more specific please?

In which way? Do you want to see the resulting crash in the commit
message or some more background information?

>
>> Avoid this problem by not deferring struct page initialization when
>> running as Xen pv guest.
>>
>> Cc: <[email protected]> #4.15
> Fixes: f7f99100d8d9 ("mm: stop zeroing memory during allocation in vmemmap")
>
> please

Okay.

Juergen

2018-02-16 19:14:02

by Michal Hocko

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

On Fri 16-02-18 15:02:17, Juergen Gross wrote:
> On 16/02/18 14:59, Michal Hocko wrote:
> > [CC Pavel]
> >
> > On Fri 16-02-18 14:37:26, Juergen Gross wrote:
> >> Commit f7f99100d8d95dbcf09e0216a143211e79418b9f ("mm: stop zeroing
> >> memory during allocation in vmemmap") broke Xen pv domains in some
> >> configurations, as the "Pinned" information in struct page of early
> >> page tables could get lost.
> >
> > Could you be more specific please?
>
> In which way? Do you want to see the resulting crash in the commit
> message or some more background information?

ideally both.
--
Michal Hocko
SUSE Labs

2018-02-16 19:21:28

by Pavel Tatashin

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

On 02/16/2018 09:02 AM, Juergen Gross wrote:
> On 16/02/18 14:59, Michal Hocko wrote:
>> [CC Pavel]
>>
>> On Fri 16-02-18 14:37:26, Juergen Gross wrote:
>>> Commit f7f99100d8d95dbcf09e0216a143211e79418b9f ("mm: stop zeroing
>>> memory during allocation in vmemmap") broke Xen pv domains in some
>>> configurations, as the "Pinned" information in struct page of early
>>> page tables could get lost.
>>
>> Could you be more specific please?
>
> In which way? Do you want to see the resulting crash in the commit
> message or some more background information?

Hi Juergen,

Could you please provide details how to reproduce the problem, and crash
message. Also, information about where exactly xen access struct pages
before deferred pages are initialized.

Thank you,
Pasha

>
>>
>>> Avoid this problem by not deferring struct page initialization when
>>> running as Xen pv guest.
>>>
>>> Cc: <[email protected]> #4.15
>> Fixes: f7f99100d8d9 ("mm: stop zeroing memory during allocation in vmemmap")
>>
>> please
>
> Okay.
>
>
> Juergen
>

2018-02-18 16:26:18

by kernel test robot

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

Attachments:

(No filename) (2.15 kB)
.config.gz (26.81 kB)
Download all attachments

2018-02-18 18:47:26

by kernel test robot

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

Hi Juergen,

I love your patch! Yet something to improve:

[auto build test ERROR on mmotm/master]
[also build test ERROR on v4.16-rc1 next-20180216]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url: https://github.com/0day-ci/linux/commits/Juergen-Gross/mm-don-t-defer-struct-page-initialization-for-Xen-pv-guests/20180218-233657
base: git://git.cmpxchg.org/linux-mmotm.git master
config: i386-randconfig-x010-201807 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386

All errors (new ones prefixed by >>):

mm/page_alloc.c: In function 'update_defer_init':
>> mm/page_alloc.c:352:6: error: implicit declaration of function 'xen_pv_domain' [-Werror=implicit-function-declaration]
if (xen_pv_domain())
^~~~~~~~~~~~~
Cyclomatic Complexity 5 include/linux/compiler.h:__read_once_size
Cyclomatic Complexity 5 include/linux/compiler.h:__write_once_size
Cyclomatic Complexity 2 arch/x86/include/asm/bitops.h:set_bit
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:__set_bit
Cyclomatic Complexity 2 arch/x86/include/asm/bitops.h:clear_bit
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:__clear_bit
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:constant_test_bit
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:variable_test_bit
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:__ffs
Cyclomatic Complexity 1 arch/x86/include/asm/bitops.h:fls
Cyclomatic Complexity 1 arch/x86/include/asm/arch_hweight.h:__arch_hweight32
Cyclomatic Complexity 2 include/linux/bitops.h:hweight_long
Cyclomatic Complexity 1 include/linux/bitops.h:fls_long
Cyclomatic Complexity 1 include/linux/log2.h:__ilog2_u32
Cyclomatic Complexity 3 include/linux/log2.h:is_power_of_2
Cyclomatic Complexity 1 include/linux/log2.h:__roundup_pow_of_two
Cyclomatic Complexity 1 include/linux/log2.h:__rounddown_pow_of_two
Cyclomatic Complexity 1 include/linux/list.h:INIT_LIST_HEAD
Cyclomatic Complexity 1 include/linux/list.h:__list_del
Cyclomatic Complexity 1 include/linux/list.h:list_empty
Cyclomatic Complexity 1 arch/x86/include/asm/current.h:get_current
Cyclomatic Complexity 3 include/linux/string.h:memset
Cyclomatic Complexity 1 arch/x86/include/asm/page_32.h:clear_page
Cyclomatic Complexity 1 include/asm-generic/getorder.h:__get_order
Cyclomatic Complexity 3 include/linux/bitmap.h:bitmap_weight
Cyclomatic Complexity 1 arch/x86/include/asm/atomic.h:atomic_read
Cyclomatic Complexity 1 arch/x86/include/asm/atomic.h:atomic_set
Cyclomatic Complexity 1 arch/x86/include/asm/atomic.h:atomic_add
Cyclomatic Complexity 1 arch/x86/include/asm/atomic.h:atomic_sub_and_test
Cyclomatic Complexity 1 arch/x86/include/asm/atomic.h:atomic_dec_and_test
Cyclomatic Complexity 1 include/asm-generic/atomic-long.h:atomic_long_read
Cyclomatic Complexity 1 include/linux/jump_label.h:static_key_count
Cyclomatic Complexity 2 include/linux/jump_label.h:static_key_false
Cyclomatic Complexity 1 arch/x86/include/asm/paravirt.h:arch_local_save_flags
Cyclomatic Complexity 1 arch/x86/include/asm/paravirt.h:arch_local_irq_restore
Cyclomatic Complexity 1 arch/x86/include/asm/paravirt.h:arch_local_irq_disable
Cyclomatic Complexity 1 arch/x86/include/asm/paravirt.h:arch_local_irq_save
Cyclomatic Complexity 1 include/linux/err.h:PTR_ERR
Cyclomatic Complexity 1 arch/x86/include/asm/processor.h:rep_nop
Cyclomatic Complexity 1 arch/x86/include/asm/processor.h:cpu_relax
Cyclomatic Complexity 1 arch/x86/include/asm/processor.h:prefetchw
Cyclomatic Complexity 1 arch/x86/include/asm/preempt.h:preempt_count
Cyclomatic Complexity 5 arch/x86/include/asm/preempt.h:__preempt_count_add
Cyclomatic Complexity 1 arch/x86/include/asm/preempt.h:__preempt_count_dec_and_test
Cyclomatic Complexity 1 arch/x86/include/asm/preempt.h:should_resched
Cyclomatic Complexity 1 include/linux/spinlock.h:spinlock_check
Cyclomatic Complexity 1 include/linux/spinlock.h:spin_lock
Cyclomatic Complexity 1 include/linux/spinlock.h:spin_unlock
Cyclomatic Complexity 1 include/linux/spinlock.h:spin_unlock_irqrestore
Cyclomatic Complexity 1 include/linux/nodemask.h:__nodemask_pr_numnodes
Cyclomatic Complexity 2 include/linux/nodemask.h:__nodemask_pr_bits
Cyclomatic Complexity 1 include/linux/nodemask.h:__nodes_weight
Cyclomatic Complexity 1 include/linux/nodemask.h:node_state
Cyclomatic Complexity 1 include/linux/nodemask.h:node_set_state
Cyclomatic Complexity 1 include/linux/nodemask.h:num_node_state
Cyclomatic Complexity 1 include/linux/mmzone.h:zone_end_pfn
Cyclomatic Complexity 3 include/linux/mmzone.h:zone_spans_pfn
Cyclomatic Complexity 1 include/linux/mmzone.h:zone_is_initialized
Cyclomatic Complexity 1 include/linux/mmzone.h:node_lruvec
Cyclomatic Complexity 1 include/linux/mmzone.h:pgdat_end_pfn
Cyclomatic Complexity 1 include/linux/rcupdate.h:rcu_read_lock_sched_notrace
Cyclomatic Complexity 1 include/linux/debugobjects.h:debug_check_no_obj_freed
Cyclomatic Complexity 1 include/linux/workqueue.h:__init_work
Cyclomatic Complexity 1 include/linux/memory_hotplug.h:pgdat_resize_init
Cyclomatic Complexity 1 include/linux/memory_hotplug.h:zone_span_seqbegin
Cyclomatic Complexity 1 include/linux/memory_hotplug.h:zone_span_seqretry
Cyclomatic Complexity 1 include/linux/memory_hotplug.h:zone_seqlock_init
Cyclomatic Complexity 1 include/linux/memory_hotplug.h:movable_node_is_enabled
Cyclomatic Complexity 1 include/linux/mmzone.h:managed_zone
Cyclomatic Complexity 1 include/linux/mmzone.h:populated_zone
Cyclomatic Complexity 1 include/linux/mmzone.h:is_highmem_idx
Cyclomatic Complexity 1 include/linux/mmzone.h:is_highmem
Cyclomatic Complexity 1 include/linux/mmzone.h:zonelist_zone
Cyclomatic Complexity 1 include/linux/mmzone.h:zonelist_zone_idx
Cyclomatic Complexity 3 include/linux/mmzone.h:next_zones_zonelist
Cyclomatic Complexity 1 include/linux/mmzone.h:first_zones_zonelist
Cyclomatic Complexity 1 include/linux/mmzone.h:pfn_to_section_nr
Cyclomatic Complexity 2 include/linux/mmzone.h:__nr_to_section
Cyclomatic Complexity 1 include/linux/mmzone.h:__section_mem_map_addr
Cyclomatic Complexity 3 include/linux/mmzone.h:valid_section
Cyclomatic Complexity 1 include/linux/mmzone.h:__pfn_to_section
Cyclomatic Complexity 2 include/linux/mmzone.h:pfn_valid
Cyclomatic Complexity 1 arch/x86/include/asm/topology.h:numa_node_id
Cyclomatic Complexity 1 include/linux/topology.h:numa_mem_id
Cyclomatic Complexity 1 include/linux/gfp.h:gfp_zonelist
Cyclomatic Complexity 1 include/linux/gfp.h:node_zonelist
Cyclomatic Complexity 1 include/linux/gfp.h:arch_free_page
Cyclomatic Complexity 1 include/linux/gfp.h:arch_alloc_page
Cyclomatic Complexity 1 include/linux/gfp.h:pm_suspended_storage
Cyclomatic Complexity 1 include/linux/page-flags.h:PageTail
Cyclomatic Complexity 3 include/linux/page-flags.h:PageCompound
Cyclomatic Complexity 1 include/linux/page-flags.h:PageLRU
Cyclomatic Complexity 1 include/linux/page-flags.h:PageReserved
Cyclomatic Complexity 1 include/linux/page-flags.h:SetPageReserved
Cyclomatic Complexity 1 include/linux/page-flags.h:ClearPageReserved
Cyclomatic Complexity 1 include/linux/page-flags.h:__ClearPageReserved
Cyclomatic Complexity 1 include/linux/page-flags.h:PageHighMem
Cyclomatic Complexity 1 include/linux/page-flags.h:PageHWPoison

vim +/xen_pv_domain +352 mm/page_alloc.c

339
340 /*
341 * Returns false when the remaining initialisation should be deferred until
342 * later in the boot cycle when it can be parallelised.
343 */
344 static inline bool update_defer_init(pg_data_t *pgdat,
345 unsigned long pfn, unsigned long zone_end,
346 unsigned long *nr_initialised)
347 {
348 /* Always populate low zones for address-constrained allocations */
349 if (zone_end < pgdat_end_pfn(pgdat))
350 return true;
351 /* Xen PV domains need page structures early */
> 352 if (xen_pv_domain())
353 return true;
354 (*nr_initialised)++;
355 if ((*nr_initialised > pgdat->static_init_pgcnt) &&
356 (pfn & (PAGES_PER_SECTION - 1)) == 0) {
357 pgdat->first_deferred_pfn = pfn;
358 return false;
359 }
360
361 return true;
362 }
363 #else
364 static inline void reset_deferred_meminit(pg_data_t *pgdat)
365 {
366 }
367

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

Attachments:

(No filename) (8.73 kB)
.config.gz (33.35 kB)
Download all attachments

2018-02-20 23:22:53

by Andrew Morton

[permalink] [raw]

Subject: Re: [PATCH] mm: don't defer struct page initialization for Xen pv guests

On Mon, 19 Feb 2018 02:45:27 +0800 kbuild test robot <[email protected]> wrote:

> [auto build test ERROR on mmotm/master]
> [also build test ERROR on v4.16-rc1 next-20180216]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url: https://github.com/0day-ci/linux/commits/Juergen-Gross/mm-don-t-defer-struct-page-initialization-for-Xen-pv-guests/20180218-233657
> base: git://git.cmpxchg.org/linux-mmotm.git master
> config: i386-randconfig-x010-201807 (attached as .config)
> compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
> reproduce:
> # save the attached .config to linux build tree
> make ARCH=i386
>
> All errors (new ones prefixed by >>):
>
> mm/page_alloc.c: In function 'update_defer_init':
> >> mm/page_alloc.c:352:6: error: implicit declaration of function 'xen_pv_domain' [-Werror=implicit-function-declaration]
> if (xen_pv_domain())
> ^~~~~~~~~~~~~

I think I already fixed this.

From: Andrew Morton <[email protected]>
Subject: mm-dont-defer-struct-page-initialization-for-xen-pv-guests-fix

explicitly include xen.h

Cc: Juergen Gross <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
---

mm/page_alloc.c | 1 +
1 file changed, 1 insertion(+)

diff -puN mm/page_alloc.c~mm-dont-defer-struct-page-initialization-for-xen-pv-guests-fix mm/page_alloc.c
--- a/mm/page_alloc.c~mm-dont-defer-struct-page-initialization-for-xen-pv-guests-fix
+++ a/mm/page_alloc.c
@@ -46,6 +46,7 @@
#include <linux/stop_machine.h>
#include <linux/sort.h>
#include <linux/pfn.h>
+#include <xen/xen.h>
#include <linux/backing-dev.h>
#include <linux/fault-inject.h>
#include <linux/page-isolation.h>
_