Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752275AbeAQKIy (ORCPT + 1 other); Wed, 17 Jan 2018 05:08:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50218 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750816AbeAQKIw (ORCPT ); Wed, 17 Jan 2018 05:08:52 -0500 Date: Wed, 17 Jan 2018 18:08:48 +0800 From: Baoquan He To: Dou Liyang Cc: linux-kernel@vger.kernel.org, mingo@redhat.com, tglx@linutronix.de, hpa@zytor.com, x86@kernel.org, rostedt@goodmis.org, jgross@suse.com, peterz@infradead.org, uobergfe@redhat.com, joro@8bytes.org Subject: Re: [RESEND PATCH 2/3] x86/apic/kexec: Enable legacy irq mode before jump to kexec/kdump kernel Message-ID: <20180117100848.GF2321@localhost.localdomain> References: <1515123732-28908-1-git-send-email-bhe@redhat.com> <20180105043838.GK7235@x1> <250edc45-03be-76bd-4ce2-f6d7f90d0c4e@cn.fujitsu.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <250edc45-03be-76bd-4ce2-f6d7f90d0c4e@cn.fujitsu.com> User-Agent: Mutt/1.9.1 (2017-09-22) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Wed, 17 Jan 2018 10:08:52 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: On 01/17/18 at 05:47pm, Dou Liyang wrote: > Hi Baoquan, > > At 01/05/2018 12:38 PM, Baoquan He wrote: > > In commit > > > > commit 522e66464467 ("x86/apic: Disable I/O APIC before shutdown of the local APIC"). > > > > lapic_shutdown() invocation is moved after disable_IO_APIC(). In fact > > in disable_IO_APIC(), it not only calls clear_IO_APIC() to disable > > IO-APIC, also sets sets LAPIC and IO-APIC to make system be PIC or > > Virtual wire mode. While the above commit putting disable_IO_APIC earlier > > causes local APIC is completely disabled. So the legacy irq mode is > > disabled too before jump to kexec/kdump kernel. > > > I have a question: > > As you said, Due to disable_IO_APIC() is triggered before > lapic_shutdown(), So the interrupt virtual wire mode will be disabled. > > but, I found that: > > After machine_crash_shutdown() is executed, Linux will call > machine_kexec(), and in machine_kexec(), disable_IO_APIC() will also be > called again, why it can't switch to virtual wire mode successfully? Or > is my understanding wrong? The disable_IO_APIC() calling has a condition check, if (image->preserve_context) { disable_IO_APIC(); } For preserve_context case, it comes from kernel_kexec(). You can check it in kexec man page, that is another scenario we use kexec for. But not kexec and kdump. > > +--------------+ > | __crash_kexec| > +--------------+ > | > | +-------------------------+ > +--> | machine_crash_shutdown | > | ++------------------------+ > | | > | | +-----------------+ > | +> | disable_IO_APIC | > | | +-----------------+ > | | > | | +----------------+ > | +-^+ lapic_shutdown | > | +----------------+ > | > | +-------------------------+ > +--> | machine_kexec | > | ++------------------------+ > | | > | | +-----------------+ > | +> | disable_IO_APIC | > | +-----------------+ > | > v > > Thanks, > dou. > > In normal kernel it defaults to be PIC mode or Virtual Wire mode during > > system initialization before APIC mode is enabled and this is done by > > BIOS initialization. But kexec/kdump kernel won't go through BIOS, so > > we should set system as PIC or Virtual Wire mode before jump to kdump > > kernel code directly. > > > > So let's take clear_IO_APIC out from disable_IO_APIC and rename > > disable_IO_APIC as switch_to_legacy_irq_mode. Then only call clear_IO_APIC > > when IO-APIC need be disabled. And call switch_to_legacy_irq_mode before > > kexec/kdump jumping. > > > > Signed-off-by: Baoquan He > > --- > > arch/x86/include/asm/io_apic.h | 3 ++- > > arch/x86/kernel/apic/io_apic.c | 12 ++++-------- > > arch/x86/kernel/crash.c | 2 +- > > arch/x86/kernel/machine_kexec_32.c | 15 +++++---------- > > arch/x86/kernel/machine_kexec_64.c | 15 +++++---------- > > arch/x86/kernel/reboot.c | 2 +- > > 6 files changed, 18 insertions(+), 31 deletions(-) > > > > diff --git a/arch/x86/include/asm/io_apic.h b/arch/x86/include/asm/io_apic.h > > index a8834dd546cd..e38ad3863a2c 100644 > > --- a/arch/x86/include/asm/io_apic.h > > +++ b/arch/x86/include/asm/io_apic.h > > @@ -192,7 +192,8 @@ static inline unsigned int io_apic_read(unsigned int apic, unsigned int reg) > > extern void setup_IO_APIC(void); > > extern void enable_IO_APIC(void); > > -extern void disable_IO_APIC(void); > > +extern void clear_IO_APIC (void); > > +extern void switch_to_legacy_irq_mode(void); > > extern int IO_APIC_get_PCI_irq_vector(int bus, int devfn, int pin); > > extern void print_IO_APICs(void); > > #else /* !CONFIG_X86_IO_APIC */ > > diff --git a/arch/x86/kernel/apic/io_apic.c b/arch/x86/kernel/apic/io_apic.c > > index 8a7963421460..a47aa915d18c 100644 > > --- a/arch/x86/kernel/apic/io_apic.c > > +++ b/arch/x86/kernel/apic/io_apic.c > > @@ -587,7 +587,7 @@ static void clear_IO_APIC_pin(unsigned int apic, unsigned int pin) > > mpc_ioapic_id(apic), pin); > > } > > -static void clear_IO_APIC (void) > > +void clear_IO_APIC (void) > > { > > int apic, pin; > > @@ -1439,15 +1439,11 @@ void native_disable_io_apic(void) > > } > > /* > > - * Not an __init, needed by the reboot code > > + * Not an __init, needed by kexec/kdump code. > > + * For safety IO-APIC and Local APIC need be cleared before this. > > */ > > -void disable_IO_APIC(void) > > +void switch_to_legacy_irq_mode(void) > > { > > - /* > > - * Clear the IO-APIC before rebooting: > > - */ > > - clear_IO_APIC(); > > - > > if (!nr_legacy_irqs()) > > return; > > diff --git a/arch/x86/kernel/crash.c b/arch/x86/kernel/crash.c > > index 10e74d4778a1..318ffeaaf55a 100644 > > --- a/arch/x86/kernel/crash.c > > +++ b/arch/x86/kernel/crash.c > > @@ -199,7 +199,7 @@ void native_machine_crash_shutdown(struct pt_regs *regs) > > #ifdef CONFIG_X86_IO_APIC > > /* Prevent crash_kexec() from deadlocking on ioapic_lock. */ > > ioapic_zap_locks(); > > - disable_IO_APIC(); > > + clear_IO_APIC(); > > #endif > > lapic_shutdown(); > > #ifdef CONFIG_HPET_TIMER > > diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_kexec_32.c > > index edfede768688..7ab10d930cc6 100644 > > --- a/arch/x86/kernel/machine_kexec_32.c > > +++ b/arch/x86/kernel/machine_kexec_32.c > > @@ -190,18 +190,13 @@ void machine_kexec(struct kimage *image) > > local_irq_disable(); > > hw_breakpoint_disable(); > > - if (image->preserve_context) { > > #ifdef CONFIG_X86_IO_APIC > > - /* > > - * We need to put APICs in legacy mode so that we can > > - * get timer interrupts in second kernel. kexec/kdump > > - * paths already have calls to disable_IO_APIC() in > > - * one form or other. kexec jump path also need > > - * one. > > - */ > > - disable_IO_APIC(); > > + /* > > + * We need to put APICs in legacy mode so that we can > > + * get timer interrupts in second kernel. > > + */ > > + switch_to_legacy_irq_mode(); > > #endif > > - } > > control_page = page_address(image->control_code_page); > > memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE); > > diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c > > index 1f790cf9d38f..b5c0cbed6791 100644 > > --- a/arch/x86/kernel/machine_kexec_64.c > > +++ b/arch/x86/kernel/machine_kexec_64.c > > @@ -288,18 +288,13 @@ void machine_kexec(struct kimage *image) > > local_irq_disable(); > > hw_breakpoint_disable(); > > - if (image->preserve_context) { > > #ifdef CONFIG_X86_IO_APIC > > - /* > > - * We need to put APICs in legacy mode so that we can > > - * get timer interrupts in second kernel. kexec/kdump > > - * paths already have calls to disable_IO_APIC() in > > - * one form or other. kexec jump path also need > > - * one. > > - */ > > - disable_IO_APIC(); > > + /* > > + * We need to put APICs in legacy mode so that we can > > + * get timer interrupts in second kernel. > > + */ > > + switch_to_legacy_irq_mode(); > > #endif > > - } > > control_page = page_address(image->control_code_page) + PAGE_SIZE; > > memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE); > > diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c > > index 2126b9d27c34..b70cc0f38a29 100644 > > --- a/arch/x86/kernel/reboot.c > > +++ b/arch/x86/kernel/reboot.c > > @@ -666,7 +666,7 @@ void native_machine_shutdown(void) > > * Even without the erratum, it still makes sense to quiet IO APIC > > * before disabling Local APIC. > > */ > > - disable_IO_APIC(); > > + clear_IO_APIC(); > > #endif > > #ifdef CONFIG_SMP > > > >