Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp1812357imm; Thu, 27 Sep 2018 02:58:12 -0700 (PDT) X-Google-Smtp-Source: ACcGV623R578Q0F33lL9tZFwzO3exrSR2R3DaUaxlnpSNXP+LiDdNZ5IS1kVtc+O7Bw9xszzXVcJ X-Received: by 2002:aa7:8118:: with SMTP id b24-v6mr10679980pfi.78.1538042292704; Thu, 27 Sep 2018 02:58:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1538042292; cv=none; d=google.com; s=arc-20160816; b=Jq8+FrN5j45cUi2rvXuRJYJtFcqm3JsYjleoLcvexypz6UWrI9Eyy5/RaHtHEwT1rQ 3es1TzlX3DtraMRZVHgubjHgFgedIYbDUHhIlFjwQ7eSVogsYJyCvYXBQ+uov5GBZ1uN EvbOh2zX+6UHyhxXHXCHgWF4LLglbOfKB1C/Z0azDwHOnBqCUrZQOyhVpheofjzaupM4 Kir5FFc+2BZXzm8q6B7zF0EEYJK2xm342c9c1LLCw6yOp1RLoZ0s6kHtePpY9aCEyADC PLO9o0JSbs2OgnafK8esilScESBt6J5dP1FW7akOc08DIdtHY8jDApShzqjNlqULANoa 8HDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=8H/NtF82T10HJZRWM/CboNvwMhwu/yGxl8dgUwRJjvc=; b=kezcyGS1ZIug+hx7R8kcvuXP2Zhs2AXrHpn7NfDqyvYmiMZTKfnVaTDrzda5PYNYjd z46JwclqcLbpsFBc/i6L7nDjjBszkt1rbIB1usS1t5GwEZwzTYvigOgjQCs7e+zCONa0 Xmdj7vvFj/uqDL+FylPj8U8J2xjQ6RjDiA9Hysbpwysgf1byRvvDpOLB1ImzsUQGHzll M4lXD+hC7N5tah58JHu0n5cVhR4JvGhKZU12ZYXkK2WtqkrgOdUjD9eyr3JTJ9RPCnXv gfLXPcYmG0OfcpVm1aYzoOb+/eH7YSRKCBIoSRpdobhmfUZO1w8EITSc9C/HmatQpGSw 9oyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q14-v6si1566597pgk.346.2018.09.27.02.57.57; Thu, 27 Sep 2018 02:58:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727358AbeI0QMy (ORCPT + 99 others); Thu, 27 Sep 2018 12:12:54 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:37955 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727018AbeI0QMy (ORCPT ); Thu, 27 Sep 2018 12:12:54 -0400 Received: by mail-pf1-f196.google.com with SMTP id x17-v6so1533869pfh.5 for ; Thu, 27 Sep 2018 02:55:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=8H/NtF82T10HJZRWM/CboNvwMhwu/yGxl8dgUwRJjvc=; b=jPHwoEtVaFsH1V4sOr7SSPwNah2B+LNxTbNAUpgfClpbuG6vu6UW+MCP5VHbjDCrM6 /1PlUv5M/Psm2ip5G4FvRWcfd1NWIdLpy5Q6+mBS+Yw1nQddb4QmXUzEgVLzqovJz7B5 iijzYSXd2e7vBswQed+8B3QtosgIsAOjXrYH4Pu3dsF22CCxavOrIKRk+OwgJHI11Bax N0ch06QzAdEwSaecIzIyxWOWKHL36lhSWFzfkczX6CnmU5Qp5jyxI/m4OdbNtF710XFZ zg2Gw6fFxJMqYEwiVV9l1wIad32G5NQo9qmFsRey1T1zaGq/GdefnAOFBPubqMk6qs1z Qmsg== X-Gm-Message-State: ABuFfohcM5rc1ihzs7wfi3OMDHNE9V4C0y64seOaOgiJJ7897p2/ijGB SBGreT/gOh7fTvOuL2BPvU+lOw== X-Received: by 2002:a17:902:5e3:: with SMTP id f90-v6mr10372313plf.286.1538042125950; Thu, 27 Sep 2018 02:55:25 -0700 (PDT) Received: from [192.168.1.6] ([171.48.34.255]) by smtp.gmail.com with ESMTPSA id y69-v6sm4157719pfd.36.2018.09.27.02.55.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 Sep 2018 02:55:24 -0700 (PDT) Subject: Re: [PATCH 00/10] GICv3 support for kexec/kdump on EFI systems To: Marc Zyngier , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Jeffrey Hugo , Thomas Gleixner , Jason Cooper , Jeremy Linton , Ard Biesheuvel , Bhupesh SHARMA , bhsharma@redhat.com References: <20180921195954.21574-1-marc.zyngier@arm.com> From: Bhupesh Sharma Message-ID: <28640b87-007f-dc72-110a-a57179e2b76a@redhat.com> Date: Thu, 27 Sep 2018 15:25:17 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20180921195954.21574-1-marc.zyngier@arm.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Marc, On 09/22/2018 01:29 AM, Marc Zyngier wrote: > The GICv3 architecture has the remarkable feature that once LPI tables > have been assigned to redistributors and that LPI delivery is enabled, > there is no guarantee that LPIs can be turned off (and most > implementations do not allow it), nor can it be reprogrammed to use > other tables. > > This is a bit of a problem for kexec, where the secondary kernel > completely looses track of the previous allocations. If the secondary > kernel doesn't allocate the tables exactly the same way, no LPIs will > be delivered by the GIC (which continues to use the old tables), and > memory previously allocated for the pending tables will be slowly > corrupted, one bit at a time. > > The workaround for this is based on a series[1] by Ard Biesheuvel, > which adds the required infrastructure for memory reservations to be > passed from one kernel to another using an EFI table. > > This infrastructure is then used to register the allocation of GIC > tables with EFI, and allow the GIC driver to safely reuse the existing > programming if it detects that the tables have been correctly > registered. On non-EFI systems, there is not much we can do. > > This has been tested on a TX2 system both as a host and a guest. I'd > welcome additional testing of different HW. For convenience, I've > stashed a branch containing the whole thing at [2]. > > [1] https://marc.info/?l=linux-efi&m=153754757208163&w=2 > [2] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=irq/gicv3-kdump > > Marc Zyngier (10): > irqchip/gic-v3-its: Change initialization ordering for LPIs > irqchip/gic-v3-its: Consolidate LPI_PENDBASE_SZ usage > irqchip/gic-v3-its: Split property table clearing from allocation > irqchip/gic-v3-its: Move pending table allocation to init time > irqchip/gic-v3-its: Keep track of property table's PA and VA > irqchip/gic-v3-its: Allow use of pre-programmed LPI tables > irqchip/gic-v3-its: Use pre-programmed redistributor tables with kdump > kernels > irqchip/gic-v3-its: Check that all RDs have the same property table > irqchip/gic-v3-its: Register LPI tables with EFI config table > irqchip/gic-v3-its: Allow use of LPI tables in reserved memory > > drivers/irqchip/irq-gic-v3-its.c | 249 ++++++++++++++++++++++------- > drivers/irqchip/irq-gic-v3.c | 20 ++- > include/linux/irqchip/arm-gic-v3.h | 4 +- > 3 files changed, 208 insertions(+), 65 deletions(-) Thanks for the patchset. I can confirm that with Ard's patchset in [1] and this patchset applied on 'efi/next' branch, I see that the "Booted with LPIs enabled, memory probably corrupted" issue that I was seeing on gigabyte boards in kdump kernel is fixed. Here are some logs: without patchset applied: ========================= [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] GICv3: Distributor has no Range Selector support [ 0.000000] GICv3: no VLPI support, direct LPI support [ 0.000000] ITS [mem 0x801000020000-0x80100021ffff] [ 0.000000] ITS@0x0000801000020000: allocated 2097152 Devices @c1000000 (flat, esz 8, psz 64K, shr 1) [ 0.000000] GIC: using LPI property table @0x00000000c03b0000 [ 0.000000] GICv3: CPU0: found redistributor a region 0:0x0000801080140000 [ 0.000000] CPU0: Booted with LPIs enabled, memory probably corrupted [ 0.000000] CPU0: Failed to disable LPIs <..snip..> [ 198.702976] dracut-initqueue[298]: Warning: dracut-initqueue timeout - starting timeout scripts [ 199.332238] dracut-initqueue[298]: Warning: dracut-initqueue timeout - starting timeout scripts [ 199.922944] dracut-initqueue[298]: Warning: dracut-initqueue timeout - starting timeout scripts [ 200.512239] dracut-initqueue[298]: Warning: dracut-initqueue timeout - starting timeout scripts <..snip..> with patchset applied: ====================== [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] GICv3: Distributor has no Range Selector support [ 0.000000] GICv3: no VLPI support, direct LPI support [ 0.000000] GICv3: CPU0: found redistributor 109 region 0:0x0000801080320000 [ 0.000000] ITS [mem 0x801000020000-0x80100021ffff] [ 0.000000] ITS@0x0000801000020000: allocated 2097152 Devices @c1000000 (flat, esz 8, psz 64K, shr 1) [ 0.000000] GICv3: Using preallocated redistributor tables [ 0.000000] GICv3: using LPI property table @0x0000000fc0420000 [ 0.000000] GICv3: CPU0: using reserved LPI pending table @0x0000000fc05c0000 So, please feel to add: Tested-by: Bhupesh Sharma Thanks, Bhupesh