Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 9 Mar 2020 23:11:38 +0100
From:   Robert Richter <rrichter@marvell.com>
To:     Marc Zyngier <maz@kernel.org>
CC:     <kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
        Eric Auger <eric.auger@redhat.com>,
        James Morse <james.morse@arm.com>,
        Julien Thierry <julien.thierry.kdev@gmail.com>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Jason Cooper <jason@lakedaemon.net>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Andrew Murray <Andrew.Murray@arm.com>,
        Zenghui Yu <yuzenghui@huawei.com>
Subject: Re: [PATCH v3 03/32] irqchip/gic-v3: Workaround Cavium TX1 erratum
 when reading GICD_TYPER2
Message-ID: <20200309221137.5pjh4vkc62ft3h2a@rric.localdomain>
References: <20191224111055.11836-1-maz@kernel.org>
 <20191224111055.11836-4-maz@kernel.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20191224111055.11836-4-maz@kernel.org>
User-Agent: NeoMutt/20170113 (1.7.2)
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from rric.localdomain (31.208.96.227) by HE1PR08CA0077.eurprd08.prod.outlook.com (2603:10a6:7:2a::48) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2793.15 via Frontend Transport; Mon, 9 Mar 2020 22:11:44 +0000
X-Originating-IP: [31.208.96.227]
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 4af0b874-6dfa-4178-e01b-08d7c476db30
X-MS-TrafficTypeDiagnostic: MN2PR18MB3390:
X-Microsoft-Antispam-PRVS: <MN2PR18MB33906A6A947423C0D0F6A47ED9FE0@MN2PR18MB3390.namprd18.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-Forefront-PRVS: 0337AFFE9A
X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4636009)(136003)(396003)(376002)(366004)(39860400002)(346002)(189003)(199004)(45080400002)(66556008)(66946007)(86362001)(66476007)(6506007)(2906002)(6916009)(478600001)(53546011)(7416002)(8936002)(81166006)(186003)(81156014)(8676002)(26005)(16526019)(956004)(1076003)(6666004)(316002)(54906003)(5660300002)(52116002)(7696005)(9686003)(55016002)(4326008);DIR:OUT;SFP:1101;SCL:1;SRVR:MN2PR18MB3390;H:MN2PR18MB3408.namprd18.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1;
Received-SPF: None (protection.outlook.com: marvell.com does not designate
 permitted sender hosts)
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: qVG6aBidd19vK9NIEuCCb9/P2zVNLNN2Sv57nEeYkDcVSOjkCyA1MSOYmIyc68v+k3ux7dENIONZI4WW0v63UTEvJQ8Ix10w26GmahsN2+WZPh7uwafpboDnmmm2IdNnNZNpXSbzBigYtvtndjnykLajSsvHBGusYX+dsbxP4J8G6xpIULJadD7RK786FLCpuLQXm44pzt+N8BPbwpsGlAm8ah1Wd2zp/QKMNOpXjfZfwmxAK28XfuKZMZDZScOxyYCcN9zLtEvV3ovo8S/O+73/yHTOZy/LW5v+nGrLZ1p4nR9SVQFMkI1CBGB7bEivI+nuxMB1LrTpmLWjPobeGlPk6S7hOjRgKSTrow9uFCFet8dRnp42cGt8AITPuSG1VJ5ncKhbbVV3TfXr4C09fU3Go0hBCQNvisWZqM/7noqbaMIY3gEuQQyvkk6818lW
X-MS-Exchange-AntiSpam-MessageData: 4gVS9hHtvutLD+fZPlbbMwiih2S52juhTm7V+HsPc99i4F6a9RSMQAZPeP/qccOJln5tuEfTuS2qMQgflIJ+C9YSl55h7HAG+ZUhM7WEfFVw1N0719AOVRGt8d8J+avAPAb5TrdHXNrXRjwyvf4O3Q==
X-MS-Exchange-CrossTenant-Network-Message-Id: 4af0b874-6dfa-4178-e01b-08d7c476db30
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2020 22:11:46.0402
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tv8piyVM8lcChJSfpLVtfusdpqRv1ZiLcAJF/pSieEJtwAP/6HHIDjNDp5pRhE+O9cj/ODCI8+4yFYyD1K9giQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3390
X-OriginatorOrg: marvell.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138,18.0.572
 definitions=2020-03-09_11:2020-03-09,2020-03-09 signatures=0
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 24.12.19 11:10:26, Marc Zyngier wrote:
> Despite the architecture spec being extremely clear about the fact
> that reserved registers in the GIC distributor memory map are RES0
> (and thus are not allowed to generate an exception), the Cavium
> ThunderX (aka TX1) SoC explodes as such:
> 
> [    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
> [    0.000000] GICv3: 128 SPIs implemented
> [    0.000000] GICv3: 0 Extended SPIs implemented
> [    0.000000] Internal error: synchronous external abort: 96000210 [#1] SMP
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.4.0-rc4-00035-g3cf6a3d5725f #7956
> [    0.000000] Hardware name: cavium,thunder-88xx (DT)
> [    0.000000] pstate: 60000085 (nZCv daIf -PAN -UAO)
> [    0.000000] pc : __raw_readl+0x0/0x8
> [    0.000000] lr : gic_init_bases+0x110/0x560
> [    0.000000] sp : ffff800011243d90
> [    0.000000] x29: ffff800011243d90 x28: 0000000000000000
> [    0.000000] x27: 0000000000000018 x26: 0000000000000002
> [    0.000000] x25: ffff8000116f0000 x24: ffff000fbe6a2c80
> [    0.000000] x23: 0000000000000000 x22: ffff010fdc322b68
> [    0.000000] x21: ffff800010a7a208 x20: 00000000009b0404
> [    0.000000] x19: ffff80001124dad0 x18: 0000000000000010
> [    0.000000] x17: 000000004d8d492b x16: 00000000f67eb9af
> [    0.000000] x15: ffffffffffffffff x14: ffff800011249908
> [    0.000000] x13: ffff800091243ae7 x12: ffff800011243af4
> [    0.000000] x11: ffff80001126e000 x10: ffff800011243a70
> [    0.000000] x9 : 00000000ffffffd0 x8 : ffff80001069c828
> [    0.000000] x7 : 0000000000000059 x6 : ffff8000113fb4d1
> [    0.000000] x5 : 0000000000000001 x4 : 0000000000000000
> [    0.000000] x3 : 0000000000000000 x2 : 0000000000000000
> [    0.000000] x1 : 0000000000000000 x0 : ffff8000116f000c
> [    0.000000] Call trace:
> [    0.000000]  __raw_readl+0x0/0x8
> [    0.000000]  gic_of_init+0x188/0x224
> [    0.000000]  of_irq_init+0x200/0x3cc
> [    0.000000]  irqchip_init+0x1c/0x40
> [    0.000000]  init_IRQ+0x160/0x1d0
> [    0.000000]  start_kernel+0x2ec/0x4b8
> [    0.000000] Code: a8c47bfd d65f03c0 d538d080 d65f03c0 (b9400000)
> 
> when reading the GICv4.1 GICD_TYPER2 register, which is unexpected...
> 
> Work around it by adding a new quirk flagging all the A1 revisions
> of the distributor, but it remains unknown whether this could affect
> other revisions of this SoC (or even other SoCs from the same silicon
> vendor).
> 
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
>  drivers/irqchip/irq-gic-v3.c | 23 ++++++++++++++++++++++-
>  1 file changed, 22 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
> index 286f98222878..640d4db65b78 100644
> --- a/drivers/irqchip/irq-gic-v3.c
> +++ b/drivers/irqchip/irq-gic-v3.c
> @@ -34,6 +34,7 @@
>  #define GICD_INT_NMI_PRI	(GICD_INT_DEF_PRI & ~0x80)
>  
>  #define FLAGS_WORKAROUND_GICR_WAKER_MSM8996	(1ULL << 0)
> +#define FLAGS_WORKAROUND_GICD_TYPER2_TX1	(1ULL << 1)
>  
>  struct redist_region {
>  	void __iomem		*redist_base;
> @@ -1464,6 +1465,15 @@ static bool gic_enable_quirk_msm8996(void *data)
>  	return true;
>  }
>  
> +static bool gic_enable_quirk_tx1(void *data)
> +{
> +	struct gic_chip_data *d = data;
> +
> +	d->flags |= FLAGS_WORKAROUND_GICD_TYPER2_TX1;
> +
> +	return true;
> +}
> +
>  static bool gic_enable_quirk_hip06_07(void *data)
>  {
>  	struct gic_chip_data *d = data;
> @@ -1502,6 +1512,12 @@ static const struct gic_quirk gic_quirks[] = {
>  		.mask	= 0xffffffff,
>  		.init	= gic_enable_quirk_hip06_07,
>  	},
> +	{
> +		.desc	= "GICv3: Cavium TX1 GICD_TYPER2 erratum",

There is no errata number yet.

> +		.iidr	= 0xa100034c,
> +		.mask	= 0xfff00fff,
> +		.init	= gic_enable_quirk_tx1,

All TX1 and OcteonTX parts are affected, which is a0-a7 and b0-b7. So
the iidr/mask should be:

		.iidr   = 0xa000034c,
		.mask   = 0xe8f00fff,

> +	},
>  	{
>  	}
>  };
> @@ -1577,7 +1593,12 @@ static int __init gic_init_bases(void __iomem *dist_base,
>  	pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
>  	pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
>  
> -	gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);
> +	/*
> +	 * ThunderX1 explodes on reading GICD_TYPER2, in total violation
> +	 * of the spec (which says that reserved addresses are RES0).
> +	 */
> +	if (!(gic_data.flags & FLAGS_WORKAROUND_GICD_TYPER2_TX1))
> +		gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);

You already said that checking for ArchRev of GICD_PIDR2 isn't an
option here. Though, it could...

Thanks,

-Robert

>  
>  	gic_data.domain = irq_domain_create_tree(handle, &gic_irq_domain_ops,
>  						 &gic_data);
> -- 
> 2.20.1
>