Received-SPF: pass (google.com: domain of linux-kernel+bounces-58134-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <d46a4d9c-7253-4318-90b7-08646493e7db@nvidia.com>
Date: Thu, 8 Feb 2024 13:11:16 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net v4] net: stmmac: xgmac: fix handling of DPP safety
 error for DMA channels
Content-Language: en-US
To: Simon Horman <horms@kernel.org>
Cc: Furong Xu <0x1207@gmail.com>, "David S. Miller" <davem@davemloft.net>,
 Alexandre Torgue <alexandre.torgue@foss.st.com>,
 Jose Abreu <joabreu@synopsys.com>, Eric Dumazet <edumazet@google.com>,
 Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
 Maxime Coquelin <mcoquelin.stm32@gmail.com>, Joao Pinto
 <jpinto@synopsys.com>, Serge Semin <fancer.lancer@gmail.com>,
 netdev@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com,
 linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
 xfr@outlook.com, rock.xu@nio.com,
 "linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>
References: <20240203051439.1127090-1-0x1207@gmail.com>
 <c25eb595-8d91-40ea-9f52-efa15ebafdbc@nvidia.com>
 <20240208092627.GP1297511@kernel.org> <20240208095348.GA1435458@kernel.org>
From: Jon Hunter <jonathanh@nvidia.com>
In-Reply-To: <20240208095348.GA1435458@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?VFArS1BsSzF2SHZMYXcwMGNpSkZWSTJaUzZQVVQ4WkFLSTNkazhqUVl2eThq?=
 =?utf-8?B?YVhZUWpjazJBZzNKMW5IMnBRYWRkcEExVkhtc1dtbjhodUhQeWRxU1ZsNWR1?=
 =?utf-8?B?dkRuNzFRQmd0NkVGZC9xS3RIL0NiRW9obENrWklac2h1R1Fsbmc4Y3VzakRu?=
 =?utf-8?B?ZUdyYWZtQW5OT2VHMmIzcU9mai9EZFFUN2VaclpTQVdvUFNFbndiSFlIZUdC?=
 =?utf-8?B?WFZzSEFEKzV0dWtuUkpKUjhxT3hLbkRMY3UwOVliemhSR056UnRJSGpTYW81?=
 =?utf-8?B?UzFvUUhkT3RmYkFIMHo3bGFMTHNlUCtHSDFSRUtvRGNDTnRGQkJzcFVJd3da?=
 =?utf-8?B?bGZzSmFjUzRHNFVGVnJzZ1o1ZUhjVENuU3Y5OEVwK1lvVHU2MDhpa29GTW14?=
 =?utf-8?B?TWhpV2hGbFJ5ZTFJV3hHS25wWkNqdm9qNmZXdnJvMFJBWlU1OVpvVW0relRj?=
 =?utf-8?B?S1Fxa0k2TG0xNlFYWTNaMUMxcldrZk0wdS9teTNQcTVlWEVlTEd3Njh2WlR5?=
 =?utf-8?B?UCtoSGJ1eS9MUWtGQVV3ZEtMTmdnQjAwU2diOG9LNEJhVTM3bmVhaTdjL0hp?=
 =?utf-8?B?R0lUNWU2VnVoRVgrcEFKNmo2NW53ZW9JTFhmcXZSeTJiVW5xOTlWa1Q2WERh?=
 =?utf-8?B?Z284VTF2eWU5cXR1SUh2MXY1T2tyeS9TSTF2Qk41SVZ4WHVhYWo3N1NhZU0x?=
 =?utf-8?B?b2YzdkxJb24vMVpZU2xMS2FGMWY4aWpjby8yZTlOTnN2YVNwc1NNT2ZxZ05I?=
 =?utf-8?B?dFB3NWpyMWFzbC83NEhwclR2UlpGcFFzbCtxR1VXT2dnd2NwK1FFbWxjSzMv?=
 =?utf-8?B?N1F4ZkhzbkJLSUkyS1g5RFRNZWVNUzlsdTNMVTNsdjNKK3N4MHdkNUlUQU9J?=
 =?utf-8?B?aEExUEhpNG9ONHpleC94N1FJaGIzN2lXNXdBRzdHTUdVeXVUamIvcEp6Q1dX?=
 =?utf-8?B?R1Q3Qm1UTzJOOXBMeWd3M2VydkNabGZacCtXR01ETXVsdnhnYi9zMGxuSDQ1?=
 =?utf-8?B?Qzhtb214MUlXMEM1ZGhCU2h1MktSR0ZlT3Jyb2wzUjlodVJoMXRmNFN2VVVJ?=
 =?utf-8?B?bXRxemEzejlBTWRVY2JqcjdXMG5NQzlLMFlZMUJYMGNwSEh1VDJrUklUMnQ4?=
 =?utf-8?B?L0N6ZmtXTDQvMzJBWFVjKy83NWhrRkF1OFEzN25TUUM0ZG1FdGcxN0RjYWc2?=
 =?utf-8?B?Q3hWTUNJSzc1eUdTQ1FadVlSR1RWdlBpQ1hUMlBwM0owM3BhajlwVFcxbnJD?=
 =?utf-8?B?NVk5aCtHQVBnZy90a2N5K0lpNEFWOFRFa2FEN0VXdCtFM25SOGdhSkxwNERV?=
 =?utf-8?B?V05yNGRZQXBkbWpOeFBMbFFhblByTG9hSVE1dFEzaCs0YStJNzlWcEdJckJr?=
 =?utf-8?B?WUtvNmJIS1oxazNsTlN6V2R3R1kzd3JkU05GOVgybE92SUhxRGFCN2lQcHhP?=
 =?utf-8?B?S0ZpcDkyVmpnN3AvVU9BMkNLclJRR3hZTk50RnFoSkw2QjAwSGFCYis3YndJ?=
 =?utf-8?B?bXZsV2RhNDA3dENZOHNwWWhGQ3ZoWVhxR0N0elk5NEJWNTNQa3d0aGtDNTVN?=
 =?utf-8?B?UlVxVDZBQWgvWityaFRSak9yYkIxVmVkeFRyc0UyQmg5bDg5MU8xaGRINzh3?=
 =?utf-8?B?RDZFZFJJRUR6SEZaZXVjY1owM2tiNXVJR1Z2eVJGbTM4Qm0rb2FjS1Z3eEJh?=
 =?utf-8?B?TGhqYTBOMTYzQklVdjJHSUhGaVFMbTdQNk5CWUpZd295a3F0U3I3MXJuUWNY?=
 =?utf-8?B?RWhUb1hRNkgvUUh5TGx6ZTQrd01sbWhWQ1g3VFFDb04zQmNROHZJczZwbXd4?=
 =?utf-8?B?dkdlRzJTZnoyUzFXVDAxQWd4TU9RVjRWWDduVVRzY3dUSndacmFNZE9rZEdC?=
 =?utf-8?B?TlFSclFmY2NLUy9ZNXFlL1p2bHZqMDljR2FJeU80Mmd1WXl1RHZuTmJrRzBY?=
 =?utf-8?B?d2Z6SFlHMEhHNU9PdUlJckNiU0hodm5WT0NDaDBjbHdmeTU4dWVEV1V0V1lK?=
 =?utf-8?B?NTR0cEwxT3dYZDFGM2tsaVJGeGFXamdscnBIbmRTYVBDYVBCVlNEUit6NXBS?=
 =?utf-8?B?cFZOY1kwKzgrd04zMGszd1p5UDJZYVFVVzZ4a3NycEpWMERiNVl4TFBrNmFi?=
 =?utf-8?Q?J7gQxOs88maN7tnSa0zTH8Xi4?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4c0bdc5b-67f4-4b1b-8cc7-08dc28a772c2
X-MS-Exchange-CrossTenant-AuthSource: CO6PR12MB5444.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 13:11:23.2559
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YQiFkUk2WI7d3nijz2eIWDPKrUR+eLhmwFTujcSP1nXQOSsDRIeFRsfXtxYoim9N2go5rxP8+7KTqvacXnYt0g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB8792


On 08/02/2024 09:53, Simon Horman wrote:
> On Thu, Feb 08, 2024 at 09:26:27AM +0000, Simon Horman wrote:
>> On Wed, Feb 07, 2024 at 11:56:26AM +0000, Jon Hunter wrote:
>>>
>>> On 03/02/2024 05:14, Furong Xu wrote:
>>>> Commit 56e58d6c8a56 ("net: stmmac: Implement Safety Features in
>>>> XGMAC core") checks and reports safety errors, but leaves the
>>>> Data Path Parity Errors for each channel in DMA unhandled at all, lead to
>>>> a storm of interrupt.
>>>> Fix it by checking and clearing the DMA_DPP_Interrupt_Status register.
>>>>
>>>> Fixes: 56e58d6c8a56 ("net: stmmac: Implement Safety Features in XGMAC core")
>>>> Signed-off-by: Furong Xu <0x1207@gmail.com>
>>>> Reviewed-by: Simon Horman <horms@kernel.org>
>>>> Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
>>>> ---
>>>> Changes in v4:
>>>>    - fix a typo name of DDPP bit, thanks Serge Semin
>>>>
>>>> Changes in v3:
>>>>    - code style fix, thanks Paolo Abeni
>>>>
>>>> Changes in v2:
>>>>     - explicit enable Data Path Parity Protection
>>>>     - add new counters to stmmac_safety_stats
>>>>     - add detailed log
>>>> ---
>>>>    drivers/net/ethernet/stmicro/stmmac/common.h  |  1 +
>>>>    .../net/ethernet/stmicro/stmmac/dwxgmac2.h    |  3 +
>>>>    .../ethernet/stmicro/stmmac/dwxgmac2_core.c   | 57 ++++++++++++++++++-
>>>>    3 files changed, 60 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h
>>>> index 721c1f8e892f..b4f60ab078d6 100644
>>>> --- a/drivers/net/ethernet/stmicro/stmmac/common.h
>>>> +++ b/drivers/net/ethernet/stmicro/stmmac/common.h
>>>> @@ -216,6 +216,7 @@ struct stmmac_safety_stats {
>>>>    	unsigned long mac_errors[32];
>>>>    	unsigned long mtl_errors[32];
>>>>    	unsigned long dma_errors[32];
>>>> +	unsigned long dma_dpp_errors[32];
>>>>    };
>>>>    /* Number of fields in Safety Stats */
>>>> diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
>>>> index 207ff1799f2c..5c67a3f89f08 100644
>>>> --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
>>>> +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h
>>>> @@ -303,6 +303,8 @@
>>>>    #define XGMAC_RXCEIE			BIT(4)
>>>>    #define XGMAC_TXCEIE			BIT(0)
>>>>    #define XGMAC_MTL_ECC_INT_STATUS	0x000010cc
>>>> +#define XGMAC_MTL_DPP_CONTROL		0x000010e0
>>>> +#define XGMAC_DPP_DISABLE		BIT(0)
>>>>    #define XGMAC_MTL_TXQ_OPMODE(x)		(0x00001100 + (0x80 * (x)))
>>>>    #define XGMAC_TQS			GENMASK(25, 16)
>>>>    #define XGMAC_TQS_SHIFT			16
>>>> @@ -385,6 +387,7 @@
>>>>    #define XGMAC_DCEIE			BIT(1)
>>>>    #define XGMAC_TCEIE			BIT(0)
>>>>    #define XGMAC_DMA_ECC_INT_STATUS	0x0000306c
>>>> +#define XGMAC_DMA_DPP_INT_STATUS	0x00003074
>>>>    #define XGMAC_DMA_CH_CONTROL(x)		(0x00003100 + (0x80 * (x)))
>>>>    #define XGMAC_SPH			BIT(24)
>>>>    #define XGMAC_PBLx8			BIT(16)
>>>> diff --git a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
>>>> index eb48211d9b0e..04d7c4dc2e35 100644
>>>> --- a/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
>>>> +++ b/drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c
>>>> @@ -830,6 +830,43 @@ static const struct dwxgmac3_error_desc dwxgmac3_dma_errors[32]= {
>>>>    	{ false, "UNKNOWN", "Unknown Error" }, /* 31 */
>>>>    };
>>>> +static const char * const dpp_rx_err = "Read Rx Descriptor Parity checker Error";
>>>> +static const char * const dpp_tx_err = "Read Tx Descriptor Parity checker Error";
>>>> +static const struct dwxgmac3_error_desc dwxgmac3_dma_dpp_errors[32] = {
>>>> +	{ true, "TDPES0", dpp_tx_err },
>>>> +	{ true, "TDPES1", dpp_tx_err },
>>>> +	{ true, "TDPES2", dpp_tx_err },
>>>> +	{ true, "TDPES3", dpp_tx_err },
>>>> +	{ true, "TDPES4", dpp_tx_err },
>>>> +	{ true, "TDPES5", dpp_tx_err },
>>>> +	{ true, "TDPES6", dpp_tx_err },
>>>> +	{ true, "TDPES7", dpp_tx_err },
>>>> +	{ true, "TDPES8", dpp_tx_err },
>>>> +	{ true, "TDPES9", dpp_tx_err },
>>>> +	{ true, "TDPES10", dpp_tx_err },
>>>> +	{ true, "TDPES11", dpp_tx_err },
>>>> +	{ true, "TDPES12", dpp_tx_err },
>>>> +	{ true, "TDPES13", dpp_tx_err },
>>>> +	{ true, "TDPES14", dpp_tx_err },
>>>> +	{ true, "TDPES15", dpp_tx_err },
>>>> +	{ true, "RDPES0", dpp_rx_err },
>>>> +	{ true, "RDPES1", dpp_rx_err },
>>>> +	{ true, "RDPES2", dpp_rx_err },
>>>> +	{ true, "RDPES3", dpp_rx_err },
>>>> +	{ true, "RDPES4", dpp_rx_err },
>>>> +	{ true, "RDPES5", dpp_rx_err },
>>>> +	{ true, "RDPES6", dpp_rx_err },
>>>> +	{ true, "RDPES7", dpp_rx_err },
>>>> +	{ true, "RDPES8", dpp_rx_err },
>>>> +	{ true, "RDPES9", dpp_rx_err },
>>>> +	{ true, "RDPES10", dpp_rx_err },
>>>> +	{ true, "RDPES11", dpp_rx_err },
>>>> +	{ true, "RDPES12", dpp_rx_err },
>>>> +	{ true, "RDPES13", dpp_rx_err },
>>>> +	{ true, "RDPES14", dpp_rx_err },
>>>> +	{ true, "RDPES15", dpp_rx_err },
>>>> +};
>>>> +
>>>>    static void dwxgmac3_handle_dma_err(struct net_device *ndev,
>>>>    				    void __iomem *ioaddr, bool correctable,
>>>>    				    struct stmmac_safety_stats *stats)
>>>> @@ -841,6 +878,13 @@ static void dwxgmac3_handle_dma_err(struct net_device *ndev,
>>>>    	dwxgmac3_log_error(ndev, value, correctable, "DMA",
>>>>    			   dwxgmac3_dma_errors, STAT_OFF(dma_errors), stats);
>>>> +
>>>> +	value = readl(ioaddr + XGMAC_DMA_DPP_INT_STATUS);
>>>> +	writel(value, ioaddr + XGMAC_DMA_DPP_INT_STATUS);
>>>> +
>>>> +	dwxgmac3_log_error(ndev, value, false, "DMA_DPP",
>>>> +			   dwxgmac3_dma_dpp_errors,
>>>> +			   STAT_OFF(dma_dpp_errors), stats);
>>>>    }
>>>>    static int
>>>> @@ -881,6 +925,12 @@ dwxgmac3_safety_feat_config(void __iomem *ioaddr, unsigned int asp,
>>>>    	value |= XGMAC_TMOUTEN; /* FSM Timeout Feature */
>>>>    	writel(value, ioaddr + XGMAC_MAC_FSM_CONTROL);
>>>> +	/* 5. Enable Data Path Parity Protection */
>>>> +	value = readl(ioaddr + XGMAC_MTL_DPP_CONTROL);
>>>> +	/* already enabled by default, explicit enable it again */
>>>> +	value &= ~XGMAC_DPP_DISABLE;
>>>> +	writel(value, ioaddr + XGMAC_MTL_DPP_CONTROL);
>>>> +
>>>>    	return 0;
>>>>    }
>>>> @@ -914,7 +964,11 @@ static int dwxgmac3_safety_feat_irq_status(struct net_device *ndev,
>>>>    		ret |= !corr;
>>>>    	}
>>>> -	err = dma & (XGMAC_DEUIS | XGMAC_DECIS);
>>>> +	/* DMA_DPP_Interrupt_Status is indicated by MCSIS bit in
>>>> +	 * DMA_Safety_Interrupt_Status, so we handle DMA Data Path
>>>> +	 * Parity Errors here
>>>> +	 */
>>>> +	err = dma & (XGMAC_DEUIS | XGMAC_DECIS | XGMAC_MCSIS);
>>>>    	corr = dma & XGMAC_DECIS;
>>>>    	if (err) {
>>>>    		dwxgmac3_handle_dma_err(ndev, ioaddr, corr, stats);
>>>> @@ -930,6 +984,7 @@ static const struct dwxgmac3_error {
>>>>    	{ dwxgmac3_mac_errors },
>>>>    	{ dwxgmac3_mtl_errors },
>>>>    	{ dwxgmac3_dma_errors },
>>>> +	{ dwxgmac3_dma_dpp_errors },
>>>>    };
>>>>    static int dwxgmac3_safety_feat_dump(struct stmmac_safety_stats *stats,
>>>
>>>
>>> This change is breaking the build on some of our builders that are still using GCC 6.x ...
>>>
>>> drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c:836:20: error: initialiser element is not constant
>>>    { true, "TDPES0", dpp_tx_err },
>>>                      ^~~~~~~~~~
>>> drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c:836:20: note: (near initialisation for ‘dwxgmac3_dma_dpp_errors[0].detailed_desc’)
>>> drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c:837:20: error: initialiser element is not constant
>>>    { true, "TDPES1", dpp_tx_err },
>>>                      ^~~~~~~~~~
>>> drivers/net/ethernet/stmicro/stmmac/dwxgmac2_core.c:837:20: note: (near initialisation for ‘dwxgmac3_dma_dpp_errors[1].detailed_desc’)
>>> ...
>>>
>>> I know that this is quite old but the minimum supported by the kernel is v5.1 ...
>>>
>>> https://www.kernel.org/doc/html/next/process/changes.html
>>
>> Thanks Jon,
>>
>> I separately received a notification about this occurring with gcc 7.
>>
>> https://lore.kernel.org/oe-kbuild-all/202402081135.lAxxBXHk-lkp@intel.com/
>>
>> It is unclear to me why this occurs, as dpp_tx_err and dpp_tx_err are const.
>> But I do seem to be able to address this problem by using #defines for
>> these values instead.
>>
>> I plan to post a patch shortly.
> 
> Patch posted:
> - [PATCH net] net: stmmac: xgmac: use #define for string constants
>    https://lore.kernel.org/netdev/20240208-xgmac-const-v1-1-e69a1eeabfc8@kernel.org/
> 


Thanks for fixing! Works for me.

Jon

-- 
nvpublic