From:   Sai Krishna Gajula <saikrishnag@marvell.com>
To:     Simon Horman <simon.horman@corigine.com>
CC:     "davem@davemloft.net" <davem@davemloft.net>,
        "edumazet@google.com" <edumazet@google.com>,
        "kuba@kernel.org" <kuba@kernel.org>,
        "pabeni@redhat.com" <pabeni@redhat.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Sunil Kovvuri Goutham <sgoutham@marvell.com>,
        Suman Ghosh <sumang@marvell.com>
Subject: Re: [net PATCH v2] octeontx2-af: Unlock contexts in the queue context
 cache in case of fault detection
Thread-Topic: [net PATCH v2] octeontx2-af: Unlock contexts in the queue
 context cache in case of fault detection
Thread-Index: AQHZTNJdrkIRAoLbtkeGZpi7N8KmvA==
Date:   Thu, 2 Mar 2023 06:43:58 +0000
Message-ID: <BY3PR18MB47079749DC04B6834AFB1F8DA0B29@BY3PR18MB4707.namprd18.prod.outlook.com>
References: <20230223110125.2172509-1-saikrishnag@marvell.com>
 <Y/dnNRD4Gpl0n2GQ@corigine.com>
 <BY3PR18MB470774ABED5E4E22DAA4535EA0A89@BY3PR18MB4707.namprd18.prod.outlook.com>
 <Y/h+TH9WTWOVaSHj@corigine.com>
In-Reply-To: <Y/h+TH9WTWOVaSHj@corigine.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?5QU8v0D9V3oj5WV19+20lemUQyoTMBkIgn7Bf2sBbN/OMIrD3v/Z6YDTSgQa?=
 =?us-ascii?Q?Wa+whIdbTSiNxiBFQcCpBqQiMfk3rWzdG97zjlelz8Xe+yv8mOByE7kK+ik+?=
 =?us-ascii?Q?pyJno/pbYuc/VzhPHufZAY/ygd9fF1mRcpA9ytDklz+MWulip2PzdKYli0kM?=
 =?us-ascii?Q?388YqRdJibz6qOybJVOvcaGw/9BoZRrVEiI3oJ8UvpOlIqpZ5IpC9ryppdL7?=
 =?us-ascii?Q?6yGUSKENk6ziwDetixhjIVDKZfa4U2TPvdY3uDkI5ztPCG4nk2947BJh+0zW?=
 =?us-ascii?Q?yidcJy81p/VNghHaJJqFZ7DaKQsDS6xJRuqq90E4I9k5Z2zJKajUcqrcQqKE?=
 =?us-ascii?Q?JK+kn3Unk6tWfBth2Q8wmOH0a9Vqnr4uvPBDCLboH0rWFBjd/7aQ7m7Re91B?=
 =?us-ascii?Q?G+k6Z44Y/3ckeeIS/Hl9k5doXoEzsIZ1M30Hz0ztWD5e1jmqh+Df9hYdor9g?=
 =?us-ascii?Q?WAsJmAOWrw5tNtUXwxV0nSHnhSjP2Z74zPdusmzH8A0GRH+8x/fgipdod76K?=
 =?us-ascii?Q?EHiv2vMaFLiw0LsD1xv4PMfMOReajFK23X2Jw4yFmwmTBIZZCcG+i/dWlbAJ?=
 =?us-ascii?Q?o8CPiLxmHknJ8VYieYy8C++1PlUtZXz6OwG1P0bksmmvZDw0v9af25O2RigC?=
 =?us-ascii?Q?jslXHJ8LUCLV0S7Z2/0c5LNM2UbrusezDQh12hK410lv7pPVLLA2Tj9mOIC9?=
 =?us-ascii?Q?wbgIaJ73A1BL0eL660eprCvMbICEoUaIIBUHXh7t182OzddEC7hAZXIykxJl?=
 =?us-ascii?Q?DjUYidDFnQqZnUk6uWpq289Jp3ffe3qdvwHACEr+hx0bFnVgVOkq/sT89RHL?=
 =?us-ascii?Q?GDJYD9u58F+Rzem1W3DcF+G438gtwhuc9kg6iR4282uOqLjsk7iinjs4zoBL?=
 =?us-ascii?Q?IU9pkUS4n/VN21YI3ge+tQD7aj9WPK4IHVBUiX1YcqvF745GrnHeDlyjNLut?=
 =?us-ascii?Q?5ljMWI0Rh2eQJdK95ncEnzGAjNlgEkMZV0q+rrrpfOLRWqkU1/FJg6XmjrgF?=
 =?us-ascii?Q?5I7+iN1C4mZeW59dtC7r5wwCeh2ARxrapGVOhUropjA99H4Vl1XAexpv/0DP?=
 =?us-ascii?Q?QVnFZ/mQQDbFaESDfKAVbVzfxNPkeSpWM89sBnhnGP+FnfKw4DOXxgIbgpqA?=
 =?us-ascii?Q?fqCxJ/Q6QAYxkhLzUREGi1a9Lk/Ti1I3NeteJwtVJBftXh6KWRwO80f/NB++?=
 =?us-ascii?Q?fmWua6HLNktKpu9BnuhMrHEzN/o9+XthAFV4Lzh8NtFpPbfORjENk3RilBp6?=
 =?us-ascii?Q?NZARpoe92YBlfgDO8+T757DjaCVCOc3FXSkvsu06Rp9KyrCxm4tVKJ3G3r70?=
 =?us-ascii?Q?F77UtLDCrkHouIgg78iBBQvv/o5nf6/23qgfhASN0b95gEerysWmOLpVsSR5?=
 =?us-ascii?Q?82p5fSuQKrZVrNqXSE2ewp/vsqzKLO12s8CobngY1IgNFHxY7p1hpnTNHUKG?=
 =?us-ascii?Q?dWjNDJck19IxxKGuL7OmcBtNCCzh5jKN8Wq7H4CDwfmP0cWcemLOjOBUsYys?=
 =?us-ascii?Q?I2Iz40ebqfHE22sPF0yNr0Jw4UAQvWFqCvXj/2/L7R2eBnDBsww/fLsL2Bag?=
 =?us-ascii?Q?FmhHkNpM3ek5ovfRqHydYqvtZfD3YgStflfSmrG0?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: marvell.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BY3PR18MB4707.namprd18.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 29be8588-e7e0-420d-16ee-08db1ae9803d
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Mar 2023 06:43:58.3777
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: hVgC6OVE/DDenxU+H/6LAnXv8rfUvjuQfUltEOHrFbYIKiHgZcH2QlgKuhi/vTEdSK0Kzanheha2oKs/Oib8WQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR18MB4053
X-Proofpoint-ORIG-GUID: Gzjs1j0SaiideNSFQMi2rqZDafLX_XGe
X-Proofpoint-GUID: Gzjs1j0SaiideNSFQMi2rqZDafLX_XGe
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-03-02_02,2023-03-01_03,2023-02-09_01
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Simon,

> -----Original Message-----
> From: Simon Horman <simon.horman@corigine.com>
> Sent: Friday, February 24, 2023 2:37 PM
> To: Sai Krishna Gajula <saikrishnag@marvell.com>
> Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> pabeni@redhat.com; netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
> Sunil Kovvuri Goutham <sgoutham@marvell.com>; Suman Ghosh
> <sumang@marvell.com>
> Subject: Re: [net PATCH v2] octeontx2-af: Unlock contexts in the queue
> context cache in case of fault detection
>=20
> ----------------------------------------------------------------------
> On Fri, Feb 24, 2023 at 08:39:20AM +0000, Sai Krishna Gajula wrote:
> > Hi Simon,
> >
> > > -----Original Message-----
> > > From: Simon Horman <simon.horman@corigine.com>
> > > Sent: Thursday, February 23, 2023 6:47 PM
> > > To: Sai Krishna Gajula <saikrishnag@marvell.com>
> > > Cc: davem@davemloft.net; edumazet@google.com; kuba@kernel.org;
> > > pabeni@redhat.com; netdev@vger.kernel.org;
> > > linux-kernel@vger.kernel.org; Sunil Kovvuri Goutham
> > > <sgoutham@marvell.com>; Suman Ghosh <sumang@marvell.com>
> > > Subject: Re: [net PATCH v2] octeontx2-af: Unlock contexts in the
> > > queue context cache in case of fault detection
> > >
> > >
> > > --------------------------------------------------------------------
> > > -- On Thu, Feb 23, 2023 at 04:31:25PM +0530, Sai Krishna wrote:
> > > > From: Suman Ghosh <sumang@marvell.com>
> > > >
> > > > NDC caches contexts of frequently used queue's (Rx and Tx queues)
> > > > contexts. Due to a HW errata when NDC detects fault/poision while
> > > > accessing contexts it could go into an illegal state where a cache
> > > > line could get locked forever. To makesure all cache lines in NDC
> > > > are available for optimum performance upon fault/lockerror/posion
> > > > errors scan through all cache lines in NDC and clear the lock bit.
> > > >
> > > > Fixes: 4a3581cd5995 ("octeontx2-af: NPA AQ instruction enqueue
> > > > support")
> > > > Signed-off-by: Suman Ghosh <sumang@marvell.com>
> > > > Signed-off-by: Sunil Kovvuri Goutham <sgoutham@marvell.com>
> > > > Signed-off-by: Sai Krishna <saikrishnag@marvell.com>
> > >
> > > ...
> > >
> > > > diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h
> > > > b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h
> > > > index 389663a13d1d..6508f25b2b37 100644
> > > > --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu.h
> > > > +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu.h
> > > > @@ -884,6 +884,12 @@ int rvu_cpt_lf_teardown(struct rvu *rvu, u16
> > > > pcifunc, int blkaddr, int lf,  int rvu_cpt_ctx_flush(struct rvu
> > > > *rvu,
> > > > u16 pcifunc);  int rvu_cpt_init(struct rvu *rvu);
> > > >
> > > > +/* NDC APIs */
> > > > +#define NDC_MAX_BANK(rvu, blk_addr) (rvu_read64(rvu, \
> > > > +					blk_addr, NDC_AF_CONST) & 0xFF)
> #define
> > > > +NDC_MAX_LINE_PER_BANK(rvu, blk_addr) ((rvu_read64(rvu, \
> > > > +					blk_addr, NDC_AF_CONST) &
> > > 0xFFFF0000) >> 16)
> > >
> > > Perhaps not appropriate to include as part of a fix, as NDC_MAX_BANK
> > > is being moved from elsewhere, but I wonder if this might be more
> > > cleanly implemented using FIELD_GET().
> >
> > We will modify and send a separate patch for all the possible macros th=
at
> can be replaced by FIELD_GET().
>=20
> Thanks, much appreciated.
>=20
> > > ...
> > >
> > > > diff --git a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h
> > > > b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h
> > > > index 1729b22580ce..bc6ca5ccc1ff 100644
> > > > --- a/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h
> > > > +++ b/drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h
> > > > @@ -694,6 +694,7 @@
> > > >  #define NDC_AF_INTR_ENA_W1S		(0x00068)
> > > >  #define NDC_AF_INTR_ENA_W1C		(0x00070)
> > > >  #define NDC_AF_ACTIVE_PC		(0x00078)
> > > > +#define NDC_AF_CAMS_RD_INTERVAL		(0x00080)
> > > >  #define NDC_AF_BP_TEST_ENABLE		(0x001F8)
> > > >  #define NDC_AF_BP_TEST(a)		(0x00200 | (a) << 3)
> > > >  #define NDC_AF_BLK_RST			(0x002F0)
> > > > @@ -709,6 +710,8 @@
> > > >  		(0x00F00 | (a) << 5 | (b) << 4)
> > > >  #define NDC_AF_BANKX_HIT_PC(a)		(0x01000 | (a) << 3)
> > > >  #define NDC_AF_BANKX_MISS_PC(a)		(0x01100 | (a) << 3)
> > > > +#define NDC_AF_BANKX_LINEX_METADATA(a, b) \
> > > > +		(0x10000 | (a) << 3 | (b) << 3)
> > >
> > > It looks a little odd that both a and b are shifted by 3 bits.
> > > If it's intended then perhaps it would be clearer to write this as:
> > >
> > > #define NDC_AF_BANKX_LINEX_METADATA(a, b) \
> > > 		(0x10000 | ((a) | (b)) << 3)
> >
> > will send v3 patch.
>=20
> Likewise, thanks.

We found a bug related to the macro NDC_AF_BANKX_LINEX_METADATA which we wi=
ll fix and send in v3 patch.

Thanks,
Sai