Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Date:   Mon, 30 Oct 2023 14:33:33 -0400
From:   Frank Li <Frank.li@nxp.com>
To:     Manivannan Sadhasivam <mani@kernel.org>
Cc:     manivannan.sadhasivam@linaro.org, aisheng.dong@nxp.com,
        bhelgaas@google.com, devicetree@vger.kernel.org,
        festevam@gmail.com, imx@lists.linux.dev, jdmason@kudzu.us,
        kernel@pengutronix.de, kishon@kernel.org, kw@linux.com,
        linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com,
        linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        lorenzo.pieralisi@arm.com, lpieralisi@kernel.org, maz@kernel.org,
        s.hauer@pengutronix.de, shawnguo@kernel.org, tglx@linutronix.de
Subject: Re: [PATCH v2 4/5] misc: pci_endpoint_test: Add doorbell test case
Message-ID: <ZT/2/SSoS4azNJnb@lizhi-Precision-Tower-5810>
References: <20230911220920.1817033-1-Frank.Li@nxp.com>
 <20230911220920.1817033-5-Frank.Li@nxp.com>
 <20231020175304.GC46191@thinkpad>
 <20231020180139.GE46191@thinkpad>
 <ZTLBxvM/qNGw/FLd@lizhi-Precision-Tower-5810>
 <20231020182643.GG46191@thinkpad>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20231020182643.GG46191@thinkpad>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SzZLbFZFUGFhcEowNlE1eDlBdzJaVHFSOUlDcVQ3cmRCV1lKWTkyUkpNWEtw?=
 =?utf-8?B?ZngzZE9zcmhLSjM2cm1VMithQXZ4S1NBaU9DRVdTbklhR3Y3MitDNFNFTTRC?=
 =?utf-8?B?VW1HRWFOVGwwNUZHS1RYSXV3dkRCNWJ4NEFSWHJNVjA0UDhBdnphWjdoMnZB?=
 =?utf-8?B?ZEhYNTMzYWp6YzRNRVlLTkdHd0VWdHNOb28rK0NIOHhuem9PRmVxbjNXK3lO?=
 =?utf-8?B?b0NFTnBtbEN4QmRZNUJoc0Qrekk1Z0k5Rm1mZFdMU0ZsajdjSTFxKysyVDVy?=
 =?utf-8?B?bnc0RFhpVnFia2UvN2pjbURnWTNKZnZyVVJCYnc4dkZ1SmtmK3UrbUprR2dW?=
 =?utf-8?B?Rjl6V3lFVUl3NnNoclRvQVpjSHZwSnhnT25XZ0hoL3dsVFVjczBCOStnOHEx?=
 =?utf-8?B?Q2I2bGZWVW1ackwwenl2OUpaUmE1RWU5a2c0a1B3YWxDWkJPK01WelExeE9W?=
 =?utf-8?B?ckRDQjh0RTFScGFpMzlFeFNRSU1PcUVTV2ZnV0YrcE8xS0dyM2dwUHFKaFlX?=
 =?utf-8?B?aWFTMmNBQnpOWENOMUdNcHRRblhJWHNkZGFTZjJaWCt1T2ZsWk1uc3NiUUIr?=
 =?utf-8?B?bkIvWUNjYmJFN0dHYXU0VkxWS0dFcWRqUFMrZ05HZlEyQ2ExV0NyNkp4dkJw?=
 =?utf-8?B?RzhUMHJVWk1Dd2QyVjhEckFJRlFCdzZpK1JOc0hwZlFpaHplU3Y2K3NKYnlV?=
 =?utf-8?B?M1J3ajZ2Y29VVzhwU2JqRmhsYitZMDdWYXl4RG9QUlhZcm5aK256TVI2U0Vz?=
 =?utf-8?B?Wis2Zmhlc2tlYW9IOVA2SFdKUTdxKzZyYVBuVzFjK3BaODMwR25tcDc4Z0Fi?=
 =?utf-8?B?c09yS29qM2F5TUVwTi9jZ0FNcUg4bng1cXUvNDNRcEdhSi9QZTd1SysyTHFh?=
 =?utf-8?B?OHNFZ2hJWUNYZFE5dVBKbitwalI3TW4zc21qZnNhbE1NZVZRWndyRkY5OVJU?=
 =?utf-8?B?RlJUZ2oxdHhYRDAyZWRaSkt2Q2dMS0VYZEVUQzBKTmFUYzRna1hzTytoUVFh?=
 =?utf-8?B?RlB6bHBka1I2OFg5L24rR3NiV1lkVnRldGRsMkhWYnVtcFlyRW5DeGtKZlJi?=
 =?utf-8?B?cGVGYXQvYnV1eFJxSzRrck43dWt6L0NTM3RXK1ZGQUFlV245NEI4RUJMV0dB?=
 =?utf-8?B?Z0NSdXhZdnpHRS9CYWFDZWFpZ3dRVkFyVjF5SXdVeWpRTk5sMGo0b2EzdUpm?=
 =?utf-8?B?Y2Qwc1FjQ0VkNjh5WnV5K0F3NlBFaUxxOU00bXFGRzMyYnRTeFdqc1FvekNa?=
 =?utf-8?B?YzRUMDM3Y1JTSU01VFNtRy84Q3UxZ0FicVh6eFNmaERDeDIrb0V3eGdQK2ZC?=
 =?utf-8?B?L2drNWVyMVIvd3VpSmZlc2lGTDUwSVkzQkxsTkY2RHJFMEU3eTRiT244cUV1?=
 =?utf-8?B?dFRZalZzZjVuNTFrQUdFOG95SU1NSk4vQndpamF1bVcvTHR4SGE3VWZ4SHFQ?=
 =?utf-8?B?QWZuWkloVFdMTGhjakxCRy9MZkk0aW9WZ0xQb1hkVVdDS2daUlVFOFMxS0lt?=
 =?utf-8?B?SE1DRTE4TnV2QmQ1TWN2dE5ZdG1paEo1OXNRcVliczd1dXlPeWdkbTlqT2RE?=
 =?utf-8?B?azY3WkRPK3QvQWRVRHc2bk5RaWtNQTJ2RjFFdUtTQldQTElSeEcvcUl2TTM1?=
 =?utf-8?B?dGNUbUJneU1EZTdOYndUUHBxZno2ckhFNGpMTHNGWVg4cWdVYVBjbmpodWgw?=
 =?utf-8?B?V0FxM3VzRDBjVC80bkJlQXkzM3pPeTdBMlRBZ1VSTVNyTjlQT2N2MDF3Nlkw?=
 =?utf-8?B?aVQwK3lmNFhxWHpDQWdsY2lXdWY3bStQQ3dTQUZWczNtS3dCaC9sbDhueGZq?=
 =?utf-8?B?cVROYVJwczdzWlVRQTc1dUd3ZmI4THdQM3RTMEwvTmFIZ0VKa1JPOURjRGY2?=
 =?utf-8?B?RERXeWZYR2JNOXNwSFJsemRDRnJLQVNycFppTUFVTlJDc3N3cHM2bXRwekNX?=
 =?utf-8?B?cEJRcElTazkwL2UxQktGdUJJd25UT0dKZjB5QmNhc1VtQlFmREZPWnNwNWRZ?=
 =?utf-8?B?N3NUUHZCbENYU25jZnh3RDZ5TmthakJ4WXZkM1pkVTdjckE2dy8zRFJoMnNj?=
 =?utf-8?B?VUtCWHg0RUZHVHZEWFl1THBHMTJNYUczdGNaNysrWXZlQzk2UXdkeWs0K1Qz?=
 =?utf-8?Q?cOww=3D?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 37dcc282-abb5-4306-7982-08dbd976be57
X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB4838.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Oct 2023 18:33:42.7583
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kg8lcAZiuqhyIEAB2P1x3itSPvW9dqGYN/Lmv2wSIntikVBREdohmEN9YuqJmcRBR/aYIPvb1eGlwqEHbKxeCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB8211
X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 30 Oct 2023 11:34:04 -0700 (PDT)

On Fri, Oct 20, 2023 at 11:56:43PM +0530, Manivannan Sadhasivam wrote:
> On Fri, Oct 20, 2023 at 02:07:02PM -0400, Frank Li wrote:
> > On Fri, Oct 20, 2023 at 11:31:39PM +0530, Manivannan Sadhasivam wrote:
> > > On Fri, Oct 20, 2023 at 11:23:04PM +0530, Manivannan Sadhasivam wrote:
> > > > On Mon, Sep 11, 2023 at 06:09:19PM -0400, Frank Li wrote:
> > > > > Using bit 0..7 of magic as version number in pci_endpoint_test struct to
> > > > > support older driver versions. Save to 'version' field of struct
> > > > > pci_endpoint_test to prevent reading non-existent address.
> > > > > 
> > > > 
> > > > Since both drivers are in the kernel, I don't see a necessity to maintain
> > > > compatibility. Does it make sense to load drivers of previous kernel revision
> > > > with a new kernel?
> > > > 
> > > 
> > > Shoot... Sorry, I completely forgot that one is EP and another is host. Yes, we
> > > do need to maintain compatibility.
> > > 
> > > But can't we use the doorbell register contents to determine that?
> > 
> > Doorbell register is not exist at old EP driver. If old EP driver register
> > size is 64Byte,  doorbell register is 64 - 68.
> >
> 
> Yes, I know!
>  
> > Read unexisted, or unmapped space will cause kernel dump or other side
> > effects.
> > 
> 
> For sure it won't be unmapped as the BAR0 size is 512B. I thought we could infer
> something from the uninitialized registers. I need to think about other options.
> But changing the semantics of MAGIC register is a no-go.

@Mani:

Any other options? I plan respin these patches soon

Frank

> 
> - Mani
> 
> > Frank
> > 
> > > 
> > > - Mani
> > > 
> > > > > Add three registers: PCIE_ENDPOINT_TEST_DB_BAR, PCIE_ENDPOINT_TEST_DB_ADDR,
> > > > > PCIE_ENDPOINT_TEST_DB_DATA.
> > > > > 
> > > > 
> > > > This patch is not adding these registers and not this driver also. So this
> > > > statement is wrong.
> > > > 
> > > > > Write data from PCI_ENDPOINT_TEST_DB_DATA to address from
> > > > > PCI_ENDPOINT_TEST_DB_ADDR to trigger doorbell and wait for endpoint
> > > > > feedback.
> > > > > 
> > > > 
> > > > You can reuse a part of the commit description I suggested for previous patch.
> > > > 
> > > > Rest looks good to me.
> > > > 
> > > > - Mani
> > > > 
> > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > > > > ---
> > > > >  drivers/misc/pci_endpoint_test.c | 48 ++++++++++++++++++++++++++++++++
> > > > >  include/uapi/linux/pcitest.h     |  1 +
> > > > >  2 files changed, 49 insertions(+)
> > > > > 
> > > > > diff --git a/drivers/misc/pci_endpoint_test.c b/drivers/misc/pci_endpoint_test.c
> > > > > index ed4d0ef5e5c31..ed0b025132d17 100644
> > > > > --- a/drivers/misc/pci_endpoint_test.c
> > > > > +++ b/drivers/misc/pci_endpoint_test.c
> > > > > @@ -33,6 +33,8 @@
> > > > >  #define IRQ_TYPE_MSIX				2
> > > > >  
> > > > >  #define PCI_ENDPOINT_TEST_MAGIC			0x0
> > > > > +#define PCI_MAGIC_VERSION_MASK			GENMASK(7, 0)
> > > > > +#define PCI_ENDPOINT_TEST_V1			0x1
> > > > >  
> > > > >  #define PCI_ENDPOINT_TEST_COMMAND		0x4
> > > > >  #define COMMAND_RAISE_LEGACY_IRQ		BIT(0)
> > > > > @@ -52,6 +54,7 @@
> > > > >  #define STATUS_IRQ_RAISED			BIT(6)
> > > > >  #define STATUS_SRC_ADDR_INVALID			BIT(7)
> > > > >  #define STATUS_DST_ADDR_INVALID			BIT(8)
> > > > > +#define STATUS_DOORBELL_SUCCESS			BIT(9)
> > > > >  
> > > > >  #define PCI_ENDPOINT_TEST_LOWER_SRC_ADDR	0x0c
> > > > >  #define PCI_ENDPOINT_TEST_UPPER_SRC_ADDR	0x10
> > > > > @@ -66,7 +69,12 @@
> > > > >  #define PCI_ENDPOINT_TEST_IRQ_NUMBER		0x28
> > > > >  
> > > > >  #define PCI_ENDPOINT_TEST_FLAGS			0x2c
> > > > > +#define PCI_ENDPOINT_TEST_DB_BAR		0x30
> > > > > +#define PCI_ENDPOINT_TEST_DB_ADDR		0x34
> > > > > +#define PCI_ENDPOINT_TEST_DB_DATA		0x38
> > > > > +
> > > > >  #define FLAG_USE_DMA				BIT(0)
> > > > > +#define FLAG_SUPPORT_DOORBELL			BIT(1)
> > > > >  
> > > > >  #define PCI_DEVICE_ID_TI_AM654			0xb00c
> > > > >  #define PCI_DEVICE_ID_TI_J7200			0xb00f
> > > > > @@ -102,6 +110,7 @@ enum pci_barno {
> > > > >  	BAR_3,
> > > > >  	BAR_4,
> > > > >  	BAR_5,
> > > > > +	NO_BAR = -1,
> > > > >  };
> > > > >  
> > > > >  struct pci_endpoint_test {
> > > > > @@ -118,6 +127,7 @@ struct pci_endpoint_test {
> > > > >  	enum pci_barno test_reg_bar;
> > > > >  	size_t alignment;
> > > > >  	const char *name;
> > > > > +	u8 version;
> > > > >  };
> > > > >  
> > > > >  struct pci_endpoint_test_data {
> > > > > @@ -713,6 +723,38 @@ static bool pci_endpoint_test_set_irq(struct pci_endpoint_test *test,
> > > > >  	return false;
> > > > >  }
> > > > >  
> > > > > +static bool pci_endpoint_test_doorbell(struct pci_endpoint_test *test)
> > > > > +{
> > > > > +	enum pci_barno bar;
> > > > > +	u32 data, status;
> > > > > +	u32 addr;
> > > > > +
> > > > > +	if (test->version < PCI_ENDPOINT_TEST_V1)
> > > > > +		return false;
> > > > > +
> > > > > +	bar = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_BAR);
> > > > > +	if (bar == NO_BAR)
> > > > > +		return false;
> > > > > +
> > > > > +	data = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_DATA);
> > > > > +	addr = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_ADDR);
> > > > > +	bar = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_BAR);
> > > > > +
> > > > > +	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type);
> > > > > +	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1);
> > > > > +
> > > > > +	pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_STATUS, 0);
> > > > > +	pci_endpoint_test_bar_writel(test, bar, addr, data);
> > > > > +
> > > > > +	wait_for_completion_timeout(&test->irq_raised, msecs_to_jiffies(1000));
> > > > > +
> > > > > +	status = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_STATUS);
> > > > > +	if (status & STATUS_DOORBELL_SUCCESS)
> > > > > +		return true;
> > > > > +
> > > > > +	return false;
> > > > > +}
> > > > > +
> > > > >  static long pci_endpoint_test_ioctl(struct file *file, unsigned int cmd,
> > > > >  				    unsigned long arg)
> > > > >  {
> > > > > @@ -760,6 +802,9 @@ static long pci_endpoint_test_ioctl(struct file *file, unsigned int cmd,
> > > > >  	case PCITEST_CLEAR_IRQ:
> > > > >  		ret = pci_endpoint_test_clear_irq(test);
> > > > >  		break;
> > > > > +	case PCITEST_DOORBELL:
> > > > > +		ret = pci_endpoint_test_doorbell(test);
> > > > > +		break;
> > > > >  	}
> > > > >  
> > > > >  ret:
> > > > > @@ -887,6 +932,9 @@ static int pci_endpoint_test_probe(struct pci_dev *pdev,
> > > > >  	misc_device->parent = &pdev->dev;
> > > > >  	misc_device->fops = &pci_endpoint_test_fops;
> > > > >  
> > > > > +	test->version = FIELD_GET(PCI_MAGIC_VERSION_MASK,
> > > > > +				  pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_MAGIC));
> > > > > +
> > > > >  	err = misc_register(misc_device);
> > > > >  	if (err) {
> > > > >  		dev_err(dev, "Failed to register device\n");
> > > > > diff --git a/include/uapi/linux/pcitest.h b/include/uapi/linux/pcitest.h
> > > > > index f9c1af8d141b4..479ca1aa3ae0b 100644
> > > > > --- a/include/uapi/linux/pcitest.h
> > > > > +++ b/include/uapi/linux/pcitest.h
> > > > > @@ -20,6 +20,7 @@
> > > > >  #define PCITEST_SET_IRQTYPE	_IOW('P', 0x8, int)
> > > > >  #define PCITEST_GET_IRQTYPE	_IO('P', 0x9)
> > > > >  #define PCITEST_CLEAR_IRQ	_IO('P', 0x10)
> > > > > +#define PCITEST_DOORBELL	_IO('P', 0x11)
> > > > >  
> > > > >  #define PCITEST_FLAGS_USE_DMA	0x00000001
> > > > >  
> > > > > -- 
> > > > > 2.34.1
> > > > > 
> > > > 
> > > > -- 
> > > > மணிவண்ணன் சதாசிவம்
> > > 
> > > -- 
> > > மணிவண்ணன் சதாசிவம்
> 
> -- 
> மணிவண்ணன் சதாசிவம்