Received-SPF: pass (google.com: domain of linux-kernel+bounces-37401-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Wed, 24 Jan 2024 12:10:01 -0500
From: Frank Li <Frank.li@nxp.com>
To: Niklas Cassel <Niklas.Cassel@wdc.com>
Cc: "imx@lists.linux.dev" <imx@lists.linux.dev>,
	Jingoo Han <jingoohan1@gmail.com>,
	Gustavo Pimentel <gustavo.pimentel@synopsys.com>,
	Manivannan Sadhasivam <mani@kernel.org>,
	Lorenzo Pieralisi <lpieralisi@kernel.org>,
	Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
	Rob Herring <robh@kernel.org>, Bjorn Helgaas <bhelgaas@google.com>,
	Jon Mason <jdmason@kudzu.us>,
	"open list:PCI DRIVER FOR SYNOPSYS DESIGNWARE" <linux-pci@vger.kernel.org>,
	open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 1/1] PCI: dwc: Fix BAR0 wrong map to iATU6 after root
 complex reinit endpoint
Message-ID: <ZbFEafUmqmE0nrPx@lizhi-Precision-Tower-5810>
References: <20231219044844.1195294-1-Frank.Li@nxp.com>
 <ZYFrUWM7JXdv7rtb@x1-carbon>
 <ZYGmpaf18pJgM/qj@lizhi-Precision-Tower-5810>
 <ZYGq6RdCfdhXFF/9@x1-carbon>
 <ZZ1Pj3MvhQNMnP8M@x1-carbon>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZZ1Pj3MvhQNMnP8M@x1-carbon>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?+91QVmSUMxPXi1wYkwET9c35QpBSqME6iNw9yAg8oG8XAGmPpfXSP22UkSQ4?=
 =?us-ascii?Q?V6qoAs1oLlwBZFseBKJ9DHjAE3fGTkNQkNS/IOncHSxPD6w+xRlaK2nJ6c6B?=
 =?us-ascii?Q?Mkg3sdpDIS9ularjx+R1pKLKwdkMdypgck2vOpP2Pw+Rz1UXFSbhkSJvVkig?=
 =?us-ascii?Q?Cc3OzzyiJ5J/MEZqHkQpl8K+HJCQ9vh6KwJbyQQK+/vQq2hlRrpKgA5aY8uh?=
 =?us-ascii?Q?KsKh75dDSHM8KhNe7I7E8i+1Ehgv5jaR7kEZXQykBzEVbknO6B+jAYtCXz7h?=
 =?us-ascii?Q?bOE57Dfm/40zqPqdmizbKqykVJDp120oxntgTqh6tBCw4HFNCLk4Qk0pyOTP?=
 =?us-ascii?Q?Y5OGFu1RL00kDMonI/gu3rgnA706lHwgAIZVGS35CE3POqQU4rp9WmTNtIAz?=
 =?us-ascii?Q?kJPng81eLPg98m7PylIfuPCSk/jDmS3J0xFrmGiqGl/ZGgS5XXVjqYtiJoSM?=
 =?us-ascii?Q?uGP2polITlvp0WefxoID3RrmaBII1Jfcd7BKQSRWJ8vjhkOD7q4+sseKSq7W?=
 =?us-ascii?Q?Er4LdDhKBAeYtnq6RPH/9LVW42bEF2+9/r4fEGHLq53fvr7hXVx6XIo7rK6f?=
 =?us-ascii?Q?azLl1efggLuuKCDRVnv1yYxizwyYckHYICnZVUW7ZF2cG4T2lyR2RyVY+Ov0?=
 =?us-ascii?Q?28ISwIuSs3jmidatk6GBziCYJqagnLG3YnTKIcNZS2vCqeiefVtLiKWoM7iZ?=
 =?us-ascii?Q?nxCJzDYR84pLxmOfdAfVN1EVs3eEmJ3YSv9NQWCw5C+Rf7ApeB4e+TacsCkm?=
 =?us-ascii?Q?pj/dCjlAXmjQ2XdpeBhqI5LS+jLFM+sv/Lb/TMQV4lkpWC8J19Kk7Cle9Oft?=
 =?us-ascii?Q?ysOCWsktM3Qyp1tSkZ/808ulIaPu5dRvZBRzMy/Ed/8QRP0dM1PjpZatstCy?=
 =?us-ascii?Q?r+2qSeXW+hGOr6ErpIP/kisfcR0v9liMYmMX3yNxETyVAEU7HO0eALyu9FVy?=
 =?us-ascii?Q?QAHNJG6ZqAW8VrueYPIaRZdFhlfAx/dVVRjE6PAwVp7o2Z6zScon6oj5fwRA?=
 =?us-ascii?Q?v5gAQ2HZtXL3IwBatWW11on/vdc3MwGMStM/V6yeq9EJwDgIm+gB9nW6NR/X?=
 =?us-ascii?Q?7cAITZiaBo4R4X33iUP3WUuUDeGTxQ9u3YkmWhpnWsqvZVdSmHkchfr4zxU9?=
 =?us-ascii?Q?9suFFLzLt+3j3xrOEJn5mLTTA2H4NnuLHUPpL4Mkd8mk/iu2Nj2RtBR1Ucgm?=
 =?us-ascii?Q?GA0+2oM8auofg4PWVOa+fxNrnTBUKLHx7L6Hz94m45Cu2tSmZPZTxDQPYVdD?=
 =?us-ascii?Q?G8RLMkYdPHKiGUAa2zd3+K75RGM8TcsjYov5irfU6iUNvAQ55OEjqK5nRl6a?=
 =?us-ascii?Q?GTOQ9nPJxsjFDo/8LG0/0iEyGISDoIMqdKM1lsMO8qbbBUMgvZxnXacRBC8i?=
 =?us-ascii?Q?Dc6sQ8zuPju0JqPxC9wLdFEKzHFdqxV9HAtS3YATYDpFmEPkHibCbTnj0s2l?=
 =?us-ascii?Q?cdqSeiin4C9B9ivgPP7jVxUsWdWnrEygr1T23TvL9xzA/XTsyiPN98CzpuoH?=
 =?us-ascii?Q?AZZTxOVbKxqDtACrvWC49BjPApIhlDqwxMP5M7R8WU73MHfHWiQjjdse6nfo?=
 =?us-ascii?Q?fYnGtBX2kZ3FcS+WVgXVZ7hLta4+fIWzf1z9Wr45?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 68f5ead5-230d-47b3-78a8-08dc1cff5265
X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB9642.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2024 17:10:10.5534
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: caRfqR3NChS66Nq0Ph0PW5Z/EyEGny58QDDGjnvU+xVLELAgufk6DN/uldU1Kh/uJksLeJd1wwSSFbIYSu0GUw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7963

On Tue, Jan 09, 2024 at 01:52:15PM +0000, Niklas Cassel wrote:
> Hello Frank,
> 
> On Tue, Dec 19, 2023 at 03:38:33PM +0100, Niklas Cassel wrote:
> > On Tue, Dec 19, 2023 at 09:20:21AM -0500, Frank Li wrote:
> > > On Tue, Dec 19, 2023 at 10:07:14AM +0000, Niklas Cassel wrote:
> > > > On Mon, Dec 18, 2023 at 11:48:43PM -0500, Frank Li wrote:
> > > > > dw_pcie_ep_inbound_atu()
> > > > > {
> > > > > 	...
> > > > > 	if (!ep->bar_to_atu[bar])
> > > > > 		free_win = find_first_zero_bit(ep->ib_window_map, pci->num_ib_windows);
> > > > > 	else
> > > > > 		free_win = ep->bar_to_atu[bar];
> > > > > 	...
> > > > > }
> > > > > 
> > > > > The atu index 0 is valid case for atu number. The find_first_zero_bit()
> > > > > will return 6 when second time call into this function if atu is 0. Suppose
> > > > > it should use branch 'free_win = ep->bar_to_atu[bar]'.
> > > > > 
> > > > > Change 'bar_to_atu' to s8. Initialize bar_to_atu as -1 to indicate it have
> > > > > not allocate atu to the bar.
> > > > > 
> > > > > Reported-by: Niklas Cassel <Niklas.Cassel@wdc.com>
> > > > > Close: https://lore.kernel.org/linux-pci/ZXt2A+Fusfz3luQV@x1-carbon/T/#u
> > > > > Fixes: 4284c88fff0e ("PCI: designware-ep: Allow pci_epc_set_bar() update inbound map address")
> > > > > Signed-off-by: Frank Li <Frank.Li@nxp.com>
> > > > > ---
> > > > > 
> > > > > Notes:
> > > > >     @Niklas:
> > > > >     	I have not test your case. I should be equal to previous's fix in
> > > > >     mail list.
> > > > 
> > > > Hello Frank,
> > > > 
> > > > Thank you for sending a proper fix for this!
> > > > 
> > > > Personally, I slightly prefer your fix that saves the iatu index + 1, and
> > > > keeps 0 to mean unused. That way, you don't need the memset, and you don't
> > > > need to change the type to signed, but either way is fine by me, so:
> > > 
> > > index + 1 don't match hardware iATU index. It will be confused because
> > > other parts is 0 based.
> > > 
> > > So I choose "-1" as free iATU.
> > 
> > A s8 can hold a max value of 127.
> > CX_ATU_NUM_OUTBOUND_REGIONS seems to be 0-255.
> > 
> > Since the DWC code can be synthesized with 256 iATUs,
> > your code will not work on systems with 128 or more iATUs.
> > 
> > If we continue to use a u8, and offset the saved value by one,
> > we will at least be able to support 255-1 == 254 iATUs.
> 
> Do you plan to send out a v2?

@mani:
	Do you have any comments about this fixes except u8's problem?

Frank

> 
> 
> Kind regards,
> Niklas