Received-SPF: pass (google.com: domain of linux-kernel+bounces-24923-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <b9a7b855-af2b-4858-b9d8-3340a0421cfe@nvidia.com>
Date: Fri, 12 Jan 2024 23:03:31 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V3] PCI/MSI: Fix MSI hwirq truncation
Content-Language: en-US
To: Thomas Gleixner <tglx@linutronix.de>, bhelgaas@google.com,
 rdunlap@infradead.org, ilpo.jarvinen@linux.intel.com,
 jiang.liu@linux.intel.com
Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
 treding@nvidia.com, jonathanh@nvidia.com, sdonthineni@nvidia.com,
 kthota@nvidia.com, mmaddireddy@nvidia.com, sagar.tv@gmail.com
References: <20240108120522.1368240-1-vidyas@nvidia.com>
 <20240111052814.713016-1-vidyas@nvidia.com> <87bk9qim4p.ffs@tglx>
From: Vidya Sagar <vidyas@nvidia.com>
In-Reply-To: <87bk9qim4p.ffs@tglx>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?STMydkNaSjFMTW9sL0NBbDI4ZGE0ZnBjWnBwenFYQmlLZDBhQnp6Wkl3dXdh?=
 =?utf-8?B?aHl2MFBTRE50U0d2S0FiVEREMkdxMm00dWNYb3lUam5uUVNKanpWMWRxZG03?=
 =?utf-8?B?TXBvWGQ1ZWxJZy9jRnhOaWZFQVIxV1YydW9hUE1yZVFjRytWWGJwbklNdVZN?=
 =?utf-8?B?S1FUS3NxeFRGSDV3c2wycnE3UmpRTDZhOEU4aS9PNjBKRTFGT1g2YTVFT2ZV?=
 =?utf-8?B?WFBpZHZjQXBPS0dMU1NNUTNGUlRGVnp1SGVkamx2OHU2U2Y0UjdaTnRxZGZp?=
 =?utf-8?B?anc0eGZiZWx1c3dxNnl1d3JwVlhjUXI4TVEydUhVVTg1aVlVSVpoUUV6dDR1?=
 =?utf-8?B?SG1sVUJEWTBHbHJmZytHSWpOcDQwTFZ3K2xXK2hRaGFBbFFIbVFKbzhoWDZM?=
 =?utf-8?B?RTg5cDhEbDg2b1hxTFJIcEsxa0NDbjNJOEJEVHBYb1JLWktPRXlzTHZVajEv?=
 =?utf-8?B?RXd0a0NFaldLK1Y5UEk0SldHVnpMaWNGR1JST3Y5REF2bVB0K0JOK01LaWZ2?=
 =?utf-8?B?bXk3dEdmejlHKzgwcURnemwydEdyY2lmaHBGaUhOQnQ3YTc3SWkwbCtEczc2?=
 =?utf-8?B?LzEzZ1ZRSmdoVUdyUEpnU3VCT3RNME1BNjZ4L0FRSTEvazgvY3RjRk9WZEV1?=
 =?utf-8?B?WXdTKzhISk40dmQ0NTRid2xBRG9reFhRQ25BU05ibVZuR0N0dzZZQ3FoQ1U0?=
 =?utf-8?B?QVZZdXQvb1dmTEVRbXo3TUF5OHhtVUpzSG9tcjNYYWlKZlNkMlRSL1dZLytp?=
 =?utf-8?B?cWZ4a084a3NqV0ltS1ZKTFA4eVV4WVNrRFVoclBzNHkwN1JDYWlSVzNDZTA4?=
 =?utf-8?B?NFV4dDN1M25wOUUweXRDSjVNMi9vYTdBUWlnNkhRNldzcU9hbWtPMTlzYzdY?=
 =?utf-8?B?Njk3bWJQUFJmRDdCWlROTERSOHRWS2hMbkdMMkxaK3dwUDlBZXZmSGtGMXRH?=
 =?utf-8?B?V3E1TDVYK3ltOUtNZFJDc2pFNzVGNVo2aHJyRGFFR1oxQnRTdWRWRDlxenpB?=
 =?utf-8?B?NjM1SUlqaTE0TWgrUWkxKzVOWXFxdGJGOEN3eDRsWGVGa05iOGh5MEJjbjlH?=
 =?utf-8?B?ODhRNVpheFY1N0pJLzBKbWtLRlJTcmNtb1MydlJ1R3doMXpRYk52TGNxRXJy?=
 =?utf-8?B?Rjh3aVYvZHFvaWgxcXd0cTAxYWlFMUJMT2orWHZrdkpQSGtEYUR0c3BLaEJN?=
 =?utf-8?B?bkV4ZStwYSsvckh4Q3pXZlNzY0ZVWFdQOUcrVkR4c3lXZ0tXQnJiaHpNTlVx?=
 =?utf-8?B?djJ4dEZqbXgxemo5aWFsdDVkOGZweFpqaDJaSWhSTC9MT2JoWW1TdUxycEgz?=
 =?utf-8?B?dGpVSWFVWDVGaUpNRlEzZytvUWpVM1JWWHNEQUdHenFxRWJ5WHRESFhkSWF3?=
 =?utf-8?B?Uyt4dEc1MEg5enM4NVNIcWFYYjNPT2RNSVBLU0VKME5WQ0ZHWmszcDl2VHZC?=
 =?utf-8?B?aUtjUEUxK0NMOFMxcVZmTWdybXVvVjRUcnBCWVVCUnhKYUd6ZXc1TFRwOENJ?=
 =?utf-8?B?Z0J1NklEdlh5cVMzL2JsMnVQRW9rU2hJTDFBUDBRdThpUzhzUXVUdVQzZ00r?=
 =?utf-8?B?aTRqUEovbll0WmdNZU5FSGhvbmVYWVJMNmR3WFFFVDFRai90Sm9JYjRNQm9Z?=
 =?utf-8?B?a3V0QkJrTDlpQnR6bEpiSTF3RWN3S1loR0xqNkJaNzVJMjY0LzJzeDdRRTNE?=
 =?utf-8?B?cGdmTU9DSFJTQzNMQ2xLL0FIbFdHWHRHQjk2YjZZRi8wWS9CN1BRNjJITHNQ?=
 =?utf-8?B?SUpPQzRnSDE5S3JQSW9ZUksxWVRmWXFuNVBtRjlkMTg4Q29iWmMxYnRJTnRW?=
 =?utf-8?B?Q2RnR2l1dkdpU1FrZDdLdjhKMDVJVlA3K2NycjNuMGh3aGk0ckxCUTk3cVc5?=
 =?utf-8?B?SnZ2QkxBSTVPcVdxOXZIOTQrMmJCaGlycHRXZVMycExSMkJJeXIxcmRzTWEx?=
 =?utf-8?B?cHgxY2xPRVJXeUlYWHBhZk5GSTl2Q0I0UzZ6UFJTNkorVEJwT21ZRXZVRzJZ?=
 =?utf-8?B?ajF2ZVNIYVJjdjR5TDk1eVdrNUZsc3VYR3J2cjBDNnBFR1VWNm5wM1dmaDBG?=
 =?utf-8?B?MHZXVnFzdDZVYkVtbXBQczV4V3hWOUtyZUtlTVRwbVRyS2VmN0hWRjNQSVlB?=
 =?utf-8?Q?wIkIJAadxI1R9gl5am57/PzqI?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dcbf2744-a2a9-40b7-ca72-08dc13949e05
X-MS-Exchange-CrossTenant-AuthSource: PH8PR12MB6674.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jan 2024 17:33:41.0499
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GsqnFe0bqUrHY5aDuQVwWiDncV+IalpIA5nBWXtlJRXSfal66ypurbxklMY8gERDoHApe7SdUPTcvd8BFlsIlA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8078


On 1/12/2024 9:23 PM, Thomas Gleixner wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Thu, Jan 11 2024 at 10:58, Vidya Sagar wrote:
>> While calculating the hwirq number for an MSI interrupt, the higher
>> bits (i.e. from bit-5 onwards a.k.a domain_nr >= 32) of the PCI domain
>> number gets truncated because of the shifted value casting to return
>> type of pci_domain_nr() which is 'int'. This for example is resulting
>> in same hwirq number for devices 0019:00:00.0 and 0039:00:00.0.
>>
>> So, cast the PCI domain number to 'irq_hw_number_t' before left shifting
>> it to calculate hwirq number.
> 
> This still does not explain that this fixes it only on 64-bit platforms
> and why we don't care for 32-bit systems.
Agree that this fixes the issue only on 64-bit platforms. It doesn't
change the behavior on 32-bit platforms. My understanding is that the
issue surfaces only if there are too many PCIe controllers in the system
which usually is the case in modern server systems and it is arguable if
the server systems really run 32-bit kernels.

One way to fix it for both 32-bit and 64-bit systems is by changing the
type of 'hwirq' to u64. This may cause two memory reads in 32-bit
systems whenever 'hwirq' is accessed and that may intern cause some perf
impact?? Is this the way you think I should be handling it?
>