Message-ID: <76781bc7-ba6c-db6d-2ed0-3aa0e53053e2@nvidia.com>
Date:   Sun, 29 Jan 2023 19:32:18 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.2
Subject: Re: [PATCH v2] arm64: gic: increase the number of IRQ descriptors
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>
Cc:     Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        James Morse <james.morse@arm.com>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
References: <20230104023738.1258925-1-sdonthineni@nvidia.com>
 <86sfgq7jb3.wl-maz@kernel.org>
 <2a0116a8-fbd0-d866-ada0-ed50f0523f1d@nvidia.com>
 <86k0216ydh.wl-maz@kernel.org>
 <4cc4114d-7fa5-1c23-3504-0ca4dbdd0f62@nvidia.com>
 <86fscjoe3i.wl-maz@kernel.org>
 <ed471259-5d32-e93b-20d6-42a32b8b3d89@nvidia.com>
 <87pmbmer6x.wl-maz@kernel.org>
 <d64145d6-966c-2e9d-beca-b8f896c8d2f0@nvidia.com> <875ydej9ur.ffs@tglx>
From:   Shanker Donthineni <sdonthineni@nvidia.com>
In-Reply-To: <875ydej9ur.ffs@tglx>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RU43UVBQTjRzcEwvM3NMQzFVMGxFY0h2K0F2S1pnbytVWFl0RFp6ME5sa001?=
 =?utf-8?B?M2EvNmhibUIxMFZyaGx5aDF0bm9HUkJkMFRvVXhZM2gyYjlkbU40SzNuTko1?=
 =?utf-8?B?UWkrR3dnSVlHSVhhREl6SGpVVHRLTmhNZ1ZMWGZpcVVuOXlVb0R5K0UyemFT?=
 =?utf-8?B?TWt3c0tlSzM3YndLZ2t6c05yZjdMOGd1U01HbmFHNGs2Y0d5YnNzMDRkZm44?=
 =?utf-8?B?bEVCMGU2MERmek1JU3JLeUE2UTFlL2l5aFZwdXo2bmh6ampuYkxnUDdnOGlJ?=
 =?utf-8?B?d0pDeUQ0aE5rQ0ZFOTdNMTY5ZHNwNXpqQkVSaWpMK2pvTFU0MGtLSEVkZEwv?=
 =?utf-8?B?Ykd5aVlPWXQ4d2pxQWNJZG1ubC9NTm1kTS9IRFZDU1Jlb2kxTExpM0M1Vjd0?=
 =?utf-8?B?WmZ5NHIzYVpsYmFJcnZlUzBTUlVDRVJUUXJEbGJnOWdOeWVIdUJKRmF2bk13?=
 =?utf-8?B?ZVZDQ3pHTklscHBWa1c0MVlEbWxLbnVldlFkbUtWMjU0ZDdUK2tldEZxZEN1?=
 =?utf-8?B?UHlyaEI2UlZHcDJSVEZZdjhXQjJrRktNUzFrK0V5VXFIMFl6aXRCOG9hVTli?=
 =?utf-8?B?azFVeUMrV3Eyb0g2NGcvaDJoUFM4dDJsTE54NlBWY3VPbCtEbVNlOWsyVVVr?=
 =?utf-8?B?czk2eVpXNnNiUkdIckw4K0o3TW9lUjYwa1lJVXg1LzVVdDczNW0yamt0dE1k?=
 =?utf-8?B?WnRCbExjakNDRXVsS0VzQjVrNU80OUlHRlN2bTZuZWpWbi9HY2gzRDhuMjY1?=
 =?utf-8?B?RUZlQTRLOG85TER6M1lhYm0wZUxlR3JYVmg0TFErMHdqcUsydWhXbnlIWTlI?=
 =?utf-8?B?MVR4VjFJREpUazh6SjJ4bkt5Qm1QWHFjT0V3bnZHV2hZRk9LeFkyYXNzV1Zh?=
 =?utf-8?B?M29STnhqU1A0eTFmUEVpVXVQUWdhTmtEbzVFUnlySk4vK1RBb25pRFE4d3ls?=
 =?utf-8?B?VU1zUXROaGxTZ1VmTkl6YXFidlJzT1BzTU85WEJTWDR5OEVtM0ZiOEdZQXJZ?=
 =?utf-8?B?QjNjeHljcHRqNVdMUEVzbjdpbWVnRjVzZzVQY1dVOHNrUjYxc0hqa0tLSWkw?=
 =?utf-8?B?U1U0anlzUHNxQVNWWGxxQnN5Zm1VdFFiY2Z4cVIwaFU3cHp5UWNhUjNFbEdF?=
 =?utf-8?B?SE1tbDgwMDdSbVMyb2w3UlpFUWR1QVVQb0pMaFUwNWRlM2VhK3RLa2thRWdk?=
 =?utf-8?B?S1I5SGx1cU5MM1cremlxdlByVkh1akE5ODRRMUs3Y3lEc3VtN0JqeklyQ0t3?=
 =?utf-8?B?bWEyczRLUi9OZFNUa3FHa0J3UHY3RmJiYnExWHZzd0tpcnk5K2FUM01ma0kw?=
 =?utf-8?B?cldrVldmQzQ5czBoanNxOTIwVkgxS0ErVDc1bytBYXE2SmxOM1J5ZnRFY1ZG?=
 =?utf-8?B?Tm02TnZBT2Ewd3EyOURUajkvTW0vbHh6eHdQWW81QkNUKzVDNnkzcElKNCtW?=
 =?utf-8?B?OXlTcnlxYnJkbngreVY4ZmFRdTVNVm84YzFoR1UvbStCZFRGa0VpcS91Y05N?=
 =?utf-8?B?K05zQ1FkdW12MWZvQitOaWJ1bEx5OVowNHovY2dBSGpFTHJLT3RDTlJydGVx?=
 =?utf-8?B?OFFuV0ZITlV0MHg3cmpGMVNoT0NFMXB6YnJieERpSDRRNnl4RVU4SzUyaUNt?=
 =?utf-8?B?Vm16b0VhS3dndllWejM3Ni9BR3ZXVXI4SFZpQnBqRTJhQUdvTXRQRUQwYjhi?=
 =?utf-8?B?NUVFdmFuQjc4dVhSblVXSEdkVnpCZUdqbDFkWGVWQkdhTG0vWnZIS2k5L2Rk?=
 =?utf-8?B?TXZZaUR1MmZDUUI0OCttWWZBT1hGNVFxSDF5c0s5anIyeVMyZjhEQ2paMjh0?=
 =?utf-8?B?ZGZNZE5FN2MrNFA4RWNsZXUyVDFyckRlMDZvcWlnYlZzQ25RWlE2ek5BK0la?=
 =?utf-8?B?VVRvak5pcHBsTStqSFhIUGVKZS9xQTRjNW96M2dtWk5vVy9RNVVyRTQ5dTcr?=
 =?utf-8?B?MFpxVTJUOXBKMXBhUDFvZmIwRm5ZNTBxK3BIM0lzSnZTdEVvRHl3eXl5RmZR?=
 =?utf-8?B?RUNXRW11WkZsdjNoc2ltajlLVEtDUnBibE5paTNsTjNETVl0MHBRZ1JCOGtp?=
 =?utf-8?B?N084enR4dVliMGxITTFLTXRIampmS01HVDhFSTdLaUV0U1hBbHVyMUpUK1N6?=
 =?utf-8?B?UDl2ZFhXVmFYNUtnUjZQUGI1ZmdqSDNQdlpJMnl2RGw3a0poWFlaOENqNVds?=
 =?utf-8?B?K1E9PQ==?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 75ef295f-6cd8-4916-449e-08db0261d42c
X-MS-Exchange-CrossTenant-AuthSource: MN0PR12MB5762.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2023 01:32:19.9841
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A0bE2b1pKZpLxUh7cPDimpsIZWCMi+sZCn05w81M4HBhj1hOYVMdcCLNhMy0W0uaRtU+k8x8Paj2X1MO3G7hGQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7428
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Thomas & Marc,

On 1/10/23 16:36, Thomas Gleixner wrote:
> You can simply use a maple_tree for this.
> 
> static MTREE_INIT_EXT(sparse_irqs, MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN,
>                        sparse_irq_lock);
> 
> And the functions become:
> 
> static int irq_find_free_area(unsigned int from, unsigned int cnt)
> {
>          MA_STATE(mas, &sparse_irqs, 0, 0);
> 
>          if (mas_empty_area(&mas, from, MAX_SPARSE_IRQS, cnt))
>                  return -ENOSPC;
>          return mas.index;
> }
> 
> static unsigned int irq_find_next_irq(unsigned int offset)
> {
>          MA_STATE(mas, &sparse_irqs, offset, nr_irqs);
>          struct irq_desc *desc = mas_next(&mas, nr_irqs);
> 
>          return desc ? irq_desc_get_irq(desc) : nr_irqs;
> }
> 
> static int irq_insert_desc(irq, desc)
> {
>          MA_STATE(mas, @sparse_irqs, irq, irq);
> 
>          return mas_store_gfp(&mas, desc, GFP_KERNEL);
> }
> 
> static void irq_remove_desc(irq)
> {
>          MA_STATE(mas, @sparse_irqs, irq, irq);
> 
>          return mas_erase(&mas);
> }

Thank you for providing the necessary functions, they have been extremely
useful in getting started with implementing patches. However, I have encountered
corruption in the maple data structure within mtree_load() when multiple virtual
machines are being shut down simultaneously. To address this, I have added the
flag MT_FLAGS_USE_RCU to ensure safe concurrent access during reads and writes.

Please review patch series https://lore.kernel.org/all/20230130005725.3517597-1-sdonthineni@nvidia.com/

I have applied 6 patches from
  https://lore.kernel.org/all/20230109205336.3665937-2-surenb@google.com/#r
to resolve RCU mode issues. The patches were tested on an ARM64 server and
underwent several hours of evaluation with multiple virtual machines, yielding
positive results.


Thanks,
Shanker