Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <f7c88827-fa50-0f5d-4a9c-ef147e8820d3@intel.com>
Date:   Wed, 9 Aug 2023 15:33:24 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.14.0
Subject: Re: [PATCH v5 05/24] x86/resctrl: Allow RMID allocation to be scoped
 by CLOSID
Content-Language: en-US
To:     James Morse <james.morse@arm.com>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>
CC:     Fenghua Yu <fenghua.yu@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        H Peter Anvin <hpa@zytor.com>,
        Babu Moger <Babu.Moger@amd.com>,
        <shameerali.kolothum.thodi@huawei.com>,
        D Scott Phillips OS <scott@os.amperecomputing.com>,
        <carl@os.amperecomputing.com>, <lcherian@marvell.com>,
        <bobo.shaobowang@huawei.com>, <tan.shaopeng@fujitsu.com>,
        <xingxin.hx@openanolis.org>, <baolin.wang@linux.alibaba.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        Xin Hao <xhao@linux.alibaba.com>, <peternewman@google.com>,
        <dfustini@baylibre.com>
References: <20230728164254.27562-1-james.morse@arm.com>
 <20230728164254.27562-6-james.morse@arm.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20230728164254.27562-6-james.morse@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SVBYOTlmbjlocGdVb0lJYlY0SjlDSkpHcHF3UnVsQmdjcjFlZG8rZ2ZhMWgw?=
 =?utf-8?B?T2F6Q1h5Z3ZQTW1kdUdGVFQxRmgrZ3J6Z0U3bm1DUkIxbjkrQjV6UzBmVks3?=
 =?utf-8?B?RlRhTkNxRXFzdVBxc0ZjcjlzRTgzWlVZWE8xWTR3L1lSalpLMFdXQXhVbE0x?=
 =?utf-8?B?S1hZM0dITHFaWjI2Skdld0tETXdSR3ZwQ0hrYkdGaExNbEYxRzNvRW1TQVJn?=
 =?utf-8?B?Z3ZZQ2lOb1I0VWc1NWJLeU9OMmNlaHc4Q3dEZTZwTnFleDNQcThSRDlmdlpm?=
 =?utf-8?B?b0JTd0NCRnZLYUdJZHZ3UExHQTQvN0NQd1JsQzZ1MVo0V3FIUkdaNlpzSHNE?=
 =?utf-8?B?U2lkSmJPWUFQWVJYbUdRcE04dXVlUCtnVzNmUTA2TmFReFc0cjE5SHQxUkVO?=
 =?utf-8?B?bmhsM1hIeDhOUXIvTy9yTlUyMjk5UFpvaWpTTkFPaXkrWUtkTFZwS3VIU0Qv?=
 =?utf-8?B?RzNDTWo4dHZVSXB0QkFGVU9acVdhclVBaTZwN3BQZHRZOHJ5dDZKRE5kRDdX?=
 =?utf-8?B?QkkwcHF4OVBOTGVMTjl5MU9SeWs0a0oyZTlxVDgwOVBRcDVnVDl3Z0kxamdW?=
 =?utf-8?B?elZib3ZtWDVSRzdPVTNhb3l0WFdwOFlDUTRLYmRwQzdjQVZHaUxXckMwd2Zs?=
 =?utf-8?B?OHc2STBGWDRVbHZRRjNRa0dXZEdKOUpMQ2h6bUVaYVQ5ZVlVYW1nWEdMbktJ?=
 =?utf-8?B?N2RQUzBuUTQ1WVh5RDltUzJBQVlUVUZER1pGM2E2T2Jic01laWVkaWtEK3pq?=
 =?utf-8?B?Y1VtMmRoV01Pek9SYkxqN2ZUUENnQzIyM0NqNnJOS2JNaTNkRDR0MGhZT0dU?=
 =?utf-8?B?ZU9ZK1JKVFFSNVgzWXU1YmFZRWNITFc5NzZSV1lQdzYyK3RvdkQzWE83N0Fa?=
 =?utf-8?B?aVQ0Z1MvYndRL3ZZbWVONzJZbjdUWFoxWkNoY0tvMEhmNzhETlQ1U2hXK1FY?=
 =?utf-8?B?cDdWc0ZqdWtpbmtSYnpGWWVDdU0xclFOZTJDMWxZY2dxUWNmTng3REJUYXc5?=
 =?utf-8?B?WC8yQmNvOE91NGM1TUkxS1V1NjNpZmRDS0RETXY5SmxyZWhzdm5BYWVhTU54?=
 =?utf-8?B?VHRPOCt2M0ZwSUZzWUpiNlpzRnRJaUV2cVNVOUowOW1EUVlkUVlxaWdEcXBJ?=
 =?utf-8?B?Y1p3V29WdGNoV09PMnRRMzhHQi9GckNZL1prSmlBSC9MTU9Sby9HSEFrdkZs?=
 =?utf-8?B?M2w0bmpZV1pjWkZiMTFmSG1jN3VUWE90K1JzZTcvcVNxMTk5bmF3ZWo2RVdT?=
 =?utf-8?B?bnhEeGRSejZzemEwT3dYcDU5L3RjbFQ0c1BNNXNWZS80QVp0cisrYlFDdWxM?=
 =?utf-8?B?UVZiWGMyMVFNbnZWRk5pZkx1QXBhU3JNNnJoR2ovZDRaRUsxY1NMVnBSWmp4?=
 =?utf-8?B?NFFNZ040YmtTbDY1ZjdXMjR6bGRPRUc4TUNwS0hqUDdHUW5uc3RObE8wOTNJ?=
 =?utf-8?B?TmY5dkxHbExDUjA5MHFGRjBGd1VncWlQUlphMWJzODNmV2JWUDkvb05Fd2FM?=
 =?utf-8?B?bWRCUElmNGdER2lIaW5WRkJyYXVuWkp6NkQrL1ZVNHFHdFNSNUNDOEUvLzZM?=
 =?utf-8?B?L2I0c2h3Y1huN05xZ3daN2NhYjV1NytvcWtJN0cwam1wWk4xemRTOEpNc3pP?=
 =?utf-8?B?SGIzeGtKMW5NdnRkRkR1cUFkZ0N6RFkwQW5BblBmK0RZZjVwRkxYbUxJdmpX?=
 =?utf-8?B?TkdHNC93cStaOU9Id2hHZG1RQlFSdDFmYUJUakRpQjB0VHUxQjVpV3B0dFZM?=
 =?utf-8?B?MHRFOWY3cWhrY2VTZkRZdFkrRm5CMGxFZjVDQ1lnRDI0MVZiZEo5ZHhudmls?=
 =?utf-8?B?T3pBTUtUZTFaTE44dkF0ZWxIemNOUDBTU2ZuNWQ2aFNUMFVQdWhJK3hDNEdB?=
 =?utf-8?B?bFBCaTF6ZUE0NisvRmhZRlVxSnN4YXBtN1NhSUNOYnhCVHJtU29QbU9DSito?=
 =?utf-8?B?MGw5dFFYUXIyUGFIVGpDYlRHRUFaaDVFMklHN3Fmcm9QN21tWVhBQldLRjRp?=
 =?utf-8?B?SzRpZTJXR2xtMVE3VjNuckFiM1pxZ1RqQXFvRTVlc092QTdncHpIQXl1WW84?=
 =?utf-8?B?UXBPOGE3ZWdxZTBkZUV4eVhTV0x3aWtHdDhScGMyQ1d6M1YxRVZQTnhVVVo1?=
 =?utf-8?B?SGcreUhreVFmSWNPbjM2MFYwR3A5RmFIMzVZQWZBQlR5SytseEs2MjlpNlZ3?=
 =?utf-8?B?NGc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8ed4f10c-2c5b-4751-747c-08db9928a643
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2023 22:33:27.1753
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Rrr8csfRiGOy7BXsVyixRCH2J+Y+Igv0DeEhPYuE1avHqOv6j8CzMQYSC3cyAvnuX3gFDvarAfpH1beNYQx7edc/xL4OG8qovZWZo9aDxas=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR11MB7413
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        SPF_HELO_NONE,SPF_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi James,

On 7/28/2023 9:42 AM, James Morse wrote:
> -int alloc_rmid(void)
> +static struct rmid_entry *resctrl_find_free_rmid(u32 closid)
>  {
> -	struct rmid_entry *entry;
> -
> -	lockdep_assert_held(&rdtgroup_mutex);
> +	struct rmid_entry *itr;
> +	u32 itr_idx, cmp_idx;
>  
>  	if (list_empty(&rmid_free_lru))
> -		return rmid_limbo_count ? -EBUSY : -ENOSPC;
> +		return rmid_limbo_count ? ERR_PTR(-EBUSY) : ERR_PTR(-ENOSPC);
> +
> +	list_for_each_entry(itr, &rmid_free_lru, list) {
> +		/*
> +		 * Get the index of this free RMID, and the index it would need
> +		 * to be if it were used with this CLOSID.
> +		 * If the CLOSID is irrelevant on this architecture, these will
> +		 * always be the same meaning the compiler can reduce this loop
> +		 * to a single list_entry_first() call.
> +		 */
> +		itr_idx = resctrl_arch_rmid_idx_encode(itr->closid, itr->rmid);
> +		cmp_idx = resctrl_arch_rmid_idx_encode(closid, itr->rmid);
> +
> +		if (itr_idx == cmp_idx)
> +			return itr;
> +	}
> +
> +	return ERR_PTR(-ENOSPC);
> +}
> +
> +/*
> + * For MPAM the RMID value is not unique, and has to be considered with
> + * the CLOSID. The (CLOSID, RMID) pair is allocated on all domains, which
> + * allows all domains to be managed by a single limbo list.
> + * Each domain also has a rmid_busy_llc to reduce the work of the limbo handler.
> + */

I find the above comment to be contradicting - it talks about a single limbo list
yet there is "also" a limbo list/bitmask per domain. Should "single limbo list"
perhaps be "single free list"?

Reinette