Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <1cf62647-20db-646d-ae83-c2ee61f257c7@amd.com>
Date:   Thu, 25 Aug 2022 17:42:45 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v3 02/10] x86/cpufeatures: Add Slow Memory Bandwidth
 Allocation feature flag
Content-Language: en-US
To:     Reinette Chatre <reinette.chatre@intel.com>,
        Babu Moger <babu.moger@amd.com>, fenghua.yu@intel.com,
        tglx@linutronix.de, mingo@redhat.com, bp@alien8.de
Cc:     eranian@google.com, dave.hansen@linux.intel.com, x86@kernel.org,
        hpa@zytor.com, corbet@lwn.net, linux-kernel@vger.kernel.org,
        linux-doc@vger.kernel.org, bagasdotme@gmail.com
References: <166117559756.6695.16047463526634290701.stgit@bmoger-ubuntu>
 <166117577662.6695.15496626554784059239.stgit@bmoger-ubuntu>
 <2b7e3511-8e69-2e47-0f11-7f1014b38940@intel.com>
From:   "Moger, Babu" <bmoger@amd.com>
In-Reply-To: <2b7e3511-8e69-2e47-0f11-7f1014b38940@intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aC9iaXlqaXM2c2tJMWNaYU95SExPR2ZNWDJoSmFvcVdlZ3pYRW1VdnRoY0xl?=
 =?utf-8?B?Tm1rZnRValI5MGl6RzlGWUVVc0d0eE1ZL2lyc3drdmg3bmdGRjRlVm5hTTU1?=
 =?utf-8?B?SURDeWNXL0lCMGtBV2VsdTRRUTN5NWVrVFFKRERKaDByTDVXTXl5b1J6NU1p?=
 =?utf-8?B?KzJoY2FmTmN6QTU2VGZGS0VHblZ4VC9oWVNTamNnb2JjWnJzR3FsSVM1Y3Bn?=
 =?utf-8?B?ak9mQ0psYlp2R0MxSzFSVENyd1N2RXZySDYrMTZzd2djUk5XL3g0S0grbDR0?=
 =?utf-8?B?R2hFNitPWngrK25hZk9sK2xZVk93UFdzWWR6UkhPakJvblRQeHZNbldXWmJt?=
 =?utf-8?B?MDI3RDlrM0tBUEFlcnlXTFJVcmE1MXJpR0dvU2xzdUVKd2JrRUc2OFllb2VI?=
 =?utf-8?B?ZEhjZGZyQjVsU2FGTGtBdzg1Y1V4NURHNHdQdC9VSHZDSWtVNVVQMnQxQjRC?=
 =?utf-8?B?NC9VZGVydFVLZTF1bVVqeEtrcDZJaFJDR0M3cC8zWWg4S2xNZmFaTk9CclFV?=
 =?utf-8?B?ZGJWS3RyOXA2QXFZWGRTU2tkeEVkcG4wYTE0ck51TkNlT3UxZzBjSE40MUoy?=
 =?utf-8?B?NFowd0ZwS0ZoMWx3aU9zaEsrM3N5ZzRlUDVFS0RsMVVadFRibWI5aEhXc25Q?=
 =?utf-8?B?MkJsNzgrQ1RjTjczTmJ0aHYyaFczdXBGUzYvYTBhTldIc2paMS8yVzBzOWFY?=
 =?utf-8?B?dmJJL3FaZlhWSEZ0RDh2Mko2S1BJR3kzY0lHU2dEV2JCa0R3TGZXLzhxYUZy?=
 =?utf-8?B?TEFERm1yZzlyQ0VOSDFJbGx0TGNJc21zSFB1YjJzUDQzL0dicmoyWi80V094?=
 =?utf-8?B?MTlkcU1sTHQ4ZSs2Zk1VckxVRG4xbEtYV1VhaXVGc2hSL1VnSmVSSzgzU1FS?=
 =?utf-8?B?NlVMNVZweThIc3pqeDlrLzdrSzNjQmNoUmZLMzkwc2w2aG5NS2l0anB5R1Rp?=
 =?utf-8?B?Rnd4cEV3YkplemJDWGU1dkNnSFBrNnB5MjJYZ2x3b1gvU3RmREpuMUpQSXdM?=
 =?utf-8?B?SDBVbHU2eHdKWG1RWFJRVkJRZmJ5OFlkNmtad256NHp2d1k4K0RQV2Z5UE53?=
 =?utf-8?B?OTRKck53R1ZKVmhTdVZwNXlWUDZxMUZaWGpUbFBaWUh1MUZlS0QwNkM2eTZo?=
 =?utf-8?B?TG5zRC9vREREMk54VjJ4TVdNbjRWM2FneFN6ZTR5YXA0WS9rRkx4L3lNMndM?=
 =?utf-8?B?QTZHbXpJWE53YjRnb2dEeTllQ2N2ajM2Q0szQW4vcGxhRHhSVnpTUE1rL0Mr?=
 =?utf-8?B?M3NFM283ckl1S21yVEVoOGtqRG13K2wyajhoMnM1V25zMXN3bVMxNW53Vmho?=
 =?utf-8?B?K1J1YzUvSldNTVJVQU9HdDhHRk1YWDROR081aEVpVzlxR2lyckd0QW4yMElP?=
 =?utf-8?B?UjBMb3RQeDNpcWIwQ0tVaGdSTVQ2T0FhRS9FMEhzZVFvNklobDRTa0JBN0lh?=
 =?utf-8?B?VlhJWnBTeENzMWhqUHlucEZGU2hoVzBXSlIrUjdSUHdZeUhhZWNOdUVaUTI1?=
 =?utf-8?B?MkswSndKSDBqR0R3VnFMdEc5cUtXUFpjTEdYS0V1bEMxQ2V0TXU5Q3V3em5R?=
 =?utf-8?B?WlRGOUhYeXkzZnovK3lTaldvRWgvU01EVjJud0d5cXYyZmQ1V0xkQ1M5Ni8r?=
 =?utf-8?B?OFZOUHNFMmdSTDg5OEJZTDl2TzduTXI2NFNzM21nbFA4K0JhNzRwMmVid3p5?=
 =?utf-8?B?MkZ0cFZsT0NQTW5QR2svZ3pOMGZ2YW9rdjg4L3VQaTMwcFBvN3FtZzVvZ1pj?=
 =?utf-8?B?NlhDc2dnQXRKUG5RUnlieEMzL3pORTBnbjAwK3M3VVF5ankvVUMweTFvb09T?=
 =?utf-8?B?TXVZK2hzeUxxd1FqT2JzbmdCOU8wTjljMEhJVE1pMDJuc1pmSkRJangzc1Ru?=
 =?utf-8?B?VXR4eTJTZ2k4WFV4SVYwRlZMakdMNVFoc3N4NDNWQ3Fuei9WRGFPWENodDJM?=
 =?utf-8?B?bVJ0U2xjTkV1MUdaV0xxMjBraEtXMjM2aEJWRGpaTUJmUE1PZnhIZkVmRFlX?=
 =?utf-8?B?aEpjRkVjOUhHb1ZZZHFuZWNQRS93NVBmdGFYTm50VXRHZFRtdmFGdzVkWEJ1?=
 =?utf-8?B?clBHUkpwSFNmVWFiT2ppL29XLzdFaWdzNEhiYVVlTVgvYkJOTGZEZkRLUlVE?=
 =?utf-8?Q?LmRsGmk9s9J7o6yIOnxs0d98K?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d0eecb44-1f3f-40eb-9842-08da86eb23a5
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Aug 2022 22:42:50.1869
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6RVgTJLj3n9t2DbM3WkAJAQ+TIzpEVUg4qt+Cg9Tj2hcgXWG+ypjSURPJ9IFPDc+
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1945
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE,
        URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 8/23/2022 5:47 PM, Reinette Chatre wrote:
> Hi Babu,
>
> On 8/22/2022 6:42 AM, Babu Moger wrote:
>> Adds the new AMD feature X86_FEATURE_SMBA. With this feature, the QOS
>> enforcement policies can be applied to external slow memory connected
>> to the host. QOS enforcement is accomplished by assigning a Class Of
>> Service (COS) to a processor and specifying allocations or limits for
>> that COS for each resource to be allocated.
>>
>> This feature is identified by the CPUID Function 8000_0020_EBX_x0.
>>
>> CPUID Fn8000_0020_EBX_x0 AMD Bandwidth Enforcement Feature Identifiers (ECX=0)
>> Bits    Field Name      Description
>> 2       L3SBE           L3 external slow memory bandwidth enforcement
>>
>> Feature description is available in the specification, "AMD64 Technology Platform Quality
>> of Service Extensions, Revision: 1.03 Publication # 56375 Revision: 1.03 Issue Date: February 2022".
>>
>> Link: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amd.com%2Fen%2Fsupport%2Ftech-docs%2Famd64-technology-platform-quality-service-extensions&amp;data=05%7C01%7Cbabu.moger%40amd.com%7C4385e95126b24de58aec08da85597c88%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637968916632283680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=dKSxIinxZGybtACbs3%2FVZr4zbeAvXYc%2FezVivq3xjx0%3D&amp;reserved=0
>> Link: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.kernel.org%2Fshow_bug.cgi%3Fid%3D206537&amp;data=05%7C01%7Cbabu.moger%40amd.com%7C4385e95126b24de58aec08da85597c88%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637968916632283680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=f7MJmrwkBGxq8BuWjNY6Ze9NdzJc6NOkXxNjUZk5c4U%3D&amp;reserved=0
>> Signed-off-by: Babu Moger <babu.moger@amd.com>
>> Reviewed-by: Ingo Molnar <mingo@kernel.org>
>> ---
> resctrl currently supports "memory bandwidth allocation" and this series adds
> "slow memory bandwidth allocation". Could you please provide more detail about
> what the difference is between "MBA" and "SMBA"? It is clear that the implementation
In this case the slow memory means memory attached to CXL device.
> treats them as different resources, but both resources are associated with L3 cache
> domains and (from what I understand) throttling always occurs at the CPU. Can both
> types of memory resources thus be seen as downstream from L3 cache? How can
Yes. that is correct. They are seen as downstream from L3.
> a user know what memory is considered when configuring MBA and what memory is
> considered when configuring SMBA? Additionally, I do find the term "slow" to be

This memory completely transparent to OS with little bit higher latency 
that regular main memory.

Yes. I know slow word is bit vague. I am not an expert of CXL. But i see 
that word slow is being used to refer the CXL memory to differentiate it 
from regular memory.

> vague as a way to distinguish between different memory types. What is the
> definition of "slow"? Would all "slow" memory on the system support SMBA?

Yes. All the slow memory in the system can support SMBA.

Thanks

Babu

>
> Reinette