Message-ID: <b7c3407f-beed-c80d-a2f4-52acdb59dff5@amd.com>
Date:   Fri, 10 Feb 2023 20:40:07 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [RFC PATCH 0/5] Memory access profiler(IBS) driven NUMA balancing
Content-Language: en-US
To:     Dave Hansen <dave.hansen@intel.com>,
        Peter Zijlstra <peterz@infradead.org>
Cc:     linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de,
        mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com,
        x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org,
        tglx@linutronix.de, yue.li@memverge.com,
        Ravikumar.Bangoria@amd.com, ying.huang@intel.com
References: <20230208073533.715-1-bharata@amd.com>
 <Y+Pj+9bbBbHpf6xM@hirez.programming.kicks-ass.net>
 <4808d3fa-bb68-d4c8-681f-0b2770d78041@intel.com>
 <369bef08-92cc-9b55-823f-1fe780532967@amd.com>
 <9848226e-2574-b8e0-4fb4-316957f57fb5@intel.com>
 <e0a8ff65-b0f9-1f63-a3a0-2986eb79846f@amd.com>
 <fbd1cf44-3643-4274-5a42-3516d4842830@intel.com>
From:   Bharata B Rao <bharata@amd.com>
In-Reply-To: <fbd1cf44-3643-4274-5a42-3516d4842830@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TjA2aUZhYWQ4cUM4dDlKcS9rYXdGRmdLZjYrSWFRbjZSajdaWUU0eVBwMnl0?=
 =?utf-8?B?SDdEU1RJSU82YzVwS1liMFRMcFc0a3YrTG8yYUF3cmFPMmk5VDNtY2dYNnY1?=
 =?utf-8?B?R2wyM1lEMWJtaHJYT094T2VNdWhWNFdFWFoxS3dwd25ZOGdYd0VkUVBUS3c1?=
 =?utf-8?B?dHBhMkpXL3Y1anhnbFJ2dmRxZ2RKVGJUSDNQWkVGUnVlUzlUNCsxYjM5VjNE?=
 =?utf-8?B?RFlOKzB5TzhaVnBhZTRWQlNvdDVNd1ZmSlViSFI5NmF5UHlwZHBOY1pSR0cr?=
 =?utf-8?B?NWgvRExzUis0YkdTQ3VkYTM4emFDRkxMV3hUVlVBclBSekJ1UTR5SUVJOGh3?=
 =?utf-8?B?VjJLanZaTm10dllNMTc4aG42T2hGSXA3WGo5UEJydVBRYXdoa0pNMHBUbVRO?=
 =?utf-8?B?L2pzOXQyL29jRE9FYzdtb3BtWU13ZDdyQVVmNTdsR0NkTHZpbkdqc3F1SnpZ?=
 =?utf-8?B?YjE0KzI5SkNyeEdYSENhUWdkOWQ5b2IzRjJxTVNyKzhZdmk4eGdrdlNuT0Ir?=
 =?utf-8?B?YXlrSjBocjB4VlJpQmZKUitOaDN1R3JQZldlRmIvM1RBMmJFUTM1a0RQYitY?=
 =?utf-8?B?MGUyRjZyNGxmQ2lPWkw5cjh4SWs0WE5zRytwd0ZoYmtWYzRvV0gyMUFSRlgz?=
 =?utf-8?B?VVljZFB3T2QxUVVXRW53MDZaWFRlUHB3bzBWZ2RkWDRYOFhJdTRKTWRuaGk1?=
 =?utf-8?B?U0l4ZDh1MU1WU1JTQXJVSkh6TUVxSVBRY0VjcXpjUmZ4SjQ5Yyt1d0htcWZN?=
 =?utf-8?B?SlNZa1ZyMytXT0dFaXdtSHBSMmpOUUZFbWxOTUlZSmxPa2Z1Yy9TalBiZ3J0?=
 =?utf-8?B?aGNBM2txaEpqYXRhdU5ST3NRSERpRXNMTlFsKzBkTW5aY2ZXSjBQL1Zvd2w4?=
 =?utf-8?B?Y29MU0NVZlY5NGc3Y3dCWW1lTXczNjlXallWR2lJamJWcUJKU3docjdlNnhE?=
 =?utf-8?B?OUE1Q2E4dTVOTWE2S2M3YjVEYktRZ0dhUXBxaXY0T0ttYUVpSkVPbm1jMFVF?=
 =?utf-8?B?WDJBVnRuK0paczlHeXVFS2U2azNwTnF5UmlQMEtBQ1ZKMys2SGxWUThtcDVh?=
 =?utf-8?B?WGwxUmgzZ1FXcVE0OVpJY1JJOHRLb0hGZVNnczBrZEdYTER6eWM2cG5FZGdj?=
 =?utf-8?B?TzE3SmJKbzlOZ0MrYlVmRi9Vd1VGNDhPbDVFUEtjL2U3MjRVZUptV1J3cm1p?=
 =?utf-8?B?Yy9ucmp5YlFPbER5K3RrOWFjSFU0cjUxMVcvcGRhejhTVkN2Y3VodnRPdkNr?=
 =?utf-8?B?c21FbXcrdEpnT1VjTkw4VGt3S0lkd2tuUEhwTDh2b1NpR3ZaQll1QjVLTDJz?=
 =?utf-8?B?bHdtRk5oMHUvWjRXRzNDVEpKQW9WYVhxbWhqOEdxVDBSRm1UNVc3MVNmdFJp?=
 =?utf-8?B?dHI5eHFja3BxYlBuekU3VHlGRnY0NnBxTG91ZHVqaEhqU2pkcFlRWHMrVDFx?=
 =?utf-8?B?QXZNblhXTElpc1ZsZ3VWUTlWaUFORU14b2prMDgxd3JwT0xHSHp1SGVnZ3Iz?=
 =?utf-8?B?R3BIaVMybEpkdVNldS9XQnYyWlhMeVIweTQrK3lhSU1KakYzRE9sVFo4WTFy?=
 =?utf-8?B?SzBHNTVuVjZpT2wwaVdhUlJKcXIrajUzSXI3d2lHdHRVS0ZETUxYbTRrT210?=
 =?utf-8?B?ZHAzcWJwR1VQYlhDQ1RrQ2xWMUJiVGQyTVg1eFFSVEY4eVNOQkJsMEJ3TUZt?=
 =?utf-8?B?RVExTnFhNFRVdEdrUDNqcHp6bG4xaVZwdDhwdXZ6RDJaSUJpR2ZjTit6V1kw?=
 =?utf-8?B?dmlrZy9pUjdabnZPR0VBR21YK2RJbWxaSUlCUFc4Sk9pUjN3TFN1WjlqcE9G?=
 =?utf-8?B?cFYxRVBxUkVIbUdNQ1RjdGRxOVhxQUpEMUdxS3hsZDZQUGRmMEl0c0ltbXNN?=
 =?utf-8?B?bCs1N25Ed3JHVG9RV25UVXUvQ043QmJxcVhwaklmclppNG0rZVdwZ1FSa3R3?=
 =?utf-8?B?Q2dXMUsxT0pGTWxrZHJLY1hLTmU5Y0wzUjR3WnZrci9RcWdEQk5tMG5rcUpx?=
 =?utf-8?B?L3B1Q0R1WXM4UTRQM3FTbGxMSk1nTUpUeGRTNkx4aFFRYjRkR1AvVEFFYTM1?=
 =?utf-8?B?ZmN6VkhPck04SmhLeVVnQ0ZDalB1QkxUQXdyOTVqaktqMkx2b1lZL1N3WHdY?=
 =?utf-8?Q?7TvFmJhsLnf/DLDKUp27fsCh8?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9bd6f52a-c62b-4077-7951-08db0b78ed0e
X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6434.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 15:10:20.8696
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sXtfFR7D7Xo/ZESIk0xOBhGvX33JHv6QsMregKeMbCqLnsryJ5LR0aCG1OSyY6qsTn4Ty8/uX+iL7wfGLhZ13Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6078
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/10/2023 10:10 AM, Dave Hansen wrote:
> On 2/9/23 20:28, Bharata B Rao wrote:
>> On 2/9/2023 7:58 PM, Dave Hansen wrote:
>>> On 2/8/23 22:04, Bharata B Rao wrote:
>>>>> First, IIRC, Intel PEBS at the time only gave guest virtual addresses in
>>>>> the PEBS records.  They had to be translated back to host addresses to
>>>>> be usable.  That was extra expensive.
>>>> Just to be clear, I am using IBS in host only and it can give both virtual
>>>> and physical address.
>>> Could you talk a little bit about how IBS might get used for NUMA
>>> balancing guest memory?
>> IBS can work for guest, but will not provide physical address. Also
>> the support for virtualized IBS isn't upstream yet.
>>
>> However I am not sure how effective or useful NUMA balancing within a guest
>> is, as the actual physical addresses are transparent to the guest.
>>
>> Additionally when using IBS in host, it is possible to prevent collection
>> of samples from secure guests by using the PreventHostIBS feature.
>> (https://lore.kernel.org/linux-perf-users/20230206060545.628502-1-manali.shukla@amd.com/T/#)
> I was wondering specifically about how a host might use IBS to balance
> guest memory transparently to the guest.  Now how a guest might use IBS
> to balance its own memory.

When guest memory accesses are captured by IBS in the host, IBS provides
the host physical address. Hence IBS based NUMA balancing in the host
should be able to balance guest memory transparently to the guest.

Regards,
Bharata.