Received-SPF: pass (google.com: domain of linux-kernel+bounces-37895-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <2c7c6afa-7750-413d-8d35-048d6813d93c@amperemail.onmicrosoft.com>
Date: Thu, 25 Jan 2024 10:42:14 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] NUMA: Early use of cpu_to_node() returns 0 instead of
 the correct node id
To: "Lameter, Christopher" <cl@os.amperecomputing.com>,
 Huang Shijie <shijie@os.amperecomputing.com>
Cc: gregkh@linuxfoundation.org, patches@amperecomputing.com,
 rafael@kernel.org, paul.walmsley@sifive.com, palmer@dabbelt.com,
 aou@eecs.berkeley.edu, yury.norov@gmail.com, kuba@kernel.org,
 vschneid@redhat.com, mingo@kernel.org, akpm@linux-foundation.org,
 vbabka@suse.cz, rppt@kernel.org, tglx@linutronix.de, jpoimboe@kernel.org,
 ndesaulniers@google.com, mikelley@microsoft.com, mhiramat@kernel.org,
 arnd@arndb.de, linux-kernel@vger.kernel.org,
 linux-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org,
 catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com,
 mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, chenhuacai@kernel.org,
 jiaxun.yang@flygoat.com, linux-mips@vger.kernel.org
References: <20240123045843.75969-1-shijie@os.amperecomputing.com>
 <4a13353c-cf4b-a388-5776-389c61c63ec0@os.amperecomputing.com>
From: Shijie Huang <shijie@amperemail.onmicrosoft.com>
In-Reply-To: <4a13353c-cf4b-a388-5776-389c61c63ec0@os.amperecomputing.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bGUrRWxjTmlTOXBHeFhvTWRyb2E3QkE1SmcxRHVFWXFleEQ4QjRNNGtPam4w?=
 =?utf-8?B?SlFXRXE0cGxaNTZIMzlqUVQyb1l0R3BzWVB3T1NlUk9jRlppbG0zU0sySnp5?=
 =?utf-8?B?bTJiT2Y3Yk5FNkZKZ3E3RmI0SDlHS2FrUGYvSzJZaGpRQ1A1Tm4rOVc1dlRv?=
 =?utf-8?B?L0tCVXUyUUNrNys1TmRRa3Rwc2ptWU1JTkhNS2c3SGNWZjNiSnNxUkN2SjJN?=
 =?utf-8?B?ODdUa202ZGZnT1oxVGlIK3FWRUE3UXFmYVBqbGxnRU1nT0ozcXFDdTg1UzBI?=
 =?utf-8?B?S2NKTFMzQnJ0WGhGMVlxbkJFR05jY2lPR1Jnc0Z2TzZ3WXVFTWtyVmhPbldX?=
 =?utf-8?B?ZmthNkdUeENmVVJmVFQ3QzB2aE5lOTE1cWJxZXpDVEJpaS8yNm9xMXRVWWQ3?=
 =?utf-8?B?ekpSOEdvcUdycS84VDNiSy83UDFYcExYdWRrYjFLcUdRaHlRRUc5MGVCTGYr?=
 =?utf-8?B?OURxYlZhbmFjekFJSUg2Vk41UklUMHBROXFoZ3NzcHlsRGZCTkhHbzRnandX?=
 =?utf-8?B?SnFuODlkWnY0Q1hsM1IvbzBBNGJDVE5WWFRzcXVtUUNPc0toN0s5MVNYSC90?=
 =?utf-8?B?cm0wTVZGS3pjdjNDcEZJNjFrbnpra3lHV2FWMTBPWUphSzFwekJHaUlyNkFn?=
 =?utf-8?B?YWJHeFBzYVBRLzJkMit1UnhmNlFpKzBJTDlqNFE1NFZINkhYVXZMcnZZNDhZ?=
 =?utf-8?B?SGtxRjVkaTVkZDRqclR4QjVGRXJpTUg5RmJWLzhSTGhlaEVqWTgvMDd4RjAy?=
 =?utf-8?B?QTM3ZnBHVjVLeW0xcElUQnh1R3ZFQkZZWUlHa2QrVDhFMEhac2s3cjJQMXJh?=
 =?utf-8?B?Z2hYS1FWOWVyWEY2U0xjMkh2UGFXNWo3Vm5wMnZPTzVjQzVOK241cnJDMmtm?=
 =?utf-8?B?aEtmbTYwbjBWYmhhM2FkTWJ2WGJpK1hVWm9Pbk0xTkJqUldYTjg2MUpQako4?=
 =?utf-8?B?Z0tUMlhNcXlLbXVBR2NBYVRhMS9KLzRsZmNFUlJlb2kwSTJ1YzJiTEtHcFU3?=
 =?utf-8?B?M0VRcTY2RXhtZVJ1WWpSR2xPWi9ZWEZLQSsvVjM0UHFkeXhNcFZodTVNZWZj?=
 =?utf-8?B?R1BmUFl4R2NMSVVpTzVDY0NBYzJQemhvS1pUL1ZWUjE0OG55aU5raVBQKzg4?=
 =?utf-8?B?R2g1WUUycTQrcXUvK0UrenJrK2FZejVIR1BEQ1MrdENTNnBQc1BoOHJqVEVP?=
 =?utf-8?B?Znl1ZmM4QjdibW5oSnV6MDI2blJudHZsQ2JSa0VxUzR1K0RvWkV2dThjajNT?=
 =?utf-8?B?S0RoNzRXcyt4bDFhTHhHTVo4bktHMlBVN3d4eFRDbm56R09vQUxncmVrK0E5?=
 =?utf-8?B?Z0ZNYlRQVGhzZVFlT2pvWnJ4aGl0VDdYNlNEUW9RNkhScU1wQUxLOEJRZUtj?=
 =?utf-8?B?dldZNWx4YXNKZDZiUDBTNkFIbmRYQkFrd0xFN2Q0YVFiOUVLcFgvY2hGMGJs?=
 =?utf-8?B?OTZwNDV4ZlJsRlRWaVRKc1d1eDhlL1Bla3RteFlSYVFEWStuYng0VDh6dy8y?=
 =?utf-8?B?d3d6cXZTV1lBN0xXUVlPSmI3TmdyVGZrYUhOM0NBVmIrTzA1bmlzcDhwYktm?=
 =?utf-8?B?dGt6cFhjZnAwRkJhTWVxczFiV0lLWTZraHF0ajFyc25CcGdhQ2JBOWJXSEdm?=
 =?utf-8?B?M2QyRFpOM1ZLU1FVV3ZkQmpYUURhelJuenp5UExDV2s0OW5WZ1FjVjBxM3hw?=
 =?utf-8?B?dGpDNmE4Uk83WGFuOGdoM1ZlRHBpQWlPQ1BjdldvUjZtUEt5WkVZY1JydllQ?=
 =?utf-8?B?dmNUMVN0WlhoNkptd0llN29Ybnh1MVNqODhSOFE0RjJoeWVhR1EySmgwRWxh?=
 =?utf-8?B?UFNpNGNUQnNpaWh5Vm5ud0RHZEkyWEZXR25ub2wvMUJWSWtpeXVpRlAvYzJs?=
 =?utf-8?B?Qit1UXlWSmE4NnpZeTNTYUNNNFM4dU1hbUtmVVU1M3BrSDYxbUN0TlM5VndX?=
 =?utf-8?B?Q3dRS0ZGemZvajFNOFFoUTNWWEFDZTVyakN1ZVNmVVJxRWtNaHNTdnZOSGZx?=
 =?utf-8?B?TS94djYxU2F6SzBNMHZKeHcwMEkweUtFbDVOS0szUjVjdlFXcUJqM3ZQWDN5?=
 =?utf-8?B?d2djdDlPNFN6cU8zUVdiRkJZazN0d0JuZDRJOHRUcVV6MmtMdDlRR2ZDNUtJ?=
 =?utf-8?B?SnB4YkV0WDZLWUhWQmpVb3loMWlMWm11UHVPa08yYzY3RFI3MzA0ZHVsRXc2?=
 =?utf-8?Q?N/c14SdkrGfuTJvileArV8s=3D?=
X-OriginatorOrg: amperemail.onmicrosoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b2648b7-57b1-4a81-ed36-08dc1d4f4dfd
X-MS-Exchange-CrossTenant-AuthSource: PH0PR01MB7975.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jan 2024 02:42:43.3496
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 41q/FE+kcUBSq0XcZ9pvymbvVxbh2aJqWriuXtRweZZziHOvTBo/rRp/nmww4ANTTPBN5ABYXMEQTxx/0134NYm7D9h9t3nzLQGPlVYlm6C0pXqEQwl9hRU5S/Sp+ZA5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR01MB8252


在 2024/1/25 1:19, Lameter, Christopher 写道:
> On Tue, 23 Jan 2024, Huang Shijie wrote:
>
>> During the kernel booting, the generic cpu_to_node() is called too 
>> early in
>> arm64, powerpc and riscv when CONFIG_NUMA is enabled.
>>
>> For arm64/powerpc/riscv, there are at least four places in the common 
>> code
>> where the generic cpu_to_node() is called before it is initialized:
>>        1.) early_trace_init()         in kernel/trace/trace.c
>>        2.) sched_init()               in kernel/sched/core.c
>>        3.) init_sched_fair_class()    in kernel/sched/fair.c
>>        4.) workqueue_init_early()     in kernel/workqueue.c
>>
>> In order to fix the bug, the patch changes generic cpu_to_node to
>> function pointer, and export it for kernel modules.
>> Introduce smp_prepare_boot_cpu_start() to wrap the original
>> smp_prepare_boot_cpu(), and set cpu_to_node with early_cpu_to_node.
>> Introduce smp_prepare_cpus_done() to wrap the original 
>> smp_prepare_cpus(),
>> and set the cpu_to_node to formal _cpu_to_node().
>
> Would  you please fix this cleanly without a function pointer?
>
> What I think needs to be done is a patch series.
>
> 1. Instrument cpu_to_node so that some warning is issued if it is used 
> too early. Preloading the array with NUMA_NO_NODE would allow us to do 
> that.
>
> 2. Implement early_cpu_to_node on platforms that currently do not have 
> it.
>
> 3. A series of patches that fix each place where cpu_to_node is used 
> too early.

okay, I will try to do it.


Thanks

Huang Shijie