Subject: Re: [RFC PATCH] perf/x86/intel/rapl: avoid access unallocate memory
To: Thomas Gleixner <tglx@linutronix.de>
References: <20161102122557.qs4rl6mb7n7l7j7p@linutronix.de>
 <24e69019-60d0-29e7-e31f-c6f00f9ed98a@brocade.com>
 <alpine.DEB.2.20.1611071717000.3709@nanos>
CC: Sebastian Andrzej Siewior <bigeasy@linutronix.de>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>, "M. Vefa Bicakci" <m.v.b@runbox.com>
From: "Charles (Chas) Williams" <ciwillia@brocade.com>
Message-ID: <58e229e2-91f4-a97f-1b9f-089f48ef994a@brocade.com>
Date: Mon, 7 Nov 2016 11:59:52 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Icedove/45.3.0
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.20.1611071717000.3709@nanos>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2914
Lines: 79

On 11/07/2016 11:19 AM, Thomas Gleixner wrote:
> On Wed, 2 Nov 2016, Charles (Chas) Williams wrote:
>
>> On 11/02/2016 08:25 AM, Sebastian Andrzej Siewior wrote:
>>> I am not sure if this a race with the new hotplug code or something that was
>>> always there. Both (M. Vefa Bicakc and Charles) say that the box boots
>>> sometimes fine (without the patch).  smp_store_boot_cpu_info() should have
>>> run
>>> before the notofoert and thus should have set the info properly. However I
>>> got
>>> the following bootlog from Charles with this patch:
>>
>> I don't this this is a race.  Here is some debugging from the two CPU VM
>> (2 sockets, 1 core per socket).  In identify_cpu() we have:
>>
>>         /* The boot/hotplug time assigment got cleared, restore it */
>>         c->logical_proc_id = topology_phys_to_logical_pkg(c->phys_proc_id);
>>
>> The values just after this:
>>
>> 	[    0.228306] identify_cpu: c ffff88023fd0a040  logical_proc_id 65535
>> c->phys_proc_id 2
>>
>> So what's interesting here, is the phys_proc_id of 2 for CPU1:
>>
>>         int topology_phys_to_logical_pkg(unsigned int phys_pkg)
>>         {
>>                 if (phys_pkg >= max_physical_pkg_id)
>>                         return -1;
>>                 return physical_to_logical_pkg[phys_pkg];
>>         }
>>
>> And we happen to know the max_physical_pkg_id is 2 in this case.
>> So apparently, topology_phys_to_logical_pkg() returns -1 and it gets
>> assigned to the logical_proc_id.
>>
>> I don't know why the CPU's phys_proc_id is 2.
>
> max_physical_pkg_id gets initialized via:
>
>     cpus = boot_cpu_data.x86_max_cores;
>     max_physical_pkg_id = DIV_ROUND_UP(MAX_LOCAL_APIC, ncpus);
>
> What's the value of boot_cpu_data.x86_max_cores and MAX_LOCAL_APIC?

I have discovered that that is not the problem.  smp_init_package_map()
is calculating the physical core id using the following:

         for_each_present_cpu(cpu) {
                 unsigned int apicid = apic->cpu_present_to_apicid(cpu);

		...
		if (!topology_update_package_map(apicid, cpu))
                         continue;

	...
	int topology_update_package_map(unsigned int apicid, unsigned int cpu)
	{
		unsigned int new, pkg = apicid >> boot_cpu_data.x86_coreid_bits;

But later when the secondary CPU's are identified they use a different
calculation using the local APIC ID from the CPU's registers:

	static void generic_identify(struct cpuinfo_x86 *c)
	...
         if (c->cpuid_level >= 0x00000001) {
                 c->initial_apicid = (cpuid_ebx(1) >> 24) & 0xFF;
	...
		c->phys_proc_id = c->initial_apicid;

So at the end of identify_cpu() when the boot/hotplug assignment is
put back:

         c->logical_proc_id = topology_phys_to_logical_pkg(c->phys_proc_id);

topology_phys_to_logical_pkg() is returning an invalid logical processor
since one isn't configured.

It's not clear to me what the right thing to do is or which is right.