Subject: Re: [RFC PATCH] perf/x86/intel/rapl: avoid access unallocate memory
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
References: <20161102122557.qs4rl6mb7n7l7j7p@linutronix.de>
 <24e69019-60d0-29e7-e31f-c6f00f9ed98a@brocade.com>
 <20161103174753.o5ynquul2rjuiq77@linutronix.de>
CC: <x86@kernel.org>, <linux-kernel@vger.kernel.org>,
        "M. Vefa Bicakci" <m.v.b@runbox.com>
From: "Charles (Chas) Williams" <ciwillia@brocade.com>
Message-ID: <e89d4987-5aab-fd6d-2b11-7a4d0eb7f57f@brocade.com>
Date: Fri, 4 Nov 2016 08:20:37 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Icedove/45.3.0
MIME-Version: 1.0
In-Reply-To: <20161103174753.o5ynquul2rjuiq77@linutronix.de>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3086
Lines: 77


On 11/03/2016 01:47 PM, Sebastian Andrzej Siewior wrote:
> On 2016-11-02 18:47:49 [-0400], Charles (Chas) Williams wrote:
>> I don't this this is a race.  Here is some debugging from the two CPU VM
>> (2 sockets, 1 core per socket).  In identify_cpu() we have:
>>
>>         /* The boot/hotplug time assigment got cleared, restore it */
>>         c->logical_proc_id = topology_phys_to_logical_pkg(c->phys_proc_id);
>>
>> The values just after this:
>>
>> 	[    0.228306] identify_cpu: c ffff88023fd0a040  logical_proc_id 65535  c->phys_proc_id 2
>>
>> So what's interesting here, is the phys_proc_id of 2 for CPU1:
>>
>>         int topology_phys_to_logical_pkg(unsigned int phys_pkg)
>>         {
>>                 if (phys_pkg >= max_physical_pkg_id)
>>                         return -1;
>>                 return physical_to_logical_pkg[phys_pkg];
>>         }
>>
>> And we happen to know the max_physical_pkg_id is 2 in this case.
>> So apparently, topology_phys_to_logical_pkg() returns -1 and it gets
>> assigned to the logical_proc_id.
>>
>> I don't know why the CPU's phys_proc_id is 2.
>
> This is the physical ID. You have two logical IDs (on your two sockets
> machine). What is max_physical_pkg_id? In order to get that -1 you would
> have to max_physical_pkg_id of 1 but code does
> 	max_physical_pkg_id = DIV_ROUND_UP(MAX_LOCAL_APIC, ncpus);
>
> and I would be a little surprised if this is 1.
>
> Sebastian

The initial CPU boots and is identified:

[    0.009018] identify_boot_cpu
[    0.009174] generic_identify: phys_proc_id is now 0
...
[    0.009427] identify_cpu: before c ffffffff81ae2680  logical_proc_id 0  c->phys_proc_id 0
[    0.009506] identify_cpu: after c ffffffff81ae2680  logical_proc_id 65535  c->phys_proc_id 0

So, this is fine because the APIC hasn't been scanned yet.  APIC
now gets scanned:

[    0.015789] smpboot: APIC(0) Converting physical 0 to logical package 0, cpu 0 (ffff88023fc0a040)
[    0.015794] smpboot: APIC(1) Converting physical 1 to logical package 1, cpu 1 (ffff88023fd0a040)
[    0.015797] smpboot: Max logical packages: 2

So, at this point, I think everything is correct.  But now the secondary
CPU's "boot":

[    0.236569] identify_secondary_cpu
[    0.236620] generic_identify: phys_proc_id is now 2

[    0.236745] identify_cpu: before c ffff88023fd0a040  logical_proc_id 65535  c->phys_proc_id 2
[    0.236747] identify_cpu: after c ffff88023fd0a040  logical_proc_id 65535  c->phys_proc_id 2

So, APIC discovered I have a cpu 0 and 1 but generic_identify() is called
my second CPU, 2.  This is >= max_physical_pkg_id, so it is going to get
set to -1.

The comment at the end of identfy_cpu() says:

         /* The boot/hotplug time assigment got cleared, restore it */

So, logical_proc_id being wrong here before restoration doesn't bother
me since I assume something in booting the secondary CPU's clears any
existing cpu data.

I know detect_extended_topology() is likely being called for both CPU's
and getting the right values (checking this now).  I don't know why
generic_identify() is resetting this value.