Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Vince Weaver <vincent.weaver@maine.edu>
Date:   Fri, 18 Jan 2019 12:24:20 -0500 (EST)
To:     Peter Zijlstra <peterz@infradead.org>
cc:     linux-kernel@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>
Subject: Re: perf: rdpmc bug when viewing all procs on remote cpu
In-Reply-To: <20190118155814.GC14054@worktop.programming.kicks-ass.net>
Message-ID: <alpine.DEB.2.21.1901181217180.5762@macbook-air>
References: <alpine.DEB.2.21.1901101229010.3358@macbook-air> <alpine.DEB.2.21.1901101246340.3358@macbook-air> <alpine.DEB.2.21.1901101500030.3717@macbook-air> <alpine.DEB.2.21.1901111649580.6487@macbook-air> <20190118120149.GC27931@hirez.programming.kicks-ass.net>
 <alpine.DEB.2.21.1901180905090.5762@macbook-air> <20190118155814.GC14054@worktop.programming.kicks-ass.net>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, 18 Jan 2019, Peter Zijlstra wrote:
> 
> You can actually use rdpmc when you attach to a CPU, but you have to
> ensure that the userspace component is guaranteed to run on that very
> CPU (sched_setaffinity(2) comes to mind).

unfortunately the HPC people using PAPI would probably be annoyed if we 
started binding their threads to cores out from under them.

> The best we could possibly do is put the (target, not current) cpu
> number in the mmap page; but userspace should already know this, for it
> created the event and therefore knows this already.

one other thing the kernel would do is just disable rdpmc (setting index 
to 0) in the case where the original perf_event_open() cpu paramater!=0

though that would stop the case where we were on the same CPU from 
working.

The issue is currently if you're not careful the rdpmc() interface will 
sometimes return plausible (but wrong) results for a cross-CPU rdpmc() 
call, even if you are properly falling back to read() on ->index being 0.
It's a bit surprising and it looks like it will take a decent amount of 
userspace code to work around the issue, which cuts into the low-overhead 
nature of rdpmc.

If the answer is simply this is the way the kernel is going to do it, 
that's fine, I just have to add workarounds to PAPI and then get the 
perf_even_open() manpage updated to make sure people are aware of the 
issue.

Vince