Hello,
I have released another version of the perfmon new base package.
This release is relative to 2.6.16-rc5
Due to the addition of a few system calls since 2.6.16-rc1, the
system call base number for all perfmon calls has changed yet again
on all architectures. As such you need to use the new version of
libpfm: libpfm-3.2-060308.
This new kernel patch includes several important changes:
- migrated all of /proc perfmon interface to /sys. The information
is split between general and per-cpu. As such it is located in
/sys/kernel/perfmon and /sys/devices/system/cpu*/perfmon/
- all data structures exchanged with the kernel now use fixed size
data types. As such they can be used without any changes by a 32-bit
application running on top on a 64-bit OS. The following changes took
place:
* all bitvectors use u64 and have fixed number of bits
* size_t is now u64
* struct timespec (set timeout) replaced by u32 (unit in micro-seconds)
* default format sampling buffer IP (instruction pointer) is u64
- added 32-bit ABI support for x86-64 arch. As a consequence, it is now
possible to run a 32-bit P4 tool on a 64-bit EM64T processor, assuming the
tool know the specifics of EM64T PMU. That goes also for PEBS support.
- Simplified the PMU description data structures. The structure pfm_pmu_config
is now split between a perfmon internal and PMU description version providing
further isolation between the core and module-provided support. There is
a simplification also in the struct pfm_reg_desc tables describing the mappings.
Tables are much smaller because they only contain the actual register mappings
for a particular PMU model as opposed to a fixed size table of 320 elements. The
perfmon core now builds its internal representation based on this information as
opposed to using it directly.
- removed extraneous locking and interrupt masking for the PMU interrupt handler
- split kernel perfmon headers files based on functionality
- added experimental support for smp_call_single() on i386. If the implementation
is satisfactory, I'll push it as a separate patch to lkml.
There was not update to the MIPS specific patch. As such it is currently broken and
cannot be used with the new libpfm.
The new version of the library, libpfm, includes the following changes:
- preliminary support for Dual-Core Itanium 2 (Montecito) based on
publicly available documentation
- update to match 2.6.16-rc5 kernel patch
- renamed all *gen_x86_64* files to *amd_x86_64* to avoid confusion
with EM64T.
- incorporated AMD provided event table for Opteron (Ray Bryant)
The two packages can be downloaded from the SourceForge website at:
http://www.sf.net/projects/perfmon2
Enjoy,
--
-Stephane
Stephane Eranian wrote:
> Hello,
>
> I have released another version of the perfmon new base package.
> This release is relative to 2.6.16-rc5
Hello Stephane,
Is there any thoughts on how perfmon2 is going to work with xen enabled
kernels or processors that support virtualization?
-Will
Will,
On Wed, Mar 08, 2006 at 03:40:16PM -0500, William Cohen wrote:
> >
> >I have released another version of the perfmon new base package.
> >This release is relative to 2.6.16-rc5
>
> Hello Stephane,
>
> Is there any thoughts on how perfmon2 is going to work with xen enabled
> kernels or processors that support virtualization?
>
AFAIK, there is currently no Xen support for PMU on any platforms. By "support"
I mean suport for guests using the PMU. As such, I do not think this works.
I am planning on looking at this next because this is becoming a pressing matter
and not just on IA-64.
My first goal is to ensure that a guest using perfmon2 works once it is virtualized.
That implies that the Xen VMM does save/restore PMU state on guest switch. That's a bare
minimum.
As for HW virutlization support, I think it helps a little bit but there still needs to
be some additional code in the VMM to make this work correctly. This is also something
I want to look at.
--
-Stephane
Hi Stephane,
I have been looking through the perfmon2 code to see how it is going to
work with OProfile. It looks like the ia64 oprofile support has not been
modified to work with the changes in perfmon2. Has the ia64 kernel been
built with perfmon2 and oprofile support? I don't have easy access to an
ia64, so I haven't been able to verify that the attached patch works.
However, I expect that the changes in the patch will be required for
OProfile to function with perfmon2.
-Will
Will,
On Mon, Mar 13, 2006 at 01:39:01PM -0500, William Cohen wrote:
> Hi Stephane,
>
> I have been looking through the perfmon2 code to see how it is going to
> work with OProfile. It looks like the ia64 oprofile support has not been
> modified to work with the changes in perfmon2. Has the ia64 kernel been
> built with perfmon2 and oprofile support? I don't have easy access to an
> ia64, so I haven't been able to verify that the attached patch works.
> However, I expect that the changes in the patch will be required for
> OProfile to function with perfmon2.
>
Good timing. I just fixed this today. Now it compiles fine on
IA64. I also started looking into using the same technique on
i386. It is very easy. It looks like opcontrol or ophelp
would need to be updated. I think the trick is to make
sure that ophelp knows the PMU mapping used by perfmon2,
i.e., knows that PERFEVTSEL0 is PMC0 for instance.
--
-Stephane
Stephane Eranian wrote:
> Will,
>
> On Mon, Mar 13, 2006 at 01:39:01PM -0500, William Cohen wrote:
>
>>Hi Stephane,
>>
>>I have been looking through the perfmon2 code to see how it is going to
>>work with OProfile. It looks like the ia64 oprofile support has not been
>>modified to work with the changes in perfmon2. Has the ia64 kernel been
>>built with perfmon2 and oprofile support? I don't have easy access to an
>>ia64, so I haven't been able to verify that the attached patch works.
>>However, I expect that the changes in the patch will be required for
>>OProfile to function with perfmon2.
>>
>
> Good timing. I just fixed this today. Now it compiles fine on
> IA64. I also started looking into using the same technique on
> i386. It is very easy. It looks like opcontrol or ophelp
> would need to be updated. I think the trick is to make
> sure that ophelp knows the PMU mapping used by perfmon2,
> i.e., knows that PERFEVTSEL0 is PMC0 for instance.
>
Yes, I have a similar patch for i386 in the kernel. I don't yet have
modifications for opcontrol or ophelp.
One question would be identifying the processor when using the perfmon2
support for i386/* processors? There is prior support in the oprofile
driver for i386 processors. Identify the processor differently depending
on whether perfmon2 is being used to distinguish between the different
interfaces? The way that OProfile has the events each name processor
requires a different directory in /usr/share/oprofile. Would prefer to
keep down the proliferation of new directories.
-Will
Will,
On Mon, Mar 13, 2006 at 02:15:53PM -0500, William Cohen wrote:
>
> Yes, I have a similar patch for i386 in the kernel. I don't yet have
> modifications for opcontrol or ophelp.
Your patch is very close to mine. I'll merge tthe two and will include
it in my next version.
My understand of opcontrol is that is passes the information from ophelp
to the kernel via /dev/oprofile. I don't know how it passes it to the oprofiled
daemon.
The daemon should not be difficult to change. I hacked something quickly
and got it up on pentium M. The only remaining problem is ophelp, I think.
> One question would be identifying the processor when using the perfmon2
> support for i386/* processors? There is prior support in the oprofile
> driver for i386 processors. Identify the processor differently depending
> on whether perfmon2 is being used to distinguish between the different
> interfaces? The way that OProfile has the events each name processor
> requires a different directory in /usr/share/oprofile. Would prefer to
> keep down the proliferation of new directories.
I think it would be easier to check in /sys/kernel/perfmon to detect
that it is running on perfmon2. Then opcontrol can pass the inormation
around to ophelp, oprofiled is necessary. Ophelp then just needs
to know the perfmon2 register mapping for each CPU. I don't know
how this information is represented today.
--
-Stephane
Stephane Eranian wrote:
> Will,
>
> On Mon, Mar 13, 2006 at 02:15:53PM -0500, William Cohen wrote:
>
>>Yes, I have a similar patch for i386 in the kernel. I don't yet have
>>modifications for opcontrol or ophelp.
>
>
> Your patch is very close to mine. I'll merge tthe two and will include
> it in my next version.
>
> My understand of opcontrol is that is passes the information from ophelp
> to the kernel via /dev/oprofile. I don't know how it passes it to the oprofiled
> daemon.
>
> The daemon should not be difficult to change. I hacked something quickly
> and got it up on pentium M. The only remaining problem is ophelp, I think.
Is new identifier cpu_type being created or is i386/pentium_m being used
also for perfmon2? It could be inferred that perfmon2 is being used if
the procesor is identified, but there are no number directories for the
counters (/dev/oprofile/[0-9]+).
I looked over the code this afternoon to refresh my memory how it actual
work (as opposed to how I thought it works).
ophelp uses oprofile/libop/op_cpu_type.c:op_get_cpu_type to read
/dev/oprofile/cpu_type. Once ophelp obtains the cpu type the appropriate
directory from /usr/share/oprofile for the events. This information is
passed to oprofiled; you can see it with "opcontrol --start --verbose".
oprofile/daemon/opd_perfmon.c and opd_perfmon.h are the place that the
performance monitoring hardware is set for perfmon.
opd_perfmon.c:write_pmu() is currently hard coded for ia64. As you have
seen, it doesn't use the libpfm to set up the performance monitoring
hardware.
Looking over the code in the linux/arch/i386/oprofile the code for
typing the processor is tied to the initialization of the performance
monitoring hardware. Probably want to factor out cpu idenification code
from oprofile native handling of the performance monitoring code. So the
pseudo code is something like in
linux/arch/i386/oprofile/init.c:oprofile_arch_init():
char *processor_id = identify_processer();
ret = op_perfmon_support(ops, processor_id);
if (ret < 0)
ret = op_nmi_init(ops, processor_id);
if (ret < 0)
ret = op_nmi_timer(ops, processor_id);
return ret;
-Will
Will,
On Mon, Mar 13, 2006 at 03:58:06PM -0500, William Cohen wrote:
>
> Is new identifier cpu_type being created or is i386/pentium_m being used
> also for perfmon2? It could be inferred that perfmon2 is being used if
> the procesor is identified, but there are no number directories for the
> counters (/dev/oprofile/[0-9]+).
>
Existing OProfile codes treats pentium M as i386/piii as it should more or less.
> I looked over the code this afternoon to refresh my memory how it actual
> work (as opposed to how I thought it works).
>
> ophelp uses oprofile/libop/op_cpu_type.c:op_get_cpu_type to read
> /dev/oprofile/cpu_type. Once ophelp obtains the cpu type the appropriate
> directory from /usr/share/oprofile for the events. This information is
> passed to oprofiled; you can see it with "opcontrol --start --verbose".
>
In the quick hack I did over the week-end, I followed what John had done
for IA-64. In other words, I also have a get_cpu() routine for i386 and
I basically did a cut&paste of what was in nmi_int.c. As such,
/dev/oprofile/cpu_type is preserved to avoid confusing ophelp.
> oprofile/daemon/opd_perfmon.c and opd_perfmon.h are the place that the
> performance monitoring hardware is set for perfmon.
> opd_perfmon.c:write_pmu() is currently hard coded for ia64. As you have
> seen, it doesn't use the libpfm to set up the performance monitoring
> hardware.
Libpfm is really a helper library and is not required to use the kernel
interface. However, there is a bit of an anomaly in libpfm at the moment.
The library also includes the kernel header files and perfmon2 syscall stubs.
Both would eventually be found in libc. As such, for the time being, some
applications may choose to link against libpfm, just for the stubs. The current
oprofiled does not do this and has every perfmon.h definitions it needs hardcoded
(syscalls stubs are also recreated).
>
> Looking over the code in the linux/arch/i386/oprofile the code for
> typing the processor is tied to the initialization of the performance
> monitoring hardware. Probably want to factor out cpu idenification code
> from oprofile native handling of the performance monitoring code. So the
> pseudo code is something like in
> linux/arch/i386/oprofile/init.c:oprofile_arch_init():
>
> char *processor_id = identify_processer();
> ret = op_perfmon_support(ops, processor_id);
> if (ret < 0)
> ret = op_nmi_init(ops, processor_id);
> if (ret < 0)
> ret = op_nmi_timer(ops, processor_id);
> return ret;
>
Yes that could be a possibility. That implies that there may be
situations where perfmon does not support a processor that is
somehow supported by OProfile. I suspect this could be the case
for very old CPUs.
--
-Stephane