Hi all,
I was trying to trace some IPIs (remote tlb shootdowns in this case) and noticed that:
1) irq_vectors:x86_platform_ipi_entry and irq_vectors:x86_platform_ipi_exit are not hit at all for my case. The backtrace on the receiving CPU:
0xffffffff81079535 flush_tlb_func_common.constprop.10+0x105/0x220 [kernel]
0xffffffff81079681 flush_tlb_func_remote+0x31/0x40 [kernel]
0xffffffff8111f76c flush_smp_call_function_queue+0x4c/0xf0 [kernel]
0xffffffff81120253 generic_smp_call_function_single_interrupt+0x13/0x30 [kernel]
0xffffffff81a030c6 smp_call_function_single_interrupt+0x36/0xd0 [kernel]
0xffffffff81a02679 call_function_single_interrupt+0xa9/0xb0 [kernel]
I would expect that we would hit those trace point somewhere around call_function_single_interrupt()
2) there is no equivalent of ipi:ipi_raise for x86. For the following call stack:
0xffffffff81055d10 native_send_call_func_single_ipi+0x0/0x20 [kernel]
0xffffffff8111f86f generic_exec_single+0x5f/0xc0 [kernel]
0xffffffff8111f9a2 smp_call_function_single+0xd2/0x100 [kernel]
0xffffffff8111fe3c smp_call_function_many+0x1cc/0x250 [kernel]
0xffffffff8107982c native_flush_tlb_others+0x3c/0xf0 [kernel]
(...)
I would expect to have a irq_vectors:x86_platform_ipi_raise (or similar) tracepoint.
Are there any reasons my expectations are wrong?
I'd love to submit a patch that addresses these issue but I'd rather get some more context (history maybe) before that.
Thanks,
Wojtek
Wojciech Kudla <[email protected]> writes:
> I was trying to trace some IPIs (remote tlb shootdowns in this case) and noticed that:
>
> 1) irq_vectors:x86_platform_ipi_entry and irq_vectors:x86_platform_ipi_exit are not hit at all for my case. The backtrace on the receiving CPU:
>
> 0xffffffff81079535 flush_tlb_func_common.constprop.10+0x105/0x220 [kernel]
> 0xffffffff81079681 flush_tlb_func_remote+0x31/0x40 [kernel]
> 0xffffffff8111f76c flush_smp_call_function_queue+0x4c/0xf0 [kernel]
> 0xffffffff81120253 generic_smp_call_function_single_interrupt+0x13/0x30 [kernel]
> 0xffffffff81a030c6 smp_call_function_single_interrupt+0x36/0xd0 [kernel]
> 0xffffffff81a02679 call_function_single_interrupt+0xa9/0xb0 [kernel]
>
> I would expect that we would hit those trace point somewhere around
> call_function_single_interrupt()
Why would the SMP call function single interrupt go through the
PLATFORM_IPI_VECTOR? It goes as the name says through the
CALL_FUNCTION_SINGLE_VECTOR.
Thanks,
tglx
On 13/05/2020 13:24, Thomas Gleixner wrote:
> Why would the SMP call function single interrupt go through the
> PLATFORM_IPI_VECTOR? It goes as the name says through the
> CALL_FUNCTION_SINGLE_VECTOR.
>
Wrong vector, my bad.
However 2) still stands in my opinion. We don't have "ipi raise" trace point for x86.
RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as well as TLB invalidation vectors are essentially inter-processor-interrupts if I'm not mistaken.
Would a patch adding such trace point be considered here?
Thanks,
Wojtek
Wojciech Kudla <[email protected]> writes:
> On 13/05/2020 13:24, Thomas Gleixner wrote:
>
>> Why would the SMP call function single interrupt go through the
>> PLATFORM_IPI_VECTOR? It goes as the name says through the
>> CALL_FUNCTION_SINGLE_VECTOR.
>>
>
> Wrong vector, my bad.
>
> However 2) still stands in my opinion. We don't have "ipi raise" trace
> point for x86. RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as
> well as TLB invalidation vectors are essentially
> inter-processor-interrupts if I'm not mistaken. Would a patch adding
> such trace point be considered here?
Maybe.
Though that IPI tracing is inconsistent across architectures. I'm not
really interested to have yet another x86 variant which is slightly
different than anything else.
ARM and ARM64 share generic tracepoints for that, though the actual
tracepoint invocation is in the architecture specific code.
If at all we really want to have the common IPIs which are required for
SMP support covered by generic tracepoints and have them in the generic
code and not sprinkled all over arch/*
Thanks,
tglx
On 13/05/2020 17:43, Thomas Gleixner wrote:
> Wojciech Kudla <[email protected]> writes:
>> On 13/05/2020 13:24, Thomas Gleixner wrote:
>>
>>> Why would the SMP call function single interrupt go through the
>>> PLATFORM_IPI_VECTOR? It goes as the name says through the
>>> CALL_FUNCTION_SINGLE_VECTOR.
>>>
>>
>> Wrong vector, my bad.
>>
>> However 2) still stands in my opinion. We don't have "ipi raise" trace
>> point for x86. RESCHEDULE_VECTOR, CALL_FUNCTION_SINGLE_VECTOR, as
>> well as TLB invalidation vectors are essentially
>> inter-processor-interrupts if I'm not mistaken. Would a patch adding
>> such trace point be considered here?
>
> Maybe.
>
> Though that IPI tracing is inconsistent across architectures. I'm not
> really interested to have yet another x86 variant which is slightly
> different than anything else.
>
> ARM and ARM64 share generic tracepoints for that, though the actual
> tracepoint invocation is in the architecture specific code.
>
> If at all we really want to have the common IPIs which are required for
> SMP support covered by generic tracepoints and have them in the generic
> code and not sprinkled all over arch/*
How about we add ipi:ipi_raise trace points before:
- arch_send_call_function_single_ipi(), and
- arch_send_call_function_ipi_mask()
Would that be a good starting point to introduce more generic IPI tracing?
Thanks,
Wojtek