Commit 72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol
write' has introduced an arch-specific method to ensure all caches are
flushed appropriately after an instruction is written to an XOL page.
However, when the XOL area is created and the out-of-line breakpoint
instruction is copied, caches are not flushed at all and stale data may
be found in icache.
Replace a simple copy_to_page() with arch_uprobe_copy_ixol() to allow
the arch to ensure all caches are updated accordingly.
This change fixes uprobes on MIPS InterAptiv (tested on Creator Ci40).
Signed-off-by: Marcin Nowakowski <[email protected]>
CC: Peter Zijlstra <[email protected]>
CC: Ingo Molnar <[email protected]>
CC: Arnaldo Carvalho de Melo <[email protected]>
CC: Alexander Shishkin <[email protected]>
CC: Victor Kamensky <[email protected]>
---
kernel/events/uprobes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index f9ec9ad..b5916b4 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1194,7 +1194,7 @@ static struct xol_area *__create_xol_area(unsigned long vaddr)
/* Reserve the 1st slot for get_trampoline_vaddr() */
set_bit(0, area->bitmap);
atomic_set(&area->slot_count, 1);
- copy_to_page(area->pages[0], 0, &insn, UPROBE_SWBP_INSN_SIZE);
+ arch_uprobe_copy_ixol(area->pages[0], 0, &insn, UPROBE_SWBP_INSN_SIZE);
if (!xol_add_vma(mm, area))
return area;
--
2.7.4
Commit-ID: 297e765e390a2ac996000b5f7228cbd84d995174
Gitweb: http://git.kernel.org/tip/297e765e390a2ac996000b5f7228cbd84d995174
Author: Marcin Nowakowski <[email protected]>
AuthorDate: Tue, 13 Dec 2016 11:40:57 +0100
Committer: Ingo Molnar <[email protected]>
CommitDate: Sun, 18 Dec 2016 09:42:11 +0100
uprobes: Fix uprobes on MIPS, allow for a cache flush after ixol breakpoint creation
Commit:
72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol write'
... has introduced an arch-specific method to ensure all caches are
flushed appropriately after an instruction is written to an XOL page.
However, when the XOL area is created and the out-of-line breakpoint
instruction is copied, caches are not flushed at all and stale data may
be found in icache.
Replace a simple copy_to_page() with arch_uprobe_copy_ixol() to allow
the arch to ensure all caches are updated accordingly.
This change fixes uprobes on MIPS InterAptiv (tested on Creator Ci40).
Signed-off-by: Marcin Nowakowski <[email protected]>
Cc: Alexander Shishkin <[email protected]>
Cc: Arnaldo Carvalho de Melo <[email protected]>
Cc: Arnaldo Carvalho de Melo <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Victor Kamensky <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
kernel/events/uprobes.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index f9ec9ad..b5916b4 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -1194,7 +1194,7 @@ static struct xol_area *__create_xol_area(unsigned long vaddr)
/* Reserve the 1st slot for get_trampoline_vaddr() */
set_bit(0, area->bitmap);
atomic_set(&area->slot_count, 1);
- copy_to_page(area->pages[0], 0, &insn, UPROBE_SWBP_INSN_SIZE);
+ arch_uprobe_copy_ixol(area->pages[0], 0, &insn, UPROBE_SWBP_INSN_SIZE);
if (!xol_add_vma(mm, area))
return area;
On 12/19, tip-bot for Marcin Nowakowski wrote:
>
> uprobes: Fix uprobes on MIPS, allow for a cache flush after ixol breakpoint creation
>
> Commit:
>
> 72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol write'
>
> ... has introduced an arch-specific method to ensure all caches are
> flushed appropriately after an instruction is written to an XOL page.
when this page is already mmaped,
> However, when the XOL area is created and the out-of-line breakpoint
> instruction is copied, caches are not flushed at all and stale data may
> be found in icache.
but in this case the page is not mmaped yet, the probed application will
take a page fault if it tries to execute this insn,
> Replace a simple copy_to_page() with arch_uprobe_copy_ixol() to allow
> the arch to ensure all caches are updated accordingly.
>
> This change fixes uprobes on MIPS InterAptiv (tested on Creator Ci40).
OK, I know nothing about MIPS, but could you help me understand this change?
See above. If we really need flush_icache_range() here then perhaps we should
modify install_special_mapping() and/or __do_fault/special_mapping_fault paths
instead?
Oleg.
Hi Oleg,
On 20.12.2016 14:08, Oleg Nesterov wrote:
> On 12/19, tip-bot for Marcin Nowakowski wrote:
>>
>> uprobes: Fix uprobes on MIPS, allow for a cache flush after ixol breakpoint creation
>>
>> Commit:
>>
>> 72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol write'
>>
>> ... has introduced an arch-specific method to ensure all caches are
>> flushed appropriately after an instruction is written to an XOL page.
>
> when this page is already mmaped,
>
>> However, when the XOL area is created and the out-of-line breakpoint
>> instruction is copied, caches are not flushed at all and stale data may
>> be found in icache.
>
> but in this case the page is not mmaped yet, the probed application will
> take a page fault if it tries to execute this insn,
In case of MIPS (and AFAICT ARM as well, and these are the only
architectures that implement arch_uprobe_copy_ixol), the cache flushing
is done through the kernel addresses of that page, so the fact that it
is not mapped yet is not an issue.
Do I understand correctly that your statement implies that after the
page fault and mmapping the xol page, the page is guaranteed to be
updated in the cache? As definitely that is not something that is
happening at the moment.
>> Replace a simple copy_to_page() with arch_uprobe_copy_ixol() to allow
>> the arch to ensure all caches are updated accordingly.
>>
>> This change fixes uprobes on MIPS InterAptiv (tested on Creator Ci40).
>
> OK, I know nothing about MIPS, but could you help me understand this change?
>
> See above. If we really need flush_icache_range() here then perhaps we should
> modify install_special_mapping() and/or __do_fault/special_mapping_fault paths
> instead?
Are you suggesting that those should be updated to force a cache update?
Marcin
On 12/20, Marcin Nowakowski wrote:
>
> Hi Oleg,
>
> On 20.12.2016 14:08, Oleg Nesterov wrote:
>> On 12/19, tip-bot for Marcin Nowakowski wrote:
>>>
>>> uprobes: Fix uprobes on MIPS, allow for a cache flush after ixol breakpoint creation
>>>
>>> Commit:
>>>
>>> 72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol write'
>>>
>>> ... has introduced an arch-specific method to ensure all caches are
>>> flushed appropriately after an instruction is written to an XOL page.
>>
>> when this page is already mmaped,
>>
>>> However, when the XOL area is created and the out-of-line breakpoint
>>> instruction is copied, caches are not flushed at all and stale data may
>>> be found in icache.
>>
>> but in this case the page is not mmaped yet, the probed application will
>> take a page fault if it tries to execute this insn,
>
> In case of MIPS (and AFAICT ARM as well, and these are the only
> architectures that implement arch_uprobe_copy_ixol), the cache flushing
> is done through the kernel addresses of that page, so the fact that it
> is not mapped yet is not an issue.
OK, thanks,
> Do I understand correctly that your statement implies that after the
> page fault and mmapping the xol page, the page is guaranteed to be
> updated in the cache? As definitely that is not something that is
> happening at the moment.
Well, I do not know. Let me repeat I don't understand this flush_.*cache
magic.
But. do_read_fault() does
__do_fault(..., &fault_page, ...);
alloc_set_pte(fault_page);
and alloc_set_pte() does flush_icache_page(vma, page)... Hmm, which is nop
on MIPS.
>> OK, I know nothing about MIPS, but could you help me understand this change?
>>
>> See above. If we really need flush_icache_range() here then perhaps we should
>> modify install_special_mapping() and/or __do_fault/special_mapping_fault paths
>> instead?
>
> Are you suggesting that those should be updated to force a cache update?
Again, I do not know. But perhaps it makes more sense to actually implement
flush_icache_page() ? Otherwise another user of install_special_mapping()
can hit the same problem?
Oleg.
On Tue, Dec 20, 2016 at 06:50:05PM +0100, Oleg Nesterov wrote:
> >>> Commit:
> >>>
> >>> 72e6ae285a1d ('ARM: 8043/1: uprobes need icache flush after xol write'
> >>>
> >>> ... has introduced an arch-specific method to ensure all caches are
> >>> flushed appropriately after an instruction is written to an XOL page.
> >>
> >> when this page is already mmaped,
> >>
> >>> However, when the XOL area is created and the out-of-line breakpoint
> >>> instruction is copied, caches are not flushed at all and stale data may
> >>> be found in icache.
> >>
> >> but in this case the page is not mmaped yet, the probed application will
> >> take a page fault if it tries to execute this insn,
> >
> > In case of MIPS (and AFAICT ARM as well, and these are the only
> > architectures that implement arch_uprobe_copy_ixol), the cache flushing
> > is done through the kernel addresses of that page, so the fact that it
> > is not mapped yet is not an issue.
>
> OK, thanks,
>
> > Do I understand correctly that your statement implies that after the
> > page fault and mmapping the xol page, the page is guaranteed to be
> > updated in the cache? As definitely that is not something that is
> > happening at the moment.
>
> Well, I do not know. Let me repeat I don't understand this flush_.*cache
> magic.
>
> But. do_read_fault() does
>
> __do_fault(..., &fault_page, ...);
>
> alloc_set_pte(fault_page);
>
> and alloc_set_pte() does flush_icache_page(vma, page)... Hmm, which is nop
> on MIPS.
>
> >> OK, I know nothing about MIPS, but could you help me understand this change?
> >>
> >> See above. If we really need flush_icache_range() here then perhaps we should
> >> modify install_special_mapping() and/or __do_fault/special_mapping_fault paths
> >> instead?
> >
> > Are you suggesting that those should be updated to force a cache update?
>
> Again, I do not know. But perhaps it makes more sense to actually implement
> flush_icache_page() ? Otherwise another user of install_special_mapping()
> can hit the same problem?
Documentation/cachetlb.txt says about flush_icache_page:
void flush_icache_page(struct vm_area_struct *vma, struct page *page)
All the functionality of flush_icache_page can be implemented in
flush_dcache_page and update_mmu_cache. In the future, the hope
is to remove this interface completely.
And that's exactly what MIPS already does, thus flush_icache_page() is a
no-op. The new interfaces flush_dcache_page and update_mmu_cache
generally are much more efficient.
Ralf