Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755219AbbGZWI3 (ORCPT ); Sun, 26 Jul 2015 18:08:29 -0400 Received: from mail-lb0-f169.google.com ([209.85.217.169]:35854 "EHLO mail-lb0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755156AbbGZWI1 (ORCPT ); Sun, 26 Jul 2015 18:08:27 -0400 MIME-Version: 1.0 In-Reply-To: <55B53636.80304@citrix.com> References: <55B53636.80304@citrix.com> From: Andy Lutomirski Date: Sun, 26 Jul 2015 15:08:06 -0700 Message-ID: Subject: Re: Getting rid of invalid SYSCALL RSP under Xen? To: Andrew Cooper Cc: X86 ML , Boris Ostrovsky , "linux-kernel@vger.kernel.org" , Borislav Petkov , Steven Rostedt , "xen-devel@lists.xen.org" Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3852 Lines: 107 On Sun, Jul 26, 2015 at 12:34 PM, Andrew Cooper wrote: > On 23/07/2015 17:49, Andy Lutomirski wrote: >> Hi- > > Hi. Apologies for the delay. I have been out of the office for a few days. > >> >> In entry_64.S, we have: >> >> ENTRY(entry_SYSCALL_64) >> /* >> * Interrupts are off on entry. >> * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON, >> * it is too small to ever cause noticeable irq latency. >> */ >> SWAPGS_UNSAFE_STACK >> /* >> * A hypervisor implementation might want to use a label >> * after the swapgs, so that it can do the swapgs >> * for the guest and jump here on syscall. >> */ >> GLOBAL(entry_SYSCALL_64_after_swapgs) >> >> movq %rsp, PER_CPU_VAR(rsp_scratch) >> movq PER_CPU_VAR(cpu_current_top_of_stack), %rsp >> >> It would be really, really nice if Xen entered the SYSCALL path >> *after* the mov to %rsp. >> >> Similarly, we have: >> >> movq RSP(%rsp), %rsp >> /* big comment */ >> USERGS_SYSRET64 >> >> It would be really nice if we could just mov to %rsp, swapgs, and >> sysret, without worrying that the sysret is actually a jump on Xen. >> >> I suspect that making Xen stop using these code paths would actually >> be a simplification. On SYSCALL entry, Xen lands in >> xen_syscall_target (AFAICT) and clearly has rsp pointing somewhere >> valid. Xen obligingly shoves the user RSP into the hardware RSP >> register and jumps into the entry code. >> >> Is that stuff running on the normal kernel stack? > > Yes. The Xen ABI takes what is essentially tss->esp0 and uses that stack > for all "switch to kernel" actions, including syscall and sysenter. > >> If so, can we just >> enter later on: >> >> pushq %r11 /* pt_regs->flags */ >> pushq $__USER_CS /* pt_regs->cs */ >> pushq %rcx /* pt_regs->ip */ >> >> <-- Xen enters here >> >> pushq %rax /* pt_regs->orig_ax */ >> pushq %rdi /* pt_regs->di */ >> pushq %rsi /* pt_regs->si */ >> pushq %rdx /* pt_regs->dx */ > > This looks plausible, and indeed preferable to the current doublestep > with undo_xen_syscall. > > One slight complication is that xen_enable_syscall() will want to > special case register_callback() to not set CALLBACKF_mask_events, as > the entry point is now after re-enabling interrupts. I wouldn't do that. Let's just move the ENABLE_INTERRUPTS a few instructions later even on native -- I want to do that anyway. > >> >> For SYSRET, I think the way to go is to force Xen to always use the >> syscall slow path. Instead, Xen could hook into >> syscall_return_via_sysret or even right before the opportunistic >> sysret stuff. Then we could remove the USERGS_SYSRET hooks entirely. >> >> Would this work? > > None of the opportunistic sysret stuff makes sense under Xen. The path > will inevitably end up in xen_iret making a hypercall. Short circuiting > all of this seems like a good idea, especially if it allows for the > removal of the UERGS_SYSRET. Doesn't Xen decide what to do based on VGCF_IN_SYSCALL? Maybe Xen should have its own opportunistic VGCF_IN_SYSCALL logic. Hmm, maybe some of this would be easier to think about if, rather than having a paravirt op, we could have: ALTERNATIVE "", "jmp xen_pop_things_and_iret", X86_FEATURE_XEN Or just IF_XEN("jmp ..."); As a practical matter, x86_64 has native and Xen -- I don't think there's any other paravirt platform that needs the asm hooks. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/