Message-ID: <04978d6d-8e1a-404d-b30d-402a7569c1f0@redhat.com>
Date:   Tue, 16 Nov 2021 20:49:08 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.0
Subject: Re: Thoughts of AMX KVM support based on latest kernel
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        Sean Christopherson <seanjc@google.com>
Cc:     "Liu, Jing2" <jing2.liu@intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Nakajima, Jun" <jun.nakajima@intel.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Jing Liu <jing2.liu@linux.intel.com>,
        "Cooper, Andrew" <andrew.cooper3@citrix.com>,
        "Bae, Chang Seok" <chang.seok.bae@intel.com>
References: <BYAPR11MB325685AB8E3DFD245846F854A9939@BYAPR11MB3256.namprd11.prod.outlook.com>
 <87k0h85m65.ffs@tglx> <YZPWsICdDTZ02UDu@google.com> <87ee7g53rp.ffs@tglx>
From:   Paolo Bonzini <pbonzini@redhat.com>
In-Reply-To: <87ee7g53rp.ffs@tglx>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 11/16/21 19:55, Thomas Gleixner wrote:
> We can do that, but I'm unhappy about this conditional in schedule(). So
> I was asking for doing a simple KVM only solution first:
> 
> vcpu_run()
>          kvm_load_guest_fpu()
>              wrmsrl(XFD, guest_fpstate->xfd);
>              XRSTORS
>            
>          do {
> 
>             local_irq_disable();
> 
>             if (test_thread_flag(TIF_NEED_FPU_LOAD))
> 		switch_fpu_return()
>                    wrmsrl(XFD, guest_fpstate->xfd);
> 
>             do {
>                  vmenter();              // Guest modifies XFD
>             } while (reenter);
> 
>             update_xfd_state();          // Restore consistency
> 
>             local_irq_enable();
> 
> and check how bad that is for KVM in terms of overhead on AMX systems.

I agree, this is how we handle SPEC_CTRL for example and it can be 
extended to XFD.  We should first do that, then switch to the MSR lists. 
  Hacking into schedule() should really be the last resort.

>            local_irq_enable();     <- Problem starts here
> 
>            preempt_enable();	   <- Becomes wider here

It doesn't become that much wider because there's always preempt 
notifiers.  So if it's okay to save XFD in the XSAVES wrapper and in 
kvm_arch_vcpu_put(), that might be already remove the need to do it 
schedule().

Paolo