Message-ID: <9afc4ca6-a326-79ea-cc0b-4ce0808217d2@redhat.com>
Date:   Wed, 17 Nov 2021 08:39:27 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.0
Subject: Re: Thoughts of AMX KVM support based on latest kernel
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        Sean Christopherson <seanjc@google.com>
Cc:     "Liu, Jing2" <jing2.liu@intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Nakajima, Jun" <jun.nakajima@intel.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Jing Liu <jing2.liu@linux.intel.com>,
        "Cooper, Andrew" <andrew.cooper3@citrix.com>,
        "Bae, Chang Seok" <chang.seok.bae@intel.com>
References: <BYAPR11MB325685AB8E3DFD245846F854A9939@BYAPR11MB3256.namprd11.prod.outlook.com>
 <87k0h85m65.ffs@tglx> <YZPWsICdDTZ02UDu@google.com> <87ee7g53rp.ffs@tglx>
 <04978d6d-8e1a-404d-b30d-402a7569c1f0@redhat.com> <87zgq34z4c.ffs@tglx>
From:   Paolo Bonzini <pbonzini@redhat.com>
In-Reply-To: <87zgq34z4c.ffs@tglx>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 11/16/21 21:36, Thomas Gleixner wrote:
>            local_irq_enable();     <- Problem starts here
> 
>            preempt_enable();	   <- Becomes wider here
>
>> It doesn't become that much wider because there's always preempt
>> notifiers.  So if it's okay to save XFD in the XSAVES wrapper and in
>> kvm_arch_vcpu_put(), that might be already remove the need to do it
>> schedule().
>
> Did not think about preemption notifiers. Probably because I hate
> notifiers with a passion since I had to deal with the CPU hotplug
> notifier trainwreck.
> 
> But yes that would work. So the places to do that would be:
> 
> 1) kvm_sched_out() -> kvm_arch_vcpu_put() > 2) kernel_fpu_begin_mask()

... which calls save_fpregs_to_fpstate

> 3) kvm_put_guest_fpu()

... which calls save_fpregs_to_fpstate via fpu_swap_kvm_fpstate

So perhaps it could be done in save_fpregs_to_fpstate (for the sched out 
path, it would be called by switch_fpu_prepare()).  But for now I would 
also start with the trivial version.

> I'd be really surprised if that RDMSR is truly noticeable within all the
> other crud this path is doing.

I agree.

Paolo