Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Vitaly Kuznetsov <vkuznets@redhat.com>
To:     Xiaoyao Li <xiaoyao.li@intel.com>,
        Chenyi Qiang <chenyi.qiang@intel.com>
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>
Subject: Re: [RFC 2/2] KVM: VMX: Enable bus lock VM exit
In-Reply-To: <adad61e8-8252-0491-7feb-992a52c1b4f3@intel.com>
References: <20200628085341.5107-1-chenyi.qiang@intel.com> <20200628085341.5107-3-chenyi.qiang@intel.com> <878sg3bo8b.fsf@vitty.brq.redhat.com> <0159554d-82d5-b388-d289-a5375ca91323@intel.com> <87366bbe1y.fsf@vitty.brq.redhat.com> <adad61e8-8252-0491-7feb-992a52c1b4f3@intel.com>
Date:   Wed, 01 Jul 2020 16:49:49 +0200
Message-ID: <87zh8j9to2.fsf@vitty.brq.redhat.com>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Xiaoyao Li <xiaoyao.li@intel.com> writes:

> On 7/1/2020 8:44 PM, Vitaly Kuznetsov wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>> 
>>> On 7/1/2020 5:04 PM, Vitaly Kuznetsov wrote:
>>>> Chenyi Qiang <chenyi.qiang@intel.com> writes:
>>> [...]
>>>>>    static const int kvm_vmx_max_exit_handlers =
>>>>> @@ -6830,6 +6838,13 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)
>>>>>    	if (unlikely(vmx->exit_reason.failed_vmentry))
>>>>>    		return EXIT_FASTPATH_NONE;
>>>>>    
>>>>> +	/*
>>>>> +	 * check the exit_reason to see if there is a bus lock
>>>>> +	 * happened in guest.
>>>>> +	 */
>>>>> +	if (vmx->exit_reason.bus_lock_detected)
>>>>> +		handle_bus_lock(vcpu);
>>>>
>>>> In case the ultimate goal is to have an exit to userspace on bus lock,
>>>
>>> I don't think we will need an exit to userspace on bus lock. See below.
>>>
>>>> the two ways to reach handle_bus_lock() are very different: in case
>>>> we're handling EXIT_REASON_BUS_LOCK we can easily drop to userspace by
>>>> returning 0 but what are we going to do in case of
>>>> exit_reason.bus_lock_detected? The 'higher priority VM exit' may require
>>>> exit to userspace too. So what's the plan? Maybe we can ignore the case
>>>> when we're exiting to userspace for some other reason as this is slow
>>>> already and force the exit otherwise?
>>>
>>>> And should we actually introduce
>>>> the KVM_EXIT_BUS_LOCK and a capability to enable it here?
>>>>
>>>
>>> Introducing KVM_EXIT_BUS_LOCK maybe help nothing. No matter
>>> EXIT_REASON_BUS_LOCK or exit_reason.bus_lock_detected, the bus lock has
>>> already happened. Exit to userspace cannot prevent bus lock, so what
>>> userspace can do is recording and counting as what this patch does in
>>> vcpu->stat.bus_locks.
>> 
>> Exiting to userspace would allow to implement custom 'throttling'
>> policies to mitigate the 'noisy neighbour' problem. The simplest would
>> be to just inject some sleep time.
>> 
>
> So you want an exit to userspace for every bus lock and leave it all to 
> userspace. Yes, it's doable.
>

In some cases we may not even want to have a VM exit: think
e.g. real-time/partitioning case when even in case of bus lock we may
not want to add additional latency just to count such events. I'd
suggest we make the new capability tri-state:
- disabled (no vmexit, default)
- stats only (what this patch does)
- userspace exit
But maybe this is an overkill, I'd like to hear what others think.

> As you said, the exit_reason.bus_lock_detected case is the tricky one. 
> We cannot do the similar to extend vcpu->run->exit_reason, this breaks 
> ABI. Maybe we can extend the vcpu->run->flags to indicate bus lock 
> detected for the other exit reason?

This is likely the easiest solution.

-- 
Vitaly