Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "dengqiao.joey" <dengqiao.joey@bytedance.com>
To:     joao.m.martins@oracle.com
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        pbonzini@redhat.com, seanjc@google.com
Subject: Re: [PATCH] KVM: SVM: Update destination when updating pi irte
Date:   Thu, 18 May 2023 11:58:06 +0800
Message-Id: <20230518035806.938517-1-dengqiao.joey@bytedance.com>
In-Reply-To: <3d05fcf1-dad3-826e-03e9-599ced7524b4@oracle.com>
References: <3d05fcf1-dad3-826e-03e9-599ced7524b4@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

On Wed, May 17, 2023 at 11:32 PM Joao Martins <joao.m.martins@oracle.com> wrote:
>
> On 17/05/2023 13:04, dengqiao.joey wrote:
> > This seems to be a different issue. Joao's patch tries to fix the issue
> > that IRTE is not changed. The problem I encountered is that IRTE does
> > get changed, thus the destination field is also cleared by
> > amd_iommu_activate_guest_mode().
>
> Whether the amd_iommu_activate_guest_mode() clears the destination field or not
> doesn't change the situation I think. So I don't think that is the problem
> you're having. amd_iommu_activate_guest_mode() will always make IRTEs with
> isRun=0. Which means your VF interrupts get injected via GALog events and the
> DestID is meaningless there :/ Even if you leave the old affinity there with
> isRun=1 bit it's still wrong as by the time you run the vcpu, the wrong affinity
> will be used by the VF. More fundamentally I am not sure that on IRTE routing
> updates in the VM that you can look on vcpu->cpu the way you are using it could
> be potentially invalid when vcpus are blocked/preempted. You're changing IRTEs
> in the whole VM in pi_update_irte()
>
> My changes essentially handle the fact where IRTEs will be always updated with
> the right GATag set in the IRTE ... and then when the vCPU migrates, wakes or
> block+unblocks it will change IRTE[DestID] with the new pcpu (which the vcpu is
> running on) via amd_iommu_update_ga(). So there's a good chance my changes fixes
> your affinity issue.
>
>         Joao

Sorry I forgot to mention before that in my application scenario, the vcpu runs on
a dedicated isolated cpu and uses "mwait" instead of "halt". So it will not be migrated,
blocked/wake and rarely get preempted by other threads. I think your patch can not fix
my issue because the vcpu rarely gets the opportunity to execute amd_iommu_update_ga()
from vcpu_load().

So each time the interrupt affinity is updated I can observe the loss of VF interrupt.
After applying my patch, the problem is solved.