Received-SPF: pass (google.com: domain of linux-kernel+bounces-119412-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <22be7d6156e38dfba1a055cc3e9cc3d10de75dbb.camel@redhat.com>
Subject: Re: [PATCH 0/5] AVIC bugfixes and workarounds
From: mlevitsk@redhat.com
To: Jim Mattson <jmattson@google.com>
Cc: kvm@vger.kernel.org, Will Deacon <will@kernel.org>, Borislav Petkov
 <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>, Suravee
 Suthikulpanit <suravee.suthikulpanit@amd.com>, Thomas Gleixner
 <tglx@linutronix.de>, Paolo Bonzini <pbonzini@redhat.com>, x86@kernel.org,
 Robin Murphy <robin.murphy@arm.com>,  iommu@lists.linux.dev, Ingo Molnar
 <mingo@redhat.com>, Joerg Roedel <joro@8bytes.org>, Sean Christopherson
 <seanjc@google.com>, "H. Peter Anvin" <hpa@zytor.com>,
 linux-kernel@vger.kernel.org, David Rientjes <rientjes@google.com>
Date: Tue, 26 Mar 2024 11:59:30 -0400
In-Reply-To: <CALMp9eSSCUSOpP64Ho16sU6iV1urbjfTafJ0nThAWGHE6oOkLw@mail.gmail.com>
References: <20230928150428.199929-1-mlevitsk@redhat.com>
	 <CALMp9eSSCUSOpP64Ho16sU6iV1urbjfTafJ0nThAWGHE6oOkLw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4 (3.44.4-3.fc36) 
Precedence: bulk
MIME-Version: 1.0

On Mon, 2024-03-25 at 20:15 -0700, Jim Mattson wrote:
> > On Thu, Sep 28, 2023 at 8:05=E2=80=AFAM Maxim Levitsky <mlevitsk@redhat=
com> wrote:
> > > >=20
> > > > Hi!
> > > >=20
> > > > This patch series includes several fixes to AVIC I found while work=
ing
> > > > on a new version of nested AVIC code.
> > > >=20
> > > > Also while developing it I realized that a very simple workaround f=
or
> > > > AVIC's errata #1235 exists and included it in this patch series as =
well.
> > > >=20
> > > > Best regards,
> > > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Maxim Levitsky
> >=20
> > Can someone explain why we're still unwilling to enable AVIC by
> > default? Have the performance issues that plagued the Rome
> > implementation been fixed? What is AMD's guidance?
> >=20
Hi

This is what I know:

Zen1:
	I never tested it, so I don't know how well AVIC works there and if it has=
 any erratas.

Zen2:
	Has CPU errata in regard to IPI virtualization that makes it unusable in p=
roduction,
 	but if AVIC's IPI virtualization (borrowing the Intel term here) is disab=
led,
	then it works just fine and 1:1 equivalent to APICv without IPI.

	I posted patches for this several times, latest version is here, it still =
applies I think:
	https://lkml.iu.edu/hypermail/linux/kernel/2310.0/00790.html

Zen3:
	For some reason AVIC got disabled by AMD in CPUID. It is still there thoug=
h and force_avic=3D1 kvm_amd option
	can make KVM use it and AFAIK it works just fine.

	It is possible that it got disabled due to Zen2 errata that is fixed on Ze=
n3,
	but maybe AMD wasn't sure back then that it will be fixed or it might be d=
ue to performance issues with broadcast
	IPIs which I think ended up being a software issue and was fixed a long ti=
me ago.

Zen4+
	I haven't tested it much, but AFAIK it should work out of the box. It also=
 got x2avic mode which allows
	to use AVIC with VMs that have more that 254 vCPUs.

IMHO if we merge the workaround I have for IPI virtualization and make IPI =
virtualization off for Zen2
(and maybe Zen1 as well), then I don't see why we can't make AVIC be the de=
fault on.

Best regards,
	Maxim Levitsky