Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
Date:   Tue, 05 Dec 2023 17:01:28 +0000
Message-ID: <86bkb4bn2v.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Jason Gunthorpe <jgg@nvidia.com>
Cc:     Catalin Marinas <catalin.marinas@arm.com>, ankita@nvidia.com,
        Shameerali Kolothum Thodi 
        <shameerali.kolothum.thodi@huawei.com>, oliver.upton@linux.dev,
        suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org,
        ardb@kernel.org, akpm@linux-foundation.org, gshan@redhat.com,
        aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com,
        targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com,
        apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com,
        mochs@nvidia.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org,
        lpieralisi@kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2 1/1] KVM: arm64: allow the VM to select DEVICE_* and NORMAL_NC for IO memory
In-Reply-To: <20231205164318.GG2692119@nvidia.com>
References: <20231205033015.10044-1-ankita@nvidia.com>
        <86fs0hatt3.wl-maz@kernel.org>
        <ZW8MP2tDt4_9ROBz@arm.com>
        <20231205130517.GD2692119@nvidia.com>
        <ZW9OSe8Z9gAmM7My@arm.com>
        <20231205164318.GG2692119@nvidia.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1
 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Tue, 05 Dec 2023 16:43:18 +0000,
Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> On Tue, Dec 05, 2023 at 04:22:33PM +0000, Catalin Marinas wrote:

> > That's an argument to restrict this feature to PCIe. It's really about
> > fewer arguments on the behaviour of other devices. Marc did raise
> > another issue with the GIC VCPU interface (does this even have a vma in
> > the host VMM?). That's a class of devices where the mapping is
> > context-switched, so the TLBI+DSB rules don't help.

There is no vma. The CPU interface is entirely under control of KVM.
Userspace only provides the IPA for the mapping.

> 
> I don't know anything about the GIC VCPU interface, to give any
> comment unfortunately. Since it seems there is something to fix here I
> would appreciate some background..
>  
> When you say it is context switched do you mean kvm does a register
> write on every vm entry to set the proper HW context for the vCPU?

The CPU interface is mapped in every guest S2 page tables as a per-CPU
device, and under complete control of the guest. There is no KVM
register write to that frame (unless we're proxying it, but that's for
another day).

> 
> We are worrying that register write will possibly not order after
> NORMAL_NC?

Guest maps the device as Normal-NC (because it now can), which means
that there is no control over the alignment or anything like that. The
accesses could also be reordered, and/or hit after a context switch to
another guest. Which is why KVM has so far used nGnRE as the mapping
type.

	M.

-- 
Without deviation from the norm, progress is not possible.