Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Mon, 04 Apr 2022 15:47:11 +0100
Message-ID: <87o81gc3dc.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Jason Gunthorpe <jgg@ziepe.ca>
Cc:     xieming <xieming@kylinos.cn>, sashal@kernel.org,
        catalin.marinas@arm.com, linux@armlinux.org.uk,
        linux-kernel@vger.kernel.org, alex.williamson@redhat.com,
        will@kernel.org, kvmarm@lists.cs.columbia.edu,
        linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2] kvm/arm64: fixed passthrough gpu into vm on arm64
In-Reply-To: <20220404132405.GQ64706@ziepe.ca>
References: <20220401090828.614167-1-xieming@kylinos.cn>
        <87tubcbvgk.wl-maz@kernel.org>
        <20220404132405.GQ64706@ziepe.ca>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Mon, 04 Apr 2022 14:24:05 +0100,
Jason Gunthorpe <jgg@ziepe.ca> wrote:
> 
> On Fri, Apr 01, 2022 at 05:48:59PM +0100, Marc Zyngier wrote:
> 
> > NAK. For a start, there is no such thing as 'write-combine' in the ARM
> > architecture, and I'm not convinced you can equate WC to Normal-NC.
> > See the previous discussion at [1].
> > 
> > [1] https://lore.kernel.org/r/20210429162906.32742-1-sdonthineni@nvidia.com
> 
> We've had a lot of discussions with ARM related to how this works with
> drivers like mlx5 that use WC.
> 
> ARM has now published some guidance on this:
> 
> https://community.arm.com/arm-research/m/resources/1012

Nicely buried where nobody would dare looking.

> 
> As an ecosystem we seem to be drifting toward Normal-NC for this
> behavior (largely because it is what Linux does). At least that is
> what we are testing and qualifing ARM CPUs against mlx5 with.
> 
> I'm guessing it will turn into a SBSA like thing where the ARM ARM is
> kind of vauge but a SOC has to implement Normal-NC in a certain way to
> be functional for the server market.

The main issue is that this equivalence isn't architected, so people
can build whatever they want. SBSA means nothing to KVM (or Linux at
large), and there is currently no way to describe which devices are
safe to map as Normal-NC vs Device.

We either have to take userspace's word for it, or rely on some other
heuristics (do this for PCIe, but not anything else). None of which
are entirely safe. Not to mention that no currently available CPU
implements FEAT_DGH.

	M.

-- 
Without deviation from the norm, progress is not possible.