Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 19 Mar 2021 16:25:40 -0600
From:   Alex Williamson <alex.williamson@redhat.com>
To:     Christoph Hellwig <hch@infradead.org>
Cc:     cohuck@redhat.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, jgg@nvidia.com, peterx@redhat.com
Subject: Re: [PATCH v1 07/14] vfio: Add a device notifier interface
Message-ID: <20210319162540.0c5fe9dd@omen.home.shazbot.org>
In-Reply-To: <20210310075639.GB662265@infradead.org>
References: <161523878883.3480.12103845207889888280.stgit@gimli.home>
        <161524010999.3480.14282676267275402685.stgit@gimli.home>
        <20210310075639.GB662265@infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, 10 Mar 2021 07:56:39 +0000
Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Mar 08, 2021 at 02:48:30PM -0700, Alex Williamson wrote:
> > Using a vfio device, a notifier block can be registered to receive
> > select device events.  Notifiers can only be registered for contained
> > devices, ie. they are available through a user context.  Registration
> > of a notifier increments the reference to that container context
> > therefore notifiers must minimally respond to the release event by
> > asynchronously removing notifiers.  
> 
> Notifiers generally are a horrible multiplexed API.  Can't we just
> add a proper method table for the intended communication channel?

I've been trying to figure out how, but I think not.  A user can have
multiple devices, each with entirely separate IOMMU contexts.  For each
device, the user can create an mmap of memory to that device and add it
to every other IOMMU context.  That enables peer to peer DMA between
all the devices, across all the IOMMU contexts.  But each individual
device has no direct reference to any IOMMU context other than its own.
A callback on the IOMMU can't reach those other contexts either, there's
no guarantee those other contexts are necessarily managed via the same
vfio IOMMU backend driver.  A notifier is the best I can come up with,
please suggest if you have other ideas.  Thanks,

Alex