Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sun, 4 Dec 2022 17:51:42 -0700
From:   Alex Williamson <alex.williamson@redhat.com>
To:     "mb@lab.how" <mb@lab.how>
Cc:     airlied@linux.ie, dri-devel@lists.freedesktop.org,
        kraxel@redhat.com, kvm@vger.kernel.org, lersek@redhat.com,
        linux-kernel@vger.kernel.org, tzimmermann@suse.de
Subject: Re: [PATCH 2/2] vfio/pci: Remove console drivers
Message-ID: <20221204175142.658d5c37.alex.williamson@redhat.com>
In-Reply-To: <CAEdEoBYXHq9cCzsbMYTpG1B41Yz=-QAjFx7bJDOnPanN5Tmo7A@mail.gmail.com>
References: <CAEdEoBYXHq9cCzsbMYTpG1B41Yz=-QAjFx7bJDOnPanN5Tmo7A@mail.gmail.com>
Organization: Red Hat
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Sat, 3 Dec 2022 17:12:38 -0700
"mb@lab.how" <mb@lab.how> wrote:

> Hi,
> 
> I hope it is ok to reply to this old thread.

It is, but the only relic of the thread is the subject.  For reference,
the latest version of this posted is here:

https://lore.kernel.org/all/20220622140134.12763-4-tzimmermann@suse.de/

Which is committed as:

d17378062079 ("vfio/pci: Remove console drivers")

> Unfortunately, I found a
> problem only now after upgrading to 6.0.
> 
> My setup has multiple GPUs (2), and I depend on EFIFB to have a working console.
> pre-patch behavior, when I bind the vfio-pci to my secondary GPU both
> the passthrough and the EFIFB keep working fine.
> post-patch behavior, when I bind the vfio-pci to the secondary GPU,
> the EFIFB disappears from the system, binding the console to the
> "dummy console".
> Whenever you try to access the terminal, you have the screen stuck in
> whatever was the last buffer content, which gives the impression of
> "freezing," but I can still type.
> Everything else works, including the passthrough.

This sounds like the call to aperture_remove_conflicting_pci_devices()
is removing the conflicting driver itself rather than removing the
device from the driver.  Is it not possible to unbind the GPU from
efifb before binding the GPU to vfio-pci to effectively nullify the
added call?
 
> I can only think about a few options:
> 
> - Is there a way to have EFIFB show up again? After all it looks like
> the kernel has just abandoned it, but the buffer is still there. I
> can't find a single message about the secondary card and EFIFB in
> dmesg, but there's a message for the primary card and EFIFB.
> - Can we have a boolean controlling the behavior of vfio-pci
> altogether or at least controlling the behavior of vfio-pci for that
> specific ID? I know there's already some option for vfio-pci and VGA
> cards, would it be appropriate to attach this behavior to that option?

I suppose we could have an opt-out module option on vfio-pci to skip
the above call, but clearly it would be better if things worked by
default.  We cannot make full use of GPUs with vfio-pci if they're
still in use by host console drivers.  The intention was certainly to
unbind the device from any low level drivers rather than disable use of
a console driver entirely.  DRM/GPU folks, is that possibly an
interface we could implement?  Thanks,

Alex