Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 19 Oct 2022 19:28:17 -0600
From:   Keith Busch <kbusch@kernel.org>
To:     Dmitrii Tcvetkov <me@demsh.org>
Cc:     Keith Busch <kbusch@fb.com>, Jens Axboe <axboe@kernel.dk>,
        linux-block@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [bisected] QEMU guest boot failure since 6.0 on x86_64 host
Message-ID: <Y1CkMS+xbwbvn8My@kbusch-mbp.dhcp.thefacebook.com>
References: <20221020031725.7d01051a@xps.demsh.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221020031725.7d01051a@xps.demsh.org>
Precedence: bulk

On Thu, Oct 20, 2022 at 03:17:25AM +0300, Dmitrii Tcvetkov wrote:
> 
> Bisect led me to commit b1a000d3b8ec5  ("block: relax direct io memory
> alignment"). I was unable to resolve revert conflicts when
> tried to revert b1a000d3b8ec5  ("block: relax direct io memory
> alignment") as I lack necessary understanding of block subsystem.

Background info: when your virtual block device's logical block size is
smaller than the host's block device backing it, qemu needs to bounce
unaligned buffers when using direct-io.

Historically for direct-io, the logical block size happened to also be
the memory page offset alignment. QEMU did this the other way around: it
used the memory offset as the block size, and that was not intended:

  https://lore.kernel.org/lkml/32db4f89-a83f-aac4-5d27-0801bdca60bf@redhat.com/

The kernel patch you bisected to detangled memory alignment from logical
block size, so now older qemu versions have the wrong idea of the
minimum vector size. That is fixed in the qemu repository here:

  https://git.qemu.org/?p=qemu.git;a=commitdiff;h=25474d90aa50bd32e0de395a33d8de42dd6f2aef
> 
> This fails to boot on 6.0+ host:
> # losetup -b 4096 -f image.raw
> # qemu-system-x86_64 -enable-kvm -drive
> file=/dev/loop0,format=raw,cache=none

In the above, your backing storage is 4k, and the default virtual device
block size is 512b, so qemu needs to bounce that, but older versions
might not do that as intended.

It should work if you include logical_block_size=4096 to the -drive
parameters.

> These boot fine on 6.0+ host:
> # losetup -b 4096 -f image.raw
> # qemu-system-x86_64 -enable-kvm -drive
> file=/dev/loop0,format=raw

The above is using cache, which doesn't have any alignment and size
constraints, so works with anything sizes.
 
> # losetup -f image.raw
> # qemu-system-x86_64 -enable-kvm -drive
> file=/dev/loop0,format=raw,cache=none

The above is using a 512b formated backing store to a 512b emulated
drive, so the matching means qemu never needs to bounce.