Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 1 Nov 2022 11:22:21 -0600
From:   Keith Busch <kbusch@kernel.org>
To:     Dmitrii Tcvetkov <me@demsh.org>
Cc:     Jens Axboe <axboe@kernel.dk>, Song Liu <song@kernel.org>,
        linux-raid@vger.kernel.org, linux-block@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [bisected] RAID1 direct IO redirecting sector loop since 6.0
Message-ID: <Y2FVzdcro8HCfODK@kbusch-mbp>
References: <20221101001558.648ee024@xps.demsh.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221101001558.648ee024@xps.demsh.org>
Precedence: bulk

On Tue, Nov 01, 2022 at 12:15:58AM +0300, Dmitrii Tcvetkov wrote:
> 
> # cat /proc/7906/stack
> [<0>] submit_bio_wait+0xdb/0x140
> [<0>] blkdev_direct_IO+0x62f/0x770
> [<0>] blkdev_read_iter+0xc1/0x140
> [<0>] vfs_read+0x34e/0x3c0
> [<0>] __x64_sys_pread64+0x74/0xc0
> [<0>] do_syscall_64+0x6a/0x90
> [<0>] entry_SYSCALL_64_after_hwframe+0x4b/0xb5
> 
> After "mdadm --fail" invocation the last line becomes:
> [pid  7906] pread64(13, 0x627c34c8d200, 4096, 0) = -1 EIO (Input/output error)

It looks like something isn't accounting for the IO size correctly when
there's an offset. It may be something specific to one of the stacking
drivers in your block setup. Does this still happen without the
cryptosetup step?

For a different experiment, it may be safer to just force all alignment
for stacking drivers. Could you try the following and see if that gets
it working again? 

---
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 8bb9eef5310e..5c16fdb00c6f 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -646,6 +646,7 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b,
 		t->misaligned = 1;
 		ret = -1;
 	}
+	blk_queue_dma_alignment(t, t->logical_block_size - 1);
 
 	t->max_sectors = blk_round_down_sectors(t->max_sectors, t->logical_block_size);
 	t->max_hw_sectors = blk_round_down_sectors(t->max_hw_sectors, t->logical_block_size);
--