Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: Spurious EIO on AIO+DIO+RWF_NOWAIT
To:     Goldwyn Rodrigues <rgoldwyn@suse.de>
Cc:     linux-kernel <linux-kernel@vger.kernel.org>, linux-aio@kvack.org
References: <9bab0f40-5748-f147-efeb-5aac4fd44533@scylladb.com>
 <20181210124823.iw4mxmdqpsdfeap4@merlin>
From:   Avi Kivity <avi@scylladb.com>
Message-ID: <8fb27ba4-c100-5d08-71b6-ef393d7bcae1@scylladb.com>
Date:   Wed, 12 Dec 2018 14:05:40 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.1
MIME-Version: 1.0
In-Reply-To: <20181210124823.iw4mxmdqpsdfeap4@merlin>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 12/10/18 2:48 PM, Goldwyn Rodrigues wrote:
> On 13:19 09/12, Avi Kivity wrote:
>> I have an application that receives spurious EIO when running with
>> RWF_NOWAIT enabled. Removing RWF_NOWAIT causes those EIOs to disappear. The
>> application uses AIO+DIO, and errors were seen on both xfs and ext4.
>>
>>
>> I suspect the following code:
>>
>>
>> /*
>>   * Process one completed BIO.  No locks are held.
>>   */
>> static blk_status_t dio_bio_complete(struct dio *dio, struct bio *bio)
>> {
>>          struct bio_vec *bvec;
>>          unsigned i;
>>          blk_status_t err = bio->bi_status;
>>
>>          if (err) {
>>                  if (err == BLK_STS_AGAIN && (bio->bi_opf & REQ_NOWAIT))
>>                          dio->io_error = -EAGAIN;
>>                  else
>>                          dio->io_error = -EIO;
>>          }
>>
>> Could it be that REQ_NOWAIT was dropped from bio->bi_opf? or that
>> bio->bi_status got changed along the way?
>>
> I don't think REQ_NOWAIT is dropped. I am assuming bio->bi_status error
> is set differently. Is the blk queue being stopped? Is it possible to
> instrument the kernel in your testcase?
>

I traced the function, and I see bio->bi_status == BLK_STS_NOTSUPP and 
bio->bi_opf == REQ_OP_WRITE|REQ_SYNC|REQ_NOMERGE|REQ_FUA|REQ_NOWAIT. 
Presumably the NOTSUPP is the result of NOWAIT not being supported down 
the stack, but shouldn't it be detected earlier? And not converted to EIO?