Received-SPF: pass (google.com: domain of linux-kernel+bounces-110005-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <fa91d7bc-f4e5-4a1b-b35a-8e0ccfe1af2b@redhat.com>
Date: Thu, 21 Mar 2024 22:06:15 +1000
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] virtio_ring: Fix the stale index in available ring
Content-Language: en-US
To: Keir Fraser <keirf@google.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>, Will Deacon <will@kernel.org>,
 virtualization@lists.linux.dev, linux-kernel@vger.kernel.org,
 jasowang@redhat.com, xuanzhuo@linux.alibaba.com, yihyu@redhat.com,
 shan.gavin@gmail.com, linux-arm-kernel@lists.infradead.org,
 Catalin Marinas <catalin.marinas@arm.com>, mochs@nvidia.com
References: <20240314074923.426688-1-gshan@redhat.com>
 <20240318165924.GA1824@willie-the-truck>
 <35a6bcef-27cf-4626-a41d-9ec0a338fe28@redhat.com>
 <20240319182251.GB3121@willie-the-truck>
 <9500adaf-0075-4ae9-92db-7e310b6598b0@redhat.com>
 <20240319203540-mutt-send-email-mst@kernel.org>
 <3a6c8b23-af9c-47a7-8c22-8e0a78154bd3@redhat.com>
 <ZfsZljA_mmHKkBsA@google.com>
From: Gavin Shan <gshan@redhat.com>
In-Reply-To: <ZfsZljA_mmHKkBsA@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 3/21/24 03:15, Keir Fraser wrote:
> On Wed, Mar 20, 2024 at 03:24:16PM +1000, Gavin Shan wrote:
>>
>> Before this patch was posted, I had debugging code to record last 16 transactions
>> to the available and used queue from guest and host side. It did reveal the wrong
>> head was fetched from the available queue.
>>
>> [   11.785745] ================ virtqueue_get_buf_ctx_split ================
>> [   11.786238] virtio_net virtio0: output.0:id 74 is not a head!
>> [   11.786655] head to be released: 036 077
>> [   11.786952]
>> [   11.786952] avail_idx:
>> [   11.787234] 000  63985  <--
>> [   11.787237] 001  63986
>> [   11.787444] 002  63987
>> [   11.787632] 003  63988
>> [   11.787821] 004  63989
>> [   11.788006] 005  63990
>> [   11.788194] 006  63991
>> [   11.788381] 007  63992
>> [   11.788567] 008  63993
>> [   11.788772] 009  63994
>> [   11.788957] 010  63995
>> [   11.789141] 011  63996
>> [   11.789327] 012  63997
>> [   11.789515] 013  63998
>> [   11.789701] 014  63999
>> [   11.789886] 015  64000
> 
> Does the error always occur at such a round idx value?
> 
> Here, 64000 == 0xFA00. Maybe coincidence but it's improbable enough to be interesting.
> 
> This debug code seems rather useful!
> 

Keir, Nope, it's just coincidence. We don't have such kind of pattern.

Thanks,
Gavin

> 
> 
>> [   11.790068]
>> [   11.790068] avail_head:
>> [   11.790529] 000  075  <--
>> [   11.790718] 001  036
>> [   11.790890] 002  077
>> [   11.791061] 003  129
>> [   11.791231] 004  072
>> [   11.791400] 005  130
>> [   11.791574] 006  015
>> [   11.791748] 007  074
>> [   11.791918] 008  130
>> [   11.792094] 009  130
>> [   11.792263] 010  074
>> [   11.792437] 011  015
>> [   11.792617] 012  072
>> [   11.792788] 013  129
>> [   11.792961] 014  077    // The last two heads from guest to host: 077, 036
>> [   11.793134] 015  036
>>
>> [root@nvidia-grace-hopper-05 qemu.main]# cat /proc/vhost
>>
>> avail_idx
>> 000  63998
>> 001  64000
>> 002  63954  <---
>> 003  63955
>> 004  63956
>> 005  63974
>> 006  63981
>> 007  63984
>> 008  63986
>> 009  63987
>> 010  63988
>> 011  63989
>> 012  63992
>> 013  63993
>> 014  63995
>> 015  63997
>>
>> avail_head
>> 000  074
>> 001  015
>> 002  072
>> 003  129
>> 004  074            // The last two heads seen by vhost is: 074, 036
>> 005  036
>> 006  075  <---
>> 007  036
>> 008  077
>> 009  129
>> 010  072
>> 011  130
>> 012  015
>> 013  074
>> 014  130
>> 015  130
>>
>> used_idx
>> 000  64000
>> 001  63882  <---
>> 002  63889
>> 003  63891
>> 004  63898
>> 005  63936
>> 006  63942
>> 007  63946
>> 008  63949
>> 009  63953
>> 010  63957
>> 011  63981
>> 012  63990
>> 013  63992
>> 014  63993
>> 015  63999
>>
>> used_head
>> 000  072
>> 001  129
>> 002  074          // The last two heads published to guest is: 074, 036
>> 003  036
>> 004  075  <---
>> 005  036
>> 006  077
>> 007  129
>> 008  072
>> 009  130
>> 010  015
>> 011  074
>> 012  130
>> 013  130
>> 014  074
>> 015  015
>>
>> Thanks,
>> Gavin
>>
>>
>>
>>
>