Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <9a96fc6f-23c5-e2f1-062b-39aa0c7ba724@suse.com>
Date:   Wed, 13 Jul 2022 11:48:59 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH] xen/netback: handle empty rx queue in
 xenvif_rx_next_skb()
Content-Language: en-US
To:     Juergen Gross <jgross@suse.com>
Cc:     Wei Liu <wei.liu@kernel.org>, Paul Durrant <paul@xen.org>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        xen-devel@lists.xenproject.org, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <20220713074823.5679-1-jgross@suse.com>
 <0e2772a3-3c3c-b447-ecb5-e2750959b527@suse.com>
 <b11693ec-5d08-62e7-7479-a631edd5b1ce@suse.com>
From:   Jan Beulich <jbeulich@suse.com>
In-Reply-To: <b11693ec-5d08-62e7-7479-a631edd5b1ce@suse.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?czRKTEtpWnh1V0F4cDBPb1RZUXlMdHNxZVd5M1hnV01VUUpzOS9qTzJrbldn?=
 =?utf-8?B?YlFHUnhUNWtZMDZjS05BVUFiaXZqTGhRTFJPdEtuUnV3L2NTRVlJMUtxMlFs?=
 =?utf-8?B?eXNQQTNMYmdLVW82bVpwRGJGTmd1dU1USi9DRXVFSXRYQzRrT0ZYaHFpM1hy?=
 =?utf-8?B?d1g1U1ZIRVFISHVZRThmWmZhUGtZSEc2Vm5ZdCtTRWZXeHNvYWNHZ25pdlhR?=
 =?utf-8?B?bzRabWtWSnNmS2dVSThDZzVOUTdFcXJtUTRGV2QvbWJTTzJiUXJ1MVlUYXlJ?=
 =?utf-8?B?aGc4VlUzV0t2WmhQTVJmYmJLRGwxRlFLblpMNldpOFF3V1FQT0VOZ0tJVk9i?=
 =?utf-8?B?RzRlTVBHcUJIUmdkYk9VRXFaODFoOGs1N3VkS2ZIUFlCMkZET2NNT3lMbEFJ?=
 =?utf-8?B?aSt3VVBPbE5VWVNnc3EwNVYrdHh3aUlBYy9WMEhESU1yVEdWdm13MHd6Z0ZF?=
 =?utf-8?B?d3hmajg4eXd0ZVhZVFBCeXdtWHJvSDZtTXNRQW4vcTlqVkYwUEFBUjd5Q1FI?=
 =?utf-8?B?VnRmWFRTTndaN3F0RHllYTlmYjBKc21mQjNQNFcvK21HdlpyeWJZOFViN2po?=
 =?utf-8?B?T0tWTnBxbU9sRTBNc1I3b0JpeW9lS2VVR2dqSHRBL3BLWlhIN1pUUWV4MFUy?=
 =?utf-8?B?bVBhMHdGLzd1MmdGRWFRU1R1UEMrYWJ1TWluTEdCUWwweDZla1c5SXhocFRF?=
 =?utf-8?B?NzE2ZktGRzFhUjA5SlpBeElFcndyQ3hNUm96ZkZIdkF1WlZTK0kzZ2FTbVJz?=
 =?utf-8?B?dXd0eFhWNnZGWG5uTDZRV3VZUkoxZ3pWK3RXUXNQbDl2N1VCdVIvYXdGOE55?=
 =?utf-8?B?Y21ySWpxTUo2Y29OL1JwL05BN3BLbnBFOTIvUWthV0EvWEs0WHd3UXAwZkRp?=
 =?utf-8?B?eWZKWURtS2JHRWFWVlZGdHlQaHVnWEpLVGwyeHdsZk1QM3lwQURGc01XZTds?=
 =?utf-8?B?LzdZTHVtQnZ5ejMyb0JoYStreXNCclVrb1UvbWc2NEVMY3VySHdVMmFxYWMv?=
 =?utf-8?B?Tko4Y2FwUmNTbHByTlUvR3NlOXBwUE9sdzhuWVBhOFc1V3JMNklOOEd5NS9R?=
 =?utf-8?B?dmZzcTBoTVB0RUhoMDdsZS9wTlNlYURXQlI4TDc4MzloVHZJQk9aeG8zUFRV?=
 =?utf-8?B?OWJlMDJDM0RqdHdqWERYdVBVS0ZnNWRNYU4vUjFOZkJFMWVCNXk4MEpVck5z?=
 =?utf-8?B?SURYRytoWmFJblpEcXQrODY0T3ZVSnpLdVpDT3JPWGdiR1dGMVhOZmZxb1NJ?=
 =?utf-8?B?dlFuSnR2MitCUkdSWDMzbzZ5ME5SVFp0ZkNrWkprN2pxQlpFaFJYRnpmSTBq?=
 =?utf-8?B?ZDE5UXhhREtlOU9DemVGMUpPZjlLL2hUU0Y3ZlJQQm9zYktpZzRJU2RJSWJi?=
 =?utf-8?B?TGRtYlZZbDhzVEhBTlFUK0ZyVnE1akQ5TG9XQnZuTjI0QkJSRTlvMXBLeE8w?=
 =?utf-8?B?eE5tMEFSWW1Ia29MV25Db0ZlNVVFT2h4RUJ4S0gwN0xjZEJrNkI5bmg1Uk93?=
 =?utf-8?B?TENaSGVYdW9sbFZOMno2ZGIycVNnS0dzU3piWXJKTGozYTkwSDJJSitKK2tj?=
 =?utf-8?B?V2JxbWh4NGRsUTdZcnBvK2ZGd0Q0Qy83cUdRWjVTVmRkN1NMRUdoYnlNem9m?=
 =?utf-8?B?OTN3M0tSby93UklIK3Y1aHB4Tkh4eDU2NUc2TDF2MStiYVlTR3ZVTUNPYXE0?=
 =?utf-8?B?Q0I5QzNvS1RsM3dIczdIUVBaWWJ3M1JhQW5idlMreHo4ZmJyWDBNZnVvNFJP?=
 =?utf-8?B?dUpIaXlzK3FXSFZEVUxlTWQvYmVjUUNMbXRzcnIvS0JrZEtQeVZ5cG8zelJp?=
 =?utf-8?B?T0pBRTNsQVlITGFIYnZrOGYzdzMxTFNQUGh6OHpVai96ZWRVZWZGbXAwc0Qx?=
 =?utf-8?B?RXJHanp3UGJkdnJuWkhJamNobUNlc0VNL2tzRUNRUk1vdUdPZTd5OU15QXFE?=
 =?utf-8?B?dnZmQXd3d2ZPaXh1d0FLOVFDNWhzTHhFcGJ3S1QxYUJpTFkrcEpYSkJIS0c3?=
 =?utf-8?B?Rkdlak45dmpSVzcybVlHRkVQbUd0cFVUNE04M1FlTVBObWtYZXd4VS90Tkl4?=
 =?utf-8?B?NE42cGNPcDRqUG45dWpLbG1wSXhBcDM5VGZnWkZBZGdSSnZZeEx0OUNtU1Vi?=
 =?utf-8?Q?wfzO406qvRqJc4oX5ueY3rCE8?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e77f8f1-e75d-4268-6250-08da64b4ea41
X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jul 2022 09:49:01.4953
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HKbrqYGYiu9sI1+QiLoj4euxAjdtlFONEKpXgyNrMkHxnbnDjn2u5dYFf27TMlwoNirNupvNYHILaTMfWNYTQA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB9074
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 13.07.2022 11:31, Juergen Gross wrote:
> On 13.07.22 09:59, Jan Beulich wrote:
>> On 13.07.2022 09:48, Juergen Gross wrote:
>>> xenvif_rx_next_skb() is expecting the rx queue not being empty, but
>>> in case the loop in xenvif_rx_action() is doing multiple iterations,
>>> the availability of another skb in the rx queue is not being checked.
>>>
>>> This can lead to crashes:
>>>
>>> [40072.537261] BUG: unable to handle kernel NULL pointer dereference at 0000000000000080
>>> [40072.537407] IP: xenvif_rx_skb+0x23/0x590 [xen_netback]
>>> [40072.537534] PGD 0 P4D 0
>>> [40072.537644] Oops: 0000 [#1] SMP NOPTI
>>> [40072.537749] CPU: 0 PID: 12505 Comm: v1-c40247-q2-gu Not tainted 4.12.14-122.121-default #1 SLE12-SP5
>>> [40072.537867] Hardware name: HP ProLiant DL580 Gen9/ProLiant DL580 Gen9, BIOS U17 11/23/2021
>>> [40072.537999] task: ffff880433b38100 task.stack: ffffc90043d40000
>>> [40072.538112] RIP: e030:xenvif_rx_skb+0x23/0x590 [xen_netback]
>>> [40072.538217] RSP: e02b:ffffc90043d43de0 EFLAGS: 00010246
>>> [40072.538319] RAX: 0000000000000000 RBX: ffffc90043cd7cd0 RCX: 00000000000000f7
>>> [40072.538430] RDX: 0000000000000000 RSI: 0000000000000006 RDI: ffffc90043d43df8
>>> [40072.538531] RBP: 000000000000003f R08: 000077ff80000000 R09: 0000000000000008
>>> [40072.538644] R10: 0000000000007ff0 R11: 00000000000008f6 R12: ffffc90043ce2708
>>> [40072.538745] R13: 0000000000000000 R14: ffffc90043d43ed0 R15: ffff88043ea748c0
>>> [40072.538861] FS: 0000000000000000(0000) GS:ffff880484600000(0000) knlGS:0000000000000000
>>> [40072.538988] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
>>> [40072.539088] CR2: 0000000000000080 CR3: 0000000407ac8000 CR4: 0000000000040660
>>> [40072.539211] Call Trace:
>>> [40072.539319] xenvif_rx_action+0x71/0x90 [xen_netback]
>>> [40072.539429] xenvif_kthread_guest_rx+0x14a/0x29c [xen_netback]
>>>
>>> Fix that by stopping the loop in case the rx queue becomes empty.
>>>
>>> Signed-off-by: Juergen Gross <jgross@suse.com>
>>
>> Reviewed-by: Jan Beulich <jbeulich@suse.com>
>>
>> Does this want a Fixes: tag and Cc: to stable@ (not the least since as per
>> above the issue was noticed with 4.12.x)?
> 
> Hmm, I _think_ the issue was introduced with eb1723a29b9a. Do you agree?

If it was that, then something must have invoked xenvif_rx_action()
without there actually being work. I'd rather see 98f6d57ced73
("xen-netback: process guest rx packets in batches") as the origin.

>>> --- a/drivers/net/xen-netback/rx.c
>>> +++ b/drivers/net/xen-netback/rx.c
>>> @@ -495,6 +495,7 @@ void xenvif_rx_action(struct xenvif_queue *queue)
>>>   	queue->rx_copy.completed = &completed_skbs;
>>>   
>>>   	while (xenvif_rx_ring_slots_available(queue) &&
>>> +	       !skb_queue_empty(&queue->rx_queue) &&
>>>   	       work_done < RX_BATCH_SIZE) {
>>>   		xenvif_rx_skb(queue);
>>>   		work_done++;
>>
>> I have to admit that I find the title a little misleading - you don't
>> deal with the issue _in_ xenvif_rx_next_skb(); you instead avoid
>> entering the function in such a case.
> 
> I'm handling the issue to avoid "an empty rx queue in xenvif_rx_next_skb()".
> 
> I can rephrase it to "avoid entering xenvif_rx_next_skb() with an empty rx
> queue".

That or simply s/handle/avoid/ in the original title.

Jan