Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752047AbdCCRMP (ORCPT ); Fri, 3 Mar 2017 12:12:15 -0500 Received: from mail-pg0-f41.google.com ([74.125.83.41]:34884 "EHLO mail-pg0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751083AbdCCRMM (ORCPT ); Fri, 3 Mar 2017 12:12:12 -0500 Date: Fri, 3 Mar 2017 09:12:04 -0800 From: Stephen Hemminger To: Vitaly Kuznetsov Cc: Dexuan Cui , Stephen Hemminger , "gregkh@linuxfoundation.org" , Haiyang Zhang , "driverdev-devel@linuxdriverproject.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH] vmbus: remove hv_event_tasklet_disable/enable Message-ID: <20170303091204.2cab4a24@xeon-e3> In-Reply-To: <874lzal3e8.fsf@vitty.brq.redhat.com> References: <874lzal3e8.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3124 Lines: 67 On Fri, 03 Mar 2017 17:40:47 +0100 Vitaly Kuznetsov wrote: > Dexuan Cui writes: > > > With the recent introduction of per-channel tasklet, we need to update > > the way we handle the 3 concurrency issues: > > > > 1. hv_process_channel_removal -> percpu_channel_deq vs. > > vmbus_chan_sched -> list_for_each_entry(..., percpu_list); > > > > 2. vmbus_process_offer -> percpu_channel_enq/deq vs. vmbus_chan_sched. > > > > 3. vmbus_close_internal vs. the per-channel tasklet vmbus_on_event; > > > > The first 2 issues can be handled by Stephen's recent patch > > "vmbus: use rcu for per-cpu channel list", and the third issue > > can be handled by calling tasklet_disable in vmbus_close_internal here. > > > > We don't need the original hv_event_tasklet_disable/enable since we > > now use per-channel tasklet instead of the previous per-CPU tasklet, > > and actually we must remove them due to the side effect now: > > vmbus_process_offer -> hv_event_tasklet_enable -> tasklet_schedule will > > start the per-channel callback prematurely, cauing NULL dereferencing > > (the channel may haven't been properly configured to run the callback yet). > > > > Fixes: 631e63a9f346 ("vmbus: change to per channel tasklet") > > > > Signed-off-by: Dexuan Cui > > Cc: "K. Y. Srinivasan" > > Cc: Haiyang Zhang > > Cc: Stephen Hemminger > > Tested-by: Vitaly Kuznetsov > > This patch fixes the following crash on boot: > > [ 1.451648] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004 > [ 1.452255] IP: netvsc_channel_cb+0x90/0x7b0 [hv_netvsc] > [ 1.452255] PGD 0 > [ 1.452255] > [ 1.452255] Oops: 0000 [#1] SMP > [ 1.452255] Modules linked in: hv_storvsc hv_netvsc(+) scsi_transport_fc hyperv_fb hyperv_keyboard hid_hyperv hv_vmbus > [ 1.452255] CPU: 1 PID: 19 Comm: ksoftirqd/1 Not tainted 4.10.0_test+ #911 > [ 1.452255] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v1.0 11/26/2012 > [ 1.452255] task: ffff880007fd2b00 task.stack: ffffc90000e34000 > [ 1.452255] RIP: 0010:netvsc_channel_cb+0x90/0x7b0 [hv_netvsc] > ... > [ 1.452255] Call Trace: > [ 1.452255] vmbus_on_event+0x22/0x90 [hv_vmbus] > [ 1.452255] tasklet_action+0x5e/0x110 > [ 1.452255] __do_softirq+0x104/0x2af > [ 1.452255] run_ksoftirqd+0x29/0x40 > ... > [ 1.548068] RIP: netvsc_channel_cb+0x90/0x7b0 [hv_netvsc] RSP: ffffc90000e37d88 > [ 1.548068] CR2: 0000000000000004 > [ 1.548068] ---[ end trace 601fd9d6588b21e5 ]--- > [ 1.548068] Kernel panic - not syncing: Fatal exception in interrupt > [ 1.548068] Kernel Offset: disabled > [ 1.548068] ---[ end Kernel panic - not syncing: Fatal exception in interrupt > [ 1.572155] ------------[ cut here ]------------ > > The crash is not imminent but it happens pretty often on boot, I think > we need to push it to 4.11. > Agree that this needs to be in 4.11, but when NAPI is merged it also will not happen.