Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754044AbaANA3B (ORCPT ); Mon, 13 Jan 2014 19:29:01 -0500 Received: from mail.linuxfoundation.org ([140.211.169.12]:54428 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753947AbaANA2p (ORCPT ); Mon, 13 Jan 2014 19:28:45 -0500 From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Yaju Cao , "K. Y. Srinivasan" , Haiyang Zhang , Jason Wang , "David S. Miller" Subject: [PATCH 3.12 19/77] netvsc: dont flush peers notifying work during setting mtu Date: Mon, 13 Jan 2014 16:27:40 -0800 Message-Id: <20140114002753.046760713@linuxfoundation.org> X-Mailer: git-send-email 1.8.5.1.163.gd7aced9 In-Reply-To: <20140114002752.497010554@linuxfoundation.org> References: <20140114002752.497010554@linuxfoundation.org> User-Agent: quilt/0.61-1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 3.12-stable review patch. If anyone has any objections, please let me know. ------------------ From: Jason Wang [ Upstream commit 50dc875f2e6e2e04aed3b3033eb0ac99192d6d02 ] There's a possible deadlock if we flush the peers notifying work during setting mtu: [ 22.991149] ====================================================== [ 22.991173] [ INFO: possible circular locking dependency detected ] [ 22.991198] 3.10.0-54.0.1.el7.x86_64.debug #1 Not tainted [ 22.991219] ------------------------------------------------------- [ 22.991243] ip/974 is trying to acquire lock: [ 22.991261] ((&(&net_device_ctx->dwork)->work)){+.+.+.}, at: [] flush_work+0x5/0x2e0 [ 22.991307] but task is already holding lock: [ 22.991330] (rtnl_mutex){+.+.+.}, at: [] rtnetlink_rcv+0x1b/0x40 [ 22.991367] which lock already depends on the new lock. [ 22.991398] the existing dependency chain (in reverse order) is: [ 22.991426] -> #1 (rtnl_mutex){+.+.+.}: [ 22.991449] [] __lock_acquire+0xb19/0x1260 [ 22.991477] [] lock_acquire+0xa2/0x1f0 [ 22.991501] [] mutex_lock_nested+0x89/0x4f0 [ 22.991529] [] rtnl_lock+0x17/0x20 [ 22.991552] [] netdev_notify_peers+0x12/0x30 [ 22.991579] [] netvsc_send_garp+0x22/0x30 [hv_netvsc] [ 22.991610] [] process_one_work+0x211/0x6e0 [ 22.991637] [] worker_thread+0x11b/0x3a0 [ 22.991663] [] kthread+0xed/0x100 [ 22.991686] [] ret_from_fork+0x7c/0xb0 [ 22.991715] -> #0 ((&(&net_device_ctx->dwork)->work)){+.+.+.}: [ 22.991715] [] check_prevs_add+0x967/0x970 [ 22.991715] [] __lock_acquire+0xb19/0x1260 [ 22.991715] [] lock_acquire+0xa2/0x1f0 [ 22.991715] [] flush_work+0x4e/0x2e0 [ 22.991715] [] __cancel_work_timer+0x95/0x130 [ 22.991715] [] cancel_delayed_work_sync+0x13/0x20 [ 22.991715] [] netvsc_change_mtu+0x84/0x200 [hv_netvsc] [ 22.991715] [] dev_set_mtu+0x34/0x80 [ 22.991715] [] do_setlink+0x23a/0xa00 [ 22.991715] [] rtnl_newlink+0x394/0x5e0 [ 22.991715] [] rtnetlink_rcv_msg+0x9c/0x260 [ 22.991715] [] netlink_rcv_skb+0xa9/0xc0 [ 22.991715] [] rtnetlink_rcv+0x2a/0x40 [ 22.991715] [] netlink_unicast+0xdd/0x190 [ 22.991715] [] netlink_sendmsg+0x337/0x750 [ 22.991715] [] sock_sendmsg+0x99/0xd0 [ 22.991715] [] ___sys_sendmsg+0x39e/0x3b0 [ 22.991715] [] __sys_sendmsg+0x42/0x80 [ 22.991715] [] SyS_sendmsg+0x12/0x20 [ 22.991715] [] system_call_fastpath+0x16/0x1b This is because we hold the rtnl_lock() before ndo_change_mtu() and try to flush the work in netvsc_change_mtu(), in the mean time, netdev_notify_peers() may be called from worker and also trying to hold the rtnl_lock. This will lead the flush won't succeed forever. Solve this by not canceling and flushing the work, this is safe because the transmission done by NETDEV_NOTIFY_PEERS was synchronized with the netif_tx_disable() called by netvsc_change_mtu(). Reported-by: Yaju Cao Tested-by: Yaju Cao Cc: K. Y. Srinivasan Cc: Haiyang Zhang Signed-off-by: Jason Wang Acked-by: Haiyang Zhang Signed-off-by: David S. Miller Signed-off-by: Greg Kroah-Hartman --- drivers/net/hyperv/netvsc_drv.c | 1 - 1 file changed, 1 deletion(-) --- a/drivers/net/hyperv/netvsc_drv.c +++ b/drivers/net/hyperv/netvsc_drv.c @@ -327,7 +327,6 @@ static int netvsc_change_mtu(struct net_ return -EINVAL; nvdev->start_remove = true; - cancel_delayed_work_sync(&ndevctx->dwork); cancel_work_sync(&ndevctx->work); netif_tx_disable(ndev); rndis_filter_device_remove(hdev); -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/