Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751227AbdIRXTC (ORCPT ); Mon, 18 Sep 2017 19:19:02 -0400 Received: from mail-sn1nam01on0130.outbound.protection.outlook.com ([104.47.32.130]:2833 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750714AbdIRXTA (ORCPT ); Mon, 18 Sep 2017 19:19:00 -0400 From: KY Srinivasan To: Vitaly Kuznetsov CC: Stephen Hemminger , "leann.ogasawara@canonical.com" , "Stephen Hemminger" , "apw@canonical.com" , "olaf@aepfle.de" , "marcelo.cerri@canonical.com" , "gregkh@linuxfoundation.org" , Haiyang Zhang , "linux-kernel@vger.kernel.org" , "jasowang@redhat.com" , "devel@linuxdriverproject.org" Subject: RE: [PATCH 1/1] Drivers: hv: vmbus: Fix rescind handling issues Thread-Topic: [PATCH 1/1] Drivers: hv: vmbus: Fix rescind handling issues Thread-Index: AQHTEsP6EQj97f9g/kmd2IKbCH40Y6KULpgAgCH80L+AAAHBf4AARN1wgABC3ACAA9aj0YAASbZngACuA5A= Date: Mon, 18 Sep 2017 23:18:57 +0000 Message-ID: References: <1502471039-5281-1-git-send-email-kys@exchange.microsoft.com> <20170824154102.62a02190@xeon-e3> <87ingkulhp.fsf@vitty.brq.redhat.com> <87efr8ul71.fsf@vitty.brq.redhat.com> <87vakgtnl2.fsf@vitty.brq.redhat.com> <87wp4wrwsx.fsf@vitty.brq.redhat.com> In-Reply-To: <87wp4wrwsx.fsf@vitty.brq.redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Ref=https://api.informationprotection.azure.com/api/72f988bf-86f1-41af-91ab-2d7cd011db47; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Owner=kys@microsoft.com; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2017-09-18T16:18:55.4240088-07:00; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Application=Microsoft Azure Information Protection; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Extended_MSFT_Method=Automatic; Sensitivity=General x-originating-ip: [2001:4898:80e8:6::461] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DM5PR21MB0508;6:LfuXbzhLx6UHfs2Kuz8UqSe8kOC4IkV8mupuuTUL7H45lSmYtjFW+XpFG1voIkw0A3woNKjw+mRsUA1bN2rLeyZCMgmPC2W7dvw7iA8JC5wR7Ie/h2+z3Jh62utOdKoecrhDr45duYEGvlod1LXtxFnvRTSnnkYHBq+9Q4KOnn2WiL3bFxtyGanmtbIVoHiDLeLPI3oQi5UFrPft6H2W0Gd0rxvx9LYtzC5PrRpWc57L27ocRBpLwhhoTYHhl66p2jxcYtme+/uzwDif7XXS+0OBWqLuZN2QJFT8ySkMxW9t0uv+Coj5HQY+jzD6zlgnUtoojZY62KH9f9r/O/p/XQ==;5:B3kDKEUfekdajNFQEk0d3MIc/IuG21LEB2Pnyrdam/okYpvCu4YXW99yPpR0kWOyK+rU0GcEh8SuW5FUVq4TYtU8QPJfG67hdO+CYPiNmloEVOIzai5OrbjuOPK86NOHwQo5vCALVQI6Ty/Xroe7NA==;24:dw0dK9/59XdG6kvnTSW44B4K/0l+K1AWvDZPXMqlyoj9AJTgBoDF/tf+Ay3060TXDr+8j+nO2STFkxCzlQjGQi3ZCAxmBWcGRo7OmhrS5v8=;7:uVpDtVvbVGom6TFBMBM2sxZ3GH3ByoB8yYjFyiKe3KkoymVdgzlSqCluyynccJW1hBIDJkGhBfbbS4+TtLtajNkAy+9R9L1JUW0c4di85+8+AoGjEZngzJQz4Xb3zRymTk2dKRwM2W8E2r3ei5FYGDiOHArgV4Kl4p7NTfU/c+/fA5TslQ7+yaJYDWmTLZhIIMSouzvPncwDZrchalTyWMtvqFej9OWZz/1blTQvDso= x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 88ae1b3b-0e67-44da-ad54-08d4feeba32b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(48565401081)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:DM5PR21MB0508; x-ms-traffictypediagnostic: DM5PR21MB0508: authentication-results: spf=none (sender IP is ) smtp.mailfrom=kys@microsoft.com; x-exchange-antispam-report-test: UriScan:(89211679590171)(9452136761055)(198206253151910); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(61425038)(6040450)(2401047)(5005006)(8121501046)(93006095)(93001095)(100000703101)(100105400095)(10201501046)(3002001)(6055026)(61426038)(61427038)(6041248)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123555025)(20161123560025)(20161123558100)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM5PR21MB0508;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM5PR21MB0508; x-forefront-prvs: 04347F8039 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(346002)(39860400002)(376002)(47760400005)(51234002)(13464003)(377454003)(189002)(199003)(14454004)(106356001)(25786009)(229853002)(105586002)(77096006)(189998001)(6506006)(6436002)(9686003)(316002)(86612001)(8936002)(53936002)(97736004)(8990500004)(93886005)(22452003)(575784001)(86362001)(10090500001)(8676002)(7416002)(53546010)(7736002)(478600001)(76176999)(54356999)(2900100001)(3280700002)(81156014)(10290500003)(2906002)(81166006)(50986999)(3660700001)(6116002)(99286003)(6246003)(6916009)(2950100002)(55016002)(102836003)(54906002)(4326008)(101416001)(33656002)(68736007)(7696004)(74316002)(305945005)(5660300001);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR21MB0508;H:DM5PR21MB0476.namprd21.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Sep 2017 23:18:57.2682 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR21MB0508 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by nfs id v8INJC0b000828 Content-Length: 4068 Lines: 96 > -----Original Message----- > From: Vitaly Kuznetsov [mailto:vkuznets@redhat.com] > Sent: Monday, September 18, 2017 5:55 AM > To: KY Srinivasan > Cc: Stephen Hemminger ; > leann.ogasawara@canonical.com; Stephen Hemminger > ; apw@canonical.com; olaf@aepfle.de; > marcelo.cerri@canonical.com; gregkh@linuxfoundation.org; Haiyang Zhang > ; linux-kernel@vger.kernel.org; > jasowang@redhat.com; devel@linuxdriverproject.org > Subject: Re: [PATCH 1/1] Drivers: hv: vmbus: Fix rescind handling issues > > Vitaly Kuznetsov writes: > > > > > Reverting 6f3d791f300618caf82a2be0c27456edd76d5164 still helps. > > In addition to the above I got the following crash while playing with > 4.14-rc1 (unmodified): > > [ 55.810080] kernel tried to execute NX-protected page - exploit attempt? > (uid: 0) > [ 55.814293] BUG: unable to handle kernel paging request at > ffff8800059985f0 > [ 55.818065] IP: 0xffff8800059985f0 > [ 55.819925] PGD 22eb067 P4D 22eb067 PUD 22ec067 PMD 5f37063 PTE > 8000000005998163 > [ 55.820018] Oops: 0011 [#1] SMP > [ 55.820018] Modules linked in: vfat fat bnx2x mdio efi_pstore hv_utils > efivars pci_hyperv ptp pps_core pcspkr hv_balloon xfs libcrc32c hv_storvsc > hyperv_fb hv_netvsc scsi_transport_fc hid_hyperv hyperv_keyboard > hv_vmbus > [ 55.834837] CPU: 0 PID: 498 Comm: kworker/0:2 Not tainted 4.14.0-rc1 #63 > [ 55.834837] Hardware name: Microsoft Corporation Virtual Machine/Virtual > Machine, BIOS Hyper-V UEFI Release v1.0 11/26/2012 > [ 55.834837] Workqueue: events vmbus_onmessage_work [hv_vmbus] > [ 55.834837] task: ffff88003f448000 task.stack: ffffc90005398000 > [ 55.834837] RIP: 0010:0xffff8800059985f0 > [ 55.834837] RSP: 0018:ffffc9000539be00 EFLAGS: 00010286 > [ 55.834837] RAX: ffff880005998010 RBX: ffff880005998000 RCX: > 0000000000000000 > [ 55.834837] RDX: ffff8800059985f0 RSI: 0000000000000246 RDI: > ffff880005998000 > [ 55.860040] RBP: ffffc9000539be18 R08: 00000000000002e6 R09: > 0000000000000000 > [ 55.865057] R10: ffffc9000539bdf0 R11: 000000000000a000 R12: > 0000000000000286 > [ 55.865057] R13: ffff88007ae1ed00 R14: 0000000000000000 R15: > ffff8800065c3200 > [ 55.865057] FS: 0000000000000000(0000) GS:ffff88007ae00000(0000) > knlGS:0000000000000000 > [ 55.865057] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 55.865057] CR2: ffff8800059985f0 CR3: 00000000075a5000 CR4: > 00000000001406f0 > [ 55.886745] Call Trace: > [ 55.886745] ? vmbus_onoffer_rescind+0xfa/0x160 [hv_vmbus] > [ 55.890968] vmbus_onmessage+0x2a/0x90 [hv_vmbus] > [ 55.891934] vmbus_onmessage_work+0x1d/0x30 [hv_vmbus] > [ 55.891934] process_one_work+0x193/0x390 > [ 55.891934] worker_thread+0x48/0x3c0 > [ 55.891934] kthread+0x120/0x140 > [ 55.891934] ? process_one_work+0x390/0x390 > [ 55.891934] ? kthread_create_on_node+0x60/0x60 > [ 55.891934] ret_from_fork+0x25/0x30 > [ 55.891934] Code: 88 ff ff c0 85 99 05 00 88 ff ff d0 85 99 05 00 88 ff ff d0 85 99 > 05 00 88 ff ff e0 85 99 05 00 88 ff ff e0 85 99 05 00 88 ff ff 85 99 05 00 88 ff > ff f0 85 99 05 00 88 ff ff 00 86 99 05 00 > [ 55.922505] RIP: 0xffff8800059985f0 RSP: ffffc9000539be00 > [ 55.922505] CR2: ffff8800059985f0 > [ 55.922505] ---[ end trace 25226e00af3f94fb ]--- > [ 55.933590] Kernel panic - not syncing: Fatal exception > [ 55.933590] Kernel Offset: disabled > [ 55.933590] ---[ end Kernel panic - not syncing: Fatal exception > > So it seems that during > > while (READ_ONCE(channel->probe_done) == false) { > /* > * We wait here until any channel offer is currently > * being processed. > */ > msleep(1); > } > > loop the channel disappeared. The issue may not be related to the netvsc > hang I mentioned before. It may make sense to do refcounting for > channels/subchannels (or employ RCU). I will work on this issue and get you a patch to try. K. Y > > -- > Vitaly