Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752546AbdLFQK6 (ORCPT ); Wed, 6 Dec 2017 11:10:58 -0500 Received: from us-smtp-delivery-194.mimecast.com ([216.205.24.194]:24211 "EHLO us-smtp-delivery-194.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752127AbdLFQK4 (ORCPT ); Wed, 6 Dec 2017 11:10:56 -0500 From: Trond Myklebust To: "anna.schumaker@netapp.com" , "geert@linux-m68k.org" CC: "linux-kernel@vger.kernel.org" , "linux-renesas-soc@vger.kernel.org" , "linux-nfs@vger.kernel.org" , "me@tobin.cc" Subject: Re: NFS crash, hashed pointers in backtrace Thread-Topic: NFS crash, hashed pointers in backtrace Thread-Index: AQHTbeJ9J10QtANYT0eTxPNCQnRpBKM2Yh4AgAAbqwA= Date: Wed, 6 Dec 2017 16:10:50 +0000 Message-ID: <1512576648.26816.3.camel@primarydata.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [68.49.162.121] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DM5PR11MB0074;20:z/s52emkcsTZ4p/yb85EO8KzsvQROEHCyhsuTsPFy3wXGY7KHiIuqTfMdWrfeMD3Klo4+TkJrT/Q5sJ8pMU69vLVvDpXz6g9xsCIC3/6NlreAXVpugDydXt9jZy7xWKPcjC0ovIRyCY1DoP1rAcVFZw2DIuFjnmfEWN4vZwifPM= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: fad5962f-7798-4232-b893-08d53cc3eb0d x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(4534020)(4602075)(4603075)(4627115)(201702281549075)(5600026)(4604075)(2017052603286);SRVR:DM5PR11MB0074; x-ms-traffictypediagnostic: DM5PR11MB0074: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040450)(2401047)(5005006)(8121501046)(3231022)(10201501046)(93006095)(93001095)(3002001)(6041248)(20161123562025)(20161123564025)(20161123558100)(20161123560025)(2016111802025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6043046)(6072148)(201708071742011);SRVR:DM5PR11MB0074;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:DM5PR11MB0074; x-forefront-prvs: 05134F8B4F x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(366004)(39830400002)(376002)(346002)(24454002)(189003)(377424004)(199004)(51234002)(86362001)(5660300001)(575784001)(4326008)(76176011)(81166006)(2950100002)(81156014)(8676002)(53936002)(6512007)(6246003)(6436002)(316002)(68736007)(53546010)(8936002)(110136005)(54906003)(305945005)(7736002)(6116002)(102836003)(3846002)(45080400002)(103116003)(3660700001)(478600001)(14454004)(3280700002)(106356001)(97736004)(33646002)(99286004)(25786009)(101416001)(105586002)(2900100001)(66066001)(2501003)(77096006)(6486002)(2906002)(6506006)(229853002)(36756003);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR11MB0074;H:DM5PR11MB0075.namprd11.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-ID: MIME-Version: 1.0 X-OriginatorOrg: primarydata.com X-MS-Exchange-CrossTenant-Network-Message-Id: fad5962f-7798-4232-b893-08d53cc3eb0d X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Dec 2017 16:10:50.1689 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 03193ed6-8726-4bb3-a832-18ab0d28adb7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR11MB0074 X-MC-Unique: yuYcVRj0NPuvxLiccNnATg-1 Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id vB6GB208001539 Content-Length: 6132 Lines: 156 Hi Geert, On Wed, 2017-12-06 at 15:31 +0100, Geert Uytterhoeven wrote: > Hi Trond. Anna, > > On Tue, Dec 5, 2017 at 5:02 PM, Geert Uytterhoeven org> wrote: > > During a failed write to a virtual sysfs file (root fs is NFS), I > > got: > > > > Unable to handle kernel NULL pointer dereference at virtual address > > 00000020 > > pgd = c448bb15 > > [00000020] *pgd=69c9c003, *pmd=69d55003, *pte=00000000 > > Internal error: Oops: 207 [#1] SMP ARM > > Modules linked in: > > CPU: 0 PID: 1230 Comm: rs:main Q:Reg Not tainted > > 4.15.0-rc2-koelsch-01160-gd389a154c640caab-dirty #3752 > > Hardware name: Generic R-Car Gen2 (Flattened Device Tree) > > task: 4a3bb6d2 task.stack: fd0c00bd > > PC is at nfs_flush_incompatible+0x54/0xf8 > > Got another nfsroot crash: > > Unable to handle kernel NULL pointer dereference at virtual address > 00000030 > pgd = 329e8f6e > [00000030] *pgd=80000040004003, *pmd=00000000 > Internal error: Oops: 206 [#1] SMP ARM > Modules linked in: > CPU: 0 PID: 101 Comm: kworker/u4:1 Not tainted > 4.15.0-rc2-koelsch-01166-g047d7d3248e08fc7-dirty #3762 > Hardware name: Generic R-Car Gen2 (Flattened Device Tree) > Workqueue: writeback wb_workfn (flush-0:15) > task: 8a5bf858 task.stack: e93c92bc > PC is at nfs_page_async_flush+0x110/0x244 > LR is at 0x10 > pc : [] lr : [<00000010>] psr: 400f0013 > sp : eaff9c98 ip : c0c5092b fp : 00000005 > r10: 00018e84 r9 : ebef92c0 r8 : eaff9d64 > r7 : ea421a00 r6 : ebef92c0 r5 : ea999040 r4 : ea9b1a00 > r3 : 00000000 r2 : 00000006 r1 : 00000000 r0 : 00000000 > Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user > Control: 30c5387d Table: 69d65680 DAC: fffffffd > Process kworker/u4:1 (pid: 101, stack limit = 0xeaff8210) > Stack: (0xeaff9c98 to 0xeaffa000) > 9c80: ebef92c0 > eaff9d64 > 9ca0: eaff9e20 ea421afc 00000000 c03bc858 eaff9e20 00000000 ffffffff > c02b11e8 > 9cc0: 00000000 ea8f4500 eb427328 00018e89 00000000 00000009 eaff9d0c > 00000000 > 9ce0: c03bc830 eaff9d64 00000000 ffffffff 00000009 00000000 ebef8440 > ebef45c0 > 9d00: ebf1abc0 ebef8860 ebef8420 ebef92c0 ebef5ce0 ebef7e80 ebef3cc0 > eaff9d1c > 9d20: eaff9d1c eb1d2d98 eb1d2d28 ea421a00 eb400700 ea421a00 eab89bc0 > ea421afc > 9d40: eaff9e20 ea421afc 00000002 ea421a50 eaff8000 c03bc94c c081590c > c02483d8 > 9d60: eaa62140 00000001 ea421a00 c08157cc c08158e0 00000000 00000000 > c08157bc > 9d80: c081590c 00000000 eab89bc0 00000000 00001000 00000001 eaff9d9c > ea999fc0 > 9da0: ea999fc0 00004000 00001000 00001000 00000000 c0745704 00000000 > 00000000 > 9dc0: ec09e250 eaff9e20 ea421afc eaff9e20 ea9c4c38 c02b2d48 00000086 > ea421a00 > 9de0: ea421a00 c0310434 ea421a00 eaff9e20 00000000 ea421ab4 ea421a00 > 00001400 > 9e00: ea9c4c38 eaff9efc 00000002 c03109b8 ea9c4c64 00003fd0 ea98b800 > 00000000 > 9e20: 000013fb 00000000 00000000 00000000 ffffffff 7fffffff 00000000 > 00000011 > 9e40: 00000000 ea9c4c38 00000000 c0e04900 00003fda eaff9efc ea9c4c4c > ea98b800 > 9e60: eb1f7584 c0310be0 ea9c4c4c ea9c4c38 eaff9efc c0e04900 ea9c4c64 > 0000175c > 9e80: ea9c4d90 c0e13020 0000000a c0310d2c 00003fd0 00003fd0 eb465198 > 00003418 > 9ea0: eaff9ea0 eaff9ea0 eaff9ea8 eaff9ea8 eaff9eb0 eaff9eb0 0000001a > ea9c4d98 > 9ec0: ea9c4c38 0000175c ea9c4d90 ea9c4c3c ea9c4d80 00000000 00000088 > c03110a0 > 9ee0: 00000000 c023b924 eb9a0d80 eafd7100 eb465100 eabe8000 00000000 > 0000175c > 9f00: 00000000 eaff9e9c 00000000 00000006 00000003 00000000 00000000 > 00000000 > 9f20: eb7f6200 ea9c4d98 eb406600 00000000 eb407f00 00000000 ea9c4d9c > c0235bdc > 9f40: eb7f6200 ea9c4d98 eb7f6200 eb406600 eb406600 eaff8000 eb406624 > c0e04900 > 9f60: eb7f6218 c023634c eafd7100 eb7f6380 eb7a7fc0 00000000 eb443ee4 > eb7f63a8 > 9f80: eb7f6200 c0236080 00000000 c023a528 eb7a7fc0 c023a40c 00000000 > 00000000 > 9fa0: 00000000 00000000 00000000 c0206f38 00000000 00000000 00000000 > 00000000 > 9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 > 00000000 > 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 > 00000000 > [] (nfs_page_async_flush) from [] > (nfs_writepages_callback+0x28/0x54) > [] (nfs_writepages_callback) from [] > (write_cache_pages+0x278/0x364) > [] (write_cache_pages) from [] > (nfs_writepages+0xa8/0xe8) > [] (nfs_writepages) from [] > (do_writepages+0x34/0x80) > [] (do_writepages) from [] > (__writeback_single_inode+0x34/0x194) > [] (__writeback_single_inode) from [] > (writeback_sb_inodes+0x1cc/0x390) > [] (writeback_sb_inodes) from [] > (__writeback_inodes_wb+0x64/0xa0) > [] (__writeback_inodes_wb) from [] > (wb_writeback+0x110/0x18c) > [] (wb_writeback) from [] (wb_workfn+0x1b8/0x304) > [] (wb_workfn) from [] > (process_one_work+0x1cc/0x31c) > [] (process_one_work) from [] > (worker_thread+0x2cc/0x408) > [] (worker_thread) from [] (kthread+0x11c/0x13c) > [] (kthread) from [] (ret_from_fork+0x14/0x3c) > Code: e3a02001 e5c32004 ebf98e95 e595300c (e5930030) > ---[ end trace 2771b70506a823a3 ]--- > > static int nfs_page_async_flush(struct nfs_pageio_descriptor *pgio, > struct page *page) > { > struct nfs_page *req; > int ret = 0; > > ... > > /* If there is a fatal error that covers this write, just > exit */ > if (nfs_error_is_fatal_on_server(req->wb_context->error)) > goto out_launder; > > c03bc644: e595300c ldr r3, [r5, #12] > c03bc648: e5930030 ldr r0, [r3, #48] ; 0x30 > c03bc64c: ebfffd1b bl c03bbac0 > > > req->wb_context must be NULL. > I'm confused. If your test involves only writing to a sysfs file, then why is the NFS code involved at all? Could this be a use-after-free? -- Trond Myklebust Linux NFS client maintainer, PrimaryData trond.myklebust@primarydata.com