Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754004AbcJKQZq (ORCPT ); Tue, 11 Oct 2016 12:25:46 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:50633 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753943AbcJKQZn (ORCPT ); Tue, 11 Oct 2016 12:25:43 -0400 Subject: Re: btrfs bio linked list corruption. To: Dave Jones , Al Viro , Josef Bacik , David Sterba , , Linux Kernel References: <20161011144507.okg6baqvodn2m2lh@codemonkey.org.uk> From: Chris Mason Message-ID: Date: Tue, 11 Oct 2016 11:54:09 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20161011144507.okg6baqvodn2m2lh@codemonkey.org.uk> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [2620:10d:c091:180::ee4c] X-ClientProxiedBy: DM5PR19CA0020.namprd19.prod.outlook.com (10.175.226.158) To DM5PR15MB1242.namprd15.prod.outlook.com (10.173.209.136) X-MS-Office365-Filtering-Correlation-Id: fd1c129b-1ff0-4241-6a89-08d3f1eed988 X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1242;2:QEb3om1rsDMOVIL4GSTGfdJqsBPlv4EodI01TPKy7AjeuvsgrkmlCHZdeWW1QHuUc8tC537LBkh5ZS7QUWilHrn8wP1O0IvxCSaYoi0r548AJlclrRvYZApk36C7A3lT5QmIBf9TAHHFMvnR8lK9+kDiBW8HglQplLldH2/Hdd6OXeb2H5WGmY9ljM49akDK7HnQS9Na7m0u9e1xJPvsSQ==;3:nluBdQrIX/7mTSQfFp+2zsKtZPAckRT1Odjw3nzGDB/AR8GuZUuVCveLUG6EBUjmJPs4ExyLkbhySQ1g5KDaMpgDYmzWC19l5uKrGGNqJZvZg1IluwkjWht5TCtL/MlYp8aOj4QwwGQql0jSu4vrHQ== X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM5PR15MB1242; X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1242;25:y+bajctp3nI9VHNT3YLxjlpIFDVbrfrD3aBz1MqVu0Rak4BgRpiM3VWg3Y473H3KCo/7fTz2Hn04eH3EoYfx3Vtx9Xi1OY7Bfcy1/Edzg5oltMx38++yF3NTSsduaa78BELECmlAeVKMcENAebt0PCDS27yUCt/0DK1cBvXuk5Hwb1R5ms9S/KVR6yW+s9Yf5okoMY3fGEYARX+eoeP1WG/QdwAiA1kvaSxraPtcfmoDdvh1XfMQM6L9oS5Mv8fKg2i4Dl66xG42kC47PFmKkdP6EtwZj6SoQkcSDSy9hTqF7FGBquvV2FSWSF2wZ3dP5YpZJsSw4wAISz4IxWwCwKqJ5SVVs0EDe1fxLFHATivenyLyMnOjn6hCc7WmjWLwD7FTxdj2c3M8KljD3QurQIAMXte0BA62heudaiwnIZ1APJrC5s9BU/D2w8+y7BU2jYAdd1Ws+9nwOtOxra5xE9GDjFQaRFmqA0tjr7IoYAkCBxqSklt3yv7qQ7YCUUWbSlns8uJFs4VgI+Dyn+s7hm16bD8AiBXsxUCIQNipdX2LRf3/+vQFsdyNDw4wx1hyB1XnujZzi09urLlO25cJ1csjbN9FSG7ZkN6kGzwRe1XIEUa+n20VNeJ5gJ3IxbdzUZ9zwr1YkQ1oz1CAWru/BsQlDJ+7gljPJ0dCZUSXrxHnp4hRxlC4MCZf2CFq0xFxnHCGSbwtPFuzmWeolfijqvYTi/u3OBvL2Ix695qlqm4= X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1242;31:9uOWupOnGWrESfBEhjA2QGa6a3hCs9zOVVROivzDSm0XugkTiqMD1Qf9w/yKCncpK4r/LLIbxvK1+MAuAp1ZyLnV1W6ZCXxFzZtU1ykDao24RQQx2Gb5qppQMzofjg5Itsv9qKyVES3eALj+kX6TpzZjeVX18/PZ0uVsE5OHFeznRESc38TbOTBpjmQWywaWPXsw11rP5npkVzpXlpGOAmLdOzMqfIDfFJPMyjog/XYqDXVapyR4/Ldf8plTDB9mGRllPZvbkbUIglcPtmNgzw==;20:IPTJEQE76ObKxoSJON3XKAB2SzmWnr/mrMf+U9W4NoucYuQPILSvUVTeg+U46bAcEJc/j7m/eu3x7P2bcUHri3QQaG/MvEw4rWYIkFjodiDljSIBIF1HOoZxp8IpAv9qPBpN7cmlXKk8TGtLLYI+YV0hA43+9i4CJSr3CDlGlcI=;4:+fnLN5X0mor8CnHeRDcE75/Zv7yVgVvGUwr8b6/XzLnHLJ5SzvuzeErg9MzfIDgyYdVOzrf+sH579pNaRbmBi2OjZvRLtYiWNGl6DShgeAOomqekX9KuoxU0Q1VKoz8m7nRcbVLl/gRdorHoXVL6sB2eJtUgYpU6AR6ZI9aMVpgBa9PUZeDSTLJwTrxpxr36aKlL12domcVfsZkZZ9NJdM8kb7VvtymnEcZIMwAKh+frn4LK/OxnXM0kiutdAEG9gZEK6UZf3pvdbr+SstkMoTsvTGS+pc5lGNYr7wQLTRBtqelwy31oWhY2CXvzd+bRiezQ9jCZkDohAEQRI7+ZcYQbt6vwTuc1rGJXU1j9S+6BP7Wx2Sb+TziEIdpCSoZboXmdiDQwIw9/DtgtdLKo/w== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:DM5PR15MB1242;BCL:0;PCL:0;RULEID:;SRVR:DM5PR15MB1242; X-Forefront-PRVS: 00922518D8 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(7916002)(199003)(24454002)(377454003)(189002)(65956001)(65806001)(189998001)(50466002)(106356001)(47776003)(5001770100001)(19580395003)(105586002)(97736004)(4001350100001)(101416001)(31686004)(54356999)(65826007)(76176999)(92566002)(36756003)(42186005)(23746002)(33646002)(107886002)(64126003)(50986999)(2950100002)(6666003)(77096005)(81156014)(81166006)(305945005)(8676002)(31696002)(2906002)(7846002)(83506001)(7736002)(5660300001)(230700001)(586003)(1706002)(86362001)(6116002)(68736007)(3826002)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR15MB1242;H:[IPv6:2620:10d:c0a1:1110::1023];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;DM5PR15MB1242;23:EaboWEjtcXtGd7IM4EZ22FxoFe2tru+DyqXCl?= =?Windows-1252?Q?3569RkrEDq+ENp9cP7+dncweFCYWPYn/BoOyramhL7UWhe1CG9Xpo043?= =?Windows-1252?Q?U06BIGruasTus5JLdQnrwYI7ETU5IRc5ECVUlkBGlM5NSnsYn+kwpVqb?= =?Windows-1252?Q?tpjUQ0luX1ti7BIrXdIYpHOFmGAFGn9trh9zx2S3JQciHIX2hGAu7fJM?= =?Windows-1252?Q?rY6pANlV3lEAPWcEODhOF0f844+OATWAj8MVP7wI+0x453hWT5FyKgmg?= =?Windows-1252?Q?fCldtti3kHXraLon4ifbRpkfe8ZvUTf/bms4ZomNt1dJxICmizhOQ4cJ?= =?Windows-1252?Q?x7x6IA4O/l3LNSSO20orVi9wQbRioEK0uY+6tbnrP5J+KrWDRU+vRV3K?= =?Windows-1252?Q?/rJpTpjNSNVxrix4UiVuxST2dedc9P0hXTJyNYUaNxt6ysLyPAlX5X6m?= =?Windows-1252?Q?6iutEY8CKntdXIbFCov5q6Ref309KdIsTGN+RMeKAi6R0Fhd8eJS0YVi?= =?Windows-1252?Q?pxQRtokNW9WARcv/bhhPsJ6apnygvUCfP0dafhaEh5c8C1NKZyQhMa2N?= =?Windows-1252?Q?FhtExpEHgsdeDCkHA5j+sdccpYWoop8jybEz70t1ZvDuQ4EEmrjiUWiw?= =?Windows-1252?Q?4ujbJ33T6MxD+YduVPzOd1zN9ifENTQYBdZ+wcArLRUVuBdUPq9E8N3C?= =?Windows-1252?Q?bWRKDlUhC6iqjsyFl9LqcjRUevbG7ENIjQmfAaVQop3sc6TFsF7/nRGL?= =?Windows-1252?Q?YVPcNa1YuAMknP2188QfDqH8Rj6RBVKlEBzeuWk/p7oJAA+DtAxk34rk?= =?Windows-1252?Q?VL+pdiG1Bpy3+10zbjeJLj7Q6S6dbAH74AElmxZXi7Wd6ya6EoYffXru?= =?Windows-1252?Q?QS2xPJ1R1O1aT2fD3iNzar8Ydzj93ahpcRuGxpcqcvAPdO/lqbu82BFS?= =?Windows-1252?Q?G2va+g4uGy4cEy2COr/xqyDunnPbRO6R5IX3zW6Vtdu69tkNwf1dALlB?= =?Windows-1252?Q?Cj0z388F6iEsraIOekSame/mj050kL9MkzFgXp4rtGlkUkl6Kx2J8Gsy?= =?Windows-1252?Q?5+dazAuaOy7w7XYmIgCeE0NJPdKbj9fdLg9rxKSIWWTzR8HQpYUjQ+lC?= =?Windows-1252?Q?Y1oPeqf7iVmbtUipop4gK8QJPEk66/Get0tC4HrZFjx3DylOq7l7LGvo?= =?Windows-1252?Q?0eGGV9OyKAdtZPCwQxULgoj3Ya0liFrTcvxdKdcjwlhZZG368Q/HBlt8?= =?Windows-1252?Q?tnLSQNjEcj1tCZ8Uyfu/EHPQ1dFuqGRUZp0qHADla8QRM+HcQtI4n7xt?= =?Windows-1252?Q?uoJ5OJeiZ2b+R8S1WCgPS2LDSQ4XEQQ/joGghVXHd3rPe6PJZm5SUH/n?= =?Windows-1252?Q?NDuIGctcubxEThnO85QcH7dUN56udOVmA=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1242;6:/6xC767L93EEOonHK4Mt9h5fTyQQNYF3hb1eDKKzhFwYmotn+/0eGxDakyOG2/vhGcRrE8nEryHoEo2gIkcmL3s7cdOVb7zsjIgSb7cz+2RTnoF2cJhfgCZZuAmRoAdYuztOpaZ7loJK7l05Gl07p+CN9X/JAUIz09ahbcD1jyzemesVGprGbbJVI4C4AdDF5kt+QRdFGRiQwFDYMQZJ1vWlA66Vwh2r8V+Mb2ONC3GcX1zztpagFg9Gi88AArxgMcZfpfkTudNBKDg/B+qN6gLFyoNvJAICdhw8AyBTklnGkgxk7CIA8MMNIT+EptCC;5:nnwYOPruNsybW8gKc/zejEzPeT9igBtQg+HGvhDCbka1MlO4B2AYQZaJBVNe/6WbS1fYL0ZAM6CmKe3ezPVw7dT/F9wXp/UsPxOfu731uCh4ep+oesptdEdx1yxUPh7nAPjsU89Je928i/PunvzCyg==;24:5lWihdCQhx1hggaX8kLsEw3+NFMvmVryhibL1lvxEXV6zvT3tgCDdAuSBQy4jckMMiVk+AvSw9p8EZ6/Vaf59arCKpPk3T1qorCHAPvP57g=;7:8jCB6PQRyCVg74j8q5jXug2s5VB7RrzgvCFhHFuEwVHTveCI38lPnF34OURP7kU8zxZZTgQ23ckh1oKTJr8hau84TEuNMMkUPSSrPPcetsqly0SOQ+MZQoiWIMVoCuk4kYq4tEJSMhJMrKgkYRQKONQDyTveZee0ZPWmoTsTS8MlhoLxIzC65wRn/q3fM2kJauisnKeqFvReFQ73HV0JMcRzXt7geYu0mpQHRJkK2RIcutMKtLGngdLS1e5D34rNJULGJvfmVjDDI/pALA7cNyR1Y+H1JwtQ/wgOf14q27PqJPosUOJ5Scj8FujJtkcq7+T+JVzV1tkLKhYEVCrF0Rdji8Y1NEQWgUCSINda4ws= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;DM5PR15MB1242;20:mRFx4rv5R3gjCsHLzOYG5E0suWCQltiiaXtKHGGt1iojT0rYAlMTBQX9cVOreGzhEpYl7rjeetLaThfUP4GegWlWMyHVxxFhuDQEw6cxWHobn/d1xy7H/EK8x8LrQbp+Jed6TJB7iFzEjHY07o54irC9mzvzm9HuajbmnqAs2/c= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Oct 2016 15:54:13.4941 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1242 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-11_09:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2516 Lines: 43 On 10/11/2016 10:45 AM, Dave Jones wrote: > This is from Linus' current tree, with Al's iovec fixups on top. > > ------------[ cut here ]------------ > WARNING: CPU: 1 PID: 3673 at lib/list_debug.c:33 __list_add+0x89/0xb0 > list_add corruption. prev->next should be next (ffffe8ffff806648), but was ffffc9000067fcd8. (prev=ffff880503878b80). > CPU: 1 PID: 3673 Comm: trinity-c0 Not tainted 4.8.0-think+ #13 > ffffc90000d87458 ffffffff8d32007c ffffc90000d874a8 0000000000000000 > ffffc90000d87498 ffffffff8d07a6c1 0000002100000246 ffff88050388e880 > ffff880503878b80 ffffe8ffff806648 ffffe8ffffc06600 ffff880502808008 > Call Trace: > [] dump_stack+0x4f/0x73 > [] __warn+0xc1/0xe0 > [] warn_slowpath_fmt+0x5a/0x80 > [] __list_add+0x89/0xb0 > [] blk_sq_make_request+0x2f8/0x350 /* * A task plug currently exists. Since this is completely lockless, * utilize that to temporarily store requests until the task is * either done or scheduled away. */ plug = current->plug; if (plug) { blk_mq_bio_to_request(rq, bio); if (!request_count) trace_block_plug(q); blk_mq_put_ctx(data.ctx); if (request_count >= BLK_MAX_REQUEST_COUNT) { blk_flush_plug_list(plug, false); trace_block_plug(q); } list_add_tail(&rq->queuelist, &plug->mq_list); ^^^^^^^^^^^^^^^^^^^^^^ Dave, is this where we're crashing? This seems strange. -chris