Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933808AbcJUUqk (ORCPT ); Fri, 21 Oct 2016 16:46:40 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:57772 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754619AbcJUUqh (ORCPT ); Fri, 21 Oct 2016 16:46:37 -0400 Subject: Re: bio linked list corruption. To: Dave Jones , Andy Lutomirski , Andy Lutomirski , Linus Torvalds , Jens Axboe , Al Viro , Josef Bacik , David Sterba , linux-btrfs , Linux Kernel References: <20161018234248.GB93792@clm-mbp.masoncoding.com> <332c8e94-a969-093f-1fb4-30d89be8993e@kernel.org> <20161020225028.czodw54tjbiwwv3o@codemonkey.org.uk> <20161020230341.jsxpia2sy53xn5l5@codemonkey.org.uk> <20161021200245.kahjzgqzdfyoe3uz@codemonkey.org.uk> <20161021202325.q6uh7k2pgnv276rg@codemonkey.org.uk> From: Chris Mason Message-ID: <0cbc7ee4-9268-9765-dde7-1e28c4cdf8f0@fb.com> Date: Fri, 21 Oct 2016 16:38:03 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20161021202325.q6uh7k2pgnv276rg@codemonkey.org.uk> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [2620:10d:c091:180::773d] X-ClientProxiedBy: BL2PR19CA0021.namprd19.prod.outlook.com (10.167.113.31) To MWHPR15MB1245.namprd15.prod.outlook.com (10.175.3.7) X-MS-Office365-Filtering-Correlation-Id: f2718f3f-01f8-48b8-46dd-08d3f9f22be0 X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1245;2:cjleSgNqI1yz7thgjeIQO1R2EiPvnuQMh20Wnp4AbALDoJ/vjV2/lsSKWYC6FXb7eUkzSA5mHku7lMeZjr1w1LRepANfyFin6kehwkpIMtW4oYGhwE+evcYp/9UeyY8MhFkRk1QGax1NspDYm02Ji8ig2EdoiJAF1aK6P1HMA2jHRN8p83KiJJS3KAb08q0+sRLC47bg+O+qNdHkQ5c6wQ==;3:1eQkk3sDZpTKirc228rLIqSIhhAd1qGw9DtW5k+ndyfDByrEyJFqb+7YVaDK60kavVgiF8ttmWpf3cLShGSZuv1ruyLPbJ5HthtShnqC3YIG/Cp3Wfj5HsbhgFTxdKHVezG+vGuAqJyYkhEMjf8BAw==;25:Gp7zIRM65+zLUDiuCSJYfx7qJcz4ClV1pLN/ZwDTqS5w7DJoUWKzsOpzIahQyURfUifhzY3u54a+SCdJinMxmMzB/Ab6vtAMXw9ynjwWVXZw49SDc8ma1zbCcvSNGAvv5hWk/ZgpjF/9UrzTTmnJt1zhYaLvdSjdEJx6k8LVfP7350Qfki0ng9jgH2yEuymWAEJZLcgtJOgvo7J6ARAlcZ7S4JEL3JcRW5NjL1WUTgHzTfBGlby0948UHwALOd732dD9dE5q8y31eYpmrsDgz/wKA9BaxASJcEuc524JtVgpEVhhbHNhnVTZa6V2YkERrmPrrZEkmTiWdQXw++guCx7Y+4WVH3ADoLBC/fnMPQC/Yn9PlkITzwY1h62wncehRnPcZrPSzGqaMEcjY3pNpmaykdsKq9MBKJHjnwcHM48xNLFqI1ZRU1lpWWTXmu6L X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1245; X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1245;31:LV3l4P+/3COvW9kTfMgivUvg+pjpALnx9OwZ24RzT41yiCcspXubcsYtRURCTP0GH20wNzRyHMTtSunXpe3wG/pyqzGau6dIEqEFphkdqTtwu9Q2iGHQuhQTvaB/wG99JlngM4Qf/tf+XNeNgk49+hKIZ6xmPLC33uh6Jjh+IXMlrK+9BaUaA2e7yVS9dL6jFNRk9mqZE0h4Ap0NyMlmNFVG6dDkd+tQ3ZwUC7B+ki4on2KVARd32ehtja8a02JN;20:fkwE4xTgAdsSesY1tl6w/Fl5/GG3yDNgqJ8HpXGqv2lJddE2Et5L+3XGIDuJi52PFKOk0W4yjTkhbfhV0eJx5Gcv/MLB0YmJ9jvNmHqInFg+Lf/WW2Sxx0EZR/y0o8JagDuHwY6aDwdYKzc+F9Mtshr/QcS+Pk3GwNeTQcAO23k=;4:VJz6mDSbgRxDD/FoMqQfdV2KBuPq/QZp9XBGIrni4l5orHFYkF/JrbnMMwefRkvsYnptw9v7mE0RvIkwP5nlrxiJ+M6lIE/oYq6hxNXJQJsFsKfhxsPCoEbs4keOE8fNfy+vVsIG2RuOlMXKKifxPsHFD/Yv/Zjx5/FPUrrIe0jIANpMw7eP5Qib2artZDwWhXBDoKtmHM09WyyvCqaoQgCvNfqDXDvM1ajub55FH0CTleBLghQJfxJ+XrfPOlYLzQ5U+NkTkXWL2ovijlR+3EIYmUFkjoCWcDiFXspRXZI45sGR5laxFz4tZt0lUZVOsOgdPUmh7EuYnlFNmvQK19hy2xw9KRnMHKFFHXW+YSoxgtNpfxHQcb0LLdsUTHVRigh/E6P3+MKXmGbgYKCW7Q== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001);SRVR:MWHPR15MB1245;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1245; X-Forefront-PRVS: 01026E1310 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(4630300001)(6009001)(7916002)(24454002)(199003)(377454003)(189002)(93886004)(7736002)(101416001)(6666003)(105586002)(305945005)(76176999)(33646002)(54356999)(31686004)(42186005)(189998001)(8676002)(106356001)(65826007)(50986999)(5660300001)(5001770100001)(81156014)(97736004)(1706002)(81166006)(2950100002)(7846002)(107886002)(4001350100001)(230700001)(36756003)(3480700004)(83506001)(50466002)(68736007)(77096005)(2906002)(6116002)(23746002)(65806001)(31696002)(64126003)(86362001)(47776003)(92566002)(65956001)(586003)(921003)(3826002)(42262002)(1121003);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1245;H:[IPv6:2620:10d:c0a1:1110:8000::204d];FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;MWHPR15MB1245;23:sQ5jTGBs4ObMLhPu0B9VqunN5Ghfb+ZZB5uem?= =?Windows-1252?Q?lim/Q1G5cYn1OB7QnEBNG7iPp4rNIxfmdg16Rpy1DESVPuwdvcrqDK3R?= =?Windows-1252?Q?X/WyCs7RWSxRIx6xF5fR9Q441+yKHLvJuszBxAdc5HZU+5UVuFfswWS0?= =?Windows-1252?Q?lyUSttTk/9j0iVWbNbJqD8PeHR4XE8FDMgv14HRfiomDMZ/iyrCwO9WL?= =?Windows-1252?Q?9XBDEN+ReqGFD2gI6w9c4kcaESo773SePEl7OAnUlBs29rm+QOiC3Wx7?= =?Windows-1252?Q?3hCs4qbd3Dy6RnvKmZIj5TtVQe5OCMjPsaQd3Ew2UCEHiX4T3xCskW8+?= =?Windows-1252?Q?XmJRbQL8gESppwT6GA/OGXuGUoZ/fZwlYBc7he0MiOxzjlDTigmv3vAX?= =?Windows-1252?Q?lnbc/oxAJhIllXhe/4i3TxVkpOckYjBo16uH2P/i7f+GrkTcx1I8l8da?= =?Windows-1252?Q?yZ7c1fEcoJ8VeNq5fcRa/s3mKB+jzkqii2OjHZ2kqu6fIVX3oqAP8ezd?= =?Windows-1252?Q?MX4sjytCADrzH6YSg+zcq8AEBy0Auw5e7QzhHy8zZk4p2jhijuifF3Yi?= =?Windows-1252?Q?xfq3K0+fEJAfa3wL9qgBIvei51w6br30GfnPMA4z6qxax2sKI6SbXQy2?= =?Windows-1252?Q?ejbYZkpLAuV2rjNg0BD0/9dHdweZcEomlMdE59+MDyS5zfE0+TuwkI1P?= =?Windows-1252?Q?+an4/jHzIVnhM1HSKU89QDi5+8TZTa2KkTAHRnleC8pOA2FLI8DAiy36?= =?Windows-1252?Q?w/HSfAhjZGWV1FpznQfThz1+iDAoRdlkupqonxgJIQoLomIgSqGJZYVA?= =?Windows-1252?Q?/ZCv0ysY9T1VNpXtNu74DSNyX80/kKTaofFQapHxdyRrDLJz96G7YUFg?= =?Windows-1252?Q?k2Lj98CcneUe/WeOPgTTG1hDZ0kuHVseJ3ljcFhXEgx3xhb9YuPKXjSS?= =?Windows-1252?Q?YWix8StpuNHCuwklj/zrhJzkg+kli91XrqCth/3wXaOkZ03vLPbDgpp3?= =?Windows-1252?Q?DmTbC1JGfGZ2SoRAZwGLiuWAFedxrBCF+A2OTgfbs3HWZbq/tjuThgI4?= =?Windows-1252?Q?zimCvIF/mysBoDYCifAgEPesRf/ajghuo1EXfF/Gph+fxaKq96PuzZB4?= =?Windows-1252?Q?R1yJ4cMW/TR3L2dIfiyu5pNum9eokGGwbBwk/g3ge2Ym4BPXARMU2wmX?= =?Windows-1252?Q?tghUZyiZOUlSbB4PYlLb7IoE2zRfW1bLUD5laZFu1zxLBIUXDpcJ0lLc?= =?Windows-1252?Q?ZP/RYMTnIjI0leVWXZhJI9a4IaRh4WrSizV14wV6a3uFh3f7C8vsvvMC?= =?Windows-1252?Q?wfVd3MPfvO+j9MwPGPFohLIv/myvUwzaT6qqS2XSZVecZiT8ruAjM4xI?= =?Windows-1252?Q?NVjz2yax1PSvJwp3Bcja5YttQspVhrH0w7J3mlkHEy8WRPT6g0+I6RrJ?= =?Windows-1252?Q?gtFLXPKqhtIfBeV3XVtS6LA0HiTMawyd0mMHXmeUw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1245;6:5IKvkaqKK/tsIu+n7acGyxaSvvvhvkU4N0xEypfiWi8yb7Ep05MkgC/UJdX17y49n+ns9RFWSXdEK8Zv/41XCB8VIbRffTOYSj/F9H1UIojw7hYty4gUvZXVpEVUg8+4TdCixv4ut/warp5ax6isFFwkL7ve5AqklTwh2PZFRxox/p7I1eoEM1W696RtExkXeoEYCdyIhqGnWK8Urgd1DUOXOx5puRrEfXZL5iEDFQUPmeoTtdYe1VjZv9tbD7dUviChd0cIOrSlaxKVwMNUFl0lTjdeK/UahuOYDLCCDNafDlZ7LfWkzN3ih2Khqz5w;5:K/KGz3gkP3wnIGeKt0CfaaTaTCpkhdPaqFnq9Cm8U7HXIa8rDU4v/PNAFWWDggRTlwN7iY2dL9KoZele9s+oxRTNALeG3p/GX5YgSNYbTyQ0rQu+XoBI8d64mpGwnUzx2znNnMbC897q2S0Z45YOOao0yocc4fwVvpNTod6+8D4=;24:tr+v6GCSN08c2vR2JaVRLHLb+ba7J7mD0m+3K/as4ppUWfSvYW0zUyfE6SC6NAqju3iWhQG7ZHechnc/hGT/P3sh83V2iLvXpFlpi5dM/0Y=;7:CZIdwZDncf+AnqB5x9Xg42jZ83WxCcqjm/K/f5rAuiSBlsH88buTZbdh409wQANdb2hR9gZ80TvUBcvJwUM5jU+itwfztThmcaCZVQyAXBjNhC3pwhP6aDWIDqaPA+03HEQVTh0ZogbJmwvWFG+Ish3oabU57/8/c+te1px1Dd3nMT9yJqAycQZ46ZJq7w37gz+QUBQnc3LGfBS/26XC144HJriVdh/eZB8DzqgTfOBLM7HEnFG2f2Ay7WgTr4szOosuqIv87XZ1e9a1csBsM3obuMcFjxQa5HhhrKS+AKTOaASNrkZb/er5VHxwNQ31Ao3FPe27XUr0C1gEZ8XfZOpD/+a7hF7VehOcqs90E4A= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;MWHPR15MB1245;20:/8fWY6jwnLjw1OudRmF8RXfVq49B3JkgKZgVQ1vXVEX3kRoWTLxH8cUlhMK+ojqoKD9HIKzl4JZbqMERXsK98eq+fL6Cqf2Y/3qn2u7xLV2k+uiROF0tWBrfkjdIm5FwmmOZ67QMWD9EUl1NXKM4Kx2Zcy50iZY7EcGy9Xd5Wl0= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2016 20:38:08.5730 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1245 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-21_12:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2099 Lines: 37 On 10/21/2016 04:23 PM, Dave Jones wrote: > On Fri, Oct 21, 2016 at 04:17:48PM -0400, Chris Mason wrote: > > > > BTRFS warning (device sda3): csum failed ino 130654 off 0 csum 2566472073 expected csum 3008371513 > > > BTRFS warning (device sda3): csum failed ino 131057 off 4096 csum 3563910319 expected csum 738595262 > > > BTRFS warning (device sda3): csum failed ino 131176 off 4096 csum 1344477721 expected csum 441864825 > > > BTRFS warning (device sda3): csum failed ino 131241 off 245760 csum 3576232181 expected csum 2566472073 > > > BTRFS warning (device sda3): csum failed ino 131429 off 0 csum 1494450239 expected csum 2646577722 > > > BTRFS warning (device sda3): csum failed ino 131471 off 0 csum 3949539320 expected csum 3828807800 > > > BTRFS warning (device sda3): csum failed ino 131471 off 4096 csum 3475108475 expected csum 2566472073 > > > BTRFS warning (device sda3): csum failed ino 131471 off 958464 csum 142982740 expected csum 2566472073 > > > BTRFS warning (device sda3): csum failed ino 131471 off 0 csum 3949539320 expected csum 3828807800 > > > BTRFS warning (device sda3): csum failed ino 131532 off 270336 csum 3138898528 expected csum 2566472073 > > > BTRFS warning (device sda3): csum failed ino 131532 off 1249280 csum 2169165042 expected csum 2566472073 > > > BTRFS warning (device sda3): csum failed ino 131649 off 16384 csum 2914965650 expected csum 1425742005 > > > > > > > > > A curious thing: the expected csum 2566472073 turns up a number of times for different inodes, and gets > > > differing actual csums each time. I suppose this could be something like a block of all zeros in multiple files, > > > but it struck me as surprising. > > > > > > btrfs people: is there an easy way to map those inodes to a filename ? I'm betting those are the > > > test files that trinity generates. If so, it might point to a race somewhere. > > > > btrfs inspect inode 130654 mntpoint > > Interesting, they all return > > ERROR: ino paths ioctl: No such file or directory > > So these files got deleted perhaps ? > Yeah, they must have. -chris