Received: by 2002:a05:7412:8d10:b0:f3:1519:9f41 with SMTP id bj16csp1667726rdb; Thu, 7 Dec 2023 05:56:05 -0800 (PST) X-Google-Smtp-Source: AGHT+IHr86Bz2rIYSquWGBlNEO+J69k2Nw2SJ96ZxiIteLUmAGdVw7cFn5HqCIeBqFeB8mWZ7JhI X-Received: by 2002:a17:90b:5281:b0:286:6cc0:b920 with SMTP id si1-20020a17090b528100b002866cc0b920mr1822100pjb.87.1701957365538; Thu, 07 Dec 2023 05:56:05 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1701957365; cv=pass; d=google.com; s=arc-20160816; b=LcY/oE3YaHmhgZkKErgx8jdIky+2/5tMX6W8jZ3JGhi4Sxt5YzXm00KmZ2Iv7oSzXa 1oO3o++RQaxsfz5f2nK2XD9Fv8jMJxMqcUC5rLVwmEJN0cPdYVBtqdiN205Qg/wSgCUD nZrtBeou759Z1+QPyOs3xtfaSdf+euUfi7yJa+x2Hj9Pd8G4CFYcPH5pZ4Mvn7Dy2diN 2Q1ATSXGbkBZ3Oh73Ylx7UCdytspcW7v7GUrB8QU3Ge/CYxYmvWOBOi+4Ew3Ri0fpjJe lqUh8NTYhxPLCltPPgTK3Wpf3RxoWzg8xfgYa264fSEXVhT54Up9/pz5rk1A5ChDU/5W YUow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id:dkim-signature:dkim-signature; bh=7GBnXR+qL+q/UpJWyclWxRNiYvCwSDJBxzxViQmXbdg=; fh=5gbjK+2YkxeZRWOwpj+3PI9h1OAX2Ziklv6qt9iQLzY=; b=I97lwuxxBACyE4l6cOL4BTPedoEgrTHduGm81euw6FkYqQaAYap21eWr6TG8ovxOg3 uKfp5RpRVY4Od4MGCK6CLBNrV1hBPSdlfZSWvHFn/ItNvKnmSrZe9EQXXxuVTg6Xj4N/ Wpan19LS6P0+qnmhuJEDWGpLj2EDkOj0vvjpYBPqM/ykdLSfIpf82R4KCrMqZw8hW1Rl 2yO90LMbY2YCSX/4AKfHiLjiROrwKNyQlXqIj2cS2bj+Snff3OY/03APUrsCu3uPTf2c 1KEOJL/1oqHjrJr0Bbf37goezMeRZ65jAhJOaAjQA8bzMQ4srUBKdFgSOiqZ6RrqqAbO RGjQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=neutral (no key) header.i=@sapience.com; dkim=pass header.i=@sapience.com header.s=dk-rsa-220413 header.b=dLGBvAxf; arc=pass (i=1); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=sapience.com Return-Path: Received: from snail.vger.email (snail.vger.email. [2620:137:e000::3:7]) by mx.google.com with ESMTPS id ei15-20020a17090ae54f00b00286df9aaaa0si1129564pjb.146.2023.12.07.05.56.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 Dec 2023 05:56:05 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7; Authentication-Results: mx.google.com; dkim=neutral (no key) header.i=@sapience.com; dkim=pass header.i=@sapience.com header.s=dk-rsa-220413 header.b=dLGBvAxf; arc=pass (i=1); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=sapience.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by snail.vger.email (Postfix) with ESMTP id 1251180A8B41; Thu, 7 Dec 2023 05:56:03 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at snail.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1442726AbjLGNzh (ORCPT + 99 others); Thu, 7 Dec 2023 08:55:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1442716AbjLGNzd (ORCPT ); Thu, 7 Dec 2023 08:55:33 -0500 Received: from s1.sapience.com (s1.sapience.com [72.84.236.66]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DAA41708 for ; Thu, 7 Dec 2023 05:55:38 -0800 (PST) Authentication-Results: dkim-srvy7; dkim=pass (Good ed25519-sha256 signature) header.d=sapience.com header.i=@sapience.com header.a=ed25519-sha256; dkim=pass (Good 2048 bit rsa-sha256 signature) header.d=sapience.com header.i=@sapience.com header.a=rsa-sha256 Received: from srv8.prv.sapience.com (srv8.prv.sapience.com [x.x.x.x]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by s1.sapience.com (Postfix) with ESMTPS id 166F0480A23; Thu, 7 Dec 2023 08:55:37 -0500 (EST) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=sapience.com; i=@sapience.com; q=dns/txt; s=dk-ed25519-220413; t=1701957337; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : from; bh=7GBnXR+qL+q/UpJWyclWxRNiYvCwSDJBxzxViQmXbdg=; b=AnvDJv4m7jFWsdahxLboF2GUYPHsNIgzvFagVox1f3bRKj9KZglEoqRiOtJB5o4ZaYSXO dzWa6gQdT1LTjrWCw== ARC-Seal: i=1; a=rsa-sha256; d=sapience.com; s=arc6-rsa-220412; t=1701957337; cv=none; b=Kuh42i9/Tc4d6JmvB+Wb9N1khi1SiyPMcf/NpA5tBhxMlXmMv/u2bOEApb4DuqlnOsZjXR9lmZQ4/qiWM90RQxztierc+CCC8FoLucllmKi2NxiFILNjRHDQqgWyrOv0oYFSnjQCMPGcP/rTOeuFCioJ0XMGR65ybWh6AEJsRYxj85FgvuVR0PQsMfgGojjaLyUXIg7wREoFHdmOFc+rHNIEl2IoAPj+IEgJLXT22FlE30mz6uvV4sF9G9jBdYbtFbRQUHzz66zh0BO4mh/mU0oXhUYyuXIxr/zoSbtDAoqHlHW3UHUlB7YVAarFxshysBuyBUVDoiPN1WAvEI4U5w== ARC-Message-Signature: i=1; a=rsa-sha256; d=sapience.com; s=arc6-rsa-220412; t=1701957337; c=relaxed/simple; bh=5gAc//SJA0AzKde1x2UEFUi3xVjmF7jW4UIpsDnKqls=; h=DKIM-Signature:DKIM-Signature:Message-ID:Date:MIME-Version: User-Agent:Subject:Content-Language:To:Cc:References:From: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=vUn2MNYlInzdGPMSB0HDqAbtE7P9flJXvC1ZllDiO0rsQIUGbcDmZIO2/Vb38ircr8VQekiookdGIt1eSFOEFiD/0Ild0TJyxRS7DznWTFvedm1k4rxoolWMXE6HVGKbinLqfxHAehYviQP9zHmHdw5q1+ZNJvsQTcSxb72WkucJAHTxpq2qeSbXV1MZHr6NwhiY+04SKG6wacjbqGFRWokIYomTtoSWo1mnQTjsEBThqrVe6ZZ/4/W5a0ZL51YqjzZX6/t5aLz96VWtabQxOL4EjnsePrx7HqVp92H0LEEZwNwWnB8eNtgLZLYWmpPJBq2/mpeNYaXYHOna4cWrTQ== ARC-Authentication-Results: i=1; arc-srv8.sapience.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sapience.com; i=@sapience.com; q=dns/txt; s=dk-rsa-220413; t=1701957337; h=message-id : date : mime-version : subject : to : cc : references : from : in-reply-to : content-type : content-transfer-encoding : from; bh=7GBnXR+qL+q/UpJWyclWxRNiYvCwSDJBxzxViQmXbdg=; b=dLGBvAxfuiXNeNrn7SVcSApDwUsu7aOomKnuM02k1aBwcsMUGhlk+KBJPP7jMMfVdR1VD QuotNm7RDRoQwBe82pyKom/r9dTrg2lYkuVeBnsdhrPaELOvSgHhxN+NCA1Q+Lr3MenWHIJ MbURcDOmzx5YkP0qwbotE28zu4vkTy5n72VZTjEN3+pbQI0vUrJE0WjLIlYRe8qKwuBkh3I WzF6yLEddzNXsPUMyWEcIupdEcZpXHHlxf4S1LIQAiTWgql0NePz13aVXwYOITuiimiV6gs 0blrMlg/I3OKYb1gOgCttfVTQGyDbcnse0pe31TVYlAxuJpHG1yviojDECUQ== Message-ID: Date: Thu, 7 Dec 2023 08:55:37 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: md raid6 oops in 6.6.4 stable Content-Language: en-US To: Bagas Sanjaya , snitzer@kernel.org, song@kernel.org, yukuai3@huawei.com, axboe@kernel.dk, mpatocka@redhat.com, heinzm@redhat.com, Linux Kernel Mailing List , Linux RAID , Linux Regressions Cc: Bhanu Victor DiCara <00bvd0+linux@gmail.com>, Xiao Ni , Guoqing Jiang , Greg Kroah-Hartman References: <6e6816dd-2ec5-4bca-9558-60cfde46ef8c@sapience.com> From: Genes Lists In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,UNPARSEABLE_RELAY autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (snail.vger.email [0.0.0.0]); Thu, 07 Dec 2023 05:56:03 -0800 (PST) On 12/7/23 08:30, Bagas Sanjaya wrote: > On Thu, Dec 07, 2023 at 08:10:04AM -0500, Genes Lists wrote: >> I have not had chance to git bisect this but since it happened in stable I >> thought it was important to share sooner than later. >> >> One possibly relevant commit between 6.6.3 and 6.6.4 could be: >> >> commit 2c975b0b8b11f1ffb1ed538609e2c89d8abf800e >> Author: Song Liu >> Date: Fri Nov 17 15:56:30 2023 -0800 >> >> md: fix bi_status reporting in md_end_clone_io >> >> log attached shows page_fault_oops. >> Machine was up for 3 days before crash happened. >> > > Can you confirm that culprit by bisection? > That's the plan - however, turn around could be horribly slow if the average wait time to crash is of order a few days between each bisect. Also machine is currently in use, so will need to deal with that as well. Will do my best. Fingers crossed someone might just spot something in the meantime. The commit mentioned above ensures underlying errors are not hidden, so it may simply have revealed some underlying issue and not be the actual 'culprit'. thanks gene