Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp6544095rwb; Mon, 5 Dec 2022 14:02:28 -0800 (PST) X-Google-Smtp-Source: AA0mqf5U2jXOO54pfMuXVQ6LgPCDtiDL0HZ0g8SxbZdWJQPorxlvgMNnH3ggBzFWro2ui4RGsSqC X-Received: by 2002:a17:906:314a:b0:7c0:c90a:a978 with SMTP id e10-20020a170906314a00b007c0c90aa978mr13158376eje.387.1670277747945; Mon, 05 Dec 2022 14:02:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1670277747; cv=none; d=google.com; s=arc-20160816; b=ecj9yiDe02GnQm6f6yb8lKtBf76ffw+U2SkjO1jVqrigLp0taxQgt0lDgymub7VPfv bsFaSvgudVZA1djBPQ1aeF5xQWgVDodQ1i/KjV3Q1zN/q9NHK0basOse2VSgzMfdkurv eu5pz1yeBp8bOf1mGWD1aAtAErB8gH0DNIaOcTMCzrmzmKfDNykMcRvhzrmYseslZh6T phSdyEGPIf/FZxkzVRYn54em/7lxL7sBnl8cBiYzY1PIjRVALUaL7LtxMocw8cGCObO5 V/gmX8jTj0RB5SRa5SPxlc6+zOORy1eyS6kgfKYfFbku6ImtJEkd2ELUojHDxlJCD9fC 3ytw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id; bh=pv5SSAEqGOvAMmg1cIpkMvSyhWjm0nbW5cnhetFcALo=; b=TkUBeII7LiIm9bgBwNXUcbiEqa5zYB7qZtntcdpcRbsHSVkpMEoA56XzUu01zyvutQ Xk3U5nCgEhWEY/IRNc/Gx5vMoo5yGiOnkykHWgGYoPeDtc8mDhy8dT3eBlp7DaLQ4qjg 7alcjxBN30jp72IoTQldFkEOLJHF4eN1ZMu+qr7F/KW9j1NtCCUYiY00nKQP+owL0Nzt PiDwYzlK/72G5Ra99T528aBxEKe/Ra64gukmmyekSgpZqtweCrcZgSXkO54NB6aUEltt 8MWhvy38nAWD+c9757xl4+PvYDxUdfJ9SpEe/v834nvKchfmD7OsKAEmDigcXL5/5thM djOw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id f12-20020a056402354c00b004571c47b13csi595637edd.418.2022.12.05.14.01.56; Mon, 05 Dec 2022 14:02:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232357AbiLEVwl (ORCPT + 99 others); Mon, 5 Dec 2022 16:52:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50454 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234156AbiLEVwF (ORCPT ); Mon, 5 Dec 2022 16:52:05 -0500 Received: from us.icdsoft.com (us.icdsoft.com [192.252.146.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62AC929376 for ; Mon, 5 Dec 2022 13:50:51 -0800 (PST) Received: (qmail 15206 invoked by uid 1001); 5 Dec 2022 21:50:50 -0000 Received: from unknown (HELO ?94.155.37.249?) (famzah@icdsoft.com@94.155.37.249) by 192.252.159.165 with ESMTPA; 5 Dec 2022 21:50:50 -0000 Message-ID: <4e946dfd-96ca-44cc-6184-a354b8235ed1@icdsoft.com> Date: Mon, 5 Dec 2022 23:50:48 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.5.1 Subject: Re: kernel BUG at fs/ext4/inode.c:1914 - page_buffers() Content-Language: en-US To: Theodore Ts'o Cc: linux-ext4@vger.kernel.org, Greg Kroah-Hartman References: From: Ivan Zahariev In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org On 5.12.2022 г. 23:10, Theodore Ts'o wrote: > Is it fair to say that your workload is using data=journaled and is > frequently truncating that might have been recently modified (hence > triggering the race between truncate and journalled writepages)? The servers are hosting hundreds of users who run their own tasks and we have no control nor a way to closely observe their usage pattern. Unless you point us in a direction to debug this somehow. "data=journaled" is definitely in place for all servers. > I wonder if you could come up with a more reliable reproducer so we > can test a particular patch. We already tried different parallel combinations of mmap()'ed reading, direct and regular write(), drop_caches, sync(), etc. but we can't trigger the panic. If you have any suggestions what we should try next as a reproducer, please share and we will try to implement and execute it. Did I understand correctly that a possible reproducer would be a loop of heavy write() followed by truncate() of the same file? Should we randomly sync() and/or "echo 3 > /proc/sys/vm/drop_caches" to increase the chance of hitting the bug? Best regards. --Ivan