Received: by 2002:a05:6358:4e97:b0:b3:742d:4702 with SMTP id ce23csp1873184rwb; Sun, 14 Aug 2022 14:56:39 -0700 (PDT) X-Google-Smtp-Source: AA6agR7U1X1EW00L0J9JX6wfpoAt2sCY4pZQO+TuunWvG/u+3U569WTF1Qr+pgASi3Ai2b4Ba7/f X-Received: by 2002:a63:1d1d:0:b0:41c:3d54:b123 with SMTP id d29-20020a631d1d000000b0041c3d54b123mr11450414pgd.357.1660514198947; Sun, 14 Aug 2022 14:56:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660514198; cv=none; d=google.com; s=arc-20160816; b=dQuXEWyfzjOXEtbG9oQdL8hjx9PVI6iu299rwLuKMxHumIUdJJxpWdm27GGUclmAsX n7ZdOwvW4Uvj/qonAt4pvh1RGVfAbuvQVzHp+duhFZzBhMyZIUwOz+wotntI9ECV9g99 k8p0tz8r79X8yfVk8r9xTuCpDSB9T3VlxM+X8zYCElVDfK0o7UP8FXoQD/El3pGWvJ+L Vk87cxyVW0kNKQM/E0spheeX52ckC9niIj2ktzEfdZjil0QZ5N7R9ybTuPfTopITWaA/ wwEGMGdP6Wk8vlY6mW7mKFvbgje47P4CukDP2RNwsKKpNMlYADFwapnvzlxHJiFNM6/b mPZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=0N4kyysm9/43RV8zlNyJuXGP2DDx8PgtF4Hpfd6IqQI=; b=yp5nl7kQdWt/cGGhlX3PV9SR/JB4M4DNm1QVrujOkjPF+ZJzIOzW/aJgB1pz7Gfubu lKl9LylutnFeyn6GWxZAxFtxn52TNkDQhnwitXh6OAhrrxOX1FYEyJQ+0CnMPSC3co+U mLXJDoPL9D5CKbRYM7N2tQkFeVnjAiUA3axMLBDk8E/RrGwNMsbZaiVFPF1+XSwAZTd7 c1LU4NqD3m1KBWcdf6mN0iTR4jHJSF5klz+thh2jqp8kxZz6P1LSxPlP4ZqizgRRDq9s iCSu2J+oVPMxGoK36CxiQZx/nEto6xQL+xo/hYOqGDEI8MblaZCIClv1AvDTbTmWIlCv prYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=google header.b=O35nkYma; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u13-20020a056a00158d00b0052e6fe35393si9584988pfk.10.2022.08.14.14.56.27; Sun, 14 Aug 2022 14:56:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=google header.b=O35nkYma; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229597AbiHNVkq (ORCPT + 99 others); Sun, 14 Aug 2022 17:40:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229640AbiHNVko (ORCPT ); Sun, 14 Aug 2022 17:40:44 -0400 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA73165A3 for ; Sun, 14 Aug 2022 14:40:43 -0700 (PDT) Received: by mail-ej1-x630.google.com with SMTP id dc19so10696847ejb.12 for ; Sun, 14 Aug 2022 14:40:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=0N4kyysm9/43RV8zlNyJuXGP2DDx8PgtF4Hpfd6IqQI=; b=O35nkYmaMrrX07dYjyoaxKR8A0rqhR+2NpVv+mUgfy/3Kp3hftiXLD/V7Tw+//q2a6 NCjR8a6YiqPQTM83P87L9hkGavh4s5YhasJUqiLKocBTjM2fnmQBWTDr8Qkf0hVshov0 8tZ1gHk6uEU6qM/IiOUkiQ29GKCdD/P8wC1Wg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=0N4kyysm9/43RV8zlNyJuXGP2DDx8PgtF4Hpfd6IqQI=; b=JRKy8U7vjfiBzbx+jTIspbIPlHrpM2xDWZ7u8exakipfyp8h27C1l/0p79yT1dHiLk 2c3OXCUf8Ou9rp2iQoBoyq+2PzU6Wr/8+ybrGUbZsQWyEM28LXoefgjfJ5zPyP97TKLq 5GNv2w7oc1chSuL0jGDSIwXUOL5jN9aZZu9BhwkuF030WIOzU4fnWLZT/toLqc4DvwyO /jbEVYXIxxLXuEp8jOHHC+xaUvDq2EDgRQt5tEvwM7dmyzmiBR3QEoBwaJW84p1MidLz LLWF1WytLa7NyA7olVA3KmEcPOHPSPl5+ccfCbn2p5cenklUXwHGPBAEQngHnmnP3gLu zNog== X-Gm-Message-State: ACgBeo0Ik35t97l7iqro1cyFcs5Hg/CnGgi41IunV6Mln/SoXYI7fEbd 7O0ahwNCtFlUo0ZPqAqPUqdx9DWVF/w95H7R X-Received: by 2002:a17:907:1c01:b0:6f4:2692:e23 with SMTP id nc1-20020a1709071c0100b006f426920e23mr8410472ejc.243.1660513242234; Sun, 14 Aug 2022 14:40:42 -0700 (PDT) Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com. [209.85.128.43]) by smtp.gmail.com with ESMTPSA id g1-20020a17090604c100b0073100dfa7b0sm3336853eja.8.2022.08.14.14.40.41 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 14 Aug 2022 14:40:41 -0700 (PDT) Received: by mail-wm1-f43.google.com with SMTP id j26so872700wms.0 for ; Sun, 14 Aug 2022 14:40:41 -0700 (PDT) X-Received: by 2002:a05:600c:4ed0:b0:3a3:3ef3:c8d1 with SMTP id g16-20020a05600c4ed000b003a33ef3c8d1mr14599046wmq.154.1660513241376; Sun, 14 Aug 2022 14:40:41 -0700 (PDT) MIME-Version: 1.0 References: <20220814212610.GA3690074@roeck-us.net> In-Reply-To: <20220814212610.GA3690074@roeck-us.net> From: Linus Torvalds Date: Sun, 14 Aug 2022 14:40:25 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: upstream kernel crashes To: Guenter Roeck , Andres Freund Cc: linux-kernel@vger.kernel.org, Greg KH Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Aug 14, 2022 at 2:26 PM Guenter Roeck wrote: > > Hi all, > > syscaller reports lots of crashes with the mainline kernel. The latest > I have seen, based on the current ToT, is attached. Backtraces are not > always the same, but this one is typical. > > I have not tried to analyze this further. This smells like the same issue that Andres Freund reported: https://lore.kernel.org/all/20220814043906.xkmhmnp23bqjzz4s@awork3.anarazel.de/ he blamed the virtio changes, mainly based on the upstream merges between his bad tree and last good tree, ie git log --merges --author=torvalds --oneline 7ebfc85e2cd7..69dac8e431af and assuming those end-points are accurate, that does seem to be the most likely area. Andres was going to try to bisect it, but if you have some idea where this all started, that would help. Linus