Received: by 2002:ab2:68c1:0:b0:1fd:9a81:d0e4 with SMTP id e1csp132958lqp; Sat, 8 Jun 2024 09:03:23 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCUHFCZWIF2ksmn2iVXkgMUuDORRqP9+Fzpiij5ynZittE+wrf9eelFDwwGcD1GkID12dax2gaCaTkkndr8LP86JeVmP9a0or7Cp4OmHdg== X-Google-Smtp-Source: AGHT+IE76+M8g4Cn//mvWbxaf6Y8esCwidC00VzOET+bhsKAfzbNEYcSnVYIGhegkvmo95p/KlGY X-Received: by 2002:a05:6830:1650:b0:6f9:8a29:8e8b with SMTP id 46e09a7af769-6f98a298f01mr919625a34.32.1717862603414; Sat, 08 Jun 2024 09:03:23 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1717862603; cv=pass; d=google.com; s=arc-20160816; b=yJjtHQfAYuPx1oPSscKd00sxlvWnkmAUNh1aFl1zLWznzQTDZcWilRr99Eh/R9creV sB8fa+hDsJZSeWITDFkfMMHfn5cCbOeH5YuA0HMOo2tdgCpLRAmxrBSL+nTsZKRwssCt ukoZ4P8sdaSLZMGiDM4wNCMV1b7w/CNGIsvpjsOruDWoalUZTQcB4WYx1DNKEdVQGfHU Cj6kic/iq7q/dKbTpOLX9Br9Xgtb8MX1FIL3Fwkp5RY8ChRLtekJDYYmW1RY8Hz48lx5 F08sMoAepmukuhMYubvyGbaH4KxWZJtwDWx07qSkpiWIeO1jWZ02Swkfmqvom9kHM85v P9hw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:list-unsubscribe:list-subscribe:list-id:precedence :dkim-signature; bh=sD+Q+gVC5+5P5PQukDTDDjpSK/EVimfSh49c8qeKnVc=; fh=UdC0TAWdHPQosZ0IcCkcDKLqxDEVnvqR0eFFMhpQMMQ=; b=PUvejb9a1qZXJ/vAIV41nA6LKBZ1NHNZXLVRVIG7Ooy57wealMmeBnQ0Uq1Ott0mr6 sNAKapAobyVD1LaLAFJfnDPh4jNNp9FGGCFzLozXAqGsF6EWvMdGlVRo2E0uWC32Vb/0 dLFWNUn9JcRajr+uNmg6wY7Jv4NnMXy4O8peDD3EVETWotLCOXHVpMUW66Lwpl5oon8k NloME0BEXACu402xwbnwChi7O7iO2E78XXglCvtZR0Mqw/fokXQ3r2xQ521eLIfD/8at 1K7GQwHqi8ORT+12ACJwMtMzlei0U13LO/3JMF+BaHVXREJ1FlDiHr89iwibepzEbHBT s2Fw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=google header.b=JezeoltD; arc=pass (i=1 spf=pass spfdomain=linuxfoundation.org dkim=pass dkdomain=linux-foundation.org); spf=pass (google.com: domain of linux-kernel+bounces-207123-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-207123-linux.lists.archive=gmail.com@vger.kernel.org" Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id ada2fe7eead31-48c1a1eb5b3si1507983137.287.2024.06.08.09.03.23 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Jun 2024 09:03:23 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-207123-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; dkim=pass header.i=@linux-foundation.org header.s=google header.b=JezeoltD; arc=pass (i=1 spf=pass spfdomain=linuxfoundation.org dkim=pass dkdomain=linux-foundation.org); spf=pass (google.com: domain of linux-kernel+bounces-207123-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-207123-linux.lists.archive=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 21F071C20BD3 for ; Sat, 8 Jun 2024 16:03:23 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2EE4F179958; Sat, 8 Jun 2024 16:03:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="JezeoltD" Received: from mail-ed1-f47.google.com (mail-ed1-f47.google.com [209.85.208.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77BED524C4 for ; Sat, 8 Jun 2024 16:03:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.47 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717862597; cv=none; b=doHn+bvmxF+GhfJ+3UuYd5znZdXYeUuxTw52pM4lPxech5qUuSurUL55jhXXHRyUuPtYK/yV49QxSHUhYAqilHX4rZN5FiBRBiYbmuoRNWBNxKWjyk4fa12PNBt9lZj7LpJO7jkJwtGIwUlkKIRzLBubltN0leMZNLxrnZYLueU= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717862597; c=relaxed/simple; bh=ZUGv6tbFAcbWuiVVg5lvhzpTD6NAzR4l2AWzqaVNoDE=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=cnqAfvIhi6JrOhXUR5I50PgquxxNPg6dLO8alRgYUSGQcbOheWSyp3o2pZTvjUWCumhPSmVjte73JQ6NbBsWu+zCEkWn+bieWEq0jtTMTr+sZX/krcqfKQ6FHLUY0bE39rwiF6Z0K2aJKWF9CxWTm3JEoLaiKJz/0P9yJvXFWJw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org; spf=pass smtp.mailfrom=linuxfoundation.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=JezeoltD; arc=none smtp.client-ip=209.85.208.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linuxfoundation.org Received: by mail-ed1-f47.google.com with SMTP id 4fb4d7f45d1cf-57a32b0211aso4230609a12.2 for ; Sat, 08 Jun 2024 09:03:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; t=1717862592; x=1718467392; darn=vger.kernel.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sD+Q+gVC5+5P5PQukDTDDjpSK/EVimfSh49c8qeKnVc=; b=JezeoltDZNN18bMZogM0h+G8NJ5PL6Pzw1mJDF1XEkiFjyedw6+bwfEJCnlc03bZcG 4HLkmWExJfXAz+qOmeeVdrPBuzVXGgTqUrul6ICayb0LHJ0TxmzpJlFVi4e+sOv6249w qOY3jTkoXR8bT6MLEeI6mabmML42ztaO94oDs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717862592; x=1718467392; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sD+Q+gVC5+5P5PQukDTDDjpSK/EVimfSh49c8qeKnVc=; b=FNRUb33EPM9B/ykBXoHwPq2Fm5NDWidrF0jlJgPKtqJR+D676yLSlMSNHwP1bv6b5R OUXyK+rS/qt2Sy5MmLQakkWQgkmmy3SkJPEU/NLtjyQBr6xJ1vLAX1r0+NCvSu6yjGfw b+AmIe9FQy1/LuwpUUvBM56fYRBYq5QqxrW3J9eR0h1Ds2Cj7aauf40Jr958PeP3o3hM II0NSJso7Zijvew4NPUXOHQydHq6QX1m6yQ71QEgZRl1nAypppLfJBMozXSP4XocPaM9 wAxL5aYAgDMKNuroiUrPpUDHwW0kkyDCjim4mSrdDHjkhpWvP6sUvEeGNPq/PR/5g5Ds SOow== X-Gm-Message-State: AOJu0Yw01svrufiK/YO6a2h0HxQSGQhwjh+7lTKM1fCGv0khq86k7/K9 fvlFLxpWnWC+p8gGnI+Hg95+fuLPi5kFiT873JTkOE5KnQoaNxCWVA3vsBYBk2i8Vt3xinQMYOB eq6Q= X-Received: by 2002:a05:6402:3106:b0:57a:3357:5967 with SMTP id 4fb4d7f45d1cf-57c5095d696mr3382493a12.30.1717862592595; Sat, 08 Jun 2024 09:03:12 -0700 (PDT) Received: from mail-ej1-f46.google.com (mail-ej1-f46.google.com. [209.85.218.46]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-57aae12ce21sm4427956a12.49.2024.06.08.09.03.11 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 08 Jun 2024 09:03:12 -0700 (PDT) Received: by mail-ej1-f46.google.com with SMTP id a640c23a62f3a-a68c5524086so364735166b.3 for ; Sat, 08 Jun 2024 09:03:11 -0700 (PDT) X-Received: by 2002:a17:906:f599:b0:a6f:dda:8514 with SMTP id a640c23a62f3a-a6f0dda8694mr89210566b.47.1717862591608; Sat, 08 Jun 2024 09:03:11 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <46cb50d65e414bfd9bef5549d68ae4ea@AcuMS.aculab.com> In-Reply-To: <46cb50d65e414bfd9bef5549d68ae4ea@AcuMS.aculab.com> From: Linus Torvalds Date: Sat, 8 Jun 2024 09:02:55 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Linux 6.10-rc2 - massive performance regression To: David Laight Cc: Linux Kernel Mailing List Content-Type: text/plain; charset="UTF-8" On Sat, 8 Jun 2024 at 07:44, David Laight wrote: > > I suspect it is due to the changes to the way the scheduler pre-empts > processes. The PREEMPT_AUTO changes haven't even made it into the kernel yet, only a couple of preliminary rcu patches that aren't relevant yet. So while there's been some scheduler changes in 6.10, they have mostly been pretty insignificant - mostly some renaming without functional changes, and one-liners. Obviously even a one-liner can end up resulting in random oddities, but could you try to bisect the issue you see to pin it down? Linus