Received: by 2002:ab2:6857:0:b0:1ef:ffd0:ce49 with SMTP id l23csp2898469lqp; Mon, 25 Mar 2024 12:21:52 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCVEr33iWcZDWivLMYLU4tqCth7NS3et/hBG1T2fEdzDdn7kTt2qrjVLfEQ5FkCDkFw1PpmX7wJ7TsyLZPFJwqNMdklV/oyWOoVO6liG2A== X-Google-Smtp-Source: AGHT+IHzsOmCiKS6HLShI2L+1iRhCwqPg6F7+NDH8EGraHEhTGtnYP4e1w979SAj2ngu94BY8pKW X-Received: by 2002:a05:6a21:338b:b0:1a3:60ab:b07a with SMTP id yy11-20020a056a21338b00b001a360abb07amr7524585pzb.38.1711394511740; Mon, 25 Mar 2024 12:21:51 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1711394511; cv=pass; d=google.com; s=arc-20160816; b=NwYmbd0hztq8fPyK0LMgQYbzjFf7TXThJpG1nk/QwDOvanbCZVaLXaOfccb8L7+w2M 9sJFvJOzUS4+VAz9+hxbxQo59DLCKPqGWnsf964oSlhyOJhvg6NlDglYkbd7VMFPJGOp J+kM7naJuJ1xetgUqlZImUOOyr37qOFYRk3w0lo2IiHecb/Ru8vghhqXv1uLtggl24Me YE+2EaGi6Jwm+fUevzfbVgFyCzjPwSqaQ4TKfG8jllxlFhMZ2n9a88pkIRaH+eTKKwZ8 zDq9uwo3KjFb6G51FHWe9sr893+6BFhrE7/d4M7cBsn4pWQufDLjv3LTMlxuFKllEVBt d4iA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature; bh=kky/1shDSavwXJhQDl5uOntgm8L39nVx0FH//gj7QE0=; fh=G31ZIOCfqokPC7OtNlcstiBbHsx4M+78EWzTtHU5T/M=; b=k8yfcilvmVPiyMK7bQ5WkUW2Xb9QuH93Dz78YW4H0VzAsYNGboh5PMUwQkFdM7dhBI VVaOOCbSv26/UnZxiSVg9dBLl/lL3Tw99HZP0PkxOYnww+egqzvLPG3hqSBkpsfBaBba HDwYxXF8Gj36q/vVV+NNS79AU1aW8RQrYlPX0WI1FcrGELCx+DMlEy6LKetzZ6LM1PeS MQzM/itOpEib/nn0Uwi7Na0P68vHeCLfk+LTMMa4m6U5gVMPCeXEoI/JJ+cd3Ym25S3C mnA+nVqF0R7ao+Z8t90oOVBzOqeIaJRoP9zfmULNsjPwvjlN1XPjK+BlsVYQp3Fceqeh gUEA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=QDe9yiSQ; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-117779-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-117779-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id q185-20020a632ac2000000b005d4a1e64c7asi7991092pgq.332.2024.03.25.12.21.51 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 12:21:51 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-117779-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=QDe9yiSQ; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-117779-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-117779-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 3B38F2C68A2 for ; Mon, 25 Mar 2024 19:12:44 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 57327168A8; Mon, 25 Mar 2024 19:12:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QDe9yiSQ" Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E580418645; Mon, 25 Mar 2024 19:12:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.53 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711393956; cv=none; b=DORRLzNFLvT6PB4FLvUOJ1+kZqak9P4K5W5pNcYpLy7tdA6gVaXhlfDp1eb33WibiQYWGyubAHPStuR1SDOT17gOuZuFsT9P4a+v/IbHmjrCt6USbTF9iUi++opdua2miWZTg90RYfQ8NNyrOcXTjP/xjp/IPZ/BLuxFnlQXI2c= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711393956; c=relaxed/simple; bh=fZOe/tLK1cgFPvkLuZbT3YDA7D2x6EJoWMvIzRbFXsQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fWtVc+ElqD2HdDLFuAn4MOtRYWtgJ8s6WeBigUI3erRMk2SINa6hW3A/xA7avk/5TP9pVjoyskG5IgZuAKbeQ8oWLYF9aRSrjp0Z99jFp/PHcD4UKWuOGpXHVfIKFDREJkXgywtXmVSnKgQkg8wzNMJ31jTCh/ILzgsH1UrUBPA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QDe9yiSQ; arc=none smtp.client-ip=209.85.208.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-ed1-f53.google.com with SMTP id 4fb4d7f45d1cf-56c1a65275bso441087a12.1; Mon, 25 Mar 2024 12:12:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711393953; x=1711998753; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kky/1shDSavwXJhQDl5uOntgm8L39nVx0FH//gj7QE0=; b=QDe9yiSQ/yj+PFqN7VlI8roQJX7Ea3+ziFvknF/QrFSLFzrM+AsW9hPrqnFDXRVMZb 3jL4GmQGwokcNcqAIWLK2kvMTm0a7nu/xVx56Ppe7Zx2BEIWXiHauriqXHj8Z/JVChvQ sei1kEmY4x4gj+dZtBONMJYvePxmX0+67wGUksvBos8IcDg33ScKF4Tg8z10PqzCzKoA qp+vTnOd9U+NBWWaMdB2eXavV3DIdlN7X59Yyut0iChdI3rjijOXIJedG9BlsijPTd1h Y8zGSOfzNWaOZEDfiLIZ2+0IOfyiRnMrH0dvcy5TlaqTQNEIMtKbAS578WZu0xPi0tGk WQqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711393953; x=1711998753; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=kky/1shDSavwXJhQDl5uOntgm8L39nVx0FH//gj7QE0=; b=bsEhzq/nYgWM55q/EzCt5SQvH78NWfU+EVG4YYx0jqZOm9f7iJi8C8cKiZf2hjkSrZ imLYyB0IZUmzQn46zkFWB1FnBatiPhQDu9WpSECQ3dZm/I+y+1fLx56Q4s495ia5DE3k WBO+DS/neCKn1Wxfz8rhCMzwBZlCDkKAOBFJXzpYt/mAFH6tcRvokgqnZRJkvtm/XaSe OVcI0s8Gfeg5vHbokgscIosF+AvR3JAM7C8kfyA9PfwPkLzBPWAqr/K4LltjzYBkAOyu +6Tc+1eWEjxY5XOycR+grcHSVUM/Nk6e9fBzOFr4CiqOmD0gd/PHiMU+XUOjsLII2X/k YYLw== X-Forwarded-Encrypted: i=1; AJvYcCV5QBcvLob8MhYzEZ13Vktl9/fxXejWEvApvYkVTOsavDd+DL86UsTjnVVKe3dfZikH84xM8lAZWoWLNb7kTZYjOhtVPOjFjMDg8ypr6pEkiLYmpa5RIEWVzrCR7X0XR4PpOy3w01G4alsiPon+hiGcqePS/8DdEyhMRE0gQkQrVL9qGg== X-Gm-Message-State: AOJu0YzRTn05niJxikKV5/b4a7AvAZn+y/sIQ8DjGetSmFj7b8tMjPu4 NFa7+1LpL7ll5aRgiPfbwAuDG9mhFm5Y+BKvf2Kqaqx5ovgfyK8V X-Received: by 2002:a50:a6d0:0:b0:56c:18b4:d2ab with SMTP id f16-20020a50a6d0000000b0056c18b4d2abmr1882154edc.42.1711393952984; Mon, 25 Mar 2024 12:12:32 -0700 (PDT) Received: from jonhaslam-mbp ([2620:10d:c092:500::7:3fad]) by smtp.gmail.com with ESMTPSA id et10-20020a056402378a00b0056bed786b9csm3223717edb.17.2024.03.25.12.12.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Mar 2024 12:12:32 -0700 (PDT) Date: Mon, 25 Mar 2024 19:12:29 +0000 From: Jonthan Haslam To: Ingo Molnar Cc: linux-trace-kernel@vger.kernel.org, andrii@kernel.org, bpf@vger.kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] uprobes: reduce contention on uprobes_tree access Message-ID: References: <20240321145736.2373846-1-jonathan.haslam@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hi Ingo, > > This change has been tested against production workloads that exhibit > > significant contention on the spinlock and an almost order of magnitude > > reduction for mean uprobe execution time is observed (28 -> 3.5 microsecs). > > Have you considered/measured per-CPU RW semaphores? No I hadn't but thanks hugely for suggesting it! In initial measurements it seems to be between 20-100% faster than the RW spinlocks! Apologies for all the exclamation marks but I'm very excited. I'll do some more testing tomorrow but so far it's looking very good. Thanks again for the input. Jon.