Received: by 2002:a05:7412:3290:b0:fa:6e18:a558 with SMTP id ev16csp898596rdb; Fri, 26 Jan 2024 14:39:21 -0800 (PST) X-Google-Smtp-Source: AGHT+IHSo7zrlj8K2vxgU65BpCE2MKfd/Co2+vemTJ4f290+UAmhWZjDDoT4ce7DdBBZohZyxiBb X-Received: by 2002:a05:6402:26d4:b0:55c:cdbf:b741 with SMTP id x20-20020a05640226d400b0055ccdbfb741mr542939edd.6.1706308761859; Fri, 26 Jan 2024 14:39:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706308761; cv=pass; d=google.com; s=arc-20160816; b=r1b4+NqmaJ2GugOHIhJgC/DzodzJi828T/5SWdNjPDC9BSzyYFudLaQXTj4g5Jl6s8 /4XxVdb1RpyYdQYS9X1ayq64ji6ZaNxAW6sDEAWODcJiEX1n4nHycyPAriFhTU8p2dcD 6skW0DMqK4GNKWbBMcjiIlbeIBVjWLhuJqdaBCNOvDYhW1AmAoMIKyiV2lh/8vBr8dvb qk6zYpiMz93Qozysjz6Up/gcD6S7H2miyfR4JQx4vROsz7ehHG/wkTvAn3ykhHeTgFEG QkLkgvPqWzeGA5z3qwXXmC9OYa5J64YqxmcAckmg9GdF74P5AeEyaBBAcNaF11QbZOoT SUog== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=Em4/kfmdug+pQ48RHT6Ptb0aualz9KCiGa765nhybis=; fh=RNLImOVVerV85xOTLs4+HReepGmzxDvGnHWtnxiQ3pE=; b=wcneQaKS9Aoof5ZjPWTwSL22yz4aThC7VzJ+i8MRdBwvvGSdCKcVxTDlK2R1a8ot+P 6aFQrOyjLd0lRBP5U8zF4d/lk/3Jr1ReXDczSUm782UWrJLW6iXduPK6/UNNcESfd/7w 9xz2xmV1CdFj+zsCoSDMz25BNqk3BDh/jNZFtKfFrXdrXgK4YoEq/gMa24dHWl7nUX+6 +LyJ9ReZIfUvi+8RK806CQVRQJxfMjJhqaSY2XLrhwDJuTG/SzWkGpldfUQdDrVBexTF F09vI4vE+25oR3j20Ge2dsXBQS39nnt9y7oV9JAH//OKJYfA5eTBYxXMLMEgN/vvpd2g ydPg== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Va6kkDTa; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-40785-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-40785-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id ij12-20020a056402158c00b0055e2da94e2esi325641edb.292.2024.01.26.14.39.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Jan 2024 14:39:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-40785-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Va6kkDTa; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-40785-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-40785-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9C2771F251FE for ; Fri, 26 Jan 2024 22:39:21 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3CEB250258; Fri, 26 Jan 2024 22:38:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Va6kkDTa" Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D9E9E2C6A0; Fri, 26 Jan 2024 22:38:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706308730; cv=none; b=OsGy/km+b8B80MjqRWbz9/I1bJUughvRqZ4s2N39GyMNY6OPSkWsZS9MzBy7jhEB9hCx/cr82UwqxRDK1G1+oPv1aazNR1rYqVJkpGfJ0gQ5dAi2OEANGZFIi7vQTJW2hFkodgLvbJwBYczYY+2PNQN4tupxwuT5REfpphF/iqs= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706308730; c=relaxed/simple; bh=Hp6f/GZsgmHvqRxAvmLEAtm19UpuDpvz8X+HdHnCBV0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PoRicdQH+N4bXI9byzocHEb3uHe30XvmUjy74June7vMkC9t1qI6+aDiOFoBQmyHW4Zpd8xXC6SPSabzzSHdU/yXJE120vHbHrgFZSYUU2JePu8EdXahd3P68Iqbancrs+B0b10irQDvuEo+qIbv4Exsy0U+2f4Hw9Kp55cNcBs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Va6kkDTa; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706308727; x=1737844727; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hp6f/GZsgmHvqRxAvmLEAtm19UpuDpvz8X+HdHnCBV0=; b=Va6kkDTalY/i4I3PAXqGbNhqyITuo3ywhwzH9EIzlLDLhC9rEg857OET zzzQGp/5D+Xch6ZNSRpydOzF+keQxLVr0t7MDaVCPl17VK5r1ELWooDfS +GJQZlfNYnT/9bVdL7iuNqdHbXFTUSsCAhVnikiBYgO0aNtsixK+UDGhr 04KdPRgELq17PEr8UiPW+f9X+MiVfbwdFGgoF+RjX4if+TFsk7wa1K09U +ZZjfAmAv3UcuEbvRoeXDH12NX4yFiTORC1EZ3Ajv9oaCaLHFDzxo8Jp7 dHA3eXQkuHz5htFIxWOH6Ni4FdjO5aeOBRo9ALdsxPUsApaWO5ZvsG6Av Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10964"; a="9707634" X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="9707634" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 14:38:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,220,1701158400"; d="scan'208";a="35560245" Received: from agluck-desk3.sc.intel.com ([172.25.222.74]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2024 14:38:46 -0800 From: Tony Luck To: Borislav Petkov Cc: Fenghua Yu , Reinette Chatre , Peter Newman , Jonathan Corbet , Shuah Khan , Shaopeng Tan , James Morse , Jamie Iles , Babu Moger , Randy Dunlap , x86@kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, patches@lists.linux.dev, Tony Luck Subject: [PATCH v14 0/8] Add support for Sub-NUMA cluster (SNC) systems Date: Fri, 26 Jan 2024 14:38:29 -0800 Message-ID: <20240126223837.21835-1-tony.luck@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231204185357.120501-1-tony.luck@intel.com> References: <20231204185357.120501-1-tony.luck@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit The Sub-NUMA cluster feature on some Intel processors partitions the CPUs that share an L3 cache into two or more sets. This plays havoc with the Resource Director Technology (RDT) monitoring features. Prior to this patch Intel has advised that SNC and RDT are incompatible. Some of these CPU support an MSR that can partition the RMID counters in the same way. This allows monitoring features to be used. With the caveat that users must be aware that Linux may migrate tasks more frequently between SNC nodes than between "regular" NUMA nodes, so reading counters from all SNC nodes may be needed to get a complete picture of activity for tasks. Cache and memory bandwidth allocation features continue to operate at the scope of the L3 cache. Signed-off-by: Tony Luck --- Changes since v13: Rebase to tip x86/cache fc747eebef73 ("x86/resctrl: Remove redundant variable in mbm_config_write_domain()") Applied all Reviewed and Tested tags Tony Luck (8): x86/resctrl: Prepare for new domain scope x86/resctrl: Prepare to split rdt_domain structure x86/resctrl: Prepare for different scope for control/monitor operations x86/resctrl: Split the rdt_domain and rdt_hw_domain structures x86/resctrl: Add node-scope to the options for feature scope x86/resctrl: Introduce snc_nodes_per_l3_cache x86/resctrl: Sub NUMA Cluster detection and enable x86/resctrl: Update documentation with Sub-NUMA cluster changes Documentation/arch/x86/resctrl.rst | 25 +- include/linux/resctrl.h | 85 +++-- arch/x86/include/asm/msr-index.h | 1 + arch/x86/kernel/cpu/resctrl/internal.h | 66 ++-- arch/x86/kernel/cpu/resctrl/core.c | 433 +++++++++++++++++----- arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 58 +-- arch/x86/kernel/cpu/resctrl/monitor.c | 68 ++-- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 26 +- arch/x86/kernel/cpu/resctrl/rdtgroup.c | 149 ++++---- 9 files changed, 629 insertions(+), 282 deletions(-) base-commit: fc747eebef734563cf68a512f57937c8f231834a -- 2.43.0