Received: by 2002:ac0:a679:0:0:0:0:0 with SMTP id p54csp677719imp; Wed, 20 Feb 2019 07:11:22 -0800 (PST) X-Google-Smtp-Source: AHgI3IawkNcitC+QVcIr5ME2K26lUiDeVZUwp4hibJq3rbFlaZs+7c7PRDqSfVWX97Wzuapx/U8l X-Received: by 2002:a17:902:5a5:: with SMTP id f34mr37403789plf.161.1550675482331; Wed, 20 Feb 2019 07:11:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550675482; cv=none; d=google.com; s=arc-20160816; b=ZCSU6xnDm7o1rOW4Xte1C2mxNiXenHnQdIYoB90vsQmAYQU7+G1IBUNRPFJqEOndqU nf7Iv36DLW+laWPBzdgAroV6Z6gQ74y7GyFcUO17PCpW8YYXlnCSPmv8YTa4FDO12sbB 4lTLi2I7K13iWlI2lKBEgLsJfwLHehLRvj9qdOJ7oZZm+S2+QqQN59Hdf4EY3IYKxv57 HG4uuw8rCn9kHKrxUzm5YMosJCM1hJsRprszEBeO+5kVOMDZ9zf/jf+Lqy/g/K9D43D0 +MwD3o+n+0l0s1hwSnoA2NlcuzY1H2E2zAjtmwmItqdRmTBxbzI0Ms/AZjiO5QN1sM6J k2tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version; bh=T7KxhyUidV5tFwycItsdaLf++bwcsQHmwpqW7sMcBjk=; b=tnba775G0cZ1SGGHBz47Nnw4b9NSFlx0W/lZ8jYSJff0rd88xNJ/ka4b4Ic8K8XTmw 3J3mG9e2cN0QfWOlMt20Qfy+0x7eeqmI36cXYNwyHvpaZYn906jjTXeZHEbGGrhpBnBR geCnIZ2GeaSkWm9mVoLl/yWeT/4HdmU3ItSRP3wvSZMyYWWZdRs27foclVpwu5f93OeC aNZQWLzt1nElnyBBYqLiF267/WR9Dxe0RFT2u000wGCD+gxCrub7sXS8PDxFn4JZ+L3V q01B00qrXGjem8MKs/1YTW0y5u8VH7MR2EzIPzI7Ytnzrqgmm8V+Zem/woJHOb/4jcAF nFJA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v2si2714571plo.212.2019.02.20.07.11.05; Wed, 20 Feb 2019 07:11:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727104AbfBTPJC (ORCPT + 99 others); Wed, 20 Feb 2019 10:09:02 -0500 Received: from mail-ed1-f68.google.com ([209.85.208.68]:37685 "EHLO mail-ed1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725890AbfBTPJC (ORCPT ); Wed, 20 Feb 2019 10:09:02 -0500 Received: by mail-ed1-f68.google.com with SMTP id m12so20123337edv.4; Wed, 20 Feb 2019 07:09:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T7KxhyUidV5tFwycItsdaLf++bwcsQHmwpqW7sMcBjk=; b=Kt8aZ/xnRX1sQF/lAWbgUM2H0C09Y/nDY3H1bo3QZOe+lRC/cgL7FoE6vTFsjX1Ew9 bPh49rI9PZyl1KIXVZVgiJNPa8B3RWfH3VHjSOKJX9DCp+vnrDnBEr+x3PnchBvDijbg PEt4DI2L/0RryPYdcQCNAbbZahhvTDRLo8XDEO9tNPr0T9wBLD7iMXN/KV2e4prsUhVF Nae60jBN4ckwVDqoxzk5B7fYgt2wKPn3L7omfnyW/5PeseKvkCGP6eZAbcRtbcN7M3b8 +QSrCFg0lMwqHLDSeH/5mWHl0Y3WfNvZ2XsJiiCSSkUfGeMTi0Hb1+gjo8pN237RP2Pv 57dA== X-Gm-Message-State: AHQUAuaAMzDL6eRR7btb71UfgdcOJC9atX0xQgbRKWqI8ocjnHFDpLLK KHfl2Hbc4n3JLsZENe9XG+zSAK9jgQXPyOdk3+0= X-Received: by 2002:a17:906:2a8b:: with SMTP id l11mr23954673eje.196.1550675339791; Wed, 20 Feb 2019 07:08:59 -0800 (PST) MIME-Version: 1.0 References: <635b2bf8b1151a191cd9299276b75791a818c0c2.1550545163.git.len.brown@intel.com> <07d2908dc72bf964b27380999e1c826587d69136.1550545163.git.len.brown@intel.com> <20190220105542.GB17969@hirez.programming.kicks-ass.net> In-Reply-To: <20190220105542.GB17969@hirez.programming.kicks-ass.net> From: Len Brown Date: Wed, 20 Feb 2019 10:08:48 -0500 Message-ID: Subject: Re: [PATCH 03/11] x86 topology: Add CPUID.1F multi-die/package support To: Peter Zijlstra Cc: X86 ML , linux-kernel@vger.kernel.org, Len Brown , linux-doc@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Thanks for the comments, Peter. I'll update the patch to address the syntax points. (Maybe checkpatch.pl should be updated to reflect your preferences?). About macros vs C. I agree with your preference. I used macros to be consistent with the existing code, and to be as backport friendly as possible. (a number of distros need to pull these patches into their supported kernels) Sure, I'm willing to write in a cosmetic-only patch, after the functional changes are upstream. > It would've been nice to have the CPUID instruction 1F leaf reference > 3B-3.9 in the SDM, and maybe mention this here too. I didn't mention SDM sections because they change -- leaving stale pointers in our commit messages. The SDM is re-published 4 times per year. > Also, figure 8-6 uses Q,R ID, without prior mention. Yeah, the tech-writer took my real example and turned it into a generic example. Probably a good idea, even if not gracefully executed. The point of undefined "Q" and "R" are that a new level can be invented at any time in the future, and the existing code that doesn't know about future levels should still function properly. The back-story is that Leaf-B was supposed to work this way, but the original SDM code example was hard-coded to assume 3-levels. Plenty of software was hard-coded and would have broken if we had actually added new levels to Leaf-B. And so Leaf-B had to be "forked" into Leaf-1F, with the implicit contract that only new code that can function properly in the face of unknown levels parses leaf-1F. Yes, the parsing routine in the Linux Kernel will work fine in the face of future unknown levels. Some utilities (such as cpuid), would have actually crashed if we added levels to Leaf-B. > You haven't explained, and I can't readily find it in the SDM either, > how these topology bits relate to caches and interconnects. > > Will these die thingies share LLC, or will LLC be per die. Will NUMA > span dies or not. Excellent question. Cache enumeration in Leaf-4 is totally unchanged. ACPI NUMA tables are totally unchanged. From a scheduler point of view, imagine that a SKX system with 4 die in 4 packages was mechanically re-designed so that those 4 die resided in 2 double-sized packages. They may have tweaked the links between the die, but logically it is identical and compatible, and the legacy kernel will function properly. So the effect of Leaf B,1F is that it defines the scope of MSRs. eg. what processors does a die-scope MSR cover. That is why the rest of the patch is about sysfs topology, and about package MSR scope. Yes, there will be more exotic MSR situations in future products -- the first ones are pretty simple -- something called a package-scope-MSR in the SDM today becomes a die-scope-MSR in this generation on a multi-die/package system. It also reflects how many packages appear in sysfs, and this can effect licensing of some kinds of software. thanks, Len Brown, Intel Open Source Technology Center