Date: Tue, 27 Jun 2017 19:44:11 +0200
From: Borislav Petkov <bp@alien8.de>
To: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org, leo.duran@amd.com,
        yazen.ghannam@amd.com, Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH 1/2] x86/CPU/AMD: Present package as die instead of socket
Message-ID: <20170627174411.gheip4jmra2ihuhq@pd.tnic>
References: <1498545653-6755-1-git-send-email-suravee.suthikulpanit@amd.com>
 <1498545653-6755-2-git-send-email-suravee.suthikulpanit@amd.com>
 <20170627104803.wlhsqhaylbeqod37@pd.tnic>
 <bb167d94-828b-2d5b-642a-d56e8db392d0@amd.com>
 <20170627134251.5ztfqqlm3rg6y7ql@pd.tnic>
 <2f3bf9f2-cbca-ca72-219a-13c6faf9d314@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <2f3bf9f2-cbca-ca72-219a-13c6faf9d314@amd.com>
User-Agent: NeoMutt/20170113 (1.7.2)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2352
Lines: 63

On Tue, Jun 27, 2017 at 11:54:12PM +0700, Suravee Suthikulpanit wrote:
> The 8 threads sharing each L3 are already in the same sched-domain1 (MC
> CCX). So, cpu0 is in the same sched-domain1 as cpu1,2,3,64,65,66,67. Here,
> we need the DIE sched-domain because it represents all cpus that are in the
> same NUMA node (since we have one memory controller per DIE).

So this is still confusing. Please drop the "DIE sched-domain" as that is
something you're trying to define and I'm trying to parse what you're
trying to define and why.

> IIUC, for Zen, w/o the DIE sched-domain, the scheduler could try to
> re-balance the tasks from one CCX (schedule group) to another CCX
> across NUMA node, and

CCX, schedule group, NUMA node, ... now my head is spinning. Do you
see what I mean with agreeing on the nomenclature and proper term
definitions first?

> potentially causing unnecessary performance due to remote memory access.
> 
> Please note also that SRAT/SLIT information are used to derive the NUMA
> sched-domains, while the DIE sched-domain is non-NUMA sched-domain (derived
> from CPUID topology extension which is available on newer families).

So let's try to discuss this without using DIE sched-domain, CCX, etc,
and let's start simple.

So in that die graphic:

              ----------------------------
          C0  | T0 T1 |    ||    | T0 T1 | C4
              --------|    ||    |--------
          C1  | T0 T1 | L3 || L3 | T0 T1 | C5
              --------|    ||    |--------
          C2  | T0 T1 | #0 || #1 | T0 T1 | C6
              --------|    ||    |--------
          C3  | T0 T1 |    ||    | T0 T1 | C7
              ----------------------------

you want all those threads to belong to a single scheduling group.
Correct?

Now that thing has a memory controller attached to it, correct?

If so, why is this thing not a logical NUMA node, as described in
SRAT/SLIT?

If not, what does a NUMA node entail on Zen as described by SRAT/SLIT?
I.e., what is the difference between the two things? I.e., how many dies
as above are in a NUMA node?

Now, SRAT should contain the assignment which core belongs to which
node. Why is that not sufficient?

Ok, that should be enough questions for now. Let's start with them.

Thanks.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.