Received-SPF: pass (google.com: domain of linux-kernel+bounces-3092-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: Hyeongtak Ji <hyeongtak.ji@sk.com>
To: gourry.memverge@gmail.com
Cc: Hasan.Maruf@amd.com,
	Jonathan.Cameron@Huawei.com,
	akpm@linux-foundation.org,
	arnd@arndb.de,
	bp@alien8.de,
	corbet@lwn.net,
	dan.j.williams@intel.com,
	dave.hansen@linux.intel.com,
	emirakhur@micron.com,
	fvdl@google.com,
	gregory.price@memverge.com,
	hannes@cmpxchg.org,
	haowang3@fb.com,
	hasanalmaruf@fb.com,
	hezhongkun.hzk@bytedance.com,
	honggyu.kim@sk.com,
	hpa@zytor.com,
	hyeongtak.ji@sk.com,
	jgroves@micron.com,
	john@jagalactic.com,
	linux-api@vger.kernel.org,
	linux-doc@vger.kernel.org,
	linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-mm@kvack.org,
	luto@kernel.org,
	mhocko@kernel.org,
	mhocko@suse.com,
	mingo@redhat.com,
	peterz@infradead.org,
	rakie.kim@sk.com,
	ravis.opensrc@micron.com,
	seungjun.ha@samsung.com,
	sthanneeru@micron.com,
	tglx@linutronix.de,
	tj@kernel.org,
	vtavarespetr@micron.com,
	x86@kernel.org,
	ying.huang@intel.com,
	kernel_team@skhynix.com
Subject: RE: [PATCH v3 00/11] mempolicy2, mbind2, and weighted interleave
Date: Mon, 18 Dec 2023 16:07:48 +0900
Message-Id: <20231218070750.2123-1-hyeongtak.ji@sk.com>
In-Reply-To: <20231213224118.1949-1-gregory.price@memverge.com>
References: <20231213224118.1949-1-gregory.price@memverge.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Hi Gregory,

Thank you for the v3 patch.

Gregory Price <gourry.memverge@gmail.com> write:

[snip]

> =====================================================================
> Performance tests - MLC
> From Ravi Jonnalagadda <ravis.opensrc@micron.com>
> 
> Workload:                               W2
> Data Signature:                         2:1 read:write
> DRAM only bandwidth (GBps):             298.8
> DRAM + CXL (default interleave) (GBps): 113.04
> DRAM + CXL (weighted interleave)(GBps): 412.5
> Gain over DRAM only:                    1.38x
> 
> Workload:                               W5
> Data Signature:                         1:1 read:write
> DRAM only bandwidth (GBps):             273.2
> DRAM + CXL (default interleave) (GBps): 117.23
> DRAM + CXL (weighted interleave)(GBps): 382.7
> Gain over DRAM only:                    1.4x

I've run XSBench based on the v3 patch and got numbers below. I used
your sample numactl extension from here:
Link: https://github.com/gmprice/numactl/tree/weighted_interleave_master

Performance tests – XSBench
NUMA node 0: 56 logical cores, 128 GB memory
NUMA node 2: 96 GB CXL memory

  1. dram only
  $ numactl -membind 0 ./XSBench -s XL –p 5000000
  Threads:     56
  Runtime:     36.235 seconds
  Lookups:     170,000,000
  Lookups/s:   4,691,618
 
  2. default interleave
  $ numactl –-interleave 0,2 ./XSBench –s XL –p 5000000
  Threads:     56
  Runtime:     55.243 seconds
  Lookups:     170,000,000
  Lookups/s:   3,077,293

  3. weighted interleave
  $ numactl --weighted --interleave 0,2 ./XSBench –s XL –p 5000000
  Threads:     56
  Runtime:     29.262 seconds
  Lookups:     170,000,000
  Lookups/s:   5,809,513

In terms of runtime, weighted-interleaving shows 1.19x improvement
compared to dram only, and 1.47x compared to default interleave.  I’ve
repeatedly run XSBench and have not observed any significant variations
across the runs.

Kind regards,
Hyeongtak