Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755982AbdLVOiI (ORCPT ); Fri, 22 Dec 2017 09:38:08 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:48114 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752170AbdLVOiB (ORCPT ); Fri, 22 Dec 2017 09:38:01 -0500 Subject: Re: [PATCH v3 0/3] create sysfs representation of ACPI HMAT To: "Kogut, Jaroslaw" , Anshuman Khandual , "linux-kernel@vger.kernel.org" , Ross Zwisler References: <20171214021019.13579-1-ross.zwisler@linux.intel.com> <2d6420f7-0a95-adfe-7390-a2aea4385ab2@linux.vnet.ibm.com> <34EF90DF7C7F0647A403B771519912C7F5382CF3@irsmsx111.ger.corp.intel.com> Cc: "Anaczkowski, Lukasz" , "Box, David E" , "Koss, Marcin" , "Koziej, Artur" , "Lahtinen, Joonas" , "Moore, Robert" , "Nachimuthu, Murugasamy" , "Odzioba, Lukasz" , "Wysocki, Rafael J" , "Rafael J. Wysocki" , "Schmauss, Erik" , "Verma, Vishal L" , "Zheng, Lv" , Andrew Morton , Balbir Singh , Brice Goglin , "Williams, Dan J" , "Hansen, Dave" , Jerome Glisse , John Hubbard , Len Brown , Tim Chen , "devel@acpica.org" , "linux-acpi@vger.kernel.org" , "linux-mm@kvack.org" , "linux-nvdimm@lists.01.org" From: Anshuman Khandual Date: Fri, 22 Dec 2017 20:07:31 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <34EF90DF7C7F0647A403B771519912C7F5382CF3@irsmsx111.ger.corp.intel.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 x-cbid: 17122214-0008-0000-0000-000004BA17D9 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17122214-0009-0000-0000-00001E4D4390 Message-Id: X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-12-22_05:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1709140000 definitions=main-1712220205 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2150 Lines: 44 On 12/22/2017 04:01 PM, Kogut, Jaroslaw wrote: >> ... first thinking about redesigning the NUMA for >> heterogeneous memory may not be a good idea. Will look into this further. > I agree with comment that first a direction should be defined how to handle heterogeneous memory system. > >> https://linuxplumbersconf.org/2017/ocw//system/presentations/4656/original/ >> Hierarchical_NUMA_Design_Plumbers_2017.pdf > I miss in the presentation a user perspective of the new approach, e.g. > - How does application developer see/understand the heterogeneous memory system? >From user perspective - Each memory node (with or without CPU) is a NUMA node with attributes - User should detect these NUMA nodes from sysfs (not part of proposal) - User allocates/operates/destroys VMA with new sys calls (_mattr based) > - How does app developer use the heterogeneous memory system? - Through existing and new system calls > - What are modification in API/sys interfaces? - The presentation has possible addition of new system calls with 'u64 _mattr' representation for memory attributes which can be used while requesting different kinds of memory from the kernel > > In other hand, if we assume that separate memory NUMA node has different memory capabilities/attributes from stand point of particular CPU, it is easy to explain for user how to describe/handle heterogeneous memory. > > Of course, current numa design is not sufficient in kernel in following areas today: > - Exposing memory attributes that describe heterogeneous memory system > - Interfaces to use the heterogeneous memory system, e.g. more sophisticated policies > - Internal mechanism in memory management, e.g. automigration, maybe something else. Right, we would need - Representation of NUMA with attributes - APIs/syscalls for accessing the intended memory from user space - Memory management policies and algorithms navigating trough all these new attributes in various situations IMHO, we should not consider sysfs interfaces for heterogeneous memory (which will be an ABI going forward and hence cannot be changed easily) before we get the NUMA redesign right.