Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756817AbbKRTzw (ORCPT ); Wed, 18 Nov 2015 14:55:52 -0500 Received: from mga02.intel.com ([134.134.136.20]:51698 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755237AbbKRTzu convert rfc822-to-8bit (ORCPT ); Wed, 18 Nov 2015 14:55:50 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,314,1444719600"; d="scan'208";a="823413894" From: "Auld, Will" To: Luiz Capitulino , Thomas Gleixner CC: LKML , Peter Zijlstra , "x86@kernel.org" , Marcelo Tosatti , "Shivappa, Vikas" , "Tejun Heo" , "Yu, Fenghua" , "Dugger, Donald D" , "riel@redhat.com" , "Luck, Tony" Subject: RE: [RFD] CAT user space interface revisited Thread-Topic: [RFD] CAT user space interface revisited Thread-Index: AQHRIji1pO80F6Dp0UunRHw6vw1eqp6iMXwg Date: Wed, 18 Nov 2015 19:55:32 +0000 Message-ID: <96EC5A4F3149B74492D2D9B9B1602C274EE8CDA4@ORSMSX105.amr.corp.intel.com> References: <20151118143822.3900dfff@redhat.com> In-Reply-To: <20151118143822.3900dfff@redhat.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.22.254.140] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3542 Lines: 94 +Tony > -----Original Message----- > From: Luiz Capitulino [mailto:lcapitulino@redhat.com] > Sent: Wednesday, November 18, 2015 11:38 AM > To: Thomas Gleixner > Cc: LKML; Peter Zijlstra; x86@kernel.org; Marcelo Tosatti; Shivappa, Vikas; Tejun > Heo; Yu, Fenghua; Auld, Will; Dugger, Donald D; riel@redhat.com > Subject: Re: [RFD] CAT user space interface revisited > > On Wed, 18 Nov 2015 19:25:03 +0100 (CET) Thomas Gleixner > wrote: > > > We really need to make this as configurable as possible from userspace > > without imposing random restrictions to it. I played around with it on > > my new intel toy and the restriction to 16 COS ids (that's 8 with CDP > > enabled) makes it really useless if we force the ids to have the same > > meaning on all sockets and restrict it to per task partitioning. > > > > Even if next generation systems will have more COS ids available, > > there are not going to be enough to have a system wide consistent view > > unless we have COS ids > nr_cpus. > > > > Aside of that I don't think that a system wide consistent view is > > useful at all. > > This is a great writeup! I agree with everything you said. > > > So now to the interface part. Unfortunately we need to expose this > > very close to the hardware implementation as there are really no > > abstractions which allow us to express the various bitmap > > combinations. Any abstraction I tried to come up with renders that > > thing completely useless. > > > > I was not able to identify any existing infrastructure where this > > really fits in. I chose a directory/file based representation. We > > certainly could do the same with a syscall, but that's just an > > implementation detail. > > > > At top level: > > > > xxxxxxx/cat/max_cosids <- Assume that all CPUs are the same > > xxxxxxx/cat/max_maskbits <- Assume that all CPUs are the same > > xxxxxxx/cat/cdp_enable <- Depends on CDP availability > > > > Per socket data: > > > > xxxxxxx/cat/socket-0/ > > ... > > xxxxxxx/cat/socket-N/l3_size > > xxxxxxx/cat/socket-N/hwsharedbits > > > > Per socket mask data: > > > > xxxxxxx/cat/socket-N/cos-id-0/ > > ... > > xxxxxxx/cat/socket-N/cos-id-N/inuse > > /cat_mask > > /cdp_mask <- Data mask if CDP enabled > > > > Per cpu default cos id for the cpus on that socket: > > > > xxxxxxx/cat/socket-N/cpu-x/default_cosid > > ... > > xxxxxxx/cat/socket-N/cpu-N/default_cosid > > > > The above allows a simple cpu based partitioning. All tasks which do > > not have a cache partition assigned on a particular socket use the > > default one of the cpu they are running on. > > > > Now for the task(s) partitioning: > > > > xxxxxxx/cat/partitions/ > > > > Under that directory one can create partitions > > > > xxxxxxx/cat/partitions/p1/tasks > > /socket-0/cosid > > ... > > /socket-n/cosid > > > > The default value for the per socket cosid is COSID_DEFAULT, which > > causes the task(s) to use the per cpu default id. > > I hope I've got all the details right, but this proposal looks awesome. > There's more people who seem to agree with something like this. > > Btw, I think it should be possible to implement this with cgroups. But I too don't > care that much on cgroups vs. syscalls. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/