Date: Mon, 16 Nov 2015 14:39:04 -0200
From: Marcelo Tosatti <mtosatti@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Vikas Shivappa <vikas.shivappa@intel.com>, Tejun Heo <tj@kernel.org>,
        Yu Fenghua <fenghua.yu@intel.com>, linux-kernel@vger.kernel.org
Subject: Re: [PATCH RFC] ioctl based CAT interface
Message-ID: <20151116163903.GA28870@amt.cnet>
References: <20151113163933.GA10222@amt.cnet>
 <20151113165100.GI17308@twins.programming.kicks-ass.net>
 <20151113173303.GB13490@amt.cnet>
 <20151116090756.GM17308@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20151116090756.GM17308@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.21 (2010-09-15)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2165
Lines: 68

On Mon, Nov 16, 2015 at 10:07:56AM +0100, Peter Zijlstra wrote:
> On Fri, Nov 13, 2015 at 03:33:04PM -0200, Marcelo Tosatti wrote:
> > On Fri, Nov 13, 2015 at 05:51:00PM +0100, Peter Zijlstra wrote:
> > > On Fri, Nov 13, 2015 at 02:39:33PM -0200, Marcelo Tosatti wrote:
> > > > + * 	* one tcrid entry can be in different locations 
> > > > + * 	  in different sockets.
> > > 
> > > NAK on that without cpuset integration.
> > > 
> > > I do not want freely migratable tasks having radically different
> > > performance profiles depending on which CPU they land.
> > 
> > Ok, so, configuration:
> > 
> > 
> > Socket-1				Socket-2
> > 
> > pinned thread-A with			100% L3 free
> > 80% of L3 
> > reserved
> > 
> > 
> > So it is a problem if a thread running on socket-2 is scheduled to 
> > socket-1 because performance is radically different, fine.
> > 
> > Then one way to avoid that is to not allow freely migratable tasks
> > to move to Socket-1. Fine.
> > 
> > Then you want to use cpusets for that.
> > 
> > Can you fill in the blanks what is missing here?
> 
> I'm still not seeing what the problem with CAT-cgroup is.
> 
> /cgroups/cpuset/
>   socket-1/cpus = $socket-1
>            tasks = $thread-A
> 
>   socket-2/cpus = $socket-2
>            tasks = $thread-B
> 
> /cgroups/cat/
>   group-A/bitmap = 0x3F / 0xFF
>   group-A/tasks = $thread-A
> 
>   group-B/bitmap = 0xFF / 0xFF
>   group-B/tasks = $thread-B
> 
> 
> That gets you thread-A on socket-1 with 6/8 of the L3 and thread-B on
> socket-2 with 8/8 of the L3.

Going that route, might as well expose the region shared with HW
to userspace and let userspace handle the problem of contiguous free regions,
which means the cgroups bitmask maps one-to-one to HW bitmap.

All is necessary then is to modify the Intel patches to

1) Support bitmaps per socket.
2) Remove hierarchical support.

3) Lazy enforcement (which can be done later as an improvement).

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/