Date: Wed, 2 Aug 2017 00:41:57 -0700 (PDT)
From: Shivappa Vikas <vikas.shivappa@intel.com>
To: Thomas Gleixner <tglx@linutronix.de>
cc: Vikas Shivappa <vikas.shivappa@linux.intel.com>,
        vikas.shivappa@intel.com, x86@kernel.org, linux-kernel@vger.kernel.org,
        hpa@zytor.com, peterz@infradead.org, ravi.v.shankar@intel.com,
        tony.luck@intel.com, fenghua.yu@intel.com, eranian@google.com,
        davidcc@google.com, ak@linux.intel.com, reinette.chatre@intel.com
Subject: Re: [PATCH 09/28] x86/intel_rdt/cqm: Add RMID(Resource monitoring
 ID) management
In-Reply-To: <alpine.DEB.2.20.1708012207531.2682@nanos>
Message-ID: <alpine.DEB.2.10.1708020039001.19147@vshiva-Udesk>
References: <1501017287-28083-1-git-send-email-vikas.shivappa@linux.intel.com> <1501017287-28083-10-git-send-email-vikas.shivappa@linux.intel.com> <alpine.DEB.2.20.1708012207531.2682@nanos>
User-Agent: Alpine 2.10 (DEB 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1679
Lines: 45


On Tue, 1 Aug 2017, Thomas Gleixner wrote:

> On Tue, 25 Jul 2017, Vikas Shivappa wrote:
>> Hardware uses RMID(Resource monitoring ID) to keep track of each of the
>> RDT events associated with tasks. The number of RMIDs is dependent on
>> the SKU and is enumerated via CPUID. We add support to manage the RMIDs
>> which include managing the RMID allocation and reading LLC occupancy
>> for an RMID.
>>
>> RMID allocation is managed by keeping a free list which is initialized
>> to all available RMIDs except for RMID 0 which is always reserved for
>> root group. RMIDs goto a limbo list once they are
>> freed since the RMIDs are still tagged to cache lines of the tasks which
>> were using them - thereby still having some occupancy. They continue to
>> be in limbo list until the occupancy < threshold_occupancy. The
>> threshold_occupancy is a user configurable value.
>> OS uses IA32_QM_CTR MSR to read the occupancy associated with an RMID
>> after programming the IA32_EVENTSEL MSR with the RMID.
>>
>> [Tony: Improved limbo search]
>
> The search is smarter, but the smp function call problem per se
> persists. It's still more than 100us worstcase on a BDW box and it's going
> to be worse with the next generation

As an alternative we could have delayed worker threads(hence not in interrupt 
context) every 1s to check the limbo lists using the same optimized search with 
1 bit per RMID.

When a mkdir tries to get RMID if no RMID is in free list and some are present 
in limbo we return -EBUSY and if both free and limbo lists are empty we return 
-ENOSPC.

If that is reasonable I can send an updated patch for the same.

Thanks,
Vikas

>
> Thanks,
>
> 	tglx
>