Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC PATCH 00/13] x86 User Interrupts support
To:     Greg KH <gregkh@linuxfoundation.org>
CC:     "Hansen, Dave" <dave.hansen@intel.com>,
        "x86@kernel.org" <x86@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Luck, Tony" <tony.luck@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H . Peter Anvin" <hpa@zytor.com>,
        "Lutomirski, Andy" <luto@kernel.org>, Jens Axboe <axboe@kernel.dk>,
        Christian Brauner <christian@brauner.io>,
        Peter Zijlstra <peterz@infradead.org>,
        Shuah Khan <shuah@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
        Jonathan Corbet <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "Kammela, Gayatri" <gayatri.kammela@intel.com>,
        "Zeng, Guang" <guang.zeng@intel.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        "Witt, Randy E" <randy.e.witt@intel.com>,
        "Shankar, Ravi V" <ravi.v.shankar@intel.com>,
        "Thomas, Ramesh" <ramesh.thomas@intel.com>,
        "linux-api@vger.kernel.org" <linux-api@vger.kernel.org>,
        "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>
References: <20210913200132.3396598-1-sohil.mehta@intel.com>
 <c08f38db-77da-c50e-23f7-b3a76688deeb@intel.com>
 <BYAPR11MB33203044CD5D7413846655F9E5DA9@BYAPR11MB3320.namprd11.prod.outlook.com>
 <YUxwuR4V+kwk1L34@kroah.com>
From:   Sohil Mehta <sohil.mehta@intel.com>
Message-ID: <e374787c-c376-be9a-b2e1-8739b36f69fc@intel.com>
Date:   Thu, 23 Sep 2021 16:09:07 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Firefox/78.0 Thunderbird/78.14.0
In-Reply-To: <YUxwuR4V+kwk1L34@kroah.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VmtaVkhWK05NT1ZvWXF4MU9yMDZ0cjF2MGx6eHNIN1VvOUhLd2FEdk4veS9U?=
 =?utf-8?B?TURXTjZ6VGhDZ2t6NnRHdWNnRU1NZzVCZjRETWtjT2ZJNS9uV21JUVZ4SGdj?=
 =?utf-8?B?TGIyRjN2c3ZrQVl3YVpvSTZ2ZXRRaUd4Mk9GVWJnZDU2ZnZUREdpZnNBTU9Y?=
 =?utf-8?B?eTZsU3NmYnVIdERnOFpybGRFVW5pbXA5WG1ubWdnUng3S0tvVHZvdW55ZVpx?=
 =?utf-8?B?QjFtVXpVR0hQQ25mNGRueVlGR2xzYzRkenlPU1EzQUQ0Sy9hazdLWFBIUzh3?=
 =?utf-8?B?bDZCQ2ZZQnFyVEJJQ2duVlBkRHJEeWx4Z1ZoaStFRStCcXY3MWtTeTdTcEU3?=
 =?utf-8?B?Sy9KdjZEWUtubFRGdmR4NXdyZzZJTXF6RUZkQWJ0L3czVHVubEV5c25MT3RN?=
 =?utf-8?B?eVN1WkZTVm9JSWMwL2s0MlJWMFNFUURaeWdWZlh2MVZ0T3puZFNjYllBc3Fh?=
 =?utf-8?B?LzRnYThvSU45VndqU3NEK2dBZGVLYkdvUU1ubm9oTDBLVndmQWFuUEhhSitj?=
 =?utf-8?B?VFpla2tvblVSbUtqTnJyUkJHT2loYTV3ZFl3aVpMME13OWUxTWhFdTYxWnY4?=
 =?utf-8?B?cmV4eUxWaEN2bUsxMjd5WHhLV1pzUzlWMVk4TDRCYTVjMzBYR3NxckJOaVJY?=
 =?utf-8?B?NkE1akFsSEpXckJ3WTQ2SitscGY0R1JydWMwOGxrTlJaYTAycUpjVEw5cUs5?=
 =?utf-8?B?RkR0R3dwSit0ampwVUYwNnRJK0V3aW5nWDJuSGp1S2tYYnNhT0lrLzcwU0VH?=
 =?utf-8?B?b1lyazNRWXhTWHUvZCtRelM0UVNBRHF5YnlWWEdlSzhhZXdkSitmRWtzOFF0?=
 =?utf-8?B?Q0dmTnp3c1hzczVPNXMydVYxVW95NFdIampiYzRIOGNFZ3ZONW9NMXI3NitF?=
 =?utf-8?B?bUxVQWtKZlZaNXRlM1NGZk5GSy8wRnQ5S1AzYXBxNU0xeHMxbXlIREJsRHRi?=
 =?utf-8?B?TDlKUGRUaGhnZHRiWTg3eC9qRi9WK1VOdHdsTHE1a252aENsdnZONEoxT0I0?=
 =?utf-8?B?MTBMSGl4cXRzdHJpeUVpZmozZEc2VmM2WHBoclY1Lzg1d2ZrZG1qM1hveExQ?=
 =?utf-8?B?Y2tkYjU1aFZTMXkva2VTamxYUnAwOHUvbW5oOEVGUTRVZ1VGaHgzZEp6Vy9P?=
 =?utf-8?B?RXZFUVFkZ3Jpd251TStFVmM2VkcyNTNnT0dWQmVaVXF3RzVQVHMwbE8zQUJQ?=
 =?utf-8?B?U0xvblhvd0tabHVqZmhYZ0dnSzFGRDQxSUhSR1A3NEZ5bG1DYXlSM3Y3Wk93?=
 =?utf-8?B?c3JRZlJIUXFHVmg4RXQxRzdCRkZDS20zYk9sdkNZeDZKWTJIYlc3dlJwbXBa?=
 =?utf-8?B?R2FPWktkb3gxc3dLZTRUSFlVVVBCcGtBZTJvZGxVUGVSdEQrZ3JLcHpWM2Rj?=
 =?utf-8?B?ZXEwZm9sOTdESG5pZUpObnk0M1NMNTdvcDlKai9FT29pZ2dHREp4a1RtYzho?=
 =?utf-8?B?ZTE1WElEK0dBb2NuUzYvMXBzengrLzQ5NUJ1VFpiL0N6T1pHQWZneXhuY3hs?=
 =?utf-8?B?blp4dEFacWVsWnc5b0h3cStWOFlIZHFmY1AzSWhiajZQeXhVNHFPbFhqakhK?=
 =?utf-8?B?VnlkYnhEcC9NREFCOTNScXVYTWQyazJFWFpERUoyZUVjb1lneWFMTFljbUdT?=
 =?utf-8?B?VERTckkrTHBJY1VuVGlFUFl3ZkpCRmx1VjBWTWRoZDBnRUVlekZGZTlBZklw?=
 =?utf-8?B?ejZ5WnM5OHRmZ3VLeUVqMEJKaVJ6NDg4eittVytRazY5aEJCMjVyajllZFZY?=
 =?utf-8?Q?O9aKVC+tIRvVokmZb5Jq3qqHLVvnc07WIHzGO73?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 546c02ff-8031-47ed-592e-08d97ee72650
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3320.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2021 23:09:09.7429
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Y7z8xJuUeFZpcrEROR0pLyf5apsNV5cKPF0B7W0ZkylH1kI8lRavuk4zhE3OdYCgzf71UzCsgrUs4WvU0CQUuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3781
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 9/23/2021 5:19 AM, Greg KH wrote:
> On Tue, Sep 14, 2021 at 07:03:36PM +0000, Mehta, Sohil wrote:
>
> Here is the updated table:
> +---------------------+-------------------------+
> | IPC type            |   Relative Latency      |
> |                     |(normalized to User IPI) |
> +---------------------+-------------------------+
> | User IPI            |                     1.0 |
> | User IPI (blocked)  |                     8.9 |
> | Signal              |                    14.8 |
> | Eventfd             |                     9.7 |
> | Pipe                |                    16.3 |
> | Domain              |                    17.3 |
> +---------------------+-------------------------+
> Relative is just that, "relative".  If the real values are extremely
> tiny, then relative is just "this goes a tiny tiny bit faster than what
> you have today in eventfd", right?
>
> So how about "absolute"?  What are we talking here?

Thanks Greg for reviewing the patches.

The reason I have not included absolute numbers is that on a 
pre-production platform it could be misleading. The data here is more of 
an approximation with the final performance expected to trend in this 
direction.

I have used the term "relative" only to signify that this is comparing 
User IPI with others.

Let's say, if eventfd took 9.7 usec on a system then User IPI (running) 
would take 1 usec. So it would still be a 9x improvement.

But, I agree with your point. This is only a micro-benchmark performance 
comparison. The overall gain in a real workload would depend on how it 
uses IPC.

+---------------------+------------------------------+
| IPC type            |       Example Latency        |
|                     |        (micro seconds)       |
+---------------------+------------------------------+
| User IPI (running)  |                     1.0 usec |
| User IPI (blocked)  |                     8.9 usec |
| Signal              |                    14.8 usec |
| Eventfd             |                     9.7 usec |
| Pipe                |                    16.3 usec |
| Domain              |                    17.3 usec |
+---------------------+------------------------------+


> And this is really only for the "one userspace task waking up another
> userspace task" policies.  What real workload can actually use this?

A User IPI sender could be registered to send IPIs to multiple targets. 
But, there is no broadcast mechanism, so it can only target one receiver 
everytime it executes the SENDUIPI instruction.

Thanks,

Sohil

> thanks,
>
> greg k-h