Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <19641ab0-ab6a-9af7-8c64-34030e187848@siemens.com>
Date:   Thu, 27 Apr 2023 15:14:47 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.9.1
Subject: Re: Unfair qspinlocks on ARM64 without LSE atomics => 3ms delay in
 interrupt handling
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        "Bouska, Zdenek" <zdenek.bouska@siemens.com>,
        Will Deacon <will@kernel.org>,
        Catalin Marinas <catalin.marinas@arm.com>
Cc:     "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-rt-users@vger.kernel.org" <linux-rt-users@vger.kernel.org>,
        Nishanth Menon <nm@ti.com>, Puranjay Mohan <p-mohan@ti.com>
References: <AS1PR10MB567534190B05A4493674173BEB659@AS1PR10MB5675.EURPRD10.PROD.OUTLOOK.COM>
 <87pm7qxrg6.ffs@tglx>
From:   Jan Kiszka <jan.kiszka@siemens.com>
In-Reply-To: <87pm7qxrg6.ffs@tglx>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SjB1MmJiYVkxbHlNQXAzTUU4SGpNdWJmTlRVZ2hwb3lLQ2pFRGp1ZFVvVSt5?=
 =?utf-8?B?a3NPdHNoTDNQcExIRjRzYXFPeFgrdlZwaEZscEJPaDk3REtHZjlBa2lSVnhO?=
 =?utf-8?B?Skhwa0Q5disrVHBBRnRBZ201dEV6RElIQk1uZFRrQ1d3RU9qb1QzQm9CNTM5?=
 =?utf-8?B?dzBiZDBpejMyZlJWa2o2VHBNU1pPUFgxOWF3dWhmZ2hzeG1QdnBjbHBLVi9X?=
 =?utf-8?B?VmJBcVdaWkl6Vnd5dEYrM29uM09TZEwzTWozL25qdmdvMGhUMnJDZkVWenNB?=
 =?utf-8?B?cWwyaHJYbEdDS1lrVW1qTEhZY0E1Zk1LaU5IM29GbVlvWWN0UHFPZVJ2SlZJ?=
 =?utf-8?B?UWNhWTd4d2szM2ViTXJqOThnY1BRbHBJamxacnBpUkN1ZG9YbmcvbngxSmgw?=
 =?utf-8?B?Q28rdmZUMHFmUmRhNzR0MExNN3FRd04zNWw5MlZDNHN6YTFvcFNkT3A4dUlp?=
 =?utf-8?B?bUhsVUdpTVNlRFpVQWR3TGNBNUZRd1Z5SHNQMEs3ek41eVdwMnRCaFFhc2lu?=
 =?utf-8?B?ZnYyKzZGUGx6WDdvQk9BSFFhSS9NcGhQQVd4K3VoRTdrRll6b2M2bCs1eEFU?=
 =?utf-8?B?UU8vTjhXdW9uMVZCNy9ETkIzekhzbWhIaTVxaW53ZkM1Qy9DUWNOeUJKRW93?=
 =?utf-8?B?M1JzVG5lWEh4VW11MXlHZWxZL0tBRzF5UUViUzliUGdwWDZVQjhwNWwzYjNY?=
 =?utf-8?B?Q0FFZUtDRGNsNVpjOGI5Y29hRFB1bG5FbTExV1ZHMUhONURTNWxJdGNJdVdo?=
 =?utf-8?B?QXl1S3dNUVdudWRiTTR5UjlUMnBMTkczajFJS1J1M21qMTdDdS9VVTc0R2s3?=
 =?utf-8?B?NTB6bXJyNFVNREhiRUptTG1rNk5jOEMzb1FTbG9VMEZONmNrMkRUbm9qMFp4?=
 =?utf-8?B?Vy9rUmRSQkpBeW5UMkovb0lvVGUxZDZLWUt3dmxBd285dWRGSzV6Tk51QmVr?=
 =?utf-8?B?aWFuRUpaT2VSMHVpY1Q0U1V4aWJVUnhpb1ZQaWlQMVAwSFlqVFJlTzZXSitR?=
 =?utf-8?B?RUFVNzVnRFoxWHh0b2J5Z2hWTWo1TytHMkVDV3ZMU1BGWFM0eGhzSWhTU0ly?=
 =?utf-8?B?VWxjTUg4MjRuQ1ppaElGczl1ei90cWFCcUVTdmdUTTRXQXZGUi8va2pkQ2Nh?=
 =?utf-8?B?SkVxNFpRa09iNVMvT3VJS0VtaUhFN1VZeUZCVUUxQm0rNGZER2FPRWFneGlB?=
 =?utf-8?B?d1pGK0I5aTlvVnFzTmd6dDlEbXVkYWowQlJXamxCRFJtaGdUaXhZb0dOcXVG?=
 =?utf-8?B?czRxeE9pWXQwbk9uM3ZlUkl4R3lNelJwaGFPZkE3Tm9yVUhTM21DZW9ENHR3?=
 =?utf-8?B?SXZ6ZGg4YmZyc2lJUGlIZW1teGtJRzF4enBaek1FaHNwUGJhbWRFWStEZVc3?=
 =?utf-8?B?NzdNWDdoL0liSFdFZGRFcjByU0F2MnlxdUpYQWZDU0M5QlJVNUFnZEU2VHlX?=
 =?utf-8?B?UEttRm8wSURobUh2WG91amtnbTk1dDhoUjNRYzBsT0JGaHdFa0tsd1I0MTNt?=
 =?utf-8?B?cURJMFdlMy83dUVLL1BWanFyeWJXNzZUZzdiQVpNM1RWeHZzU2NPYklXYnk1?=
 =?utf-8?B?K1QrZDJLU2hQVkZyVGM0Y3JTTE9ualQ2WVIrem9CbzYyTFFoZGdNamU0cDhi?=
 =?utf-8?B?K2xYNTVMcFJVdFZwTzI5RDhteDdmeFp0WGZFQjh4b3JiUVJQTHpyQ1V0LzJr?=
 =?utf-8?B?RkZzYWQ4Z1BnU09zT21RTUNSeXhiOU0zOEc3T2w0S3k1aGJ1dXlDdnBRY0dj?=
 =?utf-8?B?bUNMRGd0d1ZWbGRNWWNJMUZjZ1FZRTE4YTFsbGZOZEE5TDBkWlMwejFnSXNM?=
 =?utf-8?B?VU0vbWo4QUtlemFHWjdEWGdTOHkwVXlJNy9yKzBUL3lkOHhzUDJ3bXl0TFQ2?=
 =?utf-8?B?MktGWktwNjFSV3BWNzNhSmU1cUk2ZThRWm9VZTY5dW5YSDVDQko1dVRtdE9C?=
 =?utf-8?B?eWE0QUVJb0xxeG05aWY4OHZCL3AwSE5TTW9YdzFlT2VCMHhSbytOZk1pRTZl?=
 =?utf-8?B?UXFUdFlNSWhiWGp6b0cyTkJjam5UYmljc0h5K3djK056MEtud3VmNytRVnU0?=
 =?utf-8?B?SDZjN0V6Wis1QlM1cG83Z2lQRFk5ZXZSL3cwcG9vR2d4MUNLQ296NXVrby8w?=
 =?utf-8?Q?4gC3sQmpez/qwQ+kDMhJTT/jE?=
X-OriginatorOrg: siemens.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8434fe55-bc6f-4114-c3ff-08db47216311
X-MS-Exchange-CrossTenant-AuthSource: AS4PR10MB6181.EURPRD10.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Apr 2023 13:14:52.6751
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jhmQWfdJp0MApBBIF/p+KV59qcRMo/ULxVlLt3uJkFKSLGxh/LhkJz1FqH4+/n958gle4h8xdEFLhmn7CS073A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR10MB6228
X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIMWL_WL_MED,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 26.04.23 23:29, Thomas Gleixner wrote:
> On Wed, Apr 26 2023 at 12:03, Zdenek Bouska wrote:
>> following patch is my current approach for fixing this issue. I introduced
>> big_cpu_relax(), which uses Will's implementation [1] on ARM64 without
>> LSE atomics and original cpu_relax() on any other CPU.
> 
> Why is this interrupt handling specific? Just because it's the place
> where you observed it?
> 
> That's a general issue for any code which uses atomics for forward
> progress. LL/SC simply does not guarantee that.
> 
> So if that helps, then this needs to be addressed globaly and not with
> some crude hack in the interrupt handling code.

My impression is that the retry loop of irq_finalize_oneshot is
particularly susceptible to that issue due to the high acquire/relax
pressure and inter-dependency between holder and waiter it generates -
which does not mean it cannot occur in other places.

Are we aware of other concrete case where it bites? Even with just
"normal" contented spin_lock usage?

> 
>> Anyone has a better idea how to solve this issue properly?
> 
> Use hardware with LSE atomics :)

That would generate tons of waste of current electronic devices without
it - not very fashionable anymore. ;)

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux