Received: by 2002:a05:6a10:16a7:0:0:0:0 with SMTP id gp39csp1776628pxb; Fri, 20 Nov 2020 20:17:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJyr/9/7AsPmQ6rDl/dN9ulB9TjtQPs3oCTDL+e0n4Zd9Hi4mXBHKqs3Vbm/U2qd+0gWoxpK X-Received: by 2002:a50:a845:: with SMTP id j63mr39917158edc.32.1605932239773; Fri, 20 Nov 2020 20:17:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1605932239; cv=none; d=google.com; s=arc-20160816; b=qsUGwMOiYTwqsrrkYdxc2SKBuub0dU6FJddr4S1gZp7xJbUiliEe0RlKb3gyHSqFBp WRu8MvQzkVw087cWlYsxHQjjVa11+EsDhPhzL+8ShoTEXJH+71RIibOgJd8G+pRuLHkm oUyTWqKUuzUd7Moucbs0tcXEs9IMIvEqTopvDDerM8zjT6lO0LYpog0ch0ytb5afSz5z ZcyMoQvvPKZDgTmtdbHYtmhjyl8A6YXTFWpwQYNunDvNFA4GUNIZPY6iqcv37of33xfg wQflgLbGyuvptu8oEya5KZmMaQKl/XZFcY4YuA7ZGHUBUyvC5CsDU+LE6QDYSyzJfUJ1 3SSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:cc:to:from :dkim-signature; bh=Nysw+ALeJ2uJP9IshMO/7Qd/B8DM61mZ9shnVFeRmCk=; b=Tkh4qKrRXSAQeS3GpL9/FRRGClEGt00Z58dW5QINFmHmLjX3cnWqEHtaNEZH4rE+Ba rd1CF/NXtelJABT4uYlS4bvQAYN7LApGcakopnoQSSpvfLSZK0WpkOwKpumJjGGgiA+I V7ohR32lbK076ssYcdqMwDENgQiYzMbYjqY4a+Lsdb4jyIpzA8dw9siN1MVK83JNt0Wt 7KRd5dNH9cmetK+C5Zzv4KI53LRPu7+pl8VS1Xc0YDekLLrY12H7VHvRB+HCmjSaEQLY CdAZdy6PNIoC4NMQRE/TfpzENcY+SUvqcRX8VDY9OOumq6y9tpzeujkBVXAZXPRDmD0+ 8orw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="XkC/I4LD"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id f9si3004342edy.248.2020.11.20.20.16.56; Fri, 20 Nov 2020 20:17:19 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="XkC/I4LD"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726463AbgKUEOd (ORCPT + 99 others); Fri, 20 Nov 2020 23:14:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:24539 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726354AbgKUEOd (ORCPT ); Fri, 20 Nov 2020 23:14:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605932072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=Nysw+ALeJ2uJP9IshMO/7Qd/B8DM61mZ9shnVFeRmCk=; b=XkC/I4LD09IGI05ezRUZ9UEzWtTYxjmczAp4gaQNZcd0AxTHsdxb+Wh4rOdfGudarcOP4l JdLTDlVjQ/mlrwl8ItdBAKTWM3HPiSn57QluW4vSBWRibEQi+YZ2il5Rsg4Da+PFrkTNjU upxH/AO2bO1mPa3x+5OEoC5XotTyo3Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-98-iEFxbKjKOPOoeJSfthAsdA-1; Fri, 20 Nov 2020 23:14:30 -0500 X-MC-Unique: iEFxbKjKOPOoeJSfthAsdA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1666C1DDE7; Sat, 21 Nov 2020 04:14:29 +0000 (UTC) Received: from llong.com (ovpn-119-225.rdu2.redhat.com [10.10.119.225]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5868F6064B; Sat, 21 Nov 2020 04:14:22 +0000 (UTC) From: Waiman Long To: Peter Zijlstra , Ingo Molnar , Will Deacon Cc: linux-kernel@vger.kernel.org, Davidlohr Bueso , Phil Auld , Waiman Long Subject: [PATCH v2 0/5] locking/rwsem: Rework reader optimistic spinning Date: Fri, 20 Nov 2020 23:14:11 -0500 Message-Id: <20201121041416.12285-1-longman@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org v2: - Update some commit logs to incorporate review comments. - Patch 2: remove unnecessary comment. - Patch 3: rename osq_is_empty() to rwsem_no_spinners() as suggested. - Patch 4: correctly handle HANDOFF clearing. - Patch 5: fix !CONFIG_RWSEM_SPIN_ON_OWNER compilation errors. A recent report of SAP certification failure caused by increased system time due to rwsem reader optimistic spinning led me to reexamine the code to see the pro and cons of doing it. This led me to discover a potential lock starvation scenario as explained in patch 2. That patch does reduce reader spinning to avoid this potential problem. Patches 3 and 4 are further optimizations of the current code. Then there is the issue of reader fragmentation that can potentially reduce performance in some heavily contented cases. Two different approaches are attempted: 1) further reduce reader optimistic spinning 2) disable reader spinning See the performance shown in patch 5. This patch series adopts the second approach by dropping reader spinning for now as it simplifies the code. However, writers are still allowed to spin on a reader-owned rwsem for a limited time. Waiman Long (5): locking/rwsem: Pass the current atomic count to rwsem_down_read_slowpath() locking/rwsem: Prevent potential lock starvation locking/rwsem: Enable reader optimistic lock stealing locking/rwsem: Wake up all waiting readers if RWSEM_WAKE_READ_OWNED locking/rwsem: Remove reader optimistic spinning kernel/locking/lock_events_list.h | 6 +- kernel/locking/rwsem.c | 293 ++++++++---------------------- 2 files changed, 82 insertions(+), 217 deletions(-) -- 2.18.1