Received: by 2002:ac0:946b:0:0:0:0:0 with SMTP id j40csp2101790imj; Fri, 8 Feb 2019 12:34:52 -0800 (PST) X-Google-Smtp-Source: AHgI3IZzeHCHeZ5sv7elg748YjnaedgY7WgBmE0iBgJrnEmDRiLvG7Suq6+ztdkvlCk7jjKAwRwG X-Received: by 2002:a65:6645:: with SMTP id z5mr22354607pgv.351.1549658092280; Fri, 08 Feb 2019 12:34:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549658092; cv=none; d=google.com; s=arc-20160816; b=whLkMbkc2bIQrgipAmGsRYwV6hYSApFeIUlzMts9J5HKjCnYJd8VFcS09BFObfN4jb oBtJOUQtDShQWvSZ6NofXIGmWNDSpzLfVpxXKvlwf9/sMQTzGWpYQCAeSBfzAE3xysIh YH7081jjyKx5GL09cJPQwEHYrw7Zo7WeWA/sxvaj2Lo+Ahwj5fpIMNIGWqKmwLm/509P j1IsTAUFwlJrvhK7YqFVaN/DKVJMZc/YAuhck64pccSTgchS3vrzdDO2flHoiTwhfJI7 iH1Qdq2KVZsamwaNmk5gqlZMtbpsgI4mGm8bRBRB6T/bEvwgNPDcqZEU1mcpC/Jx9hds 4DUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-language :content-transfer-encoding:in-reply-to:mime-version:user-agent:date :message-id:organization:autocrypt:openpgp:from:references:cc:to :subject; bh=2NqwWdOpuPqBqquW1iAeChQERVMHCKjY4kP4ywnAJDs=; b=Vz7xSgZsJYQSMSRIoodMrQFAkDBu+T+Ce6H+vm5p/qrFQQPL4l3QACICZMeWmHFDhx kfnf0p22qh4qShs2oiGCXw5KA5mKx9A1Oow32U2hvyAjvlM7LZls4vmhCOLA1KQZ5kxa 2tSMAhNPIcRzdQgWX+UkzOb6W5SyAWNe8EfGUsX7dTm0OyR38BfMt4bLMlUDzPUgN7Jq ON98hdFp9CT/vcD4eSmDuLGFZkXGZffRK/LQ1ha10nGeFBf2cyYuPC9+pzYbm7UGfAMr +9MMFyLaW+sfMKCVSuZE8HUpdxu4KvM+VfO+TjMA/xVUrumL0IPm+j6IE49CUU3gBjuS 3FeQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 34si3224073plm.187.2019.02.08.12.34.36; Fri, 08 Feb 2019 12:34:52 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727246AbfBHUbY convert rfc822-to-8bit (ORCPT + 99 others); Fri, 8 Feb 2019 15:31:24 -0500 Received: from mx1.redhat.com ([209.132.183.28]:52024 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726601AbfBHUbX (ORCPT ); Fri, 8 Feb 2019 15:31:23 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7A118637EC; Fri, 8 Feb 2019 20:31:21 +0000 (UTC) Received: from llong.remote.csb (dhcp-17-35.bos.redhat.com [10.18.17.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id E87FC60474; Fri, 8 Feb 2019 20:31:18 +0000 (UTC) Subject: Re: [PATCH-tip 00/22] locking/rwsem: Rework rwsem-xadd & enable new rwsem features To: Linus Torvalds Cc: Peter Zijlstra , Ingo Molnar , Will Deacon , Thomas Gleixner , Linux List Kernel Mailing , linux-alpha@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hexagon@vger.kernel.org, linux-ia64@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Linux-sh list , sparclinux@vger.kernel.org, linux-xtensa@linux-xtensa.org, linux-arch , the arch/x86 maintainers , Arnd Bergmann , Borislav Petkov , "H. Peter Anvin" , Davidlohr Bueso , Andrew Morton , Tim Chen References: <1549566446-27967-1-git-send-email-longman@redhat.com> From: Waiman Long Openpgp: preference=signencrypt Autocrypt: addr=longman@redhat.com; prefer-encrypt=mutual; keydata= xsFNBFgsZGsBEAC3l/RVYISY3M0SznCZOv8aWc/bsAgif1H8h0WPDrHnwt1jfFTB26EzhRea XQKAJiZbjnTotxXq1JVaWxJcNJL7crruYeFdv7WUJqJzFgHnNM/upZuGsDIJHyqBHWK5X9ZO jRyfqV/i3Ll7VIZobcRLbTfEJgyLTAHn2Ipcpt8mRg2cck2sC9+RMi45Epweu7pKjfrF8JUY r71uif2ThpN8vGpn+FKbERFt4hW2dV/3awVckxxHXNrQYIB3I/G6mUdEZ9yrVrAfLw5M3fVU CRnC6fbroC6/ztD40lyTQWbCqGERVEwHFYYoxrcGa8AzMXN9CN7bleHmKZrGxDFWbg4877zX 0YaLRypme4K0ULbnNVRQcSZ9UalTvAzjpyWnlnXCLnFjzhV7qsjozloLTkZjyHimSc3yllH7 VvP/lGHnqUk7xDymgRHNNn0wWPuOpR97J/r7V1mSMZlni/FVTQTRu87aQRYu3nKhcNJ47TGY evz/U0ltaZEU41t7WGBnC7RlxYtdXziEn5fC8b1JfqiP0OJVQfdIMVIbEw1turVouTovUA39 Qqa6Pd1oYTw+Bdm1tkx7di73qB3x4pJoC8ZRfEmPqSpmu42sijWSBUgYJwsziTW2SBi4hRjU h/Tm0NuU1/R1bgv/EzoXjgOM4ZlSu6Pv7ICpELdWSrvkXJIuIwARAQABzR9Mb25nbWFuIExv bmcgPGxsb25nQHJlZGhhdC5jb20+wsF/BBMBAgApBQJYLGRrAhsjBQkJZgGABwsJCAcDAgEG FQgCCQoLBBYCAwECHgECF4AACgkQbjBXZE7vHeYwBA//ZYxi4I/4KVrqc6oodVfwPnOVxvyY oKZGPXZXAa3swtPGmRFc8kGyIMZpVTqGJYGD9ZDezxpWIkVQDnKM9zw/qGarUVKzElGHcuFN ddtwX64yxDhA+3Og8MTy8+8ZucM4oNsbM9Dx171bFnHjWSka8o6qhK5siBAf9WXcPNogUk4S fMNYKxexcUayv750GK5E8RouG0DrjtIMYVJwu+p3X1bRHHDoieVfE1i380YydPd7mXa7FrRl 7unTlrxUyJSiBc83HgKCdFC8+ggmRVisbs+1clMsK++ehz08dmGlbQD8Fv2VK5KR2+QXYLU0 rRQjXk/gJ8wcMasuUcywnj8dqqO3kIS1EfshrfR/xCNSREcv2fwHvfJjprpoE9tiL1qP7Jrq 4tUYazErOEQJcE8Qm3fioh40w8YrGGYEGNA4do/jaHXm1iB9rShXE2jnmy3ttdAh3M8W2OMK 4B/Rlr+Awr2NlVdvEF7iL70kO+aZeOu20Lq6mx4Kvq/WyjZg8g+vYGCExZ7sd8xpncBSl7b3 99AIyT55HaJjrs5F3Rl8dAklaDyzXviwcxs+gSYvRCr6AMzevmfWbAILN9i1ZkfbnqVdpaag QmWlmPuKzqKhJP+OMYSgYnpd/vu5FBbc+eXpuhydKqtUVOWjtp5hAERNnSpD87i1TilshFQm TFxHDzbOwU0EWCxkawEQALAcdzzKsZbcdSi1kgjfce9AMjyxkkZxcGc6Rhwvt78d66qIFK9D Y9wfcZBpuFY/AcKEqjTo4FZ5LCa7/dXNwOXOdB1Jfp54OFUqiYUJFymFKInHQYlmoES9EJEU yy+2ipzy5yGbLh3ZqAXyZCTmUKBU7oz/waN7ynEP0S0DqdWgJnpEiFjFN4/ovf9uveUnjzB6 lzd0BDckLU4dL7aqe2ROIHyG3zaBMuPo66pN3njEr7IcyAL6aK/IyRrwLXoxLMQW7YQmFPSw drATP3WO0x8UGaXlGMVcaeUBMJlqTyN4Swr2BbqBcEGAMPjFCm6MjAPv68h5hEoB9zvIg+fq M1/Gs4D8H8kUjOEOYtmVQ5RZQschPJle95BzNwE3Y48ZH5zewgU7ByVJKSgJ9HDhwX8Ryuia 79r86qZeFjXOUXZjjWdFDKl5vaiRbNWCpuSG1R1Tm8o/rd2NZ6l8LgcK9UcpWorrPknbE/pm MUeZ2d3ss5G5Vbb0bYVFRtYQiCCfHAQHO6uNtA9IztkuMpMRQDUiDoApHwYUY5Dqasu4ZDJk bZ8lC6qc2NXauOWMDw43z9He7k6LnYm/evcD+0+YebxNsorEiWDgIW8Q/E+h6RMS9kW3Rv1N qd2nFfiC8+p9I/KLcbV33tMhF1+dOgyiL4bcYeR351pnyXBPA66ldNWvABEBAAHCwWUEGAEC AA8FAlgsZGsCGwwFCQlmAYAACgkQbjBXZE7vHeYxSQ/+PnnPrOkKHDHQew8Pq9w2RAOO8gMg 9Ty4L54CsTf21Mqc6GXj6LN3WbQta7CVA0bKeq0+WnmsZ9jkTNh8lJp0/RnZkSUsDT9Tza9r GB0svZnBJMFJgSMfmwa3cBttCh+vqDV3ZIVSG54nPmGfUQMFPlDHccjWIvTvyY3a9SLeamaR jOGye8MQAlAD40fTWK2no6L1b8abGtziTkNh68zfu3wjQkXk4kA4zHroE61PpS3oMD4AyI9L 7A4Zv0Cvs2MhYQ4Qbbmafr+NOhzuunm5CoaRi+762+c508TqgRqH8W1htZCzab0pXHRfywtv 0P+BMT7vN2uMBdhr8c0b/hoGqBTenOmFt71tAyyGcPgI3f7DUxy+cv3GzenWjrvf3uFpxYx4 yFQkUcu06wa61nCdxXU/BWFItryAGGdh2fFXnIYP8NZfdA+zmpymJXDQeMsAEHS0BLTVQ3+M 7W5Ak8p9V+bFMtteBgoM23bskH6mgOAw6Cj/USW4cAJ8b++9zE0/4Bv4iaY5bcsL+h7TqQBH Lk1eByJeVooUa/mqa2UdVJalc8B9NrAnLiyRsg72Nurwzvknv7anSgIkL+doXDaG21DgCYTD wGA5uquIgb8p3/ENgYpDPrsZ72CxVC2NEJjJwwnRBStjJOGQX4lV1uhN1XsZjBbRHdKF2W9g weim8xU= Organization: Red Hat Message-ID: Date: Fri, 8 Feb 2019 15:31:18 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8BIT Content-Language: en-US X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 08 Feb 2019 20:31:23 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 02/08/2019 02:50 PM, Linus Torvalds wrote: > On Thu, Feb 7, 2019 at 11:08 AM Waiman Long wrote: >> This patchset revamps the current rwsem-xadd implementation to make >> it saner and easier to work with. This patchset removes all the >> architecture specific assembly code and uses generic C code for all >> architectures. This eases maintenance and enables us to enhance the >> code more easily. >> >> This patchset also implements the following 3 new features: >> >> 1) Waiter lock handoff >> 2) Reader optimistic spinning >> 3) Store write-lock owner in the atomic count (x86-64 only) > The patches are kind of hard to read, with most of them just doing > prep-work that doesn't necessarily matter to the big picture. > > What I'd really like to see is > > (a) an overview of the new locking logic The new locking logic is similar to qrwlock (see patch 11). Cmpxchg is used to acquire the write lock, while xadd is still used for read lock. Some of the bits in the count are also reserved for special purpose like has waiter or lock handoff. Patch 15 tries to compress the write-lock owner task pointer and put it into the count field for x86-64 at the expense of less bits available for reader count. I have sent out an additional patch this morning to make sure that the reader count won't overflow. In term of performance, there isn't much change with respect to read-lock performance. For write-lock, I saw a slight drop in some cases, but nothing significant. The merging of owner task pointer into the count field does impose a slightly bigger drop than I would have liked which I am going to look into a bit more. > > (b) what's the new fastpath case The only change in the fastpath is the use of cmpxchg for writer lock. > > (c) some performance numbers There are performance data at patches 11, 12, 15, 19, 20, 21. There was performance data for patch 4 as well for eliminating the arch specific file. Apparently, I might have deleted it accidentally. Anyway, no noticeable performance difference was observed when switching to use generic C code for x86, ppc and ARM64. The major gain in performance is due to reader optimistic spinning patches. The microbenchmark that I used shown an order of magnitude of performance improvement for mixed reader-writer workloads. Of course, we will see less performance gain with real world benchmarks. I am planning to run more performance test and post the data sometimes next week. Davidlohr is also going to run some of his rwsem performance test on this patchset. > > to explain the changes from a "this is the point of the whole > exercise" standpoint. > > And yes, I realize that the lock handoff and optimistic spinning is a > big deal, since I've seen the same regression numbers that presumably > caused this effort to be resurrected. So it's not that I don't find > this intriguing and worthwhile, it's literally that I'd like a summary > not so much of the individual patches, but of the new model. > > Please? Maybe I should break this patchset into a few smaller ones to make it easier to review. Any suggestion is welcome. Cheers, Longman