Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 30 Aug 2022 15:45:42 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     David Hildenbrand <david@redhat.com>
Cc:     linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Mel Gorman <mgorman@suse.de>,
        John Hubbard <jhubbard@nvidia.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Hugh Dickins <hughd@google.com>, Peter Xu <peterx@redhat.com>
Subject: Re: [PATCH v1 2/3] mm/gup: use gup_can_follow_protnone() also in
 GUP-fast
Message-ID: <Yw5a1paQJ9MNdgmW@nvidia.com>
References: <20220825164659.89824-1-david@redhat.com>
 <20220825164659.89824-3-david@redhat.com>
 <1892f6de-fd22-0e8b-3ff6-4c8641e1c68e@redhat.com>
 <2e20c90d-4d1f-dd83-aa63-9d8d17021263@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2e20c90d-4d1f-dd83-aa63-9d8d17021263@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RPn4FZrNryjYqp2gLRJfN/lkk/EwUx32r/ysw/gP/wxTahHN9M6GoOFbyEvZ?=
 =?us-ascii?Q?A1dM8pG9nngPIVvj6K5Kpe99d5axkHB0KNlurLd8zIyHGZevkP60uo+c456Q?=
 =?us-ascii?Q?Mtl+4GdQVKvraY3j4oWZ7/c84yuohPh7L6iTNos2CTAqYrsfzVoDBBb+7WL8?=
 =?us-ascii?Q?7tpOZZPYXf3lopA/9l7cZwzDeRd5+ZSlOaT2XLLm0+sF4njFXKC7R9tlGYP0?=
 =?us-ascii?Q?nQWCGaufIuzGWNh9IaJKG6rmS6aH2ZCGf+AIDGN7gIdj2XfKQ2xwOrSBdfQg?=
 =?us-ascii?Q?q+xOzb7kDE/uekB60VNku4Hm4t3QXJR2yUQ5d0qRmBiXZBKaosaUMAE9TQzH?=
 =?us-ascii?Q?pmuI0biZhBx4E5OpjnjO00g+o1VWG4vh6bdJgb+9Z1oQqfIRL0A7hbZY8S89?=
 =?us-ascii?Q?pP7aTfeGQA0584KEnURCCd+BVo6ik+uSf2tqRb1ixC2YZvps7ckwvD6gyq+N?=
 =?us-ascii?Q?RQKKMtCPT9BSmngpybaxiqPHx/w7AVSygivuhsRxh7wSMwZPFn3q3iwirL7I?=
 =?us-ascii?Q?9N5In8sJbYqk8eRNWR/Z6pd0m9+O0EEpdbLpfQPzAUQ8K8N/k3XgyGF4yzPb?=
 =?us-ascii?Q?nTMnTj5CJKoNrdV0ewZTuiMnfothv/uAVGMV7RyHPB6Nt26qwi1gArKR+QFf?=
 =?us-ascii?Q?+TiWAvyArWRA5RTlkyPv7WWjARW+WUqnpWU72h2/JrNhZD62ANyH0eFSGeKb?=
 =?us-ascii?Q?80g8fOmRPoMg11nVjgR4kbtK3+UnKBUCK7u+bVpdE7TELHG5IYSG56MAkIOo?=
 =?us-ascii?Q?MdPoEP+arVJrUc+JqjqMgSbZTGk5t72K6N1cwQw/tDcnXH+JEuutG/cPPtmj?=
 =?us-ascii?Q?CtYJHmgByHqR3r5iN06dREj/sX/EdjL9+G8o2X0OlZh0k2y28krCZmUT+j0D?=
 =?us-ascii?Q?4y/oLoIz1YhUd+0t2XHbkb6eTYLn2JDw9+sJedgqhpaelYBZfhhxzyAociCN?=
 =?us-ascii?Q?p2YLHepwSQzF6cpby9SQ6csElQAEBlotpusLp26lPmwf2iRg5NAVjpnpCwwR?=
 =?us-ascii?Q?eZY5sgxe3iWHfpYrVFw087xZfr4lOKctKIxylsqjT5+0kjB3AQ+Abm9tLl6d?=
 =?us-ascii?Q?n/ksXVi0T2xhLEPVPsy0H+1zXUF2XOigTpjSdvA2oWcTBUCYhcGItJPo9/gC?=
 =?us-ascii?Q?Fk1oxxsTydKC0zXfQwYSoxgjxw+2Mca7mphiOwXHKynOVfdPdWJm5rGS2BKQ?=
 =?us-ascii?Q?Gv2RiRvcNK0pY7/sinvZfyowD886+PWF8ymsfhEw/W6rGt57MxKoHGU5/0xV?=
 =?us-ascii?Q?zg+oQEvmLctcADBsALj/lZ+2wkya0ihJeynYqXMM3SvfUqfa12WPoMx1jzkZ?=
 =?us-ascii?Q?pmlr5Tjq4+M3J7WdraHjAJEjKbTgAKHT4E2rAo68qaEX3Yp0GRX9vMicPD/s?=
 =?us-ascii?Q?eA0xX+PIqmacDiIFMdDWHsXWuJSzlGh7BTm4SRS/WVIIU66nCGVDAHqoYVY9?=
 =?us-ascii?Q?2KnpGBqGyFrJMQUabg76fKNQvDhhxt+yFL/cfjrm/HPDqe1AZQFbaUdzVVKg?=
 =?us-ascii?Q?F06xFoSntKJAFFQk6auq3Qwn9Jkf778hK8dFV7N4Dlo35rzXqqW40RigmqvY?=
 =?us-ascii?Q?b31yg8pQzPAWzkmONROLYeWyTCJVm7pGD+CquVL2?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 958e5fc8-7ae8-4617-1aa5-08da8ab7d7ed
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2022 18:45:43.4041
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6Zf4Tlk+uisEMVPaDXlE+WaD6bJmKqYwITgd64fvQmbKX8LpsKdvQuuRR7N/ZubS
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7215
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Aug 30, 2022 at 08:23:52PM +0200, David Hildenbrand wrote:
> ... and looking into the details of TLB flush and GUP-fast interaction
> nowadays, that case is no longer relevant. A TLB flush is no longer
> sufficient to stop concurrent GUP-fast ever since we introduced generic
> RCU GUP-fast.

Yes, we've had RCU GUP fast for a while, and it is more widely used
now, IIRC.

It has been a bit, but if I remember, GUP fast in RCU mode worked on a
few principles:

 - The PTE page must not be freed without RCU
 - The PTE page content must be convertable to a struct page using the
   usual rules (eg PTE Special)
 - That struct page refcount may go from 0->1 inside the RCU
 - In the case the refcount goes from 0->1 there must be sufficient
   barriers such that GUP fast observing the refcount of 1 will also
   observe the PTE entry has changed. ie before the refcount is
   dropped in the zap it has to clear the PTE entry, the refcount
   decr has to be a 'release' and the refcount incr in gup fast has be
   to be an 'acquire'.
 - The rest of the system must tolerate speculative refcount
   increments from GUP on any random page

The basic idea being that if GUP fast obtains a valid reference on a
page *and* the PTE entry has not changed then everything is fine.

The tricks with TLB invalidation are just a "poor mans" RCU, and
arguably these days aren't really needed since I think we could make
everything use real RCU always without penalty if we really wanted.

Today we can create a unique 'struct pagetable_page' as Matthew has
been doing in other places that guarentees a rcu_head is always
available for every page used in a page table. Using that we could
drop the code in the TLB flusher that allocates memory for the
rcu_head and hopes for the best. (Or even is the common struct page
rcu_head already guarenteed to exist for pagetable pages now a days?)

IMHO that is the main reason we still have the non-RCU mode at all..

Jason