Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <f562bd43-d9bb-af27-0d44-af7902ee15df@nvidia.com>
Date:   Wed, 21 Sep 2022 17:13:29 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.2
Subject: Re: [PATCH v2 16/18] mm/memremap_pages: Support initializing pages to
 a zero reference count
Content-Language: en-US
To:     Dan Williams <dan.j.williams@intel.com>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Alistair Popple <apopple@nvidia.com>
Cc:     akpm@linux-foundation.org, Matthew Wilcox <willy@infradead.org>,
        Jan Kara <jack@suse.cz>, "Darrick J. Wong" <djwong@kernel.org>,
        Christoph Hellwig <hch@lst.de>, linux-fsdevel@vger.kernel.org,
        nvdimm@lists.linux.dev, linux-xfs@vger.kernel.org,
        linux-mm@kvack.org, linux-ext4@vger.kernel.org
References: <166329930818.2786261.6086109734008025807.stgit@dwillia2-xfh.jf.intel.com>
 <166329940343.2786261.6047770378829215962.stgit@dwillia2-xfh.jf.intel.com>
 <YyssywF6HmZrfqhD@nvidia.com>
 <632ba212e32bb_349629451@dwillia2-xfh.jf.intel.com.notmuch>
From:   John Hubbard <jhubbard@nvidia.com>
In-Reply-To: <632ba212e32bb_349629451@dwillia2-xfh.jf.intel.com.notmuch>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MnBpKzVWeCt1TWJ5N2RseVZjWnkxcTljSnJocU5zSXVUQllvMmY5c25iTjRJ?=
 =?utf-8?B?SG1HR3lLcEk5NUE0ZTk3ZCtvcHR4ZEVvdllvRkZMMU5vN3hpRVJ5dDdqMURu?=
 =?utf-8?B?SzB0MitqUlRnZE1WZFcwNnprc0U4d3p5TnNnTlN2YlE1enZUekxpL2VKTENi?=
 =?utf-8?B?bnZJdG1mcW9PMkFkM2svN0ZLQjUweWYvRW9PQVFOM3VmSDNZTXdJd21JakpD?=
 =?utf-8?B?NlNVUUNEZWJNa0NlaytoY3BIR1EwMFp0Y0RFbWpTV2c2Q0RDNndHSnhySGZC?=
 =?utf-8?B?VEsreGRMSlE0SEg5RU1hOWRYT2NQd05NYWhZSWh6cHdyQWE3WUZ0dWx1alAr?=
 =?utf-8?B?QXZBZTIwVzIxMVBPMlBNS0RMek1rRnZOb0NHUVZOQVVDOVFxL0dlemdiQ0JI?=
 =?utf-8?B?eUNnKzRROUpRMUpWWVdkNVR4azQ3OGRoSFVtMXZsUGVrS0Y3Qk9WSDk4UEZz?=
 =?utf-8?B?YWI2b1JRVEJZSy82Q3VEVjh6TU9VbVQ5djZWYmREeFdFRVUzVnE3WlhGcnZq?=
 =?utf-8?B?VDl5TEwxWEowMU94c2lsRWZDV1dHS1ZLWCtkQkQ5RGlnOUdxQkRiQi9mbmVa?=
 =?utf-8?B?d1k0K3p5YStEOVdHcjNCVllteEJ0dHJWV3JPVkN4TGFUc2twZ3dCZ0RMQndL?=
 =?utf-8?B?Rm5FZXIxQVErK0F3WWpyR1Nqa3dpM0hyTWdRTXhSeVhialA3NVlySlVBR2hP?=
 =?utf-8?B?YjFYSGJGV2l0OE40WEdoWHM1VmlhdTd4MVBhR0luRHh6MXBKVzR6NHlYYzBE?=
 =?utf-8?B?aDhNTGVtN0MveEJyOC91NEowUkRnMHJIemovdFdKbUo2Mys1RUx5TzV0QTcw?=
 =?utf-8?B?eG8vWit5bHpPQzdueUhRdTdFQ2F0OUd6QTdEVW5qWkVXa1kyMkRMU0dtcFI3?=
 =?utf-8?B?OUFqZGFKTml5NlUvSVhpR3lzWWFsc3ZKaENkOEFOOGtUaTRiZWRUWGdIalFm?=
 =?utf-8?B?b2o5YkY3dm1UaW5ncWRxUjkyUWRpZm9FMGE1VlE5amM4SUdVa3A4U3pZWml1?=
 =?utf-8?B?UTBWNm5yYjhFUnppTzhhQ2kwTWRHRU40aVhVZVozV2ozRndoOWltbjVZY3Za?=
 =?utf-8?B?NzFkK1RERUN2T29CbWdTU3FKeW04amFaV2J0UVVxUjdYTCtYS0gxVURkclFD?=
 =?utf-8?B?dG9zS2hTZ0l3VXdSa1NiaC9CN3liY0JQL1IzM0xmUXFsTi9PSk1ZQjFHRU5i?=
 =?utf-8?B?dkNWeHgvVnVPd2FUVktMb2xTVDNMMTdaWEo3NFdFYVJlTks0dkkvR2RQY0lC?=
 =?utf-8?B?bVBYOVFqd0VoWXE3eXVoRlhZQjJteFEvSmQydUVrSnp4MW1vR1BOdEUwSG10?=
 =?utf-8?B?QjlmK2dSaXNDcE13ZzlEUkV2YyszMFlNa2lqR0R3cjN0YVA3QWg5Z0pqSm9R?=
 =?utf-8?B?MktObHFYZzR3WWo4OWlIOU1PNHJzYU5IQkpIKzZYZThlbXV2cjFlWnoyMmIw?=
 =?utf-8?B?Y2x0Myt6MnFJUkwxRzZUZkdnTnAwSW9jT3NtY2t0QmlZeHltYkNYV2NyMXli?=
 =?utf-8?B?SExIQUFoYzZwMUtBL3ZZb3RJSDlmZ3NVQUErWE94L1h5a3hDeW1rR3o2eGZK?=
 =?utf-8?B?NDMreFh1cFRXeTBlVCtheGJVekl3TWlSdUVCZXVtV0xESXlmMXVWU0crS082?=
 =?utf-8?B?NUxrYjZoNm9yN2x0MVREQ011bmNQdFhYMVl6QmlkU25VVVAxNXR5ekpPdzJu?=
 =?utf-8?B?bldCTGlxSG5hbmNrQ25LTXBwb21XT2xUanNMbVJxcmNpR1JLQVpGVFM2RFFK?=
 =?utf-8?B?NnB4RUpIV0Z3VXh3VmVPK3lzYUExK0tObkJSWTU4eEFaTE1sWlZ1MFAzazVa?=
 =?utf-8?B?ZHlKS1dWOURpejZ3OCtTYzRKbGx2cEZZZkJJemh4S1lwdzZpWGlidEtLd2NJ?=
 =?utf-8?B?VnUwV3JPQytERmc2YXp2RUwxOEVMbUx0aXhiQlV1V1pNazIwUlE1NkZvQ2tB?=
 =?utf-8?B?dytFRnc0dG1Ba21QcUJDUDhtL0VvNFBNRCtoRnNXY0RuaWNGaWYzN21jRmdl?=
 =?utf-8?B?c1JuNjJxakFxT3h1VDBYc1IzR29ZekFKeG5qcERmNFFpWGRsTG1hbXNZQUZz?=
 =?utf-8?B?U3RRLzdQckFVSGJjcy9SVTdjcG1JeElMR3FYM1ZUeUNNSkpwdlZmNTdXakpK?=
 =?utf-8?Q?UUtGB3jTvSkNDnakLyP/eXceT?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 31d8de5c-b5dd-4675-9986-08da9c2f48e8
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB4140.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2022 00:13:32.8195
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cvrExU30K/fzYImpq2aDEg/5i6YGbVzTX2fnUaOLpyKYVnN7DVEh2slX+YvgBuiOjrsNj4IzdYr3SYT36dVgWg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5462
X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

On 9/21/22 16:45, Dan Williams wrote:
>> I'm shocked to read this - how does it make any sense?
> 
> I think what happened is that since memremap_pages() historically
> produced pages with an elevated reference count that GPU drivers skipped
> taking a reference on first allocation and just passed along an elevated
> reference count page to the first user.
> 
> So either we keep that assumption or update all users to be prepared for
> idle pages coming out of memremap_pages().
> 
> This is all in reaction to the "set_page_count(page, 1);" in
> free_zone_device_page(). Which I am happy to get rid of but need from
> help from MEMORY_DEVICE_{PRIVATE,COHERENT} folks to react to
> memremap_pages() starting all pages at reference count 0.
> 

Just one tiny thing to contribute to this difficult story: I think that
we can make this slightly clearer by saying things like this:

"The device driver is the allocator for device pages. And allocators
keep pages at a refcount == 0, until they hand out the pages in response
to allocation requests."

To me at least, this makes it easier to see why pages are 0 or > 0 
refcounts. In case that helps at all.


thanks,

-- 
John Hubbard
NVIDIA