Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 25 Mar 2021 14:55:04 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Thomas =?utf-8?B?SGVsbHN0csO2bSAoSW50ZWwp?= 
        <thomas_os@shipmail.org>
Cc:     Dave Hansen <dave.hansen@intel.com>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "christian.koenig@amd.com" <christian.koenig@amd.com>,
        "airlied@linux.ie" <airlied@linux.ie>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>
Subject: Re: [RFC PATCH 1/2] mm,drm/ttm: Block fast GUP to TTM huge pages
Message-ID: <20210325175504.GH2356281@nvidia.com>
References: <20210321184529.59006-1-thomas_os@shipmail.org>
 <20210321184529.59006-2-thomas_os@shipmail.org>
 <ec99146c7abc35d16b245816aba3e9d14862e624.camel@intel.com>
 <c2239da2-c514-2c88-c671-918909cdba6b@shipmail.org>
 <YFsNRIUYrwVQanVF@phenom.ffwll.local>
 <a1fa7fa2-914b-366d-9902-e5b784e8428c@shipmail.org>
 <75423f64-adef-a2c4-8e7d-2cb814127b18@intel.com>
 <e5199438-9a0d-2801-f9f6-ceb13d7a9c61@shipmail.org>
 <6b0de827-738d-b3c5-fc79-8ca9047bad35@intel.com>
 <9f789d64-940f-c728-8d5e-aab74d562fb6@shipmail.org>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9f789d64-940f-c728-8d5e-aab74d562fb6@shipmail.org>
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from mlx.ziepe.ca (206.223.160.26) by CH2PR18CA0014.namprd18.prod.outlook.com (2603:10b6:610:4f::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.26 via Frontend Transport; Thu, 25 Mar 2021 17:55:06 +0000
Received: from jgg by mlx with local (Exim 4.94)        (envelope-from <jgg@nvidia.com>)        id 1lPUCG-002l1l-Nu; Thu, 25 Mar 2021 14:55:04 -0300
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 98d2eb9c-d903-4edd-dff9-08d8efb71fa4
X-MS-TrafficTypeDiagnostic: DM6PR12MB2811:
X-Microsoft-Antispam-PRVS: <DM6PR12MB281103FF8C2AE2E6D307013CC2629@DM6PR12MB2811.namprd12.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: reIwC5hkCOBeatCkj5TeEe09+ZY9INv6/7MS5iLiih2zT99+leHub/hCtcD/b0uO3bYMF/YI/xddke5Nm47UIYNunMC2NvZOR1MP1thCyzP50JntjsED3mV/BuvbOEq3jv9Ysl9enRIwnIgM+WSAX0k0Dv7MrQYBi0IjswHSyyDvt8uAITp5rFe2v+pqfQ9SCjdVovL2geukEGCYALT9HLlDzsB79JDXriqtMRxUyDxImX+9FwuFujmhTCLxrcm+iikUAtIGS8veM8si6022KEvCiY81dxMb64crvfk+CiP1p11KRa4mx4a3y7GA7ur5aXRGd2/fctfL95jAcvctWk9JJQFrN2QEjrlFR5e2bgVuXcude1460giB/scT23xa3/NTBYH1S/2Q1pXWolCvw5TLnWcWB8oR4gEgqww3bJf6NLcQAncTfvpWhPhenW3St3H84pQJF12RqwiqUwpwSozdOVQOJnZuq+Kr8TG2csxowAbU4z5kzaadUHrS/5V8AL993CqLqO/4zMo8xjh1VM1DnZhjrr7rvAysR3Jn1T61DWLjQLU4V3X8a3v7yo183Vs+VBwU9OTPY1+uHcha1KlwdM5S4LIEYtsl3fOhTFJ3BbLcSktS3JG2195Nyw9EWvoSrruTxbRWPMLD2VKNI5xjjk4G3+Y7XzYKP7/02Kd1q8sOHmuRNomSbcOc4VK0
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3834.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(39860400002)(136003)(376002)(396003)(346002)(8936002)(4326008)(33656002)(26005)(9786002)(66476007)(478600001)(66556008)(86362001)(53546011)(5660300002)(1076003)(9746002)(186003)(54906003)(2616005)(426003)(316002)(8676002)(38100700001)(6916009)(36756003)(2906002)(66946007)(14583001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?WmtRY081WEVsSmRNMFJXcDcyd1dYK3dmeEd2eDQ5ZTFYQ0g2TlBISFhtcThJ?=
 =?utf-8?B?TksvMGJ3aC9xc1hFQk0rZGFwV2FpaWp0andKWmVYWW1XSUY5OWR6eVRXajJt?=
 =?utf-8?B?YW1vNFQ0MlhMdm40YmplNE5xaUZQTU1VZWxyTXhmNUZpaytxOHdDVmRuVUEy?=
 =?utf-8?B?MFhleVdPNUZIM2dTVGh4VWpqRmtRMHBsV2U1ZTBiTDlpWFh2Q3RHRkZiTU1m?=
 =?utf-8?B?WDN1RUF4bU9pa01pcHBqeHpWRG9FMFY1QmJQdCs3dHBkOVM2UTVEQ2dMR1ls?=
 =?utf-8?B?eXRySGgyT094M1FERkFnZGdmQ3Q4Q3BlZWlGbUtGSlBoYlo0S1ZHZW50SHN2?=
 =?utf-8?B?aHJ5OGFDVzRrcEk3VHhpNkhWZ3FZZkpROFpTbFhFWm9kZ2NBQnNneFQ3Z2FQ?=
 =?utf-8?B?ZTY5dS8rdVh4YmtFQlNUeGtPNXlzYTBhMG1obDlSbndpY1JlMWRBc3BmNzhr?=
 =?utf-8?B?b1l1ZzdzOVJBNjFOK3JVbEVHdlJzMTdpV0ZzTmlpVlp0MEFMc1VtdSt4OFpy?=
 =?utf-8?B?WGNqM0U5d1Nkc0JnTzI2aTZreElSMVczNEJhQWNLM0tnWENDWE42b09XcjVF?=
 =?utf-8?B?NnE0a3VneGFMWWpXNHVXTEcramZYMCs0Y0pHN3liajRvT21zR0dYT2F3M0Jk?=
 =?utf-8?B?UTBQbVpHZ3lvM3BsekcxcUFVNUdRZGRtYkJsRFZmNXNVd1VHZXlSSVYrSnhr?=
 =?utf-8?B?OWlLTXlaclcrRjIraW5BQW5UcjJFS1ZUWjVpWEd2VlBtTGt6ajlKa0xXRlBG?=
 =?utf-8?B?Nnh0UDFneFlyaXBIYWdGZHdWVXl4RXp2QU1mR1FmSTBMMlQzVlg4U0h1MzFQ?=
 =?utf-8?B?bUZIa0tYeWxLVy9iTkNrck9zc2xKYkhCUGIzczFiM01mR0FveUN1dGtPbEcv?=
 =?utf-8?B?UzliRzQvSXFrRDlrMUZZc0ZSVE14UzgzRk5MVnBuVSs2QTduNU9KSWtpUm1r?=
 =?utf-8?B?RkI4czBXZlA4WEtBZGcxNHgraEhIOTJxcnBlVWFFLzFqb3AzVXBmTFc2bWpt?=
 =?utf-8?B?b1A3VkxSR01BaENKRWFPYm0rZDZkSE14UlRvYkRLV2VZN1Z0Nk5XOU1INFky?=
 =?utf-8?B?aUhWSFdvV2VJQ0JJVlZxWDdraGU3ODJZblF1NC9FRHordUI3U3ZZOHhZMlZQ?=
 =?utf-8?B?Q1RlVllITFZWNUJEU3hNRjZuN29xMEltZ09jeVNhV0NPY2VuaDYzSVd5d2VB?=
 =?utf-8?B?eXlBU1ZLcFcvVGhFYXp0bjJLdnQ2ZHBUUDlRZ3ZGM1A3Q3hXMEQ1eWVTYmlH?=
 =?utf-8?B?VGpvbHFsQ2pCUlBYY1h6V2dtSi90SmpZT3RkM212YUlCYjZpNWkyMVMwclE1?=
 =?utf-8?B?aW1DbjJKdmhJS0tFTDNKWFBReTBSS1VpNGVhZUV1bWRIdWJ5aXFjMGVDYWtJ?=
 =?utf-8?B?MnRpZXdhVWY0VXVuZXVBalZPTkRLYTFSTTczSUFQb1ZSc1FINE5QL1djVDJu?=
 =?utf-8?B?TUFQWmtuM29JdnAyVjRQNHZ6bWJBYW1pTjBRNE45dXEvc3Z3S2c0WGVtTEN4?=
 =?utf-8?B?Q3ZQSnU5cDJsRlBETllKWXgyRzl1VEhwZ0dkcmxZemE2Yjg0OUNxNWM5VERX?=
 =?utf-8?B?cjVNYk5sL3E5V3JmNExrQXlNRHZNU2pFaitmc09JcFg5enRkUWJlbXRySFpI?=
 =?utf-8?B?VUYwNjhVK01xNGI5TmFMM1owSHZVTlZ1clI5d2FmZ3ExNnphK1RPTVBleExJ?=
 =?utf-8?B?MWNhaXhJdW9wR1ZpalVzNmN5cGIzQk9PVmQycTEycUhSSC9oeWZocFhoTm1L?=
 =?utf-8?Q?dNWBxVOvYo0xarzKJ4jy9aVM5FqL7hAXQmi7eiI?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 98d2eb9c-d903-4edd-dff9-08d8efb71fa4
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3834.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Mar 2021 17:55:06.3196
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tqrKTe5f83nRFnPsED0nO9FNBw8DwuDdIVsdf5ibIOjUxl13os5xweV/2fEZd8J1
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB2811
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 25, 2021 at 06:51:26PM +0100, Thomas Hellström (Intel) wrote:
> 
> On 3/24/21 9:25 PM, Dave Hansen wrote:
> > On 3/24/21 1:22 PM, Thomas Hellström (Intel) wrote:
> > > > We also have not been careful at *all* about how _PAGE_BIT_SOFTW* are
> > > > used.  It's quite possible we can encode another use even in the
> > > > existing bits.
> > > > 
> > > > Personally, I'd just try:
> > > > 
> > > > #define _PAGE_BIT_SOFTW5        57      /* available for programmer */
> > > > 
> > > OK, I'll follow your advise here. FWIW I grepped for SW1 and it seems
> > > used in a selftest, but only for PTEs AFAICT.
> > > 
> > > Oh, and we don't care about 32-bit much anymore?
> > On x86, we have 64-bit PTEs when running 32-bit kernels if PAE is
> > enabled.  IOW, we can handle the majority of 32-bit CPUs out there.
> > 
> > But, yeah, we don't care about 32-bit. :)
> 
> Hmm,
> 
> Actually it makes some sense to use SW1, to make it end up in the same dword
> as the PSE bit, as from what I can tell, reading of a 64-bit pmd_t on 32-bit
> PAE is not atomic, so in theory a huge pmd could be modified while reading
> the pmd_t making the dwords inconsistent.... How does that work with fast
> gup anyway?

It loops to get an atomic 64 bit value if the arch can't provide an
atomic 64 bit load

Jason