IronPort-Data: A9a23:bHsJL6hUHwnHAJ1YtkuBzBsgX1615hAKZh0ujC45NGQN5FlHY01je
 htvWjiDb/uIYGKned1+a4jioUMDuJTXxtRgTwJk+3owQSgb9cadCdqndUqhZCn6wu8v7q5Ex
 55HNoSfdpBcolv0/ErF3m3J9CEkvU2wbuOgTrWCYmYpHlUMpB4J0XpLg/Q+jpNjne+3CgaMv
 cKai8DEMRqu1iUc3lg8sspvkzsy+qWi0N8klgZmP6sT5waDzyB94K83fsldEVOpGuG4IcbiL
 wrz5OnR1n/U+R4rFuSknt7TGqHdauePVeQmoiM+t5mK2nCulARrukoIHKN0hXNsoyeIh7hMJ
 OBl7vRcf+uL0prkw4zxWzEAe8130DYvFLXveRBTuuTLp6HKnueFL1yDwyjaMKVBktubD12i+
 tQFcTFXR0qG3duq3ZyHc+xzltYuJZf0adZ3VnFIlVk1DN4AaLWaGeDgw48d2z09wMdTAfzZe
 swVLyJ1awjNaAFOPVFRD48imOCvhT/0dDgwRFC9/PJrpTSMilEsluGybLI5efTTLSlRtlyfq
 W/cuXzwHzkRNcCFyCrD+XWp7gPKtXqjA9lMTOTjrZaGhnXM6HIsGCwtUmCw//SrpE6BC41Cd
 FEbr39GQa8asRbDosPGdwexsTiEoxcaVPJZFuF84waIooLW7ByeHXMsVSNaZZots8pebScxy
 laDktftBDpumL6YU3SQ8vGTtzzaES8RIGwZbCkIVzwZ8sLjq4E+iBHIZtt7GavzhdrwcRn6x
 CqLt245nKkJitAQ1LST+krOiDah4JPOS2Yd9gjRG26o8A59TIqkfJCzr0jW6+5aK4SURUXHu
 2IL8+Ca7eYTHdSOmTaLTeElAr6k/bCGPSfajFopGIMunxyh4X/mcY1O7TVWIEZyLt1CdyXkb
 UPeogBN44cVO2GlBYd+Zo70BMo65azhDsj+EP7OY9dCb4NyaAifuiZ0aia41mTggEEqnawlf
 5uWbM2sAGwyAKFrijGxQo812r8xxzsixGrVA5P8whKm3pKXYXLTQrAAWHO1cu0m5bmY5iXU9
 9pSPdGDzRl3WejyJCLQ9OY7J00QP3E/ApzxtMV/eeuKIw4gE2YkY9fRxrUuf4FkxuJVi+7O8
 Vm0X0Mew1367VXFNAGLYzZxaLrpUr52qH59NispVX6K2mEiaIDpx6cScbM+eLVh/+tmpcOYV
 NEAcsSER/5JGjLO/m1HaYGn9dM+MhO2mQiJIiypJiAleIJtTBDI/dmieRbz8C4JDWy8ss5WT
 6Cc6z43iKErH2xKZPs6otrxp79tlRDxQN5PYnY=
IronPort-HdrOrdr: A9a23:nnaUGqiezvyqicBo7FPtLA9xFHBQXiwji2hC6mlwRA09TyX5ra
 2TdTogtSMc6QxhPk3I/OrrBEDuexzhHPJOj7X5eI3SPjUO21HYS72Kj7GSoAEIcheWnoJgPO
 VbAs1D4bXLZmSS5vyKhDVQfexA/DGGmprY+ts3zR1WPH9Xg3cL1XYJNu6ZeHcGNDWvHfACZe
 OhDlIsnUvcRZwQBP7LfkUtbqz4iPDgsonpWhICDw5P0njzsdv5gISKaCRxx30lIkly/Ys=
Date:   Wed, 15 Mar 2023 13:31:14 +0100
From:   Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>
To:     Huang Rui <ray.huang@amd.com>
Cc:     Juergen Gross <jgross@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org,
        dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org,
        Alex Deucher <alexander.deucher@amd.com>,
        Christian =?utf-8?B?S8O2bmln?= <christian.koenig@amd.com>,
        Stewart Hildebrand <Stewart.Hildebrand@amd.com>,
        Xenia Ragiadakou <burzalodowa@gmail.com>,
        Honglei Huang <honglei1.huang@amd.com>,
        Julia Zhang <julia.zhang@amd.com>,
        Chen Jiqian <Jiqian.Chen@amd.com>
Subject: Re: [RFC PATCH 2/5] xen/grants: update initialization order of xen
 grant table
Message-ID: <ZBG6kthkEMo4qmo6@Air-de-Roger>
References: <20230312120157.452859-1-ray.huang@amd.com>
 <20230312120157.452859-3-ray.huang@amd.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20230312120157.452859-3-ray.huang@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TXovcTJ4TkJpbXJqOE94Sm9LcXU1c2IzLzFPdU42NXZKMFBCNlNvemdnUHc2?=
 =?utf-8?B?ZTg3TUpjemcxelZ5Vld2a3d0MDhUMlBDamc3emJLZmpUQkE1VzUzc281OHAw?=
 =?utf-8?B?Vm82V1Nra0V2L3RBcS9hUlhydG5xaDIxM3k3cHlSOXBQZHRXSHZDdnNKeWpm?=
 =?utf-8?B?UWdhbmdCQUpzdHNUWVQ3djc5Snl5OXFqa1VIQm5xYVdnNDdtQ1ZIc1ByS2Ro?=
 =?utf-8?B?MGRPdk5Kd0tnbDlhN1ltMlRkVEhqS0MvMVByYnBlcURlaUdVTmx0bzBBdlhp?=
 =?utf-8?B?OW80LytTcWloM3dHb0FSZS9xRWRQV09wMzAwdGZzVkhCTFJwZVphNUs3azFB?=
 =?utf-8?B?RDVIdExVcy9pLy8zcm9oeHo2aDFkKy9UTXZQYUdEc1ZMVDRvUVJkMWJTdDF1?=
 =?utf-8?B?bjBDcUUwcFR5ZWRmVm9rdnlaU3YxakR2blVIZjlMc25SaDRYSWtDeGZMZVF5?=
 =?utf-8?B?RHpnWlRYeDJmZU5MdkhIeFNKVEpDN3VLUTBzR2lUQWNmd1V0cldqZnNOTENM?=
 =?utf-8?B?ZDY2Tk0vSkZ1SzRhWGdVTndzby9Bb3g1L3NSNmNHUXpZcnA0dnNrOVVpSWRt?=
 =?utf-8?B?Z3lNaXNxd2JrbjhBRXJKMzU5SkpoVmp1UndKMGdRSVovQ0hJRGovcEY2K2ti?=
 =?utf-8?B?UVQwSXFuVDVxR01xVlBiSitic0p5a0VOMmk3d1QxRW9hVFYwaTgwZ2xTZm5S?=
 =?utf-8?B?YjF1QnJGVTNkVlJ3NVBIY1lzNm5MME1WanZFT1g5ZVlSS3hRQ0E4L3dWRHZF?=
 =?utf-8?B?ZFBCVGxJbmQwYVpVV1ljRHhHTFpCd1RRMEYvNU5EUXE4ajBJYnhyM0Q0OHRj?=
 =?utf-8?B?c2FDNmZzcFM2d2NtYjZXVXFTdjFzZlZIeHgyd09ud0VKY1dmaXdhb3hRQ0lL?=
 =?utf-8?B?N3ZiZksxazFGOVZUcmdjdXNrZlRsMlYxcDNJbEpLM05pUGVtVzVMRVZFcGFJ?=
 =?utf-8?B?REx5ZCtPWFNtOUQxS2xZamo4MEowUnBDWWNVRFRUMTM0R0haOGRTclByZ09x?=
 =?utf-8?B?WlI5L3pJblRSbmZlZ2VMZytGL3o5WVJkMG9yR3VTbWdEbnVpTXdJUTVTMEVv?=
 =?utf-8?B?S3VGQkxNSGNocEhMdDZjT3ArREN5WUdCV0ErWHd6L1IybXhzWkgvbVhoZnpH?=
 =?utf-8?B?Ukw1d2xkTmtDQXBVL1A0Vm1OT0I4Q2ZqNkVYNFFtUFp5ZFlOaVArWHF4c2JS?=
 =?utf-8?B?TFFUNW9qTmNtS21KZjhlM0lqMy92b1NuQ3d3QjRZVHcvTGFzYnc5OEhJNFNw?=
 =?utf-8?B?OEhTdXoraW1JNk5TaCt0UWc4cGJINDI5M2lyYW1vVEQ1OEJ1WFpVSEZZb2JC?=
 =?utf-8?B?WHdsVkRxMlZDQitWSzYrdU1vQ3QzWURqT3NjMmpxR3M3TjZHODdNVHZwTlR5?=
 =?utf-8?B?MjJvTWZnbW1rMm96WFl4SnJ2MUxWNVFvbVBxd2F4RFhYdzBFUlBSUDRlczNK?=
 =?utf-8?B?Qkp5dkthRXRZQ29EUkNjT0xHV3FRVmhzaGVBZk1RUTdEbHAzQ0dXaUhQVjF1?=
 =?utf-8?B?ZlRVODR0bmFKdkVzREVJY0hKQUNsNDhLMlY5OG5WVDIrNDQ2VHRwV1Q3eUVY?=
 =?utf-8?B?QVdYZlY1N3NIUEtRRHdNRWN1clVJVVdUNFloUmQ2TThoK0c2Q1dCSVhrTXpK?=
 =?utf-8?B?WEIzbllISnBQa1FJcVNlajFvNkVBR3JZMSthM21FYXZHSVRxYUhycXhlNFFU?=
 =?utf-8?B?RUhRYnlIOCtRdTVnbjAxTFFwZit1YTQ0NHQrM0JVRjlJQnoxUERsV1FzR3do?=
 =?utf-8?B?dk1JdjhNZzdzcnpBUUtvUmFSZCt6YThUYXdrZ2JiZm9aVUpaRzFVTElMQnha?=
 =?utf-8?B?dXc5eU9VUlVrRUJaOEZVY2syaElpWkZmSHNJL0EyRGNSbHZySDdVRW9XTllW?=
 =?utf-8?B?MWlnRnVYaDdOS0UrbW1MTzIvMWhWcXJObEoyS043UE9mQjBOV0FlUFRMd1Yx?=
 =?utf-8?B?TVpvSmlNeHdYYXJmYUd5WE9kSlFRZlQ3ekloQ0hqcTUwYXB0ZFFsTnU1Ymdt?=
 =?utf-8?B?Vy9zMzMzRnNrNTNOTnNkcldsek5XZDJiOTJBTlRqZlhIY3doQlZrWmtZblJu?=
 =?utf-8?B?RWhPK2p2cUhOODAwZERYdmFKQ2o4ZlNXRGgyNXpMeE12YTljMGxGMHEvWkJB?=
 =?utf-8?B?MjNGSE1FSjUyTGhWU00wTHBmWVNGOTRxUDBWb3Blc2orY0s3cHlJcVc1dFFq?=
 =?utf-8?B?Y0E9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?MVNUMTNLR3Rkc1NFVnpDNGdDSkovRjA0RDRXOTBGelAycnVOZElxT2lLTTJU?=
 =?utf-8?B?Z1JPdUJlcml5dENxQ2RVMHFpU2x4UzVnRUg2UUdGSU9McjRYeVBzeWpGNE9R?=
 =?utf-8?B?NkZsSkxOamhRYjBSSCtrbUJyQUxOZ1RBSWQvVUFGYlF1aXNsWWZITHZnSisy?=
 =?utf-8?B?MVhLRko2ZG9OeHhsMjFrRnBzNEJwcnVnZDBUSjZaazFDUTRkcG84WXFGdDcv?=
 =?utf-8?B?SjRyeTkzWTVwaFhqbDJ6Zm5TZFYvY0h6a2tnVVAwWWlzZkV2ZFJiMFdKTGIw?=
 =?utf-8?B?YlFlSlN2VklEQjNiZVJxaW5rK3hWMkU2blM3eXFpcGhicEhLVW1mN1VvL2VC?=
 =?utf-8?B?RFlZdWZDK0o0VldCaFNvRndMOTRJY3hmaVZyVXdwK3cyTmtMZmxjSHl5WWFM?=
 =?utf-8?B?NzRpL2R3cmc3ZWdiWlNtMElWQ1dWWGVpMzdMck5OK2NkZWEvMjRFV2p5c0VK?=
 =?utf-8?B?aHlWU25reEZrMG51ODM0MmZHQ21VMXpXMWQ0cmRYcG9NODFZeDdhUEtzWXZz?=
 =?utf-8?B?V1o1eEFvSDUwdVF4NlFibXpjNFcxbVhQUDRSZE9obHBBMjhhUHc5enRhWHlN?=
 =?utf-8?B?WTQzZ1daUStxS0ttck5wZmN0M3FXQWNVY0dodEh4UWRQZnRkUFdWaiszQ1VI?=
 =?utf-8?B?T1FFYVg4UzhXWEpMMXBuUzFXRFBDNnhwZjhqcC9rVnFNdWhqTEduU3FTeDFF?=
 =?utf-8?B?M2IvcDlJSUtlWEZodGFzWmcxK1pCRjVhNlZ4S1BndFNyd2k5MzFEMUp0Sitk?=
 =?utf-8?B?V3RnMEhSS0lqeldCWkdWVGR0YjVHd0ZiSGswMVpmeFNFZFF4WmEybWJXYUJs?=
 =?utf-8?B?UnBPbGhjSE5GS3pRalB1ZDk5Tld5TGlxSkhNT2JqV29sOUpWVUxvbDJIVEdy?=
 =?utf-8?B?alM2WVMrNkE5MWlrV2pwbnVFVllNR0ZQTW5rYk51Q1loQWhVRHNueU9lQXNH?=
 =?utf-8?B?bTZQeElSckZ4N0hxOWhsZW92cW5ZUXFqVzFHcDhjd1ljbkVBL0p1b1NnM0Zh?=
 =?utf-8?B?TGFzVTNob01zVkI1dGtUWDc2QjREbnQ0ODhhWmNwb0NqdEtleEdvVEsvSWp4?=
 =?utf-8?B?VXJGNDBQVUh3dWhpNk5tTkovRUNhbUExUm5LTmxjUlNWeE1xN2ZFSkFBRk5y?=
 =?utf-8?B?N25uei9zSHp6c2NlQnVJaXg2TEFXbndBVXZpNFQ2cVdnU0xTYkEyOFo5WE56?=
 =?utf-8?B?NFNZUVlITUQ5dVlEUlFtZmVtTWhEUVpmYWF6QVdCYlJMbXlvZmFSWUZxU2du?=
 =?utf-8?B?T1pHUlErU3Q5VUJXdjJ3WjZJc25nc0pSS1E3WUhGN202enpFYjJOV2NoMEtH?=
 =?utf-8?B?RGN1aXFFOXZWQnZtV0hidzM1Wk9tL1c5ZEJ2L2RJUGJBaWxkVXVuZUgxMk85?=
 =?utf-8?Q?fG9S66Y0f2E/NS5NghoYhT3ZaYSVxT3o=3D?=
X-OriginatorOrg: citrix.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 274ca5cf-c941-447e-0e49-08db25512e06
X-MS-Exchange-CrossTenant-AuthSource: PH0PR03MB6368.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2023 12:31:20.1065
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4brcGzUI8sO+8xnsAB0+NFYKwNKRERLDP4lq8Y/HVjwBtsZm4E1eYcIXfacZgCMi5+BohRyXrCijImkhLmcEtg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR03MB5179
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, Mar 12, 2023 at 08:01:54PM +0800, Huang Rui wrote:
> The xen grant table will be initialied before parsing the PCI resources,
> so xen_alloc_unpopulated_pages() ends up using a range from the PCI
> window because Linux hasn't parsed the PCI information yet.
> 
> So modify the initialization order to make sure the real PCI resources
> are parsed before.

Has this been tested on a domU to make sure the late grant table init
doesn't interfere with PV devices getting setup?

> Signed-off-by: Huang Rui <ray.huang@amd.com>
> ---
>  arch/x86/xen/grant-table.c | 2 +-
>  drivers/xen/grant-table.c  | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/xen/grant-table.c b/arch/x86/xen/grant-table.c
> index 1e681bf62561..64a04d1e70f5 100644
> --- a/arch/x86/xen/grant-table.c
> +++ b/arch/x86/xen/grant-table.c
> @@ -165,5 +165,5 @@ static int __init xen_pvh_gnttab_setup(void)
>  }
>  /* Call it _before_ __gnttab_init as we need to initialize the
>   * xen_auto_xlat_grant_frames first. */
> -core_initcall(xen_pvh_gnttab_setup);
> +fs_initcall_sync();
>  #endif
> diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
> index e1ec725c2819..6382112f3473 100644
> --- a/drivers/xen/grant-table.c
> +++ b/drivers/xen/grant-table.c
> @@ -1680,4 +1680,4 @@ static int __gnttab_init(void)
>  }
>  /* Starts after core_initcall so that xen_pvh_gnttab_setup can be called
>   * beforehand to initialize xen_auto_xlat_grant_frames. */

Comment need to be updated, but I was thinking whether it won't be
best to simply call xen_pvh_gnttab_setup() from __gnttab_init() itself
when running as a PVH guest?

Thanks, Roger.