Received-SPF: pass (google.com: domain of linux-kernel+bounces-23850-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Message-ID: <11ebc73f-6469-4c97-9da0-0728edfb283a@oracle.com>
Date: Thu, 11 Jan 2024 10:17:58 -0600
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC V1 07/13] vhost-vdpa: flush workers on suspend
To: Jason Wang <jasowang@redhat.com>,
        Steve Sistare <steven.sistare@oracle.com>
Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Si-Wei Liu <si-wei.liu@oracle.com>,
        Eugenio Perez Martin <eperezma@redhat.com>,
        Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
        Dragos Tatulea
 <dtatulea@nvidia.com>, Eli Cohen <elic@nvidia.com>,
        Xie Yongji <xieyongji@bytedance.com>
References: <1704919215-91319-1-git-send-email-steven.sistare@oracle.com>
 <1704919215-91319-8-git-send-email-steven.sistare@oracle.com>
 <CACGkMEv8TXXTrCW=hf1sJ+V2a0eD3w1b1Yub4FnGZ0=Du-RbtQ@mail.gmail.com>
Content-Language: en-US
From: Mike Christie <michael.christie@oracle.com>
In-Reply-To: <CACGkMEv8TXXTrCW=hf1sJ+V2a0eD3w1b1Yub4FnGZ0=Du-RbtQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?dnZ0UHdFSFN3UGMvaWhtL0FJc3pFTXBaNHllR1dTaW1HMnk4VVhSaGk1ZHND?=
 =?utf-8?B?RWlsdW5LazFZUERpWHFscWYzamY1VXlnTzRseSt5akMzTWs0dEJwMDI4SXNM?=
 =?utf-8?B?MjFNU1RMZDFSdTdNVW9VUFdZMUZjZ0ZHaVExZ1hnekx1aFNWdTVJMkQ2ZnVp?=
 =?utf-8?B?MmZITmgwNGRwL0daYnZwazFRazE0bm1ENXpub3pMakswRXFJV1ZJTlpTUkRv?=
 =?utf-8?B?ZEpad1B6U2wxNGxSNjlPVkpOaXRpajlzRWNxTDg4TG5Ick1LY05iMFpjcm1x?=
 =?utf-8?B?V0U3VkkwRTlFYldwY0pVV0tRcTI2dFRWUGdsTDladVdncDlTb1MwWXZ6dkth?=
 =?utf-8?B?YWh0cFYvQ3p3VHkwM2dON2t3YVlWTjdlWURvOVA5MXAzT0oxSzBZVnR1bXUy?=
 =?utf-8?B?YlMySVg3dlNDWG9VZGxPWDkrU0wzWWFzRm01dFdiNmdacTIxWCsyU3NTSWxq?=
 =?utf-8?B?L2gzelZSS2xWOFExYnloZndMQ2hab2ZjeTdjc3U0bU0xZjE0QU4xUzlSN2h6?=
 =?utf-8?B?VjZNZkF6clF1eXVJSW13YUhKc01iRHUvRkVaYjFnakg1WG02VXlSQUJCU0Jk?=
 =?utf-8?B?bXlzd3JKeFJTTS9IYW5lYnE1Y2gwTVZ4YWRaZTBETmZ4Y0hCdlpldExiOHpk?=
 =?utf-8?B?SWJTNk9HRnJuVVNDdEUwYlkvaW9zV2hGUjczYnI0K3U3NnBLaStjZGM0Rjlk?=
 =?utf-8?B?TzR0VFlQbGpYOU0yQndpZDEvMkFRejJhQysxdFczVS95V29uS3JnMUNYbnlM?=
 =?utf-8?B?Y0hjdEJ4NW9NbCtjUVI1SVlOQ2tVWlNNdGU4TjhQdmlRRzJVM3hsZmNjU1hP?=
 =?utf-8?B?TzFKTFluakNnc0VHY1lyYjZKWW1pQ1ZtSlgwb0hDZTZSSXlROHovSTNneFls?=
 =?utf-8?B?NWtXanlBOGgrV0p5Qy9WR1lOSnhJOUY4alo1eGF1bUdSaG5RcC9rbEZkdFM1?=
 =?utf-8?B?VGlFVEJ2eEhteWlVcysxbk03bkU1R0tYcEZzSmhXdElBaVdmQW1nNTdHNHdR?=
 =?utf-8?B?MnVwN0l1aDN1VGNqSFkvREpmSlZId09idWZDdWRJbXh2Qk9JVkhBbWc5Mk9D?=
 =?utf-8?B?YzlRTERac3pua2FKSk1sU3JCSEZmc3MyVncyQU9ZVmRhZHZLZm5QVm1ZeDlv?=
 =?utf-8?B?Q2RVbXZRSXFjUnk1djArYzN5MjZyRDA1WEFYeFJReEt2c0JldG1uNTJGeHd0?=
 =?utf-8?B?SEJKL2xCZnd3SmpPYUZJOGQ5cWxIN3lKbFNoOVc0ck9BQ25GSkFXVisrUTZ3?=
 =?utf-8?B?Q0N1Z1phQkdzMS93Ty9BeVBNUHdneDgvYXc3SlduVU5UYW5CdHlSbnYycXEx?=
 =?utf-8?B?RjFrenZob1ZVTXM5a2RNemhHZnVJK2RmWVB2SkxqSlFnKzJMQkNobzNyRWpw?=
 =?utf-8?B?MjJBUElqdlA5dGkyN1pYZWhCbkl3VGpBcDdmcnNlaW0zdFQwWU0xd2IxOTBl?=
 =?utf-8?B?ZTB3OWI4aU94WFZVT1JCMmVkWGZMRlhxbDBoMC9wUzcyVTNwV042MUdpcGo1?=
 =?utf-8?B?OGV3MEE0M205cEYzUkZmNzdGekdzK0NSY2ZHNnlYWVFrUVc0ZEFpcm96bmMx?=
 =?utf-8?B?d1dQSHVwTkxpQ3lpQmRFYkpjZ3BsMkUrMHVhbmo4SnBoRXlMc0JvRTk2MzR4?=
 =?utf-8?B?VGoxanVMc1pvZkQ5citjVjFuOWxnVkVlaUxhOUh2amgxY2t4OXJsSjBlRjV3?=
 =?utf-8?B?aTNRSFJ1NnlNeHZYcnBGNTlmaVVqSnZ0VjFUNzVVRnJYRUFOTXFyQzBUV2Ev?=
 =?utf-8?B?VFFWMERBN3Yrd3NyR0N1Q0JVR1pUemQ0VmhqWElmQ2RjZ1ZyeU5yUVQ1cFUy?=
 =?utf-8?B?dHpab0tmL1NBeElaNFZiZTROU09RZ0pub21OTldNY1FhRDlxSUVrZjMyV0tY?=
 =?utf-8?B?MjlnclpJUmEyM3ZoTVUxR3ZMTURFMi9xQ1NQWVV5RWRoMmxzK3paQUhYVHR6?=
 =?utf-8?B?Q3A2ZmdSdEFRWFRMWGV4OW15a2xaamFoc3VxSE8yMUd4WEhtODJ4MnZBQnNm?=
 =?utf-8?B?VGxzLzFKWTFiaDJ3VGJPdUpuTVpaaTh1R214SE5IcWdUNStmQXkxQmtlYWpu?=
 =?utf-8?B?Q3RHU3l5bDE0aHZwTHNUcS9Td0Q2cVAxWE1VcE5hSkc2Zi9RUVZTNExFWjZH?=
 =?utf-8?B?NG5uQk1LdVpnUlZWZlpzWWl3alh2TEJLY0x3djZEclZxMXBRMjdFNkV0dDlV?=
 =?utf-8?B?NEE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	XNLqTmJd8bE90pSHuiFRJRPR+R2t3HCmDT7F20LhdYyZ7MnZQ59uBVZbo2+EZ7q6b7IYD821qbaXWG2xpxk4KhvZnL2qjhqpT167Gtso3UZLJ122DIglY9cf+vTtxvsXXrs8KwAJpVMRE2F1hCpCB59c9yadYL5okXvN53uik8/W2ZAS2FY/RCMWRqsLUQ2PnQn0tUnVXr1NOqG+FG6E9OrO/uIeK63ESl19M/8xwJWIvsRUO0rCFSsWVj4WtOtp2LfO9TqlMo3dUgqScY7Z3VL+gD09mnmuXxCrS+Qvn0JMGPON/eY/Jzvr9n7fyiN1xy9X1EDjNcVdAePZrAoHvHc4dwhrAjmZ8rVR3n42uyca/kuT+AsTYFoMrRgQPECivXm16uDXRo44Gd1MzxgwCMc/O5lHdm2FJ8IHlRgRidw35E1uhtLWT1aOcyG/JPFEYvc7N7sw8qgfUPKs3IH3wiYnTS6p9MbwdFXQ1sVKTR163TEOgqhyGWBlFEJXPRv5JUeXg0DTPEwqJGB62AQMI1HketXEYJAl78SxBXOVp4Ss+z/79cYlvU8E1JqiD7owV6NY3XNFFlS2aZc8LBDB51eH8Kg47O/TofzNHhv+EKA=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f2e82c5-df9b-40a2-1205-08dc12c0e159
X-MS-Exchange-CrossTenant-AuthSource: CY8PR10MB7243.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2024 16:18:00.5497
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: A1PwjD0nC73bBw9tkzSpthHDcRye3AsExAa0T0AAvCHG2600wTetMfHP+vv/Ml+HNNFgDODk63+p5iazRw634qoOr9Zwv9vaz/gQ+im5nzg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB5896
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-11_09,2024-01-11_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 adultscore=0
 phishscore=0 malwarescore=0 mlxlogscore=999 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2401110128
X-Proofpoint-GUID: ldSkE6H0h7e7Z-vu91NS8hMU_nXn1ZX6
X-Proofpoint-ORIG-GUID: ldSkE6H0h7e7Z-vu91NS8hMU_nXn1ZX6

On 1/10/24 9:09 PM, Jason Wang wrote:
> On Thu, Jan 11, 2024 at 4:40 AM Steve Sistare <steven.sistare@oracle.com> wrote:
>>
>> To pass ownership of a live vdpa device to a new process, the user
>> suspends the device, calls VHOST_NEW_OWNER to change the mm, and calls
>> VHOST_IOTLB_REMAP to change the user virtual addresses to match the new
>> mm.  Flush workers in suspend to guarantee that no worker sees the new
>> mm and old VA in between.
>>
>> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
>> ---
>>  drivers/vhost/vdpa.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
>> index 8fe1562d24af..9673e8e20d11 100644
>> --- a/drivers/vhost/vdpa.c
>> +++ b/drivers/vhost/vdpa.c
>> @@ -591,10 +591,14 @@ static long vhost_vdpa_suspend(struct vhost_vdpa *v)
>>  {
>>         struct vdpa_device *vdpa = v->vdpa;
>>         const struct vdpa_config_ops *ops = vdpa->config;
>> +       struct vhost_dev *vdev = &v->vdev;
>>
>>         if (!ops->suspend)
>>                 return -EOPNOTSUPP;
>>
>> +       if (vdev->use_worker)
>> +               vhost_dev_flush(vdev);
> 
> It looks to me like it's better to check use_woker in vhost_dev_flush.
> 

You can now just call vhost_dev_flush and it will do the right thing.
The xa_for_each loop will only flush workers if they have been setup,
so for vdpa it will not find/flush anything.