Received-SPF: pass (google.com: domain of linux-kernel+bounces-29418-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <3017552c-cf49-4357-8bc9-d47593024e29@oracle.com>
Date: Wed, 17 Jan 2024 15:30:50 -0500
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC V1 07/13] vhost-vdpa: flush workers on suspend
Content-Language: en-US
To: Jason Wang <jasowang@redhat.com>,
        Mike Christie <michael.christie@oracle.com>
Cc: virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Si-Wei Liu <si-wei.liu@oracle.com>,
        Eugenio Perez Martin <eperezma@redhat.com>,
        Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
        Dragos Tatulea
 <dtatulea@nvidia.com>, Eli Cohen <elic@nvidia.com>,
        Xie Yongji <xieyongji@bytedance.com>
References: <1704919215-91319-1-git-send-email-steven.sistare@oracle.com>
 <1704919215-91319-8-git-send-email-steven.sistare@oracle.com>
 <CACGkMEv8TXXTrCW=hf1sJ+V2a0eD3w1b1Yub4FnGZ0=Du-RbtQ@mail.gmail.com>
 <11ebc73f-6469-4c97-9da0-0728edfb283a@oracle.com>
 <CACGkMEs1sf6NWT76fM+D715kqrwJ56SAXrcExKvWH_AYsjcGJg@mail.gmail.com>
From: Steven Sistare <steven.sistare@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <CACGkMEs1sf6NWT76fM+D715kqrwJ56SAXrcExKvWH_AYsjcGJg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?OCtwZ1g4TUpXRzNteFBaNXpEUWd0Uyt3QkdIVXFISFdNdG1yODNRM25PZFpC?=
 =?utf-8?B?bWdTOEZEYXRqbHFmRlpUQi9UM21JV3RaYVMzVE5NOGc1VFlmalJ0R3QxMXdK?=
 =?utf-8?B?T2oxMndNYWliOC9ScUd6OVpXUGorVDJuRUFnbzhwbFVhaGE0REUzR2U0d2FQ?=
 =?utf-8?B?QVlRSlRUNyt5Nm52dGhWclB0a296RW1VTWpCbE9iOHFrNGVXd053cnUrVWVo?=
 =?utf-8?B?a0puMWpCVldqVGJEa3RNUmN1U1NxMjl1bE85NWJXRUdLNnRldThWeWJ1S09l?=
 =?utf-8?B?WVlBQ095TkdHUXVoWFhzR0J0Q0hxRkQ0TzBXclo1Z3paK0tKeU9oVGhKcjcv?=
 =?utf-8?B?YXBqbTZGNkhoN01jd0RMZmV0MWUwYWtWekNFZWYzcm5kZmtwQXd0N3J2d09m?=
 =?utf-8?B?NWViQWRQaFdPMlh0Z1o5RjZWN2Vzbjg1TjVVdklBYzZydW5PaS9DenBRV3VQ?=
 =?utf-8?B?am5yejVjZlhmK1ZnV0xUdUNtTGFFWkp2bzdkK002MFZmR29ZUUhJYzJOTUZo?=
 =?utf-8?B?V3NWemYwcWtSTStIbzdvdkQyZ1k3VFhXUUVFMjlXaTJuQmFzQVg2ZVU1cUp4?=
 =?utf-8?B?QjRqTzgza1pLVTBpa3hXTFYxYjRSbkJDZ2NMbmdacDg5WG9zVjZ1bm81UUVE?=
 =?utf-8?B?VkdNcmFVcWQ5VDQ4SE1rcGJkMi9uSnhCVi9keHFYRCtFWWl0VU9BS3E3a3Ni?=
 =?utf-8?B?cmowdG9PRkF4MU5zVjkzTE9zbnAxMU85VlJjelo3UFNNQUhabHlFQjNmdUNP?=
 =?utf-8?B?ZUp4K1NJUkZiS1RaZlV3YVgyVkJRYmJaaEFyakd4UmVYSnF0RnVvdjc5dnMx?=
 =?utf-8?B?bTEzSk5UUkpnN3d6aVoxc1VjSU5aNVZzTEJRZEFIbVErYXFVcEpqTGxWOVFV?=
 =?utf-8?B?ZjlPRk84SXlhU2EyODVBb09UTXloL2FKT09KWktKUi9waXhtdVRZVVhFVTRS?=
 =?utf-8?B?UmNHaTJWd041OTBqVkZYNkNpS094WXdLQUdLRytESFhIR1BPVEVObFErK0c5?=
 =?utf-8?B?NGhBQ0hrdmhVbjNnWXJpUmtqUUJHUkd3NFhuTVNXeUgvMUlEQ2hTYXRIUkZU?=
 =?utf-8?B?a0lsMXdTN2FYYWdpNGlPMDhEZ25wRHZPOGRwdDlnK1JzTlRuNVVlVmIrSWhD?=
 =?utf-8?B?WkVoY1NVU2tNSkxxR083ckxRRDlrUlg5UlBEejErSTJPZDVudWhZekh4MGJ6?=
 =?utf-8?B?QWNHRTFSKy82ZHB1cXlKTnlXNFVzTjNYTldOU2ZPOERRRWwvdW9RQkFjSnUz?=
 =?utf-8?B?Rkp0UVQ1Rk9wRXlLbHkxdk9ISVVqVW8ySEpwbnArRzNCaDBMT1dhZlYreXJR?=
 =?utf-8?B?VXVFeWtkOHNBdVNQYXNWUGdpWG4xSmxiTkVjMVVFTldldDFSNXpLWkEwMzRm?=
 =?utf-8?B?dlczclVZSGFPOCtjM0dWU28yaTVEL0F3ekxVOEFlckQyVXVoRngyMXIzc2wv?=
 =?utf-8?B?eUZuY3pKUFFBcUlLSWlCSDRTUGdkYlF4LzhNQzhmdE1oVUk3RWZMLzl2MVhU?=
 =?utf-8?B?YkNGWUlDTFkreWsxcWh1NTRUZERMNWFHWWdvZ0ZHZHZVSlczZU4rWlFKL3NB?=
 =?utf-8?B?NHlBY01MQ0VDMmpKbGczQlkzekVnSG5LNE9ndUo3ak8vc2d0TkFBQXFvZmpX?=
 =?utf-8?B?Y25NcHZJdEYyV0ZId2k3a2VrTG5vWXFrNlFGckF0cUY3WFFMdWZBNU5hRStM?=
 =?utf-8?B?NlJxY2RxUWl3RXYvM0ZYSDEvSWJWR3NTTzNYMlBUZE1vaHA1WGVlck85a0Rl?=
 =?utf-8?B?eGJVM3RyMWZKQVgwWGJMOURjc2xwYmxzS2Z1aTM2RGViRCtlOFZsTXg4aC9W?=
 =?utf-8?B?VWJzNXVxVTljWWJNS2xVM25GRDlSSHRBMzFhYWprcFltSkZiMCtLQ0JwN1Ny?=
 =?utf-8?B?VHdIdmUvQkxubFVDVzUwOHdIcWlkbWNOWWVWVk1sM1FxSG1hYjNZWjh5dU1w?=
 =?utf-8?B?S25YcVpOYUVlV2dFMWRURm5DUHNFRk1FYlcvT0x4anVYYU9xWVp5SHpFWmNl?=
 =?utf-8?B?UTUzZjE4YlN4QW45L2plZHdmTjg1ZEcvZ1dRTVp5TmwvVmY5RjZCSHNuVzNI?=
 =?utf-8?B?c2JkRWNYN1d2cXRGenVPakJpc2dyWS9lRGk2b3dGdGRzTHRiZ0FMaUhwOGlT?=
 =?utf-8?B?OHBuODJrR01tV3lSZ0NHdFl1ZURWVTk1WVVyT2s3enhGU0ZKVE96SDJEODFG?=
 =?utf-8?B?QWc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	HPQoj0SKlGdElHIJoWCF9enKEFKNjSwW5NC7suKwW2w+NVmOWxEn6V49Uq7RiYk2KuWXqFPGw6ZSfDUh1ziYLgdWPjGkVgFgJdPr5CM478w+OZAmZwgjfkVmiUiQ8EhwKJxQT8rvjtl4aigpf50taHZ4pRiucMCP3P/1dtShjvJ7EIb4XyCZqhVICyTeY8XuyrWKnhflQUktFbTKud2PbnzZwrgY6a9pXpXOBQUH3lDYuUHZrn3zW+9OUWAs+hzWg18J9yDlHKpKZVZGTan+FYus2vz6+hNKUsUcAxT+Xke7UPbUMISCJXvo2lxCURKtr2vRnrElIscXICz5ciY+eQZyFzcPa+/g3g0NXkUNb54uqanE6nvZeUj3VkXiSsoiJiyaNETglNSQxw/I/pEcnzWDLhyToOlxGc1IBvWpOQq3C10pOvVUEJtNKsHLfpXBB0p7ZHL1xN5YkoumnvGazqMDMRmQNKMg4xL0XOmgjxoP/RLHNE9RD/33l3D0qsPpXTXVVVieHabb2LhAAlrjZmn7Kzaq57CGxm38tvqMWH03go8+UicFvp1JCn8s3lmQuNSU5npYrMhjpDKZ7FudsFeqVo8BrnnMMK5QX3rDmlc=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 56bd8815-92ea-4236-c62e-08dc179b34ae
X-MS-Exchange-CrossTenant-AuthSource: SA2PR10MB4684.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2024 20:30:55.2172
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ELc4TPRiKyX23N4BGYhSRfmgDFjBSoOF5LFubPKFwelANUp09MA/dPdHXghq1ZXUFi5maUrTtqJBMSoOMdN/oVv+q/WXHI3ThAZIxHRrEVQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB5942
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-17_12,2024-01-17_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 phishscore=0
 mlxlogscore=999 suspectscore=0 bulkscore=0 mlxscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311290000
 definitions=main-2401170150
X-Proofpoint-GUID: P5i-xyvCOKBu2W_za45SPXDHPR0HESpV
X-Proofpoint-ORIG-GUID: P5i-xyvCOKBu2W_za45SPXDHPR0HESpV

On 1/11/2024 9:28 PM, Jason Wang wrote:
> On Fri, Jan 12, 2024 at 12:18 AM Mike Christie
> <michael.christie@oracle.com> wrote:
>>
>> On 1/10/24 9:09 PM, Jason Wang wrote:
>>> On Thu, Jan 11, 2024 at 4:40 AM Steve Sistare <steven.sistare@oracle.com> wrote:
>>>>
>>>> To pass ownership of a live vdpa device to a new process, the user
>>>> suspends the device, calls VHOST_NEW_OWNER to change the mm, and calls
>>>> VHOST_IOTLB_REMAP to change the user virtual addresses to match the new
>>>> mm.  Flush workers in suspend to guarantee that no worker sees the new
>>>> mm and old VA in between.
>>>>
>>>> Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
>>>> ---
>>>>  drivers/vhost/vdpa.c | 4 ++++
>>>>  1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
>>>> index 8fe1562d24af..9673e8e20d11 100644
>>>> --- a/drivers/vhost/vdpa.c
>>>> +++ b/drivers/vhost/vdpa.c
>>>> @@ -591,10 +591,14 @@ static long vhost_vdpa_suspend(struct vhost_vdpa *v)
>>>>  {
>>>>         struct vdpa_device *vdpa = v->vdpa;
>>>>         const struct vdpa_config_ops *ops = vdpa->config;
>>>> +       struct vhost_dev *vdev = &v->vdev;
>>>>
>>>>         if (!ops->suspend)
>>>>                 return -EOPNOTSUPP;
>>>>
>>>> +       if (vdev->use_worker)
>>>> +               vhost_dev_flush(vdev);
>>>
>>> It looks to me like it's better to check use_woker in vhost_dev_flush.
>>
>> You can now just call vhost_dev_flush and it will do the right thing.
>> The xa_for_each loop will only flush workers if they have been setup,
>> so for vdpa it will not find/flush anything.

Very good, I will drop this patch.

- Steve