Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <81e1a94d-531a-2e43-baee-da4b35644e87@oracle.com>
Date:   Thu, 18 May 2023 10:03:32 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.1
Subject: Re: [RFC PATCH 5/8] vhost: Add callback that stops new work and waits
 on running ones
Content-Language: en-US
To:     Christian Brauner <brauner@kernel.org>
Cc:     oleg@redhat.com, linux@leemhuis.info, nicolas.dichtel@6wind.com,
        axboe@kernel.dk, ebiederm@xmission.com,
        torvalds@linux-foundation.org, linux-kernel@vger.kernel.org,
        virtualization@lists.linux-foundation.org, mst@redhat.com,
        sgarzare@redhat.com, jasowang@redhat.com, stefanha@redhat.com
References: <20230518000920.191583-1-michael.christie@oracle.com>
 <20230518000920.191583-6-michael.christie@oracle.com>
 <20230518-lokomotive-aufziehen-dbc432136b76@brauner>
From:   Mike Christie <michael.christie@oracle.com>
In-Reply-To: <20230518-lokomotive-aufziehen-dbc432136b76@brauner>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YjNRY2xCUnZWZDBsSW5kSUl4dnFGaXN4SFJFZXRMTVJ0eGo2ZW9OYmV4cXF4?=
 =?utf-8?B?N3JKc1p3OU1hZUNwT3BJZGh3L0xlRGJWS01HQWNhQ28rZVVoNXRmVDBYdisx?=
 =?utf-8?B?bm01VFM2cUxWYUFqZSszUXRXZk1CVVYrQVczZU1kNmwzdFI0ZXVCWFVWZU5a?=
 =?utf-8?B?SmQ2MWxXVmNCVDRMTVE3T09NRFgvT0Z3SG1QdUFLL3pzeDNuWjdBWlAvTnNJ?=
 =?utf-8?B?VHp5L213NzhLMlBRdXkvZ1NyaElKMXUweWFwRmNtMG1CSHpkakJTL2paMnJX?=
 =?utf-8?B?Y0F6RzBqV3BxckkwL0cwV3VCUGxqOFlFeHphazNVcjFzcUtzMyt6c0o5dDF0?=
 =?utf-8?B?bXZwck1vRFFkSmxKVTFjMVN2VlA3UWgzR0t5TEp3K0VyTzFOR2FyWU9IWlhl?=
 =?utf-8?B?VFpOTDdwSXlWSEdlcTg4WUF5T3lodXE3M0NzVm94SXVhOTNPNXdvNFhsUml6?=
 =?utf-8?B?KzFJVmhTR09KeFNwQUhQd09Gc3JwNEQrcFlBU0xrN00rcGRRdm54SjRnWkxG?=
 =?utf-8?B?dkdUbVVSY1NhZ3FIZkRQd0o5b0ZQbThOY0h0WEpkcmprQUltK3pHR21uaWJU?=
 =?utf-8?B?dVpDRjRIZjc2NEZwbVd3dE1vTldHTHY3YitCaUdRcU9wVXZDdVVoejZidlpI?=
 =?utf-8?B?WC9JOFc3aU5TZWs1VHpwencwMVgxVkRFOXdFZWt0bTZ4NDhEdEpzMTMzV0Ni?=
 =?utf-8?B?YW1ReXM2SW82SWpScjVraUdXOENEUTJkSUNzNlZhaUYrakNQRTZRUWdMdmM0?=
 =?utf-8?B?SnhRSkhvdGRYNFppL3JmYXl0VVc5SDQ1RkJXS2ZhQ1YwbFY2djNBOGljM3My?=
 =?utf-8?B?dFBYVVlpZ1VBdHMrTlhwdmZpS0srL1hDaWVTWWx1ZGtBeDY3MHZoWlE4eE1p?=
 =?utf-8?B?Mk1IaFMvQWg1SmpYOWtCa1hnSURSQ1RrM3NwZzJvb3pqa0hiVGo0K1hmb3Vl?=
 =?utf-8?B?ek9BSWZNQzFzNzlnUWIxUTN2UHB2VU9mSTBTTzBxamlyRSsycm5UUkdWZVF6?=
 =?utf-8?B?UGJlcDNjWkhZa3dHUVg2Q2N0YktPMEFsNk5OelRMTHhWTzlnZUlxQm9JYkdh?=
 =?utf-8?B?YU5CVWJGRk80ekZDYmg5WHhkaEcvRjFLQkV0VDU3TUdKS21jR0M5WnpZYnY3?=
 =?utf-8?B?YzJtaEhPU2p6cGQ0c1NyeDg4SGFlM0tLa0JsUnoxd25ybXdDazVXUzNOY2Rz?=
 =?utf-8?B?UmFrUkNGaVNnZDh5Mzl1VFk1emYzQ0VwbHFzN2JVVitkTHhVUmQ5TjNmOUVa?=
 =?utf-8?B?QXAzeWhMQ2tnbmJwMkd4NmgxT1RERnlaSWZwUUdqano5OE92a2RYYm5XTXFj?=
 =?utf-8?B?VzlVYXZ2S2hlOEpvTU91WHJJc1VIZkt4amZVMjdOblZSNytrR0tObGZKSnU0?=
 =?utf-8?B?NDdkVmZUYXRKaG1DdlFWcGFrL2FQdGJBa2hHeDBTSGhGVGMwL2ZyRXprcE92?=
 =?utf-8?B?NktYNFp4ZU0vV0RndzdIK2lqeXhWL2xkeFFDb3B6WGNPdStSazFrZkFoc3Ez?=
 =?utf-8?B?TVhtTm1BbTJQemlZblZuTjRocWtmeTYyTHNhdi9yYVljMU5WS1dXM2JLMlpl?=
 =?utf-8?B?TXhmTm5uakxubVF5citqekFuU3c1TWh2UVdwSTBLQWdrVE5ZL2tjejNhZ0Nj?=
 =?utf-8?B?WCs1T3l0US9udHdHV3BKVW41L2FjVjdqb2pqZFBZUkMyaS84NDVwT25PZFRs?=
 =?utf-8?B?bVY2bDZUeDNKZC8vNkhOZ0o3aUVzNUZUWGU3VGtjYVdVOFVHL2Y2KzdyMzU1?=
 =?utf-8?B?VWkycGM0NU1IdDB2KzdvZEFYMktoL0x5N1hyVkw5TUE0STU4aS9rc3pFa0JB?=
 =?utf-8?B?Q1ZyRThoNmJydWhmeDBabi90MkVDa1A5Smw5TDF4cDVjclFqS0V6WEkwSmQy?=
 =?utf-8?B?WXVUNUdkYktDUWlObXRKYXozb1ZnQ3pzb0wwUGN0NWNvdTV1c2YwdkZ3dFNz?=
 =?utf-8?B?WVVibTBVN2ZIQkJZVitiam8wY29zbS9qWGpNNHRtNkpKNWhqV0tnSnZJUVFW?=
 =?utf-8?B?TS85cjJTUmpuR3hpL3UyekJNYTgxRGdoNVdMbEpCRHo4YThJRFExeG14ZHQ5?=
 =?utf-8?B?aHZBUW5zdEMwSkVabmhXN3FNcDBXd3VnQTJJSEk4T3R5YVYybk54Q2ViVkJr?=
 =?utf-8?B?NVdoTDVyTklmMUVrbVpyQm54dW4ydUdSNWZIbDQ2bVhmYW5pUCtGN0RNbnFJ?=
 =?utf-8?B?OUE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?b1hBSnQycnVyNy80RFJjc0lCanZ0SFN0a1Q5TW5JMG9EMWdzUDZPMzV6WVJp?=
 =?utf-8?B?YUttUG0yeVEwamZtd3hvNGdKWTN5bXkvT3RycmNjUEliZEdXRUoyWW0wTXI4?=
 =?utf-8?B?eFBWM3BSLy9mS1JXYTdGM2RUMVJMWkhURndFdE9iRVlJc3F5Sk0wRGpVay9i?=
 =?utf-8?B?OXRLZmkwQ0ZmY1FTY2xnSW1pQXNvWmtqbE9jZi9kZXdlemdWYzZ1a0FDV1Bo?=
 =?utf-8?B?aENnNlRVcVQrdDFRVmVuUXRQVkZRVEhLbjVvTEQ3aUtWWlBoSTVWanNKQUhy?=
 =?utf-8?B?MlhsRkp3SUVkazNZZm9ZOUJYOGpIQWJqUXU5RE5hYis0TGcyaGhIclVtN1Fx?=
 =?utf-8?B?bXJhajd4MHVpdHE1YWRxTlhQd29jdDFHcFFVL0FHcCtsMFBoVnNlRE1JeVNQ?=
 =?utf-8?B?YnJVMXIwRFpId3AvbUFaRWVZSXhicFhuV0Z5eGdVckFzUjZ3VGVuTGxRQWpy?=
 =?utf-8?B?L1pKTE1oYXNaVTM0WXFOSVdWL0NzazVKNDcxdGY1aUVneVVXYmY5ck9TRWww?=
 =?utf-8?B?WVZNRU1qdnhqMG1DYkRwQWFnRjIwV25sU2hLOHU3Unhoa3dIWGxVZXVsaGJG?=
 =?utf-8?B?Njl6cG9zVExKOXlxaHpWQnFwc2VraFNBd3JOcCtHMVEvKzdpbFJBQmhHVmZ2?=
 =?utf-8?B?TEozV1dVUFlaTVZGQU9odnEwa3dhOU1Bc1pBMFVZNnB4bld3YmJKM3Z4WHg4?=
 =?utf-8?B?NzRORG1BSzBhV2ZqaGlsL3pSaGJ5cThsMk80Z0J0RzdUZXVXU2ZPb092NXNj?=
 =?utf-8?B?eFlmZnM5TXdUMlRJVk1lV2JsTWRqNm9MUHFHcTM0NGVvOWRGN0dZWGpmcU04?=
 =?utf-8?B?a0lxeXFsRnVJdkJlWnQyYnRIRWZnejg4RlBqVEY3K3ljNVlmYUo5ZkMrVXcr?=
 =?utf-8?B?Q1l1bTlwWWI2anFONUR0OGRiNmE1V1JHSzdGcCtuYkxOUlZKSVJKdFhIMVgz?=
 =?utf-8?B?bDRQTGxJQ0hnZElGdWRBV0hxUHcybVkxY1ZPajVuWVQ2UmRUTEtIWHJkbzRK?=
 =?utf-8?B?ZjM4N2lEZ3JiZHdXcXZ4dTYySHdHV2pMSGd4VUo3NEtETFJxeVZNK2w5d2JU?=
 =?utf-8?B?cmVpWTcyd2xONjI1cUw4OXd2dWxhamNnOVpwOHFIdzV4clBNYlJMM2t5ZFAv?=
 =?utf-8?B?czN1U29GU0VGS0IwTXhTdHZ1a0F5TDNPeFdlZi9ybXpQSWdHUmk5MDMrcFVY?=
 =?utf-8?B?emZOazNjNUl6OTdzdVZFdjk3ZVNxZHl5aktLaG92S0czaHJlT2ZLQjlyUXdR?=
 =?utf-8?B?UHA1dGZjZUw2M0JiYVpxMDNpR0JTeDF6bzhIYkwwTkplbllzQUdDeExrYUk2?=
 =?utf-8?B?WERPdW9hZmZTWWNWVDZiekUwb1ByNmo4ZTJNVExzWUsyTU03NWNEVE9QNkVV?=
 =?utf-8?B?eXJ0M0thbmV2M00vTC9SSkhJaXhUTS92UG81aGVrK3p2d2Yxa2w5VHBWcFF3?=
 =?utf-8?Q?yBuEgPLI?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 406a05c9-3780-405d-8248-08db57b10cc2
X-MS-Exchange-CrossTenant-AuthSource: CY8PR10MB7243.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 May 2023 15:03:33.9613
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nRKmQ6QC6nazCGWxEhwgECDf8Fq0dJDsx9AIR/H5HgbI1B+Ajex2qXMD8IbvH0Qx7+cskKL/WsS6mxxaGJ/I99ZROQUpN+qUVGxAawHm+xc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB6986
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-05-18_11,2023-05-17_02,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0
 mlxlogscore=555 malwarescore=0 bulkscore=0 adultscore=0 phishscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2304280000 definitions=main-2305180121
X-Proofpoint-ORIG-GUID: gC1aYA9ZWHR8J0V6V0r7n-hwpOYJNjYv
X-Proofpoint-GUID: gC1aYA9ZWHR8J0V6V0r7n-hwpOYJNjYv
X-Spam-Status: No, score=-5.4 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 5/18/23 9:18 AM, Christian Brauner wrote:
>> @@ -352,12 +353,13 @@ static int vhost_worker(void *data)
>>  		if (!node) {
>>  			schedule();
>>  			/*
>> -			 * When we get a SIGKILL our release function will
>> -			 * be called. That will stop new IOs from being queued
>> -			 * and check for outstanding cmd responses. It will then
>> -			 * call vhost_task_stop to exit us.
>> +			 * When we get a SIGKILL we kick off a work to
>> +			 * run the driver's helper to stop new work and
>> +			 * handle completions. When they are done they will
>> +			 * call vhost_task_stop to tell us to exit.
>>  			 */
>> -			vhost_task_get_signal();
>> +			if (vhost_task_get_signal())
>> +				schedule_work(&dev->destroy_worker);
>>  		}
> 
> I'm pretty sure you still need to actually call exit here. Basically
> mirror what's done in io_worker_exit() minus the io specific bits.

We do call do_exit(). Once destory_worker has flushed the device and
all outstanding IO has completed it call vhost_task_stop(). vhost_worker()
above then breaks out of the loop and returns and vhost_task_fn() does
do_exit().