Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <c6e9a5db-798f-fa40-5ae2-a41f2d8ebab5@oracle.com>
Date:   Thu, 25 May 2023 11:15:59 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.1
From:   Mike Christie <michael.christie@oracle.com>
Subject: Re: [PATCH 3/3] fork, vhost: Use CLONE_THREAD to fix freezer/ps
 regression
To:     Oleg Nesterov <oleg@redhat.com>
Cc:     linux@leemhuis.info, nicolas.dichtel@6wind.com, axboe@kernel.dk,
        ebiederm@xmission.com, torvalds@linux-foundation.org,
        linux-kernel@vger.kernel.org,
        virtualization@lists.linux-foundation.org, mst@redhat.com,
        sgarzare@redhat.com, jasowang@redhat.com, stefanha@redhat.com,
        brauner@kernel.org
References: <20230522025124.5863-1-michael.christie@oracle.com>
 <20230522025124.5863-4-michael.christie@oracle.com>
 <20230522123029.GA22159@redhat.com>
 <cfca7764-d210-6df9-e182-2c093101c6cf@oracle.com>
 <20230522174757.GC22159@redhat.com> <20230523121506.GA6562@redhat.com>
Content-Language: en-US
In-Reply-To: <20230523121506.GA6562@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bExnUTFZSE5va1NkTzlvUXJKbkJFNkxlR0JUMUVWS2RqZTR2dG9Uay9nbEJo?=
 =?utf-8?B?aFF2TExRMjBLcUxTbG5FNVplYWF0enQ3VFBzQ056Z0dBL0JHQ3hSWmxFeTM0?=
 =?utf-8?B?MzAya1FYUXM3QnFCQ2xmWXB4MVNRdmJRSzN5dDQ1Sm5YRHd3a2l4RW52eG4w?=
 =?utf-8?B?ZXQ5SXFkcVBzLzdKaFZTaVdVak5WUmpwajdMVlN0UTJPZlJYcGpKVFhsNmlz?=
 =?utf-8?B?VC9zOTM2MXcxMXZqWGZOaUE5VVdqOUcrcUVTUlVpS3VncTVjUkRWSGJGN3hW?=
 =?utf-8?B?WjBOK1BSYklKUzhxbkNmVFllMjRUS1l2WEoxeUNpbzBEM0luUllIU0xoemFz?=
 =?utf-8?B?enZ1Ylh0bHF0SCtkaHRONFJzbXJVYm1kaU95bkNkNXdOblBvR3EvQjRNNEpw?=
 =?utf-8?B?SG5kYWE2bzJ4bllSeDhkVlc5eS9nK1AvbW16WG9VV3lzQmUrMDVTY3N1WnZa?=
 =?utf-8?B?WWpTb2h0VGlvRFdaQ3dtSldXZGdvOVlCcUpkRWxDZ2pPc0grR2tNM2xvRGhm?=
 =?utf-8?B?cjNENE9CU1ZiaXUvcXpsK3dBL005UEd6bE5aRDBXUXNzd3YwRFpOaUpLd09I?=
 =?utf-8?B?Y3BJdmgvNEpKNER3bzY2LyszWEdFSHVVcDNCNi9iNWwvYUJMaFBKWWkvRkVN?=
 =?utf-8?B?a3lKUXBDdFRZMDJzZUNsTU9qNGptRU9ReTBVU2JtcG1ydzVZdjVObExkb213?=
 =?utf-8?B?cHFNRHRsbDZhQXUvS25iTTdQMlplUnpXaU5nSXN2UmRkczB1QXdMOEw2T0p1?=
 =?utf-8?B?VDVGMEMrdlE0dktjYVpKT25leC9UVUx1djRvdnJlUENmTmFNYzJoZTA4NlJW?=
 =?utf-8?B?b1JOSW56L25TTEY2cEdWTW1LaURDY3VXbHhuTmlIMTNOR0pydTY3bDc3SXNk?=
 =?utf-8?B?SzYyVmYvbW9DS0tCNGwwM2lTR1Zxb0ZyaTJoNFhLWFRicGxjN21Ld2FhNVIx?=
 =?utf-8?B?NWhvdVZoTS9WU3FDdWJITzJPZ0Z5aDRCeGZWN0J2SFhVNFpuY2NoV0svSFFT?=
 =?utf-8?B?ekU4ZEhSUG8xd0h0endZeGpzRnhpbFpwSzRvRjlaVDMzcFJsSmpRSlN0ME92?=
 =?utf-8?B?Q0VoTTF5YzREYWE2cmYzUFJuOXpwcFRhZGtFWStiSllPVFR0ZEx2dTJZc1V0?=
 =?utf-8?B?TGFkL0xjdkd1dmtsS2t5L1AxcGs2L2tzbEJMNFQ0QTdlWnFpSS8wMWVBV0g2?=
 =?utf-8?B?NEpUSWxhVEFweDZaRVdjNGVHdWhMQXNwbHRLcXp6TUZDUlBEYmcyRzcwR0M1?=
 =?utf-8?B?ajhsckV3TmpiSzIreStFMElXbDRpRU85a3ppRmZCSk5SSHliTUNkM0YzZU1r?=
 =?utf-8?B?cStva2tMSXQ5ajcxK3dOb2c3SnFJaWREeHR3anZZdnVMcTdNVFFHdXpYTEJP?=
 =?utf-8?B?MnRtUVdnM0k4VlJIWG14TzRUWWpQRkVJV2UwOG5GbytCK1BpK2w0dzJpemw1?=
 =?utf-8?B?bnd0QlRxdXdJMy9PVm1uSy9CL3lEL1JldFhHUlhuNUZlT0JzTWJtU0dKNkVk?=
 =?utf-8?B?Ny8wcFFnZHBqT0lEUXdudFRSR1hGb3NkUFdreFdVT0V1RlFHRnpvbjYvNWor?=
 =?utf-8?B?YUFOQ2FyeWw2K2s0YzZKTzZXZ0kxaWVaR0hkaiswOHRtYzd1SnFuajNCU3Ir?=
 =?utf-8?B?TWw2RTJFazEyakZld2RpYW1KbS9lWFJvdjIwaXlYbDg3L2VQQlhuOUdTNWZa?=
 =?utf-8?B?cHg5eDMrbDJkMlgyS0F0MHIrQ3pibEwybkFRQnpuaVUzcFp3NlMvVk5qLzhy?=
 =?utf-8?B?aGZBeFNRR05mckpFTGJoK3RDWjU1dVUrUllNZ2hSc1kwNDJzM2ZFTlN4SC9R?=
 =?utf-8?B?MGwrZ3d3Q2hxMEFqaU84TXdNanJGZTVsaTdxVCt4eXNFQ1FZZEtReGFtTHlq?=
 =?utf-8?B?SVIrMnE1dXUvSFVHdzhmVjFkVHdtTVlGazI0Y0xTVkdtREptMVJFYnZsWmJ2?=
 =?utf-8?B?UURidEUrc3hvYVJlRTRRUzRGWG9kc3hocDd2TW9sTzlMT3ZaTFpzOHpTZy9y?=
 =?utf-8?B?bTVyMmFpUjdub2N0R0toc2s4WVdGNkdzS2UvNVlyOUZoWHZ4aTBpZFY4cUVU?=
 =?utf-8?B?aGM5U2ZqazNUdWwvcHJqSVRXWmtFWjI3Uk1WdE02VzR2VnV0OTVWbGhnaWRl?=
 =?utf-8?B?SGlqdllBK3JVOFhEdmxneEZ5OXZXaUFiU1ZxWVZMM2ovSXJubzZkMERKUFlD?=
 =?utf-8?B?dXc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?T0tmeW9PajJpb0tuZTJ4NDZSUm0rV0lnN2F5eTVOeEpib0lLa3dNWmorRDFE?=
 =?utf-8?B?N0hWQnJpWmFKL1VKRjRrZkUxR3RFTHQ2MWg1RVFoMFdnNXV6ek1WUDhxVkVa?=
 =?utf-8?B?VmJNWWxqeHRpMllZQ0oxTVk2Z01EL2pJTktadWFEMGs5NzJFRDRQTDI5ZzNI?=
 =?utf-8?B?d1o0V2RwQ2lyRkpVc3FzMk1UbjdUNlh5VU9jUkZiNTRvNkE0ZGdzZS9uMUg5?=
 =?utf-8?B?VURtUTlDdW9objNXV1h1SWZ3T0kvaENjZDJBTmc5SWdXSm1ycytMSm9FWDlj?=
 =?utf-8?B?bTdkMUFWaFFGVlBidHYxNk15YVN6TjNTTkZOOTkrZnFiaXBkdlQzcFhlc1Bt?=
 =?utf-8?B?Zi9HS1R6WlpFQUhuZ2JJL3VXcWJXa2dPbjFpaldaZDFxekM5eHd0a0w4aVlt?=
 =?utf-8?B?bVArN0pLdlI1S0R6djR2Ukl2UDlSQXI5UTBud3ovajNOYzIwTm51S0dEY3d2?=
 =?utf-8?B?TkY1MHBFSDZBOHdNZmlQQ2hiZkEvbzYvdjVEdDR4dnNGME1pcjhaalcva0Ru?=
 =?utf-8?B?RUMrdXQwS09CRGJJZVh0RFBuc2xPaHZDdlduL3FEdDk4Qm1NQy9SSGlhWHZp?=
 =?utf-8?B?c2VQMzdha3ExMm1jTnN6YTlSeWUwYmRmeWZ5U2xpK0djbEh1bno4U3Y2eDdJ?=
 =?utf-8?B?V0RFVkcxd2dVWGhQdDBIdnR4UWVULzBIMDZUa2ZFNnFoaFA1eUhVZlozTXFt?=
 =?utf-8?B?T1lOV1ZFMSsyQlVqVHFmNHBWdUhFY1h6ekhqbkl2NS9yakhFd1dsazdjV0tN?=
 =?utf-8?B?OThHRDcxRlJ4R3M1cWUwU0JiNjQ3MjlLTlVpajJXTGVadE1WUDBZNjAySDU1?=
 =?utf-8?B?V3B4Wkxya3NPTy9ocXNNMnFoNDQyMTg5QmtSYy84clRidUVTODVqZXZxNUNr?=
 =?utf-8?B?Nm1JVWgzZFcraGZHTlFuN3ZWaE1tWkhBOTQwanc0YUUxNjJXTTVHR2VJdzZO?=
 =?utf-8?B?WkthTjVmUDEzZWVBOWMycE5QMXN5NlR6ZExQZXVkNUJhWjZoS1puZFBvekhm?=
 =?utf-8?B?bndreFBtN050T3FEbGhNd3hhNVhwcXpBalphWDh2Q3FhbDUySXpmcTdSUTli?=
 =?utf-8?B?and6dXFpU1U0bjR2MW11Zm9JRk5GbTA4UHVqUnJYNCsxdEVaZkE3ZmpuVjZ5?=
 =?utf-8?B?YTZWRkFSSHVtYlRaMURySE9JdFVCcDROY3dLVmtqWW1XTUJWTkZ4cmZRaVlB?=
 =?utf-8?B?RVJsODJTZG9aWHY0enlobmd1TEpjU2xHdnhWMTM4WGpxbXAyZFgrTi83eTcy?=
 =?utf-8?B?ZU9GNWhaSHlvOXo5K2d6YXN6cFdhT0FjMmwxQ09sY1VMT3RUVmRUUm9uWmdU?=
 =?utf-8?B?YTFHNzlLb0dQSE9QZ290MGtIT3JsRXZyazB1RTBhVUNHNFMzNHo5S3l3eFZ5?=
 =?utf-8?B?TzFvbjhySGNEQUxKUWdrbHBKSnVFMDFsOXhzSkFicklJcGRZSEpGbU9oTnp6?=
 =?utf-8?Q?6sBm/xdT?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7607cf88-57ce-45f2-d160-08db5d3b54fd
X-MS-Exchange-CrossTenant-AuthSource: CY8PR10MB7243.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2023 16:16:01.6126
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qsYE6+zQXFN0r2KUvVFC3DqvbDEzoGOeCkL/nVM9dJ8g+GksEybKBdfUHCetX86zQoQMRoBOAwEercrNiUY9RICYgH8cY4MJH6eAkRZuvco=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB6680
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26
 definitions=2023-05-25_09,2023-05-25_03,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 malwarescore=0 adultscore=0
 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=806 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000
 definitions=main-2305250134
X-Proofpoint-ORIG-GUID: lqjMPB4dUQTywigd_Ap5yligxHQe-nxP
X-Proofpoint-GUID: lqjMPB4dUQTywigd_Ap5yligxHQe-nxP
X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 5/23/23 7:15 AM, Oleg Nesterov wrote:
> 
> Now the main question. Whatever we do, SIGKILL/SIGSTOP/etc can come right
> before we call work->fn(). Is it "safe" to run this callback with
> signal_pending() or fatal_signal_pending() ?

The questions before this one I'll leave for the core vhost devs since
they know best.

For this question and the one below, when we get SIGKILL we think it's ok
to fail the operation as in it's ok to not execute it like normal (send
it down to the net/target/scsi/block layers for execution). However, we
need to do some processing of the work to release mem, refcounts, etc.

For example we use the vhost_task to handle completions from the layers
we interact with. So when we get a SIGKILL, we could have N commands in
the target/block/scsi/nvme layers below the vhost layer. When those
complete, the current code assumes we have the vhost_task to handle the
responses. So for pending works on that work_list, we can pretty easily
kill them. However, we don't have a way to kill those outstanding
requests to some other layer, so we have to wait and handle them
somewhere.

If you are saying that once we get SIGKILL then we just can't use the
task anymore and we have to drop down to workqueue/kthread or change up
the completion paths so they can execute in the context they are completed
from by lower levels, then I can code this. On the vhost side, it's just
really ugly vs the code we have now that used to use kthreads (or in
6.4-rc looks like a process) so we couldn't get signals and just had the
one code path that removes us.

From the vhost point of view signals are not useful to us and it's just
adding complexity for something that I don't think is useful to users.
However after discussing this with guys for a week and going over the
signal code, I can see your point of views where you guys are thinking its
ugly for the signal code to try and support what we want :)


> 
> 
> Finally. I never looked into drivers/vhost/ before so I don't understand
> this code at all, but let me ask anyway... Can we change vhost_dev_flush()
> to run the pending callbacks rather than wait for vhost_worker() ?
> I guess we can't, ->mm won't be correct, but can you confirm?
> 
> Oleg.
>