Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <78c5e150-26cf-7724-74ee-4a0b16b944b1@oracle.com>
Date:   Fri, 5 May 2023 17:37:40 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.1
From:   Mike Christie <michael.christie@oracle.com>
Subject: Re: [PATCH v11 8/8] vhost: use vhost_tasks for worker threads
To:     Linus Torvalds <torvalds@linux-foundation.org>,
        nicolas.dichtel@6wind.com, Christian Brauner <brauner@kernel.org>
Cc:     hch@infradead.org, stefanha@redhat.com, jasowang@redhat.com,
        mst@redhat.com, sgarzare@redhat.com,
        virtualization@lists.linux-foundation.org, ebiederm@xmission.com,
        konrad.wilk@oracle.com, linux-kernel@vger.kernel.org
References: <20230202232517.8695-1-michael.christie@oracle.com>
 <20230202232517.8695-9-michael.christie@oracle.com>
 <aba6cca4-e66c-768f-375c-b38c8ba5e8a8@6wind.com>
 <CAHk-=wgadfsCnKHLON7op=Qs5t3w3PVz5ZDbvbKsfb=yBg=yjQ@mail.gmail.com>
Content-Language: en-US
In-Reply-To: <CAHk-=wgadfsCnKHLON7op=Qs5t3w3PVz5ZDbvbKsfb=yBg=yjQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WCtlNHIwNnhkNXk1NTRjUHpYSXFxOU5zaU9RNUlFK1NiVmxDc05RZldXWExh?=
 =?utf-8?B?SGJrRG5FN2M0Ym5xYjhiOFRYR3c3WEdET016Y0dlQkhqTCtIZzRQMXJoTmVV?=
 =?utf-8?B?d3E2UWo2QTRmOElQN1p1QndJblVSMC8ySDVHbGIyMXJOeUJ5WllQa2kyV2ky?=
 =?utf-8?B?eEthckhjeHJyb0FPY0EyNEE2MngvTjFZVktRdzJTODMzQzkwamZheDRpdWZu?=
 =?utf-8?B?RXlKYUl5YkRpcnFNcE1NMkhUcTVlajJJWlBEeG1sTXRpbEZSQXZpZm9IS2xS?=
 =?utf-8?B?eUl2SVltMTdKN2tTMGpVRUpUSW51Sll2MWtUNDFad3Z4SlNaVmJCbnA3Qnhk?=
 =?utf-8?B?L3p1SHdRYVNpS29LTkNGbW9Ed01EUWNoRktuejYwS0RIbzFsZ0UxYmM5Sll6?=
 =?utf-8?B?S0czZGJOZ3J2S2JERStIWTZ6RHY1Nk4vMkVsbk1ETWhWN2xTaWZDWVh3VmFi?=
 =?utf-8?B?V1ZNWWs1OFdBV3IvVitCRVRMM1llcFEvWlhHRFJSTjJRWTRheitrMnkvM0pH?=
 =?utf-8?B?aUYrMnBSTitkSFRNMllpMGNXVklzMG9kdzcydE95elZ0OHYwd1dUOFZvbHhQ?=
 =?utf-8?B?NVc4dlBQNXVMUzhXYnJaNzFIWjBua2ZzRnQ1R0NLNTB4WHBjZXRvT2liWmhK?=
 =?utf-8?B?Qnp4bU40YjRnOXhrbHBtQWhTV3J5U1lkeFNaSW4vWklCbG85emJOTXFSWG9C?=
 =?utf-8?B?ME5GTkhSaVphb0lndXNIdExVdEgrZnl2RFVhdkVvbW5pVkV5UTUrTHpIVzBV?=
 =?utf-8?B?eDJ6SWNFNS9LNWgzTEF5a0V6WDZUUVk3TmkvcXI5eU5BcHFVYU53ejFEUnRp?=
 =?utf-8?B?MXNRNENPaGVxQWpNbytDdml6THZ1eGRiU3J5a00yK05VaVJqK0ZhaVU4YjNO?=
 =?utf-8?B?eWU0QmNKOUZ5eFVDMEE0WGdLdXNhdFV4VHVHWW90Zmc5VEtWb3pmeW42U0to?=
 =?utf-8?B?UFJudlNmenFyUENITDNuOFhNSXVweElCZVR5ZzdWTDRlczNtWlR0K0dBcFpm?=
 =?utf-8?B?K08xSjlGelFQTm9HbWdHeWIxOE5xYWdzNnJpY3lZRkxxQTJGUmwrM1FkdFg2?=
 =?utf-8?B?WVEyN0JtcVowRnhheHZDcDdQdHdSbU5HNmpwS2FCbDNLTjlQREsvMVFzNXlP?=
 =?utf-8?B?NDJUYURkd0gyZm9DOGtwMjlGM1BQWlRmc05lSVhhNXh4VEpBTXc4M2xZR3FU?=
 =?utf-8?B?WGlQZ2p6UzlHMjNzeGxRMkEzY0tsN28xNkU5YjF0b2NCeksvOHFCRzBwdGVr?=
 =?utf-8?B?blRLb3hTa280dlZOYm1Jc1Y1Z0psT09DcTRaeTYrdFJSSGNwZzQza1lRVDhF?=
 =?utf-8?B?aXBCZjBZN1k2cEFhWVVaNEh1ZlBzVmJxeGhwNVViSXFEMTNPNEtQS2x5T1Bh?=
 =?utf-8?B?RitqdUc2Nys0UFZ0UzIvMHI4ZjROYytmMTR3OGpMS3liRGpzMkdZNUFTWEx4?=
 =?utf-8?B?MlNWTVJjRWxXNmpVcHlVOU5sWCtiaFR4SDZ3Z2RNdkRtcnJpclZEUGVDdTY3?=
 =?utf-8?B?TEp5YVRsUm1RMWxYb0psVzdFSmk1eXB5cy9DZFFrSzdwb1pIK25WTEk2L24r?=
 =?utf-8?B?ejU0NmVhUUdNN2hWYkR6MUVFNk5FMlFNS3JveFJIVUFTWCtzNU1QSkc5VXFk?=
 =?utf-8?B?MENNS2hKd1VNYkVndTVmeTk0Rm9GV3l6ZW4zQjdEK0grMHlRRC9DK3ZNUFgx?=
 =?utf-8?B?dXFxSHdWTXhuWFV0R0ZhQTdoTVhpcW9MRTgxdTlvQXBoOVQ0cFFTeUI2Wklv?=
 =?utf-8?B?akt2dmc1ckkvTGJ0Mlc2TUhqbk5leTBlTU1GRERqcDB4ZFJWbXRCQ1hodUFD?=
 =?utf-8?B?eGI5QjJrczZ3bldtQzk5dXdrRXBWbS9qS0RKaWRwL05lblNpNDQ1bmRSbWJ3?=
 =?utf-8?B?bEZLN284V2xvZG8yWEVDZmNsNi9BRzBkTXJ5ZmdmV3JMUXEvaFJ4YzdoOTdB?=
 =?utf-8?B?bUE4cWxPMkxXYkQrT1NVeG5JY0Fiek1IVlQycTJTSjhPZWM1Y1l6TEg3S0tL?=
 =?utf-8?B?SlpNaU53c2pjRGVqUnFjZ1ZzUnN0ODVSc0c0dUdDUmFDeHdic0pHcGx4NkpO?=
 =?utf-8?B?UTY3SklvSXJvWHFIeFA0Ylk5L1BJVFdLTXFGZVFyamlHUFpIcFovSVgwc2gw?=
 =?utf-8?B?Rlh1UUpGZTdlcEhnMDhqczJiZ0VZUndQUDF2Z0plZ2MwUHNhVkJjUnJlSEhv?=
 =?utf-8?B?Vnc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?R2NIbTV5am9XaXJ5RUZZcFBRQXdCK2ljQWYydHdaV0U5blo4QUdzRWx1SlF6?=
 =?utf-8?B?aWVOd0hVNjEzcERJdjRoTVZmQjZYN3EzZ3ZFYkY5czBzQXh2Wk1qOUtzN0lH?=
 =?utf-8?B?TWJOczFOV1NVQU1Hc3pmUGhUYjY3NUpXUE1saUdSRjRtTXdiNzhCVU1lcGNE?=
 =?utf-8?B?a0lBQWt0SXlGZUNmcWMyQ3BuNDdoYUVQQmNHWTlPREpkanRkVFN2dmpkMTdt?=
 =?utf-8?B?N1FiN2xKVWxKZzNIMzdsd2NkMFZDZWNtcXlOSnhBT2FXdWJHM3VuQXFzNnYz?=
 =?utf-8?B?R3lrN1UzTjZNdGJnMko3YWhwQXBkTlAveTI0eGNWbE9NU3hHckw0SEkxTytm?=
 =?utf-8?B?WGUrYVlmRG9TUU9zZmtaMk1rRC9yU2RYdlljY0JZZGUxYjNpTHVjVlVwWWty?=
 =?utf-8?B?SWhNTE8vbElramEzTytUMUozWG45L2E5S0lOeW52ZHFRb2FZOEhTYjBSaGli?=
 =?utf-8?B?TFJvaTc0dG9oWDIxWVp4aTZFWG52RVpOOFBSVHNFenFPbFU3d3dRcWJDVVJp?=
 =?utf-8?B?Vys3YkhvMVBxRE9jdmlRaDRzOUZPaUdzYUtkcklzZTdQMUtQV2ppeHQ1K0hL?=
 =?utf-8?B?anEvMWZ3Z0x0RnpyOXFZS1ZPTUw0Umo2Zmg0YnlpTUFDSVppazcwRlBjN0Va?=
 =?utf-8?B?TlhxYytYYXA4SkdFRkptN3c2N2tEV1VnUHc4UVEwUWp4TGxNcWp5ZWhWMjBE?=
 =?utf-8?B?UEtVZlZueGtZbVYxb2t6N0NVUzdlV2VWNU1mZ3N2R2Z3RkFXbWNoZjBwNGdZ?=
 =?utf-8?B?eXZvK2g3MytTVUw0ZWVSUW10ZHU4YzhyZ2QrTzZCSzMzSGRNanBRZERES09i?=
 =?utf-8?B?Rm1FUk1IcnlzUjBYSjFvcWpwWE5zazFDdUY1ZDR2VFNmMC9GeFBFdlFXWk5V?=
 =?utf-8?B?NVVCN1QwRHBpMnFjZGtCNWZ6TG9lZmpwWlhZdHh6bTdyZTc0MHB6T1FUalpD?=
 =?utf-8?B?dFBjN2xNa2VvZGI4bFlGQTQ2TFVkODJmRXlnVmRLcG1uY3NwSkpHWVE4Nmgv?=
 =?utf-8?B?NjhyNndTSlFzaWJ5cWRZMy96MlBRb2drM1lPUXErSkJRTWRRMHdYZDdhT3Qw?=
 =?utf-8?B?M0VqVXZxRUZUa09INVQ4VlZFbndrT2RjNFQ5YUt2R0cra3VwYnhRZWZBR0VY?=
 =?utf-8?B?ZnRXcVIxYU5CcnY5QnJCR0Y1eEdDT2hkL2ozdURkOVhQOTl1b01ocS9jS1Uw?=
 =?utf-8?B?ZUs3M0JQM1UzdUZPbVFRa1FFdm0vUjZ3SVN6TEk2eHpjS3lRbWJNZGg5MjNj?=
 =?utf-8?B?T1NteTNlQmNDT05qbkh5Y1lkN21CdXVsZUVMajhYT1hRVnA2WGN0b21ubDlZ?=
 =?utf-8?B?em9BdVRMd3lXcHBNbm5yOVhUYlNyYVQ2ZXQxRmcwZGYzakVGOGV3MVVQNkVt?=
 =?utf-8?Q?OILR0wrSHcp+waJX5XwcxfRSdBSMKb5E=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: af943cd4-0093-461b-178f-08db4db95680
X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 May 2023 22:37:42.0062
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dJ/fTT91m/gojNKtLI6/ZRakvOVSxGnP1asLo9WGd1G4jR+7e/mhjKGMmQiezvuUWsqwRw0J1sl0sm0SKdtiAL5zhhtF5lTSijN/etllK80=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB6864
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-05-05_27,2023-05-05_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 bulkscore=0 spamscore=0
 mlxlogscore=999 mlxscore=0 adultscore=0 phishscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000
 definitions=main-2305050182
X-Proofpoint-GUID: wJP1GlFObZhCiBKFKdGarfhi3BgJCAG1
X-Proofpoint-ORIG-GUID: wJP1GlFObZhCiBKFKdGarfhi3BgJCAG1
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 5/5/23 1:22 PM, Linus Torvalds wrote:
> On Fri, May 5, 2023 at 6:40 AM Nicolas Dichtel
> <nicolas.dichtel@6wind.com> wrote:
>>
>> Is this an intended behavior?
>> This breaks some of our scripts.
> 
> It doesn't just break your scripts (which counts as a regression), I
> think it's really wrong.
> 
> The worker threads should show up as threads of the thing that started
> them, not as processes.
> 
> So they should show up in 'ps' only when one of the "show threads" flag is set.
> 
> But I suspect the fix is trivial:  the virtio code should likely use
> CLONE_THREAD for the copy_process() it does.
> 
> It should look more like "create_io_thread()" than "copy_process()", I think.
> 
> For example, do virtio worker threads really want their own signals
> and files? That sounds wrong. create_io_thread() uses all of
> 
>  CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_IO
> 
> to share much more of the context with the process it is actually run within.
> 

For the vhost tasks and the CLONE flags:

1. I didn't use CLONE_FILES in the vhost task patches because you are right
and we didn't need our own. We needed it to work like kthreads where there
are no files, so I set the kernel_clone_args.no_files bit to have copy_files
not do a dup or clone (task->files is NULL).

2. vhost tasks didn't use CLONE_SIGHAND, because userspace apps like qemu use
signals for management operations. But, the vhost thread's worker functions
assume signals are ignored like they were with kthreads. So if they were doing
IO and got a signal like a SIGHUP they might return early and fail from whatever
network/block function they were calling. And currently the parent like qemu
handles something like a SIGSTOP by shutting everything down by calling into
the vhost interface to remove the device.

So similar to files I used the kernel_clone_args.ignore_signals bit so
copy_process has the vhost thread have it's own signal handle that just ignores
signals.

3. I didn't use CLONE_THREAD because before my patches you could do
"ps -u root" and see all the vhost threads. If we use CLONE_THREAD, then we
can only see it when we do something like "ps -T -p $parent" like you mentioned
above. I guess I messed up and did the reverse and thought it would be a
regression if "ps -u root" no longer showed the vhost threads.

If it's ok to change the behavior of "ps -u root", then we can do this patch:
(Nicolas, I confirmed it fixes the 'ps a' case, but couldn't replicate the 'ps'
case. If you could test the ps only case or give me info on what /usr/bin/example
was doing I can replicate and test here):


diff --git a/kernel/fork.c b/kernel/fork.c
index ed4e01daccaa..eb9ffc58e211 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -2269,8 +2269,14 @@ __latent_entropy struct task_struct *copy_process(
 	/*
 	 * Thread groups must share signals as well, and detached threads
 	 * can only be started up within the thread group.
+	 *
+	 * A userworker's parent thread will normally have a signal handler
+	 * that performs management operations, but the worker will not
+	 * because the parent will handle the signal then user a worker
+	 * specific interface to manage the thread and related resources.
 	 */
-	if ((clone_flags & CLONE_THREAD) && !(clone_flags & CLONE_SIGHAND))
+	if ((clone_flags & CLONE_THREAD) && !(clone_flags & CLONE_SIGHAND) &&
+	    !args->user_worker && !args->ignore_signals)
 		return ERR_PTR(-EINVAL);
 
 	/*
diff --git a/kernel/vhost_task.c b/kernel/vhost_task.c
index b7cbd66f889e..3700c21ea39d 100644
--- a/kernel/vhost_task.c
+++ b/kernel/vhost_task.c
@@ -75,7 +78,8 @@ struct vhost_task *vhost_task_create(int (*fn)(void *), void *arg,
 				     const char *name)
 {
 	struct kernel_clone_args args = {
-		.flags		= CLONE_FS | CLONE_UNTRACED | CLONE_VM,
+		.flags		= CLONE_FS | CLONE_THREAD | CLONE_VM |
+				  CLONE_UNTRACED,
 		.exit_signal	= 0,
 		.fn		= vhost_task_fn,
 		.name		= name,