Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <03c07f48-8922-f563-560c-f0d4cc3e1279@oracle.com>
Date:   Mon, 5 Jun 2023 10:46:51 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.2
Subject: Re: [CFT][PATCH v3] fork, vhost: Use CLONE_THREAD to fix freezer/ps
 regression
Content-Language: en-US
To:     Oleg Nesterov <oleg@redhat.com>
Cc:     "Eric W. Biederman" <ebiederm@xmission.com>, linux@leemhuis.info,
        nicolas.dichtel@6wind.com, axboe@kernel.dk,
        torvalds@linux-foundation.org, linux-kernel@vger.kernel.org,
        virtualization@lists.linux-foundation.org, mst@redhat.com,
        sgarzare@redhat.com, jasowang@redhat.com, stefanha@redhat.com,
        brauner@kernel.org
References: <20230601183232.8384-1-michael.christie@oracle.com>
 <20230602192254.GD555@redhat.com>
 <87r0qt18qq.fsf_-_@email.froward.int.ebiederm.org>
 <ae250076-7d55-c407-1066-86b37014c69c@oracle.com>
 <20230605151037.GE32275@redhat.com>
From:   Mike Christie <michael.christie@oracle.com>
In-Reply-To: <20230605151037.GE32275@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b3VoaVJzSnhGYkpuelBrT25LZlhzOHliaXl6TEFDWktDUnVTdWJOUzZjUDh3?=
 =?utf-8?B?MGYzZiszVm9HSk85OUk2dEtMMU51blBBcU5tdHJWOFVtcnR0aVZncWx1aUJH?=
 =?utf-8?B?ZUFhdXR6NXIzRGhJSFNQWnRvd2RVVjBwb1pWdEVXd0t1WHhpYlNzMkZPdVg4?=
 =?utf-8?B?bGVrOHN4TDFFQ0NBL1FSNktzT0pPODZJbXB0Q1hqTWNyM3B6Z2ZINzMrbFJq?=
 =?utf-8?B?M0MzQ24yQVBzbHN6QWFrSzVPTTNDMXZNWXM5YWQyWDRLbXlrZDhVOXlxN3NW?=
 =?utf-8?B?STN2WmVSN3pjSHA4WmYwak9jRWdrVjR6V203ZUQwTkgycUlUT0s1dTJLWlhx?=
 =?utf-8?B?Q0o4NE0wbmY0ekNscWJvWWVVNGp6N3o0eHIyeFVEWnlRSFdKQjN3MUJhNXpm?=
 =?utf-8?B?SDk4YTNoQ2xSWGJybmQ1VmxxN3M5Qkw5YVZaSGxibGpQMGZITUFiWDMrZ0sx?=
 =?utf-8?B?eFlQN1hsblBNdHNnQUdFTzlDTFBoaHFTK3NJSWw0dmlMWEhmT0tUdGZHN21R?=
 =?utf-8?B?Z0RMWXM2L1lhaEdUSE9RMFVITDQ5cmpsaGgxS3ZUN21MNngyQ0FVSjN3bkJ2?=
 =?utf-8?B?VE9RTzF2VjA1RmsxOHZRU0paUGdIWERWUUw0a1NodjBEOHl0QWVoOHZDeTRt?=
 =?utf-8?B?NGJ5alprWFVOYmZpaTdrSFM5U2gyNVFTQzhqNE54b3R0T2pqdFMxa3R6aVRp?=
 =?utf-8?B?b29BbkFYK2M4bFA0dXBIZUxRM0w0bEtWVG95bTB0YmxKRElXTnV4cmM5dmM5?=
 =?utf-8?B?Zk5xbXFvcFJhNXcwUlB5Qko4WFpNbjR5cjYzZ1RZZFo2ZXBWYThIY0VvRS9t?=
 =?utf-8?B?RjhiZ2F6bEZ2V0tkTWdTTmQ1Ykd1dTlCTmI4cnh1bTVWWURMVUxqeHFXSnJU?=
 =?utf-8?B?VnQ0dW5CTmpNenc0MENCRStmVHlsM2N1OVAxMzc4QUgvcERvTkJNclJJeFJz?=
 =?utf-8?B?a1R4ZHlmdEFvR1JnTWdTbitoZC9LVEszMklqNkptRTJjd2xEcUxLYUh4S3RQ?=
 =?utf-8?B?aGNsb3ZGa3BSRDZVK05mYmhQdDhwWm42SkJqa2dwY0pQbXZ0TThaVzJZM3M0?=
 =?utf-8?B?Tjgwd2tiN29QNWhmaGFVeTgvS20zaE9xQysvZjNDWFNSZWhmdHRpZExNVlhK?=
 =?utf-8?B?N25Rcyt6QWU2dThnS2kwTzdQR253OUhPZ1NEdHo5ZVFJcnVxTDR6V2EzWEo4?=
 =?utf-8?B?Y21yVzk3VytVeTVFY1Axc01qbTUvUTIyKyszSnRoc3pVMGJUbTF3djVVS2hK?=
 =?utf-8?B?bVRralFNQXFRY2JrckY5VzVoTmhBVXh2QmFPZnVyeWJFZ0IyZkdYamFIUndn?=
 =?utf-8?B?ajgwM0ZaNFM5cnBkZzBDcGJRWVMxK3VLNTE0eHUrMmVTS3V6bkJEODlDRlJ1?=
 =?utf-8?B?OEFOQkgzcjUwdmd6M0NIdHprR3ZZM1dFbVU5L3VzOVRsUmFsS25VMFV2bGVv?=
 =?utf-8?B?bFpMUzZzRXRWajdWY2VOUGdoOXlUZlpOYzE5U3R5Y001ZkswQXlxbUU1bERi?=
 =?utf-8?B?WVRMMDNuWjdNbzJlbmt0NXRCV1hyd0gvMHdReER2SUZ3aU1MeU5oUG8wNnNq?=
 =?utf-8?B?K3VHZk9zM0tCU2hmMzFETGpTaHFlUmZCRWdpeHRZRlE3VUt3TlF3eVlqK1JP?=
 =?utf-8?B?Mk5hb2FwOXFSOUxHVGZsdTd4YzBIL0pEUWJVL3Q3MTZ6YldaVlpwNWdDWlhV?=
 =?utf-8?B?UDlEcnpMWmlnQ3RWZ09wcEhCREZKdVozR0lqOHhmOFZLRHRtaXdrblQvT0Fi?=
 =?utf-8?B?NlRGYlE2TXBCb1Q5b1hFb2o1NHVrVExyOExpaXQxaE1OTkt6NWZiMm04SVBZ?=
 =?utf-8?B?NzkySGpGeExsV0lXT01ocE5WZzE1THVxZWtpbkZSUVJZejRya0VaVHhOdSt0?=
 =?utf-8?B?a3FRY25kd0h1TzVrQjRidzZxYnRQMmVscm9aZTFWYUM0SHJCY3lsMFFlMzRv?=
 =?utf-8?B?YVc4V3VHTzN5SmpiaHBwZHV3cGx4WmNYdjVPMGNQTEhSVzl0RlZWL2VkdWJF?=
 =?utf-8?B?VHlOT3g0a295Q2lXK1IzMlV6STh6Zy92T2JMcEo5eURMb1dHOVdBOGxsci9X?=
 =?utf-8?B?WW5kRUYxSnhLWThuOS9HWVUxVjBhY2toRzJQR0FsTjdLYU91ZmYrdDNOVnhW?=
 =?utf-8?B?RVNxS2V5N1RiOG96NVY5UnQ3YkVuODUzNzFoSnZNUW4relRlaVNjcnNSVHNG?=
 =?utf-8?B?UEE9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?MzJVZ2xINHBxZktzSkRTZVVSMzN1eGU3MWRYMStyOWNBYmFSNThCQmpyUStk?=
 =?utf-8?B?ZEgyM2VYQU5zQ2xwWUtRL1FKRGR6Z3lzdTl4eForbEphTzQ3K3RSV1FMSzcx?=
 =?utf-8?B?L2pOOXg4K09RdHIwcjIxaTJ6QzVmQlZqUWZGOW51K1dNSXVMUi9JU1psaDhu?=
 =?utf-8?B?ZitMY2hnUG1NcGx5eWlhclI2NFdOZHNpbkI4RmRCQm1UYUtTZWJ4eDlHNXhO?=
 =?utf-8?B?cEhNYlM2dXZFSGFyak1JZzhOL3RxeDJacTNRWTNtV0xiUURUaWFGNUZudjhE?=
 =?utf-8?B?czdsYnQ0MzFicGR5ZXpxeXREWmNyc0JQVlR6eldTZlQ3Q0RvRm5ZQ3dpSFBo?=
 =?utf-8?B?WlJCRktNbld0MkVtRzBFenY3T3oxODNWSDFRYzF0TUlrSklBMXBrVGxkUmND?=
 =?utf-8?B?TVlFb0JZRW9BNXpHZ09zNHRyTjZrS3pLMGhGL0FkbFRjVkZjbVlyOU5FRWM0?=
 =?utf-8?B?akttUlJSaGs5QXlrMGdJK1liL002UHRiM2hKT09NQ1FKL2xOa21jMnhQVVhY?=
 =?utf-8?B?aG5MdmNObk4vN1JweDNrSWp4VXJ6ME90MmtTS3IyZ1RydWJnYXJKTnFkS290?=
 =?utf-8?B?ZWZmaVpkR3R5dHFXaytYcU1vOGN4VkgrMEExVUFlbXYxbWNnNFV3U2tyVnE2?=
 =?utf-8?B?RjRIcHdDNFozeGY2R2VLaFlkZVMyOVl4SmJkdWMrck5wM3UyVDl2dURYOE9o?=
 =?utf-8?B?TTlycVJKck92THQrazVWZS9nWmVWMHU1QzRvU01uektUMkE3K1ZPUU5lOVNF?=
 =?utf-8?B?Snl6OGpXaTRreHJ0NGdRZEkwSTlyNnVuMVBzb21GWXlDT2FzcC9wRFBwV0Vw?=
 =?utf-8?B?QkJBTUxldWpYZGhKMnVTbTlxV00rNTA4dXd1OFVJd0NnQmRHNFY2SUgzbTZ4?=
 =?utf-8?B?VnFqNnJucVBheGlpN0JzNGlkRWo1VW5Hb0E4S3E5WWRSWDV2dk5xTVpyYVBE?=
 =?utf-8?B?a25WV2Q2eHZURlAxUm1ZZkVNaHhxb21YZ2Z0azJSZVltdDJxa1JESk1QZURn?=
 =?utf-8?B?b3lNVzgvWU9YcGhpNkk5MGljMGFKbjFTSkRtUEFQL0lnb3RHU1pHMmlKUHlh?=
 =?utf-8?B?aGVmNnhaSW5MeWtCVUxtc0orN1RUNTNVRElBZnBLM0NGdzdadVF4ZldMQzRK?=
 =?utf-8?B?S3ZHV3U5VTZFYXJSUGEvWFJrWW1HNzc5YWI4ZldVUUJuUkpXemFJcEtYNEhl?=
 =?utf-8?B?VmcvelhGaXlUV1N6Q2U4OEduTFlxdUthK2RGV0dEWHJOYnlWT1RDQkVJdWI1?=
 =?utf-8?B?TkthNlZlUjhpUTdKWGQxc0liMlVqV3AvQTBMbDNwaTU3K2tiVkJLbHZxek5B?=
 =?utf-8?B?bFVJanNEdGZRVnplbWc3NWxRTHBqQ3Z2aGN0ZHZkQ1l1ZG9Ic3ZnVVY2T0JG?=
 =?utf-8?B?MlI1bENVcU9taVVvbkpsb1hEN3JLRGNyVW5wYWhkdXA1RlR4NlJlYUZlQm52?=
 =?utf-8?Q?BZ/FCsSA?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dc1aece9-8c44-45ef-cf17-08db65dc1587
X-MS-Exchange-CrossTenant-AuthSource: CY8PR10MB7243.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2023 15:46:53.2911
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /JIdeByUPXgt86IRNcz0SmIjqrNif4LymTB8MLD3U1eJYZl7hbLuvZfj1qBgwlzN+7Gxi5pP1SrtXVYeR97kj7iYQYS0g/9QVhAcr/Fuukw=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB5769
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26
 definitions=2023-06-05_31,2023-06-02_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 malwarescore=0
 spamscore=0 mlxlogscore=999 suspectscore=0 bulkscore=0 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000
 definitions=main-2306050135
X-Proofpoint-ORIG-GUID: OFSlZk7lfQm1I9NwU7mLSlruzTlbcPIZ
X-Proofpoint-GUID: OFSlZk7lfQm1I9NwU7mLSlruzTlbcPIZ
X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 6/5/23 10:10 AM, Oleg Nesterov wrote:
> On 06/03, michael.christie@oracle.com wrote:
>>
>> On 6/2/23 11:15 PM, Eric W. Biederman wrote:
>> The problem is that as part of the flush the drivers/vhost/scsi.c code
>> will wait for outstanding commands, because we can't free the device and
>> it's resources before the commands complete or we will hit the accessing
>> freed memory bug.
> 
> ignoring send-fd/clone issues, can we assume that the final fput/release
> should always come from vhost_worker's sub-thread (which shares mm/etc) ?

I think I'm misunderstanding the sub-thread term.

- Is it the task_struct's context that we did the
kernel/vhost_taskc.c:vhost_task_create() from? Below it would be the
thread we did VHOST_SET_OWNER from.

If so, then yes.

- Is it the task_struct that gets created by
kernel/vhost_taskc.c:vhost_task_create()?

If so, then the answer is no. vhost_task_create has set the no_files
arg on kernel_clone_args, so copy_files() sets task_struct->files to NULL
and we don't clone or dup the files.

So it works like if we were using a kthread still:

1. Userapce thread0 opens /dev/vhost-$something.
2. thread0 does VHOST_SET_OWNER ioctl. This calls vhost_task_create() to
create the task_struct which runs the vhost_worker() function which handles
the work->fns.
3. If userspace now does a SIGKILL or just exits without doing a close() on
/dev/vhost-$something, then when thread0 does exit_files() that will do the
fput that does vhost-$something's file_operations->release.