Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <783145b7-243b-b85e-e274-44ef6c0696b9@oracle.com>
Date:   Mon, 17 Jan 2022 10:41:05 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
From:   Mike Christie <michael.christie@oracle.com>
Subject: Re: [PATCH V6 01/10] Use copy_process in vhost layer
To:     "Eric W. Biederman" <ebiederm@xmission.com>
Cc:     geert@linux-m68k.org, vverma@digitalocean.com, hdanton@sina.com,
        hch@infradead.org, stefanha@redhat.com, jasowang@redhat.com,
        mst@redhat.com, sgarzare@redhat.com,
        virtualization@lists.linux-foundation.org,
        christian.brauner@ubuntu.com, axboe@kernel.dk,
        linux-kernel@vger.kernel.org
References: <20211129194707.5863-1-michael.christie@oracle.com>
 <87tuf79gni.fsf@email.froward.int.ebiederm.org>
 <a171238e-d731-1c22-af72-0f7faf7f4bea@oracle.com>
 <87tuf11oe6.fsf@email.froward.int.ebiederm.org>
 <a3bd6fe7-1775-6fdd-3a02-e779c4d4e498@oracle.com>
 <87pmpoxzuf.fsf@email.froward.int.ebiederm.org>
Content-Language: en-US
In-Reply-To: <87pmpoxzuf.fsf@email.froward.int.ebiederm.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UHMzUUEvbTRVbmdlcnlCc0dHWDZ4L291UDQvZjg1TkR2eGZIOWZSU25ULzQv?=
 =?utf-8?B?RWpQQWprUm9vZklRa09UMEJnUE0ySno4RVhvWkhzeVZlWFM1UCttaDRaTUhn?=
 =?utf-8?B?UzBpWUhCTFgyTFp2QVprSzJuak5RYzFrSklPeVhZN0xzWHoyOHZHVlh0QnZE?=
 =?utf-8?B?a2NkOXdiL2MwcWJDM3RyR1NrbGNORVE4bnVDWkFPQnM5R0FyVmNHZGV6YmZx?=
 =?utf-8?B?TE5zZHA2ODF2Qk5EcTRDbEZnMHlmZmNpeUZQL3h5aE96a3dNanJKanl4c0Uz?=
 =?utf-8?B?MmtEWUdYUWdOa0c5cHd5RE1uVkJ0YlFIWHNmT0VpZ3JrOWlvS0VyOC9scW5W?=
 =?utf-8?B?eGlWUlM1M2ZVWDJZQXlNZnpKMEQzOXE1Uk9FOGc5Tzc3NFdFNGRXRXhrcnV6?=
 =?utf-8?B?eEZaL0hrVVdYYms0OXZMSGFJMlpocDhhcEpTL2pHcGg4VlNTZEY4L0lVVm1q?=
 =?utf-8?B?ZW10OGtuVS9mZjdQdTh3RlFNQWw1YUFrdEIwbTIxOUFIUmJERGxqVElPT081?=
 =?utf-8?B?WHZPbllLSDdjK0tTSFN3dTh2aVFYK0JjUGlsVTJwY3hsLzY3VTBRODE1a1pB?=
 =?utf-8?B?YTBZaEI5YUczSCtsajZSa3R6a0JNNzc5YTJzY3NEeVlDL0t5bzNrditRODQ3?=
 =?utf-8?B?UUExN2l5cGc1dDFTclpkTkx3Nm9Pek5Id21WcXhpbTkrTG1mOHFyelJpWisr?=
 =?utf-8?B?N0JUOVNKV29YSVY5YlRWZEgvcVNHaHd2TCtIQmFkUmUrMFRTVENUUWhGR0kv?=
 =?utf-8?B?RkFVY3c0R1d6bkJhS2NxaThaL3NVT091d0ZDTWVKb3pWWW1UY2ZhbHlzRVBW?=
 =?utf-8?B?VFlETktTdm1jQUNodU9xSFBENlV3SDkxbnhlTVlPWGF2WUI0OXh1ME5tUk9W?=
 =?utf-8?B?QUE5dllxbzdhdG44ZFpDZzU4dUptb09YUERwYk0rdFRCTHhiTTNialg0NUNX?=
 =?utf-8?B?UUpHNkN3WGxMeXJDdWQ4MGdDSFhiWnM5cWcvVzc3YkcyTXYxRFV5elZCQ1ZK?=
 =?utf-8?B?OXo3Z2E5Y1ZRZW5CbHpyQ09yaFJiZ0ZWKytFSXA2Y0pKbUlRR29teHByR1Fq?=
 =?utf-8?B?Y0VXSllzanRaZ2M5QjJ6VmwrdWx4QS9BcllESlpsa25QRWRySHFoRElLMnpV?=
 =?utf-8?B?STdTeVFZdlVZbWFidHUyc1d2Z1ZHNXJmUjlMaWVVRSswNGsxNkVKc2c1TGl4?=
 =?utf-8?B?eXRubVRkR0xXUEljUzRGNWNrRjNmUHZnUDNiNEM5Z0hWdmdXbXJldk9INXRW?=
 =?utf-8?B?VmN2aXQvMVhLU3NDbnB6UUV6M1NydGR1cDR1clJIZkFDS0RIbkJ0TlY2c1VO?=
 =?utf-8?B?azdUQUxkL0pDeXZTeG1tNStzd2VicUF0bm1OeUZXTFUrYjNQUUVVVkkzVmJi?=
 =?utf-8?B?d3ByaCs4NWZXRWFjMjYraDE3V3lyTE4vYUIvRStEeWlNd1ozOXYwRTE5Q1N1?=
 =?utf-8?B?bkxMSmVsVWx6MHhyNEpLaUc4Y2Q1Vll3eUpEc05nUFNsblBwZXZjakIvd2Yr?=
 =?utf-8?B?OFhENy9CSlF4Qm1vVUhxZXFxS2t4dDgrejd0Sm1LeWI0K2RwQ2E3dUN1Vm1l?=
 =?utf-8?B?bXptYVhiZ1hIdjBCWEF4NXFyN3dRMllkK2NZdzc4WERRZSt3UXVsVVlyRVVY?=
 =?utf-8?B?eWY1VlF3RDNRNWkvL25pU3lYcnFjTHJSL1Q5R05JUVNTSWlscGlCR3Rra3E2?=
 =?utf-8?B?NlZoTzEzaWYyZktoV2NkdlN0ODZURlVyL0FNZHZ4Q3BpN0hMZWZjTHltSGhh?=
 =?utf-8?B?VzdNUVN2ZXBoaFcxR0FWY2Jlcndibjdwd2o3YUcwbXZQbnZWUVhKK1BJN3JE?=
 =?utf-8?B?TXFsaDloVmFveWM5SUd5aDJ4SHBoemZsa3pCbGJPMi96elR3RkRVVGROdDlT?=
 =?utf-8?B?TDNON1BRc09RaTJqRVU3d0xSTDBBZm10c3YvQldkNFhldlFTUFRZZmxkZy9x?=
 =?utf-8?B?aXR0byt5ZmRFN3FCaVduVWdFeTNiZEp5Y1FXb2RXRTgxMUVFd2ZNTGxEZm1z?=
 =?utf-8?B?RTNodEtKNzdtQVMvSnFlNjIxaFZhT0llditIeWloZFNFeW43dGl3cmE0Wm1S?=
 =?utf-8?B?R3g1N2lPenNSaVBMSHVXbFNPT1ZlaDR2dGp6aGFZRkhQaFFBZUNxVlgyVkFp?=
 =?utf-8?B?S00wZmZEY1R3WTdmYm1pbGo5dnlIaFE4dmtSc2hrRjRXMnNnUG1Ed29uWjRT?=
 =?utf-8?B?enB4TlF2NXd4YWFwb2srNE9vTU5LZ0F0Slc5SzRDck1UVktETks5eFErSWsz?=
 =?utf-8?B?Y3V0REhmUGJUVlF2a1o4b0REdmV3PT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4799c5ba-f979-47e9-e8b7-08d9d9d828e7
X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2022 16:41:07.3804
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: O0pSGQr+86YHx75uz+WhZboK+uu+hu7104hbqiSoFcHSaF+Ab0VjJcKEFAmRL+aogw4uJYSHsq6xngxIXhvo22Cm+wJLIkVqueIUq+cLuHo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5438
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10230 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 bulkscore=0
 malwarescore=0 adultscore=0 phishscore=0 mlxscore=0 spamscore=0
 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2110150000 definitions=main-2201170104
X-Proofpoint-GUID: zAhHgpVAYMxpijbRINins9G22mRdlhSP
X-Proofpoint-ORIG-GUID: zAhHgpVAYMxpijbRINins9G22mRdlhSP
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/22/21 12:24 PM, Eric W. Biederman wrote:
> All I am certain of is that you need to set
> "args->exit_signal = -1;".  This prevents having to play games with
> do_notify_parent.

Hi Eric,

I have all your review comments handled except this one. It's looking like it's
more difficult than just setting the exit_signal=-1, so I wanted to check that
I understood you.

Here is what I'm currently looking at:

1. I can't just set args->exit_signal to -1, because we end up with a task_struct
that's partially setup like a CLONE_THREAD task. What happens is copy_process
will set the task's exit_signal to -1 and then thread_group_leader() will return
false. When code like the thread_group_leader check in copy_process runs, we will
then go down the CLONE_THREAD paths which are not setup and hit crashes.

We would need changes like the following which does not crash anymore but is not
correct for many reasons. I am just posting this code as an example of the issue
I am hitting.

@@ -1637,11 +1637,13 @@ static void posix_cpu_timers_init_group(struct signal_struct *sig)
 	posix_cputimers_group_init(pct, cpu_limit);
 }
 
-static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
+static int copy_signal(unsigned long clone_flags, struct task_struct *tsk,
+		       struct kernel_clone_args *args)
 {
 	struct signal_struct *sig;
 
-	if (clone_flags & CLONE_THREAD)
+	if (clone_flags & CLONE_THREAD || args->exit_signal == -1)
 		return 0;
 
 	sig = kmem_cache_zalloc(signal_cachep, GFP_KERNEL);
@@ -2194,7 +2244,7 @@ static __latent_entropy struct task_struct *copy_process(
 	retval = copy_sighand(clone_flags, p);
 	if (retval)
 		goto bad_fork_cleanup_fs;
-	retval = copy_signal(clone_flags, p);
+	retval = copy_signal(clone_flags, p, args);
 	if (retval)
 		goto bad_fork_cleanup_sighand;
 	retval = copy_mm(clone_flags, p);
@@ -2277,6 +2327,9 @@ static __latent_entropy struct task_struct *copy_process(
 	if (clone_flags & CLONE_THREAD) {
 		p->group_leader = current->group_leader;
 		p->tgid = current->tgid;
+	} else if (args->exit_signal == -1) {
+		p->group_leader = current->group_leader;
+		p->tgid = p->pid;
 	} else {
 		p->group_leader = p;
 		p->tgid = p->pid;


2. Instead of #1, I could add some code where we just set
task_struct->exit_signal to -1. We could do this twords the end of copy_process
or after it has returned, but before we do do_exit. However, hat will have similar
issues as #1 during the exit handling.

For example, __exit_signal will call thread_group_leader which would return false.
__unhash_process would then not detach the pid and we would later hit crashes due
to the task_struct being freed already. I could add code like above to the exit related
code paths, but it gets messy like above.

3. I thought I could separate the leader detection from the exit signal by adding
a flag/field to kernel_clone_args and task_struct. But then I get to the point
where I just need a check for USER/VHOST_WORKER tasks in exit_notify which is
similar to the patch you didn't like where I added the check in do_notify_parent.
So I thought you might not like this approach.

Note:
We can't set our task's exit_signal to SIGCHLD and get autoreaped like suggested in
another mail. The original idea for the do_notify_parent was we wanted the behavior
that kthreads have where they get autoreaped on exit. kthreads get autoreaped there
because the threadd task that is the parent ignores all signals and so we hit the
parent SIG_IGN check:

        psig = tsk->parent->sighand;
        spin_lock_irqsave(&psig->siglock, flags);
        if (!tsk->ptrace && sig == SIGCHLD &&
            (psig->action[SIGCHLD-1].sa.sa_handler == SIG_IGN ||
             (psig->action[SIGCHLD-1].sa.sa_flags & SA_NOCLDWAIT))) {

Our parent, the qemu task, does not ignore SIGCHLD and so will not hit the code above.

4. Maybe I am going in the wrong direction and we need kthreads. I could add a:

if (!is_ucounts_overlimit(task_ucounts(p), UCOUNT_RLIMIT_NPROC, rlimit(RLIMIT_NPROC)))
	inc_rlimit_ucounts(task_ucounts(p), UCOUNT_RLIMIT_NPROC, 1);

to vhost.c or to kthread.c when some new arg is passed in.


What do you think?