Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 5 Oct 2022 20:06:15 -0700
From:   Kees Cook <keescook@chromium.org>
To:     Jorge Merlino <jorge.merlino@canonical.com>,
        David Howells <dhowells@redhat.com>
Cc:     Alexander Viro <viro@zeniv.linux.org.uk>,
        Eric Biederman <ebiederm@xmission.com>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH] Fix race condition when exec'ing setuid files
Message-ID: <202210051950.CAF8CDBF@keescook>
References: <20220910211215.140270-1-jorge.merlino@canonical.com>
 <202209131456.76A13BC5E4@keescook>
 <c9ca551b-070b-dcee-b4b4-b7fbfc33ab5d@canonical.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <c9ca551b-070b-dcee-b4b4-b7fbfc33ab5d@canonical.com>
Precedence: bulk

On Wed, Oct 05, 2022 at 01:09:36PM -0300, Jorge Merlino wrote:
> On 13/9/22 19:03, Kees Cook wrote:
> > I'll want to spend some more time studying this race, but yes, it looks
> > like it should get fixed. I'm curious, though, how did you find this
> > problem? It seems quite unusual to have a high-load heavily threaded
> > process decide to exec.
> 
> I just got a response from our customer regarding the situation where this
> race condition occurs:

Thanks for getting back details!

> Our application is a Rust-based CLI tool that acts as a frontend to
> cloud-based testing infrastructure. In one mode of operation it uploads a
> large number of test artifacts to cloud storage, spawns compute instances,
> and then starts a VPN connection to those instances. The application creates
> the VPN connection by executing another setuid-root tool as a subprocess. We
> see that this subprocess sometimes fails to setuid. The "high-load heavily
> threaded" aspect comes from the fact that we're using the Tokio runtime.
> Each upload to cloud storage is a separate Tokio task (i.e. "green thread")
> and these are scheduled onto "N" OS-level threads, where N = nproc. In a
> large run we may upload a couple thousand artifacts but limit to 50
> concurrent uploads. Once these artifact uploads complete, we typically spawn
> the setuid subprocess within 1-2 seconds.

Interesting. Seems like the execve might be racing all the threads
exiting?

> Have you been able to look at this issue?

I'll continue looking at this.

Dave, this tracks back to commit a6f76f23d297 ("CRED: Make execve() take
advantage of copy-on-write credentials") ... any ideas what's happening
here?

-- 
Kees Cook