Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Stephen Brennan <stephen.s.brennan@oracle.com>
To:     "Eric W. Biederman" <ebiederm@xmission.com>
Cc:     Alexey Dobriyan <adobriyan@gmail.com>,
        James Morris <jmorris@namei.org>,
        "Serge E. Hallyn" <serge@hallyn.com>,
        linux-security-module@vger.kernel.org,
        Paul Moore <paul@paul-moore.com>,
        Stephen Smalley <stephen.smalley.work@gmail.com>,
        Eric Paris <eparis@parisplace.org>, selinux@vger.kernel.org,
        Casey Schaufler <casey@schaufler-ca.com>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        Matthew Wilcox <willy@infradead.org>
Subject: Re: [PATCH] proc: Allow pid_revalidate() during LOOKUP_RCU
In-Reply-To: <87zh2yh8ti.fsf@x220.int.ebiederm.org>
References: <20201130200619.84819-1-stephen.s.brennan@oracle.com>
 <87zh2yh8ti.fsf@x220.int.ebiederm.org>
Date:   Tue, 01 Dec 2020 15:49:07 -0800
Message-ID: <87zh2xjde4.fsf@stepbren-lnx.us.oracle.com>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

ebiederm@xmission.com (Eric W. Biederman) writes:
> Stephen Brennan <stephen.s.brennan@oracle.com> writes:
>
>> The pid_revalidate() function requires dropping from RCU into REF lookup
>> mode. When many threads are resolving paths within /proc in parallel,
>> this can result in heavy spinlock contention as each thread tries to
>> grab a reference to the /proc dentry (and drop it shortly thereafter).
>>
>> Allow the pid_revalidate() function to execute under LOOKUP_RCU. When
>> updates must be made to the inode due to the owning task performing
>> setuid(), drop out of RCU and into REF mode.
>
> So rather than get_task_rcu_user.  I think what we want is a function
> that verifies task->rcu_users > 0.
>
> Which frankly is just "pid_task(proc_pid(inode), PIDTYPE_PID)".
>
> Which is something that we can do unconditionally in pid_revalidate.
>
> Skipping the update of the inode is probably the only thing that needs
> to be skipped.
>
> It looks like the code can safely rely on the the security_task_to_inode
> in proc_pid_make_inode and remove the security_task_to_inode in
> pid_update_inode.
>

This makes sense, I'll get rid of the get_task_rcu_user() stuff in a v2.

>
>> Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
>> ---
>>
>> I'd like to use this patch as an RFC on this approach for reducing spinlock
>> contention during many parallel path lookups in the /proc filesystem. The
>> contention can be triggered by, for example, running ~100 parallel instances of
>> "TZ=/etc/localtime ps -fe >/dev/null" on a 100CPU machine. The %sys utilization
>> in such a case reaches around 90%, and profiles show two code paths with high
>> utilization:
>
> Do you have a real world work-load that is behaves something like this
> micro benchmark?  I am just curious how severe the problem you are
> trying to solve is.
>

We have seen this issue occur internally with monitoring scripts
(perhaps a bit misconfigured, I'll admit). However I don't have an exact
sample workload that I can give you.

Thanks,
Stephen