Received-SPF: pass (google.com: domain of linux-kernel+bounces-34652-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Precedence: bulk
MIME-Version: 1.0
References: <20240117192534.1327608-1-dylanbhatch@google.com>
 <20240119141501.GA23739@redhat.com> <20240119141529.GB23739@redhat.com>
 <CADBMgpxC+BP-wfrM-wP1nbZOcRb0LbsmMDQ3LQ8hUKYsF3QECw@mail.gmail.com>
 <20240120204552.c0708fd10fc8e2442c447049@linux-foundation.org> <20240121120754.GA2814@redhat.com>
In-Reply-To: <20240121120754.GA2814@redhat.com>
From: Dylan Hatch <dylanbhatch@google.com>
Date: Mon, 22 Jan 2024 18:53:19 -0800
Message-ID: <CADBMgpz7k=LhktfcJhSDBDWN0oLeQxPqhOVws3fq0LNpnfOSYg@mail.gmail.com>
Subject: Re: [PATCH 2/2] getrusage: use sig->stats_lock
To: Oleg Nesterov <oleg@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>, Kees Cook <keescook@chromium.org>, 
	Frederic Weisbecker <frederic@kernel.org>, "Joel Fernandes (Google)" <joel@joelfernandes.org>, 
	Ard Biesheuvel <ardb@kernel.org>, "Matthew Wilcox (Oracle)" <willy@infradead.org>, 
	Thomas Gleixner <tglx@linutronix.de>, Sebastian Andrzej Siewior <bigeasy@linutronix.de>, 
	"Eric W. Biederman" <ebiederm@xmission.com>, Vincent Whitchurch <vincent.whitchurch@axis.com>, 
	Dmitry Vyukov <dvyukov@google.com>, Luis Chamberlain <mcgrof@kernel.org>, 
	Mike Christie <michael.christie@oracle.com>, David Hildenbrand <david@redhat.com>, 
	Catalin Marinas <catalin.marinas@arm.com>, Stefan Roesch <shr@devkernel.io>, 
	Joey Gouly <joey.gouly@arm.com>, Josh Triplett <josh@joshtriplett.org>, Helge Deller <deller@gmx.de>, 
	Ondrej Mosnacek <omosnace@redhat.com>, Florent Revest <revest@chromium.org>, 
	Miguel Ojeda <ojeda@kernel.org>, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Jan 21, 2024 at 4:09=E2=80=AFAM Oleg Nesterov <oleg@redhat.com> wro=
te:
>
> Dylan, do you have a better description? Can you share your repro?

That description seems accurate to me.

> although I think that something simple like
>
>         #define NT BIG_NUMBER
>
>         pthread_barrier_t barr;
>
>         void *thread(void *arg)
>         {
>                 struct rusage ru;
>
>                 pthread_barrier_wait(&barr);
>                 for (;;)
>                         getrusage(RUSAGE_SELF, &ru);
>                 return NULL;
>         }
>
>         int main(void)
>         {
>                 pthread_barrier_init(&barr, NULL, NT);
>
>                 for (int n =3D 0; n < NT-1; ++n) {
>                         pthread_t pt;
>                         pthread_create(&pt, NULL, thread, NULL);
>                 }
>                 thread(NULL);
>
>                 return 0;
>         }
>
> should work if you have a machine with a lot of memory/cpus.
>
> Oleg.
>

Here's my repro, very similar to what you've sent:

#define _GNU_SOURCE
#include <sys/resource.h>
#include <sched.h>
#include <sys/wait.h>
#include <stdio.h>
#include <sys/mman.h>
#include <stdlib.h>
#include <unistd.h>

int thrd_func(void *data) {
       struct rusage usage;
       int *complete =3D (void *)data;

       while (!*complete);
       while (1) {
              getrusage(RUSAGE_SELF, &usage);
       }
}

#define STACK_SIZE (1024)

int main(int argc, char **argv) {
       if (argc !=3D 2) {
              printf("Usage: %s <thread count>\n", argv[0]);
              exit(EXIT_SUCCESS);
       }
       const int cnt =3D atoi(argv[1]);
       int pids[cnt];
       int complete =3D 0;
       printf("Starting test with %d threads...\n", cnt);
       for (int i =3D 0; i < cnt; i++) {
              char *stack =3D mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE=
,
                                               MAP_PRIVATE |
MAP_ANONYMOUS | MAP_STACK, -1, 0);
              if (stack =3D=3D MAP_FAILED) {
                     perror("mmap() failed\n");
                    return -1;
              }

              pids[i] =3D clone(thrd_func, stack + STACK_SIZE, CLONE_THREAD
                               | CLONE_SIGHAND | CLONE_FS | CLONE_VM |
CLONE_FILES, (void *) &complete);

              if (pids[i] =3D=3D -1) {
                     perror("clone() failed\n");
                     return pids[i];
              }
       }
       complete =3D 1;
       printf("waiting on threads...\n");
       sleep(100);
       complete =3D 0;
       printf("test finished.\n");
       exit(EXIT_SUCCESS);
}

I can't remember exactly why I chose to call mmap and clone directly instea=
d
of using pthreads... but I do know what mmap'ing in a smaller stack size
makes the repro more reliable since you can create more threads. It
seemed like around 250K threads was about enough to reliably produce
the lockup, but your mileage may vary.