Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <b9c386c1-a826-8bdc-ed4f-1d8dab6e84da@redhat.com>
Date:   Tue, 12 Apr 2022 13:03:09 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: [PATCH v8] oom_kill.c: futex: Don't OOM reap the VMA containing
 the robust_list_head
Content-Language: en-US
To:     Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>
Cc:     linux-mm@kvack.org, linux-kernel@vger.kernel.org,
        Rafael Aquini <aquini@redhat.com>,
        Waiman Long <longman@redhat.com>, Baoquan He <bhe@redhat.com>,
        Christoph von Recklinghausen <crecklin@redhat.com>,
        Don Dutile <ddutile@redhat.com>,
        "Herton R . Krzesinski" <herton@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Michal Hocko <mhocko@suse.com>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Ingo Molnar <mingo@redhat.com>,
        Joel Savitz <jsavitz@redhat.com>,
        Darren Hart <dvhart@infradead.org>, stable@kernel.org
References: <20220408032809.3696798-1-npache@redhat.com>
 <20220408081549.GM2731@worktop.programming.kicks-ass.net>
 <ee07a31c-c514-4a88-599f-14a30e93f32e@redhat.com> <87k0bzk7e5.ffs@tglx>
 <1a7944c7-d717-d5af-f71d-92326f7bb7f6@redhat.com> <87h76yff3b.ffs@tglx>
From:   Nico Pache <npache@redhat.com>
In-Reply-To: <87h76yff3b.ffs@tglx>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 4/12/22 12:20, Thomas Gleixner wrote:
> On Mon, Apr 11 2022 at 19:51, Nico Pache wrote:
>> On 4/8/22 09:54, Thomas Gleixner wrote:
>>> The below reproduces the problem nicely, i.e. the lock() in the parent
>>> times out. So why would the OOM killer fail to cause the same problem
>>> when it reaps the private anon mapping where the private futex sits?
>>>
>>> If you revert the lock order in the child the robust muck works.
>>
>> Thanks for the reproducer Thomas :)
>>
>> I think I need to re-up my knowledge around COW and how it effects
>> that stack. There are increased oddities when you add the pthread
>> library that I cant fully wrap my head around at the moment.
> 
> The pthread library functions are just conveniance so I did not have to
> hand code the futex and robust list handling.
> 
>> My confusion lies in how the parent/child share a robust list here, but they
>> obviously do. In my mind the mut_s would be different in the child/parent after
>> the fork and pthread_mutex_init (and friends) are done in the child.
> 
> They don't share a robust list, each thread has it's own.
> 
> The shared mutex mut_s is initialized in the parent before fork and it's
> the same address in the child and it's not COWed because the mapping is
> MAP_SHARED.
> 
> The child allocates private memory and initializes the private mutex in
> that private mapping.
> 
> So now child does:
> 
>    pthread_mutex_lock(mut_s);
> 
> That's the mutex in the memory shared with the parent. After that the
> childs robusts list head points to mut_s::robust_list.
> 
> Now it does:
> 
>    pthread_mutex_lock(mut_p);
> 
> after that the childs robust list head points to mut_p::robust_list and
> mut_p::robust_list points to mut_s::robust_list.
> 
> So now the child unmaps the private memory and exists.
> 
> The kernel tries to walk the robust list pointer and faults when trying
> to access mut_p. End of walk and mut_s stays locked.
> 
> So now think about the OOM case. The killed process has a shared mapping
> with some other unrelated process (file, shmem) where mut_p sits.
> 
> It gets killed after:
> 		pthread_mutex_lock(mut_s);
> 		pthread_mutex_lock(mut_p);
> 
> So the OOM reaper rips the VMA which contains mut_p and therefore breaks
> the chain which is necessary to reach mut_p.
> 
> See?
Yes, thank you for the detailed explanation, the missing piece just clicked in
my head :)

Cheers,
-- Nico
> 
> Thanks,
> 
>         tglx
> 
> 
>