From: ebiederm@xmission.com (Eric W. Biederman)
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andi Kleen <ak@linux.intel.com>, Tim Chen <tim.c.chen@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@elte.hu>,
        Kan Liang <kan.liang@intel.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Johannes Weiner <hannes@cmpxchg.org>, Jan Kara <jack@suse.cz>,
        linux-mm <linux-mm@kvack.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
References: <84c7f26182b7f4723c0fe3b34ba912a9de92b8b7.1502758114.git.tim.c.chen@linux.intel.com>
        <CA+55aFznC1wqBSfYr8=92LGqz5-F6fHMzdXoqM4aOYx8sT1Dhg@mail.gmail.com>
        <20170815022743.GB28715@tassilo.jf.intel.com>
        <CA+55aFyHVV=eTtAocUrNLymQOCj55qkF58+N+Tjr2YS9TrqFow@mail.gmail.com>
        <20170815031524.GC28715@tassilo.jf.intel.com>
        <CA+55aFw1A1C8qUeKPUzACrsqn97UDxTP3M2SRs80aEztfU=Qbg@mail.gmail.com>
        <20170815224728.GA1373@linux-80c1.suse>
        <CA+55aFyMkd8EaozxvAZo9i3ArKh7m6HLjsUB34xnDBzXz4gowg@mail.gmail.com>
        <CA+55aFw84Cu0VZdR_Rj6b03hMYBFgt9BCnSEx+OLXDsp4dDO=g@mail.gmail.com>
        <CA+55aFxbo-4M8=3BZ_VbqvRTq6_Lbw6eUQz2tTh7ve5YhLdecw@mail.gmail.com>
Date: Wed, 16 Aug 2017 18:22:57 -0500
In-Reply-To: <CA+55aFxbo-4M8=3BZ_VbqvRTq6_Lbw6eUQz2tTh7ve5YhLdecw@mail.gmail.com>
        (Linus Torvalds's message of "Tue, 15 Aug 2017 16:50:34 -0700")
Message-ID: <87inhnrtbi.fsf@xmission.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: [PATCH 1/2] sched/wait: Break up long wake list walk
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1977
Lines: 48

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Tue, Aug 15, 2017 at 3:57 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>>
>> Oh, and the page wait-queue really needs that key argument too, which
>> is another thing that swait queue code got rid of in the name of
>> simplicity.
>
> Actually, it gets worse.
>
> Because the page wait queues are hashed, it's not an all-or-nothing
> thing even for the non-exclusive cases, and it's not a "wake up first
> entry" for the exclusive case. Both have to be conditional on the wait
> entry actually matching the page and bit in question.
>
> So no way to use swait, or any of the lockless queuing code in general
> (so we can't do some clever private wait-list using llist.h either).
>
> End result: it looks like you fairly fundamentally do need to use a
> lock over the whole list traversal (like the standard wait-queues),
> and then add a cursor entry like Tim's patch if dropping the lock in
> the middle.
>
> Anyway, looking at the old code, we *used* to limit the page wait hash
> table to 4k entries, and we used to have one hash table per memory
> zone.
>
> The per-zone thing didn't work at all for the generic bit-waitqueues,
> because of how people used them on virtual addresses on the stack.
>
> But it *could* work for the page waitqueues, which are now a totally
> separate entity, and is obviously always physically addressed (since
> the indexing is by "struct page" pointer), and doesn't have that
> issue.
>
> So I guess we could re-introduce the notion of per-zone page waitqueue
> hash tables. It was disgusting to allocate and free though (and hooked
> into the memory hotplug code).
>
> So I'd still hope that we can instead just have one larger hash table,
> and that is sufficient for the problem.

If increasing the hash table size fixes the problem I am wondering if
rhash tables might be the proper solution to this problem.  They start
out small and then grow as needed.

Eric