Date: Thu, 26 Sep 2013 14:58:22 -0700
From: Andrew Morton <akpm@linux-foundation.org>
To: Peter Hurley <peter@hurleysoftware.com>
Cc: Greg KH <gregkh@linuxfoundation.org>,
        Fengguang Wu <fengguang.wu@intel.com>,
        LKML <linux-kernel@vger.kernel.org>, lkp@01.org,
        Tejun Heo <tj@kernel.org>
Subject: Re: increased vmap_area_lock contentions on
 "n_tty: Move buffers into n_tty_data"
Message-Id: <20130926145822.640206da116cdaac521c8bde@linux-foundation.org>
In-Reply-To: <5244AA5C.8060100@hurleysoftware.com>
References: <20130913005133.GA32479@localhost>
	<20130913010936.GA1291@localhost>
	<5238767D.1080606@hurleysoftware.com>
	<20130917232214.GA11390@localhost>
	<5238F252.5070905@hurleysoftware.com>
	<20130926003315.2e81bc84.akpm@linux-foundation.org>
	<52441B23.7050704@hurleysoftware.com>
	<20130926150421.GA25545@kroah.com>
	<52447064.5030902@hurleysoftware.com>
	<20130926110551.52ad8156.akpm@linux-foundation.org>
	<5244AA5C.8060100@hurleysoftware.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2056
Lines: 52

On Thu, 26 Sep 2013 17:42:52 -0400 Peter Hurley <peter@hurleysoftware.com> wrote:

> On 09/26/2013 02:05 PM, Andrew Morton wrote:
> > On Thu, 26 Sep 2013 13:35:32 -0400 Peter Hurley <peter@hurleysoftware.com> wrote:
> >
> >> The issue with a single large kmalloc is that it may fail where
> >> 3 separate, page-or-less kmallocs would not have.
> >
> > Or vmalloc fails first, because of internal fragmentation of the vmap
> > arena.  This problem plus vmalloc's slowness are the reasons why
> > vmalloc should be avoided.
> 
> Ok, no vmalloc.
> 
> > A tremendous number of places in the kernel perform higher-order
> > allocations nowadays.  The page allocator works damn hard to service
> > them and I expect that switching to kmalloc here will be OK.
> 
> I've had order-4 allocation failures before on 10Gb.

Yep.  But this allocation will be order=2, yes?  And
PAGE_ALLOC_COSTLY_ORDER=3.  So if that thing is working correctly,
order=2 will do a lot better than order=4.

> In fact, the
> nouveau driver switched to vmalloc for that very reason (commit
> d005f51eb93d71cd40ebd11dd377453fa8c8a42a, drm/nouveau: use vmalloc
> for pgt allocation).

Sigh.  I'm not aware of any reports of anyone hitting arena
fragmentation problems yet, so it remains a theoretical thing.  But the
more we use vmalloc, the more likely it becomes.  And because the usage
sites are so disparate, fixing it will be pretty horrid.

For this reason (plus vmalloc is slow), I do think it's better to do
the old

	foo = kmalloc(__GFP_NOWARN);
	if (!foo)
		foo = vmalloc();

thing.  It's ugly, but will greatly reduce the amount of vmallocing
which happens.

Someone had a patch a while back which wraps this operation (and the
corresponding free) into library functions.  I said yuk and it wasn't
merged.  Perhaps that was a mistake.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/