Allocations of larger pages are not reliable in Linux. If larger
pages have to be allocated then one faces various choices of allowing
graceful fallback or using vmalloc with a performance penalty due
to the use of a page table. Virtual Compound pages are
a simple solution out of this dilemma.
A virtual compound allocation means that there will be first of all
an attempt to satisfy the request with physically contiguous memory.
If that is not possible then a virtually contiguous memory will be
created.
This has two advantages:
1. Current uses of vmalloc can be converted to allocate virtual
compounds instead. In most cases physically contiguous
memory can be used which avoids the vmalloc performance
penalty. See f.e. the e1000 driver patch.
2. Uses of higher order allocations (stacks, buffers etc) can be
converted to use virtual compounds instead. Physically contiguous
memory will still be used for those higher order allocs in general
but the system can degrade to the use of vmalloc should memory
become heavily fragmented.
There is a compile time option to switch on fallback for
testing purposes. Virtually mapped mmemory may behave differently
and the CONFIG_FALLBACK_ALWAYS option will ensure that the code is
tested to deal with virtual memory.
V2->V3:
- Put the code into mm/vmalloc.c and leave the page allocator alone.
- Add a series of examples where virtual compound pages can be used.
- Diffed on top of the page flags and the vmalloc info patches
already in mm.
- Simplify things by omitting some of the more complex code
that used to be in there.
V1->V2
- Remove some cleanup patches and the SLUB patches from this set.
- Transparent vcompound support through page_address() and
virt_to_head_page().
- Additional use cases.
- Factor the code better for an easier read
- Add configurable stack size.
- Follow up on various suggestions made for V1
RFC->V1
- Complete support for all compound functions for virtual compound pages
(including the compound_nth_page() necessary for LBS mmap support)
- Fix various bugs
- Fix i386 build
--
On Thu, 20 Mar 2008 23:17:03 -0700
Christoph Lameter <[email protected]> wrote:
> Allocations of larger pages are not reliable in Linux. If larger
> pages have to be allocated then one faces various choices of allowing
> graceful fallback or using vmalloc with a performance penalty due
> to the use of a page table. Virtual Compound pages are
> a simple solution out of this dilemma.
can you document the drawback of large, frequent vmalloc() allocations at least?
On 32 bit x86, the effective vmalloc space is 64Mb or so (after various PCI bars are ioremaped),
so if this type of allocation is used for a "scales with nr of ABC" where "ABC" is workload dependent,
there's a rather abrupt upper limit to this.
Not saying that that is a flaw of your patch, just pointing out that we should discourage usage of
the "scales with nr of ABC" (for example "one for each thread") kind of things.
On Sat, 22 Mar 2008, Arjan van de Ven wrote:
> can you document the drawback of large, frequent vmalloc() allocations at least?
Ok. Lets add some documentation about this issue and some other
things. A similar suggestion was made by Kosaki-san.
> On 32 bit x86, the effective vmalloc space is 64Mb or so (after various PCI bars are ioremaped),
> so if this type of allocation is used for a "scales with nr of ABC" where "ABC" is workload dependent,
> there's a rather abrupt upper limit to this.
> Not saying that that is a flaw of your patch, just pointing out that we should discourage usage of
> the "scales with nr of ABC" (for example "one for each thread") kind of things.
I better take out any patches that do large scale allocs then.
On the other hand: The conversion of vmalloc to vcompound_alloc will
reduce the amount of virtually mapped space needed. Doing that to
alloc_large_system_hash() etc may help there.