2009-10-06 04:26:19

by Mike Frysinger

[permalink] [raw]
Subject: userspace firmware loader, vmap, and nommu

semi-recently (9 Apr 2009), the userspace firmware code was rewritten
to use vmap(). this causes problems for nommu systems as it isnt
possible to create a virtually contiguous map with physically
discontiguous pages. the firmware loader used to work before this
change because it would handle the realloc steps itself (allocate
larger contiguous memory, copy over older data, release older memory)
and vmalloc() on nommu is simply kmalloc().

this could be handled transparently on nommu systems by moving this
scatter gathering of pages into vmap:
void *vmap(struct page **pages, unsigned int count, unsigned long
flags, pgprot_t prot)
{
unsigned int i;
void *new_map, *page_data;

new_map = kmalloc(count << PAGE_SHIFT, GFP_KERNEL);
if (!new_map)
return NULL;

for (i = 0; i < count; ++i) {
page_data = kmap(pages[i]);
memcpy(new_map + (i << PAGE_SHIFT), page_data, PAGE_SIZE);
kunmap(page_data);
}

return new_map;
}
void vunmap(const void *addr)
{
kfree(addr);
}

or we could add nommu-specific code to the firmware loader to not use
vmap(). how would you like to go David (Howells) ?

there is a possibility for the semi-common case of vmap-ing only one
page. but i'm not familiar enough with the mm code to figure that
case out.
-mike


2009-10-06 04:39:40

by David Woodhouse

[permalink] [raw]
Subject: Re: userspace firmware loader, vmap, and nommu

On Tue, 2009-10-06 at 00:25 -0400, Mike Frysinger wrote:
> semi-recently (9 Apr 2009), the userspace firmware code was rewritten
> to use vmap(). this causes problems for nommu systems as it isnt
> possible to create a virtually contiguous map with physically
> discontiguous pages.

Oops, sorry.

> the firmware loader used to work before this
> change because it would handle the realloc steps itself (allocate
> larger contiguous memory, copy over older data, release older memory)
> and vmalloc() on nommu is simply kmalloc().
>
> this could be handled transparently on nommu systems by moving this
> scatter gathering of pages into vmap:
> void *vmap(struct page **pages, unsigned int count, unsigned long
> flags, pgprot_t prot)
> {
> unsigned int i;
> void *new_map, *page_data;
>
> new_map = kmalloc(count << PAGE_SHIFT, GFP_KERNEL);
> if (!new_map)
> return NULL;
>
> for (i = 0; i < count; ++i) {
> page_data = kmap(pages[i]);
> memcpy(new_map + (i << PAGE_SHIFT), page_data, PAGE_SIZE);
> kunmap(page_data);
> }
>
> return new_map;

I wouldn't necessarily want to do that for _all_ vmap() calls, but doing
it just for the firmware loader might make some sense. It does mean you
have to have _twice_ as much memory available as the size of the
firmware in question. And you have to have a contiguous chunk even
_after_ allocating it once piecemeal.

> void vunmap(const void *addr)
> {
> kfree(addr);
> }
>
> or we could add nommu-specific code to the firmware loader to not use
> vmap(). how would you like to go David (Howells) ?

Or we could add _generic_ code not to use vmap(). Just teach the users
that you don't get a virtually contiguous blob back from
request_firmware(); you get an array of pages instead.

--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation

2009-10-06 05:04:12

by Mike Frysinger

[permalink] [raw]
Subject: Re: userspace firmware loader, vmap, and nommu

On Tue, Oct 6, 2009 at 00:38, David Woodhouse wrote:
> On Tue, 2009-10-06 at 00:25 -0400, Mike Frysinger wrote:
>>  the firmware loader used to work before this
>> change because it would handle the realloc steps itself (allocate
>> larger contiguous memory, copy over older data, release older memory)
>> and vmalloc() on nommu is simply kmalloc().
>>
>> this could be handled transparently on nommu systems by moving this
>> scatter gathering of pages into vmap:
>> void *vmap(struct page **pages, unsigned int count, unsigned long
>> flags, pgprot_t prot)
>> {
>>     unsigned int i;
>>     void *new_map, *page_data;
>>
>>     new_map = kmalloc(count << PAGE_SHIFT, GFP_KERNEL);
>>     if (!new_map)
>>         return NULL;
>>
>>     for (i = 0; i < count; ++i) {
>>         page_data = kmap(pages[i]);
>>         memcpy(new_map + (i << PAGE_SHIFT), page_data, PAGE_SIZE);
>>         kunmap(page_data);
>>     }
>>
>>     return new_map;
>
> I wouldn't necessarily want to do that for _all_ vmap() calls,

there arent any vmap() callers currently on nommu systems since the
functions currently BUG(). looking at lxr for 2.6.31 indicates that
there are very few relevant vmap() callers in general.

> but doing
> it just for the firmware loader might make some sense. It does mean you
> have to have _twice_ as much memory available as the size of the
> firmware in question. And you have to have a contiguous chunk even
> _after_ allocating it once piecemeal.

yes, but this is how it worked before and no one complained ;).
firmware files after all tend to be on the "small" side, so getting a
small physically contiguous mapping isnt that hard.

>> void vunmap(const void *addr)
>> {
>>     kfree(addr);
>> }
>>
>> or we could add nommu-specific code to the firmware loader to not use
>> vmap().  how would you like to go David (Howells) ?
>
> Or we could add _generic_ code not to use vmap(). Just teach the users
> that you don't get a virtually contiguous blob back from
> request_firmware(); you get an array of pages instead.

wouldnt that non-trivially increase the code work for callers of the
firmware functions ? seems like a hefty penalty for a minority
(nommu) to impose on the majority (mmu).
-mike

2009-12-21 16:43:34

by Mike Frysinger

[permalink] [raw]
Subject: [PATCH] nommu: implement vmap/vunmap with kmalloc

git-svn-id: svn://localhost/svn/linux-kernel/trunk@7693 526b6c2d-f592-4532-a319-5dd88ccb003d
---
mm/nommu.c | 18 +++++++++++++++---
1 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/mm/nommu.c b/mm/nommu.c
index 8687973..d28ab94 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -360,14 +360,26 @@ EXPORT_SYMBOL(vmalloc_32_user);

void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot)
{
- BUG();
- return NULL;
+ unsigned int i;
+ void *new_map, *page_data;
+
+ new_map = kmalloc(count << PAGE_SHIFT, GFP_KERNEL);
+ if (!new_map)
+ return NULL;
+
+ for (i = 0; i < count; ++i) {
+ page_data = kmap(pages[i]);
+ memcpy(new_map + (i << PAGE_SHIFT), page_data, PAGE_SIZE);
+ kunmap(page_data);
+ }
+
+ return new_map;
}
EXPORT_SYMBOL(vmap);

void vunmap(const void *addr)
{
- BUG();
+ kfree(addr);
}
EXPORT_SYMBOL(vunmap);

--
1.6.5.4

2010-01-07 06:49:53

by Mike Frysinger

[permalink] [raw]
Subject: Re: [uClinux-dev] [PATCH] nommu: implement vmap/vunmap with kmalloc

On Mon, Dec 21, 2009 at 11:44, Mike Frysinger wrote:
> git-svn-id: svn://localhost/svn/linux-kernel/trunk@7693 526b6c2d-f592-4532-a319-5dd88ccb003d

blah, didnt realize i still had this crap. drop the changelog and add:
Signed-off-by: Mike Frysinger <[email protected]>
-mike

2010-01-16 15:57:58

by David Howells

[permalink] [raw]
Subject: Re: [PATCH] nommu: implement vmap/vunmap with kmalloc


Seems a reasonable idea.

It could be done more efficiently if vunmap() was given the page count given
to vmap().

David