2006-02-21 22:29:10

by Martin Peschke

[permalink] [raw]
Subject: How to allocate per-cpu data for online CPUs only (and safely)?

I am trying to optimize the memory footprint of some code of mine.
It's been using per-cpu data and alloc_percpu() so far. The latter has
the disadvantage of getting hold of memory for CPU's which aren't there
(yet).

I could imagine using CPU-hotplug notifications as triggers for
additional allocations or for cleaning up unneeded memory. But
alloc_percpu() appears to conflict with that idea.

I was briefly tempted to derive some code from alloc_percpu() more to my
liking, until I was scared off by this comment in alloc_percpu():

/*
* Cannot use for_each_online_cpu since a cpu may come online
* and we have no way of figuring out how to fix the array
* that we have allocated then....
*/

Well, and then there is kernel/profile.c, for example, which boldly
ignors alloc_percpu()'s qualms and allocates and releases per-cpu data
as needed.

Is that the way to go?
If so, why alloc_percpu()'s reservations?
Or, does that comment imply that the exploiter isn't expected to take
care of CPU hotplug events?
Am I missing anything?

Thank you.

Martin


2006-02-22 08:32:06

by Srivatsa Vaddagiri

[permalink] [raw]
Subject: Re: How to allocate per-cpu data for online CPUs only (and safely)?

On Tue, Feb 21, 2006 at 11:29:02PM +0100, Martin Peschke wrote:
> I am trying to optimize the memory footprint of some code of mine.
> the disadvantage of getting hold of memory for CPU's which aren't there
> (yet).
>
> I could imagine using CPU-hotplug notifications as triggers for
> additional allocations or for cleaning up unneeded memory. But
> alloc_percpu() appears to conflict with that idea.
>
> I was briefly tempted to derive some code from alloc_percpu() more to my
> liking, until I was scared off by this comment in alloc_percpu():
>
> /*
> * Cannot use for_each_online_cpu since a cpu may come online
> * and we have no way of figuring out how to fix the array
> * that we have allocated then....
> */
>
> Well, and then there is kernel/profile.c, for example, which boldly
> ignors alloc_percpu()'s qualms and allocates and releases per-cpu data
> as needed.
>
> Is that the way to go?
> If so, why alloc_percpu()'s reservations?

The above comment in alloc_percpu regarding use of for_each_online_cpu
probably refers to the fact that once we handover 'pdata' to the caller
of alloc_percpu, there is no way to go back and fix it (i.e allocate
objects) for CPUs that come online later. This could have been possible if
alloc_percpu kept track of all per-CPU allocations (chain togther all pdata's
in a linked list?), but I guess they didn't do it to keep it simple.

I would say the code in kernel/profile.c is fine enough to emulate.

> Or, does that comment imply that the exploiter isn't expected to take
> care of CPU hotplug events?
> Am I missing anything?
>
> Thank you.
>
> Martin

--
Regards,
vatsa