On arches that do not support this_cpu_cmpxchg_double slab_lock is used
to do atomic cmpxchg() on double word which contains page->_count.
page count can be changed from get_page() or put_page() without taking
slab_lock. That corrupts page counter.
Following patch fixes it by moving page->_count out of cmpxchg_double
data. So that slub does no change it while updating slub meta-data in
struct page.
Reported-by: Amey Bhide <[email protected]>
Signed-off-by: Pravin B Shelar <[email protected]>
---
include/linux/mm_types.h | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index dad95bd..5f558dc 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -57,8 +57,16 @@ struct page {
};
union {
+#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
+ defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
/* Used for cmpxchg_double in slub */
unsigned long counters;
+#else
+ /* Keep _count separate from slub cmpxchg_double data,
+ * As rest of double word is protected by slab_lock
+ * but _count is not. */
+ unsigned counters;
+#endif
struct {
--
1.7.10
On Mon, 2012-05-14 at 15:29 -0700, Pravin B Shelar wrote:
> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
> to do atomic cmpxchg() on double word which contains page->_count.
> page count can be changed from get_page() or put_page() without taking
> slab_lock. That corrupts page counter.
>
> Following patch fixes it by moving page->_count out of cmpxchg_double
> data. So that slub does no change it while updating slub meta-data in
> struct page.
I say again : Page is owned by slub, so get_page() or put_page() is not
allowed ?
How is put_page() going to work with order-1 or order-2 allocations ?
Me very confused by these Nicira patches.
On Mon, May 14, 2012 at 3:34 PM, Eric Dumazet <[email protected]> wrote:
> On Mon, 2012-05-14 at 15:29 -0700, Pravin B Shelar wrote:
>> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
>> to do atomic cmpxchg() on double word which contains page->_count.
>> page count can be changed from get_page() or put_page() without taking
>> slab_lock. That corrupts page counter.
>>
>> Following patch fixes it by moving page->_count out of cmpxchg_double
>> data. So that slub does no change it while updating slub meta-data in
>> struct page.
>
> I say again : Page is owned by slub, so get_page() or put_page() is not
> allowed ?
>
This is already done in multiple subsystem in Linux kernel. e.g.
ocfs, xfs, etc.
So object from slab can be passed to IO using DMA. I don't think this
rule you referring to is enforced anywhere.
Thanks,
Pravin.
On Tue, 15 May 2012, Eric Dumazet wrote:
> > Following patch fixes it by moving page->_count out of cmpxchg_double
> > data. So that slub does no change it while updating slub meta-data in
> > struct page.
>
> I say again : Page is owned by slub, so get_page() or put_page() is not
> allowed ?
It is allowed since slab memory can be used for DMA.
> How is put_page() going to work with order-1 or order-2 allocations ?
It is always incrementing the page count of the head page.
Hi Christoph,
Can you comment on this patch. I have changed it according to your comments.
Thanks,
Pravin.
On Mon, May 14, 2012 at 3:29 PM, Pravin B Shelar <[email protected]> wrote:
> On arches that do not support this_cpu_cmpxchg_double slab_lock is used
> to do atomic cmpxchg() on double word which contains page->_count.
> page count can be changed from get_page() or put_page() without taking
> slab_lock. That corrupts page counter.
>
> Following patch fixes it by moving page->_count out of cmpxchg_double
> data. So that slub does no change it while updating slub meta-data in
> struct page.
>
> Reported-by: Amey Bhide <[email protected]>
> Signed-off-by: Pravin B Shelar <[email protected]>
> ---
> ?include/linux/mm_types.h | ? ?8 ++++++++
> ?1 file changed, 8 insertions(+)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index dad95bd..5f558dc 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -57,8 +57,16 @@ struct page {
> ? ? ? ? ? ? ? ?};
>
> ? ? ? ? ? ? ? ?union {
> +#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE) && \
> + ? ?defined(CONFIG_HAVE_ALIGNED_STRUCT_PAGE)
> ? ? ? ? ? ? ? ? ? ? ? ?/* Used for cmpxchg_double in slub */
> ? ? ? ? ? ? ? ? ? ? ? ?unsigned long counters;
> +#else
> + ? ? ? ? ? ? ? ? ? ? ? /* Keep _count separate from slub cmpxchg_double data,
> + ? ? ? ? ? ? ? ? ? ? ? ?* As rest of double word is protected by slab_lock
> + ? ? ? ? ? ? ? ? ? ? ? ?* but _count is not. */
> + ? ? ? ? ? ? ? ? ? ? ? unsigned counters;
> +#endif
>
> ? ? ? ? ? ? ? ? ? ? ? ?struct {
>
> --
> 1.7.10
>
On Wed, 16 May 2012, Pravin Shelar wrote:
> Can you comment on this patch. I have changed it according to your comments.
Looks fine to me.
Acked-by: Christoph Lameter <[email protected]>