When the owner of a mapping fails COW because a child process is holding a
reference and no pages are available, the children VMAs are walked and the
page is unmapped. The i_mmap_lock is taken for the unmapping of the page but
not the walking of the prio_tree. In theory, that tree could be changing
while the lock is released although in practice it is protected by the
hugetlb_instantiation_mutex. This patch takes the i_mmap_lock properly for
the duration of the prio_tree walk in case the hugetlb_instantiation_mutex
ever goes away.
[[email protected]: Spotted the problem in the first place]
Signed-off-by: Mel Gorman <[email protected]>
---
mm/hugetlb.c | 9 ++++++++-
1 files changed, 8 insertions(+), 1 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index a952cb8..5adc284 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1906,6 +1906,12 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
+ (vma->vm_pgoff >> PAGE_SHIFT);
mapping = (struct address_space *)page_private(page);
+ /*
+ * Take the mapping lock for the duration of the table walk. As
+ * this mapping should be shared between all the VMAs,
+ * __unmap_hugepage_range() is called as the lock is already held
+ */
+ spin_lock(&mapping->i_mmap_lock);
vma_prio_tree_foreach(iter_vma, &iter, &mapping->i_mmap, pgoff, pgoff) {
/* Do not unmap the current VMA */
if (iter_vma == vma)
@@ -1919,10 +1925,11 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
* from the time of fork. This would look like data corruption
*/
if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
- unmap_hugepage_range(iter_vma,
+ __unmap_hugepage_range(iter_vma,
address, address + huge_page_size(h),
page);
}
+ spin_unlock(&mapping->i_mmap_lock);
return 1;
}
On Wed, 2 Dec 2009, Mel Gorman wrote:
> When the owner of a mapping fails COW because a child process is holding a
> reference and no pages are available, the children VMAs are walked and the
> page is unmapped. The i_mmap_lock is taken for the unmapping of the page but
> not the walking of the prio_tree. In theory, that tree could be changing
> while the lock is released although in practice it is protected by the
> hugetlb_instantiation_mutex. This patch takes the i_mmap_lock properly for
> the duration of the prio_tree walk in case the hugetlb_instantiation_mutex
> ever goes away.
>
> [[email protected]: Spotted the problem in the first place]
> Signed-off-by: Mel Gorman <[email protected]>
The patch looks good - thanks for taking care of that, Mel.
But the comment seems wrong to me: hugetlb_instantiation_mutex
guards against concurrent hugetlb_fault()s; but the structure of
the prio_tree shifts as vmas based on that inode are inserted into
(mmap'ed) and removed from (munmap'ed) that tree (always while
holding i_mmap_lock). I don't see hugetlb_instantiation_mutex
giving us any protection against this at present.
Hugh
> ---
> mm/hugetlb.c | 9 ++++++++-
> 1 files changed, 8 insertions(+), 1 deletions(-)
>
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index a952cb8..5adc284 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -1906,6 +1906,12 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
> + (vma->vm_pgoff >> PAGE_SHIFT);
> mapping = (struct address_space *)page_private(page);
>
> + /*
> + * Take the mapping lock for the duration of the table walk. As
> + * this mapping should be shared between all the VMAs,
> + * __unmap_hugepage_range() is called as the lock is already held
> + */
> + spin_lock(&mapping->i_mmap_lock);
> vma_prio_tree_foreach(iter_vma, &iter, &mapping->i_mmap, pgoff, pgoff) {
> /* Do not unmap the current VMA */
> if (iter_vma == vma)
> @@ -1919,10 +1925,11 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
> * from the time of fork. This would look like data corruption
> */
> if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
> - unmap_hugepage_range(iter_vma,
> + __unmap_hugepage_range(iter_vma,
> address, address + huge_page_size(h),
> page);
> }
> + spin_unlock(&mapping->i_mmap_lock);
>
> return 1;
> }
On Wed, Dec 02, 2009 at 08:13:39PM +0000, Hugh Dickins wrote:
> On Wed, 2 Dec 2009, Mel Gorman wrote:
>
> > When the owner of a mapping fails COW because a child process is holding a
> > reference and no pages are available, the children VMAs are walked and the
> > page is unmapped. The i_mmap_lock is taken for the unmapping of the page but
> > not the walking of the prio_tree. In theory, that tree could be changing
> > while the lock is released although in practice it is protected by the
> > hugetlb_instantiation_mutex. This patch takes the i_mmap_lock properly for
> > the duration of the prio_tree walk in case the hugetlb_instantiation_mutex
> > ever goes away.
> >
> > [[email protected]: Spotted the problem in the first place]
> > Signed-off-by: Mel Gorman <[email protected]>
>
> The patch looks good - thanks for taking care of that, Mel.
>
> But the comment seems wrong to me: hugetlb_instantiation_mutex
> guards against concurrent hugetlb_fault()s; but the structure of
> the prio_tree shifts as vmas based on that inode are inserted into
> (mmap'ed) and removed from (munmap'ed) that tree (always while
> holding i_mmap_lock). I don't see hugetlb_instantiation_mutex
> giving us any protection against this at present.
>
You're right of course. I'll report without that nonsense included.
Thanks
>
> > ---
> > mm/hugetlb.c | 9 ++++++++-
> > 1 files changed, 8 insertions(+), 1 deletions(-)
> >
> > diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> > index a952cb8..5adc284 100644
> > --- a/mm/hugetlb.c
> > +++ b/mm/hugetlb.c
> > @@ -1906,6 +1906,12 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
> > + (vma->vm_pgoff >> PAGE_SHIFT);
> > mapping = (struct address_space *)page_private(page);
> >
> > + /*
> > + * Take the mapping lock for the duration of the table walk. As
> > + * this mapping should be shared between all the VMAs,
> > + * __unmap_hugepage_range() is called as the lock is already held
> > + */
> > + spin_lock(&mapping->i_mmap_lock);
> > vma_prio_tree_foreach(iter_vma, &iter, &mapping->i_mmap, pgoff, pgoff) {
> > /* Do not unmap the current VMA */
> > if (iter_vma == vma)
> > @@ -1919,10 +1925,11 @@ static int unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,
> > * from the time of fork. This would look like data corruption
> > */
> > if (!is_vma_resv_set(iter_vma, HPAGE_RESV_OWNER))
> > - unmap_hugepage_range(iter_vma,
> > + __unmap_hugepage_range(iter_vma,
> > address, address + huge_page_size(h),
> > page);
> > }
> > + spin_unlock(&mapping->i_mmap_lock);
> >
> > return 1;
> > }
>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
On Wed, Dec 02, 2009 at 10:16:02PM +0000, Mel Gorman wrote:
> On Wed, Dec 02, 2009 at 08:13:39PM +0000, Hugh Dickins wrote:
> > On Wed, 2 Dec 2009, Mel Gorman wrote:
> >
> > > When the owner of a mapping fails COW because a child process is holding a
> > > reference and no pages are available, the children VMAs are walked and the
> > > page is unmapped. The i_mmap_lock is taken for the unmapping of the page but
> > > not the walking of the prio_tree. In theory, that tree could be changing
> > > while the lock is released although in practice it is protected by the
> > > hugetlb_instantiation_mutex. This patch takes the i_mmap_lock properly for
> > > the duration of the prio_tree walk in case the hugetlb_instantiation_mutex
> > > ever goes away.
> > >
> > > [[email protected]: Spotted the problem in the first place]
> > > Signed-off-by: Mel Gorman <[email protected]>
> >
> > The patch looks good - thanks for taking care of that, Mel.
> >
> > But the comment seems wrong to me: hugetlb_instantiation_mutex
> > guards against concurrent hugetlb_fault()s; but the structure of
> > the prio_tree shifts as vmas based on that inode are inserted into
> > (mmap'ed) and removed from (munmap'ed) that tree (always while
> > holding i_mmap_lock). I don't see hugetlb_instantiation_mutex
> > giving us any protection against this at present.
> >
>
> You're right of course. I'll report without that nonsense included.
>
Actually, shouldn't the mmap_sem be protecting against concurrent mmap and
munmap altering the tree? The comment is still bogus of course.
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
On Wed, 2 Dec 2009, Mel Gorman wrote:
> On Wed, Dec 02, 2009 at 10:16:02PM +0000, Mel Gorman wrote:
> > On Wed, Dec 02, 2009 at 08:13:39PM +0000, Hugh Dickins wrote:
> > >
> > > But the comment seems wrong to me: hugetlb_instantiation_mutex
> > > guards against concurrent hugetlb_fault()s; but the structure of
> > > the prio_tree shifts as vmas based on that inode are inserted into
> > > (mmap'ed) and removed from (munmap'ed) that tree (always while
> > > holding i_mmap_lock). I don't see hugetlb_instantiation_mutex
> > > giving us any protection against this at present.
> > >
> >
> > You're right of course. I'll report without that nonsense included.
> >
>
> Actually, shouldn't the mmap_sem be protecting against concurrent mmap and
> munmap altering the tree? The comment is still bogus of course.
No, the mmap_sem can only protect against other threads sharing that
same mm: whereas the prio_tree can shift around according to concurrent
mmaps and munmaps of the same file in other mms.
Hugh