Before commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
hierarchy"), anon_vma_clone() doesn't change dst->anon_vma. While after
this commit, anon_vma_clone() will try to reuse an exist one on forking.
But this commit go a little bit further for the case not forking.
anon_vma_clone() is called from __vma_split(), __split_vma(), copy_vma()
and anon_vma_fork(). For the first three places, the purpose here is get
a copy of src and we don't expect to touch dst->anon_vma even it is
NULL. While after that commit, it is possible to reuse an anon_vma when
dst->anon_vma is NULL. This is not we intend to have.
This patch stop reuse anon_vma for non-fork cases.
Fix commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
hierarchy")
Signed-off-by: Wei Yang <[email protected]>
---
v4:
* check dst->anon_vma in each iteration
v3:
* use dst->anon_vma and src->anon_vma to get reuse state
pointed by Konstantin Khlebnikov
---
mm/rmap.c | 24 +++++++++++++++---------
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/mm/rmap.c b/mm/rmap.c
index d9a23bb773bf..c34414567474 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)
* Attach the anon_vmas from src to dst.
* Returns 0 on success, -ENOMEM on failure.
*
- * If dst->anon_vma is NULL this function tries to find and reuse existing
- * anon_vma which has no vmas and only one child anon_vma. This prevents
- * degradation of anon_vma hierarchy to endless linear chain in case of
- * constantly forking task. On the other hand, an anon_vma with more than one
- * child isn't reused even if there was no alive vma, thus rmap walker has a
- * good chance of avoiding scanning the whole hierarchy when it searches where
- * page is mapped.
+ * anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and
+ * anon_vma_fork(). The first three want an exact copy of src, while the last
+ * one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent
+ * endless growth of anon_vma. Since dst->anon_vma is set to NULL before call,
+ * we can identify this case by checking (!dst->anon_vma && src->anon_vma).
+ *
+ * If (!dst->anon_vma && src->anon_vma) is true, this function tries to find
+ * and reuse existing anon_vma which has no vmas and only one child anon_vma.
+ * This prevents degradation of anon_vma hierarchy to endless linear chain in
+ * case of constantly forking task. On the other hand, an anon_vma with more
+ * than one child isn't reused even if there was no alive vma, thus rmap
+ * walker has a good chance of avoiding scanning the whole hierarchy when it
+ * searches where page is mapped.
*/
int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
{
@@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
* will always reuse it. Root anon_vma is never reused:
* it has self-parent reference and at least one child.
*/
- if (!dst->anon_vma && anon_vma != src->anon_vma &&
- anon_vma->degree < 2)
+ if (!dst->anon_vma && src->anon_vma &&
+ anon_vma != src->anon_vma && anon_vma->degree < 2)
dst->anon_vma = anon_vma;
}
if (dst->anon_vma)
--
2.17.1
In function __anon_vma_prepare(), we will try to find anon_vma if it is
possible to reuse it. While on fork, the logic is different.
Since commit 5beb49305251 ("mm: change anon_vma linking to fix
multi-process server scalability issue"), function anon_vma_clone()
tries to allocate new anon_vma for child process. But the logic here
will allocate a new anon_vma for each vma, even in parent this vma
is mergeable and share the same anon_vma with its sibling. This may do
better for scalability issue, while it is not necessary to do so
especially after interval tree is used.
Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
tries to reuse some anon_vma by counting child anon_vma and attached
vmas. While for those mergeable anon_vmas, we can just reuse it and not
necessary to go through the logic.
After this change, kernel build test reduces 20% anon_vma allocation.
Do the same kernel build test, it shows run time in sys reduced 11.6%.
Origin:
real 2m50.467s
user 17m52.002s
sys 1m51.953s
real 2m48.662s
user 17m55.464s
sys 1m50.553s
real 2m51.143s
user 17m59.687s
sys 1m53.600s
Patched:
real 2m39.933s
user 17m1.835s
sys 1m38.802s
real 2m39.321s
user 17m1.634s
sys 1m39.206s
real 2m39.575s
user 17m1.420s
sys 1m38.845s
Signed-off-by: Wei Yang <[email protected]>
---
mm/rmap.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/mm/rmap.c b/mm/rmap.c
index c34414567474..2c13e2bfd393 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -268,6 +268,19 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
{
struct anon_vma_chain *avc, *pavc;
struct anon_vma *root = NULL;
+ struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev;
+
+ /*
+ * If parent share anon_vma with its vm_prev, keep this sharing in in
+ * child.
+ *
+ * 1. Parent has vm_prev, which implies we have vm_prev.
+ * 2. Parent and its vm_prev have the same anon_vma.
+ */
+ if (!dst->anon_vma && src->anon_vma &&
+ pprev && pprev->anon_vma == src->anon_vma)
+ dst->anon_vma = prev->anon_vma;
+
list_for_each_entry_reverse(pavc, &src->anon_vma_chain, same_vma) {
struct anon_vma *anon_vma;
--
2.17.1
On 11/10/2019 10.22, Wei Yang wrote:
> Before commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
> hierarchy"), anon_vma_clone() doesn't change dst->anon_vma. While after
> this commit, anon_vma_clone() will try to reuse an exist one on forking.
>
> But this commit go a little bit further for the case not forking.
> anon_vma_clone() is called from __vma_split(), __split_vma(), copy_vma()
> and anon_vma_fork(). For the first three places, the purpose here is get
> a copy of src and we don't expect to touch dst->anon_vma even it is
> NULL. While after that commit, it is possible to reuse an anon_vma when
> dst->anon_vma is NULL. This is not we intend to have.
>
> This patch stop reuse anon_vma for non-fork cases.
>
> Fix commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
> hierarchy")
>
> Signed-off-by: Wei Yang <[email protected]>
Yes, reusing heuristic was designed for fork.
But this isn't strictly necessary - any vmas could share anon_vma.
For example all vmas in system could be linked with single anon_vma.
Acked-by: Konstantin Khlebnikov <[email protected]>
>
> ---
> v4:
> * check dst->anon_vma in each iteration
> v3:
> * use dst->anon_vma and src->anon_vma to get reuse state
> pointed by Konstantin Khlebnikov
> ---
> mm/rmap.c | 24 +++++++++++++++---------
> 1 file changed, 15 insertions(+), 9 deletions(-)
>
> diff --git a/mm/rmap.c b/mm/rmap.c
> index d9a23bb773bf..c34414567474 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)
> * Attach the anon_vmas from src to dst.
> * Returns 0 on success, -ENOMEM on failure.
> *
> - * If dst->anon_vma is NULL this function tries to find and reuse existing
> - * anon_vma which has no vmas and only one child anon_vma. This prevents
> - * degradation of anon_vma hierarchy to endless linear chain in case of
> - * constantly forking task. On the other hand, an anon_vma with more than one
> - * child isn't reused even if there was no alive vma, thus rmap walker has a
> - * good chance of avoiding scanning the whole hierarchy when it searches where
> - * page is mapped.
> + * anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and
> + * anon_vma_fork(). The first three want an exact copy of src, while the last
> + * one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent
> + * endless growth of anon_vma. Since dst->anon_vma is set to NULL before call,
> + * we can identify this case by checking (!dst->anon_vma && src->anon_vma).
> + *
> + * If (!dst->anon_vma && src->anon_vma) is true, this function tries to find
> + * and reuse existing anon_vma which has no vmas and only one child anon_vma.
> + * This prevents degradation of anon_vma hierarchy to endless linear chain in
> + * case of constantly forking task. On the other hand, an anon_vma with more
> + * than one child isn't reused even if there was no alive vma, thus rmap
> + * walker has a good chance of avoiding scanning the whole hierarchy when it
> + * searches where page is mapped.
> */
> int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
> {
> @@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
> * will always reuse it. Root anon_vma is never reused:
> * it has self-parent reference and at least one child.
> */
> - if (!dst->anon_vma && anon_vma != src->anon_vma &&
> - anon_vma->degree < 2)
> + if (!dst->anon_vma && src->anon_vma &&
> + anon_vma != src->anon_vma && anon_vma->degree < 2)
> dst->anon_vma = anon_vma;
> }
> if (dst->anon_vma)
>
On 11/10/2019 10.22, Wei Yang wrote:
> In function __anon_vma_prepare(), we will try to find anon_vma if it is
> possible to reuse it. While on fork, the logic is different.
>
> Since commit 5beb49305251 ("mm: change anon_vma linking to fix
> multi-process server scalability issue"), function anon_vma_clone()
> tries to allocate new anon_vma for child process. But the logic here
> will allocate a new anon_vma for each vma, even in parent this vma
> is mergeable and share the same anon_vma with its sibling. This may do
> better for scalability issue, while it is not necessary to do so
> especially after interval tree is used.
>
> Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy")
> tries to reuse some anon_vma by counting child anon_vma and attached
> vmas. While for those mergeable anon_vmas, we can just reuse it and not
> necessary to go through the logic.
>
> After this change, kernel build test reduces 20% anon_vma allocation.
>
> Do the same kernel build test, it shows run time in sys reduced 11.6%.
>
> Origin:
>
> real 2m50.467s
> user 17m52.002s
> sys 1m51.953s
>
> real 2m48.662s
> user 17m55.464s
> sys 1m50.553s
>
> real 2m51.143s
> user 17m59.687s
> sys 1m53.600s
>
> Patched:
>
> real 2m39.933s
> user 17m1.835s
> sys 1m38.802s
>
> real 2m39.321s
> user 17m1.634s
> sys 1m39.206s
>
> real 2m39.575s
> user 17m1.420s
> sys 1m38.845s
>
> Signed-off-by: Wei Yang <[email protected]>
Acked-by: Konstantin Khlebnikov <[email protected]>
> ---
> mm/rmap.c | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/mm/rmap.c b/mm/rmap.c
> index c34414567474..2c13e2bfd393 100644
> --- a/mm/rmap.c
> +++ b/mm/rmap.c
> @@ -268,6 +268,19 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
> {
> struct anon_vma_chain *avc, *pavc;
> struct anon_vma *root = NULL;
> + struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev;
> +
> + /*
> + * If parent share anon_vma with its vm_prev, keep this sharing in in
> + * child.
> + *
> + * 1. Parent has vm_prev, which implies we have vm_prev.
> + * 2. Parent and its vm_prev have the same anon_vma.
> + */
> + if (!dst->anon_vma && src->anon_vma &&
> + pprev && pprev->anon_vma == src->anon_vma)
> + dst->anon_vma = prev->anon_vma;
> +
I believe that in present code "prev" cannot be NULL if !dst->anon_vma && src->anon_vma is true.
It would be safer to check this explicitly.
>
> list_for_each_entry_reverse(pavc, &src->anon_vma_chain, same_vma) {
> struct anon_vma *anon_vma;
>
On Fri, Oct 11, 2019 at 11:04:16AM +0300, Konstantin Khlebnikov wrote:
>On 11/10/2019 10.22, Wei Yang wrote:
>> Before commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
>> hierarchy"), anon_vma_clone() doesn't change dst->anon_vma. While after
>> this commit, anon_vma_clone() will try to reuse an exist one on forking.
>>
>> But this commit go a little bit further for the case not forking.
>> anon_vma_clone() is called from __vma_split(), __split_vma(), copy_vma()
>> and anon_vma_fork(). For the first three places, the purpose here is get
>> a copy of src and we don't expect to touch dst->anon_vma even it is
>> NULL. While after that commit, it is possible to reuse an anon_vma when
>> dst->anon_vma is NULL. This is not we intend to have.
>>
>> This patch stop reuse anon_vma for non-fork cases.
>>
>> Fix commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma
>> hierarchy")
>>
>> Signed-off-by: Wei Yang <[email protected]>
>
>Yes, reusing heuristic was designed for fork.
>But this isn't strictly necessary - any vmas could share anon_vma.
>For example all vmas in system could be linked with single anon_vma.
Yes, agree with you.
Thanks for your comment :-)
>
>Acked-by: Konstantin Khlebnikov <[email protected]>
>
>>
>> ---
>> v4:
>> * check dst->anon_vma in each iteration
>> v3:
>> * use dst->anon_vma and src->anon_vma to get reuse state
>> pointed by Konstantin Khlebnikov
>> ---
>> mm/rmap.c | 24 +++++++++++++++---------
>> 1 file changed, 15 insertions(+), 9 deletions(-)
>>
>> diff --git a/mm/rmap.c b/mm/rmap.c
>> index d9a23bb773bf..c34414567474 100644
>> --- a/mm/rmap.c
>> +++ b/mm/rmap.c
>> @@ -250,13 +250,19 @@ static inline void unlock_anon_vma_root(struct anon_vma *root)
>> * Attach the anon_vmas from src to dst.
>> * Returns 0 on success, -ENOMEM on failure.
>> *
>> - * If dst->anon_vma is NULL this function tries to find and reuse existing
>> - * anon_vma which has no vmas and only one child anon_vma. This prevents
>> - * degradation of anon_vma hierarchy to endless linear chain in case of
>> - * constantly forking task. On the other hand, an anon_vma with more than one
>> - * child isn't reused even if there was no alive vma, thus rmap walker has a
>> - * good chance of avoiding scanning the whole hierarchy when it searches where
>> - * page is mapped.
>> + * anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and
>> + * anon_vma_fork(). The first three want an exact copy of src, while the last
>> + * one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent
>> + * endless growth of anon_vma. Since dst->anon_vma is set to NULL before call,
>> + * we can identify this case by checking (!dst->anon_vma && src->anon_vma).
>> + *
>> + * If (!dst->anon_vma && src->anon_vma) is true, this function tries to find
>> + * and reuse existing anon_vma which has no vmas and only one child anon_vma.
>> + * This prevents degradation of anon_vma hierarchy to endless linear chain in
>> + * case of constantly forking task. On the other hand, an anon_vma with more
>> + * than one child isn't reused even if there was no alive vma, thus rmap
>> + * walker has a good chance of avoiding scanning the whole hierarchy when it
>> + * searches where page is mapped.
>> */
>> int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
>> {
>> @@ -286,8 +292,8 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src)
>> * will always reuse it. Root anon_vma is never reused:
>> * it has self-parent reference and at least one child.
>> */
>> - if (!dst->anon_vma && anon_vma != src->anon_vma &&
>> - anon_vma->degree < 2)
>> + if (!dst->anon_vma && src->anon_vma &&
>> + anon_vma != src->anon_vma && anon_vma->degree < 2)
>> dst->anon_vma = anon_vma;
>> }
>> if (dst->anon_vma)
>>
--
Wei Yang
Help you, Help me