Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b6e07251-2c8c-fe73-9950-2b2a17748555@intel.com>
Date:   Sun, 4 Jun 2023 20:10:35 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.11.0
Subject: Re: [PATCH 00/14] Reduce preallocations for maple tree
Content-Language: en-US
To:     "Liam R. Howlett" <Liam.Howlett@Oracle.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        <maple-tree@lists.infradead.org>, <linux-mm@kvack.org>,
        <linux-kernel@vger.kernel.org>, "Liu, Yujie" <yujie.liu@intel.com>
References: <20230601021605.2823123-1-Liam.Howlett@oracle.com>
 <7a5dc9ce-b58f-e1b3-db1a-d00a8a556ae5@intel.com>
 <20230602185555.irsxyt7svz7h6igm@revolver>
From:   "Yin, Fengwei" <fengwei.yin@intel.com>
In-Reply-To: <20230602185555.irsxyt7svz7h6igm@revolver>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZEJMdzVjZVlmQzdDNVVlZWlPRlJMRitTcFRzUFRISms1bmFqMUtZZHVDamVN?=
 =?utf-8?B?cnRjbEVxWVNUVlNjdjNLcnoxMERKZEs5VXlxM1A5SldGMk0ycnVMYTg3UEhB?=
 =?utf-8?B?TDNUdXVvRmdzTGVLRlJ6WHNhS0R5NHlDVTdwMDV4cEg1VlVuT2VCeXdVRk0y?=
 =?utf-8?B?eE9oV1pYelRCVjhPYmdrcXpjVlFlcjVZUUNjd0VBQU5WSmlQdmxhVFArcWZU?=
 =?utf-8?B?UEFQbHAxTHpmUGVZai9jaEVIdTJ3dnNMc09EMUZYTnRNZldxc1hlOWgzNWhW?=
 =?utf-8?B?MlhwalpUYUZRRnJrRmtzVzB1VDZKRTlHc2JkdVhvbERCVDNiUkd0WmN2RGlm?=
 =?utf-8?B?VlB6Tld2dTM5QjhOeWRaUWxUajB0T2RmRU9Ia0orcTV3OEF5NnFWZFo4SjFG?=
 =?utf-8?B?ajFXM0xjZXh3WG5aWU10ek1iVWZVaTVKdlNMVUpWNE0yL0d0QmNvSTJ3bEZk?=
 =?utf-8?B?MGNEMS9sTCs3bEQ2V2lmelAwRTRKZHlKUnpJQlNJQi9IN1VqUldKYWFXL3FQ?=
 =?utf-8?B?eDdKdVA3Q0c0MXZoR0ZuLzllVzVIVFVabFZsdUFSOGp3TDE4SGtWVFdNMVZB?=
 =?utf-8?B?bUNqNnhPa0pBS1A1L2QxVmY2NXQ5dWtOMDErVVZJbzJITGxqTFNkdTQ0emlj?=
 =?utf-8?B?a3ZGcHBORXd0Q0tnLy84cXVIV3ZmT2lsb0dZUUxTYUFYRWdqVk1xS2xoeHFw?=
 =?utf-8?B?bFBzYTlodGx3dFlBRWV0bmwxUHdCb0hoZEZoaU1ZR3htUERoalBxU0FrY3JV?=
 =?utf-8?B?S3RVQnhkdnVzNkphVVR6Qmc1ekNvdHQ1bGdxdGprbU1YRC8veDVXdzBMclJp?=
 =?utf-8?B?TW9OL2RsKzVpcytCbjZVTmk5TkRyazA4N1R4KzFFdk1KaW5KVDZ1eEl3dDBR?=
 =?utf-8?B?U281Vy93bFJPYWJLdDNGbFowbXl6TGQvTVRPQVI3SUV1Z0JIWVdqU2N0VFVG?=
 =?utf-8?B?OHhWUFJoK1U0bkZGWW54ak5NVDM0eHV2dnk2MW9pYU0vbzZaMFBqc29hVWxl?=
 =?utf-8?B?SHEvZU5TY2wxejdlRXovQnFBS1plZUVvV3ViN0p2a3BnOU42Mkh1OWY4blNE?=
 =?utf-8?B?c0g3VVJpOVpnSWRlSEU4RmNMSWR6WmxBV3d5UnY2TW50MjEyWC96cVh0L3Nw?=
 =?utf-8?B?clZNT0dodUdHSE45SzFSektYWGdEYjBtWjNseGV0ZnRZQmNYVnZMU0R4d3Rk?=
 =?utf-8?B?bVEvREN5V0plcVhFS25aYXdJSzlxQ2xaNVQydTBTY3dpcTQ0bERxaWFlaFNv?=
 =?utf-8?B?RDh6M3g5d0pLeVE1NDROMXgvN0dPVjUzTkZ5UiszYzJVMUpNT2F5UGQ4dlh3?=
 =?utf-8?B?UzhjMXlvQ0R6YzFDVlNLY2hkZUdMSkorZ0Y0V0FqRWM3TmpSYVZTMmYxTGRW?=
 =?utf-8?B?a1gzV0dhajZEN2JGK0FlSmJiRVQ2WGlBdGlwYldTYmM1R0I5RWNoZU1rRlpn?=
 =?utf-8?B?WUtob3FoVnlBTmh4bmI3WjVZVURTMkYwMFdEOWU5dENZeEhpRXJpUmliY0Zk?=
 =?utf-8?B?Y1owSHVlMlNKRjJ1VkU3bDFaTTIxVldkbEV5ejhMQ05lcGpJbS9PTmxHZU1v?=
 =?utf-8?B?V3dadFdaVEVIZW5wKzJtaVllYnZ6dFRrYmRSbW1idnZ2QVcrUGs0TmZEQ1Y1?=
 =?utf-8?B?eXVDSkVMbHBBL1psUkZVUUIxa1ZaYzVHQlBiVWxsMXBzbnd4Sk9NMDM1V2dH?=
 =?utf-8?B?YXloRmg1eGNCYzJEd0JYVERFcExqQ3JGYlEwc0s0ZDZZdk9BQlprWTg1QzFs?=
 =?utf-8?B?dWlsMlBZSmhVbVM0c0VrWHJFQ0p5MG5lWkFqWWU2Zk9PWFZEeUtlb3NzVkF4?=
 =?utf-8?B?Y21aNGc1UmpNeHZrZ1R4UDFwQnRkSm42dG0rdEJrMVhmc2cvVHVnbmJXY08x?=
 =?utf-8?B?cmJ5Uzl5dW5VcWdGS21kd3VOQmdld0wyejJVZXVzR0NrNk5IdFIrbXpYYklX?=
 =?utf-8?B?OU4wS2M3N1g1VXVBdURKSEllTTA3L3BuZDAxc3B5Z0RGRzBsVmxKa3JpRUFs?=
 =?utf-8?B?a0FqcHVNUUJjcE5SelZ3NWRpY0FqWS95K09SQ084cFFDdHhBRU8xZmhuZUdv?=
 =?utf-8?B?UC9tUHZhbVBvZUNtVHQwcXhQU1F6blNka3MwZ3BIMFZkVmpiTjZmaFk2MXBU?=
 =?utf-8?Q?qN67L8PvmYddSzU52NdekmHlY?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3bfcd0ab-d8a9-4fb7-3bb7-08db64f4b87c
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Jun 2023 12:10:43.6384
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eXlKM36XO5am5WV0mbKaHsresyDnqwJ5MgwTWCyWSeflJdbHt9C4iEf6FFC7kms8r2+KrYjG6INoMCGiS/1HmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7214
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Liam,

On 6/3/2023 2:55 AM, Liam R. Howlett wrote:
> * Yin, Fengwei <fengwei.yin@intel.com> [230602 04:11]:
>> Hi Liam,
>>
>> On 6/1/2023 10:15 AM, Liam R. Howlett wrote:
>>> Initial work on preallocations showed no regression in performance
>>> during testing, but recently some users (both on [1] and off [android]
>>> list) have reported that preallocating the worst-case number of nodes
>>> has caused some slow down.  This patch set addresses the number of
>>> allocations in a few ways.
>>>
>>> During munmap() most munmap() operations will remove a single VMA, so
>>> leverage the fact that the maple tree can place a single pointer at
>>> range 0 - 0 without allocating.  This is done by changing the index in
>>> the 'sidetree'.
>>>
>>> Re-introduce the entry argument to mas_preallocate() so that a more
>>> intelligent guess of the node count can be made.
>>>
>>> Patches are in the following order:
>>> 0001-0002: Testing framework for benchmarking some operations
>>> 0003-0004: Reduction of maple node allocation in sidetree
>>> 0005:      Small cleanup of do_vmi_align_munmap()
>>> 0006-0013: mas_preallocate() calculation change
>>> 0014:      Change the vma iterator order
>> I did run The AIM:page_test on an IceLake 48C/96T + 192G RAM platform with
>> this patchset.
>>
>> The result has a little bit improvement:
>> Base (next-20230602):
>>   503880
>> Base with this patchset:
>>   519501
>>
>> But they are far from the none-regression result (commit 7be1c1a3c7b1):
>>   718080
>>
>>
>> Some other information I collected:
>> With Base, the mas_alloc_nodes are always hit with request: 7.
>> With this patchset, the request are 1 or 5.
>>
>> I suppose this is the reason for improvement from 503880 to 519501.
>>
>> With commit 7be1c1a3c7b1, mas_store_gfp() in do_brk_flags never triggered
>> mas_alloc_nodes() call. Thanks.
> 
> Thanks for retesting.  I've not been able to see the regression myself.
> Are you running in a VM of sorts?  Android and some cloud VMs seem to
I didn't run it in VM. I run it on a native env.

> see this, but I do not in kvm or the server I test on.
> 
> I am still looking to reduce/reverse the regression and a reproducer on
> my end would help.

The test is page_test of AIM9. You could get AIM9 test suite from:
http://nchc.dl.sourceforge.net/project/aimbench/aim-suite9

After build it, we could see app singleuser.

It needs a txt file named s9workfile to define the test case. The s9workfile
I am using has following content:

# @(#) s9workfile:1.2 1/22/96 00:00:00
# AIM Independent Resource Benchmark - Suite IX Workfile
FILESIZE: 5M
page_test

Then you can run the testing by command:
  ./singleuser -nl

It will ask some configuration questions and then run the real test.

One thing need be taken care is that the create-clo.c has one line:
   newbrk = sbrk(-4096 * 16);

It should be updated as:
   intptr_t inc = -4096 * 16;
   newbrk = sbrk(inc);

Otherwise, the -4096 * 16 will be treated as 32 bit and the line is
changed to extend brk to around 4G. If we don't have enough RAM, the
set_brk syscall will fail.

If you met any issue to run the test, just ping me. Thanks.


Regards
Yin, Fengwei

> 
>>
>>
>> Regards
>> Yin, Fengwei
>>
>>>
>>> [1] https://lore.kernel.org/linux-mm/202305061457.ac15990c-yujie.liu@intel.com/
>>>
>>> Liam R. Howlett (14):
>>>   maple_tree: Add benchmarking for mas_for_each
>>>   maple_tree: Add benchmarking for mas_prev()
>>>   mm: Move unmap_vmas() declaration to internal header
>>>   mm: Change do_vmi_align_munmap() side tree index
>>>   mm: Remove prev check from do_vmi_align_munmap()
>>>   maple_tree: Introduce __mas_set_range()
>>>   mm: Remove re-walk from mmap_region()
>>>   maple_tree: Re-introduce entry to mas_preallocate() arguments
>>>   mm: Use vma_iter_clear_gfp() in nommu
>>>   mm: Set up vma iterator for vma_iter_prealloc() calls
>>>   maple_tree: Move mas_wr_end_piv() below mas_wr_extend_null()
>>>   maple_tree: Update mas_preallocate() testing
>>>   maple_tree: Refine mas_preallocate() node calculations
>>>   mm/mmap: Change vma iteration order in do_vmi_align_munmap()
>>>
>>>  fs/exec.c                        |   1 +
>>>  include/linux/maple_tree.h       |  23 ++++-
>>>  include/linux/mm.h               |   4 -
>>>  lib/maple_tree.c                 |  78 ++++++++++----
>>>  lib/test_maple_tree.c            |  74 +++++++++++++
>>>  mm/internal.h                    |  40 ++++++--
>>>  mm/memory.c                      |  16 ++-
>>>  mm/mmap.c                        | 171 ++++++++++++++++---------------
>>>  mm/nommu.c                       |  45 ++++----
>>>  tools/testing/radix-tree/maple.c |  59 ++++++-----
>>>  10 files changed, 331 insertions(+), 180 deletions(-)
>>>