Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 20 Apr 2023 20:50:48 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        <linux-kernel@vger.kernel.org>, Olivier Dion <odion@efficios.com>,
        <michael.christie@oracle.com>
Subject: Re: [RFC PATCH v9 2/2] sched: Fix performance regression introduced
 by mm_cid
Message-ID: <20230420125048.GA154262@ziqianlu-desk2>
References: <20230419155012.63901-1-mathieu.desnoyers@efficios.com>
 <20230419155012.63901-2-mathieu.desnoyers@efficios.com>
 <20230420095610.GA153295@ziqianlu-desk2>
 <c01ddfc5-9410-14e1-55f7-c24f44447f8a@efficios.com>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <c01ddfc5-9410-14e1-55f7-c24f44447f8a@efficios.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M094YkhnRjFjUlFEV0JPYjBqckpQV0hmYVJpM29BNnUyeXhra09mVDczVUlv?=
 =?utf-8?B?ZURDTmNjSUo3ZzFxaXhMR2dsZlRBay9sRGNYZTR3WWtHbW9hQlBhVC9vV3Rt?=
 =?utf-8?B?SHdoaEZ0Yk5tU0M2bGJHdWw1eFYvRkl6MTNMZWY5RGFCeHBTNnE2ek1JWjNk?=
 =?utf-8?B?ZjVKNUpzNWp0d2Z1ay9HajBOM0hmVldGWjVmSlhITXlONmpLNk50QVdjcnVF?=
 =?utf-8?B?Wm9qK1RQc1R5T1dNcG1NMmJaWmRwa0tNemc5VHhhZjhIRFB1ZVVHeFh2ektv?=
 =?utf-8?B?clYvVFJpeWZSWXl2MW8ycTgwS1Z1U1VDaHpyTWp5NlNETTJjcXNGWEFxTkRP?=
 =?utf-8?B?OVBJSTVCVC92RE5uandnTWJPQW90R2dzWmFvY0xZeHZZazhObjhkMlJUb1NQ?=
 =?utf-8?B?YXF5WjlaNFhJTkNkQzlUV3p4VGRuV3piVis4cHhYbGR4dzRxdjNuMlBObUlN?=
 =?utf-8?B?c0ticjNsTEt1bHZzQWJWVmhqN0crQ1lpaGRYdjNIc2dCTFZENTh2UVFNek1r?=
 =?utf-8?B?c0hmQmp2SnJGUUw0VWRXRWJHSFNCQk1YZ0hDK0JXUElsRG9Vb0NYeTRaYTY5?=
 =?utf-8?B?S3FYak1xcGhxZTJIWmNUdE5YdUNnaUovVjhtOHVnMTZZQUtzalQwdU1hL09E?=
 =?utf-8?B?MVRJOEsrdDlFcXVMS1pXUGVmVXJLaUQ5Z2ltZi9OZXFKZU5HQW9FdmZMV3hU?=
 =?utf-8?B?Z2ZpUmYrV24xdk9GdzNSUlJqaG03M3h4SmNRTjg5YWsxbVFCSVBDRStDd0Ir?=
 =?utf-8?B?SWJpWU9HSjFvNmdFSG1jYmVRMlBnUWRXdEFENEtDOFNPS0o3eDVweWErakxK?=
 =?utf-8?B?aXpiY09lNWdPNGFlVGhCbzhUU2piclpaaHZkM3l0bXh3VkFhTEk5cVlNbmdU?=
 =?utf-8?B?eFpqd1RoRElDYzhORFkrMW9nZm1DM1daUlloS2JQR0NZeE1PNnFVblpvNTlj?=
 =?utf-8?B?VllTZ0J2dWNXY0Y3SUZZWjB3V1RxbXRFblJBVDRaVHlSa0ZNTzhXaVZpRzF6?=
 =?utf-8?B?NEZPTUx6NHlBSUR4WDh3Wi9mKyt1UEdZQTR4VFM2c25Ka0lPZm5tUERTbW0v?=
 =?utf-8?B?OU5qSzRaYjlFcm8wVUk4UFJydDdsTUphMXVKbTdqZEFPeGFZMXhwSGxrUmtV?=
 =?utf-8?B?cXdSNDVnb1pPd2xPMUFlRi8xUVNXL3d4TVlGNzcxS1YvQVlrSzYrWENkWGhv?=
 =?utf-8?B?cTN0R2VITVJhdUVVZmxib29XV0xKRlREQlZpZDM2QTBiWnoxN1p2bmhmZFNN?=
 =?utf-8?B?OWxWM2N1SWRtQWpUK0FxR01PVVFtUm1JUzRqQjluemFkbFE3TlhlV0xYclQ5?=
 =?utf-8?B?ZUVTM2ZTS3dVb1NSN2hlTW5BVEdUL1dsUlZuNTRqZ0tRRS9pendFNFAxa2xa?=
 =?utf-8?B?OTFEblZnSWI4U1lkVmx1Ynl6N2hHaUVGQmdLYnoycFFQcUx1TFBYTUxDdHUy?=
 =?utf-8?B?eXBqTEYxaG05S0FBWVo4ZmM2VUZCcGZMN2c4eThDNnNZVmlyLzRkMWl6Yjdo?=
 =?utf-8?B?bE4zOXNOcTl0amswMkNsb2lTb1hWUElyWjFSd0M5dWNQaDlBYWlvRlRiMEZr?=
 =?utf-8?B?RCt3MzFRNS92ME1JK0YrTUdLWTBRQ1lDYVhTZU54S2YzUlRaTGJ6am12aUJr?=
 =?utf-8?B?ditQLzJySGlLak9uL2N0TExVOHJNejkxalVMK0tGcXIxVGgvcGlxTkpFS2pD?=
 =?utf-8?B?TXNFaG9vNG1xRG9WNVI4dkM4bVpqUUN2cG1oZEsrQ2dpcG5lQVNOTWJ0ZXdE?=
 =?utf-8?B?ekVXWXJUNy9TajVyUDl3cHJVWnB1Y2VlTGZaNURiTlAycGROVDAvTW5hblF5?=
 =?utf-8?B?bTNnTVJZK05ZSjNTbkFCU1MvajJDelJ2Ukp6bkUwdlNRTDg3eG9YU2k4MzBE?=
 =?utf-8?B?eTRFWnlhRy9CSnBleVpwaGI2dnBvR2E2ZXZ6NzFHaTdpWHJRbGgrRFBpMndj?=
 =?utf-8?B?L244NVh2SkZXa1haNEpqQ29zVG5PTm9BUUhLbldGM2xiMkN3cU1SUHJ2aWth?=
 =?utf-8?B?TVlMQXhpRUxxREpZdnRmdFVkTE9hM2pXS3VTRzlnQWV1SlRqTzlkZS9UeHU3?=
 =?utf-8?B?UnN4SGlTdjNFcm1ITk96K3lTeU50U0NZd1JlUDh1RUtJbUhkUkYrSExjMDhN?=
 =?utf-8?Q?uR3ggvDwTjkb0fi84H5QfPJo4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b5b065fd-3b84-4446-97f1-08db419de2af
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2023 12:50:57.3908
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: SON5XSsiR7S1u2zejejQxnUCYdeUp1CfkV4C/KsQNZ0KmzAFAxlHAQHR9My/WRhx0WYeMT9a7krn1JgmAtKeDQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB5139
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Apr 20, 2023 at 08:41:05AM -0400, Mathieu Desnoyers wrote:
> On 2023-04-20 05:56, Aaron Lu wrote:
> > On Wed, Apr 19, 2023 at 11:50:12AM -0400, Mathieu Desnoyers wrote:
> > > Introduce per-mm/cpu current concurrency id (mm_cid) to fix a PostgreSQL
> > > sysbench regression reported by Aaron Lu.
> > 
> > mm_cid_get() dropped to 5.x% after I disable CONFIG_DEBUG_PREEMPT, using
> > __this_cpu_X() doesn't help, I suppose that is because __this_cpu_X()
> > still needs to fetch mm->pcpu_cid.
> > 
> > Annotate mm_cid_get():
> > 
> >         │     static inline int mm_cid_get(struct mm_struct *mm)
> >         │     {
> >    0.05 │       push   %rbp
> >    0.02 │       mov    %rsp,%rbp
> >         │       push   %r15
> >         │       push   %r14
> >         │       push   %r13
> >         │       push   %r12
> >         │       push   %rbx
> >    0.02 │       sub    $0x10,%rsp
> >         │     struct mm_cid __percpu *pcpu_cid = mm->pcpu_cid;
> >   71.30 │       mov    0x60(%rdi),%r12
> >         │     struct cpumask *cpumask;
> >         │     int cid;
> >         │
> >         │     lockdep_assert_rq_held(rq);
> >         │     cpumask = mm_cidmask(mm);
> >         │     cid = __this_cpu_read(pcpu_cid->cid);
> >   28.44 │       mov    %gs:0x8(%r12),%edx
> >         │     if (mm_cid_is_valid(cid)) {
> > 
> > 
> > sched_mm_cid_migrate_to() is 4.x% and its annotation :
> > 
> >         │     dst_pcpu_cid = per_cpu_ptr(mm->pcpu_cid, cpu_of(dst_rq));
> >         │       mov     -0x30(%rbp),%rax
> >   54.53 │       mov     0x60(%r13),%rbx
> >   19.61 │       movslq  0xaf0(%rax),%r15
> > 
> > The reason why accessing mm->pcpu_cid is so costly is still a myth to
> > me...
> 
> Then we clearly have another member of mm_struct on the same cache line as
> pcpu_cid which is bouncing all over the place and causing false-sharing. Any
> idea which field(s) are causing this ?

That's my first reaction too but as I said in an earlier reply:
https://lore.kernel.org/lkml/20230419080606.GA4247@ziqianlu-desk2/
I've tried to place pcpu_cid into a dedicate cacheline with no other
fields sharing a cacheline with it in mm_struct but it didn't help...

Thanks,
Aaron