Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e37b3007-a9eb-ff07-2efd-467a5eb8114c@oracle.com>
Date:   Wed, 11 Jan 2023 09:58:45 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.1
Subject: Re: [PATCH V4 6/6] blk-mq: Build default queue map via
 group_cpus_evenly()
Content-Language: en-US
To:     Ming Lei <ming.lei@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Jens Axboe <axboe@kernel.dk>
Cc:     linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
        Christoph Hellwig <hch@lst.de>
References: <20221227022905.352674-1-ming.lei@redhat.com>
 <20221227022905.352674-7-ming.lei@redhat.com>
From:   John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <20221227022905.352674-7-ming.lei@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Yk55SG9vMTNKaWNPWFhWTzNScU4vNnVFQ2hNM3N2TFJvWXVINlQ1aDZEQUpG?=
 =?utf-8?B?b2pkMURqVFdNZUJudWlPVjRZQjN0Zy9oMHJkWStsUEZWaHJEMjlRYTRpU1VM?=
 =?utf-8?B?eFZaNzdmdzhCWGpyVlZpOG9naW1CV3VVMFZwYVBScU9FZHRXaXA0MFY4MjBP?=
 =?utf-8?B?ZXFIWVg4U1RjZnZheklFenZWanMvMHVpdmFVeFZhRjBSRGFUaElXd0pjcXQz?=
 =?utf-8?B?dElKcTZrNk1BYzhTS1IyMm5GTW5PMjFDL0VFZ3NQS3lXd0Z3UUxRV0MwWWF1?=
 =?utf-8?B?T1VuYUk2Yndvb2V0VkRTaFZ0K0RHUG5BVHUrbklzelpIQjdRNVh5OHhBaHhZ?=
 =?utf-8?B?MmVFQ01ORkk2L1RuR1d2QUJFMjl5U0R3RmR5cW5kNlg2anJDelNHRHM0Z2or?=
 =?utf-8?B?WENLcHpPVklMQWE5cjI0ejE3YWhPbnk0b09QNGUvYzM2MUxSUGZyL0M3aktK?=
 =?utf-8?B?VE9zeGdUY3d4UU5SR2lGS1FoM05vVU1DQSt5dzdYaUQyaXB4a0pqdlZ6Qm9R?=
 =?utf-8?B?bmlWM2dwQ2tJWkN1MXQrQm5aK05PVlBmN0NpSHA3ZlQ2eTRheTQvN1FyOEYy?=
 =?utf-8?B?OCtZeEhLTUF2T2NoNGlSV01OdTljaDNlQXVybGRBWktndFhHdDJleGFmQ2lS?=
 =?utf-8?B?ZG1ndUcwVWQvSEJCWVU0UlBvV0d1Zi9lYldOTG5jUkd2VFRXTXdabXMzSjNK?=
 =?utf-8?B?czcydGRmOTVFRzZDWThkYk5KcjVQWkFIKzBBSTdtVE5COUN4djFuVk5rZHZE?=
 =?utf-8?B?SWF4Z3B1dkFUa3k1Q29KL1BLQVh0WFpUcjNqdUJyNEpGTUwvMzZkSm9zSGhK?=
 =?utf-8?B?QXRBRG5oSUtiMDFYOGxKb29ueHROUTFpWFo3YnFRdmhIcWFVYWlsSmRBcHlr?=
 =?utf-8?B?K3h0d0hrUVZQRWRQSDlqWGhSbSs4U0FyT1BaRy9BeENiMjdDKzRTRnEwQ2Q3?=
 =?utf-8?B?dTA5aXpNQ1U5eHpPQXBiSTdQZ3puTkZsMnAzd0RVRGV2dXR6eFYyMFB4bmIx?=
 =?utf-8?B?RmJYcklUSjZBUnpETFMxYktJb2l6TzRsNTNySWdCZDljalJnUW8zeW9PUnk4?=
 =?utf-8?B?UU02MjNOeFZ5MTV3R2VhZ3doQWpFVldyMTVZREdXV2d5dXorVFFGSFZ0b3Jq?=
 =?utf-8?B?UGRtcjlvNnRSaVNLYnpXZkdWeGhPNDlQRG50Z3JmV1lhUy9INHJMNGNodzB1?=
 =?utf-8?B?OUdFdXUyRkpFbk1Ncm1QSTlUSGRYVGcwaXcxUjZjTW9qUHFYODZmLzdqaHE2?=
 =?utf-8?B?S1RLSVJpNjVaNVdrM0VUWVV0Yys2K2dEOGc3UDZsUzFzSXFBaWlIa0FPeHlQ?=
 =?utf-8?B?MmUrM2tJaGFpbzhhN2ZhNjRUSFdjdmh2bHpqZ1NMbTNBS1lVYjJXWFcxeXhF?=
 =?utf-8?B?MmU2d2N2NlljeXR4ZXNseWl1VitkLzh0eFd1eHRDU0ZmNlFDa01ySHFReXRX?=
 =?utf-8?B?bDhocFBiZTBLcmt1d2RjSmppSC9wYXU5TkZGanBWcC9pdm1jT1lDZ0R3ZmlQ?=
 =?utf-8?B?Mk5RTDZIcDhmRFRrM3pvQ1E2N2VFUURsYXlVSTJJVGUxK0dHZDZoQlBzSlND?=
 =?utf-8?B?YlYzYVVXbGR5SGkrRGxmcFVLOHVCZ3BJZU9zYXRvVFY0VE5NYWZ0RWFkZmFq?=
 =?utf-8?B?WVB4L0lOMzJIK0I0dW92WHhSdURvOEZyYWpFYktjZ1NGbE1TcjBENm1ObzNw?=
 =?utf-8?B?a1FMYW1kb2l4QmYyaktxaTl2Mmkwblk0OGwzb3JGSkUrUG9aRDlTL1djM1k0?=
 =?utf-8?B?WnpwSG9iVU1lMHB5THB1NTFkcXF4RDRBOHJvZW9ka25PMWN3VFpuVlFCMFdS?=
 =?utf-8?B?Vk15MnFHRHEzMlRKSDRXRXZWUldzK3l2clR2T0ZGczU5UW5ZL2lsSWFmclpo?=
 =?utf-8?B?cWpWc0VJdGgwc3doV1NINTFSOThTZ3lCU0xVMkt2Unk2MDZNQVdsVTZ1d2hR?=
 =?utf-8?B?L2FSaXhkWHNiZ2VjbkN0TmZ1MGY0OFI4QXF0QXF1ZXduTnR1MDFyTFZkOWlF?=
 =?utf-8?B?WlNmSVRBMDA0MG1ScGcwcUdVZmEvQzkwZldJNlNadi9DVExBOU9QMFFUcTl3?=
 =?utf-8?B?UlpzWTV3ZlNvTGZpSHBuWm5mR1NmNjdFRjNhYW1ESnhrOWU3N3VMTTg4ZXhK?=
 =?utf-8?B?VjdRT0FCbGxoZ0M1SEJWbDh1SUNUMW15V0J3RDc5S2oyd1EwTG0ydjRsVmlq?=
 =?utf-8?B?YWc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: dYgwlmDAg3bHYWCUWWvPG0zxQKX2Togb/i4dZdNmzJd8375Iwz+4dk3cfbOA2yFlOZcC3zTwCdXZzXLG99ASBbmOvXlMkWZ29ttGTzdUaygHVhlba9j1BtuqAQqZdu3tzttd1cF6PTVtLd84LrSGprBDcc0A47CfMQnHboJxKFZkp0n03XyR7kEKpqfCedfsiUuWcx5hKLTuFAjmAQisv0iEV68QV7mmN729juKGJ7+urgTd5a9O6oK7bdm71fphw+MGxAUvSXxGd7GM0S/KXEMb1dPUhcBO7kxIdOH2BM/6ao2p/lJbUUkqCY6C8xTa2n87oWWmHfAOZ8Jb/plPOplbN8O6eOYZ276dO3et4/K2xv9oo4lXhO/EWpgMwr1Re/13qhTkEzfeiBll1wcQrmNnr0CogsMo5fKu67+yJbhimbdu1MMJAxzs/RBwhYPb69wF41Eec66jebnRTDOSpYOOpiUUY6W9zCNcDs9iaZBglZdnREgZGSdg7+gbeStIKH9XVK+Rzvg+UI/59WYDM8A/KUptAekqSVI+nDw69nEda8J5qKgdzEayoUv/TWmBs+CAswbsOlFcv4J/OtmuWJXTecWqhr3Ydu5rvag1dwtKQ9dVjl/GMCl9SuhG9ewbY5jmAPSF9C5JkKDjla86nGb8pvfY/glkdlyZfEGPRLRwWMz/R0/U6gX71ELYVcL8HGi8scm0/otnEZrl6thEbwTP+ELgfIdKMqpNa6ImJ0MGBMF26oeYR3gM+EuZoW7+geAE1J7R4h2s6tRhwVRJP5GAWfalrzNJG+7ZZR5jTxtWAnBfUSHOzNhlmSD/VlUMm3Wi6Su0dUuJn6v7sbzDoK+vuThsYIsieuJv4ZaA3OSHeXEvcvzMH2iqMulbCaGA
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 53d3ece6-b4c0-4d6b-23b4-08daf3ba7076
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2023 09:58:50.4946
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zte6NA1CCZxKBPz83nQIgr0Eha687VqybSSlo/cNtgIDdNwMpY6zh0/oOz+Z6xakxr8v7rUU76Sifz9hJEuzFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN0PR10MB5192
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2023-01-11_04,2023-01-10_03,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0
 adultscore=0 spamscore=0 mlxscore=0 bulkscore=0 malwarescore=0
 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2301110076
X-Proofpoint-GUID: ejmeN_HCCsfot47Zkg-OzU4Er5-tcYCM
X-Proofpoint-ORIG-GUID: ejmeN_HCCsfot47Zkg-OzU4Er5-tcYCM
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 27/12/2022 02:29, Ming Lei wrote:
> The default queue mapping builder of blk_mq_map_queues doesn't take NUMA
> topo into account, so the built mapping is pretty bad, since CPUs
> belonging to different NUMA node are assigned to same queue. It is
> observed that IOPS drops by ~30% when running two jobs on same hctx
> of null_blk from two CPUs belonging to two NUMA nodes compared with
> from same NUMA node.
> 
> Address the issue by reusing group_cpus_evenly() for building queue
> mapping since group_cpus_evenly() does group cpus according to CPU/NUMA
> locality.
> 
> Also performance data becomes more stable with this patchset given
> correct queue mapping is applied wrt. numa locality viewpoint, for
> example, on one two nodes arm64 machine with 160 cpus, node 0(cpu 0~79),
> node 1(cpu 80~159):
> 
> 1) modprobe null_blk nr_devices=1 submit_queues=2
> 
> 2) run 'fio(t/io_uring -p 0 -n 4 -r 20 /dev/nullb0)', and observe that
> IOPS becomes much stable on multiple tests:
> 
> - without patched: IOPS is 2.5M ~ 4.5M
> - patched: IOPS is 4.3 ~ 5M
> 
> Lots of drivers may benefit from the change, such as nvme pci poll,
> nvme tcp, ...
> 
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>

FWIW, but just a comment below:

Reviewed-by: John Garry <john.g.garry@oracle.com>

> ---
>   block/blk-mq-cpumap.c | 63 +++++++++----------------------------------
>   1 file changed, 13 insertions(+), 50 deletions(-)
> 
> diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
> index 9c2fce1a7b50..0c612c19feb8 100644
> --- a/block/blk-mq-cpumap.c
> +++ b/block/blk-mq-cpumap.c
> @@ -10,66 +10,29 @@
>   #include <linux/mm.h>
>   #include <linux/smp.h>
>   #include <linux/cpu.h>
> +#include <linux/group_cpus.h>
>   
>   #include <linux/blk-mq.h>
>   #include "blk.h"
>   #include "blk-mq.h"
>   
> -static int queue_index(struct blk_mq_queue_map *qmap,
> -		       unsigned int nr_queues, const int q)
> -{
> -	return qmap->queue_offset + (q % nr_queues);
> -}
> -
> -static int get_first_sibling(unsigned int cpu)
> -{
> -	unsigned int ret;
> -
> -	ret = cpumask_first(topology_sibling_cpumask(cpu));
> -	if (ret < nr_cpu_ids)
> -		return ret;
> -
> -	return cpu;
> -}
> -
>   void blk_mq_map_queues(struct blk_mq_queue_map *qmap)
>   {
> -	unsigned int *map = qmap->mq_map;
> -	unsigned int nr_queues = qmap->nr_queues;
> -	unsigned int cpu, first_sibling, q = 0;
> -
> -	for_each_possible_cpu(cpu)
> -		map[cpu] = -1;
> -
> -	/*
> -	 * Spread queues among present CPUs first for minimizing
> -	 * count of dead queues which are mapped by all un-present CPUs
> -	 */
> -	for_each_present_cpu(cpu) {
> -		if (q >= nr_queues)
> -			break;
> -		map[cpu] = queue_index(qmap, nr_queues, q++);
> +	const struct cpumask *masks;
> +	unsigned int queue, cpu;
> +
> +	masks = group_cpus_evenly(qmap->nr_queues);
> +	if (!masks) {
> +		for_each_possible_cpu(cpu)
> +			qmap->mq_map[cpu] = qmap->queue_offset;

I'm not sure if we should try something better than just assigning all 
CPUs to a single queue (which we seem to be doing), but I suppose we 
don't expect masks alloc to fail and there are bigger issues to deal 
with if it does ....

> +		return;
>   	}
>   
> -	for_each_possible_cpu(cpu) {
> -		if (map[cpu] != -1)
> -			continue;
> -		/*
> -		 * First do sequential mapping between CPUs and queues.
> -		 * In case we still have CPUs to map, and we have some number of
> -		 * threads per cores then map sibling threads to the same queue
> -		 * for performance optimizations.
> -		 */
> -		if (q < nr_queues) {
> -			map[cpu] = queue_index(qmap, nr_queues, q++);
> -		} else {
> -			first_sibling = get_first_sibling(cpu);
> -			if (first_sibling == cpu)
> -				map[cpu] = queue_index(qmap, nr_queues, q++);
> -			else
> -				map[cpu] = map[first_sibling];
> -		}
> +	for (queue = 0; queue < qmap->nr_queues; queue++) {
> +		for_each_cpu(cpu, &masks[queue])
> +			qmap->mq_map[cpu] = qmap->queue_offset + queue;
>   	}
> +	kfree(masks);
>   }
>   EXPORT_SYMBOL_GPL(blk_mq_map_queues);
>