Received-SPF: pass (google.com: domain of linux-kernel+bounces-180716-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <c795cdeb-17e3-4a34-8405-a11144dad0a4@vivo.com>
Date: Thu, 16 May 2024 15:43:01 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 0/5] dm: empty flush optimization
Content-Language: en-US
To: Alasdair Kergon <agk@redhat.com>, Mike Snitzer <snitzer@kernel.org>,
 Mikulas Patocka <mpatocka@redhat.com>, dm-devel@lists.linux.dev,
 linux-kernel@vger.kernel.org
References: <20240516040235.115651-1-yang.yang@vivo.com>
From: YangYang <yang.yang@vivo.com>
In-Reply-To: <20240516040235.115651-1-yang.yang@vivo.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?TXlDcncwUml0NWI5VkxiQkZ1M2hEcDJpZ1RyVDQ5clVCbzNwbDZYS0laSHhD?=
 =?utf-8?B?N0QrcENmUzdvbkdidUo2SmFZQ0dhM0F0TjZDbHBpSWhnMlozNHpIa2tHUmhK?=
 =?utf-8?B?UTE3SDd1di9XU01vYzVXckVKUW5rVlRNdDN3dVIvdjF6K3J6TG1JU1FFS0NM?=
 =?utf-8?B?MXA3RytMdlZOak9OSjJNTlpSNnNtNFZxeExPZjBzMEpjalV2ejFOa1czcEZU?=
 =?utf-8?B?TEk3dCtaa3JWZnJ5SWd3ZjBmZXJGTmZSTEVnVzVITHo4TEhyUGxZY0YweFht?=
 =?utf-8?B?MlFEdTJsclZJOHpPTlk0ZGk2YlY1MXFNVGtTZU80WmNNSlBhQjJWTURJVHo3?=
 =?utf-8?B?Q0NpNHFWckZNYVVHR1dHanBqT1dSZkYrektHeUJVMG13V00yaVB3NVdLTXlw?=
 =?utf-8?B?QXZMVEJhNk5JMytOQWZqYVg5VlYrbUlWMHFWUXh6amRCYUlNdGtFSFhYVlVG?=
 =?utf-8?B?MTkvUzQyOU1iQ3c1b0FxdmV6bmZxei9tekw0VllzazBLbkVGcGlZbTFoSHBy?=
 =?utf-8?B?cWJtYkdtTU9lMkVWL0xpNmFyV1dTeXF4VE1CUUR0Ri9tNW01WWRFMndOMHR5?=
 =?utf-8?B?dlpOak9mSFNGaXpUT2hiQlp3RUdWWDlsd2N1N1pmTFZlMU1aOWMxMUpzV3Qv?=
 =?utf-8?B?SEVlVDNGVE1rZUZ4bWE1RytXNDRsc3A1U3Y5VkNZVitLN3JmZmtzR3k0SXUz?=
 =?utf-8?B?dlN2bTZZMWQ0b212clkrNGdRYmNXS0hpUXJERnpFWElTUnlnMHBsaVRRRFhK?=
 =?utf-8?B?citweVF6ak5HOElTcTFvenRhTG9nZEhvQTFQVy93Q1lZUFhmcDZhZnVjaElY?=
 =?utf-8?B?c3JRMnZnUitwSWJFaFcwZGtyazk5K1pvbGFvb3I4eTF3WWhhdDZSckxLQmg1?=
 =?utf-8?B?a1pyOEhWOGhlTUVZZzBPeTREemdieklVRUxBbFVSbUpnSFF3U3pnMnpuSkpZ?=
 =?utf-8?B?dGZ6UWtDQlBlSUpvRyt5R1RDNFl2V0hYVmNsZlI3R3dDYjJQTDd5ZmVPWnRN?=
 =?utf-8?B?emh6bFpwSjRjYitWT2Z3OTdSL2tNZkxDMktBQVRRd1E2KzltOGl2QUFRWEI5?=
 =?utf-8?B?STYwd081aXVXWkJ6Q1pDKzJQRXZranoxcGZPREQ4emNtNWI5cTR3UGhRMWdq?=
 =?utf-8?B?bHg0V1NVcCtPZk1MODlKd2xxY21wSWlUUyt1VW1RQkxSYjBrT0NqM3V3SnFM?=
 =?utf-8?B?MjBLMnppVEVDNXRmaWc2c092VTdLaHo4RzVxaTVHRkQ0c0JlLzE2UjBzcDgr?=
 =?utf-8?B?VmtKMHRxcThVR3JUMitLd05Mai9lMU5qV1JuWnBWRllZaDVGVUpjbGV0UTA2?=
 =?utf-8?B?ZE5BYS9XLzdLRjFMZ2IxK2w3UUpWZzVKaVFGK3d0YzNwcW8vVVE2VnhLVG5q?=
 =?utf-8?B?MzBlaEVrc2N0dXlPYWNRYWVlUXdnZkgyNUNFeGMrNUc5YjIvUERGWkxLTmxD?=
 =?utf-8?B?NkMzY3ZyOVNaTHZmSWVES1hIWkVTOGhuRzk5c0pLQlJ3U2ZwS2w2WTdDYjNI?=
 =?utf-8?B?TzBRU1ZRSnJJUjNRTVhrUEJHWlVybFgxd1pBQVZCeG40QzU0WTJDQlVqeFdM?=
 =?utf-8?B?NEhYY0pmV0tJanVlcFFPNDhQUkMvZldSb281S3RXNzduU3IxWldPMlJacXR0?=
 =?utf-8?B?YmtGbkQ4dkxWb1dFLzczY3ZSK1QwVUxDVDA1bjBrbm5YQVlBcktoa0o1TnJ4?=
 =?utf-8?B?TWFyek5DRnAwZENhdEYwQTJPVVo4cVErNm9xNk8wdEdIMjhxalM4OGR5cUZz?=
 =?utf-8?B?NnlnWmhqcm53THNqdFcwSHNRS01wS2Zwd3EzcGM5NTRaNGNEQ25Jd3pxdys0?=
 =?utf-8?B?eGJqRGFCK1dpSHA3ajNrR2x1VHZ5bWk1Y1FiSWdKeUE1L2thZFU4MnN6TTBx?=
 =?utf-8?B?UmxQTW0ySUNmRFN3YXllYzhNTFFUVkRIT29MQTN4aVpRQjd6dmdZUnQ4a0M3?=
 =?utf-8?B?R1FCRTA2Rm5uTk9jdndjTFc0SUNraklkQ1dOL3FGNFVyanRJSDZSNHVwb0w5?=
 =?utf-8?B?YmZnL0d2TXhYN2ttNUQzOVhVRFppMm5kejltYzByTTNrMjRsd0VrTHBoL0Iw?=
 =?utf-8?B?OXVoVThMTG9iTHI1TjNYZGd4WnBNOERBMm55SGpRNzVRUmNEeTFYZEttdW4r?=
 =?utf-8?Q?t+5tvMYu6h3vZiuSxuITeYawP?=
X-OriginatorOrg: vivo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 272c859a-d973-4630-5820-08dc757bd255
X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6411.apcprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 07:43:05.1188
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GrGZl0PCjOWnQHK0j8bOTkjHHAHkA6vqVZqBJnNKt3Zf0iyE8JY6OcdhB4VZwtU+ZBfAx6h/+yzYYHuP++1Rsw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEZPR06MB7060

On 2024/5/16 12:02, Yang Yang wrote:
> __send_empty_flush() sends empty flush bios to every target in the
> dm_table. However, if the num_targets exceeds the number of block
> devices in the dm_table's device list, it could lead to multiple
> invocations of __send_duplicate_bios() for the same block device.
> Typically, a single thread sending numerous empty flush bios to one
> block device is redundant, as these bios are likely to be merged by the
> flush state machine. In scenarios where num_targets significantly
> outweighs the number of block devices, such behavior may result in a
> noteworthy decrease in performance.
> 
> This is a real-world scenario that we have encountered:
> 1) Call fallocate(file_fd, 0, 0, SZ_8G)
> 2) Call ioctl(file_fd, FS_IOC_FIEMAP, fiemap). In situations of severe
> file system fragmentation, fiemap->fm_mapped_extents may exceed 1000.
> 3) Create a dm-linear device based on fiemap->fm_extents
> 4) Create a snapshot-cow device based on the dm-linear device
> 
> Perf diff of fio test:
>    fio --group_reporting --name=benchmark --filename=/dev/mapper/example \
>        --ioengine=sync --invalidate=1 --numjobs=16 --rw=randwrite \
>        --blocksize=4k --size=2G --time_based --runtime=30 --fdatasync=1
> 
> Scenario one:
>    for i in {0..1023}; do
>      echo $((8000*$i)) 8000 linear /dev/sda2 $((16384*$i))
>    done | sudo dmsetup create example
> 
>    Before: bw=857KiB/
>    After:  bw=30.8MiB/s    +3580%
> 
> Scenario two:
>    for i in {0..1023}; do
>      if [[ $i -gt 511 ]]; then
>        echo $((8000*$i)) 8000 linear /dev/nvme0n1p6 $((16384*$i))
>      else
>        echo $((8000*$i)) 8000 linear /dev/sda2 $((16384*$i))
>      fi
>    done | sudo dmsetup create example
> 
>    Before: bw=1470KiB/
>    After:  bw=33.9MiB/s    +2261%
> 
> Any comments are welcome!
> 
> V3:
> -- Focus on targets with num_flush_bios equal to 1 to simplify the code
> -- Use t->devices_lock to protect the dm_table's device list

Please ignore V3, which has a build warning. I will send V4 with the fix.

Thanks