Received-SPF: pass (google.com: domain of linux-kernel+bounces-181816-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <8f49b95e-88df-48b8-9311-8a15414d8107@vivo.com>
Date: Fri, 17 May 2024 15:48:49 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/5] dm: support retrieving struct dm_target from struct
 dm_dev
Content-Language: en-US
To: Benjamin Marzinski <bmarzins@redhat.com>
Cc: Alasdair Kergon <agk@redhat.com>, Mike Snitzer <snitzer@kernel.org>,
 Mikulas Patocka <mpatocka@redhat.com>, dm-devel@lists.linux.dev,
 linux-kernel@vger.kernel.org
References: <20240514090445.2847-1-yang.yang@vivo.com>
 <20240514090445.2847-4-yang.yang@vivo.com> <ZkTXzG1yrPmW64Z6@redhat.com>
 <80ddcd90-2e1c-4fbf-a45a-b1b5ff4d60fb@vivo.com> <ZkYmZaGWPtGIwedG@redhat.com>
From: YangYang <yang.yang@vivo.com>
In-Reply-To: <ZkYmZaGWPtGIwedG@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YWVUNEg4SEV2a0s2WmFYSjRwZlhEaCtZczRDOTdjNGVjYllNSFVPOUxMMmdH?=
 =?utf-8?B?RDNCZ1c5bmhMb01lMzFIeVk5ckwxVlhSS2c4cG52QzJHNk5HUmMxY3p2M3Ji?=
 =?utf-8?B?ZW9wTE9oZmNpaCtZdzgzRG53OU9WeDdKVEF2U1prb3phais5cHFzWnliUFA4?=
 =?utf-8?B?c1RDejY2VDVCeDlFcFNaSWYvcGFqQzJad2xVdmpabXBZbkZ0MWpnVWp5RjFj?=
 =?utf-8?B?S1NWT0FJTW45UHRCemZ6dnZRZm9OYUlTTzl0V0JGVkRWNjBNa2hWU3pxMGxO?=
 =?utf-8?B?RnNRb1hHalZXSmVXVWJqRUZSMkhFOGJxdmFLR2IyRHlwcjYwcWJBdjU5QlVJ?=
 =?utf-8?B?Tjlwd3JwdTVCRXBoTjJrRStRbEVRSjkybU4rY1dVOGY1MkJGNmFtOXVGL3ZC?=
 =?utf-8?B?V2taRzZnUWlKd3gvcVRlUHJFdzc5d3Y0Umh3b2JBTjcyQmI5MTVHZm10emE1?=
 =?utf-8?B?MWFJUUxpM3NFdmkvUTU0aUpTdGpGUUhMaXlaUWxTc2lBNGZiaTZDeG5Va3NQ?=
 =?utf-8?B?cjFNWUUwSlRsS1BsQU93Nkljayt2OUVkUE9neWIvME5uRVJaUlhncXdRUU9G?=
 =?utf-8?B?Zit1UDl0Sk9yQ1A2L0VIVXlYTStVTXV2eXluK3hwK2N4ZGw5NjRQK2pwR1BW?=
 =?utf-8?B?bW1VWC93ZXZJY1ZvT3Nqb1FhZUZkcWczV2lKZ1FRYUs4QTBQWlBrdnlGbHdq?=
 =?utf-8?B?RWhKRTR2U002R295UDh3NGo0VkNaRFJHVlNnR3kvSTRuVzhqaEMwSk40WnFR?=
 =?utf-8?B?cGVTK1ZzaG1VYjRtSk10NWVlR2ovbDBjMEJlSmsvVkxGVEx4V3NVbldiM3lG?=
 =?utf-8?B?R00wN1NsWFN4NzBMdi9MYjRBK3Z1eEt1OGdHYzBRWktIL3ltcTNIRTl6aExF?=
 =?utf-8?B?cXdlbWxjQURNTUFYdWVDLzg2TDFUSUFyTzZZdHNHQndsRFRPWkU2bHJ6VExQ?=
 =?utf-8?B?WkNIUG1yU3FtbFhNdnM3SFlqYWxHc2NkT2FkS1d3OVdCN0FBUk5HWmpPVVh3?=
 =?utf-8?B?Um95T28yK04wV0JlSzNiRk5wSXRIb0FmS1RTako0Y2pmaGd1U2V2NFZpWmMx?=
 =?utf-8?B?ZGhLd0RwVFdBVElMNlVkTm44REx4N0xKWDREMkZCR3ZDUnhzdlFZS3dacUZR?=
 =?utf-8?B?THUyc0s4cmNQUERzdlY0eEl1djd0UXpPZjFyNFVTdUFLNzFwN21ISjBRck9S?=
 =?utf-8?B?MDd2UmFqZUdlak03L1hKMmcxZDErczJ0cTN6eWorKzBGVmhjZ1czUEU5S3Jz?=
 =?utf-8?B?RzErYzRxdThOR0JIN0RWaTJRTzB3Z1FHZWFCMU5FQXprS1lyaUloQXJRRWhU?=
 =?utf-8?B?MlVMNGY1c2hXSWRCN0haQ3dHQTI2UnpQd085dG91VWhjU0VtYVlTUDFlZ3JO?=
 =?utf-8?B?VmUzU3ZqZGpkVU04dHp0cllMYnBNRktxYVQyNTc0QnFsUWdXdjQwNU9iZ2gx?=
 =?utf-8?B?ZVNNbDZ2RUtvREdEdjF4Q2krZTZZcmhvdHJGK255LzMxaEtXcklyb3lYczNq?=
 =?utf-8?B?RURyaHJXYk9DQkt4RFhzeEhPeFdmMzgxbWQyekRJUjduSjJKUWxxdHAxVHdt?=
 =?utf-8?B?bnhESUd1ODJtZUk0cGVMZUFXWmJtdjIrcmwycGxaamRPdFJYN3hZb0tVay9o?=
 =?utf-8?B?VmRUOG9XU3BkMTFkblZLOFQrTVQwYW5ST3F5RDM3ZkdrTVh4cEN2ZlF5SENp?=
 =?utf-8?B?OWozeWNiL3RTRFNkbFF4bVNRU2FqZjRtSTNMUlhwajA5Qjhjc1dBU3FtNk14?=
 =?utf-8?B?SjY5RUNFWVZBMnJTYkJoWFJ6amI0NGpNenVOaVREOU5kbmVBLy81dUVsVGNL?=
 =?utf-8?B?MjRRNWtXOVF1YjR6bjQ0eFBMQm84cXZtV3E1SDNRR0pFQUR2eHhRbGt5NnUx?=
 =?utf-8?B?dUVwcUE0VnNqNHRxcVlydmU5Y2VzZW5wOHRJeDNVc3cyQ2NvVm1RTFVnYjdm?=
 =?utf-8?B?aVdLUFptQWJBcW9PeUdwVG9XY3FENFFxZlNVaTZyTkVlQWZGTlFaZ25xc2wv?=
 =?utf-8?B?V0Q3Y2YxNTRMMFdzVUxmN2pvYWRCMjNIcG1nQXAybWx6cDlkSnE3Zm1qYzlx?=
 =?utf-8?B?dHB2ekxGdGU4MFJDZE5MS2VEcE5CNHJCTHZaWnFpeEZIRkRqV2F4M0lsd2tE?=
 =?utf-8?Q?0Y0s8OMVYMnP8+UPQW4A+y8bY?=
X-OriginatorOrg: vivo.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b13edea2-8710-4910-bdb3-08dc7645cc7a
X-MS-Exchange-CrossTenant-AuthSource: TYSPR06MB6411.apcprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 07:48:53.6075
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KAYXvvAD4gNncV3SzY23ijtceVMqO18VV3GeLhGAoBjXlW+8jHRRksRz8gkqQY/pJT0jeiiv1ny/hrteuLBJZg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5669

On 2024/5/16 23:29, Benjamin Marzinski wrote:
> On Thu, May 16, 2024 at 09:55:53AM +0800, YangYang wrote:
>> On 2024/5/15 23:42, Benjamin Marzinski wrote:
>>> On Tue, May 14, 2024 at 05:04:42PM +0800, Yang Yang wrote:
>>>> Add a list to the struct dm_dev structure to store the associated
>>>> targets, while also allowing differentiation between different target
>>>> types.
>>>
>>> I still think this is more complex than it needs to be. If devices that
>>> support flush_pass_around can guarantee that:
>>>
>>> 1. They will send a flush bio to all of their table devices
>>> 2. They are fine with another target sending the flush bio to their
>>>      table devices
>>>
>>> Then I don't see why we need the table devices to keep track of all the
>>> different target types that are using them. Am I missing something here?
>>
>> I attempted to enhance this solution to support additional target types,
>> such as those with num_flush_bios greater than 1.
> 
> I'm still missing why sending a flush to each target type is necessary
> to handle num_flush_bios > 1. As long as the targets meet the
> requirements I listed before, AFAICS it should still work with only one
> of the targets mapped to each device.
> 
> Say the table devices are sda, sdb, sdc, and sdd.  If you have 4 linear
> targets, each mapped to a different table device, and one stripe target
> mapped to all of them.  It doesn't really matter if you don't call
> __send_empty_flush_bios() for the stripe target, does it? all if its
> stripe devs will still get flushes. Similarly, it's fine if one of the
> linear targets doesn't get called (in fact it's fine if all the linear
> targets don't get called, since the stripe target will send flushes to
> all the devices). If there were only 3 linear targets, then the stripe
> target would get linked to a table device, so it would get a flush sent
> to it. Can you explain a situation where you need the to send a flush to
> multiple targets per table device for this to work, if you assume the 2
> guarantees I mentioned above (the target sends flushes to all their
> devices, and they don't do something special so they need to be the one
> to send the flushes).

Yes, if the targets meet the requirements you listed previously, there
is no need to send a flush to each target type.
I think I may be overthinking this. I tried to handle some targets with
num_flush_bios > 1 that don't meet the requirements.

>>
>>> If we don't need to worry about sending a flush bio to a target of each
>>> type that is using a table device, then all we need to do is call
>>> __send_empty_flush_bios() for enough targets to cover all the table
>>> devices. This seems a lot easier to track. We just need another flag in
>>> dm_target, something like sends_pass_around_flush.
>>>
>>> When a target calls dm_get_device(), if it adds a new table device to
>>> t->devices, then it's the first target in this table to use that device.
>>> If flush_pass_around is set for this target, then it also sets
>>> sends_pass_around_flush. In __send_empty_flush() if the table has
>>> flush_pass_around set, when you iterate through the devices, you only
>>> call __send_empty_flush_bios() for the ones with sends_pass_around_flush
>>> set.
>>>
>>> Or am I overlooking something?
>>
>> If I understand correctly, you are suggesting to iterate through all the
>> targets, handling those with sends_pass_around_flush set, and skipping
>> those where sends_pass_around_flush is not set. I believe this approach
>> may result in some CPU wastage.
>>
>>    for i in {0..1023}; do
>>      echo $((8000*$i)) 8000 linear /dev/sda2 $((16384*$i))
>>    done | sudo dmsetup create example
>>
>> In this specific scenario, a single iteration of the loop is all that
>> is needed.
> 
> It's just one iteration of the loop either way. You either loop through
> the targets or the devices.  It's true that if you have lots of targets
> all mapped to the same device, you would waste time looping through all
> the targets instead of looping through the devices.  But if you only had
> one striped target mapped to lots of devices, you would waste time
> looping through all of the devices instead of looping through the
> targets.

Yes, I get your point. This patchset may make things even worse for
the striped target.
I am just curious, in what scenario is the "dm-strip" target mapped to
a large number of underlying devices from the same block device.

> -Ben
>   
>>>
>>> -Ben
>>>
>>>>
>>>> Signed-off-by: Yang Yang <yang.yang@vivo.com>
>>>> ---
>>>>    drivers/md/dm-table.c         | 36 +++++++++++++++++++++++++++++++++++
>>>>    include/linux/device-mapper.h |  3 +++
>>>>    2 files changed, 39 insertions(+)
>>>>
>>>> diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
>>>> index bd68af10afed..f6554590b7af 100644
>>>> --- a/drivers/md/dm-table.c
>>>> +++ b/drivers/md/dm-table.c
>>>> @@ -741,6 +741,8 @@ int dm_table_add_target(struct dm_table *t, const char *type,
>>>>    	if (ti->flush_pass_around == 0)
>>>>    		t->flush_pass_around = 0;
>>>> +	INIT_LIST_HEAD(&ti->list);
>>>> +
>>>>    	return 0;
>>>>     bad:
>>>> @@ -2134,6 +2136,25 @@ void dm_table_postsuspend_targets(struct dm_table *t)
>>>>    	suspend_targets(t, POSTSUSPEND);
>>>>    }
>>>> +static int dm_link_dev_to_target(struct dm_target *ti, struct dm_dev *dev,
>>>> +		sector_t start, sector_t len, void *data)
>>>> +{
>>>> +	struct list_head *targets = &dev->targets;
>>>> +	struct dm_target *pti;
>>>> +
>>>> +	if (!list_empty(targets)) {
>>>> +		list_for_each_entry(pti, targets, list) {
>>>> +			if (pti->type == ti->type)
>>>> +				return 0;
>>>> +		}
>>>> +	}
>>>> +
>>>> +	if (list_empty(&ti->list))
>>>> +		list_add_tail(&ti->list, targets);
>>>> +
>>>> +	return 0;
>>>> +}
>>>> +
>>>>    int dm_table_resume_targets(struct dm_table *t)
>>>>    {
>>>>    	unsigned int i;
>>>> @@ -2162,6 +2183,21 @@ int dm_table_resume_targets(struct dm_table *t)
>>>>    			ti->type->resume(ti);
>>>>    	}
>>>> +	if (t->flush_pass_around) {
>>>> +		struct list_head *devices = &t->devices;
>>>> +		struct dm_dev_internal *dd;
>>>> +
>>>> +		list_for_each_entry(dd, devices, list)
>>>> +			INIT_LIST_HEAD(&dd->dm_dev->targets);
>>>> +
>>>> +		for (i = 0; i < t->num_targets; i++) {
>>>> +			struct dm_target *ti = dm_table_get_target(t, i);
>>>> +
>>>> +			if (ti->type->iterate_devices)
>>>> +				ti->type->iterate_devices(ti, dm_link_dev_to_target, NULL);
>>>> +		}
>>>> +	}
>>>> +
>>>>    	return 0;
>>>>    }
>>>> diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h
>>>> index 0893ff8c01b6..19e03f9b2589 100644
>>>> --- a/include/linux/device-mapper.h
>>>> +++ b/include/linux/device-mapper.h
>>>> @@ -169,6 +169,7 @@ struct dm_dev {
>>>>    	struct dax_device *dax_dev;
>>>>    	blk_mode_t mode;
>>>>    	char name[16];
>>>> +	struct list_head targets;
>>>>    };
>>>>    /*
>>>> @@ -298,6 +299,8 @@ struct dm_target {
>>>>    	struct dm_table *table;
>>>>    	struct target_type *type;
>>>> +	struct list_head list;
>>>> +
>>>>    	/* target limits */
>>>>    	sector_t begin;
>>>>    	sector_t len;
>>>> -- 
>>>> 2.34.1
>>>>
>>>
>