Received-SPF: pass (google.com: domain of linux-kernel+bounces-157212-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Wed, 24 Apr 2024 11:14:08 -0400
From: Gregory Price <gregory.price@memverge.com>
To: Dongsheng Yang <dongsheng.yang@easystack.cn>
Cc: Dan Williams <dan.j.williams@intel.com>, axboe@kernel.dk,
	John Groves <John@groves.net>, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org,
	Dongsheng Yang <dongsheng.yang.linux@gmail.com>
Subject: Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)
Message-ID: <ZikhwAAIGFG0UU23@memverge.com>
References: <20240422071606.52637-1-dongsheng.yang@easystack.cn>
 <66288ac38b770_a96f294c6@dwillia2-mobl3.amr.corp.intel.com.notmuch>
 <ef34808b-d25d-c953-3407-aa833ad58e61@easystack.cn>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ef34808b-d25d-c953-3407-aa833ad58e61@easystack.cn>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?WjYrVStXd0JDYzQ3d0ZZZFI4VDdzdjNackt4QnltcEt6L1p4Ny93VmRTQndz?=
 =?utf-8?B?SGtmUjI5dlRXWFVtY0FoYUt0K2dLMlJ4S1U0WWU5SjV6YnRkYTlRb0pHd2ts?=
 =?utf-8?B?V25DVUFGV0ZoWEJUbkdhc2Uxa29jdWM0RDgwT1RGWjcxSFpiRmhFaVUzWkYz?=
 =?utf-8?B?SEtUWTBNUUVnd043QkJRZVY1QXlSWHdHUEJoNjdGWGx4R3RvY0paeURpOURL?=
 =?utf-8?B?Tk4vN20vTDFVQ1F6ZXVtZGYxbmM2VHFoamNpZGorVDV6WktRQjNCcVQ2bW84?=
 =?utf-8?B?OWhkaHZvTncyL0dvT0ZVdERxcEtiSUdHZlpES2FqbHYzR3lSQ0JlRVppaXpJ?=
 =?utf-8?B?MEVXM0FGcU1nMnJ4VmdNSlA1YkFJNUIrYkFYQzJUcDlFRkU3OWdseWxVdzBz?=
 =?utf-8?B?V1NOSno3eTNCRktkNkh2TGNCZWVxYlI4Zmx5c0xjaFp1QlJScjR4Y3FWNCtV?=
 =?utf-8?B?aEVkWmRkVFBsWlZwQ1RGMXJ3OVRTUUZTKzl3VU1MaEYyd2FTdGVPSGxOQ0Nr?=
 =?utf-8?B?Qmt5Y3BuR2pTZERESGdmR0xSc2xUNlZ4U1FERWhiNHhqaVZ4SUd0UDBhcWl0?=
 =?utf-8?B?UHE0RE9kdjA3K25yNVJuQlVBdmFodTRleWFiQUhUOUVRVkJWVDlFdUU2VWtF?=
 =?utf-8?B?aGdaSmxZN1NHQXB4dnVUUGxod2FYb3NEa0RVRGhuOWVBNFpzTnkzd0wrRFdY?=
 =?utf-8?B?M3g2SUZrVUovZTlRdTBpeVU3V2FHYUhlTXcvQzZnNVNnblBuRXA5NnBMZUds?=
 =?utf-8?B?YjZiV0JVOGlnTXlTUUdyUG10M25PWmdCbUFYVjRjcVc2WHRwcUlVTUVxTEM1?=
 =?utf-8?B?RE5xa1JQTXJjTmdDaFc3dlMrRExoeUF3RTZhTWkwQXdxc2lzYUJyWVpEYzds?=
 =?utf-8?B?MUp4RFZFWkQ2dFh1dkw4Q1V3OGNYdWVvbzN0a3BZWVJuUzExRXl1SjlJbjhJ?=
 =?utf-8?B?VjdBbnFFMWd5cUNMZFNNNUVWbXpLbnY4blZMUEpTcXUwVEJMOG9JOXZWY2pD?=
 =?utf-8?B?VCtEaW9DWkg0ZzBzTXN3MC9UUlFweVJINVhGZXUzcithc1I5UG1RdzNla3da?=
 =?utf-8?B?cFJGWUFsVWZmUkZjaytJWTV4ZUVRNDNaV1FYYjJnZUlqUUZOWUFCekR6dDZk?=
 =?utf-8?B?dk1RdU96R0puVW5uZUFnZmc2UlY1N0dWdm0rclB2M1cvTHR5M2FNd3NKTlQ1?=
 =?utf-8?B?dkRFYjhHUzhiRHpybTI1blpDcmM2d2pVL2QrOHpWdGxsbjVvRkNXYVdsd2x6?=
 =?utf-8?B?ZlVXeHU3alJSTkZ0RmNIdmxaclg3Tm1BdGZBWm1oS2xDUWJ5Vm5FUGk1OWtD?=
 =?utf-8?B?UURwUHpISXptZkg4ekVCNHI5aUd3YnM3bnBCMmZlNjRiMWhsRVlwZVo1Mmlh?=
 =?utf-8?B?cjlKZkUzYnhWNjRsb0JPaXA2RWdUNDVqeWxya1ZzMmFtVzQvem1yU0gybld1?=
 =?utf-8?B?aDZ3cjN2SDdWeU1PY2xIcVdmc3F0d291ek5mL2NjdmlDTUtyWmMyMWhhdG8z?=
 =?utf-8?B?bXdLV0NaanRnTWNqU2ZDNG5SMUVWc2NIVDJkd2FGRUxiYzZvaFhKQVM5UG4w?=
 =?utf-8?B?Vit5UWJOa3kwcHJRbk5hNk9QSEdZN3hqUCtvazRjWTFQU2w1NkNldFUxT3p1?=
 =?utf-8?B?TDVMOUhtd0dlaDJxbXZSbXlFdzVnbGtJZys5TWJDZU9yTzh2dFk3VjRrK3Uy?=
 =?utf-8?B?bzVIa28rQURhVlZZZlBUYnZaUTVESTNVZzIwZUhVaW9wMzF4S2R0NnVNOUFs?=
 =?utf-8?B?TCt6WXNZL2kvckZqVStEam9pMkZ5MVZ4V2ljd1hqemVNWDl1Z1N0SVhQUWVK?=
 =?utf-8?B?VFovaGR5bjZhMmhrNXdzTkE5RGxzdjZNcGpySU1zNzBQMkU0b1JENXBFaDkz?=
 =?utf-8?B?Qnc5V1A4Z29JOUkya0p2MmdySXZyblQ4OEhFL2pYK0RTZlBRaGVXTTVsSnlX?=
 =?utf-8?B?emQwV004QWhDV3R4MlBZNHl2TWZ4bHRqWG9BMjk4L29xWkl3R1YzdFdRVGxs?=
 =?utf-8?B?MnVqU01rVk96bFhyQ2p5VG1QdGV4V0hCWmg0cTJKUElKZ0U4dWtIYTB2eFgr?=
 =?utf-8?B?YzVnYmptTjl2Q0pWcTdnRk1KdGJCUnEwcEJFUHRocWVHTkhNQXBLMElpRGV0?=
 =?utf-8?B?R0RWOVlDZndvb084YWVuSk5wY3VwVTJadVAwbEtXT08rSklRcTRYQmQ5aXI1?=
 =?utf-8?B?dVE9PQ==?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f52e2f1-eebe-435d-5dfb-08dc647417ef
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2024 15:34:56.2458
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tkE5orBfS7zeH+MPgYVjmkVh35jJRbiIqTw7+BB6szvUogMXNayDyMha5AIV6rSYzuUumrAE1Mhd2IgpIjkUnCZdpts3k3n7rxy6pHzjGXg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR17MB7134

On Wed, Apr 24, 2024 at 02:33:28PM +0800, Dongsheng Yang wrote:
> 
> 
> 在 2024/4/24 星期三 下午 12:29, Dan Williams 写道:
> > Dongsheng Yang wrote:
> > > From: Dongsheng Yang <dongsheng.yang.linux@gmail.com>
> > > 
> > > Hi all,
> > > 	This patchset introduce cbd (CXL block device). It's based on linux 6.8, and available at:
> > > 	https://github.com/DataTravelGuide/linux
> > > 
> > [..]
> > > (4) dax is not supported yet:
> > > 	same with famfs, dax device is not supported here, because dax device does not support
> > > dev_dax_iomap so far. Once dev_dax_iomap is supported, CBD can easily support DAX mode.
> > 
> > I am glad that famfs is mentioned here, it demonstrates you know about
> > it. However, unfortunately this cover letter does not offer any analysis
> > of *why* the Linux project should consider this additional approach to
> > the inter-host shared-memory enabling problem.
> > 
> > To be clear I am neutral at best on some of the initiatives around CXL
> > memory sharing vs pooling, but famfs at least jettisons block-devices
> > and gets closer to a purpose-built memory semantic.
> > 
> > So my primary question is why would Linux need both famfs and cbd? I am
> > sure famfs would love feedback and help vs developing competing efforts.
> 
> Hi,
> 	Thanks for your reply, IIUC about FAMfs, the data in famfs is stored in
> shared memory, and related nodes can share the data inside this file system;
> whereas cbd does not store data in shared memory, it uses shared memory as a
> channel for data transmission, and the actual data is stored in the backend
> block device of remote nodes. In cbd, shared memory works more like network
> to connect different hosts.
>

Couldn't you basically just allocate a file for use as a uni-directional
buffer on top of FAMFS and achieve the same thing without the need for
additional kernel support? Similar in a sense to allocating a file on
network storage and pinging the remote host when it's ready (except now
it's fast!)

(The point here is not "FAMFS is better" or "CBD is better", simply
trying to identify the function that will ultimately dictate the form).

~Gregory