Received-SPF: pass (google.com: domain of linux-kernel+bounces-157441-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Wed, 24 Apr 2024 11:08:26 -0700
From: Dan Williams <dan.j.williams@intel.com>
To: Dongsheng Yang <dongsheng.yang@easystack.cn>, Dan Williams
	<dan.j.williams@intel.com>, <axboe@kernel.dk>, John Groves <John@groves.net>
CC: <linux-block@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-cxl@vger.kernel.org>, Dongsheng Yang <dongsheng.yang.linux@gmail.com>
Subject: Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)
Message-ID: <66294a9a8e9a8_b6e02947d@dwillia2-mobl3.amr.corp.intel.com.notmuch>
References: <20240422071606.52637-1-dongsheng.yang@easystack.cn>
 <66288ac38b770_a96f294c6@dwillia2-mobl3.amr.corp.intel.com.notmuch>
 <ef34808b-d25d-c953-3407-aa833ad58e61@easystack.cn>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ef34808b-d25d-c953-3407-aa833ad58e61@easystack.cn>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UW9MQjJzYlBuc3ZvK0RNSGFPcnprc3IwUTV0UTI4ckJYYUlGWmdoVXg0Vyto?=
 =?utf-8?B?djVSMDZ2MmVobGw2M0dWVDViWnIrTzZCL2ZBMkgwRkE3M1pQZWUrREVSbkNL?=
 =?utf-8?B?L2VxMWlTS2tBUnN2ZFdOZ050K1pNdHZLZGlka0Y2UWpuNXozZVFTeG12ZE9Y?=
 =?utf-8?B?dGM1cXZQSklycyt1L3M4M2JCK2dMZWd2MXFXenkwaFBFZ05VeVZjNnFUU2Vr?=
 =?utf-8?B?UmNsaWFsREVTUGJaZFp0aDVHM2pEZm9FZElkMVRTVEtia0VnNGF5bVhWUWRP?=
 =?utf-8?B?TVF1T0FPOVRjemZUNGZKdC9ScmFXdExTa3ExZEp4TmNvVDBSNUUzNmFTMkxO?=
 =?utf-8?B?WkdueFlMazRkZVR6dEIzOTc0a1ZHdEdyOXU5SjJTUnpkYnQyMmNhbU1HS3V1?=
 =?utf-8?B?MktyVG5WcTc1SVBEc1F6c3NnVlAycVpBMEgwMjdsTTVybnpZWFU3Z29vZ3RS?=
 =?utf-8?B?aTAzbGwva3h2WEhrU2s3WEFKM2tOc3liYjZkdDVJTE05U0RrMlBoUmNJakxR?=
 =?utf-8?B?TFJhMWtZWTBZcmF6S3U2cjZSR1RMemI5d1V3eEdRSkNJbjRpMTNLVlJ5eXhF?=
 =?utf-8?B?ZjBBUmo4clJrd2FXV2hsUjlhR1psWUFZZFBpazZYNmdORmdxVDF1amZiZCty?=
 =?utf-8?B?RHM3VjI4M2ZrMlRLbVBUWUV5M0RpeWlZSitrZGIxY2YyTmVSdnFSRDlMQUp4?=
 =?utf-8?B?Tmh2MEQ2TDM3Sm1iZDlUK29SVkJCeDY5bFRzOWJscTNEbXMvd2Vld2w0QUxC?=
 =?utf-8?B?a2w5aVB0b3VlVUUwaFFWRm4wZHRoRk9sVzRSd0kxS0VrMjNvUVNKd0Z4dGkw?=
 =?utf-8?B?SGZ6TEFsWS9oUlQrMHZaOC9lVzhPdTQxSlUyVkJDa2RMdmE1OGNXdnBscXBx?=
 =?utf-8?B?eDE4Y0JCVW8xayt2MDcyR0NuZHE0Q1I2OWNMMDBjWkhPcWdUb1gwVStJTVl1?=
 =?utf-8?B?TFVvUFNQb3Z3WUN3MGl3M0I4TzRQUHFkUUlEeEF1NXNWSjl5ZmxpQ29pbDNU?=
 =?utf-8?B?TVlMd1F5USt1TndLUWFaT1ZyVjYxWTlpNllZNUtDQjRaL1oxU0l0bEJIeHRV?=
 =?utf-8?B?Rm1PWkg0SlFmYklyOHRvM3cyaGxNSHk5MnRSaGhDNzJ3Q1hnNnhHazFFMTdn?=
 =?utf-8?B?eStyTjZnM0gycVZIL21tN3V3QjFubnZ5RzBMM1BtRXhxYjNjR1dQbnJDUFBi?=
 =?utf-8?B?L3d5Q0xTKzZSRVZ2cW9WU3dzZGtSN09sc3VYdzN0d2liOW02NFNYMG03Z3FB?=
 =?utf-8?B?bjFQUVFnc1ludWNhTllObm0veVMyTE9ZMTJySmg3YjBmZ0ZBV0VwZFB4djNO?=
 =?utf-8?B?dkl3OEU5MXA3b0NUQ0FUYjJSVGs4d0VyNi8wUFRGaU5VRjZ0K2VqT3VBaUFr?=
 =?utf-8?B?QyswcGt0emE4eElFczZITVl1Um12Y1ppWFJYdnRmbEJOZ295bnBkZXJqdDc4?=
 =?utf-8?B?TXV3ajRpT0k1dGNuUEVIbkxwK0JIL09RN0QyTzVzc1NqeUlpRTllSTR3bWVh?=
 =?utf-8?B?d2N1MVpQRVhrVVRSOFdPblcvTGQvZ1AzRmJ1eHJMSUtORDVnT3B6bXFldERs?=
 =?utf-8?B?VEZ5TGVqaTE3VlBQVStSY2Y5cmRYRjlwZ0VyR1F6UVZSRE5wRlFmQTFVZTJl?=
 =?utf-8?B?TTQwYzhjQ3l0R25tdmlhWU9SdWJqdDdlUzNlYkZLdGJmY2drTTg2R3JVS3BZ?=
 =?utf-8?B?MjYzckxpVURwc1NXa0gwMU1TclJGZlBJc1g2YjRKL215bVNuVnZxMFRGVkdO?=
 =?utf-8?B?WjFORWxjVklrKzBaVVBNakR0dlNRc3BTWHdTVzRyZVhiQWJBeTlGU2FOWVVu?=
 =?utf-8?B?QVF5ZzVQbUFIbkV6eDFHSWx2N0tXaU5RcDRlUHdoTTBSdFJTZS8wMXZ5TUVD?=
 =?utf-8?B?bEhWSSs1TlREUDhsSTZkMnRTaXh6TGhyditnSGhWVEZwc2pybVN1WDdPSVhV?=
 =?utf-8?B?VitVQWdEYXNWaS9jdS9jT1dtZWlkY0NqU29XL1BIWjVsTnRld1NsQUJPWGdV?=
 =?utf-8?B?QURZUzFkYXJvQ1hZV2xvdFZPajBaS2wrNmpXRXBoc1N5NFVXR1hQbDZMdk9l?=
 =?utf-8?B?dHdlMFZLajFOWXhjQ2J0TE1WcFUvdlNCT3dkOER3Tk8yYXBFSWhROEs2OGJt?=
 =?utf-8?B?eVNtVkdlb3hyNEp5dVpGbCtxNGFRR25Ibk15cW81QU5kOVc5VjZUUGtzM1o2?=
 =?utf-8?B?SGc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 04d2bb8c-7e24-4b02-070e-08dc64898b38
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2024 18:08:29.0051
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3meHLN5zRdagjfbRnsAszvJDhUcJRt4EBh0ZCxObPHfmCMuilDW9nSzyLO2ZvH+Zsu6oiv5Cv2bPOWH33xnTSgHEC0KZ/kfXvOX1hOxWAD0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB7428
X-OriginatorOrg: intel.com

Dongsheng Yang wrote:
> 
> 
> 在 2024/4/24 星期三 下午 12:29, Dan Williams 写道:
> > Dongsheng Yang wrote:
> >> From: Dongsheng Yang <dongsheng.yang.linux@gmail.com>
> >>
> >> Hi all,
> >> 	This patchset introduce cbd (CXL block device). It's based on linux 6.8, and available at:
> >> 	https://github.com/DataTravelGuide/linux
> >>
> > [..]
> >> (4) dax is not supported yet:
> >> 	same with famfs, dax device is not supported here, because dax device does not support
> >> dev_dax_iomap so far. Once dev_dax_iomap is supported, CBD can easily support DAX mode.
> > 
> > I am glad that famfs is mentioned here, it demonstrates you know about
> > it. However, unfortunately this cover letter does not offer any analysis
> > of *why* the Linux project should consider this additional approach to
> > the inter-host shared-memory enabling problem.
> > 
> > To be clear I am neutral at best on some of the initiatives around CXL
> > memory sharing vs pooling, but famfs at least jettisons block-devices
> > and gets closer to a purpose-built memory semantic.
> > 
> > So my primary question is why would Linux need both famfs and cbd? I am
> > sure famfs would love feedback and help vs developing competing efforts.
> 
> Hi,
> 	Thanks for your reply, IIUC about FAMfs, the data in famfs is stored in 
> shared memory, and related nodes can share the data inside this file 
> system; whereas cbd does not store data in shared memory, it uses shared 
> memory as a channel for data transmission, and the actual data is stored 
> in the backend block device of remote nodes. In cbd, shared memory works 
> more like network to connect different hosts.
> 
> That is to say, in my view, FAMfs and cbd do not conflict at all; they 
> meet different scenario requirements. cbd simply uses shared memory to 
> transmit data, shared memory plays the role of a data transmission 
> channel, while in FAMfs, shared memory serves as a data store role.

If shared memory is just a communication transport then a block-device
abstraction does not seem a proper fit. From the above description this
sounds similar to what CONFIG_NTB_TRANSPORT offers which is a way for
two hosts to communicate over a shared memory channel.

So, I am not really looking for an analysis of famfs vs CBD I am looking
for CBD to clarify why Linux should consider it, and why the
architecture is fit for purpose.