Received-SPF: pass (google.com: domain of linux-kernel+bounces-196969-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 31 May 2024 10:23:47 -0400
From: Gregory Price <gregory.price@memverge.com>
To: Dongsheng Yang <dongsheng.yang@easystack.cn>
Cc: Dan Williams <dan.j.williams@intel.com>,
	Jonathan Cameron <Jonathan.Cameron@huawei.com>,
	John Groves <John@groves.net>, axboe@kernel.dk,
	linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev
Subject: Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)
Message-ID: <Zlndc8NI0eK3MmuR@memverge.com>
References: <20240508131125.00003d2b@Huawei.com>
 <ef0ee621-a2d2-e59a-f601-e072e8790f06@easystack.cn>
 <20240508164417.00006c69@Huawei.com>
 <3d547577-e8f2-8765-0f63-07d1700fcefc@easystack.cn>
 <20240509132134.00000ae9@Huawei.com>
 <a571be12-2fd3-e0ee-a914-0a6e2c46bdbc@easystack.cn>
 <664cead8eb0b6_add32947d@dwillia2-mobl3.amr.corp.intel.com.notmuch>
 <8f161b2d-eacd-ad35-8959-0f44c8d132b3@easystack.cn>
 <ZldIzp0ncsRX5BZE@memverge.com>
 <5db870de-ecb3-f127-f31c-b59443b4fbb4@easystack.cn>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <5db870de-ecb3-f127-f31c-b59443b4fbb4@easystack.cn>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Q0NVL2JBbU83Q0o5V2Y0M2tKMlcwREs0dW1rZFROcGZXTVU0dlJycDdDU0Ru?=
 =?utf-8?B?TXY4Y1VDNjltSTV6M0dYYmhDWk9zeVp1ZGRwb3RPc0lTd3ZrSjl4dTY2Y29N?=
 =?utf-8?B?c2FCS2xwMTRkTW5GbVMwR2NSVExsK2drdFB5NFFRRG85S3dsUUNXQXBVUFlJ?=
 =?utf-8?B?c0xLakd0SDRTMGtJWVpLT2YxbDBSUzNCTVNOQ2o5UzFzS2NBMTdpazJCMUMz?=
 =?utf-8?B?aTcyalpMc3ltLzJ6aXB1M0VvVWFFVitnS2l0Mm5LK1ZoQlpXNEkzSkptd3pO?=
 =?utf-8?B?ODJLSzhNWnJmT05KQloxWXFLSGNOWnllUzUzOFo5SHNOSXF2QWE5U3FiYXp4?=
 =?utf-8?B?dmE3WDFlcS9talh4SEoyT3U0bm9IUUJwaTNtWUY2Y2FaSGZkM2J2ektyeDEy?=
 =?utf-8?B?STdrV1F2Q2gzVzJvMHRWTVNxYVJyN0VrSEFYa0I5WDRLRmgyUWQ0c1IrdUxk?=
 =?utf-8?B?QjZNYjhteVQyWm15UGpHMUpwUlA1SE4zYUN3VC9FdjJ6YUgwaENLbW9WYUE1?=
 =?utf-8?B?S3VFT2JmL082N0lXNFJvTjZ5ZUhVRkkzUzJEZmVWNXZMbW5BZ3A4N09xTmdM?=
 =?utf-8?B?SVkyR2ZLbUliaW4vL0hsK0NtTTQyK0dwRmhCMG8ySHdUeGdsSTZ3UklHMnhO?=
 =?utf-8?B?Q1FKZ2hQbndDQnlOVXBLL1lSckNVVlpUaFZoeFVqTHVDdndCTExNdHdPWXZC?=
 =?utf-8?B?ZmhHd0RnY3VyazRGbjBtL0wwQitLNmNNV1JBTTFZNlVyOVh4WXNVK0ZQeXho?=
 =?utf-8?B?aXV1Yy9vRi9EV2dUUGh1Vm1GU3FMbkZWMENKUkpEejhIaXlaOGhXWmtIV0Uv?=
 =?utf-8?B?c1lnVWVVQ2ZURXgwTk1DVGpuTUd0SkZ3bTJ0NFZ6RjBSeVZmODZEejJ1WlVK?=
 =?utf-8?B?MG1sUVBBLzdJRzJNVkdjZG9ydTEyVGNDK0hERzJlRDN5NkU1VmxnSFlVWk9C?=
 =?utf-8?B?Ty82Mlh5N3FubERLWGFKM2pMMVpHOFBlRnpzVnoybzZrZUp5K1gvMDRId3Ji?=
 =?utf-8?B?UlphTWttTTNSLy80MmhnckthSm9EOUVGTWtzcnJyVnBNaUNvc3FQNG9Od2Mz?=
 =?utf-8?B?dVgwLytqNkdIYUlIRHpocnBpZzBKRmxqbXk0YUlrMzI3NndidEc0MEFSM2Rv?=
 =?utf-8?B?SnE5Uit6dDdaODEvemdYL3Z2TjhTdGJYL2hudEIxMGlSclFOcTNTREpmN1M4?=
 =?utf-8?B?dWJONmVBWDJnd0hlSU56RFFhRENSUU1NaGlncXVINzJ2MnBRamZJMzcyeHdp?=
 =?utf-8?B?bVNxTWNWbTdEQ1dhOWREaGxFSG9OeXlRc09aUzdwZUF5L1VaaGxFcUU2Z1Zt?=
 =?utf-8?B?SHRzbW9NOWk1RzdKQnJ5RGUyU2FxNFR0VHZPSjluR3dtQk56ZGxlWjUxdVox?=
 =?utf-8?B?VWhCQldKUUFXTTFhRTlJZHFjc0pXTEl2ZmJ5d1Y1ZTJYeWtnaEpHYTlTOXJT?=
 =?utf-8?B?LzVSQ1M3OWJ1K1BtOHNld2lwN0VKNTJnQVdsYmdYM0orVUZYc09EeDdFUDJy?=
 =?utf-8?B?M3ZRYnZlY2RYOVBHckhaeUF2dEN0TzJTNytJa0d4REpxUmM0c1VleGd6Rld1?=
 =?utf-8?B?RUdvblJOSWRyWGdqTkp1dWg1QkFHK0FiWGFuWnYyanZvbHhLYU4vOVplSExD?=
 =?utf-8?B?bFU5cy80azAyVVFCd0p3SnBoeW42M2JKV0JCNjVpY2F2RWwvWGtKL3dSYlNL?=
 =?utf-8?B?NXptbjdNMDdVK1ZySVduVmNpRzVjRlNGL1NaWVZoWW9laUtobG1HOHF6dnlU?=
 =?utf-8?B?djF3Vm5OYVVtRGltTW1UUUE4UlhVZGp1L0FoeXI0TEhzZnIwUVM4WStWSzR3?=
 =?utf-8?B?M0pTUk45Y1VpdUtnTDVVMnE3QTA2T1Jib1JlenB2VFJvYldCSmFxSGtpTVpJ?=
 =?utf-8?B?WUFhbUg1VnlkTmtmcHRYaTZ6Szg0azNrdjZFaGNHc1BEeGFkajJubFRoU3F6?=
 =?utf-8?B?L29KQWV3dE5TZFJUZFpVWUpsOGx5SDJXQjBPZDNjbGZ4ZFU1QmhEL1gydGhm?=
 =?utf-8?B?NEE4Sm5xTU43NklIS3dHRkZ3dkErZ0lZZW1DdENuUm5XWDh1WWk3UFI0bk1Y?=
 =?utf-8?B?Zms2UDdlOGNhR2p5RlZqcS9EMk94b1hMZkN6SnY3RFFtVTlWLy9rcUNrWHZB?=
 =?utf-8?B?TlJhTTBoNWNiajA3dmpSWklBY1VMMmsvcVBkbXhyRXNraCtndnFnc3NEWkJk?=
 =?utf-8?B?Z2c9PQ==?=
X-OriginatorOrg: memverge.com
X-MS-Exchange-CrossTenant-Network-Message-Id: af8599d2-1dcc-4765-623f-08dc817d4b08
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 May 2024 14:23:51.1316
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +U6Cg99HFHH8M1MbVofZginPuV42yn52sXdWcjrWN0Vb9SAZO1v+ne96OsbS4B99RhLIssqcLWukQHNo3xDSSEVeBFL5FWPmTK8a76zgbgc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR17MB6666

On Thu, May 30, 2024 at 02:59:38PM +0800, Dongsheng Yang wrote:
> 
> 
> 在 2024/5/29 星期三 下午 11:25, Gregory Price 写道:
> > 
> > There are some (FAMFS, for example). The coherence state of these
> > systems tend to be less volatile (e.g. mappings are read-only), or
> > they have inherent design limitations (cacheline-sized message passing
> > via write-ahead logging only).
> 
> Can you explain more about this? I understand that if the reader in the
> writer-reader model is using a readonly mapping, the interaction will be
> much simpler. However, after the writer writes data, if we don't have a
> mechanism to flush and invalidate puncturing all caches, how can the
> readonly reader access the new data?

This is exactly right, so the coherence/correctness of the data needs to
be enforced in some other way.

Generally speaking, the WPQs will *eventually* get flushed.  As such,
the memory will *eventually* become coherent.  So if you set up the
following pattern, you will end up with an "eventually coherent" system

1) Writer instantiates the memory to be used
2) Writer calculates and records a checksum of that data into memory
3) Writer invalidates everything
4) Reader maps the memory
5) Reader reads the checksum and calculates the checksum of the data
   a) if the checksums match, the data is coherent
   b) if they don't, we must wait longer for the queues to flush

This is just one example of a system design which enforces coherence by
placing the limitation on the system that the data will never change
once it becomes coherent.

Whatever the case, regardless of the scheme you come up with, you will
end up with a system where the data must be inspected and validated
before it can be used.  This has the limiting factor of performance:
throughput will be limited by how fast you can validate the data.

~Gregory