by Darrick J. Wong

[permalink] [raw]

Subject: Re: [PATCH v2.2 1/8] fsdax: introduce page->share for fsdax in reflink mode

On Wed, Dec 07, 2022 at 02:49:19AM +0000, Shiyang Ruan wrote:
> fsdax page is used not only when CoW, but also mapread. To make the it
> easily understood, use 'share' to indicate that the dax page is shared
> by more than one extent. And add helper functions to use it.
>
> Also, the flag needs to be renamed to PAGE_MAPPING_DAX_SHARED.
>
> Signed-off-by: Shiyang Ruan <[email protected]>
> Reviewed-by: Allison Henderson <[email protected]>

Looks fine to me,
Reviewed-by: Darrick J. Wong <[email protected]>

--D

> ---
> fs/dax.c | 38 ++++++++++++++++++++++----------------
> include/linux/mm_types.h | 5 ++++-
> include/linux/page-flags.h | 2 +-
> 3 files changed, 27 insertions(+), 18 deletions(-)
>
> diff --git a/fs/dax.c b/fs/dax.c
> index 1c6867810cbd..84fadea08705 100644
> --- a/fs/dax.c
> +++ b/fs/dax.c
> @@ -334,35 +334,41 @@ static unsigned long dax_end_pfn(void *entry)
> for (pfn = dax_to_pfn(entry); \
> pfn < dax_end_pfn(entry); pfn++)
>
> -static inline bool dax_mapping_is_cow(struct address_space *mapping)
> +static inline bool dax_page_is_shared(struct page *page)
> {
> - return (unsigned long)mapping == PAGE_MAPPING_DAX_COW;
> + return page->mapping == PAGE_MAPPING_DAX_SHARED;
> }
>
> /*
> - * Set the page->mapping with FS_DAX_MAPPING_COW flag, increase the refcount.
> + * Set the page->mapping with PAGE_MAPPING_DAX_SHARED flag, increase the
> + * refcount.
> */
> -static inline void dax_mapping_set_cow(struct page *page)
> +static inline void dax_page_share_get(struct page *page)
> {
> - if ((uintptr_t)page->mapping != PAGE_MAPPING_DAX_COW) {
> + if (page->mapping != PAGE_MAPPING_DAX_SHARED) {
> /*
> * Reset the index if the page was already mapped
> * regularly before.
> */
> if (page->mapping)
> - page->index = 1;
> - page->mapping = (void *)PAGE_MAPPING_DAX_COW;
> + page->share = 1;
> + page->mapping = PAGE_MAPPING_DAX_SHARED;
> }
> - page->index++;
> + page->share++;
> +}
> +
> +static inline unsigned long dax_page_share_put(struct page *page)
> +{
> + return --page->share;
> }
>
> /*
> - * When it is called in dax_insert_entry(), the cow flag will indicate that
> + * When it is called in dax_insert_entry(), the shared flag will indicate that
> * whether this entry is shared by multiple files. If so, set the page->mapping
> - * FS_DAX_MAPPING_COW, and use page->index as refcount.
> + * PAGE_MAPPING_DAX_SHARED, and use page->share as refcount.
> */
> static void dax_associate_entry(void *entry, struct address_space *mapping,
> - struct vm_area_struct *vma, unsigned long address, bool cow)
> + struct vm_area_struct *vma, unsigned long address, bool shared)
> {
> unsigned long size = dax_entry_size(entry), pfn, index;
> int i = 0;
> @@ -374,8 +380,8 @@ static void dax_associate_entry(void *entry, struct address_space *mapping,
> for_each_mapped_pfn(entry, pfn) {
> struct page *page = pfn_to_page(pfn);
>
> - if (cow) {
> - dax_mapping_set_cow(page);
> + if (shared) {
> + dax_page_share_get(page);
> } else {
> WARN_ON_ONCE(page->mapping);
> page->mapping = mapping;
> @@ -396,9 +402,9 @@ static void dax_disassociate_entry(void *entry, struct address_space *mapping,
> struct page *page = pfn_to_page(pfn);
>
> WARN_ON_ONCE(trunc && page_ref_count(page) > 1);
> - if (dax_mapping_is_cow(page->mapping)) {
> - /* keep the CoW flag if this page is still shared */
> - if (page->index-- > 0)
> + if (dax_page_is_shared(page)) {
> + /* keep the shared flag if this page is still shared */
> + if (dax_page_share_put(page) > 0)
> continue;
> } else
> WARN_ON_ONCE(page->mapping && page->mapping != mapping);
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 500e536796ca..f46cac3657ad 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -103,7 +103,10 @@ struct page {
> };
> /* See page-flags.h for PAGE_MAPPING_FLAGS */
> struct address_space *mapping;
> - pgoff_t index; /* Our offset within mapping. */
> + union {
> + pgoff_t index; /* Our offset within mapping. */
> + unsigned long share; /* share count for fsdax */
> + };
> /**
> * @private: Mapping-private opaque data.
> * Usually used for buffer_heads if PagePrivate.
> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 0b0ae5084e60..d8e94f2f704a 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -641,7 +641,7 @@ PAGEFLAG_FALSE(VmemmapSelfHosted, vmemmap_self_hosted)
> * Different with flags above, this flag is used only for fsdax mode. It
> * indicates that this page->mapping is now under reflink case.
> */
> -#define PAGE_MAPPING_DAX_COW 0x1
> +#define PAGE_MAPPING_DAX_SHARED ((void *)0x1)
>
> static __always_inline bool folio_mapping_flags(struct folio *folio)
> {
> --
> 2.38.1
>

2022-12-29 08:46:03

by Shiyang Ruan

[permalink] [raw]

Subject: Re: [PATCH v2 0/8] fsdax,xfs: fix warning messages

在 2022/12/3 9:21, Dan Williams 写道:
> Shiyang Ruan wrote:
>> Changes since v1:
>> 1. Added a snippet of the warning message and some of the failed cases
>> 2. Separated the patch for easily review
>> 3. Added page->share and its helper functions
>> 4. Included the patch[1] that removes the restrictions of fsdax and reflink
>> [1] https://lore.kernel.org/linux-xfs/[email protected]/
>>
...
>>
>> This also effects dax+noreflink mode if we run the test after a
>> dax+reflink test. So, the most urgent thing is solving the warning
>> messages.
>>
>> With these fixes, most warning messages in dax_associate_entry() are
>> gone. But honestly, generic/388 will randomly failed with the warning.
>> The case shutdown the xfs when fsstress is running, and do it for many
>> times. I think the reason is that dax pages in use are not able to be
>> invalidated in time when fs is shutdown. The next time dax page to be
>> associated, it still remains the mapping value set last time. I'll keep
>> on solving it.
>
> This one also sounds like it is going to be relevant for CXL PMEM, and
> the improvements to the reference counting. CXL has a facility where the
> driver asserts that no more writes are in-flight to the device so that
> the device can assert a clean shutdown. Part of that will be making sure
> that page access ends at fs shutdown.

I was trying to locate the root cause of the fail on generic/388. But
since it's a fsstress test, I can't relpay the operation sequence to
help me locate the operations. So, I tried to replace fsstress with
fsx, which can do replay after the case fails, but it can't reproduce
the fail. I think another important factor is that fsstress tests with
multiple threads. So, for now, it's hard for me to locate the cause by
running the test.

Then I updated the kernel to the latest v6.2-rc1 and run generic/388 for
many times. The warning dmesg doesn't show any more.

How is your test on this case? Does it still fail on the latest kernel?
If so, I think I have to keep on locating the cause, and need your advice.

--
Thanks,
Ruan.

>
>> The warning message in dax_writeback_one() can also be fixed because of
>> the dax unshare.
>>
>>
>> Shiyang Ruan (8):
>> fsdax: introduce page->share for fsdax in reflink mode
>> fsdax: invalidate pages when CoW
>> fsdax: zero the edges if source is HOLE or UNWRITTEN
>> fsdax,xfs: set the shared flag when file extent is shared
>> fsdax: dedupe: iter two files at the same time
>> xfs: use dax ops for zero and truncate in fsdax mode
>> fsdax,xfs: port unshare to fsdax
>> xfs: remove restrictions for fsdax and reflink
>>
>> fs/dax.c | 220 +++++++++++++++++++++++++------------
>> fs/xfs/xfs_ioctl.c | 4 -
>> fs/xfs/xfs_iomap.c | 6 +-
>> fs/xfs/xfs_iops.c | 4 -
>> fs/xfs/xfs_reflink.c | 8 +-
>> include/linux/dax.h | 2 +
>> include/linux/mm_types.h | 5 +-
>> include/linux/page-flags.h | 2 +-
>> 8 files changed, 166 insertions(+), 85 deletions(-)
>>
>> --
>> 2.38.1
>>
>>
>
>