[permalink] [raw]

Subject: Re: [PATCH v7 4/8] pagemap,pmem: Introduce ->memory_failure()

Except for the error code inversion noticed by Darrick this looks fine
to me:

Reviewed-by: Christoph Hellwig <[email protected]>

2021-10-15 13:03:36

by Christoph Hellwig

[permalink] [raw]

Subject: Re: [PATCH v7 7/8] xfs: Implement ->notify_failure() for XFS

On Fri, Sep 24, 2021 at 09:09:58PM +0800, Shiyang Ruan wrote:
> +void fs_dax_register_holder(struct dax_device *dax_dev, void *holder,
> + const struct dax_holder_operations *ops)
> +{
> + dax_set_holder(dax_dev, holder, ops);
> +}
> +EXPORT_SYMBOL_GPL(fs_dax_register_holder);
> +
> +void fs_dax_unregister_holder(struct dax_device *dax_dev)
> +{
> + dax_set_holder(dax_dev, NULL, NULL);
> +}
> +EXPORT_SYMBOL_GPL(fs_dax_unregister_holder);
> +
> +void *fs_dax_get_holder(struct dax_device *dax_dev)
> +{
> + return dax_get_holder(dax_dev);
> +}
> +EXPORT_SYMBOL_GPL(fs_dax_get_holder);

These should not be in a XFS patch. But why do we even need this
wrappers?

> @@ -377,6 +385,8 @@ xfs_close_devices(
>
> xfs_free_buftarg(mp->m_logdev_targp);
> xfs_blkdev_put(logdev);
> + if (dax_logdev)
> + fs_dax_unregister_holder(dax_logdev);
> fs_put_dax(dax_logdev);

I'd prefer to include the fs_dax_unregister_holder in the fs_put_dax
call to avoid callers failing to unregister it.

> @@ -411,6 +425,9 @@ xfs_open_devices(
> struct block_device *logdev = NULL, *rtdev = NULL;
> int error;
>
> + if (dax_ddev)
> + fs_dax_register_holder(dax_ddev, mp,
> + &xfs_dax_holder_operations);

I'd include the holder registration with fs_dax_get_by_bdev as well.

2021-10-15 13:04:03

by Christoph Hellwig

[permalink] [raw]

Subject: Re: [PATCH v7 8/8] fsdax: add exception for reflinked files

On Thu, Oct 14, 2021 at 12:24:50PM -0700, Darrick J. Wong wrote:
> It feels a little dangerous to have page->mapping for shared storage
> point to an actual address_space when there are really multiple
> potential address_spaces out there. If the mm or dax folks are ok with
> doing this this way then I'll live with it, but it seems like you'd want
> to leave /some/ kind of marker once you know that the page has multiple
> owners and therefore regular mm rmap via page->mapping won't work.

Yes, I thing poisoning page->mapping for the rmap enabled case seems
like a better idea.

2021-10-20 05:28:33

by Shiyang Ruan

[permalink] [raw]

Subject: Re: [PATCH v7 4/8] pagemap,pmem: Introduce ->memory_failure()

在 2021/10/15 2:05, Darrick J. Wong 写道:
> On Fri, Sep 24, 2021 at 09:09:55PM +0800, Shiyang Ruan wrote:
>> When memory-failure occurs, we call this function which is implemented
>> by each kind of devices. For the fsdax case, pmem device driver
>> implements it. Pmem device driver will find out the filesystem in which
>> the corrupted page located in.
>>
>> With dax_holder notify support, we are able to notify the memory failure
>> from pmem driver to upper layers. If there is something not support in
>> the notify routine, memory_failure will fall back to the generic hanlder.
>>
>> Signed-off-by: Shiyang Ruan <[email protected]>
>> ---
>> drivers/nvdimm/pmem.c | 11 +++++++++++
>> include/linux/memremap.h | 9 +++++++++
>> mm/memory-failure.c | 14 ++++++++++++++
>> 3 files changed, 34 insertions(+)
>>
>> diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
>> index 72de88ff0d30..0dfafad8fcc5 100644
>> --- a/drivers/nvdimm/pmem.c
>> +++ b/drivers/nvdimm/pmem.c
>> @@ -362,9 +362,20 @@ static void pmem_release_disk(void *__pmem)
>> del_gendisk(pmem->disk);
>> }
>>
>> +static int pmem_pagemap_memory_failure(struct dev_pagemap *pgmap,
>> + unsigned long pfn, size_t size, int flags)
>> +{
>> + struct pmem_device *pmem =
>> + container_of(pgmap, struct pmem_device, pgmap);
>> + loff_t offset = PFN_PHYS(pfn) - pmem->phys_addr - pmem->data_offset;
>> +
>> + return dax_holder_notify_failure(pmem->dax_dev, offset, size, flags);
>> +}
>> +
>> static const struct dev_pagemap_ops fsdax_pagemap_ops = {
>> .kill = pmem_pagemap_kill,
>> .cleanup = pmem_pagemap_cleanup,
>> + .memory_failure = pmem_pagemap_memory_failure,
>> };
>>
>> static int pmem_attach_disk(struct device *dev,
>> diff --git a/include/linux/memremap.h b/include/linux/memremap.h
>> index c0e9d35889e8..36d47bacd46d 100644
>> --- a/include/linux/memremap.h
>> +++ b/include/linux/memremap.h
>> @@ -87,6 +87,15 @@ struct dev_pagemap_ops {
>> * the page back to a CPU accessible page.
>> */
>> vm_fault_t (*migrate_to_ram)(struct vm_fault *vmf);
>> +
>> + /*
>> + * Handle the memory failure happens on a range of pfns. Notify the
>> + * processes who are using these pfns, and try to recover the data on
>> + * them if necessary. The flag is finally passed to the recover
>> + * function through the whole notify routine.
>> + */
>> + int (*memory_failure)(struct dev_pagemap *pgmap, unsigned long pfn,
>> + size_t size, int flags);
>> };
>>
>> #define PGMAP_ALTMAP_VALID (1 << 0)
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 8ff9b52823c0..85eab206b68f 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -1605,6 +1605,20 @@ static int memory_failure_dev_pagemap(unsigned long pfn, int flags,
>> if (!pgmap_pfn_valid(pgmap, pfn))
>> goto out;
>>
>> + /*
>> + * Call driver's implementation to handle the memory failure, otherwise
>> + * fall back to generic handler.
>> + */
>> + if (pgmap->ops->memory_failure) {
>> + rc = pgmap->ops->memory_failure(pgmap, pfn, PAGE_SIZE, flags);
>> + /*
>> + * Fall back to generic handler too if operation is not
>> + * supported inside the driver/device/filesystem.
>> + */
>> + if (rc != EOPNOTSUPP)
>
> -EOPNOTSUPP? (negative errno)

Yes, my mistake. Thanks for pointing out.

--
Thanks,
Ruan.

>
> --D
>
>> + goto out;
>> + }
>> +
>> rc = mf_generic_kill_procs(pfn, flags, pgmap);
>> out:
>> /* drop pgmap ref acquired in caller */
>> --
>> 2.33.0
>>
>>
>>

2021-10-20 05:50:08

by Shiyang Ruan

[permalink] [raw]

Subject: Re: [PATCH v7 6/8] mm: Introduce mf_dax_kill_procs() for fsdax case

在 2021/10/15 3:32, Darrick J. Wong 写道:
> On Fri, Sep 24, 2021 at 09:09:57PM +0800, Shiyang Ruan wrote:
>> This function is called at the end of RMAP routine, i.e. filesystem
>> recovery function, to collect and kill processes using a shared page of
>> DAX file. The difference between mf_generic_kill_procs() is,
>> it accepts file's mapping,offset instead of struct page. Because
>> different file's mappings and offsets may share the same page in fsdax
>> mode. So, it is called when filesystem RMAP results are found.
>>
>> Signed-off-by: Shiyang Ruan <[email protected]>
>> ---
>> fs/dax.c | 10 ------
>> include/linux/dax.h | 9 +++++
>> include/linux/mm.h | 2 ++
>> mm/memory-failure.c | 83 ++++++++++++++++++++++++++++++++++++++++-----
>> 4 files changed, 86 insertions(+), 18 deletions(-)
>>
>> diff --git a/fs/dax.c b/fs/dax.c
>> index 509b65e60478..2536c105ec7f 100644
>> --- a/fs/dax.c
>> +++ b/fs/dax.c
>> @@ -852,16 +852,6 @@ static void *dax_insert_entry(struct xa_state *xas,
>> return entry;
>> }
>>
>> -static inline
>> -unsigned long pgoff_address(pgoff_t pgoff, struct vm_area_struct *vma)
>> -{
>> - unsigned long address;
>> -
>> - address = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT);
>> - VM_BUG_ON_VMA(address < vma->vm_start || address >= vma->vm_end, vma);
>> - return address;
>> -}
>> -
>> /* Walk all mappings of a given index of a file and writeprotect them */
>> static void dax_entry_mkclean(struct address_space *mapping, pgoff_t index,
>> unsigned long pfn)
>> diff --git a/include/linux/dax.h b/include/linux/dax.h
>> index 65411bee4312..3d90becbd160 100644
>> --- a/include/linux/dax.h
>> +++ b/include/linux/dax.h
>> @@ -258,6 +258,15 @@ static inline bool dax_mapping(struct address_space *mapping)
>> {
>> return mapping->host && IS_DAX(mapping->host);
>> }
>> +static inline unsigned long pgoff_address(pgoff_t pgoff,
>> + struct vm_area_struct *vma)
>> +{
>> + unsigned long address;
>> +
>> + address = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT);
>> + VM_BUG_ON_VMA(address < vma->vm_start || address >= vma->vm_end, vma);
>> + return address;
>> +}
>>
>> #ifdef CONFIG_DEV_DAX_HMEM_DEVICES
>> void hmem_register_device(int target_nid, struct resource *r);
>> diff --git a/include/linux/mm.h b/include/linux/mm.h
>> index 73a52aba448f..d06af0051e53 100644
>> --- a/include/linux/mm.h
>> +++ b/include/linux/mm.h
>> @@ -3114,6 +3114,8 @@ enum mf_flags {
>> MF_MUST_KILL = 1 << 2,
>> MF_SOFT_OFFLINE = 1 << 3,
>> };
>> +extern int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index,
>> + size_t size, int flags);
>> extern int memory_failure(unsigned long pfn, int flags);
>> extern void memory_failure_queue(unsigned long pfn, int flags);
>> extern void memory_failure_queue_kick(int cpu);
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 85eab206b68f..a9d0d487d205 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -302,10 +302,9 @@ void shake_page(struct page *p)
>> }
>> EXPORT_SYMBOL_GPL(shake_page);
>>
>> -static unsigned long dev_pagemap_mapping_shift(struct page *page,
>> +static unsigned long dev_pagemap_mapping_shift(unsigned long address,
>> struct vm_area_struct *vma)
>> {
>> - unsigned long address = vma_address(page, vma);
>> pgd_t *pgd;
>> p4d_t *p4d;
>> pud_t *pud;
>> @@ -345,7 +344,7 @@ static unsigned long dev_pagemap_mapping_shift(struct page *page,
>> * Schedule a process for later kill.
>> * Uses GFP_ATOMIC allocations to avoid potential recursions in the VM.
>> */
>> -static void add_to_kill(struct task_struct *tsk, struct page *p,
>> +static void add_to_kill(struct task_struct *tsk, struct page *p, pgoff_t pgoff,
>
> Hm, so I guess you're passing the page and the pgoff now because
> page->index is meaningless for shared dax pages? Ok.

Yes, it is for that case.

>
>> struct vm_area_struct *vma,
>> struct list_head *to_kill)
>> {
>> @@ -358,9 +357,15 @@ static void add_to_kill(struct task_struct *tsk, struct page *p,
>> }
>>
>> tk->addr = page_address_in_vma(p, vma);
>> - if (is_zone_device_page(p))
>> - tk->size_shift = dev_pagemap_mapping_shift(p, vma);
>> - else
>> + if (is_zone_device_page(p)) {
>> + /*
>> + * Since page->mapping is no more used for fsdax, we should
>> + * calculate the address in a fsdax way.
>> + */
>> + if (p->pgmap->type == MEMORY_DEVICE_FS_DAX)
>> + tk->addr = pgoff_address(pgoff, vma);
>> + tk->size_shift = dev_pagemap_mapping_shift(tk->addr, vma);
>> + } else
>> tk->size_shift = page_shift(compound_head(p));
>>
>> /*
>> @@ -508,7 +513,7 @@ static void collect_procs_anon(struct page *page, struct list_head *to_kill,
>> if (!page_mapped_in_vma(page, vma))
>> continue;
>> if (vma->vm_mm == t->mm)
>> - add_to_kill(t, page, vma, to_kill);
>> + add_to_kill(t, page, 0, vma, to_kill);
>> }
>> }
>> read_unlock(&tasklist_lock);
>> @@ -544,7 +549,32 @@ static void collect_procs_file(struct page *page, struct list_head *to_kill,
>> * to be informed of all such data corruptions.
>> */
>> if (vma->vm_mm == t->mm)
>> - add_to_kill(t, page, vma, to_kill);
>> + add_to_kill(t, page, 0, vma, to_kill);
>> + }
>> + }
>> + read_unlock(&tasklist_lock);
>> + i_mmap_unlock_read(mapping);
>> +}
>> +
>> +/*
>> + * Collect processes when the error hit a fsdax page.
>> + */
>> +static void collect_procs_fsdax(struct page *page, struct address_space *mapping,
>> + pgoff_t pgoff, struct list_head *to_kill)
>> +{
>> + struct vm_area_struct *vma;
>> + struct task_struct *tsk;
>> +
>> + i_mmap_lock_read(mapping);
>> + read_lock(&tasklist_lock);
>> + for_each_process(tsk) {
>> + struct task_struct *t = task_early_kill(tsk, true);
>> +
>> + if (!t)
>> + continue;
>> + vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) {
>> + if (vma->vm_mm == t->mm)
>> + add_to_kill(t, page, pgoff, vma, to_kill);
>> }
>> }
>> read_unlock(&tasklist_lock);
>> @@ -1503,6 +1533,43 @@ static int mf_generic_kill_procs(unsigned long long pfn, int flags,
>> return 0;
>> }
>>
>> +/**
>> + * mf_dax_kill_procs - Collect and kill processes who are using this file range
>> + * @mapping: the file in use
>> + * @index: start offset of the range
>> + * @size: length of the range
>
> It feels odd that one argument is in units of pgoff_t but the other is
> in bytes.

The index is page aligned but @size may not be. I will explain it in
detail in the comments.

>
>> + * @flags: memory failure flags
>> + */
>> +int mf_dax_kill_procs(struct address_space *mapping, pgoff_t index,
>> + size_t size, int flags)
>> +{
>> + LIST_HEAD(to_kill);
>> + dax_entry_t cookie;
>> + struct page *page;
>> + size_t end = (index << PAGE_SHIFT) + size;
>> +
>> + flags |= MF_ACTION_REQUIRED | MF_MUST_KILL;
>
> Hm. What flags will we be passing to the xfs_dax_notify_failure_fn?
> Does XFS itself have to care about what's in the flags values, or is it
> really just a magic cookie to be passed from the mm layer into the fs
> and back to mf_dax_kill_procs?
>

Just to pass the flag from mm layer to mf_dax_kill_procs(). No one
inside this RMAP progress will care about or change it. As you
mentioned in the next patch, I think this should be named with a "mf_"
prefix to make it easier to understand.

--
Thanks,
Ruan.

> --D
>
>> +
>> + for (; (index << PAGE_SHIFT) < end; index++) {
>> + page = NULL;
>> + cookie = dax_lock_mapping_entry(mapping, index, &page);
>> + if (!cookie)
>> + return -EBUSY;
>> + if (!page)
>> + goto unlock;
>> +
>> + SetPageHWPoison(page);
>> +
>> + collect_procs_fsdax(page, mapping, index, &to_kill);
>> + unmap_and_kill(&to_kill, page_to_pfn(page), mapping,
>> + index, flags);
>> +unlock:
>> + dax_unlock_mapping_entry(mapping, index, cookie);
>> + }
>> + return 0;
>> +}
>> +EXPORT_SYMBOL_GPL(mf_dax_kill_procs);
>> +
>> static int memory_failure_hugetlb(unsigned long pfn, int flags)
>> {
>> struct page *p = pfn_to_page(pfn);
>> --
>> 2.33.0
>>
>>
>>

2021-10-20 07:02:24

by Shiyang Ruan

[permalink] [raw]

Subject: Re: [PATCH v7 2/8] dax: Introduce holder for dax_device

在 2021/10/15 2:00, Darrick J. Wong 写道:
> On Fri, Sep 24, 2021 at 09:09:53PM +0800, Shiyang Ruan wrote:
>> To easily track filesystem from a pmem device, we introduce a holder for
>> dax_device structure, and also its operation. This holder is used to
>> remember who is using this dax_device:
>> - When it is the backend of a filesystem, the holder will be the
>> superblock of this filesystem.
>> - When this pmem device is one of the targets in a mapped device, the
>> holder will be this mapped device. In this case, the mapped device
>> has its own dax_device and it will follow the first rule. So that we
>> can finally track to the filesystem we needed.
>>
>> The holder and holder_ops will be set when filesystem is being mounted,
>> or an target device is being activated.
>>
>> Signed-off-by: Shiyang Ruan <[email protected]>
>> ---
>> drivers/dax/super.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
>> include/linux/dax.h | 29 ++++++++++++++++++++++
>> 2 files changed, 88 insertions(+)
>>
>> diff --git a/drivers/dax/super.c b/drivers/dax/super.c
>> index 48ce86501d93..7d4a11dcba90 100644
>> --- a/drivers/dax/super.c
>> +++ b/drivers/dax/super.c
>> @@ -23,7 +23,10 @@
>> * @cdev: optional character interface for "device dax"
>> * @host: optional name for lookups where the device path is not available
>> * @private: dax driver private data
>> + * @holder_data: holder of a dax_device: could be filesystem or mapped device
>> * @flags: state and boolean properties
>> + * @ops: operations for dax_device
>> + * @holder_ops: operations for the inner holder
>> */
>> struct dax_device {
>> struct hlist_node list;
>> @@ -31,8 +34,10 @@ struct dax_device {
>> struct cdev cdev;
>> const char *host;
>> void *private;
>> + void *holder_data;
>> unsigned long flags;
>> const struct dax_operations *ops;
>> + const struct dax_holder_operations *holder_ops;
>> };
>>
>> static dev_t dax_devt;
>> @@ -374,6 +379,29 @@ int dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff,
>> }
>> EXPORT_SYMBOL_GPL(dax_zero_page_range);
>>
>> +int dax_holder_notify_failure(struct dax_device *dax_dev, loff_t offset,
>> + size_t size, int flags)
>> +{
>> + int rc;
>> +
>> + dax_read_lock();
>> + if (!dax_alive(dax_dev)) {
>> + rc = -ENXIO;
>> + goto out;
>> + }
>> +
>> + if (!dax_dev->holder_data) {
>> + rc = -EOPNOTSUPP;
>> + goto out;
>> + }
>> +
>> + rc = dax_dev->holder_ops->notify_failure(dax_dev, offset, size, flags);
>
> Shouldn't this check if dax_dev->holder_ops != NULL before dereferencing
> it for the function call? Imagine an implementation that wants to
> attach a ->notify_failure function to a dax_device, maintains its own
> lookup table, and decides that it doesn't need to set holder_data.
>
> (Or, imagine someone who writes a garbage into holder_data and *boom*)

My mistake. I should check @holder_ops instead of @holder_data.

>
> How does the locking work here? If there's a media failure, we'll take
> dax_rwsem and call ->notify_failure. If the ->notify_failure function
> wants to access the pmem to handle the error by calling back into the
> dax code, will that cause nested locking on dax_rwsem?

Won't for now. I have tested it with my simple testcases.
>
> Jumping ahead a bit, I think the rmap btree accesses that the xfs
> implementation performs can cause xfs_buf(fer) cache IO, which would
> trigger that if the buffers aren't already in memory, if I'm reading
> this correctly?

I didn't think of this case. But I think this uses read lock too. It
won't be blocked. Only dax_set_holder() takes write lock.

>
>> +out:
>> + dax_read_unlock();
>> + return rc;
>> +}
>> +EXPORT_SYMBOL_GPL(dax_holder_notify_failure);
>> +
>> #ifdef CONFIG_ARCH_HAS_PMEM_API
>> void arch_wb_cache_pmem(void *addr, size_t size);
>> void dax_flush(struct dax_device *dax_dev, void *addr, size_t size)
>> @@ -618,6 +646,37 @@ void put_dax(struct dax_device *dax_dev)
>> }
>> EXPORT_SYMBOL_GPL(put_dax);
>>
>> +void dax_set_holder(struct dax_device *dax_dev, void *holder,
>> + const struct dax_holder_operations *ops)
>> +{
>> + dax_write_lock();
>> + if (!dax_alive(dax_dev)) {
>> + dax_write_unlock();
>> + return;
>> + }
>> +
>> + dax_dev->holder_data = holder;
>> + dax_dev->holder_ops = ops;
>> + dax_write_unlock();
>
> I guess this means that the holder has to detach itself before anyone
> calls kill_dax, or else a dead dax device ends up with a dangling
> reference to the holder?

Yes.

>
>> +}
>> +EXPORT_SYMBOL_GPL(dax_set_holder);
>> +
>> +void *dax_get_holder(struct dax_device *dax_dev)
>> +{
>> + void *holder;
>> +
>> + dax_read_lock();
>> + if (!dax_alive(dax_dev)) {
>> + dax_read_unlock();
>> + return NULL;
>> + }
>> +
>> + holder = dax_dev->holder_data;
>> + dax_read_unlock();
>> + return holder;
>> +}
>> +EXPORT_SYMBOL_GPL(dax_get_holder);
>> +
>> /**
>> * inode_dax: convert a public inode into its dax_dev
>> * @inode: An inode with i_cdev pointing to a dax_dev
>> diff --git a/include/linux/dax.h b/include/linux/dax.h
>> index 097b3304f9b9..d273d59723cd 100644
>> --- a/include/linux/dax.h
>> +++ b/include/linux/dax.h
>> @@ -38,9 +38,24 @@ struct dax_operations {
>> int (*zero_page_range)(struct dax_device *, pgoff_t, size_t);
>> };
>>
>> +struct dax_holder_operations {
>> + /*
>> + * notify_failure - notify memory failure into inner holder device
>> + * @dax_dev: the dax device which contains the holder
>> + * @offset: offset on this dax device where memory failure occurs
>> + * @size: length of this memory failure event
>> + * @flags: action flags for memory failure handler
>> + */
>> + int (*notify_failure)(struct dax_device *dax_dev, loff_t offset,
>> + size_t size, int flags);
>
> Shouldn't size be u64 or something? Let's say that 8GB of your pmem go
> bad, wouldn't you want a single call? Though I guess the current
> implementation only goes a single page at a time, doesn't it?

Right.

>
>> +};
>> +
>> extern struct attribute_group dax_attribute_group;
>>
>> #if IS_ENABLED(CONFIG_DAX)
>> +void dax_set_holder(struct dax_device *dax_dev, void *holder,
>> + const struct dax_holder_operations *ops);
>> +void *dax_get_holder(struct dax_device *dax_dev);
>> struct dax_device *alloc_dax(void *private, const char *host,
>> const struct dax_operations *ops, unsigned long flags);
>> void put_dax(struct dax_device *dax_dev);
>> @@ -70,6 +85,18 @@ static inline bool daxdev_mapping_supported(struct vm_area_struct *vma,
>> return dax_synchronous(dax_dev);
>> }
>> #else
>> +static inline struct dax_device *dax_get_by_host(const char *host)
>
> Not sure why this is being added here? AFAICT none of the patches call
> this function...?

It's mistake when I rebase my code to the latest. These lines were
deleted but I didn't notice. Will fix it.

--
Thanks,
Ruan.

>
> --D
>
>> +{
>> + return NULL;
>> +}
>> +static inline void dax_set_holder(struct dax_device *dax_dev, void *holder,
>> + const struct dax_holder_operations *ops)
>> +{
>> +}
>> +static inline void *dax_get_holder(struct dax_device *dax_dev)
>> +{
>> + return NULL;
>> +}
>> static inline struct dax_device *alloc_dax(void *private, const char *host,
>> const struct dax_operations *ops, unsigned long flags)
>> {
>> @@ -198,6 +225,8 @@ size_t dax_copy_to_iter(struct dax_device *dax_dev, pgoff_t pgoff, void *addr,
>> size_t bytes, struct iov_iter *i);
>> int dax_zero_page_range(struct dax_device *dax_dev, pgoff_t pgoff,
>> size_t nr_pages);
>> +int dax_holder_notify_failure(struct dax_device *dax_dev, loff_t offset,
>> + size_t size, int flags);
>> void dax_flush(struct dax_device *dax_dev, void *addr, size_t size);
>>
>> ssize_t dax_iomap_rw(struct kiocb *iocb, struct iov_iter *iter,
>> --
>> 2.33.0
>>
>>
>>