From: Darrick J. Wong <[email protected]>
This is the fourth attempt at documenting the design of iomap and how to
port filesystems to use it. Apologies for all the rst formatting, but
it's necessary to distinguish code from regular text.
A lot of this has been collected from various email conversations, code
comments, commit messages, my own understanding of iomap, and
Ritesh/Luis' previous efforts to create a document. Please note a large
part of this has been taken from Dave's reply to last iomap doc
patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
other iomap developers who have taken time to explain the iomap design
in various emails, commits, comments etc.
Cc: Dave Chinner <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Christoph Hellwig <[email protected]>
Cc: Christian Brauner <[email protected]>
Cc: Ojaswin Mujoo <[email protected]>
Cc: Jan Kara <[email protected]>
Cc: Luis Chamberlain <[email protected]>
Inspired-by: Ritesh Harjani (IBM) <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
Documentation/filesystems/index.rst | 1
Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
MAINTAINERS | 1
3 files changed, 1062 insertions(+)
create mode 100644 Documentation/filesystems/iomap.rst
diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
index 8f5c1ee02e2f..b010cc8df32d 100644
--- a/Documentation/filesystems/index.rst
+++ b/Documentation/filesystems/index.rst
@@ -34,6 +34,7 @@ algorithms work.
seq_file
sharedsubtree
idmappings
+ iomap
automount-support
diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
new file mode 100644
index 000000000000..a478b55e4135
--- /dev/null
+++ b/Documentation/filesystems/iomap.rst
@@ -0,0 +1,1060 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. _iomap:
+
+..
+ Dumb style notes to maintain the author's sanity:
+ Please try to start sentences on separate lines so that
+ sentence changes don't bleed colors in diff.
+ Heading decorations are documented in sphinx.rst.
+
+============================
+VFS iomap Design and Porting
+============================
+
+.. toctree::
+
+Introduction
+============
+
+iomap is a filesystem library for handling various filesystem operations
+that involves mapping of file's logical offset ranges to physical
+extents.
+This origins of this library is the file I/O path that XFS once used; it
+has now been extended to cover several other operations.
+The library provides various APIs for implementing various file and
+pagecache operations, such as:
+
+ * Pagecache reads and writes
+ * Folio write faults to the pagecache
+ * Writeback of dirty folios
+ * Direct I/O reads and writes
+ * FIEMAP
+ * lseek ``SEEK_DATA`` and ``SEEK_HOLE``
+ * swapfile activation
+
+Who Should Read This?
+=====================
+
+The target audience for this document are filesystem, storage, and
+pagecache programmers and code reviewers.
+The goal of this document is to provide a brief discussion of the
+design and capabilities of iomap, followed by a more detailed catalog
+of the interfaces presented by iomap.
+If you change iomap, please update this design document.
+
+But Why?
+========
+
+Unlike the classic Linux I/O model which breaks file I/O into small
+units (generally memory pages or blocks) and looks up space mappings on
+the basis of that unit, the iomap model asks the filesystem for the
+largest space mappings that it can create for a given file operation and
+initiates operations on that basis.
+This strategy improves the filesystem's visibility into the size of the
+operation being performed, which enables it to combat fragmentation with
+larger space allocations when possible.
+Larger space mappings improve runtime performance by amortizing the cost
+of a mapping function call into the filesystem across a larger amount of
+data.
+
+At a high level, an iomap operation `looks like this
+<https://lore.kernel.org/all/[email protected]/>`_:
+
+1. For each byte in the operation range...
+
+ 1. Obtain space mapping via ->iomap_begin
+ 2. For each sub-unit of work...
+
+ 1. Revalidate the mapping and go back to (1) above, if necessary
+ 2. Do the work
+
+ 3. Increment operation cursor
+ 4. Release the mapping via ->iomap_end, if necessary
+
+Each iomap operation will be covered in more detail below.
+This library was covered previously by an `LWN article
+<https://lwn.net/Articles/935934/>`_ and a `KernelNewbies page
+<https://kernelnewbies.org/KernelProjects/iomap>`_.
+
+Data Structures and Algorithms
+==============================
+
+Definitions
+-----------
+
+ * ``bufferhead``: Shattered remnants of the old buffer cache.
+ * ``fsblock``: The block size of a file, also known as ``i_blocksize``.
+ * ``i_rwsem``: The VFS ``struct inode`` rwsemaphore.
+ * ``invalidate_lock``: The pagecache ``struct address_space``
+ rwsemaphore that protects against folio removal.
+
+struct iomap_ops
+----------------
+
+Every iomap function requires the filesystem to pass an operations
+structure to obtain a mapping and (optionally) to release the mapping.
+
+.. code-block:: c
+
+ struct iomap_ops {
+ int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
+ unsigned flags, struct iomap *iomap,
+ struct iomap *srcmap);
+
+ int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
+ ssize_t written, unsigned flags,
+ struct iomap *iomap);
+ };
+
+The ``->iomap_begin`` function is called to obtain one mapping for the
+range of bytes specified by ``pos`` and ``length`` for the file
+``inode``.
+
+Each iomap operation describes the requested operation through the
+``flags`` argument.
+The exact value of ``flags`` will be documented in the
+operation-specific sections below, but these principles apply generally:
+
+ * For a write operation, ``IOMAP_WRITE`` will be set.
+ Filesystems must not return ``IOMAP_HOLE`` mappings.
+
+ * For any other operation, ``IOMAP_WRITE`` will not be set.
+
+ * For any operation targetting direct access to storage (fsdax),
+ ``IOMAP_DAX`` will be set.
+
+If it is necessary to read existing file contents from a `different
+<https://lore.kernel.org/all/[email protected]/>`_ device or
+address range on a device, the filesystem should return that information via
+``srcmap``.
+Only pagecache and fsdax operations support reading from one mapping and
+writing to another.
+
+After the operation completes, the ``->iomap_end`` function, if present,
+is called to signal that iomap is finished with a mapping.
+Typically, implementations will use this function to tear down any
+context that were set up in ``->iomap_begin``.
+For example, a write might wish to commit the reservations for the bytes
+that were operated upon and unreserve any space that was not operated
+upon.
+``written`` might be zero if no bytes were touched.
+``flags`` will contain the same value passed to ``->iomap_begin``.
+iomap ops for reads are not likely to need to supply this function.
+
+Both functions should return a negative errno code on error, or zero.
+
+struct iomap
+------------
+
+The filesystem returns the mappings via the following structure.
+For documentation purposes, the structure has been reordered to group
+fields that go together logically.
+
+.. code-block:: c
+
+ struct iomap {
+ loff_t offset;
+ u64 length;
+
+ u16 type;
+ u16 flags;
+
+ u64 addr;
+ struct block_device *bdev;
+ struct dax_device *dax_dev;
+ void *inline_data;
+
+ void *private;
+
+ const struct iomap_folio_ops *folio_ops;
+
+ u64 validity_cookie;
+ };
+
+The information is useful for translating file operations into action.
+The actions taken are specific to the target of the operation, such as
+disk cache, physical storage devices, or another part of the kernel.
+
+ * ``offset`` and ``length`` describe the range of file offsets, in
+ bytes, covered by this mapping.
+ These fields must always be set by the filesystem.
+
+ * ``type`` describes the type of the space mapping:
+
+ * **IOMAP_HOLE**: No storage has been allocated.
+ This type must never be returned in response to an IOMAP_WRITE
+ operation because writes must allocate and map space, and return
+ the mapping.
+ The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
+ iomap does not support writing (whether via pagecache or direct
+ I/O) to a hole.
+
+ * **IOMAP_DELALLOC**: A promise to allocate space at a later time
+ ("delayed allocation").
+ If the filesystem returns IOMAP_F_NEW here and the write fails, the
+ ``->iomap_end`` function must delete the reservation.
+ The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
+
+ * **IOMAP_MAPPED**: The file range maps to specific space on the
+ storage device.
+ The device is returned in ``bdev`` or ``dax_dev``.
+ The device address, in bytes, is returned via ``addr``.
+
+ * **IOMAP_UNWRITTEN**: The file range maps to specific space on the
+ storage device, but the space has not yet been initialized.
+ The device is returned in ``bdev`` or ``dax_dev``.
+ The device address, in bytes, is returned via ``addr``.
+ Reads will return zeroes to userspace.
+ For a write or writeback operation, the ioend should update the
+ mapping to MAPPED.
+
+ * **IOMAP_INLINE**: The file range maps to the memory buffer
+ specified by ``inline_data``.
+ For write operation, the ``->iomap_end`` function presumably
+ handles persisting the data.
+ The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
+
+ * ``flags`` describe the status of the space mapping.
+ These flags should be set by the filesystem in ``->iomap_begin``:
+
+ * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
+ Areas that will not be written to must be zeroed.
+ If a write fails and the mapping is a space reservation, the
+ reservation must be deleted.
+
+ * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
+ to access any data written.
+ fdatasync is required to commit these changes to persistent
+ storage.
+ This needs to take into account metadata changes that *may* be made
+ at I/O completion, such as file size updates from direct I/O.
+
+ * **IOMAP_F_SHARED**: The space under the mapping is shared.
+ Copy on write is necessary to avoid corrupting other file data.
+
+ * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
+ heads for pagecache operations.
+ Do not add more uses of this.
+
+ * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
+ coalesced into this single mapping.
+ This is only useful for FIEMAP.
+
+ * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
+ regular file data.
+ This is only useful for FIEMAP.
+
+ * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
+ be set by the filesystem for its own purposes.
+
+ These flags can be set by iomap itself during file operations.
+ The filesystem should supply an ``->iomap_end`` function to observe
+ these flags:
+
+ * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
+ using this mapping.
+
+ * **IOMAP_F_STALE**: The mapping was found to be stale.
+ iomap will call ``->iomap_end`` on this mapping and then
+ ``->iomap_begin`` to obtain a new mapping.
+
+ Currently, these flags are only set by pagecache operations.
+
+ * ``addr`` describes the device address, in bytes.
+
+ * ``bdev`` describes the block device for this mapping.
+ This only needs to be set for mapped or unwritten operations.
+
+ * ``dax_dev`` describes the DAX device for this mapping.
+ This only needs to be set for mapped or unwritten operations, and
+ only for a fsdax operation.
+
+ * ``inline_data`` points to a memory buffer for I/O involving
+ ``IOMAP_INLINE`` mappings.
+ This value is ignored for all other mapping types.
+
+ * ``private`` is a pointer to `filesystem-private information
+ <https://lore.kernel.org/all/[email protected]/>`_.
+ This value will be passed unchanged to ``->iomap_end``.
+
+ * ``folio_ops`` will be covered in the section on pagecache operations.
+
+ * ``validity_cookie`` is a magic freshness value set by the filesystem
+ that should be used to detect stale mappings.
+ For pagecache operations this is critical for correct operation
+ because page faults can occur, which implies that filesystem locks
+ should not be held between ``->iomap_begin`` and ``->iomap_end``.
+ Filesystems with completely static mappings need not set this value.
+ Only pagecache operations revalidate mappings.
+
+ XXX: Should fsdax revalidate as well?
+
+Validation
+==========
+
+**NOTE**: iomap only handles mapping and I/O.
+Filesystems must still call out to the VFS to check input parameters
+and file state before initiating an I/O operation.
+It does not handle updating of timestamps, stripping privileges, or
+access control.
+
+Locking Hierarchy
+=================
+
+iomap requires that filesystems provide their own locking.
+There are no locks within iomap itself, though in the course of an
+operation iomap may take other locks (e.g. folio/dax locks) as part of
+an I/O operation.
+Locking with iomap can be split into two categories: above and below
+iomap.
+
+The upper level of lock must coordinate the iomap operation with other
+iomap operations.
+Generally, the filesystem must take VFS/pagecache locks such as
+``i_rwsem`` or ``invalidate_lock`` before calling into iomap.
+The exact locking requirements are specific to the type of operation.
+
+The lower level of lock must coordinate access to the mapping
+information.
+This lock is filesystem specific and should be held during
+``->iomap_begin`` while sampling the mapping and validity cookie.
+
+The general locking hierarchy in iomap is:
+
+ * VFS or pagecache lock
+
+ * Internal filesystem specific mapping lock
+
+ * iomap operation-specific lock
+
+The exact locking requirements are specific to the filesystem; for
+certain operations, some of these locks can be elided.
+All further mention of locking are *recommendations*, not mandates.
+Each filesystem author must figure out the locking for themself.
+
+iomap Operations
+================
+
+Below are a discussion of the file operations that iomap implements.
+
+Buffered I/O
+------------
+
+Buffered I/O is the default file I/O path in Linux.
+File contents are cached in memory ("pagecache") to satisfy reads and
+writes.
+Dirty cache will be written back to disk at some point that can be
+forced via ``fsync`` and variants.
+
+iomap implements nearly all the folio and pagecache management that
+filesystems once had to implement themselves.
+This means that the filesystem need not know the details of allocating,
+mapping, managing uptodate and dirty state, or writeback of pagecache
+folios.
+Unless the filesystem explicitly opts in to buffer heads, they will not
+be used, which makes buffered I/O much more efficient, and ``willy``
+much happier.
+
+struct address_space_operations
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The following iomap functions can be referenced directly from the
+address space operations structure:
+
+ * ``iomap_dirty_folio``
+ * ``iomap_release_folio``
+ * ``iomap_invalidate_folio``
+ * ``iomap_is_partially_uptodate``
+
+The following address space operations can be wrapped easily:
+
+ * ``read_folio``
+ * ``readahead``
+ * ``writepages``
+ * ``bmap``
+ * ``swap_activate``
+
+struct iomap_folio_ops
+~~~~~~~~~~~~~~~~~~~~~~
+
+The ``->iomap_begin`` function for pagecache operations may set the
+``struct iomap::folio_ops`` field to an ops structure to override
+default behaviors of iomap:
+
+.. code-block:: c
+
+ struct iomap_folio_ops {
+ struct folio *(*get_folio)(struct iomap_iter *iter, loff_t pos,
+ unsigned len);
+ void (*put_folio)(struct inode *inode, loff_t pos, unsigned copied,
+ struct folio *folio);
+ bool (*iomap_valid)(struct inode *inode, const struct iomap *iomap);
+ };
+
+iomap calls these functions:
+
+ - ``get_folio``: Called to allocate and return an active reference to
+ a locked folio prior to starting a write.
+ If this function is not provided, iomap will call
+ ``iomap_get_folio``.
+ This could be used to `set up per-folio filesystem state
+ <https://lore.kernel.org/all/[email protected]/>`_
+ for a write.
+
+ - ``put_folio``: Called to unlock and put a folio after a pagecache
+ operation completes.
+ If this function is not provided, iomap will ``folio_unlock`` and
+ ``folio_put`` on its own.
+ This could be used to `commit per-folio filesystem state
+ <https://lore.kernel.org/all/[email protected]/>`_
+ that was set up by ``->get_folio``.
+
+ - ``iomap_valid``: The filesystem may not hold locks between
+ ``->iomap_begin`` and ``->iomap_end`` because pagecache operations
+ can take folio locks, fault on userspace pages, initiate writeback
+ for memory reclamation, or engage in other time-consuming actions.
+ If a file's space mapping data are mutable, it is possible that the
+ mapping for a particular pagecache folio can `change in the time it
+ takes
+ <https://lore.kernel.org/all/[email protected]/>`_
+ to allocate, install, and lock that folio.
+ For such files, the mapping *must* be revalidated after the folio
+ lock has been taken so that iomap can manage the folio correctly.
+ The filesystem's ``->iomap_begin`` function must sample a sequence
+ counter into ``struct iomap::validity_cookie`` at the same time that
+ it populates the mapping fields.
+ It must then provide a ``->iomap_valid`` function to compare the
+ validity cookie against the source counter and return whether or not
+ the mapping is still valid.
+ If the mapping is not valid, the mapping will be sampled again.
+
+These ``struct kiocb`` flags are significant for buffered I/O with
+iomap:
+
+ * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
+ already in memory, we do not have to initiate other I/O, and we
+ acquire all filesystem locks without blocking.
+ Neither this flag nor its definition ``RWF_NOWAIT`` actually define
+ what this flag means, so this is the best the author could come up
+ with.
+
+Internal per-Folio State
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the fsblock size matches the size of a pagecache folio, it is assumed
+that all disk I/O operations will operate on the entire folio.
+The uptodate (memory contents are at least as new as what's on disk) and
+dirty (memory contents are newer than what's on disk) status of the
+folio are all that's needed for this case.
+
+If the fsblock size is less than the size of a pagecache folio, iomap
+tracks the per-fsblock uptodate and dirty state itself.
+This enables iomap to handle both "bs < ps" `filesystems
+<https://lore.kernel.org/all/[email protected]/>`_
+and large folios in the pagecache.
+
+iomap internally tracks two state bits per fsblock:
+
+ * ``uptodate``: iomap will try to keep folios fully up to date.
+ If there are read(ahead) errors, those fsblocks will not be marked
+ uptodate.
+ The folio itself will be marked uptodate when all fsblocks within the
+ folio are uptodate.
+
+ * ``dirty``: iomap will set the per-block dirty state when programs
+ write to the file.
+ The folio itself will be marked dirty when any fsblock within the
+ folio is dirty.
+
+iomap also tracks the amount of read and write disk IOs that are in
+flight.
+This structure is much lighter weight than ``struct buffer_head``.
+
+Filesystems wishing to turn on large folios in the pagecache should call
+``mapping_set_large_folios`` when initializing the incore inode.
+
+Readahead and Reads
+~~~~~~~~~~~~~~~~~~~
+
+The ``iomap_readahead`` function initiates readahead to the pagecache.
+The ``iomap_read_folio`` function reads one folio's worth of data into
+the pagecache.
+The ``flags`` argument to ``->iomap_begin`` will be set to zero.
+The pagecache takes whatever locks it needs before calling the
+filesystem.
+
+Writes
+~~~~~~
+
+The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
+pagecache.
+``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
+the ``flags`` argument to ``->iomap_begin``.
+Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
+
+mmap Write Faults
+^^^^^^^^^^^^^^^^^
+
+The ``iomap_page_mkwrite`` function handles a write fault to a folio the
+pagecache.
+``IOMAP_WRITE | IOMAP_FAULT`` will be passed as the ``flags`` argument
+to ``->iomap_begin``.
+Callers commonly take the mmap ``invalidate_lock`` in shared or
+exclusive mode.
+
+Write Failures
+^^^^^^^^^^^^^^
+
+After a short write to the pagecache, the areas not written will not
+become marked dirty.
+The filesystem must arrange to `cancel
+<https://lore.kernel.org/all/[email protected]/>`_
+such `reservations
+<https://lore.kernel.org/linux-xfs/[email protected]/>`_
+because writeback will not consume the reservation.
+The ``iomap_file_buffered_write_punch_delalloc`` can be called from a
+``->iomap_end`` function to find all the clean areas of the folios
+caching a fresh (``IOMAP_F_NEW``) delalloc mapping.
+It takes the ``invalidate_lock``.
+
+The filesystem should supply a callback ``punch`` will be called for
+each file range in this state.
+This function must *only* remove delayed allocation reservations, in
+case another thread racing with the current thread writes successfully
+to the same region and triggers writeback to flush the dirty data out to
+disk.
+
+Truncation
+^^^^^^^^^^
+
+Filesystems can call ``iomap_truncate_page`` to zero the bytes in the
+pagecache from EOF to the end of the fsblock during a file truncation
+operation.
+``truncate_setsize`` or ``truncate_pagecache`` will take care of
+everything after the EOF block.
+``IOMAP_ZERO`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
+mode.
+
+Zeroing for File Operations
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Filesystems can call ``iomap_zero_range`` to perform zeroing of the
+pagecache for non-truncation file operations that are not aligned to
+the fsblock size.
+``IOMAP_ZERO`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
+mode.
+
+Unsharing Reflinked File Data
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Filesystems can call ``iomap_file_unshare`` to force a file sharing
+storage with another file to preemptively copy the shared data to newly
+allocate storage.
+``IOMAP_WRITE | IOMAP_UNSHARE`` will be passed as the ``flags`` argument
+to ``->iomap_begin``.
+Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
+mode.
+
+Writeback
+~~~~~~~~~
+
+Filesystems can call ``iomap_writepages`` to respond to a request to
+write dirty pagecache folios to disk.
+The ``mapping`` and ``wbc`` parameters should be passed unchanged.
+The ``wpc`` pointer should be allocated by the filesystem and must
+be initialized to zero.
+
+The pagecache will lock each folio before trying to schedule it for
+writeback.
+It does not lock ``i_rwsem`` or ``invalidate_lock``.
+
+The dirty bit will be cleared for all folios run through the
+``->map_blocks`` machinery described below even if the writeback fails.
+This is to prevent dirty folio clots when storage devices fail; an
+``-EIO`` is recorded for userspace to collect via ``fsync``.
+
+The ``ops`` structure must be specified and is as follows:
+
+struct iomap_writeback_ops
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. code-block:: c
+
+ struct iomap_writeback_ops {
+ int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
+ loff_t offset, unsigned len);
+ int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
+ void (*discard_folio)(struct folio *folio, loff_t pos);
+ };
+
+The fields are as follows:
+
+ - ``map_blocks``: Sets ``wpc->iomap`` to the space mapping of the file
+ range (in bytes) given by ``offset`` and ``len``.
+ iomap calls this function for each fs block in each dirty folio,
+ even if the mapping returned is longer than one fs block.
+ Do not return ``IOMAP_INLINE`` mappings here; the ``->iomap_end``
+ function must deal with persisting written data.
+ Filesystems can skip a potentially expensive mapping lookup if the
+ mappings have not changed.
+ This revalidation must be open-coded by the filesystem; it is
+ unclear if ``iomap::validity_cookie`` can be reused for this
+ purpose.
+ This function is required.
+
+ - ``prepare_ioend``: Enables filesystems to transform the writeback
+ ioend or perform any other prepatory work before the writeback I/O
+ is submitted.
+ A filesystem can override the ``->bi_end_io`` function for its own
+ purposes, such as kicking the ioend completion to a workqueue if the
+ bio is completed in interrupt context.
+ This function is optional.
+
+ - ``discard_folio``: iomap calls this function after ``->map_blocks``
+ fails schedule I/O for any part of a dirty folio.
+ The function should throw away any reservations that may have been
+ made for the write.
+ The folio will be marked clean and an ``-EIO`` recorded in the
+ pagecache.
+ Filesystems can use this callback to `remove
+ <https://lore.kernel.org/all/[email protected]/>`_
+ delalloc reservations to avoid having delalloc reservations for
+ clean pagecache.
+ This function is optional.
+
+Writeback ioend Completion
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+iomap creates chains of ``struct iomap_ioend`` objects that wrap the
+``bio`` that is used to write pagecache data to disk.
+By default, iomap finishes writeback ioends by clearing the writeback
+bit on the folios attached to the ``ioend``.
+If the write failed, it will also set the error bits on the folios and
+the address space.
+This can happen in interrupt or process context, depending on the
+storage device.
+
+Filesystems that need to update internal bookkeeping (e.g. unwritten
+extent conversions) should provide a ``->prepare_ioend`` function to
+override the ``struct iomap_end::bio::bi_end_io`` with its own function.
+This function should call ``iomap_finish_ioends`` after finishing its
+own work.
+
+Some filesystems may wish to `amortize the cost of running metadata
+transactions
+<https://lore.kernel.org/all/[email protected]/>`_
+for post-writeback updates by batching them.
+They may also require transactions to run from process context, which
+implies punting batches to a workqueue.
+iomap ioends contain a ``list_head`` to enable batching.
+
+Given a batch of ioends, iomap has a few helpers to assist with
+amortization:
+
+ * ``iomap_sort_ioends``: Sort all the ioends in the list by file
+ offset.
+
+ * ``iomap_ioend_try_merge``: Given an ioend that is not in any list and
+ a separate list of sorted ioends, merge as many of the ioends from
+ the head of the list into the given ioend.
+ ioends can only be merged if the file range and storage addresses are
+ contiguous; the unwritten and shared status are the same; and the
+ write I/O outcome is the same.
+ The merged ioends become their own list.
+
+ * ``iomap_finish_ioends``: Finish an ioend that possibly has other
+ ioends linked to it.
+
+Direct I/O
+----------
+
+In Linux, direct I/O is defined as file I/O that is issued directly to
+storage, bypassing the pagecache.
+
+The ``iomap_dio_rw`` function implements O_DIRECT (direct I/O) reads and
+writes for files.
+An optional ``ops`` parameter can be passed to change the behavior of
+direct I/O.
+The ``done_before`` parameter should be set if writes have been
+initiated prior to the call.
+The direction of the I/O is determined from the ``iocb`` passed in.
+
+The ``flags`` argument can be any of the following values:
+
+ * ``IOMAP_DIO_FORCE_WAIT``: Wait for the I/O to complete even if the
+ kiocb is not synchronous.
+
+ * ``IOMAP_DIO_OVERWRITE_ONLY``: Allocating blocks, zeroing partial
+ blocks, and extensions of the file size are not allowed.
+ The entire file range must to map to a single written or unwritten
+ extent.
+ This flag exists to enable issuing concurrent direct IOs with only
+ the shared ``i_rwsem`` held when the file I/O range is not aligned to
+ the filesystem block size.
+ ``-EAGAIN`` will be returned if the operation cannot proceed.
+
+ * ``IOMAP_DIO_PARTIAL``: If a page fault occurs, return whatever
+ progress has already been made.
+ The caller may deal with the page fault and retry the operation.
+
+These ``struct kiocb`` flags are significant for direct I/O with iomap:
+
+ * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
+ already in memory, we do not have to initiate other I/O, and we
+ acquire all filesystem locks without blocking.
+
+ * ``IOCB_SYNC``: Ensure that the device has persisted data to disk
+ before completing the call.
+ In the case of pure overwrites, the I/O may be issued with FUA
+ enabled.
+
+ * ``IOCB_HIPRI``: Poll for I/O completion instead of waiting for an
+ interrupt.
+ Only meaningful for asynchronous I/O, and only if the entire I/O can
+ be issued as a single ``struct bio``.
+
+ * ``IOCB_DIO_CALLER_COMP``: Try to run I/O completion from the caller's
+ process context.
+ See ``linux/fs.h`` for more details.
+
+Filesystems should call ``iomap_dio_rw`` from ``->read_iter`` and
+``->write_iter``, and set ``FMODE_CAN_ODIRECT`` in the ``->open``
+function for the file.
+They should not set ``->direct_IO``, which is deprecated.
+
+If a filesystem wishes to perform its own work before direct I/O
+completion, it should call ``__iomap_dio_rw``.
+If its return value is not an error pointer or a NULL pointer, the
+filesystem should pass the return value to ``iomap_dio_complete`` after
+finishing its internal work.
+
+Direct Reads
+~~~~~~~~~~~~
+
+A direct I/O read initiates a read I/O from the storage device to the
+caller's buffer.
+Dirty parts of the pagecache are flushed to storage before initiating
+the read io.
+The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT`` with
+any combination of the following enhancements:
+
+ * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
+ Does not initiate other I/O or block on filesystem locks.
+
+Callers commonly hold ``i_rwsem`` in shared mode.
+
+Direct Writes
+~~~~~~~~~~~~~
+
+A direct I/O write initiates a write I/O to the storage device to the
+caller's buffer.
+Dirty parts of the pagecache are flushed to storage before initiating
+the write io.
+The pagecache is invalidated both before and after the write io.
+The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT |
+IOMAP_WRITE`` with any combination of the following enhancements:
+
+ * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
+ Does not initiate other I/O or block on filesystem locks.
+ * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
+ blocks is not allowed.
+ The entire file range must to map to a single written or unwritten
+ extent.
+ The file I/O range must be aligned to the filesystem block size.
+
+Callers commonly hold ``i_rwsem`` in shared or exclusive mode.
+
+struct iomap_dio_ops:
+~~~~~~~~~~~~~~~~~~~~~
+.. code-block:: c
+
+ struct iomap_dio_ops {
+ void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
+ loff_t file_offset);
+ int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
+ unsigned flags);
+ struct bio_set *bio_set;
+ };
+
+The fields of this structure are as follows:
+
+ - ``submit_io``: iomap calls this function when it has constructed a
+ ``struct bio`` object for the I/O requested, and wishes to submit it
+ to the block device.
+ If no function is provided, ``submit_bio`` will be called directly.
+ Filesystems that would like to perform additional work before (e.g.
+ data replication for btrfs) should implement this function.
+
+ - ``end_io``: This is called after the ``struct bio`` completes.
+ This function should perform post-write conversions of unwritten
+ extent mappings, handle write failures, etc.
+ The ``flags`` argument may be set to a combination of the following:
+
+ * ``IOMAP_DIO_UNWRITTEN``: The mapping was unwritten, so the ioend
+ should mark the extent as written.
+
+ * ``IOMAP_DIO_COW``: Writing to the space in the mapping required a
+ copy on write operation, so the ioend should switch mappings.
+
+ - ``bio_set``: This allows the filesystem to provide a custom bio_set
+ for allocating direct I/O bios.
+ This enables filesystems to `stash additional per-bio information
+ <https://lore.kernel.org/all/[email protected]/>`_
+ for private use.
+ If this field is NULL, generic ``struct bio`` objects will be used.
+
+Filesystems that want to perform extra work after an I/O completion
+should set a custom ``->bi_end_io`` function via ``->submit_io``.
+Afterwards, the custom endio function must call
+``iomap_dio_bio_end_io`` to finish the direct I/O.
+
+DAX I/O
+-------
+
+Storage devices that can be directly mapped as memory support a new
+access mode known as "fsdax".
+
+fsdax Reads
+~~~~~~~~~~~
+
+A fsdax read performs a memcpy from storage device to the caller's
+buffer.
+The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX`` with any
+combination of the following enhancements:
+
+ * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
+ Does not initiate other I/O or block on filesystem locks.
+
+Callers commonly hold ``i_rwsem`` in shared mode.
+
+fsdax Writes
+~~~~~~~~~~~~
+
+A fsdax write initiates a memcpy to the storage device to the caller's
+buffer.
+The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX |
+IOMAP_WRITE`` with any combination of the following enhancements:
+
+ * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
+ Does not initiate other I/O or block on filesystem locks.
+
+ * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
+ blocks is not allowed.
+ The entire file range must to map to a single written or unwritten
+ extent.
+ The file I/O range must be aligned to the filesystem block size.
+
+Callers commonly hold ``i_rwsem`` in exclusive mode.
+
+mmap Faults
+~~~~~~~~~~~
+
+The ``dax_iomap_fault`` function handles read and write faults to fsdax
+storage.
+For a read fault, ``IOMAP_DAX | IOMAP_FAULT`` will be passed as the
+``flags`` argument to ``->iomap_begin``.
+For a write fault, ``IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE`` will be
+passed as the ``flags`` argument to ``->iomap_begin``.
+
+Callers commonly hold the same locks as they do to call their iomap
+pagecache counterparts.
+
+Truncation, fallocate, and Unsharing
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For fsdax files, the following functions are provided to replace their
+iomap pagecache I/O counterparts.
+The ``flags`` argument to ``->iomap_begin`` are the same as the
+pagecache counterparts, with ``IOMAP_DIO`` added.
+
+ * ``dax_file_unshare``
+ * ``dax_zero_range``
+ * ``dax_truncate_page``
+
+Callers commonly hold the same locks as they do to call their iomap
+pagecache counterparts.
+
+SEEK_DATA
+---------
+
+The ``iomap_seek_data`` function implements the SEEK_DATA "whence" value
+for llseek.
+``IOMAP_REPORT`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+
+For unwritten mappings, the pagecache will be searched.
+Regions of the pagecache with a folio mapped and uptodate fsblocks
+within those folios will be reported as data areas.
+
+Callers commonly hold ``i_rwsem`` in shared mode.
+
+SEEK_HOLE
+---------
+
+The ``iomap_seek_hole`` function implements the SEEK_HOLE "whence" value
+for llseek.
+``IOMAP_REPORT`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+
+For unwritten mappings, the pagecache will be searched.
+Regions of the pagecache with no folio mapped, or a !uptodate fsblock
+within a folio will be reported as sparse hole areas.
+
+Callers commonly hold ``i_rwsem`` in shared mode.
+
+Swap File Activation
+--------------------
+
+The ``iomap_swapfile_activate`` function finds all the base-page aligned
+regions in a file and sets them up as swap space.
+The file will be ``fsync()``'d before activation.
+``IOMAP_REPORT`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+All mappings must be mapped or unwritten; cannot be dirty or shared, and
+cannot span multiple block devices.
+Callers must hold ``i_rwsem`` in exclusive mode; this is already
+provided by ``swapon``.
+
+Extent Map Reporting (FS_IOC_FIEMAP)
+------------------------------------
+
+The ``iomap_fiemap`` function exports file extent mappings to userspace
+in the format specified by the ``FS_IOC_FIEMAP`` ioctl.
+``IOMAP_REPORT`` will be passed as the ``flags`` argument to
+``->iomap_begin``.
+Callers commonly hold ``i_rwsem`` in shared mode.
+
+Block Map Reporting (FIBMAP)
+----------------------------
+
+``iomap_bmap`` implements FIBMAP.
+The calling conventions are the same as for FIEMAP.
+This function is only provided to maintain compatibility for filesystems
+that implemented FIBMAP prior to conversion.
+This ioctl is deprecated; do not add a FIBMAP implementation to
+filesystems that do not have it.
+Callers should probably hold ``i_rwsem`` in shared mode, but this is
+unclear.
+
+Porting Guide
+=============
+
+Why Convert to iomap?
+---------------------
+
+There are several reasons to convert a filesystem to iomap:
+
+ 1. The classic Linux I/O path is not terribly efficient.
+ Pagecache operations lock a single base page at a time and then call
+ into the filesystem to return a mapping for only that page.
+ Direct I/O operations build I/O requests a single file block at a
+ time.
+ This worked well enough for direct/indirect-mapped filesystems such
+ as ext2, but is very inefficient for extent-based filesystems such
+ as XFS.
+
+ 2. Large folios are only supported via iomap; there are no plans to
+ convert the old buffer_head path to use them.
+
+ 3. Direct access to storage on memory-like devices (fsdax) is only
+ supported via iomap.
+
+ 4. Lower maintenance overhead for individual filesystem maintainers.
+ iomap handles common pagecache related operations itself, such as
+ allocating, instantiating, locking, and unlocking of folios.
+ No ->write_begin(), ->write_end() or direct_IO
+ address_space_operations are required to be implemented by
+ filesystem using iomap.
+
+How to Convert to iomap?
+------------------------
+
+First, add ``#include <linux/iomap.h>`` from your source code and add
+``select FS_IOMAP`` to your filesystem's Kconfig option.
+Build the kernel, run fstests with the ``-g all`` option across a wide
+variety of your filesystem's supported configurations to build a
+baseline of which tests pass and which ones fail.
+
+The recommended approach is first to implement ``->iomap_begin`` (and
+``->iomap->end`` if necessary) to allow iomap to obtain a read-only
+mapping of a file range.
+In most cases, this is a relatively trivial conversion of the existing
+``get_block()`` function for read-only mappings.
+``FS_IOC_FIEMAP`` is a good first target because it is trivial to
+implement support for it and then to determine that the extent map
+iteration is correct from userspace.
+If FIEMAP is returning the correct information, it's a good sign that
+other read-only mapping operations will do the right thing.
+
+Next, modify the filesystem's ``get_block(create = false)``
+implementation to use the new ``->iomap_begin`` implementation to map
+file space for selected read operations.
+Hide behind a debugging knob the ability to switch on the iomap mapping
+functions for selected call paths.
+It is necessary to write some code to fill out the bufferhead-based
+mapping information from the ``iomap`` structure, but the new functions
+can be tested without needing to implement any iomap APIs.
+
+Once the read-only functions are working like this, convert each high
+level file operation one by one to use iomap native APIs instead of
+going through ``get_block()``.
+Done one at a time, regressions should be self evident.
+You *do* have a regression test baseline for fstests, right?
+It is suggested to convert swap file activation, ``SEEK_DATA``, and
+``SEEK_HOLE`` before tackling the I/O paths.
+A likely complexity at this point will be converting the buffered read
+I/O path because of bufferheads.
+The buffered read I/O paths doesn't need to be converted yet, though the
+direct I/O read path should be converted in this phase.
+
+At this point, you should look over your ``->iomap_begin`` function.
+If it switches between large blocks of code based on dispatching of the
+``flags`` argument, you should consider breaking it up into
+per-operation iomap ops with smaller, more cohesive functions.
+XFS is a good example of this.
+
+The next thing to do is implement ``get_blocks(create == true)``
+functionality in the ``->iomap_begin``/``->iomap_end`` methods.
+It is strongly recommended to create separate mapping functions and
+iomap ops for write operations.
+Then convert the direct I/O write path to iomap, and start running fsx
+w/ DIO enabled in earnest on filesystem.
+This will flush out lots of data integrity corner case bugs that the new
+write mapping implementation introduces.
+
+Now, convert any remaining file operations to call the iomap functions.
+This will get the entire filesystem using the new mapping functions, and
+they should largely be debugged and working correctly after this step.
+
+Most likely at this point, the buffered read and write paths will still
+to be converted.
+The mapping functions should all work correctly, so all that needs to be
+done is rewriting all the code that interfaces with bufferheads to
+interface with iomap and folios.
+It is much easier first to get regular file I/O (without any fancy
+features like fscrypt, fsverity, compression, or data=journaling)
+converted to use iomap.
+Some of those fancy features (fscrypt and compression) aren't
+implemented yet in iomap.
+For unjournalled filesystems that use the pagecache for symbolic links
+and directories, you might also try converting their handling to iomap.
+
+The rest is left as an exercise for the reader, as it will be different
+for every filesystem.
+If you encounter problems, email the people and lists in
+``get_maintainers.pl`` for help.
+
+Bugs and Limitations
+====================
+
+ * No support for fscrypt.
+ * No support for compression.
+ * No support for fsverity yet.
+ * Strong assumptions that IO should work the way it does on XFS.
+ * Does iomap *actually* work for non-regular file data?
+
+Patches welcome!
diff --git a/MAINTAINERS b/MAINTAINERS
index 8754ac2c259d..2ddd94d43ecf 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8483,6 +8483,7 @@ R: Darrick J. Wong <[email protected]>
L: [email protected]
L: [email protected]
S: Supported
+F: Documentation/filesystems/iomap.txt
F: fs/iomap/
F: include/linux/iomap.h
On Fri, Jun 07, 2024 at 05:17:07PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <[email protected]>
>
> This is the fourth attempt at documenting the design of iomap and how to
The number of attempts should go out of the final commit version..
> port filesystems to use it. Apologies for all the rst formatting, but
> it's necessary to distinguish code from regular text.
Maybe we should do this as a normal text file and not rst then?
> +.. SPDX-License-Identifier: GPL-2.0
> +.. _iomap:
> +
> +..
> + Dumb style notes to maintain the author's sanity:
> + Please try to start sentences on separate lines so that
> + sentence changes don't bleed colors in diff.
> + Heading decorations are documented in sphinx.rst.
Should this be in the document and not a README in the directory?
That being said starting every sentence on a new line makes the text
really hard to read. To the point that I'll really need to go off
and reformat it before making it beyond the first few paragraphs.
I'll try to do that and will return to it later, sorry for just
dropping these procedural notes for now.
On Sat, Jun 08, 2024 at 11:36:30PM -0700, Christoph Hellwig wrote:
> On Fri, Jun 07, 2024 at 05:17:07PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <[email protected]>
> >
> > This is the fourth attempt at documenting the design of iomap and how to
>
> The number of attempts should go out of the final commit version..
>
> > port filesystems to use it. Apologies for all the rst formatting, but
> > it's necessary to distinguish code from regular text.
>
> Maybe we should do this as a normal text file and not rst then?
>
> > +.. SPDX-License-Identifier: GPL-2.0
> > +.. _iomap:
> > +
> > +..
> > + Dumb style notes to maintain the author's sanity:
> > + Please try to start sentences on separate lines so that
> > + sentence changes don't bleed colors in diff.
> > + Heading decorations are documented in sphinx.rst.
>
> Should this be in the document and not a README in the directory?
>
> That being said starting every sentence on a new line makes the text
> really hard to read. To the point that I'll really need to go off
> and reformat it before making it beyond the first few paragraphs.
> I'll try to do that and will return to it later, sorry for just
> dropping these procedural notes for now.
HTML version here, text version below.
https://djwong.org/docs/iomap.html
--D
VFS iomap Design and Porting
Introduction
iomap is a filesystem library for handling various filesystem
operations that involves mapping of file's logical offset ranges
to physical extents. This origins of this library is the file I/O
path that XFS once used; it has now been extended to cover several
other operations. The library provides various APIs for
implementing various file and pagecache operations, such as:
* Pagecache reads and writes
* Folio write faults to the pagecache
* Writeback of dirty folios
* Direct I/O reads and writes
* FIEMAP
* lseek SEEK_DATA and SEEK_HOLE
* swapfile activation
Who Should Read This?
The target audience for this document are filesystem, storage, and
pagecache programmers and code reviewers. The goal of this
document is to provide a brief discussion of the design and
capabilities of iomap, followed by a more detailed catalog of the
interfaces presented by iomap. If you change iomap, please update
this design document.
But Why?
Unlike the classic Linux I/O model which breaks file I/O into
small units (generally memory pages or blocks) and looks up space
mappings on the basis of that unit, the iomap model asks the
filesystem for the largest space mappings that it can create for a
given file operation and initiates operations on that basis. This
strategy improves the filesystem's visibility into the size of the
operation being performed, which enables it to combat
fragmentation with larger space allocations when possible. Larger
space mappings improve runtime performance by amortizing the cost
of a mapping function call into the filesystem across a larger
amount of data.
At a high level, an iomap operation looks like this:
1. For each byte in the operation range...
1. Obtain space mapping via ->iomap_begin
2. For each sub-unit of work...
1. Revalidate the mapping and go back to (1) above, if
necessary
2. Do the work
3. Increment operation cursor
4. Release the mapping via ->iomap_end, if necessary
Each iomap operation will be covered in more detail below. This
library was covered previously by an LWN article and a
KernelNewbies page.
Data Structures and Algorithms
Definitions
* bufferhead: Shattered remnants of the old buffer cache.
* fsblock: The block size of a file, also known as
i_blocksize.
* i_rwsem: The VFS struct inode rwsemaphore.
* invalidate_lock: The pagecache struct address_space
rwsemaphore that protects against folio removal.
struct iomap_ops
Every iomap function requires the filesystem to pass an operations
structure to obtain a mapping and (optionally) to release the
mapping.
struct iomap_ops {
int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
unsigned flags, struct iomap *iomap,
struct iomap *srcmap);
int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
ssize_t written, unsigned flags,
struct iomap *iomap);
};
The ->iomap_begin function is called to obtain one mapping for the
range of bytes specified by pos and length for the file inode.
Each iomap operation describes the requested operation through the
flags argument. The exact value of flags will be documented in the
operation-specific sections below, but these principles apply
generally:
* For a write operation, IOMAP_WRITE will be set. Filesystems
must not return IOMAP_HOLE mappings.
* For any other operation, IOMAP_WRITE will not be set.
* For any operation targetting direct access to storage
(fsdax), IOMAP_DAX will be set.
If it is necessary to read existing file contents from a different
device or address range on a device, the filesystem should return
that information via srcmap. Only pagecache and fsdax operations
support reading from one mapping and writing to another.
After the operation completes, the ->iomap_end function, if
present, is called to signal that iomap is finished with a
mapping. Typically, implementations will use this function to tear
down any context that were set up in ->iomap_begin. For example, a
write might wish to commit the reservations for the bytes that
were operated upon and unreserve any space that was not operated
upon. written might be zero if no bytes were touched. flags will
contain the same value passed to ->iomap_begin. iomap ops for
reads are not likely to need to supply this function.
Both functions should return a negative errno code on error, or
zero.
struct iomap
The filesystem returns the mappings via the following structure.
For documentation purposes, the structure has been reordered to
group fields that go together logically.
struct iomap {
loff_t offset;
u64 length;
u16 type;
u16 flags;
u64 addr;
struct block_device *bdev;
struct dax_device *dax_dev;
void *inline_data;
void *private;
const struct iomap_folio_ops *folio_ops;
u64 validity_cookie;
};
The information is useful for translating file operations into
action. The actions taken are specific to the target of the
operation, such as disk cache, physical storage devices, or
another part of the kernel.
* offset and length describe the range of file offsets, in
bytes, covered by this mapping. These fields must always be
set by the filesystem.
* type describes the type of the space mapping:
* IOMAP_HOLE: No storage has been allocated. This type
must never be returned in response to an IOMAP_WRITE
operation because writes must allocate and map space,
and return the mapping. The addr field must be set to
IOMAP_NULL_ADDR. iomap does not support writing
(whether via pagecache or direct I/O) to a hole.
* IOMAP_DELALLOC: A promise to allocate space at a later
time ("delayed allocation"). If the filesystem returns
IOMAP_F_NEW here and the write fails, the ->iomap_end
function must delete the reservation. The addr field
must be set to IOMAP_NULL_ADDR.
* IOMAP_MAPPED: The file range maps to specific space on
the storage device. The device is returned in bdev or
dax_dev. The device address, in bytes, is returned via
addr.
* IOMAP_UNWRITTEN: The file range maps to specific space
on the storage device, but the space has not yet been
initialized. The device is returned in bdev or dax_dev.
The device address, in bytes, is returned via addr.
Reads will return zeroes to userspace. For a write or
writeback operation, the ioend should update the
mapping to MAPPED.
* IOMAP_INLINE: The file range maps to the memory buffer
specified by inline_data. For write operation, the
->iomap_end function presumably handles persisting the
data. The addr field must be set to IOMAP_NULL_ADDR.
* flags describe the status of the space mapping. These flags
should be set by the filesystem in ->iomap_begin:
* IOMAP_F_NEW: The space under the mapping is newly
allocated. Areas that will not be written to must be
zeroed. If a write fails and the mapping is a space
reservation, the reservation must be deleted.
* IOMAP_F_DIRTY: The inode will have uncommitted metadata
needed to access any data written. fdatasync is
required to commit these changes to persistent storage.
This needs to take into account metadata changes that
may be made at I/O completion, such as file size
updates from direct I/O.
* IOMAP_F_SHARED: The space under the mapping is shared.
Copy on write is necessary to avoid corrupting other
file data.
* IOMAP_F_BUFFER_HEAD: This mapping requires the use of
buffer heads for pagecache operations. Do not add more
uses of this.
* IOMAP_F_MERGED: Multiple contiguous block mappings were
coalesced into this single mapping. This is only useful
for FIEMAP.
* IOMAP_F_XATTR: The mapping is for extended attribute
data, not regular file data. This is only useful for
FIEMAP.
* IOMAP_F_PRIVATE: Starting with this value, the upper
bits can be set by the filesystem for its own purposes.
These flags can be set by iomap itself during file
operations. The filesystem should supply an ->iomap_end
function to observe these flags:
* IOMAP_F_SIZE_CHANGED: The file size has changed as a
result of using this mapping.
* IOMAP_F_STALE: The mapping was found to be stale. iomap
will call ->iomap_end on this mapping and then
->iomap_begin to obtain a new mapping.
Currently, these flags are only set by pagecache operations.
* addr describes the device address, in bytes.
* bdev describes the block device for this mapping. This only
needs to be set for mapped or unwritten operations.
* dax_dev describes the DAX device for this mapping. This only
needs to be set for mapped or unwritten operations, and only
for a fsdax operation.
* inline_data points to a memory buffer for I/O involving
IOMAP_INLINE mappings. This value is ignored for all other
mapping types.
* private is a pointer to filesystem-private information. This
value will be passed unchanged to ->iomap_end.
* folio_ops will be covered in the section on pagecache
operations.
* validity_cookie is a magic freshness value set by the
filesystem that should be used to detect stale mappings. For
pagecache operations this is critical for correct operation
because page faults can occur, which implies that filesystem
locks should not be held between ->iomap_begin and
->iomap_end. Filesystems with completely static mappings
need not set this value. Only pagecache operations
revalidate mappings.
XXX: Should fsdax revalidate as well?
Validation
NOTE: iomap only handles mapping and I/O. Filesystems must still
call out to the VFS to check input parameters and file state
before initiating an I/O operation. It does not handle updating of
timestamps, stripping privileges, or access control.
Locking Hierarchy
iomap requires that filesystems provide their own locking. There
are no locks within iomap itself, though in the course of an
operation iomap may take other locks (e.g. folio/dax locks) as
part of an I/O operation. Locking with iomap can be split into two
categories: above and below iomap.
The upper level of lock must coordinate the iomap operation with
other iomap operations. Generally, the filesystem must take
VFS/pagecache locks such as i_rwsem or invalidate_lock before
calling into iomap. The exact locking requirements are specific to
the type of operation.
The lower level of lock must coordinate access to the mapping
information. This lock is filesystem specific and should be held
during ->iomap_begin while sampling the mapping and validity
cookie.
The general locking hierarchy in iomap is:
* VFS or pagecache lock
* Internal filesystem specific mapping lock
* iomap operation-specific lock
The exact locking requirements are specific to the filesystem; for
certain operations, some of these locks can be elided. All further
mention of locking are recommendations, not mandates. Each
filesystem author must figure out the locking for themself.
iomap Operations
Below are a discussion of the file operations that iomap
implements.
Buffered I/O
Buffered I/O is the default file I/O path in Linux. File contents
are cached in memory ("pagecache") to satisfy reads and writes.
Dirty cache will be written back to disk at some point that can be
forced via fsync and variants.
iomap implements nearly all the folio and pagecache management
that filesystems once had to implement themselves. This means that
the filesystem need not know the details of allocating, mapping,
managing uptodate and dirty state, or writeback of pagecache
folios. Unless the filesystem explicitly opts in to buffer heads,
they will not be used, which makes buffered I/O much more
efficient, and willy much happier.
struct address_space_operations
The following iomap functions can be referenced directly from the
address space operations structure:
* iomap_dirty_folio
* iomap_release_folio
* iomap_invalidate_folio
* iomap_is_partially_uptodate
The following address space operations can be wrapped easily:
* read_folio
* readahead
* writepages
* bmap
* swap_activate
struct iomap_folio_ops
The ->iomap_begin function for pagecache operations may set the
struct iomap::folio_ops field to an ops structure to override
default behaviors of iomap:
struct iomap_folio_ops {
struct folio *(*get_folio)(struct iomap_iter *iter, loff_t pos,
unsigned len);
void (*put_folio)(struct inode *inode, loff_t pos, unsigned
copied,
struct folio *folio);
bool (*iomap_valid)(struct inode *inode, const struct iomap
*iomap);
};
iomap calls these functions:
* get_folio: Called to allocate and return an active reference
to a locked folio prior to starting a write. If this
function is not provided, iomap will call iomap_get_folio.
This could be used to set up per-folio filesystem state for
a write.
* put_folio: Called to unlock and put a folio after a
pagecache operation completes. If this function is not
provided, iomap will folio_unlock and folio_put on its own.
This could be used to commit per-folio filesystem state that
was set up by ->get_folio.
* iomap_valid: The filesystem may not hold locks between
->iomap_begin and ->iomap_end because pagecache operations
can take folio locks, fault on userspace pages, initiate
writeback for memory reclamation, or engage in other
time-consuming actions. If a file's space mapping data are
mutable, it is possible that the mapping for a particular
pagecache folio can change in the time it takes to allocate,
install, and lock that folio. For such files, the mapping
must be revalidated after the folio lock has been taken so
that iomap can manage the folio correctly. The filesystem's
->iomap_begin function must sample a sequence counter into
struct iomap::validity_cookie at the same time that it
populates the mapping fields. It must then provide a
->iomap_valid function to compare the validity cookie
against the source counter and return whether or not the
mapping is still valid. If the mapping is not valid, the
mapping will be sampled again.
These struct kiocb flags are significant for buffered I/O with
iomap:
* IOCB_NOWAIT: Only proceed with the I/O if mapping data are
already in memory, we do not have to initiate other I/O, and
we acquire all filesystem locks without blocking. Neither
this flag nor its definition RWF_NOWAIT actually define what
this flag means, so this is the best the author could come
up with.
Internal per-Folio State
If the fsblock size matches the size of a pagecache folio, it is
assumed that all disk I/O operations will operate on the entire
folio. The uptodate (memory contents are at least as new as what's
on disk) and dirty (memory contents are newer than what's on disk)
status of the folio are all that's needed for this case.
If the fsblock size is less than the size of a pagecache folio,
iomap tracks the per-fsblock uptodate and dirty state itself. This
enables iomap to handle both "bs < ps" filesystems and large
folios in the pagecache.
iomap internally tracks two state bits per fsblock:
* uptodate: iomap will try to keep folios fully up to date. If
there are read(ahead) errors, those fsblocks will not be
marked uptodate. The folio itself will be marked uptodate
when all fsblocks within the folio are uptodate.
* dirty: iomap will set the per-block dirty state when
programs write to the file. The folio itself will be marked
dirty when any fsblock within the folio is dirty.
iomap also tracks the amount of read and write disk IOs that are
in flight. This structure is much lighter weight than struct
buffer_head.
Filesystems wishing to turn on large folios in the pagecache
should call mapping_set_large_folios when initializing the incore
inode.
Readahead and Reads
The iomap_readahead function initiates readahead to the pagecache.
The iomap_read_folio function reads one folio's worth of data into
the pagecache. The flags argument to ->iomap_begin will be set to
zero. The pagecache takes whatever locks it needs before calling
the filesystem.
Writes
The iomap_file_buffered_write function writes an iocb to the
pagecache. IOMAP_WRITE or IOMAP_WRITE | IOMAP_NOWAIT will be
passed as the flags argument to ->iomap_begin. Callers commonly
take i_rwsem in either shared or exclusive mode.
mmap Write Faults
The iomap_page_mkwrite function handles a write fault to a folio
the pagecache. IOMAP_WRITE | IOMAP_FAULT will be passed as the
flags argument to ->iomap_begin. Callers commonly take the mmap
invalidate_lock in shared or exclusive mode.
Write Failures
After a short write to the pagecache, the areas not written will
not become marked dirty. The filesystem must arrange to cancel
such reservations because writeback will not consume the
reservation. The iomap_file_buffered_write_punch_delalloc can be
called from a ->iomap_end function to find all the clean areas of
the folios caching a fresh (IOMAP_F_NEW) delalloc mapping. It
takes the invalidate_lock.
The filesystem should supply a callback punch will be called for
each file range in this state. This function must only remove
delayed allocation reservations, in case another thread racing
with the current thread writes successfully to the same region and
triggers writeback to flush the dirty data out to disk.
Truncation
Filesystems can call iomap_truncate_page to zero the bytes in the
pagecache from EOF to the end of the fsblock during a file
truncation operation. truncate_setsize or truncate_pagecache will
take care of everything after the EOF block. IOMAP_ZERO will be
passed as the flags argument to ->iomap_begin. Callers typically
take i_rwsem and invalidate_lock in exclusive mode.
Zeroing for File Operations
Filesystems can call iomap_zero_range to perform zeroing of the
pagecache for non-truncation file operations that are not aligned
to the fsblock size. IOMAP_ZERO will be passed as the flags
argument to ->iomap_begin. Callers typically take i_rwsem and
invalidate_lock in exclusive mode.
Unsharing Reflinked File Data
Filesystems can call iomap_file_unshare to force a file sharing
storage with another file to preemptively copy the shared data to
newly allocate storage. IOMAP_WRITE | IOMAP_UNSHARE will be passed
as the flags argument to ->iomap_begin. Callers typically take
i_rwsem and invalidate_lock in exclusive mode.
Writeback
Filesystems can call iomap_writepages to respond to a request to
write dirty pagecache folios to disk. The mapping and wbc
parameters should be passed unchanged. The wpc pointer should be
allocated by the filesystem and must be initialized to zero.
The pagecache will lock each folio before trying to schedule it
for writeback. It does not lock i_rwsem or invalidate_lock.
The dirty bit will be cleared for all folios run through the
->map_blocks machinery described below even if the writeback
fails. This is to prevent dirty folio clots when storage devices
fail; an -EIO is recorded for userspace to collect via fsync.
The ops structure must be specified and is as follows:
struct iomap_writeback_ops
struct iomap_writeback_ops {
int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode
*inode,
loff_t offset, unsigned len);
int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
void (*discard_folio)(struct folio *folio, loff_t pos);
};
The fields are as follows:
* map_blocks: Sets wpc->iomap to the space mapping of the file
range (in bytes) given by offset and len. iomap calls this
function for each fs block in each dirty folio, even if the
mapping returned is longer than one fs block. Do not return
IOMAP_INLINE mappings here; the ->iomap_end function must
deal with persisting written data. Filesystems can skip a
potentially expensive mapping lookup if the mappings have
not changed. This revalidation must be open-coded by the
filesystem; it is unclear if iomap::validity_cookie can be
reused for this purpose. This function is required.
* prepare_ioend: Enables filesystems to transform the
writeback ioend or perform any other prepatory work before
the writeback I/O is submitted. A filesystem can override
the ->bi_end_io function for its own purposes, such as
kicking the ioend completion to a workqueue if the bio is
completed in interrupt context. This function is optional.
* discard_folio: iomap calls this function after ->map_blocks
fails schedule I/O for any part of a dirty folio. The
function should throw away any reservations that may have
been made for the write. The folio will be marked clean and
an -EIO recorded in the pagecache. Filesystems can use this
callback to remove delalloc reservations to avoid having
delalloc reservations for clean pagecache. This function is
optional.
Writeback ioend Completion
iomap creates chains of struct iomap_ioend objects that wrap the
bio that is used to write pagecache data to disk. By default,
iomap finishes writeback ioends by clearing the writeback bit on
the folios attached to the ioend. If the write failed, it will
also set the error bits on the folios and the address space. This
can happen in interrupt or process context, depending on the
storage device.
Filesystems that need to update internal bookkeeping (e.g.
unwritten extent conversions) should provide a ->prepare_ioend
function to override the struct iomap_end::bio::bi_end_io with its
own function. This function should call iomap_finish_ioends after
finishing its own work.
Some filesystems may wish to amortize the cost of running metadata
transactions for post-writeback updates by batching them. They may
also require transactions to run from process context, which
implies punting batches to a workqueue. iomap ioends contain a
list_head to enable batching.
Given a batch of ioends, iomap has a few helpers to assist with
amortization:
* iomap_sort_ioends: Sort all the ioends in the list by file
offset.
* iomap_ioend_try_merge: Given an ioend that is not in any
list and a separate list of sorted ioends, merge as many of
the ioends from the head of the list into the given ioend.
ioends can only be merged if the file range and storage
addresses are contiguous; the unwritten and shared status
are the same; and the write I/O outcome is the same. The
merged ioends become their own list.
* iomap_finish_ioends: Finish an ioend that possibly has other
ioends linked to it.
Direct I/O
In Linux, direct I/O is defined as file I/O that is issued
directly to storage, bypassing the pagecache.
The iomap_dio_rw function implements O_DIRECT (direct I/O) reads
and writes for files. An optional ops parameter can be passed to
change the behavior of direct I/O. The done_before parameter
should be set if writes have been initiated prior to the call. The
direction of the I/O is determined from the iocb passed in.
The flags argument can be any of the following values:
* IOMAP_DIO_FORCE_WAIT: Wait for the I/O to complete even if
the kiocb is not synchronous.
* IOMAP_DIO_OVERWRITE_ONLY: Allocating blocks, zeroing partial
blocks, and extensions of the file size are not allowed. The
entire file range must to map to a single written or
unwritten extent. This flag exists to enable issuing
concurrent direct IOs with only the shared i_rwsem held when
the file I/O range is not aligned to the filesystem block
size. -EAGAIN will be returned if the operation cannot
proceed.
* IOMAP_DIO_PARTIAL: If a page fault occurs, return whatever
progress has already been made. The caller may deal with the
page fault and retry the operation.
These struct kiocb flags are significant for direct I/O with
iomap:
* IOCB_NOWAIT: Only proceed with the I/O if mapping data are
already in memory, we do not have to initiate other I/O, and
we acquire all filesystem locks without blocking.
* IOCB_SYNC: Ensure that the device has persisted data to disk
before completing the call. In the case of pure overwrites,
the I/O may be issued with FUA enabled.
* IOCB_HIPRI: Poll for I/O completion instead of waiting for
an interrupt. Only meaningful for asynchronous I/O, and only
if the entire I/O can be issued as a single struct bio.
* IOCB_DIO_CALLER_COMP: Try to run I/O completion from the
caller's process context. See linux/fs.h for more details.
Filesystems should call iomap_dio_rw from ->read_iter and
->write_iter, and set FMODE_CAN_ODIRECT in the ->open function for
the file. They should not set ->direct_IO, which is deprecated.
If a filesystem wishes to perform its own work before direct I/O
completion, it should call __iomap_dio_rw. If its return value is
not an error pointer or a NULL pointer, the filesystem should pass
the return value to iomap_dio_complete after finishing its
internal work.
Direct Reads
A direct I/O read initiates a read I/O from the storage device to
the caller's buffer. Dirty parts of the pagecache are flushed to
storage before initiating the read io. The flags value for
->iomap_begin will be IOMAP_DIRECT with any combination of the
following enhancements:
* IOMAP_NOWAIT: Read if mapping data are already in memory.
Does not initiate other I/O or block on filesystem locks.
Callers commonly hold i_rwsem in shared mode.
Direct Writes
A direct I/O write initiates a write I/O to the storage device to
the caller's buffer. Dirty parts of the pagecache are flushed to
storage before initiating the write io. The pagecache is
invalidated both before and after the write io. The flags value
for ->iomap_begin will be IOMAP_DIRECT | IOMAP_WRITE with any
combination of the following enhancements:
* IOMAP_NOWAIT: Write if mapping data are already in memory.
Does not initiate other I/O or block on filesystem locks.
* IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
blocks is not allowed. The entire file range must to map to
a single written or unwritten extent. The file I/O range
must be aligned to the filesystem block size.
Callers commonly hold i_rwsem in shared or exclusive mode.
struct iomap_dio_ops:
struct iomap_dio_ops {
void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
loff_t file_offset);
int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
unsigned flags);
struct bio_set *bio_set;
};
The fields of this structure are as follows:
* submit_io: iomap calls this function when it has constructed
a struct bio object for the I/O requested, and wishes to
submit it to the block device. If no function is provided,
submit_bio will be called directly. Filesystems that would
like to perform additional work before (e.g. data
replication for btrfs) should implement this function.
* end_io: This is called after the struct bio completes. This
function should perform post-write conversions of unwritten
extent mappings, handle write failures, etc. The flags
argument may be set to a combination of the following:
* IOMAP_DIO_UNWRITTEN: The mapping was unwritten, so the
ioend should mark the extent as written.
* IOMAP_DIO_COW: Writing to the space in the mapping
required a copy on write operation, so the ioend should
switch mappings.
* bio_set: This allows the filesystem to provide a custom
bio_set for allocating direct I/O bios. This enables
filesystems to stash additional per-bio information for
private use. If this field is NULL, generic struct bio
objects will be used.
Filesystems that want to perform extra work after an I/O
completion should set a custom ->bi_end_io function via
->submit_io. Afterwards, the custom endio function must call
iomap_dio_bio_end_io to finish the direct I/O.
DAX I/O
Storage devices that can be directly mapped as memory support a
new access mode known as "fsdax".
fsdax Reads
A fsdax read performs a memcpy from storage device to the caller's
buffer. The flags value for ->iomap_begin will be IOMAP_DAX with
any combination of the following enhancements:
* IOMAP_NOWAIT: Read if mapping data are already in memory.
Does not initiate other I/O or block on filesystem locks.
Callers commonly hold i_rwsem in shared mode.
fsdax Writes
A fsdax write initiates a memcpy to the storage device to the
caller's buffer. The flags value for ->iomap_begin will be
IOMAP_DAX | IOMAP_WRITE with any combination of the following
enhancements:
* IOMAP_NOWAIT: Write if mapping data are already in memory.
Does not initiate other I/O or block on filesystem locks.
* IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
blocks is not allowed. The entire file range must to map to
a single written or unwritten extent. The file I/O range
must be aligned to the filesystem block size.
Callers commonly hold i_rwsem in exclusive mode.
mmap Faults
The dax_iomap_fault function handles read and write faults to
fsdax storage. For a read fault, IOMAP_DAX | IOMAP_FAULT will be
passed as the flags argument to ->iomap_begin. For a write fault,
IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE will be passed as the flags
argument to ->iomap_begin.
Callers commonly hold the same locks as they do to call their
iomap pagecache counterparts.
Truncation, fallocate, and Unsharing
For fsdax files, the following functions are provided to replace
their iomap pagecache I/O counterparts. The flags argument to
->iomap_begin are the same as the pagecache counterparts, with
IOMAP_DIO added.
* dax_file_unshare
* dax_zero_range
* dax_truncate_page
Callers commonly hold the same locks as they do to call their
iomap pagecache counterparts.
SEEK_DATA
The iomap_seek_data function implements the SEEK_DATA "whence"
value for llseek. IOMAP_REPORT will be passed as the flags
argument to ->iomap_begin.
For unwritten mappings, the pagecache will be searched. Regions of
the pagecache with a folio mapped and uptodate fsblocks within
those folios will be reported as data areas.
Callers commonly hold i_rwsem in shared mode.
SEEK_HOLE
The iomap_seek_hole function implements the SEEK_HOLE "whence"
value for llseek. IOMAP_REPORT will be passed as the flags
argument to ->iomap_begin.
For unwritten mappings, the pagecache will be searched. Regions of
the pagecache with no folio mapped, or a !uptodate fsblock within
a folio will be reported as sparse hole areas.
Callers commonly hold i_rwsem in shared mode.
Swap File Activation
The iomap_swapfile_activate function finds all the base-page
aligned regions in a file and sets them up as swap space. The file
will be fsync()'d before activation. IOMAP_REPORT will be passed
as the flags argument to ->iomap_begin. All mappings must be
mapped or unwritten; cannot be dirty or shared, and cannot span
multiple block devices. Callers must hold i_rwsem in exclusive
mode; this is already provided by swapon.
Extent Map Reporting (FS_IOC_FIEMAP)
The iomap_fiemap function exports file extent mappings to
userspace in the format specified by the FS_IOC_FIEMAP ioctl.
IOMAP_REPORT will be passed as the flags argument to
->iomap_begin. Callers commonly hold i_rwsem in shared mode.
Block Map Reporting (FIBMAP)
iomap_bmap implements FIBMAP. The calling conventions are the same
as for FIEMAP. This function is only provided to maintain
compatibility for filesystems that implemented FIBMAP prior to
conversion. This ioctl is deprecated; do not add a FIBMAP
implementation to filesystems that do not have it. Callers should
probably hold i_rwsem in shared mode, but this is unclear.
Porting Guide
Why Convert to iomap?
There are several reasons to convert a filesystem to iomap:
1. The classic Linux I/O path is not terribly efficient.
Pagecache operations lock a single base page at a time and
then call into the filesystem to return a mapping for only
that page. Direct I/O operations build I/O requests a single
file block at a time. This worked well enough for
direct/indirect-mapped filesystems such as ext2, but is very
inefficient for extent-based filesystems such as XFS.
2. Large folios are only supported via iomap; there are no
plans to convert the old buffer_head path to use them.
3. Direct access to storage on memory-like devices (fsdax) is
only supported via iomap.
4. Lower maintenance overhead for individual filesystem
maintainers. iomap handles common pagecache related
operations itself, such as allocating, instantiating,
locking, and unlocking of folios. No ->write_begin(),
->write_end() or direct_IO address_space_operations are
required to be implemented by filesystem using iomap.
How to Convert to iomap?
First, add #include <linux/iomap.h> from your source code and add
select FS_IOMAP to your filesystem's Kconfig option. Build the
kernel, run fstests with the -g all option across a wide variety
of your filesystem's supported configurations to build a baseline
of which tests pass and which ones fail.
The recommended approach is first to implement ->iomap_begin (and
->iomap->end if necessary) to allow iomap to obtain a read-only
mapping of a file range. In most cases, this is a relatively
trivial conversion of the existing get_block() function for
read-only mappings. FS_IOC_FIEMAP is a good first target because
it is trivial to implement support for it and then to determine
that the extent map iteration is correct from userspace. If FIEMAP
is returning the correct information, it's a good sign that other
read-only mapping operations will do the right thing.
Next, modify the filesystem's get_block(create = false)
implementation to use the new ->iomap_begin implementation to map
file space for selected read operations. Hide behind a debugging
knob the ability to switch on the iomap mapping functions for
selected call paths. It is necessary to write some code to fill
out the bufferhead-based mapping information from the iomap
structure, but the new functions can be tested without needing to
implement any iomap APIs.
Once the read-only functions are working like this, convert each
high level file operation one by one to use iomap native APIs
instead of going through get_block(). Done one at a time,
regressions should be self evident. You do have a regression test
baseline for fstests, right? It is suggested to convert swap file
activation, SEEK_DATA, and SEEK_HOLE before tackling the I/O
paths. A likely complexity at this point will be converting the
buffered read I/O path because of bufferheads. The buffered read
I/O paths doesn't need to be converted yet, though the direct I/O
read path should be converted in this phase.
At this point, you should look over your ->iomap_begin function.
If it switches between large blocks of code based on dispatching
of the flags argument, you should consider breaking it up into
per-operation iomap ops with smaller, more cohesive functions. XFS
is a good example of this.
The next thing to do is implement get_blocks(create == true)
functionality in the ->iomap_begin/->iomap_end methods. It is
strongly recommended to create separate mapping functions and
iomap ops for write operations. Then convert the direct I/O write
path to iomap, and start running fsx w/ DIO enabled in earnest on
filesystem. This will flush out lots of data integrity corner case
bugs that the new write mapping implementation introduces.
Now, convert any remaining file operations to call the iomap
functions. This will get the entire filesystem using the new
mapping functions, and they should largely be debugged and working
correctly after this step.
Most likely at this point, the buffered read and write paths will
still to be converted. The mapping functions should all work
correctly, so all that needs to be done is rewriting all the code
that interfaces with bufferheads to interface with iomap and
folios. It is much easier first to get regular file I/O (without
any fancy features like fscrypt, fsverity, compression, or
data=journaling) converted to use iomap. Some of those fancy
features (fscrypt and compression) aren't implemented yet in
iomap. For unjournalled filesystems that use the pagecache for
symbolic links and directories, you might also try converting
their handling to iomap.
The rest is left as an exercise for the reader, as it will be
different for every filesystem. If you encounter problems, email
the people and lists in get_maintainers.pl for help.
Bugs and Limitations
* No support for fscrypt.
* No support for compression.
* No support for fsverity yet.
* Strong assumptions that IO should work the way it does on
XFS.
* Does iomap actually work for non-regular file data?
Patches welcome!
Hello Darrick,
"Darrick J. Wong" <[email protected]> writes:
> From: Darrick J. Wong <[email protected]>
>
> This is the fourth attempt at documenting the design of iomap and how to
I agree that this isn't needed in the commit msg ("fourth attempt").
> port filesystems to use it. Apologies for all the rst formatting, but
> it's necessary to distinguish code from regular text.
>
> A lot of this has been collected from various email conversations, code
> comments, commit messages, my own understanding of iomap, and
> Ritesh/Luis' previous efforts to create a document. Please note a large
> part of this has been taken from Dave's reply to last iomap doc
> patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
> other iomap developers who have taken time to explain the iomap design
> in various emails, commits, comments etc.
>
> Cc: Dave Chinner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Christoph Hellwig <[email protected]>
> Cc: Christian Brauner <[email protected]>
> Cc: Ojaswin Mujoo <[email protected]>
> Cc: Jan Kara <[email protected]>
> Cc: Luis Chamberlain <[email protected]>
> Inspired-by: Ritesh Harjani (IBM) <[email protected]>
I am not sure if this is even a valid or accepted tag.
But sure thanks! :)
> Signed-off-by: Darrick J. Wong <[email protected]>
> ---
> Documentation/filesystems/index.rst | 1
> Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
> MAINTAINERS | 1
> 3 files changed, 1062 insertions(+)
> create mode 100644 Documentation/filesystems/iomap.rst
>
> diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
> index 8f5c1ee02e2f..b010cc8df32d 100644
> --- a/Documentation/filesystems/index.rst
> +++ b/Documentation/filesystems/index.rst
> @@ -34,6 +34,7 @@ algorithms work.
> seq_file
> sharedsubtree
> idmappings
> + iomap
>
> automount-support
>
> diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
> new file mode 100644
> index 000000000000..a478b55e4135
> --- /dev/null
> +++ b/Documentation/filesystems/iomap.rst
> @@ -0,0 +1,1060 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. _iomap:
> +
> +..
> + Dumb style notes to maintain the author's sanity:
> + Please try to start sentences on separate lines so that
> + sentence changes don't bleed colors in diff.
> + Heading decorations are documented in sphinx.rst.
> +
> +============================
> +VFS iomap Design and Porting
> +============================
> +
> +.. toctree::
> +
> +Introduction
> +============
> +
> +iomap is a filesystem library for handling various filesystem operations
> +that involves mapping of file's logical offset ranges to physical
> +extents.
> +This origins of this library is the file I/O path that XFS once used; it
> +has now been extended to cover several other operations.
> +The library provides various APIs for implementing various file and
^^^^ redundant "various"
> +pagecache operations, such as:
> +
> + * Pagecache reads and writes
> + * Folio write faults to the pagecache
> + * Writeback of dirty folios
> + * Direct I/O reads and writes
Dax I/O reads and writes.
... as well please?
> + * FIEMAP
> + * lseek ``SEEK_DATA`` and ``SEEK_HOLE``
> + * swapfile activation
> +
> +Who Should Read This?
> +=====================
> +
> +The target audience for this document are filesystem, storage, and
/s/and/,/
> +pagecache programmers and code reviewers.
Not sure if we even need this secion "Who Should Read This".
> +The goal of this document is to provide a brief discussion of the
> +design and capabilities of iomap, followed by a more detailed catalog
> +of the interfaces presented by iomap.
> +If you change iomap, please update this design document.
The details of "goal of this document..." -> can be a part of
separate paragraph in "Introduction" section itself.
> +
> +But Why?
> +========
"Why Iomap?" is more clean IMO.
> +
> +Unlike the classic Linux I/O model which breaks file I/O into small
> +units (generally memory pages or blocks) and looks up space mappings on
> +the basis of that unit, the iomap model asks the filesystem for the
> +largest space mappings that it can create for a given file operation and
> +initiates operations on that basis.
> +This strategy improves the filesystem's visibility into the size of the
> +operation being performed, which enables it to combat fragmentation with
> +larger space allocations when possible.
> +Larger space mappings improve runtime performance by amortizing the cost
> +of a mapping function call into the filesystem across a larger amount of
s/call/calls
> +data.
> +
> +At a high level, an iomap operation `looks like this
> +<https://lore.kernel.org/all/[email protected]/>`_:
> +
> +1. For each byte in the operation range...
> +
> + 1. Obtain space mapping via ->iomap_begin
> + 2. For each sub-unit of work...
> +
> + 1. Revalidate the mapping and go back to (1) above, if necessary
> + 2. Do the work
> +
> + 3. Increment operation cursor
> + 4. Release the mapping via ->iomap_end, if necessary
> +
> +Each iomap operation will be covered in more detail below.
> +This library was covered previously by an `LWN article
> +<https://lwn.net/Articles/935934/>`_ and a `KernelNewbies page
> +<https://kernelnewbies.org/KernelProjects/iomap>`_.
> +
> +Data Structures and Algorithms
> +==============================
> +
> +Definitions
> +-----------
> +
> + * ``bufferhead``: Shattered remnants of the old buffer cache.
> + * ``fsblock``: The block size of a file, also known as ``i_blocksize``.
> + * ``i_rwsem``: The VFS ``struct inode`` rwsemaphore.
> + * ``invalidate_lock``: The pagecache ``struct address_space``
> + rwsemaphore that protects against folio removal.
This definition is a bit confusing & maybe even incomplete.
I think we should use this from header file.
@invalidate_lock: The pagecache ``struct address_sapce`` rwsemaphore
that guards coherency between page cache contents and file offset->disk
block mappings in the filesystem during invalidates. It is also used to
block modification of page cache contents through memory mappings.
Also if we are describing definitions above - then I think we should
also clarify these locks/terms used in this document (I just looked
"lock" related terms for now)
- folio lock:
- dax lock:
- pagecache lock:
- FS internal mapping lock:
- Iomap internal operation lock:
> +
> +struct iomap_ops
> +----------------
IMO, we should define "struct iomap" in the begining. The reason is
because iomap_ops functions take "struct iomap" in it's function
arguments. So it's easier if we describe "struct iomap" before.
> +
> +Every iomap function requires the filesystem to pass an operations
> +structure to obtain a mapping and (optionally) to release the mapping.
> +
> +.. code-block:: c
> +
> + struct iomap_ops {
> + int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
> + unsigned flags, struct iomap *iomap,
> + struct iomap *srcmap);
> +
> + int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
> + ssize_t written, unsigned flags,
> + struct iomap *iomap);
> + };
> +
> +The ``->iomap_begin`` function is called to obtain one mapping for the
> +range of bytes specified by ``pos`` and ``length`` for the file
> +``inode``.
I think it is better if we describe ->iomap_begin and ->iomap_end
in proper sub-sections. Otherwise this looks like we have clobbered
all the information together :)
->iomap_begin
^^^^^^^^^^^^^^^^^
This either returns an existing mapping or reserve/allocates a new
mapping. logical file pos and length are in bytes which gets passed
as function arguments. Filesystem returns the new mapping information
within ``struct iomap`` which also gets passed as a function argument.
Filesystems should provide the details of this mapping by filling
various fields within ``struct iomap``.
@srcmap agument:
Note that ->iomap_begin call has srcmap passed as another argument. This is
mainly used only during the begin phase for COW mappings to identify where
the reads are to be performed from. Filesystems needs to fill that mapping
information if iomap should read data for partially written blocks from a
different location than the write target [4].
@flags argument:
These are the operation types which iomap supports.
IOMAP_WRITE: For doing write I/O.
<...>
IOMAP_ZERO:
IOMAP_REPORT:
IOMAP_FAULT:
IOMAP_DIRECT:
IOMAP_NOWAIT:
IOMAP_OVERWRITE_ONLY:
IOMAP_UNSHARE:
IOMAP_DAX:
->iomap_end
^^^^^^^^^^^^^^^^^
Commit and/or unreserve space which was previously allocated/reserved
in ``->iomap_begin``. For e.g. during buffered-io, when a short writes
occurs, filesystem may need to remove the reserved space that was
allocated during ->iomap_begin.
For filesystems that use delalloc allocation, we may need to punch out
delalloc extents from the range that are not dirty in
the page cache. See comments in
iomap_file_buffered_write_punch_delalloc() for more info [5][6].
(IMHO) I find above definitions more descriptive.
> +
> +Each iomap operation describes the requested operation through the
> +``flags`` argument.
> +The exact value of ``flags`` will be documented in the
> +operation-specific sections below, but these principles apply generally:
> +
> + * For a write operation, ``IOMAP_WRITE`` will be set.
> + Filesystems must not return ``IOMAP_HOLE`` mappings.
> +
> + * For any other operation, ``IOMAP_WRITE`` will not be set.
> +
Direct-io related operation which bypasses pagecache use IOMAP_DIRECT.
> + * For any operation targetting direct access to storage (fsdax),
> + ``IOMAP_DAX`` will be set.
> +
> +If it is necessary to read existing file contents from a `different
> +<https://lore.kernel.org/all/[email protected]/>`_ device or
> +address range on a device, the filesystem should return that information via
> +``srcmap``.
> +Only pagecache and fsdax operations support reading from one mapping and
> +writing to another.
> +
> +After the operation completes, the ``->iomap_end`` function, if present,
> +is called to signal that iomap is finished with a mapping.
> +Typically, implementations will use this function to tear down any
> +context that were set up in ``->iomap_begin``.
> +For example, a write might wish to commit the reservations for the bytes
> +that were operated upon and unreserve any space that was not operated
> +upon.
> +``written`` might be zero if no bytes were touched.
> +``flags`` will contain the same value passed to ``->iomap_begin``.
> +iomap ops for reads are not likely to need to supply this function.
> +
> +Both functions should return a negative errno code on error, or zero.
minor nit: ... or zero on success.
> +
> +struct iomap
> +------------
> +
> +The filesystem returns the mappings via the following structure.
Filesystem returns the contiguous file mapping information of logical
file offset range to a physically mapped extent via the following
structure which iomap uses to perform various file and pagecache
related operations listed above.
> +For documentation purposes, the structure has been reordered to group
> +fields that go together logically.
> +
> +.. code-block:: c
> +
> + struct iomap {
> + loff_t offset;
> + u64 length;
> +
> + u16 type;
> + u16 flags;
> +
> + u64 addr;
> + struct block_device *bdev;
> + struct dax_device *dax_dev;
> + void *inline_data;
> +
> + void *private;
> +
> + const struct iomap_folio_ops *folio_ops;
> +
> + u64 validity_cookie;
> + };
> +
> +The information is useful for translating file operations into action.
> +The actions taken are specific to the target of the operation, such as
> +disk cache, physical storage devices, or another part of the kernel.
I think the wording "action" & trying to make it so generic w/o mapping
what "action" refers here for "disk cache", "physical storage device" or
"other parts of the kernel", gets a bit confusing.
Do you think we should map those to some examples maybe?
BTW, with added definition of "struct iomap" which I mentioned above,
I am even fine if we want to drop this paragraph.
> +
> + * ``offset`` and ``length`` describe the range of file offsets, in
> + bytes, covered by this mapping.
> + These fields must always be set by the filesystem.
> +
> + * ``type`` describes the type of the space mapping:
This field is set by the filesystem in ->iomap_begin call.
> +
> + * **IOMAP_HOLE**: No storage has been allocated.
> + This type must never be returned in response to an IOMAP_WRITE
> + operation because writes must allocate and map space, and return
> + the mapping.
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> + iomap does not support writing (whether via pagecache or direct
> + I/O) to a hole.
> +
> + * **IOMAP_DELALLOC**: A promise to allocate space at a later time
> + ("delayed allocation").
> + If the filesystem returns IOMAP_F_NEW here and the write fails, the
> + ``->iomap_end`` function must delete the reservation.
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> +
> + * **IOMAP_MAPPED**: The file range maps to specific space on the
> + storage device.
> + The device is returned in ``bdev`` or ``dax_dev``.
> + The device address, in bytes, is returned via ``addr``.
> +
> + * **IOMAP_UNWRITTEN**: The file range maps to specific space on the
> + storage device, but the space has not yet been initialized.
> + The device is returned in ``bdev`` or ``dax_dev``.
> + The device address, in bytes, is returned via ``addr``.
> + Reads will return zeroes to userspace.
Reads to this type of mapping will return zeroes to the caller.
> + For a write or writeback operation, the ioend should update the
> + mapping to MAPPED.
Refer to section "Writeback ioend Completion" for more details.
> +
> + * **IOMAP_INLINE**: The file range maps to the memory buffer
> + specified by ``inline_data``.
> + For write operation, the ``->iomap_end`` function presumably
> + handles persisting the data.
Is it? Or do we just mark the inode as dirty?
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> +
> + * ``flags`` describe the status of the space mapping.
> + These flags should be set by the filesystem in ``->iomap_begin``:
> +
> + * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
> + Areas that will not be written to must be zeroed.
In case of DAX, we have to invalidate those existing mappings which
might have a "hole" page mapped.
> + If a write fails and the mapping is a space reservation, the
> + reservation must be deleted.
> +
> + * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
> + to access any data written.
> + fdatasync is required to commit these changes to persistent
> + storage.
> + This needs to take into account metadata changes that *may* be made
> + at I/O completion, such as file size updates from direct I/O.
> +
> + * **IOMAP_F_SHARED**: The space under the mapping is shared.
> + Copy on write is necessary to avoid corrupting other file data.
> +
> + * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
> + heads for pagecache operations.
> + Do not add more uses of this.
> +
> + * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
> + coalesced into this single mapping.
> + This is only useful for FIEMAP.
> +
> + * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
> + regular file data.
> + This is only useful for FIEMAP.
> +
> + * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
> + be set by the filesystem for its own purposes.
> +
> + These flags can be set by iomap itself during file operations.
> + The filesystem should supply an ``->iomap_end`` function to observe
> + these flags:
> +
> + * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
> + using this mapping.
> +
> + * **IOMAP_F_STALE**: The mapping was found to be stale.
> + iomap will call ``->iomap_end`` on this mapping and then
> + ``->iomap_begin`` to obtain a new mapping.
> +
> + Currently, these flags are only set by pagecache operations.
> +
> + * ``addr`` describes the device address, in bytes.
> +
> + * ``bdev`` describes the block device for this mapping.
> + This only needs to be set for mapped or unwritten operations.
> +
> + * ``dax_dev`` describes the DAX device for this mapping.
> + This only needs to be set for mapped or unwritten operations, and
> + only for a fsdax operation.
Looks like we can make this union (bdev and dax_dev). Since depending
upon IOMAP_DAX or not we only set either dax_dev or bdev.
Sorry Darrick. I will stop here for now.
I will continue it from here later.
-ritesh
> +
> + * ``inline_data`` points to a memory buffer for I/O involving
> + ``IOMAP_INLINE`` mappings.
> + This value is ignored for all other mapping types.
> +
> + * ``private`` is a pointer to `filesystem-private information
> + <https://lore.kernel.org/all/[email protected]/>`_.
> + This value will be passed unchanged to ``->iomap_end``.
> +
> + * ``folio_ops`` will be covered in the section on pagecache operations.
> +
> + * ``validity_cookie`` is a magic freshness value set by the filesystem
> + that should be used to detect stale mappings.
> + For pagecache operations this is critical for correct operation
> + because page faults can occur, which implies that filesystem locks
> + should not be held between ``->iomap_begin`` and ``->iomap_end``.
> + Filesystems with completely static mappings need not set this value.
> + Only pagecache operations revalidate mappings.
> +
> + XXX: Should fsdax revalidate as well?
> +
> +Validation
> +==========
> +
> +**NOTE**: iomap only handles mapping and I/O.
> +Filesystems must still call out to the VFS to check input parameters
> +and file state before initiating an I/O operation.
> +It does not handle updating of timestamps, stripping privileges, or
> +access control.
> +
> +Locking Hierarchy
> +=================
> +
> +iomap requires that filesystems provide their own locking.
> +There are no locks within iomap itself, though in the course of an
> +operation iomap may take other locks (e.g. folio/dax locks) as part of
> +an I/O operation.
> +Locking with iomap can be split into two categories: above and below
> +iomap.
> +
> +The upper level of lock must coordinate the iomap operation with other
> +iomap operations.
> +Generally, the filesystem must take VFS/pagecache locks such as
> +``i_rwsem`` or ``invalidate_lock`` before calling into iomap.
> +The exact locking requirements are specific to the type of operation.
> +
> +The lower level of lock must coordinate access to the mapping
> +information.
> +This lock is filesystem specific and should be held during
> +``->iomap_begin`` while sampling the mapping and validity cookie.
> +
> +The general locking hierarchy in iomap is:
> +
> + * VFS or pagecache lock
> +
> + * Internal filesystem specific mapping lock
> +
> + * iomap operation-specific lock
> +
> +The exact locking requirements are specific to the filesystem; for
> +certain operations, some of these locks can be elided.
> +All further mention of locking are *recommendations*, not mandates.
> +Each filesystem author must figure out the locking for themself.
> +
> +iomap Operations
> +================
> +
> +Below are a discussion of the file operations that iomap implements.
> +
> +Buffered I/O
> +------------
> +
> +Buffered I/O is the default file I/O path in Linux.
> +File contents are cached in memory ("pagecache") to satisfy reads and
> +writes.
> +Dirty cache will be written back to disk at some point that can be
> +forced via ``fsync`` and variants.
> +
> +iomap implements nearly all the folio and pagecache management that
> +filesystems once had to implement themselves.
> +This means that the filesystem need not know the details of allocating,
> +mapping, managing uptodate and dirty state, or writeback of pagecache
> +folios.
> +Unless the filesystem explicitly opts in to buffer heads, they will not
> +be used, which makes buffered I/O much more efficient, and ``willy``
> +much happier.
> +
> +struct address_space_operations
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The following iomap functions can be referenced directly from the
> +address space operations structure:
> +
> + * ``iomap_dirty_folio``
> + * ``iomap_release_folio``
> + * ``iomap_invalidate_folio``
> + * ``iomap_is_partially_uptodate``
> +
> +The following address space operations can be wrapped easily:
> +
> + * ``read_folio``
> + * ``readahead``
> + * ``writepages``
> + * ``bmap``
> + * ``swap_activate``
> +
> +struct iomap_folio_ops
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``->iomap_begin`` function for pagecache operations may set the
> +``struct iomap::folio_ops`` field to an ops structure to override
> +default behaviors of iomap:
> +
> +.. code-block:: c
> +
> + struct iomap_folio_ops {
> + struct folio *(*get_folio)(struct iomap_iter *iter, loff_t pos,
> + unsigned len);
> + void (*put_folio)(struct inode *inode, loff_t pos, unsigned copied,
> + struct folio *folio);
> + bool (*iomap_valid)(struct inode *inode, const struct iomap *iomap);
> + };
> +
> +iomap calls these functions:
> +
> + - ``get_folio``: Called to allocate and return an active reference to
> + a locked folio prior to starting a write.
> + If this function is not provided, iomap will call
> + ``iomap_get_folio``.
> + This could be used to `set up per-folio filesystem state
> + <https://lore.kernel.org/all/[email protected]/>`_
> + for a write.
> +
> + - ``put_folio``: Called to unlock and put a folio after a pagecache
> + operation completes.
> + If this function is not provided, iomap will ``folio_unlock`` and
> + ``folio_put`` on its own.
> + This could be used to `commit per-folio filesystem state
> + <https://lore.kernel.org/all/[email protected]/>`_
> + that was set up by ``->get_folio``.
> +
> + - ``iomap_valid``: The filesystem may not hold locks between
> + ``->iomap_begin`` and ``->iomap_end`` because pagecache operations
> + can take folio locks, fault on userspace pages, initiate writeback
> + for memory reclamation, or engage in other time-consuming actions.
> + If a file's space mapping data are mutable, it is possible that the
> + mapping for a particular pagecache folio can `change in the time it
> + takes
> + <https://lore.kernel.org/all/[email protected]/>`_
> + to allocate, install, and lock that folio.
> + For such files, the mapping *must* be revalidated after the folio
> + lock has been taken so that iomap can manage the folio correctly.
> + The filesystem's ``->iomap_begin`` function must sample a sequence
> + counter into ``struct iomap::validity_cookie`` at the same time that
> + it populates the mapping fields.
> + It must then provide a ``->iomap_valid`` function to compare the
> + validity cookie against the source counter and return whether or not
> + the mapping is still valid.
> + If the mapping is not valid, the mapping will be sampled again.
> +
> +These ``struct kiocb`` flags are significant for buffered I/O with
> +iomap:
> +
> + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> + already in memory, we do not have to initiate other I/O, and we
> + acquire all filesystem locks without blocking.
> + Neither this flag nor its definition ``RWF_NOWAIT`` actually define
> + what this flag means, so this is the best the author could come up
> + with.
> +
> +Internal per-Folio State
> +~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +If the fsblock size matches the size of a pagecache folio, it is assumed
> +that all disk I/O operations will operate on the entire folio.
> +The uptodate (memory contents are at least as new as what's on disk) and
> +dirty (memory contents are newer than what's on disk) status of the
> +folio are all that's needed for this case.
> +
> +If the fsblock size is less than the size of a pagecache folio, iomap
> +tracks the per-fsblock uptodate and dirty state itself.
> +This enables iomap to handle both "bs < ps" `filesystems
> +<https://lore.kernel.org/all/[email protected]/>`_
> +and large folios in the pagecache.
> +
> +iomap internally tracks two state bits per fsblock:
> +
> + * ``uptodate``: iomap will try to keep folios fully up to date.
> + If there are read(ahead) errors, those fsblocks will not be marked
> + uptodate.
> + The folio itself will be marked uptodate when all fsblocks within the
> + folio are uptodate.
> +
> + * ``dirty``: iomap will set the per-block dirty state when programs
> + write to the file.
> + The folio itself will be marked dirty when any fsblock within the
> + folio is dirty.
> +
> +iomap also tracks the amount of read and write disk IOs that are in
> +flight.
> +This structure is much lighter weight than ``struct buffer_head``.
> +
> +Filesystems wishing to turn on large folios in the pagecache should call
> +``mapping_set_large_folios`` when initializing the incore inode.
> +
> +Readahead and Reads
> +~~~~~~~~~~~~~~~~~~~
> +
> +The ``iomap_readahead`` function initiates readahead to the pagecache.
> +The ``iomap_read_folio`` function reads one folio's worth of data into
> +the pagecache.
> +The ``flags`` argument to ``->iomap_begin`` will be set to zero.
> +The pagecache takes whatever locks it needs before calling the
> +filesystem.
> +
> +Writes
> +~~~~~~
> +
> +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
> +pagecache.
> +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
> +the ``flags`` argument to ``->iomap_begin``.
> +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
> +
> +mmap Write Faults
> +^^^^^^^^^^^^^^^^^
> +
> +The ``iomap_page_mkwrite`` function handles a write fault to a folio the
> +pagecache.
> +``IOMAP_WRITE | IOMAP_FAULT`` will be passed as the ``flags`` argument
> +to ``->iomap_begin``.
> +Callers commonly take the mmap ``invalidate_lock`` in shared or
> +exclusive mode.
> +
> +Write Failures
> +^^^^^^^^^^^^^^
> +
> +After a short write to the pagecache, the areas not written will not
> +become marked dirty.
> +The filesystem must arrange to `cancel
> +<https://lore.kernel.org/all/[email protected]/>`_
> +such `reservations
> +<https://lore.kernel.org/linux-xfs/[email protected]/>`_
> +because writeback will not consume the reservation.
> +The ``iomap_file_buffered_write_punch_delalloc`` can be called from a
> +``->iomap_end`` function to find all the clean areas of the folios
> +caching a fresh (``IOMAP_F_NEW``) delalloc mapping.
> +It takes the ``invalidate_lock``.
> +
> +The filesystem should supply a callback ``punch`` will be called for
> +each file range in this state.
> +This function must *only* remove delayed allocation reservations, in
> +case another thread racing with the current thread writes successfully
> +to the same region and triggers writeback to flush the dirty data out to
> +disk.
> +
> +Truncation
> +^^^^^^^^^^
> +
> +Filesystems can call ``iomap_truncate_page`` to zero the bytes in the
> +pagecache from EOF to the end of the fsblock during a file truncation
> +operation.
> +``truncate_setsize`` or ``truncate_pagecache`` will take care of
> +everything after the EOF block.
> +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Zeroing for File Operations
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Filesystems can call ``iomap_zero_range`` to perform zeroing of the
> +pagecache for non-truncation file operations that are not aligned to
> +the fsblock size.
> +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Unsharing Reflinked File Data
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Filesystems can call ``iomap_file_unshare`` to force a file sharing
> +storage with another file to preemptively copy the shared data to newly
> +allocate storage.
> +``IOMAP_WRITE | IOMAP_UNSHARE`` will be passed as the ``flags`` argument
> +to ``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Writeback
> +~~~~~~~~~
> +
> +Filesystems can call ``iomap_writepages`` to respond to a request to
> +write dirty pagecache folios to disk.
> +The ``mapping`` and ``wbc`` parameters should be passed unchanged.
> +The ``wpc`` pointer should be allocated by the filesystem and must
> +be initialized to zero.
> +
> +The pagecache will lock each folio before trying to schedule it for
> +writeback.
> +It does not lock ``i_rwsem`` or ``invalidate_lock``.
> +
> +The dirty bit will be cleared for all folios run through the
> +``->map_blocks`` machinery described below even if the writeback fails.
> +This is to prevent dirty folio clots when storage devices fail; an
> +``-EIO`` is recorded for userspace to collect via ``fsync``.
> +
> +The ``ops`` structure must be specified and is as follows:
> +
> +struct iomap_writeback_ops
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +.. code-block:: c
> +
> + struct iomap_writeback_ops {
> + int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
> + loff_t offset, unsigned len);
> + int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
> + void (*discard_folio)(struct folio *folio, loff_t pos);
> + };
> +
> +The fields are as follows:
> +
> + - ``map_blocks``: Sets ``wpc->iomap`` to the space mapping of the file
> + range (in bytes) given by ``offset`` and ``len``.
> + iomap calls this function for each fs block in each dirty folio,
> + even if the mapping returned is longer than one fs block.
> + Do not return ``IOMAP_INLINE`` mappings here; the ``->iomap_end``
> + function must deal with persisting written data.
> + Filesystems can skip a potentially expensive mapping lookup if the
> + mappings have not changed.
> + This revalidation must be open-coded by the filesystem; it is
> + unclear if ``iomap::validity_cookie`` can be reused for this
> + purpose.
> + This function is required.
> +
> + - ``prepare_ioend``: Enables filesystems to transform the writeback
> + ioend or perform any other prepatory work before the writeback I/O
> + is submitted.
> + A filesystem can override the ``->bi_end_io`` function for its own
> + purposes, such as kicking the ioend completion to a workqueue if the
> + bio is completed in interrupt context.
> + This function is optional.
> +
> + - ``discard_folio``: iomap calls this function after ``->map_blocks``
> + fails schedule I/O for any part of a dirty folio.
> + The function should throw away any reservations that may have been
> + made for the write.
> + The folio will be marked clean and an ``-EIO`` recorded in the
> + pagecache.
> + Filesystems can use this callback to `remove
> + <https://lore.kernel.org/all/[email protected]/>`_
> + delalloc reservations to avoid having delalloc reservations for
> + clean pagecache.
> + This function is optional.
> +
> +Writeback ioend Completion
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +iomap creates chains of ``struct iomap_ioend`` objects that wrap the
> +``bio`` that is used to write pagecache data to disk.
> +By default, iomap finishes writeback ioends by clearing the writeback
> +bit on the folios attached to the ``ioend``.
> +If the write failed, it will also set the error bits on the folios and
> +the address space.
> +This can happen in interrupt or process context, depending on the
> +storage device.
> +
> +Filesystems that need to update internal bookkeeping (e.g. unwritten
> +extent conversions) should provide a ``->prepare_ioend`` function to
> +override the ``struct iomap_end::bio::bi_end_io`` with its own function.
> +This function should call ``iomap_finish_ioends`` after finishing its
> +own work.
> +
> +Some filesystems may wish to `amortize the cost of running metadata
> +transactions
> +<https://lore.kernel.org/all/[email protected]/>`_
> +for post-writeback updates by batching them.
> +They may also require transactions to run from process context, which
> +implies punting batches to a workqueue.
> +iomap ioends contain a ``list_head`` to enable batching.
> +
> +Given a batch of ioends, iomap has a few helpers to assist with
> +amortization:
> +
> + * ``iomap_sort_ioends``: Sort all the ioends in the list by file
> + offset.
> +
> + * ``iomap_ioend_try_merge``: Given an ioend that is not in any list and
> + a separate list of sorted ioends, merge as many of the ioends from
> + the head of the list into the given ioend.
> + ioends can only be merged if the file range and storage addresses are
> + contiguous; the unwritten and shared status are the same; and the
> + write I/O outcome is the same.
> + The merged ioends become their own list.
> +
> + * ``iomap_finish_ioends``: Finish an ioend that possibly has other
> + ioends linked to it.
> +
> +Direct I/O
> +----------
> +
> +In Linux, direct I/O is defined as file I/O that is issued directly to
> +storage, bypassing the pagecache.
> +
> +The ``iomap_dio_rw`` function implements O_DIRECT (direct I/O) reads and
> +writes for files.
> +An optional ``ops`` parameter can be passed to change the behavior of
> +direct I/O.
> +The ``done_before`` parameter should be set if writes have been
> +initiated prior to the call.
> +The direction of the I/O is determined from the ``iocb`` passed in.
> +
> +The ``flags`` argument can be any of the following values:
> +
> + * ``IOMAP_DIO_FORCE_WAIT``: Wait for the I/O to complete even if the
> + kiocb is not synchronous.
> +
> + * ``IOMAP_DIO_OVERWRITE_ONLY``: Allocating blocks, zeroing partial
> + blocks, and extensions of the file size are not allowed.
> + The entire file range must to map to a single written or unwritten
> + extent.
> + This flag exists to enable issuing concurrent direct IOs with only
> + the shared ``i_rwsem`` held when the file I/O range is not aligned to
> + the filesystem block size.
> + ``-EAGAIN`` will be returned if the operation cannot proceed.
> +
> + * ``IOMAP_DIO_PARTIAL``: If a page fault occurs, return whatever
> + progress has already been made.
> + The caller may deal with the page fault and retry the operation.
> +
> +These ``struct kiocb`` flags are significant for direct I/O with iomap:
> +
> + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> + already in memory, we do not have to initiate other I/O, and we
> + acquire all filesystem locks without blocking.
> +
> + * ``IOCB_SYNC``: Ensure that the device has persisted data to disk
> + before completing the call.
> + In the case of pure overwrites, the I/O may be issued with FUA
> + enabled.
> +
> + * ``IOCB_HIPRI``: Poll for I/O completion instead of waiting for an
> + interrupt.
> + Only meaningful for asynchronous I/O, and only if the entire I/O can
> + be issued as a single ``struct bio``.
> +
> + * ``IOCB_DIO_CALLER_COMP``: Try to run I/O completion from the caller's
> + process context.
> + See ``linux/fs.h`` for more details.
> +
> +Filesystems should call ``iomap_dio_rw`` from ``->read_iter`` and
> +``->write_iter``, and set ``FMODE_CAN_ODIRECT`` in the ``->open``
> +function for the file.
> +They should not set ``->direct_IO``, which is deprecated.
> +
> +If a filesystem wishes to perform its own work before direct I/O
> +completion, it should call ``__iomap_dio_rw``.
> +If its return value is not an error pointer or a NULL pointer, the
> +filesystem should pass the return value to ``iomap_dio_complete`` after
> +finishing its internal work.
> +
> +Direct Reads
> +~~~~~~~~~~~~
> +
> +A direct I/O read initiates a read I/O from the storage device to the
> +caller's buffer.
> +Dirty parts of the pagecache are flushed to storage before initiating
> +the read io.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT`` with
> +any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Direct Writes
> +~~~~~~~~~~~~~
> +
> +A direct I/O write initiates a write I/O to the storage device to the
> +caller's buffer.
> +Dirty parts of the pagecache are flushed to storage before initiating
> +the write io.
> +The pagecache is invalidated both before and after the write io.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT |
> +IOMAP_WRITE`` with any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> + blocks is not allowed.
> + The entire file range must to map to a single written or unwritten
> + extent.
> + The file I/O range must be aligned to the filesystem block size.
> +
> +Callers commonly hold ``i_rwsem`` in shared or exclusive mode.
> +
> +struct iomap_dio_ops:
> +~~~~~~~~~~~~~~~~~~~~~
> +.. code-block:: c
> +
> + struct iomap_dio_ops {
> + void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
> + loff_t file_offset);
> + int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
> + unsigned flags);
> + struct bio_set *bio_set;
> + };
> +
> +The fields of this structure are as follows:
> +
> + - ``submit_io``: iomap calls this function when it has constructed a
> + ``struct bio`` object for the I/O requested, and wishes to submit it
> + to the block device.
> + If no function is provided, ``submit_bio`` will be called directly.
> + Filesystems that would like to perform additional work before (e.g.
> + data replication for btrfs) should implement this function.
> +
> + - ``end_io``: This is called after the ``struct bio`` completes.
> + This function should perform post-write conversions of unwritten
> + extent mappings, handle write failures, etc.
> + The ``flags`` argument may be set to a combination of the following:
> +
> + * ``IOMAP_DIO_UNWRITTEN``: The mapping was unwritten, so the ioend
> + should mark the extent as written.
> +
> + * ``IOMAP_DIO_COW``: Writing to the space in the mapping required a
> + copy on write operation, so the ioend should switch mappings.
> +
> + - ``bio_set``: This allows the filesystem to provide a custom bio_set
> + for allocating direct I/O bios.
> + This enables filesystems to `stash additional per-bio information
> + <https://lore.kernel.org/all/[email protected]/>`_
> + for private use.
> + If this field is NULL, generic ``struct bio`` objects will be used.
> +
> +Filesystems that want to perform extra work after an I/O completion
> +should set a custom ``->bi_end_io`` function via ``->submit_io``.
> +Afterwards, the custom endio function must call
> +``iomap_dio_bio_end_io`` to finish the direct I/O.
> +
> +DAX I/O
> +-------
> +
> +Storage devices that can be directly mapped as memory support a new
> +access mode known as "fsdax".
> +
> +fsdax Reads
> +~~~~~~~~~~~
> +
> +A fsdax read performs a memcpy from storage device to the caller's
> +buffer.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX`` with any
> +combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +fsdax Writes
> +~~~~~~~~~~~~
> +
> +A fsdax write initiates a memcpy to the storage device to the caller's
> +buffer.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX |
> +IOMAP_WRITE`` with any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> + blocks is not allowed.
> + The entire file range must to map to a single written or unwritten
> + extent.
> + The file I/O range must be aligned to the filesystem block size.
> +
> +Callers commonly hold ``i_rwsem`` in exclusive mode.
> +
> +mmap Faults
> +~~~~~~~~~~~
> +
> +The ``dax_iomap_fault`` function handles read and write faults to fsdax
> +storage.
> +For a read fault, ``IOMAP_DAX | IOMAP_FAULT`` will be passed as the
> +``flags`` argument to ``->iomap_begin``.
> +For a write fault, ``IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE`` will be
> +passed as the ``flags`` argument to ``->iomap_begin``.
> +
> +Callers commonly hold the same locks as they do to call their iomap
> +pagecache counterparts.
> +
> +Truncation, fallocate, and Unsharing
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For fsdax files, the following functions are provided to replace their
> +iomap pagecache I/O counterparts.
> +The ``flags`` argument to ``->iomap_begin`` are the same as the
> +pagecache counterparts, with ``IOMAP_DIO`` added.
> +
> + * ``dax_file_unshare``
> + * ``dax_zero_range``
> + * ``dax_truncate_page``
> +
> +Callers commonly hold the same locks as they do to call their iomap
> +pagecache counterparts.
> +
> +SEEK_DATA
> +---------
> +
> +The ``iomap_seek_data`` function implements the SEEK_DATA "whence" value
> +for llseek.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +
> +For unwritten mappings, the pagecache will be searched.
> +Regions of the pagecache with a folio mapped and uptodate fsblocks
> +within those folios will be reported as data areas.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +SEEK_HOLE
> +---------
> +
> +The ``iomap_seek_hole`` function implements the SEEK_HOLE "whence" value
> +for llseek.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +
> +For unwritten mappings, the pagecache will be searched.
> +Regions of the pagecache with no folio mapped, or a !uptodate fsblock
> +within a folio will be reported as sparse hole areas.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Swap File Activation
> +--------------------
> +
> +The ``iomap_swapfile_activate`` function finds all the base-page aligned
> +regions in a file and sets them up as swap space.
> +The file will be ``fsync()``'d before activation.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +All mappings must be mapped or unwritten; cannot be dirty or shared, and
> +cannot span multiple block devices.
> +Callers must hold ``i_rwsem`` in exclusive mode; this is already
> +provided by ``swapon``.
> +
> +Extent Map Reporting (FS_IOC_FIEMAP)
> +------------------------------------
> +
> +The ``iomap_fiemap`` function exports file extent mappings to userspace
> +in the format specified by the ``FS_IOC_FIEMAP`` ioctl.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Block Map Reporting (FIBMAP)
> +----------------------------
> +
> +``iomap_bmap`` implements FIBMAP.
> +The calling conventions are the same as for FIEMAP.
> +This function is only provided to maintain compatibility for filesystems
> +that implemented FIBMAP prior to conversion.
> +This ioctl is deprecated; do not add a FIBMAP implementation to
> +filesystems that do not have it.
> +Callers should probably hold ``i_rwsem`` in shared mode, but this is
> +unclear.
> +
> +Porting Guide
> +=============
> +
> +Why Convert to iomap?
> +---------------------
> +
> +There are several reasons to convert a filesystem to iomap:
> +
> + 1. The classic Linux I/O path is not terribly efficient.
> + Pagecache operations lock a single base page at a time and then call
> + into the filesystem to return a mapping for only that page.
> + Direct I/O operations build I/O requests a single file block at a
> + time.
> + This worked well enough for direct/indirect-mapped filesystems such
> + as ext2, but is very inefficient for extent-based filesystems such
> + as XFS.
> +
> + 2. Large folios are only supported via iomap; there are no plans to
> + convert the old buffer_head path to use them.
> +
> + 3. Direct access to storage on memory-like devices (fsdax) is only
> + supported via iomap.
> +
> + 4. Lower maintenance overhead for individual filesystem maintainers.
> + iomap handles common pagecache related operations itself, such as
> + allocating, instantiating, locking, and unlocking of folios.
> + No ->write_begin(), ->write_end() or direct_IO
> + address_space_operations are required to be implemented by
> + filesystem using iomap.
> +
> +How to Convert to iomap?
> +------------------------
> +
> +First, add ``#include <linux/iomap.h>`` from your source code and add
> +``select FS_IOMAP`` to your filesystem's Kconfig option.
> +Build the kernel, run fstests with the ``-g all`` option across a wide
> +variety of your filesystem's supported configurations to build a
> +baseline of which tests pass and which ones fail.
> +
> +The recommended approach is first to implement ``->iomap_begin`` (and
> +``->iomap->end`` if necessary) to allow iomap to obtain a read-only
> +mapping of a file range.
> +In most cases, this is a relatively trivial conversion of the existing
> +``get_block()`` function for read-only mappings.
> +``FS_IOC_FIEMAP`` is a good first target because it is trivial to
> +implement support for it and then to determine that the extent map
> +iteration is correct from userspace.
> +If FIEMAP is returning the correct information, it's a good sign that
> +other read-only mapping operations will do the right thing.
> +
> +Next, modify the filesystem's ``get_block(create = false)``
> +implementation to use the new ``->iomap_begin`` implementation to map
> +file space for selected read operations.
> +Hide behind a debugging knob the ability to switch on the iomap mapping
> +functions for selected call paths.
> +It is necessary to write some code to fill out the bufferhead-based
> +mapping information from the ``iomap`` structure, but the new functions
> +can be tested without needing to implement any iomap APIs.
> +
> +Once the read-only functions are working like this, convert each high
> +level file operation one by one to use iomap native APIs instead of
> +going through ``get_block()``.
> +Done one at a time, regressions should be self evident.
> +You *do* have a regression test baseline for fstests, right?
> +It is suggested to convert swap file activation, ``SEEK_DATA``, and
> +``SEEK_HOLE`` before tackling the I/O paths.
> +A likely complexity at this point will be converting the buffered read
> +I/O path because of bufferheads.
> +The buffered read I/O paths doesn't need to be converted yet, though the
> +direct I/O read path should be converted in this phase.
> +
> +At this point, you should look over your ``->iomap_begin`` function.
> +If it switches between large blocks of code based on dispatching of the
> +``flags`` argument, you should consider breaking it up into
> +per-operation iomap ops with smaller, more cohesive functions.
> +XFS is a good example of this.
> +
> +The next thing to do is implement ``get_blocks(create == true)``
> +functionality in the ``->iomap_begin``/``->iomap_end`` methods.
> +It is strongly recommended to create separate mapping functions and
> +iomap ops for write operations.
> +Then convert the direct I/O write path to iomap, and start running fsx
> +w/ DIO enabled in earnest on filesystem.
> +This will flush out lots of data integrity corner case bugs that the new
> +write mapping implementation introduces.
> +
> +Now, convert any remaining file operations to call the iomap functions.
> +This will get the entire filesystem using the new mapping functions, and
> +they should largely be debugged and working correctly after this step.
> +
> +Most likely at this point, the buffered read and write paths will still
> +to be converted.
> +The mapping functions should all work correctly, so all that needs to be
> +done is rewriting all the code that interfaces with bufferheads to
> +interface with iomap and folios.
> +It is much easier first to get regular file I/O (without any fancy
> +features like fscrypt, fsverity, compression, or data=journaling)
> +converted to use iomap.
> +Some of those fancy features (fscrypt and compression) aren't
> +implemented yet in iomap.
> +For unjournalled filesystems that use the pagecache for symbolic links
> +and directories, you might also try converting their handling to iomap.
> +
> +The rest is left as an exercise for the reader, as it will be different
> +for every filesystem.
> +If you encounter problems, email the people and lists in
> +``get_maintainers.pl`` for help.
> +
> +Bugs and Limitations
> +====================
> +
> + * No support for fscrypt.
> + * No support for compression.
> + * No support for fsverity yet.
> + * Strong assumptions that IO should work the way it does on XFS.
> + * Does iomap *actually* work for non-regular file data?
> +
> +Patches welcome!
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8754ac2c259d..2ddd94d43ecf 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8483,6 +8483,7 @@ R: Darrick J. Wong <[email protected]>
> L: [email protected]
> L: [email protected]
> S: Supported
> +F: Documentation/filesystems/iomap.txt
> F: fs/iomap/
> F: include/linux/iomap.h
>
On Sun 09-06-24 08:55:06, Darrick J. Wong wrote:
> * invalidate_lock: The pagecache struct address_space
> rwsemaphore that protects against folio removal.
invalidate_lock lock is held for read during insertions and for write
during removals. So holding it pro read indeed protects against folio
removal but holding it for write protects against folio insertion (which
some places also use).
> * validity_cookie is a magic freshness value set by the
> filesystem that should be used to detect stale mappings. For
> pagecache operations this is critical for correct operation
> because page faults can occur, which implies that filesystem
> locks should not be held between ->iomap_begin and
> ->iomap_end. Filesystems with completely static mappings
> need not set this value. Only pagecache operations
> revalidate mappings.
>
> XXX: Should fsdax revalidate as well?
AFAICT no. DAX is more like using direct IO for everything. So no writeback
changing mapping state behind your back (and that's the only thing that is
not serialized with i_rwsem or invalidate_lock). Maybe this fact can be
mentioned somewhere around the discussion of iomap_valid() as a way how
locking usually works out?
> iomap implements nearly all the folio and pagecache management
> that filesystems once had to implement themselves. This means that
> the filesystem need not know the details of allocating, mapping,
> managing uptodate and dirty state, or writeback of pagecache
> folios. Unless the filesystem explicitly opts in to buffer heads,
> they will not be used, which makes buffered I/O much more
> efficient, and willy much happier.
^^^ unless we make it a general noun for someone doing
thankless neverending conversion job, we should give him a capital W ;).
> These struct kiocb flags are significant for buffered I/O with
> iomap:
>
> * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> already in memory, we do not have to initiate other I/O, and
> we acquire all filesystem locks without blocking. Neither
> this flag nor its definition RWF_NOWAIT actually define what
> this flag means, so this is the best the author could come
> up with.
RWF_NOWAIT is a performance feature, not a correctness one, hence the
meaning is somewhat vague. It is meant to mean "do the IO only if it
doesn't involve waiting for other IO or other time expensive operations".
Generally we translate it to "don't wait for i_rwsem, page locks, don't do
block allocation, etc." OTOH we don't bother to specialcase internal
filesystem locks (such as EXT4_I(inode)->i_data_sem) and we get away with
it because blocking on it under constraints we generally perform RWF_NOWAIT
IO is exceedingly rare.
> mmap Write Faults
>
> The iomap_page_mkwrite function handles a write fault to a folio
> the pagecache.
^^^ to a folio *in* the pagecache? I cannot quite parse the sentence.
> Truncation
>
> Filesystems can call iomap_truncate_page to zero the bytes in the
> pagecache from EOF to the end of the fsblock during a file
> truncation operation. truncate_setsize or truncate_pagecache will
> take care of everything after the EOF block. IOMAP_ZERO will be
> passed as the flags argument to ->iomap_begin. Callers typically
> take i_rwsem and invalidate_lock in exclusive mode.
Hum, but i_rwsem and invalidate_lock are usually acquired *before*
iomap_truncate_page() is even called, aren't they? This locking note looks
a bit confusing to me. I'd rather write: "The callers typically hold i_rwsem
and invalidate_lock when calling iomap_truncate_page()." if you want to
mention any locking.
> Zeroing for File Operations
>
> Filesystems can call iomap_zero_range to perform zeroing of the
> pagecache for non-truncation file operations that are not aligned
> to the fsblock size. IOMAP_ZERO will be passed as the flags
> argument to ->iomap_begin. Callers typically take i_rwsem and
> invalidate_lock in exclusive mode.
Ditto here...
> Unsharing Reflinked File Data
>
> Filesystems can call iomap_file_unshare to force a file sharing
> storage with another file to preemptively copy the shared data to
> newly allocate storage. IOMAP_WRITE | IOMAP_UNSHARE will be passed
> as the flags argument to ->iomap_begin. Callers typically take
> i_rwsem and invalidate_lock in exclusive mode.
And here.
> Direct I/O
>
> In Linux, direct I/O is defined as file I/O that is issued
> directly to storage, bypassing the pagecache.
>
> The iomap_dio_rw function implements O_DIRECT (direct I/O) reads
> and writes for files. An optional ops parameter can be passed to
> change the behavior of direct I/O. The done_before parameter
> should be set if writes have been initiated prior to the call. The
> direction of the I/O is determined from the iocb passed in.
>
> The flags argument can be any of the following values:
>
> * IOMAP_DIO_FORCE_WAIT: Wait for the I/O to complete even if
> the kiocb is not synchronous.
>
> * IOMAP_DIO_OVERWRITE_ONLY: Allocating blocks, zeroing partial
> blocks, and extensions of the file size are not allowed. The
> entire file range must to map to a single written or
^^ extra "to"
> unwritten extent. This flag exists to enable issuing
> concurrent direct IOs with only the shared i_rwsem held when
> the file I/O range is not aligned to the filesystem block
> size. -EAGAIN will be returned if the operation cannot
> proceed.
<snip>
> Direct Writes
>
> A direct I/O write initiates a write I/O to the storage device to
> the caller's buffer. Dirty parts of the pagecache are flushed to
> storage before initiating the write io. The pagecache is
> invalidated both before and after the write io. The flags value
> for ->iomap_begin will be IOMAP_DIRECT | IOMAP_WRITE with any
> combination of the following enhancements:
>
> * IOMAP_NOWAIT: Write if mapping data are already in memory.
> Does not initiate other I/O or block on filesystem locks.
>
> * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> blocks is not allowed. The entire file range must to map to
^^ extra "to"
> a single written or unwritten extent. The file I/O range
> must be aligned to the filesystem block size.
This seems to be XFS specific thing? At least I don't see anything in
generic iomap code depending on this?
> fsdax Writes
>
> A fsdax write initiates a memcpy to the storage device to the
^^ from
> caller's buffer. The flags value for ->iomap_begin will be
> IOMAP_DAX | IOMAP_WRITE with any combination of the following
> enhancements:
>
> * IOMAP_NOWAIT: Write if mapping data are already in memory.
> Does not initiate other I/O or block on filesystem locks.
>
> * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> blocks is not allowed. The entire file range must to map to
^^ extra "to"
> a single written or unwritten extent. The file I/O range
> must be aligned to the filesystem block size.
>
> Callers commonly hold i_rwsem in exclusive mode.
>
> mmap Faults
>
> The dax_iomap_fault function handles read and write faults to
> fsdax storage. For a read fault, IOMAP_DAX | IOMAP_FAULT will be
> passed as the flags argument to ->iomap_begin. For a write fault,
> IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE will be passed as the flags
> argument to ->iomap_begin.
>
> Callers commonly hold the same locks as they do to call their
> iomap pagecache counterparts.
>
> Truncation, fallocate, and Unsharing
>
> For fsdax files, the following functions are provided to replace
> their iomap pagecache I/O counterparts. The flags argument to
> ->iomap_begin are the same as the pagecache counterparts, with
> IOMAP_DIO added.
^^^ IOMAP_DAX?
> * dax_file_unshare
>
> * dax_zero_range
>
> * dax_truncate_page
>
> Callers commonly hold the same locks as they do to call their
> iomap pagecache counterparts.
> How to Convert to iomap?
>
> First, add #include <linux/iomap.h> from your source code and add
> select FS_IOMAP to your filesystem's Kconfig option. Build the
> kernel, run fstests with the -g all option across a wide variety
> of your filesystem's supported configurations to build a baseline
> of which tests pass and which ones fail.
>
> The recommended approach is first to implement ->iomap_begin (and
> ->iomap->end if necessary) to allow iomap to obtain a read-only
^^^^ ->iomap_end
<snip>
> Most likely at this point, the buffered read and write paths will
> still to be converted. The mapping functions should all work
^^ need to be
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Mon, Jun 10, 2024 at 04:18:08PM +0200, Jan Kara wrote:
> On Sun 09-06-24 08:55:06, Darrick J. Wong wrote:
> > * invalidate_lock: The pagecache struct address_space
> > rwsemaphore that protects against folio removal.
>
> invalidate_lock lock is held for read during insertions and for write
> during removals. So holding it pro read indeed protects against folio
> removal but holding it for write protects against folio insertion (which
> some places also use).
Ok, I've changed the two lock bulletpoints to read:
"i_rwsem: The VFS struct inode rwsemaphore. Processes hold this in
shared mode to read file state and contents. Some filesystems may allow
shared mode for writes. Processes often hold this in exclusive mode to
change file state and contents.
"invalidate_lock: The pagecache struct address_space rwsemaphore that
protects against folio insertion and removal for filesystems that
support punching out folios below EOF. Processes wishing to insert
folios must hold this lock in shared mode to prevent removal, though
concurrent insertion is allowed. Processes wishing to remove folios
must hold this lock in exclusive mode to prevent insertions. Concurrent
removals are not allowed."
> > * validity_cookie is a magic freshness value set by the
> > filesystem that should be used to detect stale mappings. For
> > pagecache operations this is critical for correct operation
> > because page faults can occur, which implies that filesystem
> > locks should not be held between ->iomap_begin and
> > ->iomap_end. Filesystems with completely static mappings
> > need not set this value. Only pagecache operations
> > revalidate mappings.
> >
> > XXX: Should fsdax revalidate as well?
>
> AFAICT no. DAX is more like using direct IO for everything. So no writeback
> changing mapping state behind your back (and that's the only thing that is
> not serialized with i_rwsem or invalidate_lock). Maybe this fact can be
> mentioned somewhere around the discussion of iomap_valid() as a way how
> locking usually works out?
<nod> I'll put that in the section about iomap_valid, which documents
the whole mechanism more thoroughly:
"iomap_valid: The filesystem may not hold locks between ->iomap_begin
and ->iomap_end because pagecache operations can take folio locks, fault
on userspace pages, initiate writeback for memory reclamation, or engage
in other time-consuming actions. If a file's space mapping data are
mutable, it is possible that the mapping for a particular pagecache
folio can change in the time it takes to allocate, install, and lock
that folio.
"For the pagecache, races can happen if writeback doesn't take i_rwsem
or invalidate_lock and updates mapping information. Races can also
happen if the filesytem allows concurrent writes. For such files, the
mapping *must* be revalidated after the folio lock has been taken so
that iomap can manage the folio correctly.
"fsdax does not need this revalidation because there's no writeback and
no support for unwritten extents.
"The filesystem's ->iomap_begin function must sample a sequence counter
into struct iomap::validity_cookie at the same time that it populates
the mapping fields. It must then provide a ->iomap_valid function to
compare the validity cookie against the source counter and return
whether or not the mapping is still valid. If the mapping is not valid,
the mapping will be sampled again."
> > iomap implements nearly all the folio and pagecache management
> > that filesystems once had to implement themselves. This means that
> > the filesystem need not know the details of allocating, mapping,
> > managing uptodate and dirty state, or writeback of pagecache
> > folios. Unless the filesystem explicitly opts in to buffer heads,
> > they will not be used, which makes buffered I/O much more
> > efficient, and willy much happier.
> ^^^ unless we make it a general noun for someone doing
> thankless neverending conversion job, we should give him a capital W ;).
I'll change it to 'the pagecache maintainer'
> > These struct kiocb flags are significant for buffered I/O with
> > iomap:
> >
> > * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> > already in memory, we do not have to initiate other I/O, and
> > we acquire all filesystem locks without blocking. Neither
> > this flag nor its definition RWF_NOWAIT actually define what
> > this flag means, so this is the best the author could come
> > up with.
>
> RWF_NOWAIT is a performance feature, not a correctness one, hence the
> meaning is somewhat vague. It is meant to mean "do the IO only if it
> doesn't involve waiting for other IO or other time expensive operations".
> Generally we translate it to "don't wait for i_rwsem, page locks, don't do
> block allocation, etc." OTOH we don't bother to specialcase internal
> filesystem locks (such as EXT4_I(inode)->i_data_sem) and we get away with
> it because blocking on it under constraints we generally perform RWF_NOWAIT
> IO is exceedingly rare.
I hate this flag's undocumented nature. It now makes *documenting*
things around it hard. How about:
"IOCB_NOWAIT: Neither this flag nor its associated definition RWF_NOWAIT
actually specify what this flag means. Community members seem to think
that it means only proceed with the I/O if it doesn't involve waiting
for expensive operations. XFS and ext4 appear to reject the IO unless
the mapping data are already in memory, the filesystem does not have to
initiate other I/O, and the kernel can acquire all filesystem locks
without blocking."
> > mmap Write Faults
> >
> > The iomap_page_mkwrite function handles a write fault to a folio
> > the pagecache.
> ^^^ to a folio *in* the pagecache? I cannot quite parse the sentence.
Correct. Fixed.
> > Truncation
> >
> > Filesystems can call iomap_truncate_page to zero the bytes in the
> > pagecache from EOF to the end of the fsblock during a file
> > truncation operation. truncate_setsize or truncate_pagecache will
> > take care of everything after the EOF block. IOMAP_ZERO will be
> > passed as the flags argument to ->iomap_begin. Callers typically
> > take i_rwsem and invalidate_lock in exclusive mode.
>
> Hum, but i_rwsem and invalidate_lock are usually acquired *before*
> iomap_truncate_page() is even called, aren't they?
Yes, I think so.
> This locking note looks
> a bit confusing to me. I'd rather write: "The callers typically hold i_rwsem
> and invalidate_lock when calling iomap_truncate_page()." if you want to
> mention any locking.
Ok.
> > Zeroing for File Operations
> >
> > Filesystems can call iomap_zero_range to perform zeroing of the
> > pagecache for non-truncation file operations that are not aligned
> > to the fsblock size. IOMAP_ZERO will be passed as the flags
> > argument to ->iomap_begin. Callers typically take i_rwsem and
> > invalidate_lock in exclusive mode.
>
> Ditto here...
>
> > Unsharing Reflinked File Data
> >
> > Filesystems can call iomap_file_unshare to force a file sharing
> > storage with another file to preemptively copy the shared data to
> > newly allocate storage. IOMAP_WRITE | IOMAP_UNSHARE will be passed
> > as the flags argument to ->iomap_begin. Callers typically take
> > i_rwsem and invalidate_lock in exclusive mode.
>
> And here.
Yeah, I'm not sure -- this is the XFS model (and I guess ext4 now too),
but other systems don't have to follow that. I'll just weaselword it:
"Callers typically hold i_rwsem and invalidate_lock in exclusive
mode before calling this function."
> > Direct I/O
> >
> > In Linux, direct I/O is defined as file I/O that is issued
> > directly to storage, bypassing the pagecache.
> >
> > The iomap_dio_rw function implements O_DIRECT (direct I/O) reads
> > and writes for files. An optional ops parameter can be passed to
> > change the behavior of direct I/O. The done_before parameter
> > should be set if writes have been initiated prior to the call. The
> > direction of the I/O is determined from the iocb passed in.
> >
> > The flags argument can be any of the following values:
> >
> > * IOMAP_DIO_FORCE_WAIT: Wait for the I/O to complete even if
> > the kiocb is not synchronous.
> >
> > * IOMAP_DIO_OVERWRITE_ONLY: Allocating blocks, zeroing partial
> > blocks, and extensions of the file size are not allowed. The
> > entire file range must to map to a single written or
> ^^ extra "to"
Fixed, thanks.
> > unwritten extent. This flag exists to enable issuing
> > concurrent direct IOs with only the shared i_rwsem held when
> > the file I/O range is not aligned to the filesystem block
> > size. -EAGAIN will be returned if the operation cannot
> > proceed.
>
> <snip>
>
> > Direct Writes
> >
> > A direct I/O write initiates a write I/O to the storage device to
> > the caller's buffer. Dirty parts of the pagecache are flushed to
> > storage before initiating the write io. The pagecache is
> > invalidated both before and after the write io. The flags value
> > for ->iomap_begin will be IOMAP_DIRECT | IOMAP_WRITE with any
> > combination of the following enhancements:
> >
> > * IOMAP_NOWAIT: Write if mapping data are already in memory.
> > Does not initiate other I/O or block on filesystem locks.
> >
> > * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> > blocks is not allowed. The entire file range must to map to
> ^^ extra "to"
>
> > a single written or unwritten extent. The file I/O range
> > must be aligned to the filesystem block size.
>
> This seems to be XFS specific thing? At least I don't see anything in
> generic iomap code depending on this?
Hmm. XFS bails out if the mapping is unwritten and the directio write
range isn't aligned to the fsblock size. I think the reason for that is
because we'd have to zero the unaligned regions outside of the write
range, and xfs can't do that without synchronizing. (Or we didn't think
that was common enough to bother with the code complexity.)
"The file I/O range must be aligned to the filesystem block size
if the filesystem supports unwritten mappings but cannot zero unaligned
regions without exposing stale contents."?
>
> > fsdax Writes
> >
> > A fsdax write initiates a memcpy to the storage device to the
> ^^ from
>
> > caller's buffer. The flags value for ->iomap_begin will be
> > IOMAP_DAX | IOMAP_WRITE with any combination of the following
> > enhancements:
> >
> > * IOMAP_NOWAIT: Write if mapping data are already in memory.
> > Does not initiate other I/O or block on filesystem locks.
> >
> > * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> > blocks is not allowed. The entire file range must to map to
> ^^ extra "to"
>
> > a single written or unwritten extent. The file I/O range
> > must be aligned to the filesystem block size.
> >
> > Callers commonly hold i_rwsem in exclusive mode.
> >
> > mmap Faults
> >
> > The dax_iomap_fault function handles read and write faults to
> > fsdax storage. For a read fault, IOMAP_DAX | IOMAP_FAULT will be
> > passed as the flags argument to ->iomap_begin. For a write fault,
> > IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE will be passed as the flags
> > argument to ->iomap_begin.
> >
> > Callers commonly hold the same locks as they do to call their
> > iomap pagecache counterparts.
> >
> > Truncation, fallocate, and Unsharing
> >
> > For fsdax files, the following functions are provided to replace
> > their iomap pagecache I/O counterparts. The flags argument to
> > ->iomap_begin are the same as the pagecache counterparts, with
> > IOMAP_DIO added.
> ^^^ IOMAP_DAX?
Oops yes.
> > * dax_file_unshare
> >
> > * dax_zero_range
> >
> > * dax_truncate_page
> >
> > Callers commonly hold the same locks as they do to call their
> > iomap pagecache counterparts.
>
> > How to Convert to iomap?
> >
> > First, add #include <linux/iomap.h> from your source code and add
> > select FS_IOMAP to your filesystem's Kconfig option. Build the
> > kernel, run fstests with the -g all option across a wide variety
> > of your filesystem's supported configurations to build a baseline
> > of which tests pass and which ones fail.
> >
> > The recommended approach is first to implement ->iomap_begin (and
> > ->iomap->end if necessary) to allow iomap to obtain a read-only
> ^^^^ ->iomap_end
Fixed, thanks.
> <snip>
>
> > Most likely at this point, the buffered read and write paths will
> > still to be converted. The mapping functions should all work
> ^^ need to be
Fixed, thanks.
--D
>
> Honza
> --
> Jan Kara <[email protected]>
> SUSE Labs, CR
>
On Mon 10-06-24 14:59:28, Darrick J. Wong wrote:
> > > These struct kiocb flags are significant for buffered I/O with
> > > iomap:
> > >
> > > * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> > > already in memory, we do not have to initiate other I/O, and
> > > we acquire all filesystem locks without blocking. Neither
> > > this flag nor its definition RWF_NOWAIT actually define what
> > > this flag means, so this is the best the author could come
> > > up with.
> >
> > RWF_NOWAIT is a performance feature, not a correctness one, hence the
> > meaning is somewhat vague. It is meant to mean "do the IO only if it
> > doesn't involve waiting for other IO or other time expensive operations".
> > Generally we translate it to "don't wait for i_rwsem, page locks, don't do
> > block allocation, etc." OTOH we don't bother to specialcase internal
> > filesystem locks (such as EXT4_I(inode)->i_data_sem) and we get away with
> > it because blocking on it under constraints we generally perform RWF_NOWAIT
> > IO is exceedingly rare.
>
> I hate this flag's undocumented nature. It now makes *documenting*
> things around it hard. How about:
>
> "IOCB_NOWAIT: Neither this flag nor its associated definition RWF_NOWAIT
> actually specify what this flag means. Community members seem to think
> that it means only proceed with the I/O if it doesn't involve waiting
> for expensive operations. XFS and ext4 appear to reject the IO unless
> the mapping data are already in memory, the filesystem does not have to
> initiate other I/O, and the kernel can acquire all filesystem locks
> without blocking."
I guess this is good enough :)
> > > Direct Writes
> > >
> > > A direct I/O write initiates a write I/O to the storage device to
> > > the caller's buffer. Dirty parts of the pagecache are flushed to
> > > storage before initiating the write io. The pagecache is
> > > invalidated both before and after the write io. The flags value
> > > for ->iomap_begin will be IOMAP_DIRECT | IOMAP_WRITE with any
> > > combination of the following enhancements:
> > >
> > > * IOMAP_NOWAIT: Write if mapping data are already in memory.
> > > Does not initiate other I/O or block on filesystem locks.
> > >
> > > * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> > > blocks is not allowed. The entire file range must to map to
> > ^^ extra "to"
> >
> > > a single written or unwritten extent. The file I/O range
> > > must be aligned to the filesystem block size.
> >
> > This seems to be XFS specific thing? At least I don't see anything in
> > generic iomap code depending on this?
>
> Hmm. XFS bails out if the mapping is unwritten and the directio write
> range isn't aligned to the fsblock size. I think the reason for that is
> because we'd have to zero the unaligned regions outside of the write
> range, and xfs can't do that without synchronizing. (Or we didn't think
> that was common enough to bother with the code complexity.)
>
> "The file I/O range must be aligned to the filesystem block size
> if the filesystem supports unwritten mappings but cannot zero unaligned
> regions without exposing stale contents."?
Sounds good.
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Mon, Jun 10, 2024 at 02:27:28PM +0530, Ritesh Harjani wrote:
>
> Hello Darrick,
>
> "Darrick J. Wong" <[email protected]> writes:
>
> > From: Darrick J. Wong <[email protected]>
> >
> > This is the fourth attempt at documenting the design of iomap and how to
>
> I agree that this isn't needed in the commit msg ("fourth attempt").
Ok. "Coapture the design of iomap and how to port..."
> > port filesystems to use it. Apologies for all the rst formatting, but
> > it's necessary to distinguish code from regular text.
> >
> > A lot of this has been collected from various email conversations, code
> > comments, commit messages, my own understanding of iomap, and
> > Ritesh/Luis' previous efforts to create a document. Please note a large
> > part of this has been taken from Dave's reply to last iomap doc
> > patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
> > other iomap developers who have taken time to explain the iomap design
> > in various emails, commits, comments etc.
> >
> > Cc: Dave Chinner <[email protected]>
> > Cc: Matthew Wilcox <[email protected]>
> > Cc: Christoph Hellwig <[email protected]>
> > Cc: Christian Brauner <[email protected]>
> > Cc: Ojaswin Mujoo <[email protected]>
> > Cc: Jan Kara <[email protected]>
> > Cc: Luis Chamberlain <[email protected]>
> > Inspired-by: Ritesh Harjani (IBM) <[email protected]>
>
> I am not sure if this is even a valid or accepted tag.
> But sure thanks! :)
They're freeform tags, so they can be everything everyone wants them to
be! Drum circle kumbaya etc. :P
> > Signed-off-by: Darrick J. Wong <[email protected]>
> > ---
> > Documentation/filesystems/index.rst | 1
> > Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
> > MAINTAINERS | 1
> > 3 files changed, 1062 insertions(+)
> > create mode 100644 Documentation/filesystems/iomap.rst
> >
> > diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
> > index 8f5c1ee02e2f..b010cc8df32d 100644
> > --- a/Documentation/filesystems/index.rst
> > +++ b/Documentation/filesystems/index.rst
> > @@ -34,6 +34,7 @@ algorithms work.
> > seq_file
> > sharedsubtree
> > idmappings
> > + iomap
> >
> > automount-support
> >
> > diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
> > new file mode 100644
> > index 000000000000..a478b55e4135
> > --- /dev/null
> > +++ b/Documentation/filesystems/iomap.rst
> > @@ -0,0 +1,1060 @@
> > +.. SPDX-License-Identifier: GPL-2.0
> > +.. _iomap:
> > +
> > +..
> > + Dumb style notes to maintain the author's sanity:
> > + Please try to start sentences on separate lines so that
> > + sentence changes don't bleed colors in diff.
> > + Heading decorations are documented in sphinx.rst.
> > +
> > +============================
> > +VFS iomap Design and Porting
> > +============================
> > +
> > +.. toctree::
> > +
> > +Introduction
> > +============
> > +
> > +iomap is a filesystem library for handling various filesystem operations
> > +that involves mapping of file's logical offset ranges to physical
> > +extents.
> > +This origins of this library is the file I/O path that XFS once used; it
> > +has now been extended to cover several other operations.
> > +The library provides various APIs for implementing various file and
> ^^^^ redundant "various"
>
> > +pagecache operations, such as:
> > +
> > + * Pagecache reads and writes
> > + * Folio write faults to the pagecache
> > + * Writeback of dirty folios
> > + * Direct I/O reads and writes
>
> Dax I/O reads and writes.
> ... as well please?
It's really fsdax I/O reads, writes, loads, and stores, isn't it?
>
> > + * FIEMAP
> > + * lseek ``SEEK_DATA`` and ``SEEK_HOLE``
> > + * swapfile activation
> > +
>
> > +Who Should Read This?
> > +=====================
> > +
> > +The target audience for this document are filesystem, storage, and
>
> /s/and/,/
>
>
> > +pagecache programmers and code reviewers.
>
> Not sure if we even need this secion "Who Should Read This".
That was a review comment from Luis' attempt to write this document:
https://lore.kernel.org/linux-xfs/[email protected]/
Also I guess I should state explicitly:
"If you are working on PCI, machine architectures, or device drivers,
you are most likely in the wrong place."
> > +The goal of this document is to provide a brief discussion of the
> > +design and capabilities of iomap, followed by a more detailed catalog
> > +of the interfaces presented by iomap.
> > +If you change iomap, please update this design document.
>
> The details of "goal of this document..." -> can be a part of
> separate paragraph in "Introduction" section itself.
>
> > +
> > +But Why?
> > +========
>
> "Why Iomap?" is more clean IMO.
"Why VFS iomap?", then.
> > +
> > +Unlike the classic Linux I/O model which breaks file I/O into small
> > +units (generally memory pages or blocks) and looks up space mappings on
> > +the basis of that unit, the iomap model asks the filesystem for the
> > +largest space mappings that it can create for a given file operation and
> > +initiates operations on that basis.
> > +This strategy improves the filesystem's visibility into the size of the
> > +operation being performed, which enables it to combat fragmentation with
> > +larger space allocations when possible.
> > +Larger space mappings improve runtime performance by amortizing the cost
> > +of a mapping function call into the filesystem across a larger amount of
>
> s/call/calls
Done.
> > +data.
> > +
> > +At a high level, an iomap operation `looks like this
> > +<https://lore.kernel.org/all/[email protected]/>`_:
> > +
> > +1. For each byte in the operation range...
> > +
> > + 1. Obtain space mapping via ->iomap_begin
> > + 2. For each sub-unit of work...
> > +
> > + 1. Revalidate the mapping and go back to (1) above, if necessary
> > + 2. Do the work
> > +
> > + 3. Increment operation cursor
> > + 4. Release the mapping via ->iomap_end, if necessary
> > +
> > +Each iomap operation will be covered in more detail below.
> > +This library was covered previously by an `LWN article
> > +<https://lwn.net/Articles/935934/>`_ and a `KernelNewbies page
> > +<https://kernelnewbies.org/KernelProjects/iomap>`_.
> > +
> > +Data Structures and Algorithms
> > +==============================
> > +
> > +Definitions
> > +-----------
> > +
> > + * ``bufferhead``: Shattered remnants of the old buffer cache.
> > + * ``fsblock``: The block size of a file, also known as ``i_blocksize``.
> > + * ``i_rwsem``: The VFS ``struct inode`` rwsemaphore.
> > + * ``invalidate_lock``: The pagecache ``struct address_space``
> > + rwsemaphore that protects against folio removal.
>
> This definition is a bit confusing & maybe even incomplete.
> I think we should use this from header file.
>
> @invalidate_lock: The pagecache ``struct address_sapce`` rwsemaphore
> that guards coherency between page cache contents and file offset->disk
> block mappings in the filesystem during invalidates. It is also used to
> block modification of page cache contents through memory mappings.
>
> Also if we are describing definitions above - then I think we should
> also clarify these locks/terms used in this document (I just looked
> "lock" related terms for now)
>
> - folio lock:
> - dax lock:
Er, what /is/ the dax lock? Is that the dax_read_lock thing that (I
think) wraps the dax rcu lock? Which in turn exists so that we don't
return from the dax offlining function before everyone's dropped all
their references to internal structures?
> - pagecache lock:
> - FS internal mapping lock:
> - Iomap internal operation lock:
>
> > +
> > +struct iomap_ops
> > +----------------
>
> IMO, we should define "struct iomap" in the begining. The reason is
> because iomap_ops functions take "struct iomap" in it's function
> arguments. So it's easier if we describe "struct iomap" before.
Yeah, I agree.
> > +
> > +Every iomap function requires the filesystem to pass an operations
> > +structure to obtain a mapping and (optionally) to release the mapping.
> > +
> > +.. code-block:: c
> > +
> > + struct iomap_ops {
> > + int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
> > + unsigned flags, struct iomap *iomap,
> > + struct iomap *srcmap);
> > +
> > + int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
> > + ssize_t written, unsigned flags,
> > + struct iomap *iomap);
> > + };
> > +
> > +The ``->iomap_begin`` function is called to obtain one mapping for the
> > +range of bytes specified by ``pos`` and ``length`` for the file
> > +``inode``.
>
> I think it is better if we describe ->iomap_begin and ->iomap_end
> in proper sub-sections. Otherwise this looks like we have clobbered
> all the information together :)
>
> ->iomap_begin
> ^^^^^^^^^^^^^^^^^
Yes, I like the explicit section headings better.
> This either returns an existing mapping or reserve/allocates a new
> mapping.
That's a filesystem specific detail -- all that iomap cares about is
that the fs communicates a mapping. Maybe the fs actually had to do a
bunch of work to get that mapping, or maybe it's already laid out
statically, ala zonefs. Either way, it's not a concern of the iomap
library.
> logical file pos and length are in bytes which gets passed
> as function arguments. Filesystem returns the new mapping information
> within ``struct iomap`` which also gets passed as a function argument.
> Filesystems should provide the details of this mapping by filling
> various fields within ``struct iomap``.
"iomap operations call ->iomap_begin to obtain one file mapping for the
range of bytes specified by pos and length for the file inode. This
mapping should be returned through the iomap pointer. The mapping must
cover at least the first byte of the supplied file range, but it does
not need to cover the entire requested range."
> @srcmap agument:
> Note that ->iomap_begin call has srcmap passed as another argument. This is
> mainly used only during the begin phase for COW mappings to identify where
> the reads are to be performed from. Filesystems needs to fill that mapping
> information if iomap should read data for partially written blocks from a
> different location than the write target [4].
> @flags argument:
> These are the operation types which iomap supports.
> IOMAP_WRITE: For doing write I/O.
> <...>
> IOMAP_ZERO:
> IOMAP_REPORT:
> IOMAP_FAULT:
> IOMAP_DIRECT:
> IOMAP_NOWAIT:
> IOMAP_OVERWRITE_ONLY:
> IOMAP_UNSHARE:
> IOMAP_DAX:
I think it's /much/ more valuable to document the exact combinations
that will be passed to ->iomap_begin further down where we talk about
specific operations that iomap performs.
Otherwise, someone is going to look at this list and wonder if they
really need to figure out what IOMAP_ZERO|IOMAP_FAULT|IOMAP_DAX means,
and if it's actually possible (it's not).
>
> ->iomap_end
> ^^^^^^^^^^^^^^^^^
>
> Commit and/or unreserve space which was previously allocated/reserved
> in ``->iomap_begin``. For e.g. during buffered-io, when a short writes
> occurs, filesystem may need to remove the reserved space that was
> allocated during ->iomap_begin.
> For filesystems that use delalloc allocation, we may need to punch out
> delalloc extents from the range that are not dirty in
> the page cache. See comments in
> iomap_file_buffered_write_punch_delalloc() for more info [5][6].
>
> (IMHO) I find above definitions more descriptive.
I don't want to merge the general description with pagecache specific
areas. They already cover punch_delalloc.
> > +
> > +Each iomap operation describes the requested operation through the
> > +``flags`` argument.
> > +The exact value of ``flags`` will be documented in the
> > +operation-specific sections below, but these principles apply generally:
> > +
> > + * For a write operation, ``IOMAP_WRITE`` will be set.
> > + Filesystems must not return ``IOMAP_HOLE`` mappings.
> > +
> > + * For any other operation, ``IOMAP_WRITE`` will not be set.
> > +
>
> Direct-io related operation which bypasses pagecache use IOMAP_DIRECT.
That's covered in the pagecache/directio/dax subsection because I wanted
to document specific combinations that filesystem authors should expect.
> > + * For any operation targetting direct access to storage (fsdax),
> > + ``IOMAP_DAX`` will be set.
> > +
> > +If it is necessary to read existing file contents from a `different
> > +<https://lore.kernel.org/all/[email protected]/>`_ device or
> > +address range on a device, the filesystem should return that information via
> > +``srcmap``.
> > +Only pagecache and fsdax operations support reading from one mapping and
> > +writing to another.
> > +
> > +After the operation completes, the ``->iomap_end`` function, if present,
> > +is called to signal that iomap is finished with a mapping.
> > +Typically, implementations will use this function to tear down any
> > +context that were set up in ``->iomap_begin``.
> > +For example, a write might wish to commit the reservations for the bytes
> > +that were operated upon and unreserve any space that was not operated
> > +upon.
> > +``written`` might be zero if no bytes were touched.
> > +``flags`` will contain the same value passed to ``->iomap_begin``.
> > +iomap ops for reads are not likely to need to supply this function.
> > +
> > +Both functions should return a negative errno code on error, or zero.
>
> minor nit: ... or zero on success.
done.
> > +
> > +struct iomap
> > +------------
> > +
> > +The filesystem returns the mappings via the following structure.
>
> Filesystem returns the contiguous file mapping information of logical
> file offset range to a physically mapped extent via the following
> structure which iomap uses to perform various file and pagecache
> related operations listed above.
How about:
"The filesystem communicates to iomap operations the mappings of byte
ranges of a file to byte ranges of a storage device with the structure
below."
> > +For documentation purposes, the structure has been reordered to group
> > +fields that go together logically.
> > +
> > +.. code-block:: c
> > +
> > + struct iomap {
> > + loff_t offset;
> > + u64 length;
> > +
> > + u16 type;
> > + u16 flags;
> > +
> > + u64 addr;
> > + struct block_device *bdev;
> > + struct dax_device *dax_dev;
> > + void *inline_data;
> > +
> > + void *private;
> > +
> > + const struct iomap_folio_ops *folio_ops;
> > +
> > + u64 validity_cookie;
> > + };
> > +
> > +The information is useful for translating file operations into action.
> > +The actions taken are specific to the target of the operation, such as
> > +disk cache, physical storage devices, or another part of the kernel.
>
> I think the wording "action" & trying to make it so generic w/o mapping
> what "action" refers here for "disk cache", "physical storage device" or
> "other parts of the kernel", gets a bit confusing.
>
> Do you think we should map those to some examples maybe?
> BTW, with added definition of "struct iomap" which I mentioned above,
> I am even fine if we want to drop this paragraph.
Yeah, I'll delete the paragraph.
> > +
> > + * ``offset`` and ``length`` describe the range of file offsets, in
> > + bytes, covered by this mapping.
> > + These fields must always be set by the filesystem.
> > +
> > + * ``type`` describes the type of the space mapping:
>
> This field is set by the filesystem in ->iomap_begin call.
>
> > +
> > + * **IOMAP_HOLE**: No storage has been allocated.
> > + This type must never be returned in response to an IOMAP_WRITE
> > + operation because writes must allocate and map space, and return
> > + the mapping.
> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> > + iomap does not support writing (whether via pagecache or direct
> > + I/O) to a hole.
> > +
> > + * **IOMAP_DELALLOC**: A promise to allocate space at a later time
> > + ("delayed allocation").
> > + If the filesystem returns IOMAP_F_NEW here and the write fails, the
> > + ``->iomap_end`` function must delete the reservation.
> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> > +
> > + * **IOMAP_MAPPED**: The file range maps to specific space on the
> > + storage device.
> > + The device is returned in ``bdev`` or ``dax_dev``.
> > + The device address, in bytes, is returned via ``addr``.
> > +
> > + * **IOMAP_UNWRITTEN**: The file range maps to specific space on the
> > + storage device, but the space has not yet been initialized.
> > + The device is returned in ``bdev`` or ``dax_dev``.
> > + The device address, in bytes, is returned via ``addr``.
> > + Reads will return zeroes to userspace.
>
> Reads to this type of mapping will return zeroes to the caller.
Reads from this type of mapping, but yes.
> > + For a write or writeback operation, the ioend should update the
> > + mapping to MAPPED.
>
> Refer to section "Writeback ioend Completion" for more details.
There are two here -- one for the pagecache, and one for directio.
> > +
> > + * **IOMAP_INLINE**: The file range maps to the memory buffer
> > + specified by ``inline_data``.
> > + For write operation, the ``->iomap_end`` function presumably
> > + handles persisting the data.
>
> Is it? Or do we just mark the inode as dirty?
gfs2 actually starts a transaction in ->iomap_begin and commits or
cancels it in ->iomap_end.
> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> > +
> > + * ``flags`` describe the status of the space mapping.
> > + These flags should be set by the filesystem in ``->iomap_begin``:
> > +
> > + * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
> > + Areas that will not be written to must be zeroed.
>
> In case of DAX, we have to invalidate those existing mappings which
> might have a "hole" page mapped.
Isn't that an internal detail of the fs/dax.c code? The filesystem
doesn't have to do the invalidation or even know about hole pages.
> > + If a write fails and the mapping is a space reservation, the
> > + reservation must be deleted.
> > +
> > + * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
> > + to access any data written.
> > + fdatasync is required to commit these changes to persistent
> > + storage.
> > + This needs to take into account metadata changes that *may* be made
> > + at I/O completion, such as file size updates from direct I/O.
> > +
> > + * **IOMAP_F_SHARED**: The space under the mapping is shared.
> > + Copy on write is necessary to avoid corrupting other file data.
> > +
> > + * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
> > + heads for pagecache operations.
> > + Do not add more uses of this.
> > +
> > + * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
> > + coalesced into this single mapping.
> > + This is only useful for FIEMAP.
> > +
> > + * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
> > + regular file data.
> > + This is only useful for FIEMAP.
> > +
> > + * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
> > + be set by the filesystem for its own purposes.
> > +
> > + These flags can be set by iomap itself during file operations.
> > + The filesystem should supply an ``->iomap_end`` function to observe
> > + these flags:
> > +
> > + * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
> > + using this mapping.
> > +
> > + * **IOMAP_F_STALE**: The mapping was found to be stale.
> > + iomap will call ``->iomap_end`` on this mapping and then
> > + ``->iomap_begin`` to obtain a new mapping.
> > +
> > + Currently, these flags are only set by pagecache operations.
> > +
> > + * ``addr`` describes the device address, in bytes.
> > +
> > + * ``bdev`` describes the block device for this mapping.
> > + This only needs to be set for mapped or unwritten operations.
> > +
> > + * ``dax_dev`` describes the DAX device for this mapping.
> > + This only needs to be set for mapped or unwritten operations, and
> > + only for a fsdax operation.
>
> Looks like we can make this union (bdev and dax_dev). Since depending
> upon IOMAP_DAX or not we only set either dax_dev or bdev.
Separate patch. ;)
> Sorry Darrick. I will stop here for now.
> I will continue it from here later.
Ok. The rest of the doc will hopefully make it more obvious why there's
the generic discussion up here.
--D
> -ritesh
>
>
> > +
> > + * ``inline_data`` points to a memory buffer for I/O involving
> > + ``IOMAP_INLINE`` mappings.
> > + This value is ignored for all other mapping types.
> > +
> > + * ``private`` is a pointer to `filesystem-private information
> > + <https://lore.kernel.org/all/[email protected]/>`_.
> > + This value will be passed unchanged to ``->iomap_end``.
> > +
> > + * ``folio_ops`` will be covered in the section on pagecache operations.
> > +
> > + * ``validity_cookie`` is a magic freshness value set by the filesystem
> > + that should be used to detect stale mappings.
> > + For pagecache operations this is critical for correct operation
> > + because page faults can occur, which implies that filesystem locks
> > + should not be held between ``->iomap_begin`` and ``->iomap_end``.
> > + Filesystems with completely static mappings need not set this value.
> > + Only pagecache operations revalidate mappings.
> > +
> > + XXX: Should fsdax revalidate as well?
> > +
> > +Validation
> > +==========
> > +
> > +**NOTE**: iomap only handles mapping and I/O.
> > +Filesystems must still call out to the VFS to check input parameters
> > +and file state before initiating an I/O operation.
> > +It does not handle updating of timestamps, stripping privileges, or
> > +access control.
> > +
> > +Locking Hierarchy
> > +=================
> > +
> > +iomap requires that filesystems provide their own locking.
> > +There are no locks within iomap itself, though in the course of an
> > +operation iomap may take other locks (e.g. folio/dax locks) as part of
> > +an I/O operation.
> > +Locking with iomap can be split into two categories: above and below
> > +iomap.
> > +
> > +The upper level of lock must coordinate the iomap operation with other
> > +iomap operations.
> > +Generally, the filesystem must take VFS/pagecache locks such as
> > +``i_rwsem`` or ``invalidate_lock`` before calling into iomap.
> > +The exact locking requirements are specific to the type of operation.
> > +
> > +The lower level of lock must coordinate access to the mapping
> > +information.
> > +This lock is filesystem specific and should be held during
> > +``->iomap_begin`` while sampling the mapping and validity cookie.
> > +
> > +The general locking hierarchy in iomap is:
> > +
> > + * VFS or pagecache lock
> > +
> > + * Internal filesystem specific mapping lock
> > +
> > + * iomap operation-specific lock
> > +
> > +The exact locking requirements are specific to the filesystem; for
> > +certain operations, some of these locks can be elided.
> > +All further mention of locking are *recommendations*, not mandates.
> > +Each filesystem author must figure out the locking for themself.
> > +
> > +iomap Operations
> > +================
> > +
> > +Below are a discussion of the file operations that iomap implements.
> > +
> > +Buffered I/O
> > +------------
> > +
> > +Buffered I/O is the default file I/O path in Linux.
> > +File contents are cached in memory ("pagecache") to satisfy reads and
> > +writes.
> > +Dirty cache will be written back to disk at some point that can be
> > +forced via ``fsync`` and variants.
> > +
> > +iomap implements nearly all the folio and pagecache management that
> > +filesystems once had to implement themselves.
> > +This means that the filesystem need not know the details of allocating,
> > +mapping, managing uptodate and dirty state, or writeback of pagecache
> > +folios.
> > +Unless the filesystem explicitly opts in to buffer heads, they will not
> > +be used, which makes buffered I/O much more efficient, and ``willy``
> > +much happier.
> > +
> > +struct address_space_operations
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The following iomap functions can be referenced directly from the
> > +address space operations structure:
> > +
> > + * ``iomap_dirty_folio``
> > + * ``iomap_release_folio``
> > + * ``iomap_invalidate_folio``
> > + * ``iomap_is_partially_uptodate``
> > +
> > +The following address space operations can be wrapped easily:
> > +
> > + * ``read_folio``
> > + * ``readahead``
> > + * ``writepages``
> > + * ``bmap``
> > + * ``swap_activate``
> > +
> > +struct iomap_folio_ops
> > +~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The ``->iomap_begin`` function for pagecache operations may set the
> > +``struct iomap::folio_ops`` field to an ops structure to override
> > +default behaviors of iomap:
> > +
> > +.. code-block:: c
> > +
> > + struct iomap_folio_ops {
> > + struct folio *(*get_folio)(struct iomap_iter *iter, loff_t pos,
> > + unsigned len);
> > + void (*put_folio)(struct inode *inode, loff_t pos, unsigned copied,
> > + struct folio *folio);
> > + bool (*iomap_valid)(struct inode *inode, const struct iomap *iomap);
> > + };
> > +
> > +iomap calls these functions:
> > +
> > + - ``get_folio``: Called to allocate and return an active reference to
> > + a locked folio prior to starting a write.
> > + If this function is not provided, iomap will call
> > + ``iomap_get_folio``.
> > + This could be used to `set up per-folio filesystem state
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + for a write.
> > +
> > + - ``put_folio``: Called to unlock and put a folio after a pagecache
> > + operation completes.
> > + If this function is not provided, iomap will ``folio_unlock`` and
> > + ``folio_put`` on its own.
> > + This could be used to `commit per-folio filesystem state
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + that was set up by ``->get_folio``.
> > +
> > + - ``iomap_valid``: The filesystem may not hold locks between
> > + ``->iomap_begin`` and ``->iomap_end`` because pagecache operations
> > + can take folio locks, fault on userspace pages, initiate writeback
> > + for memory reclamation, or engage in other time-consuming actions.
> > + If a file's space mapping data are mutable, it is possible that the
> > + mapping for a particular pagecache folio can `change in the time it
> > + takes
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + to allocate, install, and lock that folio.
> > + For such files, the mapping *must* be revalidated after the folio
> > + lock has been taken so that iomap can manage the folio correctly.
> > + The filesystem's ``->iomap_begin`` function must sample a sequence
> > + counter into ``struct iomap::validity_cookie`` at the same time that
> > + it populates the mapping fields.
> > + It must then provide a ``->iomap_valid`` function to compare the
> > + validity cookie against the source counter and return whether or not
> > + the mapping is still valid.
> > + If the mapping is not valid, the mapping will be sampled again.
> > +
> > +These ``struct kiocb`` flags are significant for buffered I/O with
> > +iomap:
> > +
> > + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> > + already in memory, we do not have to initiate other I/O, and we
> > + acquire all filesystem locks without blocking.
> > + Neither this flag nor its definition ``RWF_NOWAIT`` actually define
> > + what this flag means, so this is the best the author could come up
> > + with.
> > +
> > +Internal per-Folio State
> > +~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +If the fsblock size matches the size of a pagecache folio, it is assumed
> > +that all disk I/O operations will operate on the entire folio.
> > +The uptodate (memory contents are at least as new as what's on disk) and
> > +dirty (memory contents are newer than what's on disk) status of the
> > +folio are all that's needed for this case.
> > +
> > +If the fsblock size is less than the size of a pagecache folio, iomap
> > +tracks the per-fsblock uptodate and dirty state itself.
> > +This enables iomap to handle both "bs < ps" `filesystems
> > +<https://lore.kernel.org/all/[email protected]/>`_
> > +and large folios in the pagecache.
> > +
> > +iomap internally tracks two state bits per fsblock:
> > +
> > + * ``uptodate``: iomap will try to keep folios fully up to date.
> > + If there are read(ahead) errors, those fsblocks will not be marked
> > + uptodate.
> > + The folio itself will be marked uptodate when all fsblocks within the
> > + folio are uptodate.
> > +
> > + * ``dirty``: iomap will set the per-block dirty state when programs
> > + write to the file.
> > + The folio itself will be marked dirty when any fsblock within the
> > + folio is dirty.
> > +
> > +iomap also tracks the amount of read and write disk IOs that are in
> > +flight.
> > +This structure is much lighter weight than ``struct buffer_head``.
> > +
> > +Filesystems wishing to turn on large folios in the pagecache should call
> > +``mapping_set_large_folios`` when initializing the incore inode.
> > +
> > +Readahead and Reads
> > +~~~~~~~~~~~~~~~~~~~
> > +
> > +The ``iomap_readahead`` function initiates readahead to the pagecache.
> > +The ``iomap_read_folio`` function reads one folio's worth of data into
> > +the pagecache.
> > +The ``flags`` argument to ``->iomap_begin`` will be set to zero.
> > +The pagecache takes whatever locks it needs before calling the
> > +filesystem.
> > +
> > +Writes
> > +~~~~~~
> > +
> > +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
> > +pagecache.
> > +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
> > +the ``flags`` argument to ``->iomap_begin``.
> > +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
> > +
> > +mmap Write Faults
> > +^^^^^^^^^^^^^^^^^
> > +
> > +The ``iomap_page_mkwrite`` function handles a write fault to a folio the
> > +pagecache.
> > +``IOMAP_WRITE | IOMAP_FAULT`` will be passed as the ``flags`` argument
> > +to ``->iomap_begin``.
> > +Callers commonly take the mmap ``invalidate_lock`` in shared or
> > +exclusive mode.
> > +
> > +Write Failures
> > +^^^^^^^^^^^^^^
> > +
> > +After a short write to the pagecache, the areas not written will not
> > +become marked dirty.
> > +The filesystem must arrange to `cancel
> > +<https://lore.kernel.org/all/[email protected]/>`_
> > +such `reservations
> > +<https://lore.kernel.org/linux-xfs/[email protected]/>`_
> > +because writeback will not consume the reservation.
> > +The ``iomap_file_buffered_write_punch_delalloc`` can be called from a
> > +``->iomap_end`` function to find all the clean areas of the folios
> > +caching a fresh (``IOMAP_F_NEW``) delalloc mapping.
> > +It takes the ``invalidate_lock``.
> > +
> > +The filesystem should supply a callback ``punch`` will be called for
> > +each file range in this state.
> > +This function must *only* remove delayed allocation reservations, in
> > +case another thread racing with the current thread writes successfully
> > +to the same region and triggers writeback to flush the dirty data out to
> > +disk.
> > +
> > +Truncation
> > +^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_truncate_page`` to zero the bytes in the
> > +pagecache from EOF to the end of the fsblock during a file truncation
> > +operation.
> > +``truncate_setsize`` or ``truncate_pagecache`` will take care of
> > +everything after the EOF block.
> > +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Zeroing for File Operations
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_zero_range`` to perform zeroing of the
> > +pagecache for non-truncation file operations that are not aligned to
> > +the fsblock size.
> > +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Unsharing Reflinked File Data
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_file_unshare`` to force a file sharing
> > +storage with another file to preemptively copy the shared data to newly
> > +allocate storage.
> > +``IOMAP_WRITE | IOMAP_UNSHARE`` will be passed as the ``flags`` argument
> > +to ``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Writeback
> > +~~~~~~~~~
> > +
> > +Filesystems can call ``iomap_writepages`` to respond to a request to
> > +write dirty pagecache folios to disk.
> > +The ``mapping`` and ``wbc`` parameters should be passed unchanged.
> > +The ``wpc`` pointer should be allocated by the filesystem and must
> > +be initialized to zero.
> > +
> > +The pagecache will lock each folio before trying to schedule it for
> > +writeback.
> > +It does not lock ``i_rwsem`` or ``invalidate_lock``.
> > +
> > +The dirty bit will be cleared for all folios run through the
> > +``->map_blocks`` machinery described below even if the writeback fails.
> > +This is to prevent dirty folio clots when storage devices fail; an
> > +``-EIO`` is recorded for userspace to collect via ``fsync``.
> > +
> > +The ``ops`` structure must be specified and is as follows:
> > +
> > +struct iomap_writeback_ops
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +.. code-block:: c
> > +
> > + struct iomap_writeback_ops {
> > + int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
> > + loff_t offset, unsigned len);
> > + int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
> > + void (*discard_folio)(struct folio *folio, loff_t pos);
> > + };
> > +
> > +The fields are as follows:
> > +
> > + - ``map_blocks``: Sets ``wpc->iomap`` to the space mapping of the file
> > + range (in bytes) given by ``offset`` and ``len``.
> > + iomap calls this function for each fs block in each dirty folio,
> > + even if the mapping returned is longer than one fs block.
> > + Do not return ``IOMAP_INLINE`` mappings here; the ``->iomap_end``
> > + function must deal with persisting written data.
> > + Filesystems can skip a potentially expensive mapping lookup if the
> > + mappings have not changed.
> > + This revalidation must be open-coded by the filesystem; it is
> > + unclear if ``iomap::validity_cookie`` can be reused for this
> > + purpose.
> > + This function is required.
> > +
> > + - ``prepare_ioend``: Enables filesystems to transform the writeback
> > + ioend or perform any other prepatory work before the writeback I/O
> > + is submitted.
> > + A filesystem can override the ``->bi_end_io`` function for its own
> > + purposes, such as kicking the ioend completion to a workqueue if the
> > + bio is completed in interrupt context.
> > + This function is optional.
> > +
> > + - ``discard_folio``: iomap calls this function after ``->map_blocks``
> > + fails schedule I/O for any part of a dirty folio.
> > + The function should throw away any reservations that may have been
> > + made for the write.
> > + The folio will be marked clean and an ``-EIO`` recorded in the
> > + pagecache.
> > + Filesystems can use this callback to `remove
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + delalloc reservations to avoid having delalloc reservations for
> > + clean pagecache.
> > + This function is optional.
> > +
> > +Writeback ioend Completion
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +iomap creates chains of ``struct iomap_ioend`` objects that wrap the
> > +``bio`` that is used to write pagecache data to disk.
> > +By default, iomap finishes writeback ioends by clearing the writeback
> > +bit on the folios attached to the ``ioend``.
> > +If the write failed, it will also set the error bits on the folios and
> > +the address space.
> > +This can happen in interrupt or process context, depending on the
> > +storage device.
> > +
> > +Filesystems that need to update internal bookkeeping (e.g. unwritten
> > +extent conversions) should provide a ``->prepare_ioend`` function to
> > +override the ``struct iomap_end::bio::bi_end_io`` with its own function.
> > +This function should call ``iomap_finish_ioends`` after finishing its
> > +own work.
> > +
> > +Some filesystems may wish to `amortize the cost of running metadata
> > +transactions
> > +<https://lore.kernel.org/all/[email protected]/>`_
> > +for post-writeback updates by batching them.
> > +They may also require transactions to run from process context, which
> > +implies punting batches to a workqueue.
> > +iomap ioends contain a ``list_head`` to enable batching.
> > +
> > +Given a batch of ioends, iomap has a few helpers to assist with
> > +amortization:
> > +
> > + * ``iomap_sort_ioends``: Sort all the ioends in the list by file
> > + offset.
> > +
> > + * ``iomap_ioend_try_merge``: Given an ioend that is not in any list and
> > + a separate list of sorted ioends, merge as many of the ioends from
> > + the head of the list into the given ioend.
> > + ioends can only be merged if the file range and storage addresses are
> > + contiguous; the unwritten and shared status are the same; and the
> > + write I/O outcome is the same.
> > + The merged ioends become their own list.
> > +
> > + * ``iomap_finish_ioends``: Finish an ioend that possibly has other
> > + ioends linked to it.
> > +
> > +Direct I/O
> > +----------
> > +
> > +In Linux, direct I/O is defined as file I/O that is issued directly to
> > +storage, bypassing the pagecache.
> > +
> > +The ``iomap_dio_rw`` function implements O_DIRECT (direct I/O) reads and
> > +writes for files.
> > +An optional ``ops`` parameter can be passed to change the behavior of
> > +direct I/O.
> > +The ``done_before`` parameter should be set if writes have been
> > +initiated prior to the call.
> > +The direction of the I/O is determined from the ``iocb`` passed in.
> > +
> > +The ``flags`` argument can be any of the following values:
> > +
> > + * ``IOMAP_DIO_FORCE_WAIT``: Wait for the I/O to complete even if the
> > + kiocb is not synchronous.
> > +
> > + * ``IOMAP_DIO_OVERWRITE_ONLY``: Allocating blocks, zeroing partial
> > + blocks, and extensions of the file size are not allowed.
> > + The entire file range must to map to a single written or unwritten
> > + extent.
> > + This flag exists to enable issuing concurrent direct IOs with only
> > + the shared ``i_rwsem`` held when the file I/O range is not aligned to
> > + the filesystem block size.
> > + ``-EAGAIN`` will be returned if the operation cannot proceed.
> > +
> > + * ``IOMAP_DIO_PARTIAL``: If a page fault occurs, return whatever
> > + progress has already been made.
> > + The caller may deal with the page fault and retry the operation.
> > +
> > +These ``struct kiocb`` flags are significant for direct I/O with iomap:
> > +
> > + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> > + already in memory, we do not have to initiate other I/O, and we
> > + acquire all filesystem locks without blocking.
> > +
> > + * ``IOCB_SYNC``: Ensure that the device has persisted data to disk
> > + before completing the call.
> > + In the case of pure overwrites, the I/O may be issued with FUA
> > + enabled.
> > +
> > + * ``IOCB_HIPRI``: Poll for I/O completion instead of waiting for an
> > + interrupt.
> > + Only meaningful for asynchronous I/O, and only if the entire I/O can
> > + be issued as a single ``struct bio``.
> > +
> > + * ``IOCB_DIO_CALLER_COMP``: Try to run I/O completion from the caller's
> > + process context.
> > + See ``linux/fs.h`` for more details.
> > +
> > +Filesystems should call ``iomap_dio_rw`` from ``->read_iter`` and
> > +``->write_iter``, and set ``FMODE_CAN_ODIRECT`` in the ``->open``
> > +function for the file.
> > +They should not set ``->direct_IO``, which is deprecated.
> > +
> > +If a filesystem wishes to perform its own work before direct I/O
> > +completion, it should call ``__iomap_dio_rw``.
> > +If its return value is not an error pointer or a NULL pointer, the
> > +filesystem should pass the return value to ``iomap_dio_complete`` after
> > +finishing its internal work.
> > +
> > +Direct Reads
> > +~~~~~~~~~~~~
> > +
> > +A direct I/O read initiates a read I/O from the storage device to the
> > +caller's buffer.
> > +Dirty parts of the pagecache are flushed to storage before initiating
> > +the read io.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT`` with
> > +any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Direct Writes
> > +~~~~~~~~~~~~~
> > +
> > +A direct I/O write initiates a write I/O to the storage device to the
> > +caller's buffer.
> > +Dirty parts of the pagecache are flushed to storage before initiating
> > +the write io.
> > +The pagecache is invalidated both before and after the write io.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT |
> > +IOMAP_WRITE`` with any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> > + blocks is not allowed.
> > + The entire file range must to map to a single written or unwritten
> > + extent.
> > + The file I/O range must be aligned to the filesystem block size.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared or exclusive mode.
> > +
> > +struct iomap_dio_ops:
> > +~~~~~~~~~~~~~~~~~~~~~
> > +.. code-block:: c
> > +
> > + struct iomap_dio_ops {
> > + void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
> > + loff_t file_offset);
> > + int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
> > + unsigned flags);
> > + struct bio_set *bio_set;
> > + };
> > +
> > +The fields of this structure are as follows:
> > +
> > + - ``submit_io``: iomap calls this function when it has constructed a
> > + ``struct bio`` object for the I/O requested, and wishes to submit it
> > + to the block device.
> > + If no function is provided, ``submit_bio`` will be called directly.
> > + Filesystems that would like to perform additional work before (e.g.
> > + data replication for btrfs) should implement this function.
> > +
> > + - ``end_io``: This is called after the ``struct bio`` completes.
> > + This function should perform post-write conversions of unwritten
> > + extent mappings, handle write failures, etc.
> > + The ``flags`` argument may be set to a combination of the following:
> > +
> > + * ``IOMAP_DIO_UNWRITTEN``: The mapping was unwritten, so the ioend
> > + should mark the extent as written.
> > +
> > + * ``IOMAP_DIO_COW``: Writing to the space in the mapping required a
> > + copy on write operation, so the ioend should switch mappings.
> > +
> > + - ``bio_set``: This allows the filesystem to provide a custom bio_set
> > + for allocating direct I/O bios.
> > + This enables filesystems to `stash additional per-bio information
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + for private use.
> > + If this field is NULL, generic ``struct bio`` objects will be used.
> > +
> > +Filesystems that want to perform extra work after an I/O completion
> > +should set a custom ``->bi_end_io`` function via ``->submit_io``.
> > +Afterwards, the custom endio function must call
> > +``iomap_dio_bio_end_io`` to finish the direct I/O.
> > +
> > +DAX I/O
> > +-------
> > +
> > +Storage devices that can be directly mapped as memory support a new
> > +access mode known as "fsdax".
> > +
> > +fsdax Reads
> > +~~~~~~~~~~~
> > +
> > +A fsdax read performs a memcpy from storage device to the caller's
> > +buffer.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX`` with any
> > +combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +fsdax Writes
> > +~~~~~~~~~~~~
> > +
> > +A fsdax write initiates a memcpy to the storage device to the caller's
> > +buffer.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX |
> > +IOMAP_WRITE`` with any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> > + blocks is not allowed.
> > + The entire file range must to map to a single written or unwritten
> > + extent.
> > + The file I/O range must be aligned to the filesystem block size.
> > +
> > +Callers commonly hold ``i_rwsem`` in exclusive mode.
> > +
> > +mmap Faults
> > +~~~~~~~~~~~
> > +
> > +The ``dax_iomap_fault`` function handles read and write faults to fsdax
> > +storage.
> > +For a read fault, ``IOMAP_DAX | IOMAP_FAULT`` will be passed as the
> > +``flags`` argument to ``->iomap_begin``.
> > +For a write fault, ``IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE`` will be
> > +passed as the ``flags`` argument to ``->iomap_begin``.
> > +
> > +Callers commonly hold the same locks as they do to call their iomap
> > +pagecache counterparts.
> > +
> > +Truncation, fallocate, and Unsharing
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +For fsdax files, the following functions are provided to replace their
> > +iomap pagecache I/O counterparts.
> > +The ``flags`` argument to ``->iomap_begin`` are the same as the
> > +pagecache counterparts, with ``IOMAP_DIO`` added.
> > +
> > + * ``dax_file_unshare``
> > + * ``dax_zero_range``
> > + * ``dax_truncate_page``
> > +
> > +Callers commonly hold the same locks as they do to call their iomap
> > +pagecache counterparts.
> > +
> > +SEEK_DATA
> > +---------
> > +
> > +The ``iomap_seek_data`` function implements the SEEK_DATA "whence" value
> > +for llseek.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +
> > +For unwritten mappings, the pagecache will be searched.
> > +Regions of the pagecache with a folio mapped and uptodate fsblocks
> > +within those folios will be reported as data areas.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +SEEK_HOLE
> > +---------
> > +
> > +The ``iomap_seek_hole`` function implements the SEEK_HOLE "whence" value
> > +for llseek.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +
> > +For unwritten mappings, the pagecache will be searched.
> > +Regions of the pagecache with no folio mapped, or a !uptodate fsblock
> > +within a folio will be reported as sparse hole areas.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Swap File Activation
> > +--------------------
> > +
> > +The ``iomap_swapfile_activate`` function finds all the base-page aligned
> > +regions in a file and sets them up as swap space.
> > +The file will be ``fsync()``'d before activation.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +All mappings must be mapped or unwritten; cannot be dirty or shared, and
> > +cannot span multiple block devices.
> > +Callers must hold ``i_rwsem`` in exclusive mode; this is already
> > +provided by ``swapon``.
> > +
> > +Extent Map Reporting (FS_IOC_FIEMAP)
> > +------------------------------------
> > +
> > +The ``iomap_fiemap`` function exports file extent mappings to userspace
> > +in the format specified by the ``FS_IOC_FIEMAP`` ioctl.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Block Map Reporting (FIBMAP)
> > +----------------------------
> > +
> > +``iomap_bmap`` implements FIBMAP.
> > +The calling conventions are the same as for FIEMAP.
> > +This function is only provided to maintain compatibility for filesystems
> > +that implemented FIBMAP prior to conversion.
> > +This ioctl is deprecated; do not add a FIBMAP implementation to
> > +filesystems that do not have it.
> > +Callers should probably hold ``i_rwsem`` in shared mode, but this is
> > +unclear.
> > +
> > +Porting Guide
> > +=============
> > +
> > +Why Convert to iomap?
> > +---------------------
> > +
> > +There are several reasons to convert a filesystem to iomap:
> > +
> > + 1. The classic Linux I/O path is not terribly efficient.
> > + Pagecache operations lock a single base page at a time and then call
> > + into the filesystem to return a mapping for only that page.
> > + Direct I/O operations build I/O requests a single file block at a
> > + time.
> > + This worked well enough for direct/indirect-mapped filesystems such
> > + as ext2, but is very inefficient for extent-based filesystems such
> > + as XFS.
> > +
> > + 2. Large folios are only supported via iomap; there are no plans to
> > + convert the old buffer_head path to use them.
> > +
> > + 3. Direct access to storage on memory-like devices (fsdax) is only
> > + supported via iomap.
> > +
> > + 4. Lower maintenance overhead for individual filesystem maintainers.
> > + iomap handles common pagecache related operations itself, such as
> > + allocating, instantiating, locking, and unlocking of folios.
> > + No ->write_begin(), ->write_end() or direct_IO
> > + address_space_operations are required to be implemented by
> > + filesystem using iomap.
> > +
> > +How to Convert to iomap?
> > +------------------------
> > +
> > +First, add ``#include <linux/iomap.h>`` from your source code and add
> > +``select FS_IOMAP`` to your filesystem's Kconfig option.
> > +Build the kernel, run fstests with the ``-g all`` option across a wide
> > +variety of your filesystem's supported configurations to build a
> > +baseline of which tests pass and which ones fail.
> > +
> > +The recommended approach is first to implement ``->iomap_begin`` (and
> > +``->iomap->end`` if necessary) to allow iomap to obtain a read-only
> > +mapping of a file range.
> > +In most cases, this is a relatively trivial conversion of the existing
> > +``get_block()`` function for read-only mappings.
> > +``FS_IOC_FIEMAP`` is a good first target because it is trivial to
> > +implement support for it and then to determine that the extent map
> > +iteration is correct from userspace.
> > +If FIEMAP is returning the correct information, it's a good sign that
> > +other read-only mapping operations will do the right thing.
> > +
> > +Next, modify the filesystem's ``get_block(create = false)``
> > +implementation to use the new ``->iomap_begin`` implementation to map
> > +file space for selected read operations.
> > +Hide behind a debugging knob the ability to switch on the iomap mapping
> > +functions for selected call paths.
> > +It is necessary to write some code to fill out the bufferhead-based
> > +mapping information from the ``iomap`` structure, but the new functions
> > +can be tested without needing to implement any iomap APIs.
> > +
> > +Once the read-only functions are working like this, convert each high
> > +level file operation one by one to use iomap native APIs instead of
> > +going through ``get_block()``.
> > +Done one at a time, regressions should be self evident.
> > +You *do* have a regression test baseline for fstests, right?
> > +It is suggested to convert swap file activation, ``SEEK_DATA``, and
> > +``SEEK_HOLE`` before tackling the I/O paths.
> > +A likely complexity at this point will be converting the buffered read
> > +I/O path because of bufferheads.
> > +The buffered read I/O paths doesn't need to be converted yet, though the
> > +direct I/O read path should be converted in this phase.
> > +
> > +At this point, you should look over your ``->iomap_begin`` function.
> > +If it switches between large blocks of code based on dispatching of the
> > +``flags`` argument, you should consider breaking it up into
> > +per-operation iomap ops with smaller, more cohesive functions.
> > +XFS is a good example of this.
> > +
> > +The next thing to do is implement ``get_blocks(create == true)``
> > +functionality in the ``->iomap_begin``/``->iomap_end`` methods.
> > +It is strongly recommended to create separate mapping functions and
> > +iomap ops for write operations.
> > +Then convert the direct I/O write path to iomap, and start running fsx
> > +w/ DIO enabled in earnest on filesystem.
> > +This will flush out lots of data integrity corner case bugs that the new
> > +write mapping implementation introduces.
> > +
> > +Now, convert any remaining file operations to call the iomap functions.
> > +This will get the entire filesystem using the new mapping functions, and
> > +they should largely be debugged and working correctly after this step.
> > +
> > +Most likely at this point, the buffered read and write paths will still
> > +to be converted.
> > +The mapping functions should all work correctly, so all that needs to be
> > +done is rewriting all the code that interfaces with bufferheads to
> > +interface with iomap and folios.
> > +It is much easier first to get regular file I/O (without any fancy
> > +features like fscrypt, fsverity, compression, or data=journaling)
> > +converted to use iomap.
> > +Some of those fancy features (fscrypt and compression) aren't
> > +implemented yet in iomap.
> > +For unjournalled filesystems that use the pagecache for symbolic links
> > +and directories, you might also try converting their handling to iomap.
> > +
> > +The rest is left as an exercise for the reader, as it will be different
> > +for every filesystem.
> > +If you encounter problems, email the people and lists in
> > +``get_maintainers.pl`` for help.
> > +
> > +Bugs and Limitations
> > +====================
> > +
> > + * No support for fscrypt.
> > + * No support for compression.
> > + * No support for fsverity yet.
> > + * Strong assumptions that IO should work the way it does on XFS.
> > + * Does iomap *actually* work for non-regular file data?
> > +
> > +Patches welcome!
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 8754ac2c259d..2ddd94d43ecf 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -8483,6 +8483,7 @@ R: Darrick J. Wong <[email protected]>
> > L: [email protected]
> > L: [email protected]
> > S: Supported
> > +F: Documentation/filesystems/iomap.txt
> > F: fs/iomap/
> > F: include/linux/iomap.h
> >
>
"Darrick J. Wong" <[email protected]> writes:
> On Mon, Jun 10, 2024 at 02:27:28PM +0530, Ritesh Harjani wrote:
>>
>> Hello Darrick,
>>
>> "Darrick J. Wong" <[email protected]> writes:
>>
>> > From: Darrick J. Wong <[email protected]>
>> >
>> > This is the fourth attempt at documenting the design of iomap and how to
>>
>> I agree that this isn't needed in the commit msg ("fourth attempt").
>
> Ok. "Coapture the design of iomap and how to port..."
>
>> > port filesystems to use it. Apologies for all the rst formatting, but
>> > it's necessary to distinguish code from regular text.
>> >
>> > A lot of this has been collected from various email conversations, code
>> > comments, commit messages, my own understanding of iomap, and
>> > Ritesh/Luis' previous efforts to create a document. Please note a large
>> > part of this has been taken from Dave's reply to last iomap doc
>> > patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
>> > other iomap developers who have taken time to explain the iomap design
>> > in various emails, commits, comments etc.
>> >
>> > Cc: Dave Chinner <[email protected]>
>> > Cc: Matthew Wilcox <[email protected]>
>> > Cc: Christoph Hellwig <[email protected]>
>> > Cc: Christian Brauner <[email protected]>
>> > Cc: Ojaswin Mujoo <[email protected]>
>> > Cc: Jan Kara <[email protected]>
>> > Cc: Luis Chamberlain <[email protected]>
>> > Inspired-by: Ritesh Harjani (IBM) <[email protected]>
>>
>> I am not sure if this is even a valid or accepted tag.
>> But sure thanks! :)
>
> They're freeform tags, so they can be everything everyone wants them to
> be! Drum circle kumbaya etc. :P
>
>> > Signed-off-by: Darrick J. Wong <[email protected]>
>> > ---
>> > Documentation/filesystems/index.rst | 1
>> > Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
>> > MAINTAINERS | 1
>> > 3 files changed, 1062 insertions(+)
>> > create mode 100644 Documentation/filesystems/iomap.rst
>> >
>> > diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
>> > index 8f5c1ee02e2f..b010cc8df32d 100644
>> > --- a/Documentation/filesystems/index.rst
>> > +++ b/Documentation/filesystems/index.rst
>> > @@ -34,6 +34,7 @@ algorithms work.
>> > seq_file
>> > sharedsubtree
>> > idmappings
>> > + iomap
>> >
>> > automount-support
>> >
>> > diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
>> > new file mode 100644
>> > index 000000000000..a478b55e4135
>> > --- /dev/null
>> > +++ b/Documentation/filesystems/iomap.rst
>> > @@ -0,0 +1,1060 @@
>> > +.. SPDX-License-Identifier: GPL-2.0
>> > +.. _iomap:
>> > +
>> > +..
>> > + Dumb style notes to maintain the author's sanity:
>> > + Please try to start sentences on separate lines so that
>> > + sentence changes don't bleed colors in diff.
>> > + Heading decorations are documented in sphinx.rst.
>> > +
>> > +============================
>> > +VFS iomap Design and Porting
>> > +============================
>> > +
>> > +.. toctree::
>> > +
>> > +Introduction
>> > +============
>> > +
>> > +iomap is a filesystem library for handling various filesystem operations
>> > +that involves mapping of file's logical offset ranges to physical
>> > +extents.
>> > +This origins of this library is the file I/O path that XFS once used; it
>> > +has now been extended to cover several other operations.
>> > +The library provides various APIs for implementing various file and
>> ^^^^ redundant "various"
>>
>> > +pagecache operations, such as:
>> > +
>> > + * Pagecache reads and writes
>> > + * Folio write faults to the pagecache
>> > + * Writeback of dirty folios
>> > + * Direct I/O reads and writes
>>
>> Dax I/O reads and writes.
>> ... as well please?
>
> It's really fsdax I/O reads, writes, loads, and stores, isn't it?
>
It felt like dax_iomap_rw() belongs to fs/iomap.
But nevertheless, we could skip it if we are targetting fs/iomap/
lib.
>>
>> > + * FIEMAP
>> > + * lseek ``SEEK_DATA`` and ``SEEK_HOLE``
>> > + * swapfile activation
>> > +
>>
>> > +Who Should Read This?
>> > +=====================
>> > +
>> > +The target audience for this document are filesystem, storage, and
>>
>> /s/and/,/
>>
>>
>> > +pagecache programmers and code reviewers.
>>
>> Not sure if we even need this secion "Who Should Read This".
>
> That was a review comment from Luis' attempt to write this document:
> https://lore.kernel.org/linux-xfs/[email protected]/
>
> Also I guess I should state explicitly:
>
> "If you are working on PCI, machine architectures, or device drivers,
> you are most likely in the wrong place."
>
>> > +The goal of this document is to provide a brief discussion of the
>> > +design and capabilities of iomap, followed by a more detailed catalog
>> > +of the interfaces presented by iomap.
>> > +If you change iomap, please update this design document.
>>
>> The details of "goal of this document..." -> can be a part of
>> separate paragraph in "Introduction" section itself.
>>
>> > +
>> > +But Why?
>> > +========
>>
>> "Why Iomap?" is more clean IMO.
>
> "Why VFS iomap?", then.
>
>> > +
>> > +Unlike the classic Linux I/O model which breaks file I/O into small
>> > +units (generally memory pages or blocks) and looks up space mappings on
>> > +the basis of that unit, the iomap model asks the filesystem for the
>> > +largest space mappings that it can create for a given file operation and
>> > +initiates operations on that basis.
>> > +This strategy improves the filesystem's visibility into the size of the
>> > +operation being performed, which enables it to combat fragmentation with
>> > +larger space allocations when possible.
>> > +Larger space mappings improve runtime performance by amortizing the cost
>> > +of a mapping function call into the filesystem across a larger amount of
>>
>> s/call/calls
>
> Done.
>
>> > +data.
>> > +
>> > +At a high level, an iomap operation `looks like this
>> > +<https://lore.kernel.org/all/[email protected]/>`_:
>> > +
>> > +1. For each byte in the operation range...
>> > +
>> > + 1. Obtain space mapping via ->iomap_begin
>> > + 2. For each sub-unit of work...
>> > +
>> > + 1. Revalidate the mapping and go back to (1) above, if necessary
>> > + 2. Do the work
>> > +
>> > + 3. Increment operation cursor
>> > + 4. Release the mapping via ->iomap_end, if necessary
>> > +
>> > +Each iomap operation will be covered in more detail below.
>> > +This library was covered previously by an `LWN article
>> > +<https://lwn.net/Articles/935934/>`_ and a `KernelNewbies page
>> > +<https://kernelnewbies.org/KernelProjects/iomap>`_.
>> > +
>> > +Data Structures and Algorithms
>> > +==============================
>> > +
>> > +Definitions
>> > +-----------
>> > +
>> > + * ``bufferhead``: Shattered remnants of the old buffer cache.
>> > + * ``fsblock``: The block size of a file, also known as ``i_blocksize``.
>> > + * ``i_rwsem``: The VFS ``struct inode`` rwsemaphore.
>> > + * ``invalidate_lock``: The pagecache ``struct address_space``
>> > + rwsemaphore that protects against folio removal.
>>
>> This definition is a bit confusing & maybe even incomplete.
>> I think we should use this from header file.
>>
>> @invalidate_lock: The pagecache ``struct address_sapce`` rwsemaphore
>> that guards coherency between page cache contents and file offset->disk
>> block mappings in the filesystem during invalidates. It is also used to
>> block modification of page cache contents through memory mappings.
>>
>> Also if we are describing definitions above - then I think we should
>> also clarify these locks/terms used in this document (I just looked
>> "lock" related terms for now)
>>
>> - folio lock:
>> - dax lock:
>
> Er, what /is/ the dax lock? Is that the dax_read_lock thing that (I
> think) wraps the dax rcu lock? Which in turn exists so that we don't
> return from the dax offlining function before everyone's dropped all
> their references to internal structures?
>
>> - pagecache lock:
>> - FS internal mapping lock:
>> - Iomap internal operation lock:
>>
>> > +
>> > +struct iomap_ops
>> > +----------------
>>
>> IMO, we should define "struct iomap" in the begining. The reason is
>> because iomap_ops functions take "struct iomap" in it's function
>> arguments. So it's easier if we describe "struct iomap" before.
>
> Yeah, I agree.
>
>> > +
>> > +Every iomap function requires the filesystem to pass an operations
>> > +structure to obtain a mapping and (optionally) to release the mapping.
>> > +
>> > +.. code-block:: c
>> > +
>> > + struct iomap_ops {
>> > + int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
>> > + unsigned flags, struct iomap *iomap,
>> > + struct iomap *srcmap);
>> > +
>> > + int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
>> > + ssize_t written, unsigned flags,
>> > + struct iomap *iomap);
>> > + };
>> > +
>> > +The ``->iomap_begin`` function is called to obtain one mapping for the
>> > +range of bytes specified by ``pos`` and ``length`` for the file
>> > +``inode``.
>>
>> I think it is better if we describe ->iomap_begin and ->iomap_end
>> in proper sub-sections. Otherwise this looks like we have clobbered
>> all the information together :)
>>
>> ->iomap_begin
>> ^^^^^^^^^^^^^^^^^
>
> Yes, I like the explicit section headings better.
>
yup.
>> This either returns an existing mapping or reserve/allocates a new
>> mapping.
>
> That's a filesystem specific detail -- all that iomap cares about is
> that the fs communicates a mapping. Maybe the fs actually had to do a
> bunch of work to get that mapping, or maybe it's already laid out
> statically, ala zonefs. Either way, it's not a concern of the iomap
> library.
>
>> logical file pos and length are in bytes which gets passed
>> as function arguments. Filesystem returns the new mapping information
>> within ``struct iomap`` which also gets passed as a function argument.
>> Filesystems should provide the details of this mapping by filling
>> various fields within ``struct iomap``.
>
> "iomap operations call ->iomap_begin to obtain one file mapping for the
> range of bytes specified by pos and length for the file inode. This
> mapping should be returned through the iomap pointer. The mapping must
> cover at least the first byte of the supplied file range, but it does
> not need to cover the entire requested range."
>
I like it. Thanks for adding that detail in the last line.
>> @srcmap agument:
>> Note that ->iomap_begin call has srcmap passed as another argument. This is
>> mainly used only during the begin phase for COW mappings to identify where
>> the reads are to be performed from. Filesystems needs to fill that mapping
>> information if iomap should read data for partially written blocks from a
>> different location than the write target [4].
>> @flags argument:
>> These are the operation types which iomap supports.
>> IOMAP_WRITE: For doing write I/O.
>> <...>
>> IOMAP_ZERO:
>> IOMAP_REPORT:
>> IOMAP_FAULT:
>> IOMAP_DIRECT:
>> IOMAP_NOWAIT:
>> IOMAP_OVERWRITE_ONLY:
>> IOMAP_UNSHARE:
>> IOMAP_DAX:
>
> I think it's /much/ more valuable to document the exact combinations
> that will be passed to ->iomap_begin further down where we talk about
> specific operations that iomap performs.
>
> Otherwise, someone is going to look at this list and wonder if they
> really need to figure out what IOMAP_ZERO|IOMAP_FAULT|IOMAP_DAX means,
> and if it's actually possible (it's not).
>
Sure.
>>
>> ->iomap_end
>> ^^^^^^^^^^^^^^^^^
>>
>> Commit and/or unreserve space which was previously allocated/reserved
>> in ``->iomap_begin``. For e.g. during buffered-io, when a short writes
>> occurs, filesystem may need to remove the reserved space that was
>> allocated during ->iomap_begin.
>> For filesystems that use delalloc allocation, we may need to punch out
>> delalloc extents from the range that are not dirty in
>> the page cache. See comments in
>> iomap_file_buffered_write_punch_delalloc() for more info [5][6].
>>
>> (IMHO) I find above definitions more descriptive.
>
> I don't want to merge the general description with pagecache specific
> areas. They already cover punch_delalloc.
>
sure.
>> > +
>> > +Each iomap operation describes the requested operation through the
>> > +``flags`` argument.
>> > +The exact value of ``flags`` will be documented in the
>> > +operation-specific sections below, but these principles apply generally:
>> > +
>> > + * For a write operation, ``IOMAP_WRITE`` will be set.
>> > + Filesystems must not return ``IOMAP_HOLE`` mappings.
>> > +
>> > + * For any other operation, ``IOMAP_WRITE`` will not be set.
>> > +
>>
>> Direct-io related operation which bypasses pagecache use IOMAP_DIRECT.
>
> That's covered in the pagecache/directio/dax subsection because I wanted
> to document specific combinations that filesystem authors should expect.
>
The points mentioned above were targetting buffered-io, dax, so I
thought we could add direct-io related flag as well here.
>> > + * For any operation targetting direct access to storage (fsdax),
>> > + ``IOMAP_DAX`` will be set.
>> > +
>> > +If it is necessary to read existing file contents from a `different
>> > +<https://lore.kernel.org/all/[email protected]/>`_ device or
>> > +address range on a device, the filesystem should return that information via
>> > +``srcmap``.
>> > +Only pagecache and fsdax operations support reading from one mapping and
>> > +writing to another.
>> > +
>> > +After the operation completes, the ``->iomap_end`` function, if present,
>> > +is called to signal that iomap is finished with a mapping.
>> > +Typically, implementations will use this function to tear down any
>> > +context that were set up in ``->iomap_begin``.
>> > +For example, a write might wish to commit the reservations for the bytes
>> > +that were operated upon and unreserve any space that was not operated
>> > +upon.
>> > +``written`` might be zero if no bytes were touched.
>> > +``flags`` will contain the same value passed to ``->iomap_begin``.
>> > +iomap ops for reads are not likely to need to supply this function.
>> > +
>> > +Both functions should return a negative errno code on error, or zero.
>>
>> minor nit: ... or zero on success.
>
> done.
>
>> > +
>> > +struct iomap
>> > +------------
>> > +
>> > +The filesystem returns the mappings via the following structure.
>>
>> Filesystem returns the contiguous file mapping information of logical
>> file offset range to a physically mapped extent via the following
>> structure which iomap uses to perform various file and pagecache
>> related operations listed above.
>
> How about:
>
> "The filesystem communicates to iomap operations the mappings of byte
> ranges of a file to byte ranges of a storage device with the structure
> below."
>
Sounds good.
>> > +For documentation purposes, the structure has been reordered to group
>> > +fields that go together logically.
>> > +
>> > +.. code-block:: c
>> > +
>> > + struct iomap {
>> > + loff_t offset;
>> > + u64 length;
>> > +
>> > + u16 type;
>> > + u16 flags;
>> > +
>> > + u64 addr;
>> > + struct block_device *bdev;
>> > + struct dax_device *dax_dev;
>> > + void *inline_data;
>> > +
>> > + void *private;
>> > +
>> > + const struct iomap_folio_ops *folio_ops;
>> > +
>> > + u64 validity_cookie;
>> > + };
>> > +
>> > +The information is useful for translating file operations into action.
>> > +The actions taken are specific to the target of the operation, such as
>> > +disk cache, physical storage devices, or another part of the kernel.
>>
>> I think the wording "action" & trying to make it so generic w/o mapping
>> what "action" refers here for "disk cache", "physical storage device" or
>> "other parts of the kernel", gets a bit confusing.
>>
>> Do you think we should map those to some examples maybe?
>> BTW, with added definition of "struct iomap" which I mentioned above,
>> I am even fine if we want to drop this paragraph.
>
> Yeah, I'll delete the paragraph.
>
>> > +
>> > + * ``offset`` and ``length`` describe the range of file offsets, in
>> > + bytes, covered by this mapping.
>> > + These fields must always be set by the filesystem.
>> > +
>> > + * ``type`` describes the type of the space mapping:
>>
>> This field is set by the filesystem in ->iomap_begin call.
>>
>> > +
>> > + * **IOMAP_HOLE**: No storage has been allocated.
>> > + This type must never be returned in response to an IOMAP_WRITE
>> > + operation because writes must allocate and map space, and return
>> > + the mapping.
>> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
>> > + iomap does not support writing (whether via pagecache or direct
>> > + I/O) to a hole.
>> > +
>> > + * **IOMAP_DELALLOC**: A promise to allocate space at a later time
>> > + ("delayed allocation").
>> > + If the filesystem returns IOMAP_F_NEW here and the write fails, the
>> > + ``->iomap_end`` function must delete the reservation.
>> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
>> > +
>> > + * **IOMAP_MAPPED**: The file range maps to specific space on the
>> > + storage device.
>> > + The device is returned in ``bdev`` or ``dax_dev``.
>> > + The device address, in bytes, is returned via ``addr``.
>> > +
>> > + * **IOMAP_UNWRITTEN**: The file range maps to specific space on the
>> > + storage device, but the space has not yet been initialized.
>> > + The device is returned in ``bdev`` or ``dax_dev``.
>> > + The device address, in bytes, is returned via ``addr``.
>> > + Reads will return zeroes to userspace.
>>
>> Reads to this type of mapping will return zeroes to the caller.
>
> Reads from this type of mapping, but yes.
>
>> > + For a write or writeback operation, the ioend should update the
>> > + mapping to MAPPED.
>>
>> Refer to section "Writeback ioend Completion" for more details.
>
> There are two here -- one for the pagecache, and one for directio.
>
>> > +
>> > + * **IOMAP_INLINE**: The file range maps to the memory buffer
>> > + specified by ``inline_data``.
>> > + For write operation, the ``->iomap_end`` function presumably
>> > + handles persisting the data.
>>
>> Is it? Or do we just mark the inode as dirty?
>
> gfs2 actually starts a transaction in ->iomap_begin and commits or
> cancels it in ->iomap_end.
>
ok.
>> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
>> > +
>> > + * ``flags`` describe the status of the space mapping.
>> > + These flags should be set by the filesystem in ``->iomap_begin``:
>> > +
>> > + * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
>> > + Areas that will not be written to must be zeroed.
>>
>> In case of DAX, we have to invalidate those existing mappings which
>> might have a "hole" page mapped.
>
> Isn't that an internal detail of the fs/dax.c code? The filesystem
> doesn't have to do the invalidation or even know about hole pages.
>
Right. Sorry about that. I assumed dax_iomap_rw() implementation
is a part of iomap :)
>> > + If a write fails and the mapping is a space reservation, the
>> > + reservation must be deleted.
>> > +
>> > + * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
>> > + to access any data written.
>> > + fdatasync is required to commit these changes to persistent
>> > + storage.
>> > + This needs to take into account metadata changes that *may* be made
>> > + at I/O completion, such as file size updates from direct I/O.
>> > +
>> > + * **IOMAP_F_SHARED**: The space under the mapping is shared.
>> > + Copy on write is necessary to avoid corrupting other file data.
>> > +
>> > + * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
>> > + heads for pagecache operations.
>> > + Do not add more uses of this.
>> > +
>> > + * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
>> > + coalesced into this single mapping.
>> > + This is only useful for FIEMAP.
>> > +
>> > + * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
>> > + regular file data.
>> > + This is only useful for FIEMAP.
>> > +
>> > + * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
>> > + be set by the filesystem for its own purposes.
>> > +
>> > + These flags can be set by iomap itself during file operations.
>> > + The filesystem should supply an ``->iomap_end`` function to observe
>> > + these flags:
>> > +
>> > + * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
>> > + using this mapping.
>> > +
>> > + * **IOMAP_F_STALE**: The mapping was found to be stale.
>> > + iomap will call ``->iomap_end`` on this mapping and then
>> > + ``->iomap_begin`` to obtain a new mapping.
>> > +
>> > + Currently, these flags are only set by pagecache operations.
>> > +
>> > + * ``addr`` describes the device address, in bytes.
>> > +
>> > + * ``bdev`` describes the block device for this mapping.
>> > + This only needs to be set for mapped or unwritten operations.
>> > +
>> > + * ``dax_dev`` describes the DAX device for this mapping.
>> > + This only needs to be set for mapped or unwritten operations, and
>> > + only for a fsdax operation.
>>
>> Looks like we can make this union (bdev and dax_dev). Since depending
>> upon IOMAP_DAX or not we only set either dax_dev or bdev.
>
> Separate patch. ;)
>
Yes, in a way I was trying to get an opinion from you and others on
whether it make sense to make bdev and dax_dev as union :)
Looks like this series [1] could be the reason for that.
[1]: https://lore.kernel.org/all/[email protected]/#t
I also don't see any reference to dax code from fs/iomap/buffered-io.c
So maybe we don't need this dax.h header in this file.
diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index c5802a459334..e1a6cca3cec2 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -10,7 +10,6 @@
#include <linux/pagemap.h>
#include <linux/uio.h>
#include <linux/buffer_head.h>
-#include <linux/dax.h>
#include <linux/writeback.h>
#include <linux/list_sort.h>
#include <linux/swap.h>
>> Sorry Darrick. I will stop here for now.
>> I will continue it from here later.
>
> Ok. The rest of the doc will hopefully make it more obvious why there's
> the generic discussion up here.
>
Sure. I am going through it.
-ritesh
Hi Darrick,
Resuming my review from where I left off yesterday.
"Darrick J. Wong" <[email protected]> writes:
> From: Darrick J. Wong <[email protected]>
>
> This is the fourth attempt at documenting the design of iomap and how to
> port filesystems to use it. Apologies for all the rst formatting, but
> it's necessary to distinguish code from regular text.
>
> A lot of this has been collected from various email conversations, code
> comments, commit messages, my own understanding of iomap, and
> Ritesh/Luis' previous efforts to create a document. Please note a large
> part of this has been taken from Dave's reply to last iomap doc
> patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
> other iomap developers who have taken time to explain the iomap design
> in various emails, commits, comments etc.
>
> Cc: Dave Chinner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Christoph Hellwig <[email protected]>
> Cc: Christian Brauner <[email protected]>
> Cc: Ojaswin Mujoo <[email protected]>
> Cc: Jan Kara <[email protected]>
> Cc: Luis Chamberlain <[email protected]>
> Inspired-by: Ritesh Harjani (IBM) <[email protected]>
> Signed-off-by: Darrick J. Wong <[email protected]>
> ---
> Documentation/filesystems/index.rst | 1
> Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
> MAINTAINERS | 1
> 3 files changed, 1062 insertions(+)
> create mode 100644 Documentation/filesystems/iomap.rst
>
> diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
> index 8f5c1ee02e2f..b010cc8df32d 100644
> --- a/Documentation/filesystems/index.rst
> +++ b/Documentation/filesystems/index.rst
> @@ -34,6 +34,7 @@ algorithms work.
> seq_file
> sharedsubtree
> idmappings
> + iomap
>
> automount-support
>
> diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
> new file mode 100644
> index 000000000000..a478b55e4135
> --- /dev/null
> +++ b/Documentation/filesystems/iomap.rst
> @@ -0,0 +1,1060 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +.. _iomap:
> +
> +..
> + Dumb style notes to maintain the author's sanity:
> + Please try to start sentences on separate lines so that
> + sentence changes don't bleed colors in diff.
> + Heading decorations are documented in sphinx.rst.
> +
> +============================
> +VFS iomap Design and Porting
> +============================
> +
> +.. toctree::
> +
> +Introduction
> +============
> +
> +iomap is a filesystem library for handling various filesystem operations
> +that involves mapping of file's logical offset ranges to physical
> +extents.
> +This origins of this library is the file I/O path that XFS once used; it
> +has now been extended to cover several other operations.
> +The library provides various APIs for implementing various file and
> +pagecache operations, such as:
> +
> + * Pagecache reads and writes
> + * Folio write faults to the pagecache
> + * Writeback of dirty folios
> + * Direct I/O reads and writes
> + * FIEMAP
> + * lseek ``SEEK_DATA`` and ``SEEK_HOLE``
> + * swapfile activation
> +
> +Who Should Read This?
> +=====================
> +
> +The target audience for this document are filesystem, storage, and
> +pagecache programmers and code reviewers.
> +The goal of this document is to provide a brief discussion of the
> +design and capabilities of iomap, followed by a more detailed catalog
> +of the interfaces presented by iomap.
> +If you change iomap, please update this design document.
> +
> +But Why?
> +========
> +
> +Unlike the classic Linux I/O model which breaks file I/O into small
> +units (generally memory pages or blocks) and looks up space mappings on
> +the basis of that unit, the iomap model asks the filesystem for the
> +largest space mappings that it can create for a given file operation and
> +initiates operations on that basis.
> +This strategy improves the filesystem's visibility into the size of the
> +operation being performed, which enables it to combat fragmentation with
> +larger space allocations when possible.
> +Larger space mappings improve runtime performance by amortizing the cost
> +of a mapping function call into the filesystem across a larger amount of
> +data.
> +
> +At a high level, an iomap operation `looks like this
> +<https://lore.kernel.org/all/[email protected]/>`_:
> +
> +1. For each byte in the operation range...
> +
> + 1. Obtain space mapping via ->iomap_begin
> + 2. For each sub-unit of work...
> +
> + 1. Revalidate the mapping and go back to (1) above, if necessary
> + 2. Do the work
> +
> + 3. Increment operation cursor
> + 4. Release the mapping via ->iomap_end, if necessary
> +
> +Each iomap operation will be covered in more detail below.
> +This library was covered previously by an `LWN article
> +<https://lwn.net/Articles/935934/>`_ and a `KernelNewbies page
> +<https://kernelnewbies.org/KernelProjects/iomap>`_.
> +
> +Data Structures and Algorithms
> +==============================
> +
> +Definitions
> +-----------
> +
> + * ``bufferhead``: Shattered remnants of the old buffer cache.
> + * ``fsblock``: The block size of a file, also known as ``i_blocksize``.
> + * ``i_rwsem``: The VFS ``struct inode`` rwsemaphore.
> + * ``invalidate_lock``: The pagecache ``struct address_space``
> + rwsemaphore that protects against folio removal.
> +
> +struct iomap_ops
> +----------------
> +
> +Every iomap function requires the filesystem to pass an operations
> +structure to obtain a mapping and (optionally) to release the mapping.
> +
> +.. code-block:: c
> +
> + struct iomap_ops {
> + int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length,
> + unsigned flags, struct iomap *iomap,
> + struct iomap *srcmap);
> +
> + int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length,
> + ssize_t written, unsigned flags,
> + struct iomap *iomap);
> + };
> +
> +The ``->iomap_begin`` function is called to obtain one mapping for the
> +range of bytes specified by ``pos`` and ``length`` for the file
> +``inode``.
> +
> +Each iomap operation describes the requested operation through the
> +``flags`` argument.
> +The exact value of ``flags`` will be documented in the
> +operation-specific sections below, but these principles apply generally:
> +
> + * For a write operation, ``IOMAP_WRITE`` will be set.
> + Filesystems must not return ``IOMAP_HOLE`` mappings.
> +
> + * For any other operation, ``IOMAP_WRITE`` will not be set.
> +
> + * For any operation targetting direct access to storage (fsdax),
> + ``IOMAP_DAX`` will be set.
> +
> +If it is necessary to read existing file contents from a `different
> +<https://lore.kernel.org/all/[email protected]/>`_ device or
> +address range on a device, the filesystem should return that information via
> +``srcmap``.
> +Only pagecache and fsdax operations support reading from one mapping and
> +writing to another.
> +
> +After the operation completes, the ``->iomap_end`` function, if present,
> +is called to signal that iomap is finished with a mapping.
> +Typically, implementations will use this function to tear down any
> +context that were set up in ``->iomap_begin``.
> +For example, a write might wish to commit the reservations for the bytes
> +that were operated upon and unreserve any space that was not operated
> +upon.
> +``written`` might be zero if no bytes were touched.
> +``flags`` will contain the same value passed to ``->iomap_begin``.
> +iomap ops for reads are not likely to need to supply this function.
> +
> +Both functions should return a negative errno code on error, or zero.
> +
> +struct iomap
> +------------
> +
> +The filesystem returns the mappings via the following structure.
> +For documentation purposes, the structure has been reordered to group
> +fields that go together logically.
> +
> +.. code-block:: c
> +
> + struct iomap {
> + loff_t offset;
> + u64 length;
> +
> + u16 type;
> + u16 flags;
> +
> + u64 addr;
> + struct block_device *bdev;
> + struct dax_device *dax_dev;
> + void *inline_data;
> +
> + void *private;
> +
> + const struct iomap_folio_ops *folio_ops;
> +
> + u64 validity_cookie;
> + };
> +
> +The information is useful for translating file operations into action.
> +The actions taken are specific to the target of the operation, such as
> +disk cache, physical storage devices, or another part of the kernel.
> +
> + * ``offset`` and ``length`` describe the range of file offsets, in
> + bytes, covered by this mapping.
> + These fields must always be set by the filesystem.
> +
> + * ``type`` describes the type of the space mapping:
> +
> + * **IOMAP_HOLE**: No storage has been allocated.
> + This type must never be returned in response to an IOMAP_WRITE
> + operation because writes must allocate and map space, and return
> + the mapping.
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> + iomap does not support writing (whether via pagecache or direct
> + I/O) to a hole.
> +
> + * **IOMAP_DELALLOC**: A promise to allocate space at a later time
> + ("delayed allocation").
> + If the filesystem returns IOMAP_F_NEW here and the write fails, the
> + ``->iomap_end`` function must delete the reservation.
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> +
> + * **IOMAP_MAPPED**: The file range maps to specific space on the
> + storage device.
> + The device is returned in ``bdev`` or ``dax_dev``.
> + The device address, in bytes, is returned via ``addr``.
> +
> + * **IOMAP_UNWRITTEN**: The file range maps to specific space on the
> + storage device, but the space has not yet been initialized.
> + The device is returned in ``bdev`` or ``dax_dev``.
> + The device address, in bytes, is returned via ``addr``.
> + Reads will return zeroes to userspace.
> + For a write or writeback operation, the ioend should update the
> + mapping to MAPPED.
> +
> + * **IOMAP_INLINE**: The file range maps to the memory buffer
> + specified by ``inline_data``.
> + For write operation, the ``->iomap_end`` function presumably
> + handles persisting the data.
> + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> +
> + * ``flags`` describe the status of the space mapping.
> + These flags should be set by the filesystem in ``->iomap_begin``:
> +
> + * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
> + Areas that will not be written to must be zeroed.
> + If a write fails and the mapping is a space reservation, the
> + reservation must be deleted.
> +
> + * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
> + to access any data written.
> + fdatasync is required to commit these changes to persistent
> + storage.
> + This needs to take into account metadata changes that *may* be made
> + at I/O completion, such as file size updates from direct I/O.
> +
> + * **IOMAP_F_SHARED**: The space under the mapping is shared.
> + Copy on write is necessary to avoid corrupting other file data.
> +
> + * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
> + heads for pagecache operations.
> + Do not add more uses of this.
> +
> + * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
> + coalesced into this single mapping.
> + This is only useful for FIEMAP.
> +
> + * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
> + regular file data.
> + This is only useful for FIEMAP.
> +
> + * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
> + be set by the filesystem for its own purposes.
> +
> + These flags can be set by iomap itself during file operations.
> + The filesystem should supply an ``->iomap_end`` function to observe
> + these flags:
> +
> + * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
> + using this mapping.
> +
> + * **IOMAP_F_STALE**: The mapping was found to be stale.
> + iomap will call ``->iomap_end`` on this mapping and then
> + ``->iomap_begin`` to obtain a new mapping.
> +
> + Currently, these flags are only set by pagecache operations.
> +
> + * ``addr`` describes the device address, in bytes.
> +
> + * ``bdev`` describes the block device for this mapping.
> + This only needs to be set for mapped or unwritten operations.
> +
> + * ``dax_dev`` describes the DAX device for this mapping.
> + This only needs to be set for mapped or unwritten operations, and
> + only for a fsdax operation.
> +
> + * ``inline_data`` points to a memory buffer for I/O involving
> + ``IOMAP_INLINE`` mappings.
> + This value is ignored for all other mapping types.
> +
> + * ``private`` is a pointer to `filesystem-private information
> + <https://lore.kernel.org/all/[email protected]/>`_.
> + This value will be passed unchanged to ``->iomap_end``.
> +
> + * ``folio_ops`` will be covered in the section on pagecache operations.
> +
> + * ``validity_cookie`` is a magic freshness value set by the filesystem
> + that should be used to detect stale mappings.
> + For pagecache operations this is critical for correct operation
> + because page faults can occur, which implies that filesystem locks
> + should not be held between ``->iomap_begin`` and ``->iomap_end``.
> + Filesystems with completely static mappings need not set this value.
> + Only pagecache operations revalidate mappings.
> +
> + XXX: Should fsdax revalidate as well?
> +
> +Validation
> +==========
> +
> +**NOTE**: iomap only handles mapping and I/O.
> +Filesystems must still call out to the VFS to check input parameters
> +and file state before initiating an I/O operation.
> +It does not handle updating of timestamps, stripping privileges, or
> +access control.
> +
> +Locking Hierarchy
> +=================
> +
> +iomap requires that filesystems provide their own locking.
> +There are no locks within iomap itself, though in the course of an
That might not be totally true. There is a state_lock within iomap_folio_state ;)
> +operation iomap may take other locks (e.g. folio/dax locks) as part of
> +an I/O operation.
I think we need not mention "dax locks" here right? Since most of that
code is in fs/dax.c anyways?
> +Locking with iomap can be split into two categories: above and below
> +iomap.
> +
> +The upper level of lock must coordinate the iomap operation with other
> +iomap operations.
Can we add some more details in this line or maybe an example?
Otherwise confusing use of "iomap operation" term.
> +Generally, the filesystem must take VFS/pagecache locks such as
> +``i_rwsem`` or ``invalidate_lock`` before calling into iomap.
> +The exact locking requirements are specific to the type of operation.
> +
> +The lower level of lock must coordinate access to the mapping
> +information.
> +This lock is filesystem specific and should be held during
> +``->iomap_begin`` while sampling the mapping and validity cookie.
> +
> +The general locking hierarchy in iomap is:
> +
> + * VFS or pagecache lock
> +
There is also a folio lock within iomap which now comes below VFS or
pagecache lock.
> + * Internal filesystem specific mapping lock
I think it will also be helpful if we give an example of this lock for
e.g. XFS(XFS_ILOCK) or ext4(i_data_sem)
> +
> + * iomap operation-specific lock
some e.g. of what you mean here please?
> +
> +The exact locking requirements are specific to the filesystem; for
> +certain operations, some of these locks can be elided.
> +All further mention of locking are *recommendations*, not mandates.
> +Each filesystem author must figure out the locking for themself.
Is it also possible to explicitly list down the fact that folio_lock
order w.r.t VFS lock (i_rwsem) (is it even with pagecache lock??) is now
reversed with iomap v/s the legacy I/O model.
There was an internal ext4 issue which got exposed due to this [1].
So it might be useful to document the lock order change now.
[1]: https://lore.kernel.org/linux-ext4/[email protected]/
> +
> +iomap Operations
> +================
> +
> +Below are a discussion of the file operations that iomap implements.
> +
> +Buffered I/O
> +------------
> +
> +Buffered I/O is the default file I/O path in Linux.
> +File contents are cached in memory ("pagecache") to satisfy reads and
> +writes.
> +Dirty cache will be written back to disk at some point that can be
> +forced via ``fsync`` and variants.
> +
> +iomap implements nearly all the folio and pagecache management that
> +filesystems once had to implement themselves.
nit: that "earlier in the legacy I/O model filesystems had to implement
themselves"
> +This means that the filesystem need not know the details of allocating,
> +mapping, managing uptodate and dirty state, or writeback of pagecache
> +folios.
> +Unless the filesystem explicitly opts in to buffer heads, they will not
> +be used, which makes buffered I/O much more efficient, and ``willy``
Could also please list down why buffered I/O is more efficient with
iomap (other than the fact that iomap has large folios)?
If I am not wrong, it comes from the fact that iomap only maintains
(other than sizeof iomap_folio_state once) 2 extra bytes per fsblock v/s
the 104 extra bytes of struct buffer_head per fsblock in the legacy I/O model.
And while iterating over the pagecache pages, it is much faster to
set/clear the uptodate/dirty bits of a folio in iomap v/s iterating over
each bufferhead within a folio in legacy I/O model.
Right?
> +much happier.
> +
> +struct address_space_operations
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +The following iomap functions can be referenced directly from the
> +address space operations structure:
> +
> + * ``iomap_dirty_folio``
> + * ``iomap_release_folio``
> + * ``iomap_invalidate_folio``
> + * ``iomap_is_partially_uptodate``
> +
> +The following address space operations can be wrapped easily:
> +
> + * ``read_folio``
> + * ``readahead``
> + * ``writepages``
> + * ``bmap``
> + * ``swap_activate``
> +
> +struct iomap_folio_ops
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +The ``->iomap_begin`` function for pagecache operations may set the
> +``struct iomap::folio_ops`` field to an ops structure to override
> +default behaviors of iomap:
> +
> +.. code-block:: c
> +
> + struct iomap_folio_ops {
> + struct folio *(*get_folio)(struct iomap_iter *iter, loff_t pos,
> + unsigned len);
> + void (*put_folio)(struct inode *inode, lofs, unsigned copied,
> + struct folio *folio);
> + bool (*iomap_valid)(struct inode *inode, const struct iomap *iomap);
> + };
> +
> +iomap calls these functions:
> +
> + - ``get_folio``: Called to allocate and return an active reference to
> + a locked folio prior to starting a write.
> + If this function is not provided, iomap will call
> + ``iomap_get_folio``.
> + This could be used to `set up per-folio filesystem state
> + <https://lore.kernel.org/all/[email protected]/>`_
> + for a write.
> +
> + - ``put_folio``: Called to unlock and put a folio after a pagecache
> + operation completes.
> + If this function is not provided, iomap will ``folio_unlock`` and
> + ``folio_put`` on its own.
> + This could be used to `commit per-folio filesystem state
> + <https://lore.kernel.org/all/[email protected]/>`_
> + that was set up by ``->get_folio``.
> +
> + - ``iomap_valid``: The filesystem may not hold locks between
> + ``->iomap_begin`` and ``->iomap_end`` because pagecache operations
> + can take folio locks, fault on userspace pages, initiate writeback
> + for memory reclamation, or engage in other time-consuming actions.
> + If a file's space mapping data are mutable, it is possible that the
> + mapping for a particular pagecache folio can `change in the time it
> + takes
> + <https://lore.kernel.org/all/[email protected]/>`_
> + to allocate, install, and lock that folio.
> + For such files, the mapping *must* be revalidated after the folio
> + lock has been taken so that iomap can manage the folio correctly.
> + The filesystem's ``->iomap_begin`` function must sample a sequence
> + counter into ``struct iomap::validity_cookie`` at the same time that
> + it populates the mapping fields.
> + It must then provide a ``->iomap_valid`` function to compare the
> + validity cookie against the source counter and return whether or not
> + the mapping is still valid.
> + If the mapping is not valid, the mapping will be sampled again.
> +
> +These ``struct kiocb`` flags are significant for buffered I/O with
> +iomap:
> +
> + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> + already in memory, we do not have to initiate other I/O, and we
> + acquire all filesystem locks without blocking.
> + Neither this flag nor its definition ``RWF_NOWAIT`` actually define
> + what this flag means, so this is the best the author could come up
> + with.
> +
> +Internal per-Folio State
> +~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +If the fsblock size matches the size of a pagecache folio, it is assumed
> +that all disk I/O operations will operate on the entire folio.
> +The uptodate (memory contents are at least as new as what's on disk) and
> +dirty (memory contents are newer than what's on disk) status of the
> +folio are all that's needed for this case.
> +
> +If the fsblock size is less than the size of a pagecache folio, iomap
> +tracks the per-fsblock uptodate and dirty state itself.
> +This enables iomap to handle both "bs < ps" `filesystems
> +<https://lore.kernel.org/all/[email protected]/>`_
> +and large folios in the pagecache.
> +
> +iomap internally tracks two state bits per fsblock:
> +
> + * ``uptodate``: iomap will try to keep folios fully up to date.
> + If there are read(ahead) errors, those fsblocks will not be marked
> + uptodate.
> + The folio itself will be marked uptodate when all fsblocks within the
> + folio are uptodate.
> +
> + * ``dirty``: iomap will set the per-block dirty state when programs
> + write to the file.
> + The folio itself will be marked dirty when any fsblock within the
> + folio is dirty.
> +
> +iomap also tracks the amount of read and write disk IOs that are in
> +flight.
> +This structure is much lighter weight than ``struct buffer_head``.
> +
> +Filesystems wishing to turn on large folios in the pagecache should call
> +``mapping_set_large_folios`` when initializing the incore inode.
> +
> +Readahead and Reads
> +~~~~~~~~~~~~~~~~~~~
> +
> +The ``iomap_readahead`` function initiates readahead to the pagecache.
> +The ``iomap_read_folio`` function reads one folio's worth of data into
> +the pagecache.
> +The ``flags`` argument to ``->iomap_begin`` will be set to zero.
> +The pagecache takes whatever locks it needs before calling the
> +filesystem.
> +
> +Writes
> +~~~~~~
> +
> +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
> +pagecache.
> +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
> +the ``flags`` argument to ``->iomap_begin``.
> +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
shared(e.g. aligned overwrites)
> +
> +mmap Write Faults
> +^^^^^^^^^^^^^^^^^
> +
> +The ``iomap_page_mkwrite`` function handles a write fault to a folio the
> +pagecache.
"handles a write fault to the pagecache" ?
> +``IOMAP_WRITE | IOMAP_FAULT`` will be passed as the ``flags`` argument
> +to ``->iomap_begin``.
> +Callers commonly take the mmap ``invalidate_lock`` in shared or
> +exclusive mode.
> +
> +Write Failures
> +^^^^^^^^^^^^^^
> +
> +After a short write to the pagecache, the areas not written will not
> +become marked dirty.
> +The filesystem must arrange to `cancel
> +<https://lore.kernel.org/all/[email protected]/>`_
> +such `reservations
> +<https://lore.kernel.org/linux-xfs/[email protected]/>`_
> +because writeback will not consume the reservation.
> +The ``iomap_file_buffered_write_punch_delalloc`` can be called from a
> +``->iomap_end`` function to find all the clean areas of the folios
> +caching a fresh (``IOMAP_F_NEW``) delalloc mapping.
> +It takes the ``invalidate_lock``.
> +
> +The filesystem should supply a callback ``punch`` will be called for
The filesystem supplied ``punch`` callback will be called for...
> +each file range in this state.
> +This function must *only* remove delayed allocation reservations, in
> +case another thread racing with the current thread writes successfully
> +to the same region and triggers writeback to flush the dirty data out to
> +disk.
> +
> +Truncation
> +^^^^^^^^^^
> +
> +Filesystems can call ``iomap_truncate_page`` to zero the bytes in the
> +pagecache from EOF to the end of the fsblock during a file truncation
> +operation.
> +``truncate_setsize`` or ``truncate_pagecache`` will take care of
> +everything after the EOF block.
> +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Zeroing for File Operations
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Filesystems can call ``iomap_zero_range`` to perform zeroing of the
> +pagecache for non-truncation file operations that are not aligned to
> +the fsblock size.
> +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Unsharing Reflinked File Data
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Filesystems can call ``iomap_file_unshare`` to force a file sharing
> +storage with another file to preemptively copy the shared data to newly
> +allocate storage.
> +``IOMAP_WRITE | IOMAP_UNSHARE`` will be passed as the ``flags`` argument
> +to ``->iomap_begin``.
> +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> +mode.
> +
> +Writeback
> +~~~~~~~~~
> +
> +Filesystems can call ``iomap_writepages`` to respond to a request to
> +write dirty pagecache folios to disk.
> +The ``mapping`` and ``wbc`` parameters should be passed unchanged.
> +The ``wpc`` pointer should be allocated by the filesystem and must
> +be initialized to zero.
> +
> +The pagecache will lock each folio before trying to schedule it for
> +writeback.
> +It does not lock ``i_rwsem`` or ``invalidate_lock``.
> +
> +The dirty bit will be cleared for all folios run through the
> +``->map_blocks`` machinery described below even if the writeback fails.
> +This is to prevent dirty folio clots when storage devices fail; an
> +``-EIO`` is recorded for userspace to collect via ``fsync``.
> +
> +The ``ops`` structure must be specified and is as follows:
> +
> +struct iomap_writeback_ops
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +.. code-block:: c
> +
> + struct iomap_writeback_ops {
> + int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
> + loff_t offset, unsigned len);
> + int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
> + void (*discard_folio)(struct folio *folio, loff_t pos);
> + };
> +
> +The fields are as follows:
> +
> + - ``map_blocks``: Sets ``wpc->iomap`` to the space mapping of the file
> + range (in bytes) given by ``offset`` and ``len``.
> + iomap calls this function for each fs block in each dirty folio,
> + even if the mapping returned is longer than one fs block.
It's no longer true after this patch right [1]. iomap calls this
function for each contiguous range of dirty fsblocks within a dirty folio.
[1]: https://lore.kernel.org/all/[email protected]/
> + Do not return ``IOMAP_INLINE`` mappings here; the ``->iomap_end``
> + function must deal with persisting written data.
> + Filesystems can skip a potentially expensive mapping lookup if the
> + mappings have not changed.
> + This revalidation must be open-coded by the filesystem; it is
> + unclear if ``iomap::validity_cookie`` can be reused for this
> + purpose.
struct iomap_writepage_ctx defines it's own ``struct iomap`` as a member.
struct iomap_writepage_ctx {
struct iomap iomap;
struct iomap_ioend *ioend;
const struct iomap_writeback_ops *ops;
u32 nr_folios; /* folios added to the ioend */
};
That means it does not conflict with the context which is doing buffered
writes (i.e. write_iter) and writeback is anyway single threaded.
So we should be able to use wpc->iomap.validity_cookie for validating
whether the cookie is valid or not during the course of writeback
operation - (IMO)
> + This function is required.
This line is left incomplete.
I think we should also mention this right? -
If the filesystem reserved delalloc extents during buffered-writes, than
they should allocate extents for those delalloc mappings in this
->map_blocks call.
> +
> + - ``prepare_ioend``: Enables filesystems to transform the writeback
> + ioend or perform any other prepatory work before the writeback I/O
IMO, some e.g. will be very helpful to add wherever possible. I
understand we should keep the document generic enough, but it is much
easier if we state some common examples of what XFS / other filesystems
do with such callback methods.
e.g.
- What do we mean by "transform the writeback ioend"? I guess it is -
XFS uses this for conversion of COW extents to regular extents?
- What do we mean by "perform any other preparatory work before the
writeback I/O"? - I guess it is -
XFS hooks in custom a completion handler in ->prepare_ioend callback for
conversion of unwritten extents.
> + is submitted.
> + A filesystem can override the ``->bi_end_io`` function for its own
> + purposes, such as kicking the ioend completion to a workqueue if the
> + bio is completed in interrupt context.
Thanks this is also helpful.
> + This function is optional.
> +
> + - ``discard_folio``: iomap calls this function after ``->map_blocks``
> + fails schedule I/O for any part of a dirty folio.
fails "to" schedule
> + The function should throw away any reservations that may have been
> + made for the write.
> + The folio will be marked clean and an ``-EIO`` recorded in the
> + pagecache.
> + Filesystems can use this callback to `remove
> + <https://lore.kernel.org/all/[email protected]/>`_
> + delalloc reservations to avoid having delalloc reservations for
> + clean pagecache.
> + This function is optional.
> +
> +Writeback ioend Completion
> +^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +iomap creates chains of ``struct iomap_ioend`` objects that wrap the
> +``bio`` that is used to write pagecache data to disk.
> +By default, iomap finishes writeback ioends by clearing the writeback
> +bit on the folios attached to the ``ioend``.
> +If the write failed, it will also set the error bits on the folios and
> +the address space.
> +This can happen in interrupt or process context, depending on the
> +storage device.
> +
> +Filesystems that need to update internal bookkeeping (e.g. unwritten
> +extent conversions) should provide a ``->prepare_ioend`` function to
Ok, you did actually mention the unwritten conversion example here.
However no harm in also mentioning this in the section which gives info
about ->prepare_ioend callback :)
> +override the ``struct iomap_end::bio::bi_end_io`` with its own function.
> +This function should call ``iomap_finish_ioends`` after finishing its
> +own work.
> +
> +Some filesystems may wish to `amortize the cost of running metadata
> +transactions
> +<https://lore.kernel.org/all/[email protected]/>`_
> +for post-writeback updates by batching them.
> +They may also require transactions to run from process context, which
> +implies punting batches to a workqueue.
> +iomap ioends contain a ``list_head`` to enable batching.
> +
> +Given a batch of ioends, iomap has a few helpers to assist with
> +amortization:
> +
> + * ``iomap_sort_ioends``: Sort all the ioends in the list by file
> + offset.
> +
> + * ``iomap_ioend_try_merge``: Given an ioend that is not in any list and
> + a separate list of sorted ioends, merge as many of the ioends from
> + the head of the list into the given ioend.
> + ioends can only be merged if the file range and storage addresses are
> + contiguous; the unwritten and shared status are the same; and the
> + write I/O outcome is the same.
> + The merged ioends become their own list.
> +
> + * ``iomap_finish_ioends``: Finish an ioend that possibly has other
> + ioends linked to it.
> +
Again sorry for stopping here. I will continue the review from Direct-io later.
Thanks!
-ritesh
On Sun, Jun 09, 2024 at 08:55:06AM -0700, Darrick J. Wong wrote:
> HTML version here, text version below.
That is so much nicer than all the RST stuff..
> iomap is a filesystem library for handling various filesystem
> operations that involves mapping of file's logical offset ranges
> to physical extents. This origins of this library is the file I/O
> path that XFS once used; it has now been extended to cover several
> other operations. The library provides various APIs for
> implementing various file and pagecache operations, such as:
Does anyone care about the origin?
>
> * Pagecache reads and writes
>
> * Folio write faults to the pagecache
>
> * Writeback of dirty folios
>
> * Direct I/O reads and writes
>
> * FIEMAP
>
> * lseek SEEK_DATA and SEEK_HOLE
>
> * swapfile activation
One useful bit might be that there are two layer in iomap.
1) the very simply underlying layer in iter.c that just provides
a nicer iteration over logical file offsets
2) anything built on top. That's the things mentioned above plus
DAX.
What is also kinda interesting as it keeps confusing people is that
nothing in the iterator is block device specific. In fact the DAX
code now has no block device dependencies, as does the lseek and
FIEMAP code.
Because of that it might make sense to split this document up a bit
for the different layers and libraries. Or maybe not if too many
documents are too confusing.
> 2. For each sub-unit of work...
>
> 1. Revalidate the mapping and go back to (1) above, if
> necessary
That's something only really done in the buffered write path.
> Each iomap operation will be covered in more detail below. This
> library was covered previously by an LWN article and a
> KernelNewbies page.
Maybe these are links in other formats, but if not this information
isn't very useful. Depending on how old that information is it
probably isn't even with links.
> The filesystem returns the mappings via the following structure.
> For documentation purposes, the structure has been reordered to
> group fields that go together logically.
I don't think putting a different layout in here is a good idea.
In fact duplicating the definition means it will be out of sync
rather sooner than later. Given that we have to deal with RST anyway
we might as well want to pull this in as kerneldoc comments.
And maybe reorder the actual definition while we're at it,
as the version below still packs nicely.
> struct block_device *bdev;
> struct dax_device *dax_dev;
> void *inline_data;
Note: The could become a union these days. I tried years ago
before fully decoupling the DAX code and that didn't work,
but we should be fine now.
> * type describes the type of the space mapping:
>
> * IOMAP_HOLE: No storage has been allocated. This type
> must never be returned in response to an IOMAP_WRITE
> operation because writes must allocate and map space,
> and return the mapping. The addr field must be set to
> IOMAP_NULL_ADDR. iomap does not support writing
> (whether via pagecache or direct I/O) to a hole.
...
These should probably also be kerneldoc comments instead of being
away from the definitions?
>
> * IOMAP_F_XATTR: The mapping is for extended attribute
> data, not regular file data. This is only useful for
> FIEMAP.
.. and only used inside XFS. Maybe we should look into killing it.
> These struct kiocb flags are significant for buffered I/O with
> iomap:
>
> * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> already in memory, we do not have to initiate other I/O, and
> we acquire all filesystem locks without blocking. Neither
> this flag nor its definition RWF_NOWAIT actually define what
> this flag means, so this is the best the author could come
> up with.
I don't think that's true. But if it feels true to you submitting
a patch to describe it better is probably more helpful than this.
> iomap internally tracks two state bits per fsblock:
>
> * uptodate: iomap will try to keep folios fully up to date. If
> there are read(ahead) errors, those fsblocks will not be
> marked uptodate. The folio itself will be marked uptodate
> when all fsblocks within the folio are uptodate.
>
> * dirty: iomap will set the per-block dirty state when
> programs write to the file. The folio itself will be marked
> dirty when any fsblock within the folio is dirty.
>
> iomap also tracks the amount of read and write disk IOs that are
> in flight. This structure is much lighter weight than struct
> buffer_head.
Is this really something that should go into an API documentation?
Note that the structure not only is lighter weight than a buffer_head,
but more importantly there are a lot less of them as there is only
one per folio and not one per FSB.
> Why Convert to iomap?
Make this a separate document?
On Tue, Jun 11, 2024 at 09:12:13AM -0700, Christoph Hellwig wrote:
> On Sun, Jun 09, 2024 at 08:55:06AM -0700, Darrick J. Wong wrote:
> > HTML version here, text version below.
>
> That is so much nicer than all the RST stuff..
>
> > iomap is a filesystem library for handling various filesystem
> > operations that involves mapping of file's logical offset ranges
> > to physical extents. This origins of this library is the file I/O
> > path that XFS once used; it has now been extended to cover several
> > other operations. The library provides various APIs for
> > implementing various file and pagecache operations, such as:
>
> Does anyone care about the origin?
I do; occasionally people who are totally new to iomap wonder why
suchandsuch works in the odd way it does, and I can point them at its
XFS origins.
> >
> > * Pagecache reads and writes
> >
> > * Folio write faults to the pagecache
> >
> > * Writeback of dirty folios
> >
> > * Direct I/O reads and writes
> >
> > * FIEMAP
> >
> > * lseek SEEK_DATA and SEEK_HOLE
> >
> > * swapfile activation
>
> One useful bit might be that there are two layer in iomap.
>
> 1) the very simply underlying layer in iter.c that just provides
> a nicer iteration over logical file offsets
> 2) anything built on top. That's the things mentioned above plus
> DAX.
>
> What is also kinda interesting as it keeps confusing people is that
> nothing in the iterator is block device specific. In fact the DAX
> code now has no block device dependencies, as does the lseek and
> FIEMAP code.
<nod>
> Because of that it might make sense to split this document up a bit
> for the different layers and libraries. Or maybe not if too many
> documents are too confusing.
Hmm. The internal design is about ~400 lines of text. The actual
operations iomap implements are another ~600 LoT, and the porting
guidelines at the end are about 140 LoT. Maybe that's a reasonable
length for splitting?
> > 2. For each sub-unit of work...
> >
> > 1. Revalidate the mapping and go back to (1) above, if
> > necessary
>
> That's something only really done in the buffered write path.
Yeah.
> > Each iomap operation will be covered in more detail below. This
> > library was covered previously by an LWN article and a
> > KernelNewbies page.
>
> Maybe these are links in other formats, but if not this information
> isn't very useful. Depending on how old that information is it
> probably isn't even with links.
There are links, which are visible in the HTML version. Maybe I
should've run links in whichever mode spits out all the links as
footnotes. (rst2html -> links is how I made the text version in the
first place).
> > The filesystem returns the mappings via the following structure.
> > For documentation purposes, the structure has been reordered to
> > group fields that go together logically.
>
> I don't think putting a different layout in here is a good idea.
> In fact duplicating the definition means it will be out of sync
> rather sooner than later. Given that we have to deal with RST anyway
> we might as well want to pull this in as kerneldoc comments.
> And maybe reorder the actual definition while we're at it,
> as the version below still packs nicely.
Ok, I'll copy struct iomap as is.
> > struct block_device *bdev;
> > struct dax_device *dax_dev;
> > void *inline_data;
>
> Note: The could become a union these days. I tried years ago
> before fully decoupling the DAX code and that didn't work,
> but we should be fine now.
You and Ritesh have both suggested that today.
> > * type describes the type of the space mapping:
> >
> > * IOMAP_HOLE: No storage has been allocated. This type
> > must never be returned in response to an IOMAP_WRITE
> > operation because writes must allocate and map space,
> > and return the mapping. The addr field must be set to
> > IOMAP_NULL_ADDR. iomap does not support writing
> > (whether via pagecache or direct I/O) to a hole.
>
> ...
>
> These should probably also be kerneldoc comments instead of being
> away from the definitions?
I don't like how kerneldoc makes it hard to associate iomap::type with
the IOMAP_* constants that go in it. This would probably be ok for
::type if we turned it into an actual enum, but as C doesn't actually
have a bitset type, the only way to tell the reader which flags go where
is either strong namespacing (we blew it on that) or writing linking
text into the kerneldoc.
With this format I can lay out the document with relevant topics
adjacent and indented, so the association is obvious. The oneline
comments in the header file can jog readers' memories, without us
needing to stuff a whole ton of documentation into a C header.
Besides, kerneldoc only tells the reader what the interfaces are, not
how all those pieces fit together.
> > * IOMAP_F_XATTR: The mapping is for extended attribute
> > data, not regular file data. This is only useful for
> > FIEMAP.
>
> .. and only used inside XFS. Maybe we should look into killing it.
Yeah.
> > These struct kiocb flags are significant for buffered I/O with
> > iomap:
> >
> > * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> > already in memory, we do not have to initiate other I/O, and
> > we acquire all filesystem locks without blocking. Neither
> > this flag nor its definition RWF_NOWAIT actually define what
> > this flag means, so this is the best the author could come
> > up with.
>
> I don't think that's true. But if it feels true to you submitting
> a patch to describe it better is probably more helpful than this.
I think Dave just told me off for this, so I'll probably replace the
whole section with what he and Jan wrote.
> > iomap internally tracks two state bits per fsblock:
> >
> > * uptodate: iomap will try to keep folios fully up to date. If
> > there are read(ahead) errors, those fsblocks will not be
> > marked uptodate. The folio itself will be marked uptodate
> > when all fsblocks within the folio are uptodate.
> >
> > * dirty: iomap will set the per-block dirty state when
> > programs write to the file. The folio itself will be marked
> > dirty when any fsblock within the folio is dirty.
> >
> > iomap also tracks the amount of read and write disk IOs that are
> > in flight. This structure is much lighter weight than struct
> > buffer_head.
>
> Is this really something that should go into an API documentation?
Strictly speaking, no. It should be in a separate internals document.
> Note that the structure not only is lighter weight than a buffer_head,
> but more importantly there are a lot less of them as there is only
> one per folio and not one per FSB.
>
> > Why Convert to iomap?
>
> Make this a separate document?
I was pondering splitting these into two pieces:
Documentation/iomap/{design,porting}.rst
Though the porting guide is 10% of the document. Maybe that's worth it.
--D
On Tue, Jun 11, 2024 at 12:13:22PM +0530, Ritesh Harjani wrote:
> "Darrick J. Wong" <[email protected]> writes:
>
> > On Mon, Jun 10, 2024 at 02:27:28PM +0530, Ritesh Harjani wrote:
> >>
> >> Hello Darrick,
> >>
> >> "Darrick J. Wong" <[email protected]> writes:
> >>
> >> > From: Darrick J. Wong <[email protected]>
> >> >
> >> > This is the fourth attempt at documenting the design of iomap and how to
> >>
> >> I agree that this isn't needed in the commit msg ("fourth attempt").
> >
> > Ok. "Coapture the design of iomap and how to port..."
> >
> >> > port filesystems to use it. Apologies for all the rst formatting, but
> >> > it's necessary to distinguish code from regular text.
> >> >
> >> > A lot of this has been collected from various email conversations, code
> >> > comments, commit messages, my own understanding of iomap, and
> >> > Ritesh/Luis' previous efforts to create a document. Please note a large
> >> > part of this has been taken from Dave's reply to last iomap doc
> >> > patchset. Thanks to Ritesh, Luis, Dave, Darrick, Matthew, Christoph and
> >> > other iomap developers who have taken time to explain the iomap design
> >> > in various emails, commits, comments etc.
> >> >
> >> > Cc: Dave Chinner <[email protected]>
> >> > Cc: Matthew Wilcox <[email protected]>
> >> > Cc: Christoph Hellwig <[email protected]>
> >> > Cc: Christian Brauner <[email protected]>
> >> > Cc: Ojaswin Mujoo <[email protected]>
> >> > Cc: Jan Kara <[email protected]>
> >> > Cc: Luis Chamberlain <[email protected]>
> >> > Inspired-by: Ritesh Harjani (IBM) <[email protected]>
> >>
> >> I am not sure if this is even a valid or accepted tag.
> >> But sure thanks! :)
> >
> > They're freeform tags, so they can be everything everyone wants them to
> > be! Drum circle kumbaya etc. :P
> >
> >> > Signed-off-by: Darrick J. Wong <[email protected]>
> >> > ---
> >> > Documentation/filesystems/index.rst | 1
> >> > Documentation/filesystems/iomap.rst | 1060 +++++++++++++++++++++++++++++++++++
> >> > MAINTAINERS | 1
> >> > 3 files changed, 1062 insertions(+)
> >> > create mode 100644 Documentation/filesystems/iomap.rst
> >> >
> >> > diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
> >> > index 8f5c1ee02e2f..b010cc8df32d 100644
> >> > --- a/Documentation/filesystems/index.rst
> >> > +++ b/Documentation/filesystems/index.rst
> >> > @@ -34,6 +34,7 @@ algorithms work.
> >> > seq_file
> >> > sharedsubtree
> >> > idmappings
> >> > + iomap
> >> >
> >> > automount-support
> >> >
> >> > diff --git a/Documentation/filesystems/iomap.rst b/Documentation/filesystems/iomap.rst
> >> > new file mode 100644
> >> > index 000000000000..a478b55e4135
> >> > --- /dev/null
> >> > +++ b/Documentation/filesystems/iomap.rst
> >> > @@ -0,0 +1,1060 @@
> >> > +.. SPDX-License-Identifier: GPL-2.0
> >> > +.. _iomap:
> >> > +
> >> > +..
> >> > + Dumb style notes to maintain the author's sanity:
> >> > + Please try to start sentences on separate lines so that
> >> > + sentence changes don't bleed colors in diff.
> >> > + Heading decorations are documented in sphinx.rst.
> >> > +
> >> > +============================
> >> > +VFS iomap Design and Porting
> >> > +============================
> >> > +
> >> > +.. toctree::
> >> > +
> >> > +Introduction
> >> > +============
> >> > +
> >> > +iomap is a filesystem library for handling various filesystem operations
> >> > +that involves mapping of file's logical offset ranges to physical
> >> > +extents.
> >> > +This origins of this library is the file I/O path that XFS once used; it
> >> > +has now been extended to cover several other operations.
> >> > +The library provides various APIs for implementing various file and
> >> ^^^^ redundant "various"
> >>
> >> > +pagecache operations, such as:
> >> > +
> >> > + * Pagecache reads and writes
> >> > + * Folio write faults to the pagecache
> >> > + * Writeback of dirty folios
> >> > + * Direct I/O reads and writes
> >>
> >> Dax I/O reads and writes.
> >> ... as well please?
> >
> > It's really fsdax I/O reads, writes, loads, and stores, isn't it?
> >
>
> It felt like dax_iomap_rw() belongs to fs/iomap.
> But nevertheless, we could skip it if we are targetting fs/iomap/
> lib.
Logically, it does. However, there's a fair bit of code in fs/dax.c
that is used by the iomap iterators; all of those would have to become
non-static symbols to make that separation happen.
Maybe it still should; the pagecache is already like that.
<snip>
> >> I think it is better if we describe ->iomap_begin and ->iomap_end
> >> in proper sub-sections. Otherwise this looks like we have clobbered
> >> all the information together :)
> >>
> >> ->iomap_begin
> >> ^^^^^^^^^^^^^^^^^
> >
> > Yes, I like the explicit section headings better.
> >
>
> yup.
>
> >> This either returns an existing mapping or reserve/allocates a new
> >> mapping.
> >
> > That's a filesystem specific detail -- all that iomap cares about is
> > that the fs communicates a mapping. Maybe the fs actually had to do a
> > bunch of work to get that mapping, or maybe it's already laid out
> > statically, ala zonefs. Either way, it's not a concern of the iomap
> > library.
> >
> >> logical file pos and length are in bytes which gets passed
> >> as function arguments. Filesystem returns the new mapping information
> >> within ``struct iomap`` which also gets passed as a function argument.
> >> Filesystems should provide the details of this mapping by filling
> >> various fields within ``struct iomap``.
> >
> > "iomap operations call ->iomap_begin to obtain one file mapping for the
> > range of bytes specified by pos and length for the file inode. This
> > mapping should be returned through the iomap pointer. The mapping must
> > cover at least the first byte of the supplied file range, but it does
> > not need to cover the entire requested range."
> >
>
> I like it. Thanks for adding that detail in the last line.
>
> >> @srcmap agument:
> >> Note that ->iomap_begin call has srcmap passed as another argument. This is
> >> mainly used only during the begin phase for COW mappings to identify where
> >> the reads are to be performed from. Filesystems needs to fill that mapping
> >> information if iomap should read data for partially written blocks from a
> >> different location than the write target [4].
> >> @flags argument:
> >> These are the operation types which iomap supports.
> >> IOMAP_WRITE: For doing write I/O.
> >> <...>
> >> IOMAP_ZERO:
> >> IOMAP_REPORT:
> >> IOMAP_FAULT:
> >> IOMAP_DIRECT:
> >> IOMAP_NOWAIT:
> >> IOMAP_OVERWRITE_ONLY:
> >> IOMAP_UNSHARE:
> >> IOMAP_DAX:
> >
> > I think it's /much/ more valuable to document the exact combinations
> > that will be passed to ->iomap_begin further down where we talk about
> > specific operations that iomap performs.
> >
> > Otherwise, someone is going to look at this list and wonder if they
> > really need to figure out what IOMAP_ZERO|IOMAP_FAULT|IOMAP_DAX means,
> > and if it's actually possible (it's not).
> >
>
> Sure.
>
> >>
> >> ->iomap_end
> >> ^^^^^^^^^^^^^^^^^
> >>
> >> Commit and/or unreserve space which was previously allocated/reserved
> >> in ``->iomap_begin``. For e.g. during buffered-io, when a short writes
> >> occurs, filesystem may need to remove the reserved space that was
> >> allocated during ->iomap_begin.
> >> For filesystems that use delalloc allocation, we may need to punch out
> >> delalloc extents from the range that are not dirty in
> >> the page cache. See comments in
> >> iomap_file_buffered_write_punch_delalloc() for more info [5][6].
> >>
> >> (IMHO) I find above definitions more descriptive.
> >
> > I don't want to merge the general description with pagecache specific
> > areas. They already cover punch_delalloc.
> >
>
> sure.
>
> >> > +
> >> > +Each iomap operation describes the requested operation through the
> >> > +``flags`` argument.
> >> > +The exact value of ``flags`` will be documented in the
> >> > +operation-specific sections below, but these principles apply generally:
> >> > +
> >> > + * For a write operation, ``IOMAP_WRITE`` will be set.
> >> > + Filesystems must not return ``IOMAP_HOLE`` mappings.
> >> > +
> >> > + * For any other operation, ``IOMAP_WRITE`` will not be set.
> >> > +
> >>
> >> Direct-io related operation which bypasses pagecache use IOMAP_DIRECT.
> >
> > That's covered in the pagecache/directio/dax subsection because I wanted
> > to document specific combinations that filesystem authors should expect.
> >
>
> The points mentioned above were targetting buffered-io, dax, so I
> thought we could add direct-io related flag as well here.
Given all the confusion with the later sections I'll just remove it.
<snip>
> >> > +
> >> > + * **IOMAP_INLINE**: The file range maps to the memory buffer
> >> > + specified by ``inline_data``.
> >> > + For write operation, the ``->iomap_end`` function presumably
> >> > + handles persisting the data.
> >>
> >> Is it? Or do we just mark the inode as dirty?
> >
> > gfs2 actually starts a transaction in ->iomap_begin and commits or
> > cancels it in ->iomap_end.
> >
>
> ok.
>
> >> > + The ``addr`` field must be set to ``IOMAP_NULL_ADDR``.
> >> > +
> >> > + * ``flags`` describe the status of the space mapping.
> >> > + These flags should be set by the filesystem in ``->iomap_begin``:
> >> > +
> >> > + * **IOMAP_F_NEW**: The space under the mapping is newly allocated.
> >> > + Areas that will not be written to must be zeroed.
> >>
> >> In case of DAX, we have to invalidate those existing mappings which
> >> might have a "hole" page mapped.
> >
> > Isn't that an internal detail of the fs/dax.c code? The filesystem
> > doesn't have to do the invalidation or even know about hole pages.
> >
>
> Right. Sorry about that. I assumed dax_iomap_rw() implementation
> is a part of iomap :)
>
> >> > + If a write fails and the mapping is a space reservation, the
> >> > + reservation must be deleted.
> >> > +
> >> > + * **IOMAP_F_DIRTY**: The inode will have uncommitted metadata needed
> >> > + to access any data written.
> >> > + fdatasync is required to commit these changes to persistent
> >> > + storage.
> >> > + This needs to take into account metadata changes that *may* be made
> >> > + at I/O completion, such as file size updates from direct I/O.
> >> > +
> >> > + * **IOMAP_F_SHARED**: The space under the mapping is shared.
> >> > + Copy on write is necessary to avoid corrupting other file data.
> >> > +
> >> > + * **IOMAP_F_BUFFER_HEAD**: This mapping requires the use of buffer
> >> > + heads for pagecache operations.
> >> > + Do not add more uses of this.
> >> > +
> >> > + * **IOMAP_F_MERGED**: Multiple contiguous block mappings were
> >> > + coalesced into this single mapping.
> >> > + This is only useful for FIEMAP.
> >> > +
> >> > + * **IOMAP_F_XATTR**: The mapping is for extended attribute data, not
> >> > + regular file data.
> >> > + This is only useful for FIEMAP.
> >> > +
> >> > + * **IOMAP_F_PRIVATE**: Starting with this value, the upper bits can
> >> > + be set by the filesystem for its own purposes.
> >> > +
> >> > + These flags can be set by iomap itself during file operations.
> >> > + The filesystem should supply an ``->iomap_end`` function to observe
> >> > + these flags:
> >> > +
> >> > + * **IOMAP_F_SIZE_CHANGED**: The file size has changed as a result of
> >> > + using this mapping.
> >> > +
> >> > + * **IOMAP_F_STALE**: The mapping was found to be stale.
> >> > + iomap will call ``->iomap_end`` on this mapping and then
> >> > + ``->iomap_begin`` to obtain a new mapping.
> >> > +
> >> > + Currently, these flags are only set by pagecache operations.
> >> > +
> >> > + * ``addr`` describes the device address, in bytes.
> >> > +
> >> > + * ``bdev`` describes the block device for this mapping.
> >> > + This only needs to be set for mapped or unwritten operations.
> >> > +
> >> > + * ``dax_dev`` describes the DAX device for this mapping.
> >> > + This only needs to be set for mapped or unwritten operations, and
> >> > + only for a fsdax operation.
> >>
> >> Looks like we can make this union (bdev and dax_dev). Since depending
> >> upon IOMAP_DAX or not we only set either dax_dev or bdev.
> >
> > Separate patch. ;)
> >
>
> Yes, in a way I was trying to get an opinion from you and others on
> whether it make sense to make bdev and dax_dev as union :)
>
> Looks like this series [1] could be the reason for that.
>
> [1]: https://lore.kernel.org/all/[email protected]/#t
>
> I also don't see any reference to dax code from fs/iomap/buffered-io.c
> So maybe we don't need this dax.h header in this file.
>
> diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
> index c5802a459334..e1a6cca3cec2 100644
> --- a/fs/iomap/buffered-io.c
> +++ b/fs/iomap/buffered-io.c
> @@ -10,7 +10,6 @@
> #include <linux/pagemap.h>
> #include <linux/uio.h>
> #include <linux/buffer_head.h>
> -#include <linux/dax.h>
> #include <linux/writeback.h>
> #include <linux/list_sort.h>
> #include <linux/swap.h>
Yes, given that both you and hch have mentioned it, could one of you
send a cleanup series for that?
>
> >> Sorry Darrick. I will stop here for now.
> >> I will continue it from here later.
> >
> > Ok. The rest of the doc will hopefully make it more obvious why there's
> > the generic discussion up here.
> >
>
> Sure. I am going through it.
<nod>
--D
> -ritesh
>
On Tue, Jun 11, 2024 at 04:15:02PM +0530, Ritesh Harjani wrote:
>
> Hi Darrick,
>
> Resuming my review from where I left off yesterday.
<snip>
> > +Validation
> > +==========
> > +
> > +**NOTE**: iomap only handles mapping and I/O.
> > +Filesystems must still call out to the VFS to check input parameters
> > +and file state before initiating an I/O operation.
> > +It does not handle updating of timestamps, stripping privileges, or
> > +access control.
> > +
> > +Locking Hierarchy
> > +=================
> > +
> > +iomap requires that filesystems provide their own locking.
> > +There are no locks within iomap itself, though in the course of an
>
> That might not be totally true. There is a state_lock within iomap_folio_state ;)
>
> > +operation iomap may take other locks (e.g. folio/dax locks) as part of
> > +an I/O operation.
>
> I think we need not mention "dax locks" here right? Since most of that
> code is in fs/dax.c anyways?
Well they're examples, so I think we can leave them.
> > +Locking with iomap can be split into two categories: above and below
> > +iomap.
> > +
> > +The upper level of lock must coordinate the iomap operation with other
> > +iomap operations.
>
> Can we add some more details in this line or maybe an example?
> Otherwise confusing use of "iomap operation" term.
How about:
"iomap requires that filesystems supply their own locking model. There
are three categories of synchronization primitives, as far as iomap is
concerned:
* The **upper** level primitive is provided by the filesystem to
coordinate access to different iomap operations.
The exact primitive is specifc to the filesystem and operation,
but is often a VFS inode, pagecache invalidation, or folio lock.
For example, a filesystem might take ``i_rwsem`` before calling
``iomap_file_buffered_write`` and ``iomap_file_unshare`` to prevent
these two file operations from clobbering each other.
Pagecache writeback may lock a folio to prevent other threads from
accessing the folio until writeback is underway.
* The **lower** level primitive is taken by the filesystem in the
``->iomap_begin`` and ``->iomap_end`` functions to coordinate
access to the file space mapping information.
The fields of the iomap object should be filled out while holding
this primitive.
The upper level synchronization primitive, if any, remains held
while acquiring the lower level synchronization primitive.
For example, XFS takes ``ILOCK_EXCL`` and ext4 takes ``i_data_sem``
while sampling mappings.
Filesystems with immutable mapping information may not require
synchronization here.
* The **operation** primitive is taken by an iomap operation to
coordinate access to its own internal data structures.
The upper level synchronization primitive, if any, remains held
while acquiring this primitive.
The lower level primitive is not held while acquiring this
primitive.
For example, pagecache write operations will obtain a file mapping,
then grab and lock a folio to copy new contents.
It may also lock an internal folio state object to update metadata.
The exact locking requirements are specific to the filesystem; for
certain operations, some of these locks can be elided.
All further mention of locking are *recommendations*, not mandates.
Each filesystem author must figure out the locking for themself."
> > +Generally, the filesystem must take VFS/pagecache locks such as
> > +``i_rwsem`` or ``invalidate_lock`` before calling into iomap.
> > +The exact locking requirements are specific to the type of operation.
> > +
> > +The lower level of lock must coordinate access to the mapping
> > +information.
> > +This lock is filesystem specific and should be held during
> > +``->iomap_begin`` while sampling the mapping and validity cookie.
> > +
> > +The general locking hierarchy in iomap is:
> > +
> > + * VFS or pagecache lock
> > +
>
> There is also a folio lock within iomap which now comes below VFS or
> pagecache lock.
>
> > + * Internal filesystem specific mapping lock
>
> I think it will also be helpful if we give an example of this lock for
> e.g. XFS(XFS_ILOCK) or ext4(i_data_sem)
>
> > +
> > + * iomap operation-specific lock
>
> some e.g. of what you mean here please?
>
> > +
> > +The exact locking requirements are specific to the filesystem; for
> > +certain operations, some of these locks can be elided.
> > +All further mention of locking are *recommendations*, not mandates.
> > +Each filesystem author must figure out the locking for themself.
>
> Is it also possible to explicitly list down the fact that folio_lock
> order w.r.t VFS lock (i_rwsem) (is it even with pagecache lock??) is now
> reversed with iomap v/s the legacy I/O model.
>
> There was an internal ext4 issue which got exposed due to this [1].
> So it might be useful to document the lock order change now.
>
> [1]: https://lore.kernel.org/linux-ext4/[email protected]/
>
> > +
> > +iomap Operations
> > +================
> > +
> > +Below are a discussion of the file operations that iomap implements.
> > +
> > +Buffered I/O
> > +------------
> > +
> > +Buffered I/O is the default file I/O path in Linux.
> > +File contents are cached in memory ("pagecache") to satisfy reads and
> > +writes.
> > +Dirty cache will be written back to disk at some point that can be
> > +forced via ``fsync`` and variants.
> > +
> > +iomap implements nearly all the folio and pagecache management that
> > +filesystems once had to implement themselves.
>
> nit: that "earlier in the legacy I/O model filesystems had to implement
> themselves"
"iomap implements nearly all the folio and pagecache management that
filesystems have to implement themselves for the legacy I/O model."
?
> > +This means that the filesystem need not know the details of allocating,
> > +mapping, managing uptodate and dirty state, or writeback of pagecache
> > +folios.
> > +Unless the filesystem explicitly opts in to buffer heads, they will not
> > +be used, which makes buffered I/O much more efficient, and ``willy``
>
> Could also please list down why buffered I/O is more efficient with
> iomap (other than the fact that iomap has large folios)?
>
> If I am not wrong, it comes from the fact that iomap only maintains
> (other than sizeof iomap_folio_state once) 2 extra bytes per fsblock v/s
> the 104 extra bytes of struct buffer_head per fsblock in the legacy I/O model.
> And while iterating over the pagecache pages, it is much faster to
> set/clear the uptodate/dirty bits of a folio in iomap v/s iterating over
> each bufferhead within a folio in legacy I/O model.
>
> Right?
Yeah. How about:
"iomap implements nearly all the folio and pagecache management that
filesystems have to implement themselves under the legacy I/O model.
This means that the filesystem need not know the details of allocating,
mapping, managing uptodate and dirty state, or writeback of pagecache
folios. Under the legacy I/O model, this was managed very inefficiently
with linked lists of buffer heads instead of the per-folio bitmaps that
iomap uses. Unless the filesystem explicitly opts in to buffer heads,
they will not be used, which makes buffered I/O much more efficient, and
the pagecache maintainer much happier."
<snip>
> > +Writes
> > +~~~~~~
> > +
> > +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
> > +pagecache.
> > +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
> > +the ``flags`` argument to ``->iomap_begin``.
> > +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
>
> shared(e.g. aligned overwrites)
That's a matter of debate -- xfs locks out concurrent reads by taking
i_rwsem in exclusive mode, whereas (I think?) ext4 and most other
filesystems take it in shared mode and synchronizes readers and writers
with folio locks.
There was some discussion before/during LSF about relaxing XFS' locking
model since most linux programs don't seem to care that readers can see
partially written contents if a write crosses a folio boundary.
> > +
> > +mmap Write Faults
> > +^^^^^^^^^^^^^^^^^
> > +
> > +The ``iomap_page_mkwrite`` function handles a write fault to a folio the
> > +pagecache.
>
> "handles a write fault to the pagecache" ?
I'd earlier corrected that to read "...to a folio in the pagecache."
>
>
> > +``IOMAP_WRITE | IOMAP_FAULT`` will be passed as the ``flags`` argument
> > +to ``->iomap_begin``.
> > +Callers commonly take the mmap ``invalidate_lock`` in shared or
> > +exclusive mode.
> > +
> > +Write Failures
> > +^^^^^^^^^^^^^^
> > +
> > +After a short write to the pagecache, the areas not written will not
> > +become marked dirty.
> > +The filesystem must arrange to `cancel
> > +<https://lore.kernel.org/all/[email protected]/>`_
> > +such `reservations
> > +<https://lore.kernel.org/linux-xfs/[email protected]/>`_
> > +because writeback will not consume the reservation.
> > +The ``iomap_file_buffered_write_punch_delalloc`` can be called from a
> > +``->iomap_end`` function to find all the clean areas of the folios
> > +caching a fresh (``IOMAP_F_NEW``) delalloc mapping.
> > +It takes the ``invalidate_lock``.
> > +
> > +The filesystem should supply a callback ``punch`` will be called for
>
> The filesystem supplied ``punch`` callback will be called for...
"The filesystem must supply a function ``punch`` to be called for..."
> > +each file range in this state.
> > +This function must *only* remove delayed allocation reservations, in
> > +case another thread racing with the current thread writes successfully
> > +to the same region and triggers writeback to flush the dirty data out to
> > +disk.
> > +
> > +Truncation
> > +^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_truncate_page`` to zero the bytes in the
> > +pagecache from EOF to the end of the fsblock during a file truncation
> > +operation.
> > +``truncate_setsize`` or ``truncate_pagecache`` will take care of
> > +everything after the EOF block.
> > +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Zeroing for File Operations
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_zero_range`` to perform zeroing of the
> > +pagecache for non-truncation file operations that are not aligned to
> > +the fsblock size.
> > +``IOMAP_ZERO`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Unsharing Reflinked File Data
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +Filesystems can call ``iomap_file_unshare`` to force a file sharing
> > +storage with another file to preemptively copy the shared data to newly
> > +allocate storage.
> > +``IOMAP_WRITE | IOMAP_UNSHARE`` will be passed as the ``flags`` argument
> > +to ``->iomap_begin``.
> > +Callers typically take ``i_rwsem`` and ``invalidate_lock`` in exclusive
> > +mode.
> > +
> > +Writeback
> > +~~~~~~~~~
> > +
> > +Filesystems can call ``iomap_writepages`` to respond to a request to
> > +write dirty pagecache folios to disk.
> > +The ``mapping`` and ``wbc`` parameters should be passed unchanged.
> > +The ``wpc`` pointer should be allocated by the filesystem and must
> > +be initialized to zero.
> > +
> > +The pagecache will lock each folio before trying to schedule it for
> > +writeback.
> > +It does not lock ``i_rwsem`` or ``invalidate_lock``.
> > +
> > +The dirty bit will be cleared for all folios run through the
> > +``->map_blocks`` machinery described below even if the writeback fails.
> > +This is to prevent dirty folio clots when storage devices fail; an
> > +``-EIO`` is recorded for userspace to collect via ``fsync``.
> > +
> > +The ``ops`` structure must be specified and is as follows:
> > +
> > +struct iomap_writeback_ops
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +.. code-block:: c
> > +
> > + struct iomap_writeback_ops {
> > + int (*map_blocks)(struct iomap_writepage_ctx *wpc, struct inode *inode,
> > + loff_t offset, unsigned len);
> > + int (*prepare_ioend)(struct iomap_ioend *ioend, int status);
> > + void (*discard_folio)(struct folio *folio, loff_t pos);
> > + };
> > +
> > +The fields are as follows:
> > +
> > + - ``map_blocks``: Sets ``wpc->iomap`` to the space mapping of the file
> > + range (in bytes) given by ``offset`` and ``len``.
> > + iomap calls this function for each fs block in each dirty folio,
> > + even if the mapping returned is longer than one fs block.
>
> It's no longer true after this patch right [1]. iomap calls this
> function for each contiguous range of dirty fsblocks within a dirty folio.
>
> [1]: https://lore.kernel.org/all/[email protected]/
Oh! It does! I forgot about this series a second time. :(
"iomap calls this function for each dirty fs block in each dirty folio,
though it will reuse mappings for runs of contiguous dirty fsblocks
within a folio."
> > + Do not return ``IOMAP_INLINE`` mappings here; the ``->iomap_end``
> > + function must deal with persisting written data.
> > + Filesystems can skip a potentially expensive mapping lookup if the
> > + mappings have not changed.
> > + This revalidation must be open-coded by the filesystem; it is
> > + unclear if ``iomap::validity_cookie`` can be reused for this
> > + purpose.
>
> struct iomap_writepage_ctx defines it's own ``struct iomap`` as a member.
>
> struct iomap_writepage_ctx {
> struct iomap iomap;
> struct iomap_ioend *ioend;
> const struct iomap_writeback_ops *ops;
> u32 nr_folios; /* folios added to the ioend */
> };
>
> That means it does not conflict with the context which is doing buffered
> writes (i.e. write_iter) and writeback is anyway single threaded.
> So we should be able to use wpc->iomap.validity_cookie for validating
> whether the cookie is valid or not during the course of writeback
> operation - (IMO)
We could, since the validity cookie is merely a u64 value that the
filesystem gets to define completely. But someone will have to check
the xfs mechanisms very carefully to make sure we encode it correctly.
I think it's a simple matter of combining the value that gets written to
data_seq/cow_seq into a single u64, passing that to xfs_bmbt_to_iomap,
and revalidating it later in xfs_imap_valid. However, the behavior of
the validity cookie and cow/data_seq are different when IOMAP_F_SHARED
is set, so this is tricky.
>
> > + This function is required.
>
> This line is left incomplete.
I disagree, but perhaps it would be clearer if it said:
"This function must be supplied by the filesystem." ?
> I think we should also mention this right? -
>
> If the filesystem reserved delalloc extents during buffered-writes, than
> they should allocate extents for those delalloc mappings in this
> ->map_blocks call.
Technically speaking iomap doesn't screen for that, but writeback will
probably do a very wrong thing if the fs supplies a delalloc mapping.
I'll update the doc to say that:
"Do not return IOMAP_DELALLOC mappings here; iomap currently requires
mapping to allocated space."
Though I guess if hch or someone gets back to the "write and tell me
where you wrote it" patchset then I guess it /would/ be appropriate to
use IOMAP_DELALLOC here, and let the block device tell us what to map.
> > +
> > + - ``prepare_ioend``: Enables filesystems to transform the writeback
> > + ioend or perform any other prepatory work before the writeback I/O
>
> IMO, some e.g. will be very helpful to add wherever possible. I
> understand we should keep the document generic enough, but it is much
> easier if we state some common examples of what XFS / other filesystems
> do with such callback methods.
>
> e.g.
>
> - What do we mean by "transform the writeback ioend"? I guess it is -
> XFS uses this for conversion of COW extents to regular extents?
Yes, the xfs ioend processing will move the mappings for freshly written
extents from the cow fork to the data fork.
> - What do we mean by "perform any other preparatory work before the
> writeback I/O"? - I guess it is -
> XFS hooks in custom a completion handler in ->prepare_ioend callback for
> conversion of unwritten extents.
Yes.
"prepare_ioend: Enables filesystems to transform the writeback ioend or
perform any other preparatory work before the writeback I/O is
submitted. This might include pre-write space accounting updates, or
installing a custom ->bi_end_io function for internal purposes such as
deferring the ioend completion to a workqueue to run metadata update
transactions from process context. This function is optional."
> > + is submitted.
> > + A filesystem can override the ``->bi_end_io`` function for its own
> > + purposes, such as kicking the ioend completion to a workqueue if the
> > + bio is completed in interrupt context.
>
> Thanks this is also helpful.
<nod>
> > + This function is optional.
> > +
> > + - ``discard_folio``: iomap calls this function after ``->map_blocks``
> > + fails schedule I/O for any part of a dirty folio.
>
> fails "to" schedule
Thanks, fixed.
> > + The function should throw away any reservations that may have been
> > + made for the write.
> > + The folio will be marked clean and an ``-EIO`` recorded in the
> > + pagecache.
> > + Filesystems can use this callback to `remove
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + delalloc reservations to avoid having delalloc reservations for
> > + clean pagecache.
> > + This function is optional.
> > +
> > +Writeback ioend Completion
> > +^^^^^^^^^^^^^^^^^^^^^^^^^^
> > +
> > +iomap creates chains of ``struct iomap_ioend`` objects that wrap the
> > +``bio`` that is used to write pagecache data to disk.
> > +By default, iomap finishes writeback ioends by clearing the writeback
> > +bit on the folios attached to the ``ioend``.
> > +If the write failed, it will also set the error bits on the folios and
> > +the address space.
> > +This can happen in interrupt or process context, depending on the
> > +storage device.
> > +
> > +Filesystems that need to update internal bookkeeping (e.g. unwritten
> > +extent conversions) should provide a ``->prepare_ioend`` function to
>
> Ok, you did actually mention the unwritten conversion example here.
> However no harm in also mentioning this in the section which gives info
> about ->prepare_ioend callback :)
Ok, I'll reference that again:
"This function should call iomap_finish_ioends after finishing its own
work (e.g. unwritten extent conversion)."
> > +override the ``struct iomap_end::bio::bi_end_io`` with its own function.
> > +This function should call ``iomap_finish_ioends`` after finishing its
> > +own work.
> > +
> > +Some filesystems may wish to `amortize the cost of running metadata
> > +transactions
> > +<https://lore.kernel.org/all/[email protected]/>`_
> > +for post-writeback updates by batching them.
>
> > +They may also require transactions to run from process context, which
> > +implies punting batches to a workqueue.
> > +iomap ioends contain a ``list_head`` to enable batching.
> > +
> > +Given a batch of ioends, iomap has a few helpers to assist with
> > +amortization:
> > +
> > + * ``iomap_sort_ioends``: Sort all the ioends in the list by file
> > + offset.
> > +
> > + * ``iomap_ioend_try_merge``: Given an ioend that is not in any list and
> > + a separate list of sorted ioends, merge as many of the ioends from
> > + the head of the list into the given ioend.
> > + ioends can only be merged if the file range and storage addresses are
> > + contiguous; the unwritten and shared status are the same; and the
> > + write I/O outcome is the same.
> > + The merged ioends become their own list.
> > +
> > + * ``iomap_finish_ioends``: Finish an ioend that possibly has other
> > + ioends linked to it.
> > +
>
> Again sorry for stopping here. I will continue the review from Direct-io later.
Ok, see you tomorrow.
--D
> Thanks!
> -ritesh
>
On Tue, Jun 11, 2024 at 11:32:31AM +1000, Dave Chinner wrote:
> On Mon, Jun 10, 2024 at 02:59:28PM -0700, Darrick J. Wong wrote:
> > On Mon, Jun 10, 2024 at 04:18:08PM +0200, Jan Kara wrote:
> > > On Sun 09-06-24 08:55:06, Darrick J. Wong wrote:
> > > > * validity_cookie is a magic freshness value set by the
> > > > filesystem that should be used to detect stale mappings. For
> > > > pagecache operations this is critical for correct operation
> > > > because page faults can occur, which implies that filesystem
> > > > locks should not be held between ->iomap_begin and
> > > > ->iomap_end. Filesystems with completely static mappings
> > > > need not set this value. Only pagecache operations
> > > > revalidate mappings.
> > > >
> > > > XXX: Should fsdax revalidate as well?
> > >
> > > AFAICT no. DAX is more like using direct IO for everything. So no writeback
> > > changing mapping state behind your back (and that's the only thing that is
> > > not serialized with i_rwsem or invalidate_lock). Maybe this fact can be
> > > mentioned somewhere around the discussion of iomap_valid() as a way how
> > > locking usually works out?
> >
> > <nod> I'll put that in the section about iomap_valid, which documents
> > the whole mechanism more thoroughly:
> >
> > "iomap_valid: The filesystem may not hold locks between ->iomap_begin
> > and ->iomap_end because pagecache operations can take folio locks, fault
> > on userspace pages, initiate writeback for memory reclamation, or engage
> > in other time-consuming actions. If a file's space mapping data are
> > mutable, it is possible that the mapping for a particular pagecache
> > folio can change in the time it takes to allocate, install, and lock
> > that folio.
> >
> > "For the pagecache, races can happen if writeback doesn't take i_rwsem
> > or invalidate_lock and updates mapping information. Races can also
> > happen if the filesytem allows concurrent writes. For such files, the
> > mapping *must* be revalidated after the folio lock has been taken so
> > that iomap can manage the folio correctly.
> >
> > "fsdax does not need this revalidation because there's no writeback and
> > no support for unwritten extents.
> >
> > "The filesystem's ->iomap_begin function must sample a sequence counter
> > into struct iomap::validity_cookie at the same time that it populates
> > the mapping fields. It must then provide a ->iomap_valid function to
> > compare the validity cookie against the source counter and return
> > whether or not the mapping is still valid. If the mapping is not valid,
> > the mapping will be sampled again."
>
> Everything is fine except for the last paragraph. Filesystems do not
> need to use a sequence counter for the validity cookie - they can
> use any mechanism they want to determine that the extent state has
> changed. If they can track extent status on a fine grained basis
> (e.g. ext4 has individual extent state cache entries) then that
> validity cookie could be a pointer to the filesystem's internal
> extent state cache entry.
>
> There's also nothing that says the cookie needs to remain a u64;
> I've been toying with replacing it with a pointer to an xfs iext
> btree cursor and checking that the cursor still points to the same
> extent that iomap was made from. Hence we get fine grained extent
> checks and don't have to worry about changes outside the range of
> the iomap causing spurious revalidation failures.
>
> IOWs, the intention of the cookie is simply an opaque blob that the
> filesystem can keep whatever validity information in it that it
> wants - a sequence counter is just one of many possible
> implementations. hence I think this should be rephrased to reflect
> this.
>
> "The filesystem's ->iomap_begin function must write the data it
> needs to validate the iomap into struct iomap::validity_cookie at
> the same time that it populates the mapping fields. It must then
> provide a ->iomap_valid function to compare the validity cookie
> against the source data and return whether or not the mapping is
> still valid. If the mapping is not valid, the mapping will be
> sampled again.
>
> A simple validation cookie implementation is a sequence counter. If
> the filesystem bumps the sequence counter every time it modifies the
> inode's extent map, it can be placed in the struct
> iomap::validity_cookie during ->iomap_begin. If the value in the
> cookie is found to be different to the value the filesystem holds
> when it is passed back to ->iomap_valid, then the iomap should
> considered stale and the validation failed."
Ok, I'll rework my last paragraph and add yours after it:
"Filesystems subject to this kind of race must provide a ->iomap_valid
function to decide if the mapping is still valid. If the mapping is not
valid, the mapping will be sampled again. To support making the
validity decision, the filesystem's ->iomap_begin function may set
struct iomap::validity_cookie at the same time that it populates the
other iomap fields.
"A simple validation cookie implementation is a sequence counter. If
the filesystem bumps the sequence counter every time it modifies the
inode's extent map, it can be placed in the struct
iomap::validity_cookie during ->iomap_begin. If the value in the cookie
is found to be different to the value the filesystem holds when the
mapping is passed back to ->iomap_valid, then the iomap should
considered stale and the validation failed."
> > > > These struct kiocb flags are significant for buffered I/O with
> > > > iomap:
> > > >
> > > > * IOCB_NOWAIT: Only proceed with the I/O if mapping data are
> > > > already in memory, we do not have to initiate other I/O, and
> > > > we acquire all filesystem locks without blocking. Neither
> > > > this flag nor its definition RWF_NOWAIT actually define what
> > > > this flag means, so this is the best the author could come
> > > > up with.
> > >
> > > RWF_NOWAIT is a performance feature, not a correctness one, hence the
> > > meaning is somewhat vague. It is meant to mean "do the IO only if it
> > > doesn't involve waiting for other IO or other time expensive operations".
> > > Generally we translate it to "don't wait for i_rwsem, page locks, don't do
> > > block allocation, etc." OTOH we don't bother to specialcase internal
> > > filesystem locks (such as EXT4_I(inode)->i_data_sem)
>
> But we have support for this - the IOMAP_NOWAIT flag tells the
> filesystem iomap methods that it should:
>
> a) not block on internal mapping operations; and
> b) fail with -EAGAIN if it can't map the entire range in a
> single iomap.
>
> XFS implements these semantics for internal locks and operations
> needed for mapping operations, and any filesystem that uses iomap
> -should- be doing the same thing.
>
> > > and we get away with
> > > it because blocking on it under constraints we generally perform RWF_NOWAIT
> > > IO is exceedingly rare.
> >
> > I hate this flag's undocumented nature. It now makes *documenting*
> > things around it hard. How about:
>
> But it is documented.
>
> RWF_NOWAIT (since Linux 4.14)
>
> Do not wait for data which is not immediately available. If
> this flag is specified, the preadv2() system call will return
> instantly if it would have to read data from the backing
> storage or wait for a lock. [...]
>
> Yes, not well documented. But the *intent* is clear. It's basically
> the same thing as O_NONBLOCK:
>
> O_NONBLOCK or O_NDELAY
> When possible, the file is opened in nonblocking mode.
> Neither the open() nor any subsequent I/O operations on the
> file descriptor which is returned will cause the calling
> process to wait.
>
> That's what we are supposed to be implementing with IOCB_NOWAIT -
> don't block the submitting task if at all possible.
Sorry. I didn't realize that it /had/ been documented, just not in the
kernel.
"IOCB_NOWAIT: Do not wait for data which is not immediately available.
XFS and ext4 appear to reject the IO unless the mapping data are already
in memory, the filesystem does not have to initiate other I/O, and the
kernel can acquire all filesystem locks without blocking."
> > "IOCB_NOWAIT: Neither this flag nor its associated definition
> > RWF_NOWAIT actually specify what this flag means. Community
> > members seem to think that it means only proceed with the I/O if
> > it doesn't involve waiting for expensive operations. XFS and ext4
> > appear to reject the IO unless the mapping data are already in
> > memory, the filesystem does not have to initiate other I/O, and
> > the kernel can acquire all filesystem locks without blocking."
>
> I think the passive-aggressive nature of this statement doesn't read
> very well. Blaming "community members" for organic code development
> that solved poorly defined, undocumented bleeding edge issues isn't
> a winning strategy. And other developers don't care about this; they
> just want to know what they should be implementing.
Some day soon I'll be gone and you won't have me poisoning the
community anymore.
> So let's just make a clear statement about the intent of
> IOCB_NOWAIT, because that is *always* been very clear right from the
> start, even though there was no way we could implement the intent
> right from the start.
>
> "IOCB_NOWAIT: Perform a best effort attempt to avoid any operation
> that would result in blocking the submitting task. This is similar
> in intent to O_NONBLOCK for network APIs - it is intended for
> asynchronous applications to keep doing other work instead of
> waiting for the specific unavailable filesystem resource to become
> available.
>
> This means the filesystem implementing IOCB_NOWAIT semantics need to
> use trylock algorithms. They need to be able to satisfy the entire
> IO request range in a single iomap mapping. They need to avoid
> reading or writing metadata synchronously. They need to avoid
> blocking memory allocations. They need to avoid waiting on
> transaction reservations to allow modifications to take place. And
> so on.
>
> If there is any doubt in the filesystem developer's mind as to
> whether any specific IOCB_NOWAIT operation may end up blocking, then
> they should return -EAGAIN as early as possible rather than start
> the operation and force the submitting task to block."
>
> This clearly documents the intent and what the submitters are
> expecting from filesystems when this flag is set. IT also tells
> filesystem implementers the way to handle IOCB_NOWAIT if they
> haven't actually coded any of this - abort at the top of the IO
> stack with -EAGAIN - and applications using this will work
> correctly.
Yes, that's better. Thank you for that.
> > > > Direct Writes
> > > >
> > > > A direct I/O write initiates a write I/O to the storage device to
> > > > the caller's buffer. Dirty parts of the pagecache are flushed to
> > > > storage before initiating the write io. The pagecache is
> > > > invalidated both before and after the write io. The flags value
> > > > for ->iomap_begin will be IOMAP_DIRECT | IOMAP_WRITE with any
> > > > combination of the following enhancements:
> > > >
> > > > * IOMAP_NOWAIT: Write if mapping data are already in memory.
> > > > Does not initiate other I/O or block on filesystem locks.
>
> IOMAP_NOWAIT is not specific to direct IO. It is supported for both
> buffered reads and writes in iomap and XFS.
>
> It also tends to imply "don't allocate new blocks" for journalling
> filesysetms because that requires journal space reservation (which
> blocks), memory allocation, metadata creation and metadata IO.
>
> > > > * IOMAP_OVERWRITE_ONLY: Allocating blocks and zeroing partial
> > > > blocks is not allowed. The entire file range must to map to
> > > ^^ extra "to"
> > >
> > > > a single written or unwritten extent. The file I/O range
> > > > must be aligned to the filesystem block size.
>
>
> > > This seems to be XFS specific thing? At least I don't see anything in
> > > generic iomap code depending on this?
> >
> > Hmm. XFS bails out if the mapping is unwritten and the directio write
> > range isn't aligned to the fsblock size.
>
> I think that is wrong. IOMAP_OVERWRITE_ONLY is specifically for
> allowing unaligned IO to be performed without triggering allocation
> or sub-block zeroing. It requests a mapping to allow a pure
> overwrite to be performed, otherwise it fails.
>
> XFS first it checks if allocation is required. If so, it bails.
>
> Second it checks if the extent spans the range requested. If not, it
> bails.
>
> Finally, it checks if the extent is IOMAP_WRITTEN. If not, it bails.
>
> > I think the reason for that is
> > because we'd have to zero the unaligned regions outside of the write
> > range, and xfs can't do that without synchronizing. (Or we didn't think
>
> Right, it enables an optimistic fast path for unaligned direct
> IOs. We take the i_rwsem shared, attempt the fast path, if it is
> rejected with -EAGAIN we drop the shared lock, take it exclusive
> and run the IO again without IOMAP_DIO_OVERWRITE_ONLY being set
> to allow allocation and/or sub-block zeroing to be performed.
>
> I think this needs to be written in terms of what a "pure overwrite"
> is. We use that term repeatedly in the iomap code (e.g. for FUA
> optimisations), and IOMAP_OVERWRITE_ONLY is a mechanism for the
> caller to ask "give me a pure overwrite mapping for this range or
> fail with -EAGAIN". This is the mechanism that provides the required
> IOMAP_DIO_OVERWRITE_ONLY semantics.
>
>
> "Pure Overwrite: A write operation that does not require any
> metadata or zeroing operations to perform during either submission
> or completion. This implies that the fileystem must have already
> allocated space on disk as IOMAP_WRITTEN and the filesystem must not
> place any constaints on IO alignment or size - the only constraints
> on IO alignment are device level (minimum IO size and alignment,
> typically sector size).
>
> IOMAP_DIO_OVERWRITE_ONLY: Perform a pure overwrite for this range or
> fail with -EAGAIN.
>
> This can be used by filesystems with complex unaligned IO
> write paths to provide an optimised fast path for unaligned writes.
> If a pure overwrite can be performed, then serialisation against
> other IOs to the same filesystem block(s) is unnecessary as there is
> no risk of stale data exposure or data loss.
>
> If a pure overwrite cannot be performed, then the filesystem can
> perform the serialisation steps needed to provide exclusive access
> to the unaligned IO range so that it can perform allocation and
> sub-block zeroing safely.
>
> IOMAP_OVERWRITE_ONLY: The caller requires a pure overwrite to be
> performed from this mapping. This requires the filesystem extent
> mapping to already exist as an IOMAP_MAPPED type and span the entire
> range of the write IO request. If the filesystem cannot map this
> request in a way that allows the iomap infrastructure to perform
> a pure overwrite, it must fail the mapping operation with -EAGAIN."
Yes, that's a much better explanation. I'll incorporate those. Thank
you.
--D
> -Dave.
> --
> Dave Chinner
> [email protected]
>
"Darrick J. Wong" <[email protected]> writes:
> On Tue, Jun 11, 2024 at 04:15:02PM +0530, Ritesh Harjani wrote:
>>
>> Hi Darrick,
>>
>> Resuming my review from where I left off yesterday.
>
<snip>
>> > +Writes
>> > +~~~~~~
>> > +
>> > +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
>> > +pagecache.
>> > +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
>> > +the ``flags`` argument to ``->iomap_begin``.
>> > +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
>>
>> shared(e.g. aligned overwrites)
>
Ok, I see we were in buffered I/O section (Sorry, I misunderstood
thinking this was for direct-io)
> That's a matter of debate -- xfs locks out concurrent reads by taking
> i_rwsem in exclusive mode, whereas (I think?) ext4 and most other
> filesystems take it in shared mode and synchronizes readers and writers
> with folio locks.
Ext4 too takes inode lock in exclusive mode in case of
buffered-write. It's the DIO writes/overwrites in ext4 which has special
casing for shared/exclusive mode locking.
But ext4 buffered-read does not take any inode lock (it uses
generic_file_read_iter()). So the synchronization must happen via folio
lock w.r.t buffered-writes.
However, I am not sure if we have any filesystem taking VFS inode lock in
shared more for buffered-writes.
BTW -
I really like all of the other updates that you made w.r.t the review
comments. All of those looks more clear to me. (so not commenting on them
individually).
Thanks!
-ritesh
"Darrick J. Wong" <[email protected]> writes:
> On Tue, Jun 11, 2024 at 12:13:22PM +0530, Ritesh Harjani wrote:
>> "Darrick J. Wong" <[email protected]> writes:
>>
<snip>
>> >> > + * ``addr`` describes the device address, in bytes.
>> >> > +
>> >> > + * ``bdev`` describes the block device for this mapping.
>> >> > + This only needs to be set for mapped or unwritten operations.
>> >> > +
>> >> > + * ``dax_dev`` describes the DAX device for this mapping.
>> >> > + This only needs to be set for mapped or unwritten operations, and
>> >> > + only for a fsdax operation.
>> >>
>> >> Looks like we can make this union (bdev and dax_dev). Since depending
>> >> upon IOMAP_DAX or not we only set either dax_dev or bdev.
>> >
>> > Separate patch. ;)
>> >
>>
>> Yes, in a way I was trying to get an opinion from you and others on
>> whether it make sense to make bdev and dax_dev as union :)
>>
>> Looks like this series [1] could be the reason for that.
>>
>> [1]: https://lore.kernel.org/all/[email protected]/#t
>>
>> I also don't see any reference to dax code from fs/iomap/buffered-io.c
>> So maybe we don't need this dax.h header in this file.
>>
>> diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
>> index c5802a459334..e1a6cca3cec2 100644
>> --- a/fs/iomap/buffered-io.c
>> +++ b/fs/iomap/buffered-io.c
>> @@ -10,7 +10,6 @@
>> #include <linux/pagemap.h>
>> #include <linux/uio.h>
>> #include <linux/buffer_head.h>
>> -#include <linux/dax.h>
>> #include <linux/writeback.h>
>> #include <linux/list_sort.h>
>> #include <linux/swap.h>
>
> Yes, given that both you and hch have mentioned it, could one of you
> send a cleanup series for that?
>
Sure, Thanks Darrick and Christoph.
I can queue this with my other work where I am improving iomap for
indirect-block mapping, so that it will be easier to get testing done on
all of this together.
-ritesh
On Wed, Jun 12, 2024 at 12:07:40PM +0530, Ritesh Harjani wrote:
>
> "Darrick J. Wong" <[email protected]> writes:
>
> > On Tue, Jun 11, 2024 at 04:15:02PM +0530, Ritesh Harjani wrote:
> >>
> >> Hi Darrick,
> >>
> >> Resuming my review from where I left off yesterday.
> >
>
> <snip>
> >> > +Writes
> >> > +~~~~~~
> >> > +
> >> > +The ``iomap_file_buffered_write`` function writes an ``iocb`` to the
> >> > +pagecache.
> >> > +``IOMAP_WRITE`` or ``IOMAP_WRITE`` | ``IOMAP_NOWAIT`` will be passed as
> >> > +the ``flags`` argument to ``->iomap_begin``.
> >> > +Callers commonly take ``i_rwsem`` in either shared or exclusive mode.
> >>
> >> shared(e.g. aligned overwrites)
> >
>
> Ok, I see we were in buffered I/O section (Sorry, I misunderstood
> thinking this was for direct-io)
Aha. I'll change these headings to "Buffered Readahead and Reads" and
"Buffered Writes".
> > That's a matter of debate -- xfs locks out concurrent reads by taking
> > i_rwsem in exclusive mode, whereas (I think?) ext4 and most other
> > filesystems take it in shared mode and synchronizes readers and writers
> > with folio locks.
>
> Ext4 too takes inode lock in exclusive mode in case of
> buffered-write. It's the DIO writes/overwrites in ext4 which has special
> casing for shared/exclusive mode locking.
>
> But ext4 buffered-read does not take any inode lock (it uses
> generic_file_read_iter()). So the synchronization must happen via folio
> lock w.r.t buffered-writes.
>
> However, I am not sure if we have any filesystem taking VFS inode lock in
> shared more for buffered-writes.
In theory you could if no other metadata needed updating, such as a dumb
filesystem with fixed size files where timestamps don't matter.
> BTW -
> I really like all of the other updates that you made w.r.t the review
> comments. All of those looks more clear to me. (so not commenting on them
> individually).
>
> Thanks!
No, thank /you/ and everyone else for reading all the way through it.
I'll finish cleaning things up and put out a v2.
--D
> -ritesh
>
<snip>
> +Direct I/O
> +----------
> +
> +In Linux, direct I/O is defined as file I/O that is issued directly to
> +storage, bypassing the pagecache.
> +
> +The ``iomap_dio_rw`` function implements O_DIRECT (direct I/O) reads and
> +writes for files.
> +An optional ``ops`` parameter can be passed to change the behavior of
> +direct I/O.
Did you mean "dops" iomap_dio_ops (instead of ops)?
ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
const struct iomap_ops *ops, const struct iomap_dio_ops *dops,
unsigned int dio_flags, void *private, size_t done_before);
1. Also can you please explain what you meant by "change the behavior of
direct-io"?
2. Do you think we should add the function declaration of
iomap_dio_rw() here, given it has so many arguments?
> +The ``done_before`` parameter should be set if writes have been
> +initiated prior to the call.
I don't think this is specific to "writes" alone.
Maybe this?
The ``done_before`` parameter tells the how much of the request has
already been transferred. It gets used for finishing a request
asynchronously when part of the request has already been complete
synchronously.
Maybe please also add a the link to this (for easy reference).
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c03098d4b9ad76bca2966a8769dcfe59f7f85103
> +The direction of the I/O is determined from the ``iocb`` passed in.
> +
> +The ``flags`` argument can be any of the following values:
Callers of iomap_dio_rw() can set the flags argument which can be any of
the following values:
Just a bit more descriptive ^^^
> +
> + * ``IOMAP_DIO_FORCE_WAIT``: Wait for the I/O to complete even if the
> + kiocb is not synchronous.
Adding an example would be nice.
e.g. callers might want to consider setting this flag for extending writes.
> +
> + * ``IOMAP_DIO_OVERWRITE_ONLY``: Allocating blocks, zeroing partial
> + blocks, and extensions of the file size are not allowed.
> + The entire file range must to map to a single written or unwritten
^^^ an extra to
> + extent.
> + This flag exists to enable issuing concurrent direct IOs with only
> + the shared ``i_rwsem`` held when the file I/O range is not aligned to
> + the filesystem block size.
> + ``-EAGAIN`` will be returned if the operation cannot proceed.
Can we please add these below details too. I would rather avoid wasting
my time in searching the history about, why EXT4 does not use this flag :)
Currently XFS uses this flag. EXT4 does not use it since it checks for
overwrites or unaligned overwrites and uses appropriate locking
up front rather than on a retry response to -EAGAIN [1] [2].
[1]: https://lore.kernel.org/linux-ext4/[email protected]/
[2]: https://lore.kernel.org/linux-ext4/[email protected]/
> +
> + * ``IOMAP_DIO_PARTIAL``: If a page fault occurs, return whatever
> + progress has already been made.
> + The caller may deal with the page fault and retry the operation.
Callers use ``dio_before`` argument along with ``IOMAP_DIO_PARTIAL`` to
tell the iomap subsystem about how much of the requested I/O was already
done.
> +
> +These ``struct kiocb`` flags are significant for direct I/O with iomap:
> +
> + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> + already in memory, we do not have to initiate other I/O, and we
> + acquire all filesystem locks without blocking.
> +
Maybe explicitly mentioning about "no block allocation"?
* ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
already in memory, we do not have to initiate other I/O or do any
block allocations, and we acquire all filesystem locks without
blocking.
> + * ``IOCB_SYNC``: Ensure that the device has persisted data to disk
> + BEFORE completing the call.
> + In the case of pure overwrites, the I/O may be issued with FUA
> + enabled.
> +
> + * ``IOCB_HIPRI``: Poll for I/O completion instead of waiting for an
> + interrupt.
> + Only meaningful for asynchronous I/O, and only if the entire I/O can
> + be issued as a single ``struct bio``.
> +
> + * ``IOCB_DIO_CALLER_COMP``: Try to run I/O completion from the caller's
> + process context.
> + See ``linux/fs.h`` for more details.
> +
> +Filesystems should call ``iomap_dio_rw`` from ``->read_iter`` and
> +``->write_iter``, and set ``FMODE_CAN_ODIRECT`` in the ``->open``
> +function for the file.
> +They should not set ``->direct_IO``, which is deprecated.
> +
Return value:
~~~~~~~~~~~~~
On success it will return the number of bytes transferred. On failure it
will return a negative error value.
Note:
-ENOTBLK is a magic return value which callers may use for falling back
to buffered-io. ->iomap_end()/->iomap_begin() can decide to return this
magic return value if it decides to fallback to buffered-io. iomap
subsystem return this value in case if it fails to invalidate the
pagecache pages belonging to the direct-io range before initiating the
direct-io.
-EIOCBQUEUED is returned when an async direct-io request is queued for I/O.
> +If a filesystem wishes to perform its own work before direct I/O
> +completion, it should call ``__iomap_dio_rw``.
> +If its return value is not an error pointer or a NULL pointer, the
> +filesystem should pass the return value to ``iomap_dio_complete`` after
> +finishing its internal work.
> +
> +Direct Reads
> +~~~~~~~~~~~~
> +
> +A direct I/O read initiates a read I/O from the storage device to the
> +caller's buffer.
> +Dirty parts of the pagecache are flushed to storage before initiating
> +the read io.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT`` with
> +any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Direct Writes
> +~~~~~~~~~~~~~
> +
> +A direct I/O write initiates a write I/O to the storage device to the
> +caller's buffer.
to the storage device "from" the caller's buffer.
> +Dirty parts of the pagecache are flushed to storage before initiating
> +the write io.
> +The pagecache is invalidated both before and after the write io.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT |
> +IOMAP_WRITE`` with any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> + blocks is not allowed.
> + The entire file range must to map to a single written or unwritten
> + extent.
> + The file I/O range must be aligned to the filesystem block size.
> +
> +Callers commonly hold ``i_rwsem`` in shared or exclusive mode.
> +
> +struct iomap_dio_ops:
> +~~~~~~~~~~~~~~~~~~~~~
> +.. code-block:: c
> +
> + struct iomap_dio_ops {
> + void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
> + loff_t file_offset);
> + int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
> + unsigned flags);
> + struct bio_set *bio_set;
> + };
> +
> +The fields of this structure are as follows:
> +
> + - ``submit_io``: iomap calls this function when it has constructed a
> + ``struct bio`` object for the I/O requested, and wishes to submit it
> + to the block device.
> + If no function is provided, ``submit_bio`` will be called directly.
> + Filesystems that would like to perform additional work before (e.g.
> + data replication for btrfs) should implement this function.
> +
> + - ``end_io``: This is called after the ``struct bio`` completes.
> + This function should perform post-write conversions of unwritten
> + extent mappings, handle write failures, etc.
> + The ``flags`` argument may be set to a combination of the following:
> +
> + * ``IOMAP_DIO_UNWRITTEN``: The mapping was unwritten, so the ioend
> + should mark the extent as written.
> +
> + * ``IOMAP_DIO_COW``: Writing to the space in the mapping required a
> + copy on write operation, so the ioend should switch mappings.
> +
> + - ``bio_set``: This allows the filesystem to provide a custom bio_set
> + for allocating direct I/O bios.
> + This enables filesystems to `stash additional per-bio information
> + <https://lore.kernel.org/all/[email protected]/>`_
> + for private use.
> + If this field is NULL, generic ``struct bio`` objects will be used.
> +
> +Filesystems that want to perform extra work after an I/O completion
> +should set a custom ``->bi_end_io`` function via ``->submit_io``.
> +Afterwards, the custom endio function must call
> +``iomap_dio_bio_end_io`` to finish the direct I/O.
> +
> +DAX I/O
> +-------
> +
> +Storage devices that can be directly mapped as memory support a new
> +access mode known as "fsdax".
Added a comma before "support" for better readability.
Storage devices that can be directly mapped as memory, support a new
access mode known as "fsdax".
> +
> +fsdax Reads
> +~~~~~~~~~~~
> +
> +A fsdax read performs a memcpy from storage device to the caller's
> +buffer.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX`` with any
> +combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +fsdax Writes
> +~~~~~~~~~~~~
> +
> +A fsdax write initiates a memcpy to the storage device to the caller's
"from" the storage device
> +buffer.
> +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX |
> +IOMAP_WRITE`` with any combination of the following enhancements:
> +
> + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> + Does not initiate other I/O or block on filesystem locks.
> +
> + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> + blocks is not allowed.
> + The entire file range must to map to a single written or unwritten
> + extent.
> + The file I/O range must be aligned to the filesystem block size.
> +
> +Callers commonly hold ``i_rwsem`` in exclusive mode.
> +
> +mmap Faults
> +~~~~~~~~~~~
> +
> +The ``dax_iomap_fault`` function handles read and write faults to fsdax
> +storage.
> +For a read fault, ``IOMAP_DAX | IOMAP_FAULT`` will be passed as the
> +``flags`` argument to ``->iomap_begin``.
> +For a write fault, ``IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE`` will be
> +passed as the ``flags`` argument to ``->iomap_begin``.
> +
> +Callers commonly hold the same locks as they do to call their iomap
> +pagecache counterparts.
> +
> +Truncation, fallocate, and Unsharing
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +For fsdax files, the following functions are provided to replace their
> +iomap pagecache I/O counterparts.
> +The ``flags`` argument to ``->iomap_begin`` are the same as the
> +pagecache counterparts, with ``IOMAP_DIO`` added.
with "IOMAP_DAX"
> +
> + * ``dax_file_unshare``
> + * ``dax_zero_range``
> + * ``dax_truncate_page``
Shall we mention
"dax_remap_file_range_prep()/dax_dedupe_file_range_compare()" ?
> +
> +Callers commonly hold the same locks as they do to call their iomap
> +pagecache counterparts.
> +
Stopping here for now. Will resume the rest of the document review
later.
-ritesh
On Wed, Jun 12, 2024 at 06:54:02PM +0530, Ritesh Harjani wrote:
>
> <snip>
> > +Direct I/O
> > +----------
> > +
> > +In Linux, direct I/O is defined as file I/O that is issued directly to
> > +storage, bypassing the pagecache.
> > +
> > +The ``iomap_dio_rw`` function implements O_DIRECT (direct I/O) reads and
> > +writes for files.
> > +An optional ``ops`` parameter can be passed to change the behavior of
> > +direct I/O.
>
> Did you mean "dops" iomap_dio_ops (instead of ops)?
Oops, yes.
> ssize_t iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
> const struct iomap_ops *ops, const struct iomap_dio_ops *dops,
> unsigned int dio_flags, void *private, size_t done_before);
>
> 1. Also can you please explain what you meant by "change the behavior of
> direct-io"?
"The filesystem can provide the ``dops`` parameter if it needs to
perform extra work before or after the I/O is issued to storage."
>
> 2. Do you think we should add the function declaration of
> iomap_dio_rw() here, given it has so many arguments?
Will do.
> > +The ``done_before`` parameter should be set if writes have been
> > +initiated prior to the call.
>
> I don't think this is specific to "writes" alone.
>
> Maybe this?
>
> The ``done_before`` parameter tells the how much of the request has
> already been transferred. It gets used for finishing a request
> asynchronously when part of the request has already been complete
> synchronously.
>
> Maybe please also add a the link to this (for easy reference).
> [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c03098d4b9ad76bca2966a8769dcfe59f7f85103
Yep, thanks for that.
> > +The direction of the I/O is determined from the ``iocb`` passed in.
> > +
> > +The ``flags`` argument can be any of the following values:
>
> Callers of iomap_dio_rw() can set the flags argument which can be any of
> the following values:
>
> Just a bit more descriptive ^^^
>
> > +
> > + * ``IOMAP_DIO_FORCE_WAIT``: Wait for the I/O to complete even if the
> > + kiocb is not synchronous.
>
> Adding an example would be nice.
>
> e.g. callers might want to consider setting this flag for extending writes.
>
> > +
> > + * ``IOMAP_DIO_OVERWRITE_ONLY``: Allocating blocks, zeroing partial
> > + blocks, and extensions of the file size are not allowed.
> > + The entire file range must to map to a single written or unwritten
> ^^^ an extra to
>
> > + extent.
> > + This flag exists to enable issuing concurrent direct IOs with only
> > + the shared ``i_rwsem`` held when the file I/O range is not aligned to
> > + the filesystem block size.
> > + ``-EAGAIN`` will be returned if the operation cannot proceed.
>
> Can we please add these below details too. I would rather avoid wasting
> my time in searching the history about, why EXT4 does not use this flag :)
>
> Currently XFS uses this flag. EXT4 does not use it since it checks for
> overwrites or unaligned overwrites and uses appropriate locking
> up front rather than on a retry response to -EAGAIN [1] [2].
>
> [1]: https://lore.kernel.org/linux-ext4/[email protected]/
> [2]: https://lore.kernel.org/linux-ext4/[email protected]/
Ok. I'll just mention that it's a performance optimization to reduce
lock contention, but that a lot of detailed checking is required to do
it correctly.
> > +
> > + * ``IOMAP_DIO_PARTIAL``: If a page fault occurs, return whatever
> > + progress has already been made.
> > + The caller may deal with the page fault and retry the operation.
>
> Callers use ``dio_before`` argument along with ``IOMAP_DIO_PARTIAL`` to
> tell the iomap subsystem about how much of the requested I/O was already
> done.
Let's be more specific than that -- if the caller decides to perform
multiple retries, then the done_before parameter to the next call should
be the accumulated return values of all the previous attempts.
> > +
> > +These ``struct kiocb`` flags are significant for direct I/O with iomap:
> > +
> > + * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> > + already in memory, we do not have to initiate other I/O, and we
> > + acquire all filesystem locks without blocking.
> > +
>
> Maybe explicitly mentioning about "no block allocation"?
>
> * ``IOCB_NOWAIT``: Only proceed with the I/O if mapping data are
> already in memory, we do not have to initiate other I/O or do any
> block allocations, and we acquire all filesystem locks without
> blocking.
Oh, I changed all these NOWAIT bits to define what nowait means (i.e.
dave's long paragraph about it) in its own section, and all these bullet
points merely point back to that definition.
> > + * ``IOCB_SYNC``: Ensure that the device has persisted data to disk
> > + BEFORE completing the call.
> > + In the case of pure overwrites, the I/O may be issued with FUA
> > + enabled.
> > +
> > + * ``IOCB_HIPRI``: Poll for I/O completion instead of waiting for an
> > + interrupt.
> > + Only meaningful for asynchronous I/O, and only if the entire I/O can
> > + be issued as a single ``struct bio``.
> > +
> > + * ``IOCB_DIO_CALLER_COMP``: Try to run I/O completion from the caller's
> > + process context.
> > + See ``linux/fs.h`` for more details.
> > +
> > +Filesystems should call ``iomap_dio_rw`` from ``->read_iter`` and
> > +``->write_iter``, and set ``FMODE_CAN_ODIRECT`` in the ``->open``
> > +function for the file.
> > +They should not set ``->direct_IO``, which is deprecated.
> > +
>
> Return value:
> ~~~~~~~~~~~~~
> On success it will return the number of bytes transferred. On failure it
> will return a negative error value.
>
> Note:
> -ENOTBLK is a magic return value which callers may use for falling back
> to buffered-io. ->iomap_end()/->iomap_begin() can decide to return this
> magic return value if it decides to fallback to buffered-io. iomap
> subsystem return this value in case if it fails to invalidate the
> pagecache pages belonging to the direct-io range before initiating the
> direct-io.
>
> -EIOCBQUEUED is returned when an async direct-io request is queued for I/O.
Done.
> > +If a filesystem wishes to perform its own work before direct I/O
> > +completion, it should call ``__iomap_dio_rw``.
> > +If its return value is not an error pointer or a NULL pointer, the
> > +filesystem should pass the return value to ``iomap_dio_complete`` after
> > +finishing its internal work.
> > +
> > +Direct Reads
> > +~~~~~~~~~~~~
> > +
> > +A direct I/O read initiates a read I/O from the storage device to the
> > +caller's buffer.
> > +Dirty parts of the pagecache are flushed to storage before initiating
> > +the read io.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT`` with
> > +any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Direct Writes
> > +~~~~~~~~~~~~~
> > +
> > +A direct I/O write initiates a write I/O to the storage device to the
> > +caller's buffer.
>
> to the storage device "from" the caller's buffer.
Heh, oooops. Thanks for catching that.
> > +Dirty parts of the pagecache are flushed to storage before initiating
> > +the write io.
> > +The pagecache is invalidated both before and after the write io.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DIRECT |
> > +IOMAP_WRITE`` with any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> > + blocks is not allowed.
> > + The entire file range must to map to a single written or unwritten
> > + extent.
> > + The file I/O range must be aligned to the filesystem block size.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared or exclusive mode.
> > +
> > +struct iomap_dio_ops:
> > +~~~~~~~~~~~~~~~~~~~~~
> > +.. code-block:: c
> > +
> > + struct iomap_dio_ops {
> > + void (*submit_io)(const struct iomap_iter *iter, struct bio *bio,
> > + loff_t file_offset);
> > + int (*end_io)(struct kiocb *iocb, ssize_t size, int error,
> > + unsigned flags);
> > + struct bio_set *bio_set;
> > + };
> > +
> > +The fields of this structure are as follows:
> > +
> > + - ``submit_io``: iomap calls this function when it has constructed a
> > + ``struct bio`` object for the I/O requested, and wishes to submit it
> > + to the block device.
> > + If no function is provided, ``submit_bio`` will be called directly.
> > + Filesystems that would like to perform additional work before (e.g.
> > + data replication for btrfs) should implement this function.
> > +
> > + - ``end_io``: This is called after the ``struct bio`` completes.
> > + This function should perform post-write conversions of unwritten
> > + extent mappings, handle write failures, etc.
> > + The ``flags`` argument may be set to a combination of the following:
> > +
> > + * ``IOMAP_DIO_UNWRITTEN``: The mapping was unwritten, so the ioend
> > + should mark the extent as written.
> > +
> > + * ``IOMAP_DIO_COW``: Writing to the space in the mapping required a
> > + copy on write operation, so the ioend should switch mappings.
> > +
> > + - ``bio_set``: This allows the filesystem to provide a custom bio_set
> > + for allocating direct I/O bios.
> > + This enables filesystems to `stash additional per-bio information
> > + <https://lore.kernel.org/all/[email protected]/>`_
> > + for private use.
> > + If this field is NULL, generic ``struct bio`` objects will be used.
> > +
> > +Filesystems that want to perform extra work after an I/O completion
> > +should set a custom ``->bi_end_io`` function via ``->submit_io``.
> > +Afterwards, the custom endio function must call
> > +``iomap_dio_bio_end_io`` to finish the direct I/O.
> > +
> > +DAX I/O
> > +-------
> > +
> > +Storage devices that can be directly mapped as memory support a new
> > +access mode known as "fsdax".
>
> Added a comma before "support" for better readability.
>
> Storage devices that can be directly mapped as memory, support a new
> access mode known as "fsdax".
Eh, I don't like the comma. Maybe this should say a teensy bit more
about what fsdax even is?
"Some storage devices can be directly mapped as memory.
These devices support a new access mode known as fsdax that allows
loads and stores through the CPU and memory controller."
>
> > +
> > +fsdax Reads
> > +~~~~~~~~~~~
> > +
> > +A fsdax read performs a memcpy from storage device to the caller's
> > +buffer.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX`` with any
> > +combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Read if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +fsdax Writes
> > +~~~~~~~~~~~~
> > +
> > +A fsdax write initiates a memcpy to the storage device to the caller's
>
> "from" the storage device
Fixed, thank you.
> > +buffer.
> > +The ``flags`` value for ``->iomap_begin`` will be ``IOMAP_DAX |
> > +IOMAP_WRITE`` with any combination of the following enhancements:
> > +
> > + * ``IOMAP_NOWAIT``: Write if mapping data are already in memory.
> > + Does not initiate other I/O or block on filesystem locks.
> > +
> > + * ``IOMAP_OVERWRITE_ONLY``: Allocating blocks and zeroing partial
> > + blocks is not allowed.
> > + The entire file range must to map to a single written or unwritten
> > + extent.
> > + The file I/O range must be aligned to the filesystem block size.
> > +
> > +Callers commonly hold ``i_rwsem`` in exclusive mode.
> > +
> > +mmap Faults
> > +~~~~~~~~~~~
> > +
> > +The ``dax_iomap_fault`` function handles read and write faults to fsdax
> > +storage.
> > +For a read fault, ``IOMAP_DAX | IOMAP_FAULT`` will be passed as the
> > +``flags`` argument to ``->iomap_begin``.
> > +For a write fault, ``IOMAP_DAX | IOMAP_FAULT | IOMAP_WRITE`` will be
> > +passed as the ``flags`` argument to ``->iomap_begin``.
> > +
> > +Callers commonly hold the same locks as they do to call their iomap
> > +pagecache counterparts.
> > +
> > +Truncation, fallocate, and Unsharing
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +For fsdax files, the following functions are provided to replace their
> > +iomap pagecache I/O counterparts.
> > +The ``flags`` argument to ``->iomap_begin`` are the same as the
> > +pagecache counterparts, with ``IOMAP_DIO`` added.
>
> with "IOMAP_DAX"
Fixed, thanks.
> > +
> > + * ``dax_file_unshare``
> > + * ``dax_zero_range``
> > + * ``dax_truncate_page``
>
> Shall we mention
> "dax_remap_file_range_prep()/dax_dedupe_file_range_compare()" ?
I think dax_remap_file_range_prep is a rather silly wrapper. But I
suppose filesystems that support dax and reflink need to know about it,
so I'll add a section:
"Filesystems implementing the ``FIDEDUPERANGE`` ioctl must call the
``dax_remap_file_range_prep`` function with their own iomap read ops."
The only caller of dax_dedupe_file_range_compare is the vfs itself, so I
don't think this is worth mentioning.
> > +
> > +Callers commonly hold the same locks as they do to call their iomap
> > +pagecache counterparts.
> > +
>
> Stopping here for now. Will resume the rest of the document review
> later.
Ok. I didn't see this email before my previous reply. You're pretty
close to the end, so I'll respin the series on the list after I see your
next reply. An interim version is here:
https://djwong.org/docs/iomap/
--D
> -ritesh
>
> +SEEK_DATA
> +---------
> +
> +The ``iomap_seek_data`` function implements the SEEK_DATA "whence" value
> +for llseek.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +
> +For unwritten mappings, the pagecache will be searched.
> +Regions of the pagecache with a folio mapped and uptodate fsblocks
> +within those folios will be reported as data areas.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +SEEK_HOLE
> +---------
> +
> +The ``iomap_seek_hole`` function implements the SEEK_HOLE "whence" value
> +for llseek.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +
> +For unwritten mappings, the pagecache will be searched.
> +Regions of the pagecache with no folio mapped, or a !uptodate fsblock
> +within a folio will be reported as sparse hole areas.
> +
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Swap File Activation
> +--------------------
> +
> +The ``iomap_swapfile_activate`` function finds all the base-page aligned
> +regions in a file and sets them up as swap space.
> +The file will be ``fsync()``'d before activation.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +All mappings must be mapped or unwritten; cannot be dirty or shared, and
> +cannot span multiple block devices.
> +Callers must hold ``i_rwsem`` in exclusive mode; this is already
> +provided by ``swapon``.
> +
> +Extent Map Reporting (FS_IOC_FIEMAP)
> +------------------------------------
> +
> +The ``iomap_fiemap`` function exports file extent mappings to userspace
> +in the format specified by the ``FS_IOC_FIEMAP`` ioctl.
> +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> +``->iomap_begin``.
> +Callers commonly hold ``i_rwsem`` in shared mode.
> +
> +Block Map Reporting (FIBMAP)
> +----------------------------
> +
> +``iomap_bmap`` implements FIBMAP.
> +The calling conventions are the same as for FIEMAP.
> +This function is only provided to maintain compatibility for filesystems
> +that implemented FIBMAP prior to conversion.
> +This ioctl is deprecated; do not add a FIBMAP implementation to
> +filesystems that do not have it.
> +Callers should probably hold ``i_rwsem`` in shared mode, but this is
> +unclear.
looking at fiemap callers is also confusing w.r.t i_rwsem ;)
> +
> +Porting Guide
> +=============
> +
> +Why Convert to iomap?
> +---------------------
> +
> +There are several reasons to convert a filesystem to iomap:
> +
> + 1. The classic Linux I/O path is not terribly efficient.
> + Pagecache operations lock a single base page at a time and then call
> + into the filesystem to return a mapping for only that page.
> + Direct I/O operations build I/O requests a single file block at a
> + time.
> + This worked well enough for direct/indirect-mapped filesystems such
> + as ext2, but is very inefficient for extent-based filesystems such
> + as XFS.
> +
> + 2. Large folios are only supported via iomap; there are no plans to
> + convert the old buffer_head path to use them.
> +
> + 3. Direct access to storage on memory-like devices (fsdax) is only
> + supported via iomap.
> +
> + 4. Lower maintenance overhead for individual filesystem maintainers.
> + iomap handles common pagecache related operations itself, such as
> + allocating, instantiating, locking, and unlocking of folios.
> + No ->write_begin(), ->write_end() or direct_IO
> + address_space_operations are required to be implemented by
> + filesystem using iomap.
> +
> +How to Convert to iomap?
> +------------------------
> +
> +First, add ``#include <linux/iomap.h>`` from your source code and add
> +``select FS_IOMAP`` to your filesystem's Kconfig option.
> +Build the kernel, run fstests with the ``-g all`` option across a wide
> +variety of your filesystem's supported configurations to build a
> +baseline of which tests pass and which ones fail.
> +
> +The recommended approach is first to implement ``->iomap_begin`` (and
> +``->iomap->end`` if necessary) to allow iomap to obtain a read-only
small correction: ``->iomap_end``
> +mapping of a file range.
> +In most cases, this is a relatively trivial conversion of the existing
> +``get_block()`` function for read-only mappings.
> +``FS_IOC_FIEMAP`` is a good first target because it is trivial to
> +implement support for it and then to determine that the extent map
> +iteration is correct from userspace.
> +If FIEMAP is returning the correct information, it's a good sign that
> +other read-only mapping operations will do the right thing.
> +
> +Next, modify the filesystem's ``get_block(create = false)``
> +implementation to use the new ``->iomap_begin`` implementation to map
> +file space for selected read operations.
> +Hide behind a debugging knob the ability to switch on the iomap mapping
> +functions for selected call paths.
> +It is necessary to write some code to fill out the bufferhead-based
> +mapping information from the ``iomap`` structure, but the new functions
> +can be tested without needing to implement any iomap APIs.
> +
> +Once the read-only functions are working like this, convert each high
> +level file operation one by one to use iomap native APIs instead of
> +going through ``get_block()``.
> +Done one at a time, regressions should be self evident.
> +You *do* have a regression test baseline for fstests, right?
> +It is suggested to convert swap file activation, ``SEEK_DATA``, and
> +``SEEK_HOLE`` before tackling the I/O paths.
> +A likely complexity at this point will be converting the buffered read
> +I/O path because of bufferheads.
> +The buffered read I/O paths doesn't need to be converted yet, though the
> +direct I/O read path should be converted in this phase.
> +
> +At this point, you should look over your ``->iomap_begin`` function.
> +If it switches between large blocks of code based on dispatching of the
> +``flags`` argument, you should consider breaking it up into
> +per-operation iomap ops with smaller, more cohesive functions.
> +XFS is a good example of this.
> +
> +The next thing to do is implement ``get_blocks(create == true)``
> +functionality in the ``->iomap_begin``/``->iomap_end`` methods.
> +It is strongly recommended to create separate mapping functions and
> +iomap ops for write operations.
> +Then convert the direct I/O write path to iomap, and start running fsx
> +w/ DIO enabled in earnest on filesystem.
> +This will flush out lots of data integrity corner case bugs that the new
> +write mapping implementation introduces.
> +
> +Now, convert any remaining file operations to call the iomap functions.
> +This will get the entire filesystem using the new mapping functions, and
> +they should largely be debugged and working correctly after this step.
> +
> +Most likely at this point, the buffered read and write paths will still
> +to be converted.
> +The mapping functions should all work correctly, so all that needs to be
> +done is rewriting all the code that interfaces with bufferheads to
> +interface with iomap and folios.
> +It is much easier first to get regular file I/O (without any fancy
> +features like fscrypt, fsverity, compression, or data=journaling)
> +converted to use iomap.
> +Some of those fancy features (fscrypt and compression) aren't
> +implemented yet in iomap.
> +For unjournalled filesystems that use the pagecache for symbolic links
> +and directories, you might also try converting their handling to iomap.
> +
> +The rest is left as an exercise for the reader, as it will be different
> +for every filesystem.
> +If you encounter problems, email the people and lists in
> +``get_maintainers.pl`` for help.
> +
> +Bugs and Limitations
> +====================
> +
> + * No support for fscrypt.
> + * No support for compression.
> + * No support for fsverity yet.
> + * Strong assumptions that IO should work the way it does on XFS.
> + * Does iomap *actually* work for non-regular file data?
> +
> +Patches welcome!
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 8754ac2c259d..2ddd94d43ecf 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8483,6 +8483,7 @@ R: Darrick J. Wong <[email protected]>
> L: [email protected]
> L: [email protected]
> S: Supported
> +F: Documentation/filesystems/iomap.txt
> F: fs/iomap/
> F: include/linux/iomap.h
>
Rest looks good to me.
-ritesh
On Fri, Jun 14, 2024 at 08:31:55PM +0530, Ritesh Harjani wrote:
>
> > +SEEK_DATA
> > +---------
> > +
> > +The ``iomap_seek_data`` function implements the SEEK_DATA "whence" value
> > +for llseek.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +
> > +For unwritten mappings, the pagecache will be searched.
> > +Regions of the pagecache with a folio mapped and uptodate fsblocks
> > +within those folios will be reported as data areas.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +SEEK_HOLE
> > +---------
> > +
> > +The ``iomap_seek_hole`` function implements the SEEK_HOLE "whence" value
> > +for llseek.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +
> > +For unwritten mappings, the pagecache will be searched.
> > +Regions of the pagecache with no folio mapped, or a !uptodate fsblock
> > +within a folio will be reported as sparse hole areas.
> > +
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Swap File Activation
> > +--------------------
> > +
> > +The ``iomap_swapfile_activate`` function finds all the base-page aligned
> > +regions in a file and sets them up as swap space.
> > +The file will be ``fsync()``'d before activation.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +All mappings must be mapped or unwritten; cannot be dirty or shared, and
> > +cannot span multiple block devices.
> > +Callers must hold ``i_rwsem`` in exclusive mode; this is already
> > +provided by ``swapon``.
> > +
> > +Extent Map Reporting (FS_IOC_FIEMAP)
> > +------------------------------------
> > +
> > +The ``iomap_fiemap`` function exports file extent mappings to userspace
> > +in the format specified by the ``FS_IOC_FIEMAP`` ioctl.
> > +``IOMAP_REPORT`` will be passed as the ``flags`` argument to
> > +``->iomap_begin``.
> > +Callers commonly hold ``i_rwsem`` in shared mode.
> > +
> > +Block Map Reporting (FIBMAP)
> > +----------------------------
> > +
> > +``iomap_bmap`` implements FIBMAP.
> > +The calling conventions are the same as for FIEMAP.
> > +This function is only provided to maintain compatibility for filesystems
> > +that implemented FIBMAP prior to conversion.
> > +This ioctl is deprecated; do not add a FIBMAP implementation to
> > +filesystems that do not have it.
> > +Callers should probably hold ``i_rwsem`` in shared mode, but this is
> > +unclear.
>
> looking at fiemap callers is also confusing w.r.t i_rwsem ;)
Yes indeed -- if the FIEMAP code does not access any state that's
protected only by i_rwsem then I guess you don't need it? AFAICT that's
the case for xfs.
> > +
> > +Porting Guide
> > +=============
> > +
> > +Why Convert to iomap?
> > +---------------------
> > +
> > +There are several reasons to convert a filesystem to iomap:
> > +
> > + 1. The classic Linux I/O path is not terribly efficient.
> > + Pagecache operations lock a single base page at a time and then call
> > + into the filesystem to return a mapping for only that page.
> > + Direct I/O operations build I/O requests a single file block at a
> > + time.
> > + This worked well enough for direct/indirect-mapped filesystems such
> > + as ext2, but is very inefficient for extent-based filesystems such
> > + as XFS.
> > +
> > + 2. Large folios are only supported via iomap; there are no plans to
> > + convert the old buffer_head path to use them.
> > +
> > + 3. Direct access to storage on memory-like devices (fsdax) is only
> > + supported via iomap.
> > +
> > + 4. Lower maintenance overhead for individual filesystem maintainers.
> > + iomap handles common pagecache related operations itself, such as
> > + allocating, instantiating, locking, and unlocking of folios.
> > + No ->write_begin(), ->write_end() or direct_IO
> > + address_space_operations are required to be implemented by
> > + filesystem using iomap.
> > +
> > +How to Convert to iomap?
> > +------------------------
> > +
> > +First, add ``#include <linux/iomap.h>`` from your source code and add
> > +``select FS_IOMAP`` to your filesystem's Kconfig option.
> > +Build the kernel, run fstests with the ``-g all`` option across a wide
> > +variety of your filesystem's supported configurations to build a
> > +baseline of which tests pass and which ones fail.
> > +
> > +The recommended approach is first to implement ``->iomap_begin`` (and
> > +``->iomap->end`` if necessary) to allow iomap to obtain a read-only
>
> small correction: ``->iomap_end``
Fixed, thanks.
> > +mapping of a file range.
> > +In most cases, this is a relatively trivial conversion of the existing
> > +``get_block()`` function for read-only mappings.
> > +``FS_IOC_FIEMAP`` is a good first target because it is trivial to
> > +implement support for it and then to determine that the extent map
> > +iteration is correct from userspace.
> > +If FIEMAP is returning the correct information, it's a good sign that
> > +other read-only mapping operations will do the right thing.
> > +
> > +Next, modify the filesystem's ``get_block(create = false)``
> > +implementation to use the new ``->iomap_begin`` implementation to map
> > +file space for selected read operations.
> > +Hide behind a debugging knob the ability to switch on the iomap mapping
> > +functions for selected call paths.
> > +It is necessary to write some code to fill out the bufferhead-based
> > +mapping information from the ``iomap`` structure, but the new functions
> > +can be tested without needing to implement any iomap APIs.
> > +
> > +Once the read-only functions are working like this, convert each high
> > +level file operation one by one to use iomap native APIs instead of
> > +going through ``get_block()``.
> > +Done one at a time, regressions should be self evident.
> > +You *do* have a regression test baseline for fstests, right?
> > +It is suggested to convert swap file activation, ``SEEK_DATA``, and
> > +``SEEK_HOLE`` before tackling the I/O paths.
> > +A likely complexity at this point will be converting the buffered read
> > +I/O path because of bufferheads.
> > +The buffered read I/O paths doesn't need to be converted yet, though the
> > +direct I/O read path should be converted in this phase.
> > +
> > +At this point, you should look over your ``->iomap_begin`` function.
> > +If it switches between large blocks of code based on dispatching of the
> > +``flags`` argument, you should consider breaking it up into
> > +per-operation iomap ops with smaller, more cohesive functions.
> > +XFS is a good example of this.
> > +
> > +The next thing to do is implement ``get_blocks(create == true)``
> > +functionality in the ``->iomap_begin``/``->iomap_end`` methods.
> > +It is strongly recommended to create separate mapping functions and
> > +iomap ops for write operations.
> > +Then convert the direct I/O write path to iomap, and start running fsx
> > +w/ DIO enabled in earnest on filesystem.
> > +This will flush out lots of data integrity corner case bugs that the new
> > +write mapping implementation introduces.
> > +
> > +Now, convert any remaining file operations to call the iomap functions.
> > +This will get the entire filesystem using the new mapping functions, and
> > +they should largely be debugged and working correctly after this step.
> > +
> > +Most likely at this point, the buffered read and write paths will still
> > +to be converted.
> > +The mapping functions should all work correctly, so all that needs to be
> > +done is rewriting all the code that interfaces with bufferheads to
> > +interface with iomap and folios.
> > +It is much easier first to get regular file I/O (without any fancy
> > +features like fscrypt, fsverity, compression, or data=journaling)
> > +converted to use iomap.
> > +Some of those fancy features (fscrypt and compression) aren't
> > +implemented yet in iomap.
> > +For unjournalled filesystems that use the pagecache for symbolic links
> > +and directories, you might also try converting their handling to iomap.
> > +
> > +The rest is left as an exercise for the reader, as it will be different
> > +for every filesystem.
> > +If you encounter problems, email the people and lists in
> > +``get_maintainers.pl`` for help.
> > +
> > +Bugs and Limitations
> > +====================
> > +
> > + * No support for fscrypt.
> > + * No support for compression.
> > + * No support for fsverity yet.
> > + * Strong assumptions that IO should work the way it does on XFS.
> > + * Does iomap *actually* work for non-regular file data?
> > +
> > +Patches welcome!
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 8754ac2c259d..2ddd94d43ecf 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -8483,6 +8483,7 @@ R: Darrick J. Wong <[email protected]>
> > L: [email protected]
> > L: [email protected]
> > S: Supported
> > +F: Documentation/filesystems/iomap.txt
> > F: fs/iomap/
> > F: include/linux/iomap.h
> >
>
> Rest looks good to me.
Yay, thanks for giving feedback on the whole thing!
--D
> -ritesh
>