Provide a function, readahead_expand(), that expands the set of pages
specified by a readahead_control object to encompass a revised area with a
proposed size and length.
The proposed area must include all of the old area and may be expanded yet
more by this function so that the edges align on (transparent huge) page
boundaries as allocated.
The expansion will be cut short if a page already exists in either of the
areas being expanded into. Note that any expansion made in such a case is
not rolled back.
This will be used by fscache so that reads can be expanded to cache granule
boundaries, thereby allowing whole granules to be stored in the cache, but
there are other potential users also.
Changes:
v6:
- Fold in a patch from Matthew Wilcox to tell the ondemand readahead
algorithm about the expansion so that the next readahead starts at the
right place[2].
v4:
- Moved the declaration of readahead_expand() to a better place[1].
Suggested-by: Matthew Wilcox (Oracle) <[email protected]>
Signed-off-by: David Howells <[email protected]>
cc: Matthew Wilcox (Oracle) <[email protected]>
cc: Alexander Viro <[email protected]>
cc: Christoph Hellwig <[email protected]>
cc: Mike Marshall <[email protected]>
cc: [email protected]
cc: [email protected]
cc: [email protected]
cc: [email protected]
cc: [email protected]
cc: [email protected]
cc: [email protected]
cc: [email protected]
Link: https://lore.kernel.org/r/[email protected]/ [1]
Link: https://lore.kernel.org/r/159974633888.2094769.8326206446358128373.stgit@warthog.procyon.org.uk/
Link: https://lore.kernel.org/r/160588479816.3465195.553952688795241765.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/161118131787.1232039.4863969952441067985.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/161161028670.2537118.13831420617039766044.stgit@warthog.procyon.org.uk/ # v2
Link: https://lore.kernel.org/r/161340389201.1303470.14353807284546854878.stgit@warthog.procyon.org.uk/ # v3
Link: https://lore.kernel.org/r/161539530488.286939.18085961677838089157.stgit@warthog.procyon.org.uk/ # v4
Link: https://lore.kernel.org/r/161653789422.2770958.2108046612147345000.stgit@warthog.procyon.org.uk/ # v5
Link: https://lore.kernel.org/r/[email protected]/ [2]
---
include/linux/pagemap.h | 2 +
mm/readahead.c | 75 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 77 insertions(+)
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 9a9e558ce4c7..ef511364cc0c 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -838,6 +838,8 @@ void page_cache_ra_unbounded(struct readahead_control *,
void page_cache_sync_ra(struct readahead_control *, unsigned long req_count);
void page_cache_async_ra(struct readahead_control *, struct page *,
unsigned long req_count);
+void readahead_expand(struct readahead_control *ractl,
+ loff_t new_start, size_t new_len);
/**
* page_cache_sync_readahead - generic file readahead
diff --git a/mm/readahead.c b/mm/readahead.c
index 2088569a947e..f02dbebf1cef 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -638,3 +638,78 @@ SYSCALL_DEFINE3(readahead, int, fd, loff_t, offset, size_t, count)
{
return ksys_readahead(fd, offset, count);
}
+
+/**
+ * readahead_expand - Expand a readahead request
+ * @ractl: The request to be expanded
+ * @new_start: The revised start
+ * @new_len: The revised size of the request
+ *
+ * Attempt to expand a readahead request outwards from the current size to the
+ * specified size by inserting locked pages before and after the current window
+ * to increase the size to the new window. This may involve the insertion of
+ * THPs, in which case the window may get expanded even beyond what was
+ * requested.
+ *
+ * The algorithm will stop if it encounters a conflicting page already in the
+ * pagecache and leave a smaller expansion than requested.
+ *
+ * The caller must check for this by examining the revised @ractl object for a
+ * different expansion than was requested.
+ */
+void readahead_expand(struct readahead_control *ractl,
+ loff_t new_start, size_t new_len)
+{
+ struct address_space *mapping = ractl->mapping;
+ struct file_ra_state *ra = ractl->ra;
+ pgoff_t new_index, new_nr_pages;
+ gfp_t gfp_mask = readahead_gfp_mask(mapping);
+
+ new_index = new_start / PAGE_SIZE;
+
+ /* Expand the leading edge downwards */
+ while (ractl->_index > new_index) {
+ unsigned long index = ractl->_index - 1;
+ struct page *page = xa_load(&mapping->i_pages, index);
+
+ if (page && !xa_is_value(page))
+ return; /* Page apparently present */
+
+ page = __page_cache_alloc(gfp_mask);
+ if (!page)
+ return;
+ if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) {
+ put_page(page);
+ return;
+ }
+
+ ractl->_nr_pages++;
+ ractl->_index = page->index;
+ }
+
+ new_len += new_start - readahead_pos(ractl);
+ new_nr_pages = DIV_ROUND_UP(new_len, PAGE_SIZE);
+
+ /* Expand the trailing edge upwards */
+ while (ractl->_nr_pages < new_nr_pages) {
+ unsigned long index = ractl->_index + ractl->_nr_pages;
+ struct page *page = xa_load(&mapping->i_pages, index);
+
+ if (page && !xa_is_value(page))
+ return; /* Page apparently present */
+
+ page = __page_cache_alloc(gfp_mask);
+ if (!page)
+ return;
+ if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) {
+ put_page(page);
+ return;
+ }
+ ractl->_nr_pages++;
+ if (ra) {
+ ra->size++;
+ ra->async_size++;
+ }
+ }
+}
+EXPORT_SYMBOL(readahead_expand);
On Thu, Apr 08, 2021 at 03:04:58PM +0100, David Howells wrote:
> Suggested-by: Matthew Wilcox (Oracle) <[email protected]>
> Signed-off-by: David Howells <[email protected]>
Reviewed-by: Matthew Wilcox (Oracle) <[email protected]>