On Sat, Jul 8, 2023 at 1:39 AM Hyeonggon Yoo <[email protected]> wrote:
>
> On Wed, Jun 28, 2023 at 11:48:52AM +0100, David Howells wrote:
> > Fscache has an optimisation by which reads from the cache are skipped until
> > we know that (a) there's data there to be read and (b) that data isn't
> > entirely covered by pages resident in the netfs pagecache. This is done
> > with two flags manipulated by fscache_note_page_release():
> >
> > if (...
> > test_bit(FSCACHE_COOKIE_HAVE_DATA, &cookie->flags) &&
> > test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags))
> > clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
> >
> > where the NO_DATA_TO_READ flag causes cachefiles_prepare_read() to indicate
> > that netfslib should download from the server or clear the page instead.
> >
> > The fscache_note_page_release() function is intended to be called from
> > ->releasepage() - but that only gets called if PG_private or PG_private_2
> > is set - and currently the former is at the discretion of the network
> > filesystem and the latter is only set whilst a page is being written to the
> > cache, so sometimes we miss clearing the optimisation.
> >
> > Fix this by following Willy's suggestion[1] and adding an address_space
> > flag, AS_RELEASE_ALWAYS, that causes filemap_release_folio() to always call
> > ->release_folio() if it's set, even if PG_private or PG_private_2 aren't
> > set.
> >
> > Note that this would require folio_test_private() and page_has_private() to
> > become more complicated. To avoid that, in the places[*] where these are
> > used to conditionalise calls to filemap_release_folio() and
> > try_to_release_page(), the tests are removed the those functions just
> > jumped to unconditionally and the test is performed there.
> >
> > [*] There are some exceptions in vmscan.c where the check guards more than
> > just a call to the releaser. I've added a function, folio_needs_release()
> > to wrap all the checks for that.
> >
> > AS_RELEASE_ALWAYS should be set if a non-NULL cookie is obtained from
> > fscache and cleared in ->evict_inode() before truncate_inode_pages_final()
> > is called.
> >
> > Additionally, the FSCACHE_COOKIE_NO_DATA_TO_READ flag needs to be cleared
> > and the optimisation cancelled if a cachefiles object already contains data
> > when we open it.
> >
> > Fixes: 1f67e6d0b188 ("fscache: Provide a function to note the release of a page")
> > Fixes: 047487c947e8 ("cachefiles: Implement the I/O routines")
> > Reported-by: Rohith Surabattula <[email protected]>
> > Suggested-by: Matthew Wilcox <[email protected]>
> > Signed-off-by: David Howells <[email protected]>
>
> Hi David,
>
> I was bisecting a use-after-free BUG on the latest mm-unstable,
> where HEAD is 347e208de0e4 ("rmap: pass the folio to __page_check_anon_rmap()").
>
> According to my bisection, this is the first bad commit.
> Use-After-Free is triggered on reclamation path when swap is enabled.
This was originally occurred during kernel compilation but
can easily be reproduced via:
stress-ng --bigheap $(nproc)
> (and couldn't trigger without swap enabled)
>
> the config, KASAN splat, bisect log are attached.
> hope this isn't too late :(
>
> > cc: Matthew Wilcox <[email protected]>
> > cc: Linus Torvalds <[email protected]>
> > cc: Steve French <[email protected]>
> > cc: Shyam Prasad N <[email protected]>
> > cc: Rohith Surabattula <[email protected]>
> > cc: Dave Wysochanski <[email protected]>
> > cc: Dominique Martinet <[email protected]>
> > cc: Ilya Dryomov <[email protected]>
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > cc: [email protected]
> > ---
> >
> > Notes:
> > ver #7)
> > - Make NFS set AS_RELEASE_ALWAYS.
> >
> > ver #4)
> > - Split out merging of folio_has_private()/filemap_release_folio() call
> > pairs into a preceding patch.
> > - Don't need to clear AS_RELEASE_ALWAYS in ->evict_inode().
> >
> > ver #3)
> > - Fixed mapping_clear_release_always() to use clear_bit() not set_bit().
> > - Moved a '&&' to the correct line.
> >
> > ver #2)
> > - Rewrote entirely according to Willy's suggestion[1].
> >
> > fs/9p/cache.c | 2 ++
> > fs/afs/internal.h | 2 ++
> > fs/cachefiles/namei.c | 2 ++
> > fs/ceph/cache.c | 2 ++
> > fs/nfs/fscache.c | 3 +++
> > fs/smb/client/fscache.c | 2 ++
> > include/linux/pagemap.h | 16 ++++++++++++++++
> > mm/internal.h | 5 ++++-
> > 8 files changed, 33 insertions(+), 1 deletion(-)
On Fri, Jul 7, 2023 at 12:46 PM Hyeonggon Yoo <[email protected]> wrote:
>
> On Sat, Jul 8, 2023 at 1:39 AM Hyeonggon Yoo <[email protected]> wrote:
> >
> > On Wed, Jun 28, 2023 at 11:48:52AM +0100, David Howells wrote:
> > > Fscache has an optimisation by which reads from the cache are skipped until
> > > we know that (a) there's data there to be read and (b) that data isn't
> > > entirely covered by pages resident in the netfs pagecache. This is done
> > > with two flags manipulated by fscache_note_page_release():
> > >
> > > if (...
> > > test_bit(FSCACHE_COOKIE_HAVE_DATA, &cookie->flags) &&
> > > test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags))
> > > clear_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags);
> > >
> > > where the NO_DATA_TO_READ flag causes cachefiles_prepare_read() to indicate
> > > that netfslib should download from the server or clear the page instead.
> > >
> > > The fscache_note_page_release() function is intended to be called from
> > > ->releasepage() - but that only gets called if PG_private or PG_private_2
> > > is set - and currently the former is at the discretion of the network
> > > filesystem and the latter is only set whilst a page is being written to the
> > > cache, so sometimes we miss clearing the optimisation.
> > >
> > > Fix this by following Willy's suggestion[1] and adding an address_space
> > > flag, AS_RELEASE_ALWAYS, that causes filemap_release_folio() to always call
> > > ->release_folio() if it's set, even if PG_private or PG_private_2 aren't
> > > set.
> > >
> > > Note that this would require folio_test_private() and page_has_private() to
> > > become more complicated. To avoid that, in the places[*] where these are
> > > used to conditionalise calls to filemap_release_folio() and
> > > try_to_release_page(), the tests are removed the those functions just
> > > jumped to unconditionally and the test is performed there.
> > >
> > > [*] There are some exceptions in vmscan.c where the check guards more than
> > > just a call to the releaser. I've added a function, folio_needs_release()
> > > to wrap all the checks for that.
> > >
> > > AS_RELEASE_ALWAYS should be set if a non-NULL cookie is obtained from
> > > fscache and cleared in ->evict_inode() before truncate_inode_pages_final()
> > > is called.
> > >
> > > Additionally, the FSCACHE_COOKIE_NO_DATA_TO_READ flag needs to be cleared
> > > and the optimisation cancelled if a cachefiles object already contains data
> > > when we open it.
> > >
> > > Fixes: 1f67e6d0b188 ("fscache: Provide a function to note the release of a page")
> > > Fixes: 047487c947e8 ("cachefiles: Implement the I/O routines")
> > > Reported-by: Rohith Surabattula <[email protected]>
> > > Suggested-by: Matthew Wilcox <[email protected]>
> > > Signed-off-by: David Howells <[email protected]>
> >
> > Hi David,
> >
> > I was bisecting a use-after-free BUG on the latest mm-unstable,
> > where HEAD is 347e208de0e4 ("rmap: pass the folio to __page_check_anon_rmap()").
> >
> > According to my bisection, this is the first bad commit.
> > Use-After-Free is triggered on reclamation path when swap is enabled.
>
> This was originally occurred during kernel compilation but
> can easily be reproduced via:
>
> stress-ng --bigheap $(nproc)
>
> > (and couldn't trigger without swap enabled)
> >
> > the config, KASAN splat, bisect log are attached.
> > hope this isn't too late :(
> >
> > > cc: Matthew Wilcox <[email protected]>
> > > cc: Linus Torvalds <[email protected]>
> > > cc: Steve French <[email protected]>
> > > cc: Shyam Prasad N <[email protected]>
> > > cc: Rohith Surabattula <[email protected]>
> > > cc: Dave Wysochanski <[email protected]>
> > > cc: Dominique Martinet <[email protected]>
> > > cc: Ilya Dryomov <[email protected]>
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > cc: [email protected]
> > > ---
> > >
> > > Notes:
> > > ver #7)
> > > - Make NFS set AS_RELEASE_ALWAYS.
> > >
> > > ver #4)
> > > - Split out merging of folio_has_private()/filemap_release_folio() call
> > > pairs into a preceding patch.
> > > - Don't need to clear AS_RELEASE_ALWAYS in ->evict_inode().
> > >
> > > ver #3)
> > > - Fixed mapping_clear_release_always() to use clear_bit() not set_bit().
> > > - Moved a '&&' to the correct line.
> > >
> > > ver #2)
> > > - Rewrote entirely according to Willy's suggestion[1].
> > >
> > > fs/9p/cache.c | 2 ++
> > > fs/afs/internal.h | 2 ++
> > > fs/cachefiles/namei.c | 2 ++
> > > fs/ceph/cache.c | 2 ++
> > > fs/nfs/fscache.c | 3 +++
> > > fs/smb/client/fscache.c | 2 ++
> > > include/linux/pagemap.h | 16 ++++++++++++++++
> > > mm/internal.h | 5 ++++-
> > > 8 files changed, 33 insertions(+), 1 deletion(-)
I think myself / Daire Byrne may have already tracked this down and I
found a 1-liner that fixed a similar crash in his environment.
Can you try this patch on top and let me know if it still crashes?
https://github.com/DaveWysochanskiRH/kernel/commit/902c990e311120179fa5de99d68364b2947b79ec
On Fri, Jul 07, 2023 at 02:12:06PM -0400, David Wysochanski wrote:
> I think myself / Daire Byrne may have already tracked this down and I
> found a 1-liner that fixed a similar crash in his environment.
>
> Can you try this patch on top and let me know if it still crashes?
> https://github.com/DaveWysochanskiRH/kernel/commit/902c990e311120179fa5de99d68364b2947b79ec
Said one-liner:
- struct address_space *mapping = folio->mapping;
+ struct address_space *mapping = folio_mapping(folio);
This will definitely fix the problem. shrink_folio_list() sees
anonymous folios as well as file folios.
I wonder if we want to go a step further and introduce ...
+static inline bool __folio_needs_release(struct address_space *mapping,
+ struct folio *folio)
+{
+ return folio_has_private(folio) ||
+ (mapping && mapping_release_always(mapping));
+}
+
/*
* Return true if a folio needs ->release_folio() calling upon it.
*/
static inline bool folio_needs_release(struct folio *folio)
{
- struct address_space *mapping = folio->mapping;
-
- return folio_has_private(folio) ||
- (mapping && mapping_release_always(mapping));
+ return __folio_needs_release(folio_mapping(folio), folio);
}
since two of the three callers already have done the necessary dance to
get the mapping (and they're the two which happen regularly; the third
is an unusual situation).