Received-SPF: pass (google.com: domain of linux-kernel+bounces-76404-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Thu, 22 Feb 2024 06:27:14 -0500
From: Kent Overstreet <kent.overstreet@linux.dev>
To: Jan Kara <jack@suse.cz>
Cc: Miklos Szeredi <miklos@szeredi.hu>, Josef Bacik <josef@toxicpanda.com>, 
	linux-kernel@vger.kernel.org, linux-bcachefs@vger.kernel.org, linux-fsdevel@vger.kernel.org, 
	lsf-pc@lists.linux-foundation.org, linux-btrfs@vger.kernel.org
Subject: Re: [Lsf-pc] [LSF TOPIC] statx extensions for subvol/snapshot
 filesystems & more
Message-ID: <u4btyvpsohnf77r5rm43tz426u3advjc4goea4obt2wtv6xyog@7bukhgoyumed>
References: <2uvhm6gweyl7iyyp2xpfryvcu2g3padagaeqcbiavjyiis6prl@yjm725bizncq>
 <CAJfpeguBzbhdcknLG4CjFr12_PdGo460FSRONzsYBKmT9uaSMA@mail.gmail.com>
 <20240221210811.GA1161565@perftesting>
 <CAJfpegucM5R_pi_EeDkg9yPNTj_esWYrFd6vG178_asram0=Ew@mail.gmail.com>
 <w534uujga5pqcbhbc5wad7bdt5lchxu6gcmwvkg6tdnkhnkujs@wjqrhv5uqxyx>
 <20240222110138.ckai4sxiin3a74ku@quack3>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240222110138.ckai4sxiin3a74ku@quack3>

On Thu, Feb 22, 2024 at 12:01:38PM +0100, Jan Kara wrote:
> On Thu 22-02-24 04:42:07, Kent Overstreet wrote:
> > On Thu, Feb 22, 2024 at 10:14:20AM +0100, Miklos Szeredi wrote:
> > > On Wed, 21 Feb 2024 at 22:08, Josef Bacik <josef@toxicpanda.com> wrote:
> > > >
> > > > On Wed, Feb 21, 2024 at 04:06:34PM +0100, Miklos Szeredi wrote:
> > > > > On Wed, 21 Feb 2024 at 01:51, Kent Overstreet <kent.overstreet@linux.dev> wrote:
> > > > > >
> > > > > > Recently we had a pretty long discussion on statx extensions, which
> > > > > > eventually got a bit offtopic but nevertheless hashed out all the major
> > > > > > issues.
> > > > > >
> > > > > > To summarize:
> > > > > >  - guaranteeing inode number uniqueness is becoming increasingly
> > > > > >    infeasible, we need a bit to tell userspace "inode number is not
> > > > > >    unique, use filehandle instead"
> > > > >
> > > > > This is a tough one.   POSIX says "The st_ino and st_dev fields taken
> > > > > together uniquely identify the file within the system."
> > > > >
> > > >
> > > > Which is what btrfs has done forever, and we've gotten yelled at forever for
> > > > doing it.  We have a compromise and a way forward, but it's not a widely held
> > > > view that changing st_dev to give uniqueness is an acceptable solution.  It may
> > > > have been for overlayfs because you guys are already doing something special,
> > > > but it's not an option that is afforded the rest of us.
> > > 
> > > Overlayfs tries hard not to use st_dev to give uniqueness and instead
> > > partitions the 64bit st_ino space within the same st_dev.  There are
> > > various fallback cases, some involve switching st_dev and some using
> > > non-persistent st_ino.
> > 
> > Yeah no, you can't crap multiple 64 bit inode number spaces into 64
> > bits: pigeonhole principle.
> > 
> > We need something better than "hacks".
> 
> I agree we should have a better long-term plan than finding ways how to
> cram things into 64-bits inos. However I don't see a realistic short-term
> solution other than that.
> 
> To explicit: Currently, tar and patch and very likely other less well-known
> tools are broken on bcachefs due to non-unique inode numbers. If you want
> ot fix them, either you find ways how bcachefs can cram things into 64-bit
> ino_t or you go and modify these tools (or prod maintainers or whatever) to
> not depend on ino_t for uniqueness. The application side of things isn't
> going to magically fix itself by us telling "bad luck, ino_t isn't unique
> anymore".

My intent is to make a real effort towards getting better interfaces
going, prod those maintainers, _then_ look at adding those hacks (that
will necessarily be short term solutions since 64 bits is already
looking cramped).

Kind of seems to me like you guys think kicking the can down the road
is supposed to to be the first priority, and that's not what I'm doing
here.

> > > What overlayfs does may or may not be applicable to btrfs/bcachefs,
> > > but that's not my point.  My point is that adding a flag to statx does
> > > not solve anything.   You can't just say that from now on btrfs
> > > doesn't have use unique st_ino/st_dev because we've just indicated
> > > that in statx and everything is fine.   That will trigger the
> > > no-regressions rule and then it's game over.  At least I would expect
> > > that to happen.
> > > 
> > > What we can do instead is introduce a new API that is better,
> > 
> > This isn't a serious proposal.
> 
> I think for "unique inode identifier" we don't even have to come up with
> new APIs. The file handle + fsid pair is an established way to do this,
> fanotify successfully uses this as object identifier and Amir did quite
> some work for this to be usable for vast majority of filesystems (including
> virtual ones). The problem is with rewriting all these applications to use
> it. If statx flag telling whether inode numbers are unique helps with that,
> sure we can add it, but that seems like a trivial part of the problem.

Basically, it turns it into a checklist item for userspace. "Are we
still doing things the old way (and we've got a documented list of where
that will break", or "are we handling the INO_NOT_UNIQUE case
correctly".

In general when semantics change, APIs ought to change.