I've started to use the new quota implementation (3.7.3 running now).
I used tune2fs -Q to turn non {usr,grp}quota. It's nice, no need to
run quotaon and all the jquota=file options. It works but it starts
counting only what comes after the mount with the new quota. If there
is already stuff in the filesystem it's not counted. How can we do a
quotacheck?
Also, what happens if the machine crashes without umounting? How is
the usage going to be handled? Is it in the journal? Anyway, it'd also
be important to do a quotacheck.
Is it possible?
On Sat, Jan 19, 2013 at 12:40:35AM -0200, Carlos Carvalho wrote:
> I've started to use the new quota implementation (3.7.3 running now).
> I used tune2fs -Q to turn non {usr,grp}quota. It's nice, no need to
> run quotaon and all the jquota=file options. It works but it starts
> counting only what comes after the mount with the new quota. If there
> is already stuff in the filesystem it's not counted. How can we do a
> quotacheck?
tune2fs -Q should have set up the quota inodes with the correct usage
information. What version of e2fsprogs were you using? If the quota
information is incorrect, e2fsck will correct it --- so e2fsck will
serve as the quota check.
That being said, I've been doing some testing, and I have found some
shortcomings with the e2fsck's handling of the quota option that we
need to fix before I can recommend people use the new quota
implementation.
1) The e2fsck check of the on-disk quota inodes won't notice if there
is a missing uid record. (i.e., if some uid, say daemon owns a bunch
of files, but that uid record is not in the quota inode, e2fsck won't
say boo.)
2) If e2fsck *does* notice a discrepancy between the usage information
recorded in the hidden quota inodes, and the actual number of blocks
used by a particular user id or group id, it will overwrite the user
or group quota inode with all of the information it has.
Unfortunately, in the process it will zero out all of the current
quota limits set. This is unfortunate....
I've also found two kernel bugs. If the quota code is built as a
kernel module, and the quota_v2 module is not loaded, and the file
system has a hidden quota module, currently the kernel (a) won't
request that the quota_v2 module be loaded, since the array which
control the module loading only know about QFMT_VFS_OLD and
QFMT_VFS_V0, but not QFMT_VFS_CV1, and then (b)it will return a
mysterious error, "No such process" (ESRCH) without printing any kind
of explanatory printk message, and then fail to delete the
/sys/fs/ext4/<dev> kobject, which will cause future attempts to load
that device to fail and to trigger a sysfs WARN_ON when ext4 tries to
create the /sys/fs/ext4/<dev> object, and sysfs discovers that the old
one still exists.
I'll send patches for the kernel bugs, but that still leaves some
additional work that needs to be done before I'd be ready to call the
quota feature fully ready for prime time....
> Also, what happens if the machine crashes without umounting? How is
> the usage going to be handled? Is it in the journal? Anyway, it'd also
> be important to do a quotacheck.
The quota updates are all journalled, so that's all OK. And if they
aren't correct, e2fsck will correct them, although as mentioned above,
there are some cases when e2fsck might not notice a problem (a bug)
and when it does try to correct them, it will blow away all of the
stored quota limits in the quota inode (another bug). Also, we're
still missing repquota functionality for ocfs2 and ext4 with the
hidden inode feature --- this requires a new quotactl kernel interface
to support iterating over the uid/gid values in the dquot tree, and
userspace support in the quotatools package.
- Ted
Otherwise, ext4 file systems with the quota feature enable will get a
very confusing "No such process" error message if the quota code is
built as a mdoule and the quota_v2 module has not been loaded.
Signed-off-by: "Theodore Ts'o" <[email protected]>
Cc: Jan Kara <[email protected]>
Cc: [email protected]
---
include/linux/quota.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/linux/quota.h b/include/linux/quota.h
index 58fdef12..d133711 100644
--- a/include/linux/quota.h
+++ b/include/linux/quota.h
@@ -405,6 +405,7 @@ struct quota_module_name {
#define INIT_QUOTA_MODULE_NAMES {\
{QFMT_VFS_OLD, "quota_v1"},\
{QFMT_VFS_V0, "quota_v2"},\
+ {QFMT_VFS_V1, "quota_v2"},\
{0, NULL}}
#endif /* _QUOTA_ */
--
1.7.12.rc0.22.gcdd159b
In addition, print the error returned from ext4_enable_quotas()
Signed-off-by: "Theodore Ts'o" <[email protected]>
Cc: [email protected]
---
fs/ext4/super.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index 30651bd..0a6e9d5 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -4015,7 +4015,7 @@ no_journal:
!(sb->s_flags & MS_RDONLY)) {
err = ext4_enable_quotas(sb);
if (err)
- goto failed_mount7;
+ goto failed_mount8;
}
#endif /* CONFIG_QUOTA */
@@ -4042,6 +4042,8 @@ cantfind_ext4:
ext4_msg(sb, KERN_ERR, "VFS: Can't find ext4 filesystem");
goto failed_mount;
+failed_mount8:
+ kobject_del(&sbi->s_kobj);
failed_mount7:
ext4_unregister_li_request(sb);
failed_mount6:
@@ -5012,9 +5014,9 @@ static int ext4_enable_quotas(struct super_block *sb)
DQUOT_USAGE_ENABLED);
if (err) {
ext4_warning(sb,
- "Failed to enable quota (type=%d) "
- "tracking. Please run e2fsck to fix.",
- type);
+ "Failed to enable quota tracking "
+ "(type=%d, err=%d). Please run "
+ "e2fsck to fix.", type, err);
return err;
}
}
--
1.7.12.rc0.22.gcdd159b
On Mon 21-01-13 01:46:21, Ted Tso wrote:
> Otherwise, ext4 file systems with the quota feature enable will get a
> very confusing "No such process" error message if the quota code is
> built as a mdoule and the quota_v2 module has not been loaded.
Thanks for the fix. Should I merge it via my tree or do you plan to do it
via your tree?
Honza
>
> Signed-off-by: "Theodore Ts'o" <[email protected]>
> Cc: Jan Kara <[email protected]>
> Cc: [email protected]
> ---
> include/linux/quota.h | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/include/linux/quota.h b/include/linux/quota.h
> index 58fdef12..d133711 100644
> --- a/include/linux/quota.h
> +++ b/include/linux/quota.h
> @@ -405,6 +405,7 @@ struct quota_module_name {
> #define INIT_QUOTA_MODULE_NAMES {\
> {QFMT_VFS_OLD, "quota_v1"},\
> {QFMT_VFS_V0, "quota_v2"},\
> + {QFMT_VFS_V1, "quota_v2"},\
> {0, NULL}}
>
> #endif /* _QUOTA_ */
> --
> 1.7.12.rc0.22.gcdd159b
>
--
Jan Kara <[email protected]>
SUSE Labs, CR
Looks Good,
Reviewed-by: Carlos Maiolino <[email protected]>
> In addition, print the error returned from ext4_enable_quotas()
>
> Signed-off-by: "Theodore Ts'o" <[email protected]>
> Cc: [email protected]
> ---
> fs/ext4/super.c | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/fs/ext4/super.c b/fs/ext4/super.c
> index 30651bd..0a6e9d5 100644
> --- a/fs/ext4/super.c
> +++ b/fs/ext4/super.c
> @@ -4015,7 +4015,7 @@ no_journal:
> !(sb->s_flags & MS_RDONLY)) {
> err = ext4_enable_quotas(sb);
> if (err)
> - goto failed_mount7;
> + goto failed_mount8;
> }
> #endif /* CONFIG_QUOTA */
>
> @@ -4042,6 +4042,8 @@ cantfind_ext4:
> ext4_msg(sb, KERN_ERR, "VFS: Can't find ext4 filesystem");
> goto failed_mount;
>
> +failed_mount8:
> + kobject_del(&sbi->s_kobj);
> failed_mount7:
> ext4_unregister_li_request(sb);
> failed_mount6:
> @@ -5012,9 +5014,9 @@ static int ext4_enable_quotas(struct super_block *sb)
> DQUOT_USAGE_ENABLED);
> if (err) {
> ext4_warning(sb,
> - "Failed to enable quota (type=%d) "
> - "tracking. Please run e2fsck to fix.",
> - type);
> + "Failed to enable quota tracking "
> + "(type=%d, err=%d). Please run "
> + "e2fsck to fix.", type, err);
> return err;
> }
> }
> --
> 1.7.12.rc0.22.gcdd159b
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Carlos
Looks good,
Reviewed-by: Carlos Maiolino <[email protected]>
On Mon, Jan 21, 2013 at 01:46:21AM -0500, Theodore Ts'o wrote:
> Otherwise, ext4 file systems with the quota feature enable will get a
> very confusing "No such process" error message if the quota code is
> built as a mdoule and the quota_v2 module has not been loaded.
>
> Signed-off-by: "Theodore Ts'o" <[email protected]>
> Cc: Jan Kara <[email protected]>
> Cc: [email protected]
> ---
> include/linux/quota.h | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/include/linux/quota.h b/include/linux/quota.h
> index 58fdef12..d133711 100644
> --- a/include/linux/quota.h
> +++ b/include/linux/quota.h
> @@ -405,6 +405,7 @@ struct quota_module_name {
> #define INIT_QUOTA_MODULE_NAMES {\
> {QFMT_VFS_OLD, "quota_v1"},\
> {QFMT_VFS_V0, "quota_v2"},\
> + {QFMT_VFS_V1, "quota_v2"},\
> {0, NULL}}
>
> #endif /* _QUOTA_ */
> --
> 1.7.12.rc0.22.gcdd159b
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Carlos
On Mon, Jan 21, 2013 at 11:40:17AM +0100, Jan Kara wrote:
> On Mon 21-01-13 01:46:21, Ted Tso wrote:
> > Otherwise, ext4 file systems with the quota feature enable will get a
> > very confusing "No such process" error message if the quota code is
> > built as a mdoule and the quota_v2 module has not been loaded.
> Thanks for the fix. Should I merge it via my tree or do you plan to do it
> via your tree?
I'm happy to keep the patch in my tree (since I need it for my ext4
quota testing), unless you'd like to take it. It seems unlikely to
cause any conflicts with any changes you might have in your quota
tree, yes?
- Ted
On Mon 21-01-13 10:01:34, Ted Tso wrote:
> On Mon, Jan 21, 2013 at 11:40:17AM +0100, Jan Kara wrote:
> > On Mon 21-01-13 01:46:21, Ted Tso wrote:
> > > Otherwise, ext4 file systems with the quota feature enable will get a
> > > very confusing "No such process" error message if the quota code is
> > > built as a mdoule and the quota_v2 module has not been loaded.
> > Thanks for the fix. Should I merge it via my tree or do you plan to do it
> > via your tree?
>
> I'm happy to keep the patch in my tree (since I need it for my ext4
> quota testing), unless you'd like to take it. It seems unlikely to
> cause any conflicts with any changes you might have in your quota
> tree, yes?
Exactly. There shouldn't be any conflicts so just keep it in your tree.
You can add to the patch:
Acked-by: Jan Kara <[email protected]>
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Mon, Jan 21, 2013 at 04:10:08PM +0100, Jan Kara wrote:
> Exactly. There shouldn't be any conflicts so just keep it in your tree.
> You can add to the patch:
> Acked-by: Jan Kara <[email protected]>
Great, thanks.
BTW, do you have any plans to work on getting repquota support for
ocfs2 and ext4 with the internal quota support into quotactl(2) and
quotatools? It's missing functionality that I think users who want to
switch to the internal quota support will very much care about.
The work list that I have currently for ext4 w/ internal quota support
are:
1) Debugfs support for manipulating the quota inode directly. (So we
can introduce fs corruptions and make sure e2fsck deals
appropriately).
2) Make sure e2fsck preserves the quota limits information when it
repairs the quota information, and that it notices a missing quota
record in the quota inode.
3) Make sure e2fsck notices repairs a quota inode which is corrupted
in some way (and not just contains missing or incorrect usage
information).
4) More regression tests in e2fsprogs.
5) Add some kind of functional replacement for repquota. If
quotactl(2) support is not forthcoming in the near future, at the very
least debugfs should have a way of dumping out the repquota
information for an unmounted ext4 file system with internal quotas
enabled.
Can anyone think of other things which might be missing?
- Ted
Theodore Ts'o ([email protected]) wrote on 21 January 2013 00:47:
>Also, we're still missing repquota functionality for ocfs2 and ext4
>with the hidden inode feature
It works here with ext4. I haven't removed users yet but current ones
are shown. Both repquota and repquota -v work in the same way as the
vfs one.
On Mon 21-01-13 10:50:20, Ted Tso wrote:
> On Mon, Jan 21, 2013 at 04:10:08PM +0100, Jan Kara wrote:
> > Exactly. There shouldn't be any conflicts so just keep it in your tree.
> > You can add to the patch:
> > Acked-by: Jan Kara <[email protected]>
>
> Great, thanks.
>
> BTW, do you have any plans to work on getting repquota support for
> ocfs2 and ext4 with the internal quota support into quotactl(2) and
> quotatools? It's missing functionality that I think users who want to
> switch to the internal quota support will very much care about.
As Carlos mentioned, it kind-of works. If repquota(8) doesn't have a way
to iterate over quota structures directly, it iterates over entries in
/etc/passwd and uses Q_GETQUOTA to find the information. It is not a
perfect solution (e.g. when you have a huge LDAP database instead of
/etc/passwd) but in lots of cases it works just fine.
> 5) Add some kind of functional replacement for repquota. If
> quotactl(2) support is not forthcoming in the near future, at the very
> least debugfs should have a way of dumping out the repquota
> information for an unmounted ext4 file system with internal quotas
> enabled.
I have it on my todo list for a long time. The problem is noone wants it
enough to get to the top of the list... Also it won't be completely trivial
interface because we'll have to return some handle with which we could
associate scanning state. My plan would be to likely return an ordinary
read-only file handle (mostly to avoid various problems with what happens
when process exits without closing the handle or what happens on fork)
where f_pos would mean [ug]id to report next and reading would fill the
buffer with quota structures (if_dqblk + [ug]id). But I haven't tried
coding this...
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Sun, Jan 20, 2013 at 9:47 PM, Theodore Ts'o <[email protected]> wrote:
> On Sat, Jan 19, 2013 at 12:40:35AM -0200, Carlos Carvalho wrote:
>> I've started to use the new quota implementation (3.7.3 running now).
>> I used tune2fs -Q to turn non {usr,grp}quota. It's nice, no need to
>> run quotaon and all the jquota=file options. It works but it starts
>> counting only what comes after the mount with the new quota. If there
>> is already stuff in the filesystem it's not counted. How can we do a
>> quotacheck?
>
> tune2fs -Q should have set up the quota inodes with the correct usage
> information. What version of e2fsprogs were you using? If the quota
> information is incorrect, e2fsck will correct it --- so e2fsck will
> serve as the quota check.
>
> That being said, I've been doing some testing, and I have found some
> shortcomings with the e2fsck's handling of the quota option that we
> need to fix before I can recommend people use the new quota
> implementation.
>
> 1) The e2fsck check of the on-disk quota inodes won't notice if there
> is a missing uid record. (i.e., if some uid, say daemon owns a bunch
> of files, but that uid record is not in the quota inode, e2fsck won't
> say boo.)
>
> 2) If e2fsck *does* notice a discrepancy between the usage information
> recorded in the hidden quota inodes, and the actual number of blocks
> used by a particular user id or group id, it will overwrite the user
> or group quota inode with all of the information it has.
> Unfortunately, in the process it will zero out all of the current
> quota limits set. This is unfortunate....
>
I thought I had made the change to e2fsck to maintain limits as well,
but looking at code now it does seem that e2fsck will discard the
limits during fixing. tune2fs does maintain limits if the previous
quota files (in V1 format) are present. I can send a patch to fix
e2fsck. (1) above will be a bit tricky to fix though. I will take a
look at it too.
Thanks,
--
Aditya
I've just seen that the new quota implementation doesn't stop writes
after the limit has been reached. This means it's useless... The
machine is running 3.7.3.
To check: dd if=/dev/zero of=anything bs=2M
It'll never stop. I've gone over twice my limit to be sure.
By default, only usage is enabled when the filesystem is mounted. You
will need to call quotaon to turn on limit enforcement. If it doesn't
even after that, then its a bug.
On Thu, Jan 31, 2013 at 11:01 AM, Carlos Carvalho <[email protected]> wrote:
> I've just seen that the new quota implementation doesn't stop writes
> after the limit has been reached. This means it's useless... The
> machine is running 3.7.3.
>
> To check: dd if=/dev/zero of=anything bs=2M
>
> It'll never stop. I've gone over twice my limit to be sure.
--
Aditya
Aditya Kali ([email protected]) wrote on 31 January 2013 11:04:
>By default, only usage is enabled when the filesystem is mounted. You
>will need to call quotaon to turn on limit enforcement. If it doesn't
>even after that, then its a bug.
Ah, fine. It works, fortunately :-). Thanks.
It's not clear what can be dropped and what must still be used in the
new implementation. It'd be nice to state it in the man page of
tune2fs, for example. I thought that if we wanted to stop quotas it'd
be necessary to umount and tune2fs again.
On Thu, Jan 31, 2013 at 05:13:16PM -0200, Carlos Carvalho wrote:
> Ah, fine. It works, fortunately :-). Thanks.
>
> It's not clear what can be dropped and what must still be used in the
> new implementation. It'd be nice to state it in the man page of
> tune2fs, for example. I thought that if we wanted to stop quotas it'd
> be necessary to umount and tune2fs again.
This sequence documents the biggest set of issues with the new
implementation:
1) Create a file system. Populate it with files. I untarred the
e2fsprogs source tree as root, so there were a lot of files owned by
root.
2) unmount the filesystem and run tune2fs -O quota /dev/XXX
3) mount the file system; observe that the quota tools don't realize
that they should be trying to do the quota thing.
4) mount the file system with the -o quota flag; this sets the quota
option in /etc/mtab, which is necessary for the quota tools to
correctly deal with the ext4 file system.
5) mount the file system; observe (via "quota -v root") that the usage
information for root is _not_ correct; in fact the usage for root is
nonexistent.
6) unmount the file system and run e2fsck
7) mount the file system; observe that the usage information for root
still doesn't exist.
8) Set a quota for root: "setquota -u root 1024 2048 500 1000 /mnt"
9) Unmount the file system and run e2fsck. Now e2fsck will complain
and ask if you should fix the usage information for rooot. Say yes.
10) Mount the file system (with -o quota), and run "quota -v root".
Observe that the usage information is now correct, but the quota
limits for root (and all users/groups) have been cleared.
Basically, there are a bunch of e2fsprogs bugs with quota which we
need to fix before we can really recommend end users to use the first
class quota support. If you use quota with a freshly created file
system, it works pretty well, as long as the usage quota information
never gets inconsistent (or if you are using the quota system for
usage tracking only and not for quota enforcement). But if you try to
add the quota feature to an already existing file system which has
files already pre-populated, there are some definite issues that need
fixing. :-(
Most of these issues have been written up at:
https://ext4.wiki.kernel.org/index.php/Quota
- Ted
On Thu 31-01-13 17:03:12, Ted Tso wrote:
> On Thu, Jan 31, 2013 at 05:13:16PM -0200, Carlos Carvalho wrote:
> > Ah, fine. It works, fortunately :-). Thanks.
> >
> > It's not clear what can be dropped and what must still be used in the
> > new implementation. It'd be nice to state it in the man page of
> > tune2fs, for example. I thought that if we wanted to stop quotas it'd
> > be necessary to umount and tune2fs again.
>
> This sequence documents the biggest set of issues with the new
> implementation:
>
> 1) Create a file system. Populate it with files. I untarred the
> e2fsprogs source tree as root, so there were a lot of files owned by
> root.
>
> 2) unmount the filesystem and run tune2fs -O quota /dev/XXX
>
> 3) mount the file system; observe that the quota tools don't realize
> that they should be trying to do the quota thing.
What version of quota-tools are you using? If I didn't screw up
something, version 4.01 shouldn't need the -o quota mount option anymore...
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
Thanks for the detailed instructions.
Theodore Ts'o ([email protected]) wrote on 31 January 2013 17:03:
>If you use quota with a freshly created file
>system, it works pretty well, as long as the usage quota information
>never gets inconsistent (or if you are using the quota system for
>usage tracking only and not for quota enforcement). But if you try to
>add the quota feature to an already existing file system which has
>files already pre-populated, there are some definite issues that need
>fixing. :-(
That's what I'm doing: in a new filesystem I use the new
implementation, in another I use the vfs one, because it's populated.
Jan Kara ([email protected]) wrote on 31 January 2013 23:28:
>> 3) mount the file system; observe that the quota tools don't realize
>> that they should be trying to do the quota thing.
> What version of quota-tools are you using? If I didn't screw up
>something, version 4.01 shouldn't need the -o quota mount option anymore...
4.01, from Debian unstable. -o quota is necessary for the new
implementation to track usage, no? I put it in fstab, so quota appears
in /proc/mounts.
On Thu, Jan 31, 2013 at 11:28:17PM +0100, Jan Kara wrote:
> What version of quota-tools are you using? If I didn't screw up
> something, version 4.01 shouldn't need the -o quota mount option anymore...
Ah, I was using 4.00-4 from Debian Testing (which hopefully will soon
be the new Debian Obsolete^H^H^H^H^H^H^H^H Stable).
I'll have to upgrade to a newer quota tools for my testing, but we
should definitely note that users who want to use the internal quota
needs to be using a newer quota-tools than what is shipping in Debian
Stable, Debian Testing, Ubuntu Precise, etc.
Is there an easy way to find what version of quota-tools is being used
in various versions of Fedora or OpenSuSE (I assume all of the
enterprise distro's will have prehistoric softare, so I'm not even
worrying about them :-). This would be good information to put up on:
https://ext4.wiki.kernel.org/index.php/Quota
- Ted
On Thu 31-01-13 18:03:30, Ted Tso wrote:
> On Thu, Jan 31, 2013 at 11:28:17PM +0100, Jan Kara wrote:
> > What version of quota-tools are you using? If I didn't screw up
> > something, version 4.01 shouldn't need the -o quota mount option anymore...
>
> Ah, I was using 4.00-4 from Debian Testing (which hopefully will soon
> be the new Debian Obsolete^H^H^H^H^H^H^H^H Stable).
>
> I'll have to upgrade to a newer quota tools for my testing, but we
> should definitely note that users who want to use the internal quota
> needs to be using a newer quota-tools than what is shipping in Debian
> Stable, Debian Testing, Ubuntu Precise, etc.
>
> Is there an easy way to find what version of quota-tools is being used
> in various versions of Fedora or OpenSuSE (I assume all of the
> enterprise distro's will have prehistoric softare, so I'm not even
> worrying about them :-). This would be good information to put up on:
>
> https://ext4.wiki.kernel.org/index.php/Quota
I don't know about Fedora but for openSUSE one can have a look at:
http://software.opensuse.org/package/quota
There you see the version for the latest distro (12.2 - has version
4.00), if you click on "Show other versions", you can check the version in
older releases.
Actually I can see there that openSUSE Factory still has 4.00 (so that
will end up in soon to be released 12.3) so I have to ping our quota
packager to update.
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR