2016-07-07 23:38:41

by Kamran Khan

[permalink] [raw]
Subject: Cannot umount ext4 fs, no user-space procs using the fs, lsof points to jbd2

Hi,

I am unmounting an ext4 filesystem (Ubuntu 16.04), fuser shows that no
user-space processes are holding a handle to it:

> root@kakhan-ubuntu:/# fuser -vm /oldroot
> USER PID ACCESS COMMAND
> /oldroot: root kernel mount /oldroot

lsof gives this:

> root@kakhan-ubuntu:/# lsof | grep sda
> jbd2/sda1 368 root cwd DIR 0,20 340 2 /
> jbd2/sda1 368 root rtd DIR 0,20 340 2 /
> jbd2/sda1 368 root txt unknown /proc/368/exe

If I look into /proc/../mounts I see lots of kernel specific processes
which have /dev/sda1 listed in their mounts:

> root@kakhan-ubuntu:/# find /proc -name mounts | xargs grep /dev/sda1
> /proc/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/1/task/1/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/1/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/2/task/2/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/2/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/3/task/3/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/3/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/5/task/5/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/5/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> /proc/7/task/7/mounts:/dev/sda1 /oldroot ext4 rw,relatime,errors=remount-ro,data=ordered 0 0
> # and whole bunch of others ...)

I am unable to unmount the filesystem:

> root@kakhan-ubuntu:/# umount /oldroot
> umount: /oldroot: target is busy
> (In some cases useful info about processes that
> use the device is found by lsof(8) or fuser(1).)

Can anyone please provide some pointers on how should I proceed with
unmounting the filesystem?

(Context: I copied binaries into a tmpfs and did a pivot_root to it,
i.e., the opposite of what happens from initramfs during boot. I
killed all the userspace processes holding a handle to the oldroot.
This issue is different than the one I faced a couple of months ago
during the same exercise on CentOS [1], where systemd-udevd needed a
restart to release the handle to the old filesystem. In this case
there's no userspace procs accessing the mountpoint.)

[1] http://permalink.gmane.org/gmane.comp.file-systems.ext4/53314

--
Kamran.

http://inspirated.com/


2016-07-08 02:39:12

by Theodore Ts'o

[permalink] [raw]
Subject: Re: Cannot umount ext4 fs, no user-space procs using the fs, lsof points to jbd2

On Thu, Jul 07, 2016 at 04:38:39PM -0700, Kamran Khan wrote:
> Hi,
>
> I am unmounting an ext4 filesystem (Ubuntu 16.04), fuser shows that no
> user-space processes are holding a handle to it:
>
> > root@kakhan-ubuntu:/# fuser -vm /oldroot
> > USER PID ACCESS COMMAND
> > /oldroot: root kernel mount /oldroot
>
> lsof gives this:
>
> > root@kakhan-ubuntu:/# lsof | grep sda
> > jbd2/sda1 368 root cwd DIR 0,20 340 2 /
> > jbd2/sda1 368 root rtd DIR 0,20 340 2 /
> > jbd2/sda1 368 root txt unknown /proc/368/exe

This is the jbd2 kernel thread for /dev/sda1; there is one of these
for each mounted file system, and the thread will automatically exit
with the relevant file system is umounted: unmount /dev/sda1.

For example:

# ps augxww | grep jbd2
root 319 0.0 0.0 0 0 ? S Jul02 0:45 [jbd2/sda3-8]
root 1284 0.0 0.0 0 0 ? S Jul02 0:00 [jbd2/dm-9-8]
root 1286 0.0 0.0 0 0 ? S Jul02 0:03 [jbd2/dm-8-8]
root 1303 0.0 0.0 0 0 ? S Jul02 0:01 [jbd2/dm-1-8]
root 1314 0.0 0.0 0 0 ? S Jul02 0:00 [jbd2/dm-10-8]
root 1334 0.0 0.0 0 0 ? S Jul02 0:00 [jbd2/dm-0-8]
root 7338 0.0 0.0 12748 2180 pts/3 S+ 22:32 0:00 grep jbd2

# lsof -p 319
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
jbd2/sda3 319 root cwd DIR 8,3 4096 2 /
jbd2/sda3 319 root rtd DIR 8,3 4096 2 /
jbd2/sda3 319 root txt unknown /proc/319/exe

# lsof -p 1284
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
jbd2/dm-9 1284 root cwd DIR 8,3 4096 2 /
jbd2/dm-9 1284 root rtd DIR 8,3 4096 2 /
jbd2/dm-9 1284 root txt unknown /proc/1284/exe

# ls -l /dev/mapper/lambda-android
0 lrwxrwxrwx. 1 root root 7 Jul 2 23:02 /dev/mapper/lambda-android -> ../dm-9
# umount /dev/mapper/lambda-android

# ps augxww | grep jbd2
root 319 0.0 0.0 0 0 ? S Jul02 0:45 [jbd2/sda3-8]
root 1286 0.0 0.0 0 0 ? S Jul02 0:03 [jbd2/dm-8-8]
root 1303 0.0 0.0 0 0 ? S Jul02 0:01 [jbd2/dm-1-8]
root 1314 0.0 0.0 0 0 ? S Jul02 0:00 [jbd2/dm-10-8]
root 1334 0.0 0.0 0 0 ? S Jul02 0:00 [jbd2/dm-0-8]
root 7469 0.0 0.0 12748 2184 pts/3 S+ 22:35 0:00 grep jbd2

So there's nothing wrong with the fact that you see a jbd2 thread for
/dev/sda1, and that's not the reason why you are having trouble unmounting /oldroot.

> I am unable to unmount the filesystem:
>
> > root@kakhan-ubuntu:/# umount /oldroot
> > umount: /oldroot: target is busy
> > (In some cases useful info about processes that
> > use the device is found by lsof(8) or fuser(1).)
>
> Can anyone please provide some pointers on how should I proceed with
> unmounting the filesystem?

This is more of a VFS question than an ext4 question, and
unfortunately fuser and lsof are not the only reliable ways if a file
system is still busy.

You might want to try searching /proc/.../mounts for any potential
mountpoints on top of /oldroot that might be left over from a process
which created a separate namespace, and hence wasn't affected by the
umount of /oldroot/proc, or /oldroot/sys, etc.

Cheers,

- Ted