2019-11-17 17:26:55

by Jaskaran Singh

[permalink] [raw]
Subject: [PATCH v2 0/3] docs: filesystems: convert autofs.txt to reST

The following patch series is for converting autofs.txt to reST, and
updating some of the content.

Changes from v1:
----------------
- Split patch into multiple logical changes as per Jonathan Corbet's
request.
- Few more formatting changes and fixes, as pointed out by Jonathan.
- Add short description of master map used by autofs.

autofs.rst | 273 ++++++++++++++++++++++++++++++++-----------------------------
index.rst | 1
2 files changed, 148 insertions(+), 126 deletions(-)



2019-11-17 17:26:56

by Jaskaran Singh

[permalink] [raw]
Subject: [PATCH v2 1/3] docs: filesystems: convert autofs.txt to reST

Convert autofs.txt to reST.

The following changes abound:

- Introduce reST formatting for headings, lists et al.
- Add an indentation of an 8 space tab wherever suitable, so as
to maintain consistency.
- Remove indentation of the description of the ioctls which are similar
to the AUTOFS_IOC ioctls, as it does not come out quite right in HTML.
- Add an entry for autofs in the index.

Signed-off-by: Jaskaran Singh <[email protected]>
---
.../filesystems/{autofs.txt => autofs.rst} | 253 ++++++++++--------
Documentation/filesystems/index.rst | 1 +
2 files changed, 135 insertions(+), 119 deletions(-)
rename Documentation/filesystems/{autofs.txt => autofs.rst} (77%)

diff --git a/Documentation/filesystems/autofs.txt b/Documentation/filesystems/autofs.rst
similarity index 77%
rename from Documentation/filesystems/autofs.txt
rename to Documentation/filesystems/autofs.rst
index 3af38c7fd26d..6702a5f61f50 100644
--- a/Documentation/filesystems/autofs.txt
+++ b/Documentation/filesystems/autofs.rst
@@ -1,12 +1,9 @@
-<head>
-<style> p { max-width:50em} ol, ul {max-width: 40em}</style>
-</head>
-
+=====================
autofs - how it works
=====================

Purpose
--------
+=======

The goal of autofs is to provide on-demand mounting and race free
automatic unmounting of various other filesystems. This provides two
@@ -28,7 +25,7 @@ key advantages:
first accessed a name.

Context
--------
+=======

The "autofs" filesystem module is only one part of an autofs system.
There also needs to be a user-space program which looks up names
@@ -43,7 +40,7 @@ filesystem type. Several "autofs" filesystems can be mounted and they
can each be managed separately, or all managed by the same daemon.

Content
--------
+=======

An autofs filesystem can contain 3 sorts of objects: directories,
symbolic links and mount traps. Mount traps are directories with
@@ -52,7 +49,7 @@ extra properties as described in the next section.
Objects can only be created by the automount daemon: symlinks are
created with a regular `symlink` system call, while directories and
mount traps are created with `mkdir`. The determination of whether a
-directory should be a mount trap or not is quite _ad hoc_, largely for
+directory should be a mount trap or not is quite ad hoc, largely for
historical reasons, and is determined in part by the
*direct*/*indirect*/*offset* mount options, and the *maxproto* mount option.

@@ -80,7 +77,7 @@ where in the tree they are (root, top level, or lower), the *maxproto*,
and whether the mount was *indirect* or not.

Mount Traps
----------------
+===========

A core element of the implementation of autofs is the Mount Traps
which are provided by the Linux VFS. Any directory provided by a
@@ -201,7 +198,7 @@ initiated or is being considered, otherwise it returns 0.


Mountpoint expiry
------------------
+=================

The VFS has a mechanism for automatically expiring unused mounts,
much as it can expire any unused dentry information from the dcache.
@@ -301,7 +298,7 @@ completed (together with removing any directories that might have been
necessary), or has been aborted.

Communicating with autofs: detecting the daemon
------------------------------------------------
+===============================================

There are several forms of communication between the automount daemon
and the filesystem. As we have already seen, the daemon can create and
@@ -317,33 +314,33 @@ If the daemon ever has to be stopped and restarted a new pgid can be
provided through an ioctl as will be described below.

Communicating with autofs: the event pipe
------------------------------------------
+=========================================

When an autofs filesystem is mounted, the 'write' end of a pipe must
be passed using the 'fd=' mount option. autofs will write
notification messages to this pipe for the daemon to respond to.
-For version 5, the format of the message is:
-
- struct autofs_v5_packet {
- int proto_version; /* Protocol version */
- int type; /* Type of packet */
- autofs_wqt_t wait_queue_token;
- __u32 dev;
- __u64 ino;
- __u32 uid;
- __u32 gid;
- __u32 pid;
- __u32 tgid;
- __u32 len;
- char name[NAME_MAX+1];
+For version 5, the format of the message is::
+
+ struct autofs_v5_packet {
+ int proto_version; /* Protocol version */
+ int type; /* Type of packet */
+ autofs_wqt_t wait_queue_token;
+ __u32 dev;
+ __u64 ino;
+ __u32 uid;
+ __u32 gid;
+ __u32 pid;
+ __u32 tgid;
+ __u32 len;
+ char name[NAME_MAX+1];
};

-where the type is one of
+where the type is one of ::

- autofs_ptype_missing_indirect
- autofs_ptype_expire_indirect
- autofs_ptype_missing_direct
- autofs_ptype_expire_direct
+ autofs_ptype_missing_indirect
+ autofs_ptype_expire_indirect
+ autofs_ptype_missing_direct
+ autofs_ptype_expire_direct

so messages can indicate that a name is missing (something tried to
access it but it isn't there) or that it has been selected for expiry.
@@ -360,7 +357,7 @@ acknowledged using one of the ioctls below with the relevant
`wait_queue_token`.

Communicating with autofs: root directory ioctls
-------------------------------------------------
+================================================

The root directory of an autofs filesystem will respond to a number of
ioctls. The process issuing the ioctl must have the CAP_SYS_ADMIN
@@ -368,58 +365,67 @@ capability, or must be the automount daemon.

The available ioctl commands are:

-- **AUTOFS_IOC_READY**: a notification has been handled. The argument
- to the ioctl command is the "wait_queue_token" number
- corresponding to the notification being acknowledged.
-- **AUTOFS_IOC_FAIL**: similar to above, but indicates failure with
- the error code `ENOENT`.
-- **AUTOFS_IOC_CATATONIC**: Causes the autofs to enter "catatonic"
- mode meaning that it stops sending notifications to the daemon.
- This mode is also entered if a write to the pipe fails.
-- **AUTOFS_IOC_PROTOVER**: This returns the protocol version in use.
-- **AUTOFS_IOC_PROTOSUBVER**: Returns the protocol sub-version which
- is really a version number for the implementation.
-- **AUTOFS_IOC_SETTIMEOUT**: This passes a pointer to an unsigned
- long. The value is used to set the timeout for expiry, and
- the current timeout value is stored back through the pointer.
-- **AUTOFS_IOC_ASKUMOUNT**: Returns, in the pointed-to `int`, 1 if
- the filesystem could be unmounted. This is only a hint as
- the situation could change at any instant. This call can be
- used to avoid a more expensive full unmount attempt.
-- **AUTOFS_IOC_EXPIRE**: as described above, this asks if there is
- anything suitable to expire. A pointer to a packet:
-
- struct autofs_packet_expire_multi {
- int proto_version; /* Protocol version */
- int type; /* Type of packet */
- autofs_wqt_t wait_queue_token;
- int len;
- char name[NAME_MAX+1];
- };
+- **AUTOFS_IOC_READY**:
+ a notification has been handled. The argument
+ to the ioctl command is the "wait_queue_token" number
+ corresponding to the notification being acknowledged.
+- **AUTOFS_IOC_FAIL**:
+ similar to above, but indicates failure with
+ the error code `ENOENT`.
+- **AUTOFS_IOC_CATATONIC**:
+ Causes the autofs to enter "catatonic"
+ mode meaning that it stops sending notifications to the daemon.
+ This mode is also entered if a write to the pipe fails.
+- **AUTOFS_IOC_PROTOVER**:
+ This returns the protocol version in use.
+- **AUTOFS_IOC_PROTOSUBVER**:
+ Returns the protocol sub-version which
+ is really a version number for the implementation.
+- **AUTOFS_IOC_SETTIMEOUT**:
+ This passes a pointer to an unsigned
+ long. The value is used to set the timeout for expiry, and
+ the current timeout value is stored back through the pointer.
+- **AUTOFS_IOC_ASKUMOUNT**:
+ Returns, in the pointed-to `int`, 1 if
+ the filesystem could be unmounted. This is only a hint as
+ the situation could change at any instant. This call can be
+ used to avoid a more expensive full unmount attempt.
+- **AUTOFS_IOC_EXPIRE**:
+ as described above, this asks if there is
+ anything suitable to expire. A pointer to a packet::
+
+ struct autofs_packet_expire_multi {
+ int proto_version; /* Protocol version */
+ int type; /* Type of packet */
+ autofs_wqt_t wait_queue_token;
+ int len;
+ char name[NAME_MAX+1];
+ };

- is required. This is filled in with the name of something
- that can be unmounted or removed. If nothing can be expired,
- `errno` is set to `EAGAIN`. Even though a `wait_queue_token`
- is present in the structure, no "wait queue" is established
- and no acknowledgment is needed.
-- **AUTOFS_IOC_EXPIRE_MULTI**: This is similar to
- **AUTOFS_IOC_EXPIRE** except that it causes notification to be
- sent to the daemon, and it blocks until the daemon acknowledges.
- The argument is an integer which can contain two different flags.
+ is required. This is filled in with the name of something
+ that can be unmounted or removed. If nothing can be expired,
+ `errno` is set to `EAGAIN`. Even though a `wait_queue_token`
+ is present in the structure, no "wait queue" is established
+ and no acknowledgment is needed.
+- **AUTOFS_IOC_EXPIRE_MULTI**:
+ This is similar to
+ **AUTOFS_IOC_EXPIRE** except that it causes notification to be
+ sent to the daemon, and it blocks until the daemon acknowledges.
+ The argument is an integer which can contain two different flags.

- **AUTOFS_EXP_IMMEDIATE** causes `last_used` time to be ignored
- and objects are expired if the are not in use.
+ **AUTOFS_EXP_IMMEDIATE** causes `last_used` time to be ignored
+ and objects are expired if the are not in use.

- **AUTOFS_EXP_FORCED** causes the in use status to be ignored
- and objects are expired ieven if they are in use. This assumes
- that the daemon has requested this because it is capable of
- performing the umount.
+ **AUTOFS_EXP_FORCED** causes the in use status to be ignored
+ and objects are expired ieven if they are in use. This assumes
+ that the daemon has requested this because it is capable of
+ performing the umount.

- **AUTOFS_EXP_LEAVES** will select a leaf rather than a top-level
- name to expire. This is only safe when *maxproto* is 4.
+ **AUTOFS_EXP_LEAVES** will select a leaf rather than a top-level
+ name to expire. This is only safe when *maxproto* is 4.

Communicating with autofs: char-device ioctls
----------------------------------------------
+=============================================

It is not always possible to open the root of an autofs filesystem,
particularly a *direct* mounted filesystem. If the automount daemon
@@ -429,9 +435,9 @@ need there is a "miscellaneous" character device (major 10, minor 235)
which can be used to communicate directly with the autofs filesystem.
It requires CAP_SYS_ADMIN for access.

-The `ioctl`s that can be used on this device are described in a separate
+The 'ioctl's that can be used on this device are described in a separate
document `autofs-mount-control.txt`, and are summarised briefly here.
-Each ioctl is passed a pointer to an `autofs_dev_ioctl` structure:
+Each ioctl is passed a pointer to an `autofs_dev_ioctl` structure::

struct autofs_dev_ioctl {
__u32 ver_major;
@@ -469,41 +475,50 @@ that the kernel module can support.

Commands are:

-- **AUTOFS_DEV_IOCTL_VERSION_CMD**: does nothing, except validate and
- set version numbers.
-- **AUTOFS_DEV_IOCTL_OPENMOUNT_CMD**: return an open file descriptor
- on the root of an autofs filesystem. The filesystem is identified
- by name and device number, which is stored in `openmount.devid`.
- Device numbers for existing filesystems can be found in
- `/proc/self/mountinfo`.
-- **AUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD**: same as `close(ioctlfd)`.
-- **AUTOFS_DEV_IOCTL_SETPIPEFD_CMD**: if the filesystem is in
- catatonic mode, this can provide the write end of a new pipe
- in `setpipefd.pipefd` to re-establish communication with a daemon.
- The process group of the calling process is used to identify the
- daemon.
-- **AUTOFS_DEV_IOCTL_REQUESTER_CMD**: `path` should be a
- name within the filesystem that has been auto-mounted on.
- On successful return, `requester.uid` and `requester.gid` will be
- the UID and GID of the process which triggered that mount.
-- **AUTOFS_DEV_IOCTL_ISMOUNTPOINT_CMD**: Check if path is a
- mountpoint of a particular type - see separate documentation for
- details.
-- **AUTOFS_DEV_IOCTL_PROTOVER_CMD**:
-- **AUTOFS_DEV_IOCTL_PROTOSUBVER_CMD**:
-- **AUTOFS_DEV_IOCTL_READY_CMD**:
-- **AUTOFS_DEV_IOCTL_FAIL_CMD**:
-- **AUTOFS_DEV_IOCTL_CATATONIC_CMD**:
-- **AUTOFS_DEV_IOCTL_TIMEOUT_CMD**:
-- **AUTOFS_DEV_IOCTL_EXPIRE_CMD**:
-- **AUTOFS_DEV_IOCTL_ASKUMOUNT_CMD**: These all have the same
- function as the similarly named **AUTOFS_IOC** ioctls, except
- that **FAIL** can be given an explicit error number in `fail.status`
- instead of assuming `ENOENT`, and this **EXPIRE** command
- corresponds to **AUTOFS_IOC_EXPIRE_MULTI**.
+- **AUTOFS_DEV_IOCTL_VERSION_CMD**:
+ does nothing, except validate and
+ set version numbers.
+- **AUTOFS_DEV_IOCTL_OPENMOUNT_CMD**:
+ return an open file descriptor
+ on the root of an autofs filesystem. The filesystem is identified
+ by name and device number, which is stored in `openmount.devid`.
+ Device numbers for existing filesystems can be found in
+ `/proc/self/mountinfo`.
+- **AUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD**:
+ same as `close(ioctlfd)`.
+- **AUTOFS_DEV_IOCTL_SETPIPEFD_CMD**:
+ if the filesystem is in
+ catatonic mode, this can provide the write end of a new pipe
+ in `setpipefd.pipefd` to re-establish communication with a daemon.
+ The process group of the calling process is used to identify the
+ daemon.
+- **AUTOFS_DEV_IOCTL_REQUESTER_CMD**:
+ `path` should be a
+ name within the filesystem that has been auto-mounted on.
+ On successful return, `requester.uid` and `requester.gid` will be
+ the UID and GID of the process which triggered that mount.
+- **AUTOFS_DEV_IOCTL_ISMOUNTPOINT_CMD**:
+ Check if path is a
+ mountpoint of a particular type - see separate documentation for
+ details.
+
+- **AUTOFS_DEV_IOCTL_PROTOVER_CMD**
+- **AUTOFS_DEV_IOCTL_PROTOSUBVER_CMD**
+- **AUTOFS_DEV_IOCTL_READY_CMD**
+- **AUTOFS_DEV_IOCTL_FAIL_CMD**
+- **AUTOFS_DEV_IOCTL_CATATONIC_CMD**
+- **AUTOFS_DEV_IOCTL_TIMEOUT_CMD**
+- **AUTOFS_DEV_IOCTL_EXPIRE_CMD**
+- **AUTOFS_DEV_IOCTL_ASKUMOUNT_CMD**
+
+These all have the same
+function as the similarly named **AUTOFS_IOC** ioctls, except
+that **FAIL** can be given an explicit error number in `fail.status`
+instead of assuming `ENOENT`, and this **EXPIRE** command
+corresponds to **AUTOFS_IOC_EXPIRE_MULTI**.

Catatonic mode
---------------
+==============

As mentioned, an autofs mount can enter "catatonic" mode. This
happens if a write to the notification pipe fails, or if it is
@@ -527,7 +542,7 @@ Catatonic mode can only be left via the
**AUTOFS_DEV_IOCTL_OPENMOUNT_CMD** ioctl on the `/dev/autofs`.

The "ignore" mount option
--------------------------
+=========================

The "ignore" mount option can be used to provide a generic indicator
to applications that the mount entry should be ignored when displaying
@@ -542,18 +557,18 @@ This is intended to be used by user space programs to exclude autofs
mounts from consideration when reading the mounts list.

autofs, name spaces, and shared mounts
---------------------------------------
+======================================

With bind mounts and name spaces it is possible for an autofs
filesystem to appear at multiple places in one or more filesystem
name spaces. For this to work sensibly, the autofs filesystem should
-always be mounted "shared". e.g.
+always be mounted "shared". e.g. ::

-> `mount --make-shared /autofs/mount/point`
+ mount --make-shared /autofs/mount/point

The automount daemon is only able to manage a single mount location for
an autofs filesystem and if mounts on that are not 'shared', other
locations will not behave as expected. In particular access to those
-other locations will likely result in the `ELOOP` error
+other locations will likely result in the `ELOOP` error ::

-> Too many levels of symbolic links
+ Too many levels of symbolic links
diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
index 2c3a9f761205..ad6315a48d14 100644
--- a/Documentation/filesystems/index.rst
+++ b/Documentation/filesystems/index.rst
@@ -46,4 +46,5 @@ Documentation for filesystem implementations.
.. toctree::
:maxdepth: 2

+ autofs
virtiofs
--
2.21.0

2019-11-17 17:29:42

by Jaskaran Singh

[permalink] [raw]
Subject: [PATCH v2 2/3] docs: filesystems: Update code snippets in autofs.rst

Some of the struct definitions now have an autofs packet header.
Reflect these changes by adding a definition of this header and
place it wherever suitable.

Signed-off-by: Jaskaran Singh <[email protected]>
---
Documentation/filesystems/autofs.rst | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/Documentation/filesystems/autofs.rst b/Documentation/filesystems/autofs.rst
index 6702a5f61f50..2f704e9c5dc0 100644
--- a/Documentation/filesystems/autofs.rst
+++ b/Documentation/filesystems/autofs.rst
@@ -322,8 +322,7 @@ notification messages to this pipe for the daemon to respond to.
For version 5, the format of the message is::

struct autofs_v5_packet {
- int proto_version; /* Protocol version */
- int type; /* Type of packet */
+ struct autofs_packet_hdr hdr;
autofs_wqt_t wait_queue_token;
__u32 dev;
__u64 ino;
@@ -335,6 +334,13 @@ For version 5, the format of the message is::
char name[NAME_MAX+1];
};

+And the format of the header is::
+
+ struct autofs_packet_hdr {
+ int proto_version; /* Protocol version */
+ int type; /* Type of packet */
+ };
+
where the type is one of ::

autofs_ptype_missing_indirect
@@ -395,8 +401,7 @@ The available ioctl commands are:
anything suitable to expire. A pointer to a packet::

struct autofs_packet_expire_multi {
- int proto_version; /* Protocol version */
- int type; /* Type of packet */
+ struct autofs_packet_hdr hdr;
autofs_wqt_t wait_queue_token;
int len;
char name[NAME_MAX+1];
--
2.21.0

2019-11-17 17:30:36

by Jaskaran Singh

[permalink] [raw]
Subject: [PATCH v2 3/3] docs: filesystems: Add mount map description in Content

The second paragraph of the content section does not properly
describe how mount points are determined by autofs.

Replace the lines detailing how the determination of these mount
points is "ad hoc" by a short description of the mount map syntax
used by autofs.

Signed-off-by: Jaskaran Singh <[email protected]>
---
Documentation/filesystems/autofs.rst | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/autofs.rst b/Documentation/filesystems/autofs.rst
index 2f704e9c5dc0..c17f6bf0eb5d 100644
--- a/Documentation/filesystems/autofs.rst
+++ b/Documentation/filesystems/autofs.rst
@@ -49,9 +49,10 @@ extra properties as described in the next section.
Objects can only be created by the automount daemon: symlinks are
created with a regular `symlink` system call, while directories and
mount traps are created with `mkdir`. The determination of whether a
-directory should be a mount trap or not is quite ad hoc, largely for
-historical reasons, and is determined in part by the
-*direct*/*indirect*/*offset* mount options, and the *maxproto* mount option.
+directory should be a mount trap is based on a master map. This master
+map is consulted by autofs to determine which directories are mount
+points. Mount points can be *direct*/*indirect*/*offset*.
+On most systems, the default master map is located at */etc/auto.master*.

If neither the *direct* or *offset* mount options are given (so the
mount is considered to be *indirect*), then the root directory is
--
2.21.0

2019-11-18 00:03:16

by Ian Kent

[permalink] [raw]
Subject: Re: [PATCH v2 3/3] docs: filesystems: Add mount map description in Content

On Sun, 2019-11-17 at 22:54 +0530, Jaskaran Singh wrote:
> The second paragraph of the content section does not properly
> describe how mount points are determined by autofs.
>
> Replace the lines detailing how the determination of these mount
> points is "ad hoc" by a short description of the mount map syntax
> used by autofs.
>
> Signed-off-by: Jaskaran Singh <[email protected]>
> ---
> Documentation/filesystems/autofs.rst | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/filesystems/autofs.rst
> b/Documentation/filesystems/autofs.rst
> index 2f704e9c5dc0..c17f6bf0eb5d 100644
> --- a/Documentation/filesystems/autofs.rst
> +++ b/Documentation/filesystems/autofs.rst
> @@ -49,9 +49,10 @@ extra properties as described in the next section.
> Objects can only be created by the automount daemon: symlinks are
> created with a regular `symlink` system call, while directories and
> mount traps are created with `mkdir`. The determination of whether
> a
> -directory should be a mount trap or not is quite ad hoc, largely for
> -historical reasons, and is determined in part by the
> -*direct*/*indirect*/*offset* mount options, and the *maxproto* mount
> option.
> +directory should be a mount trap is based on a master map. This
> master
> +map is consulted by autofs to determine which directories are mount
> +points. Mount points can be *direct*/*indirect*/*offset*.
> +On most systems, the default master map is located at
> */etc/auto.master*.

Ok, lets just use this for now since it's better than what was
originally there and I'll put together an update after these
changes are merged.

What happens is actually a bit more complicated, the opening
sentence above this is talking about what happens with maps
associated with autofs mount points given in the master map,
not so much master map entries themselves.

>
> If neither the *direct* or *offset* mount options are given (so the
> mount is considered to be *indirect*), then the root directory is

2019-11-18 19:22:50

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH v2 0/3] docs: filesystems: convert autofs.txt to reST

On Sun, 17 Nov 2019 22:54:33 +0530
Jaskaran Singh <[email protected]> wrote:

> The following patch series is for converting autofs.txt to reST, and
> updating some of the content.
>
> Changes from v1:
> ----------------
> - Split patch into multiple logical changes as per Jonathan Corbet's
> request.
> - Few more formatting changes and fixes, as pointed out by Jonathan.
> - Add short description of master map used by autofs.
>
> autofs.rst | 273 ++++++++++++++++++++++++++++++++-----------------------------
> index.rst | 1
> 2 files changed, 148 insertions(+), 126 deletions(-)
>
OK, I have applied this set. Thanks for doing this work!

jon