2021-03-05 01:04:15

by NeilBrown

[permalink] [raw]
Subject: [PATCH 0/7 v2] nfs-utils: provide audit-logging of NFSv4 access

This version improves the man-pages, usage messages, etc and
adds a new feature: logging client attach/detach based on
/proc/fs/nfsd/clients/
That feature is in patch 7 and should be considered RFC at this point.
i.e. patches 1-6 are ready to apply, patch 7 isn't.

A problem with patch7 is double-reporting due to the extra transient
entries in the clients directory. I'm wondering if we could add a
"Confirmed: Y" line for confirmed clients, so that the code could
ignore any clients which are not confirmed.
Maybe those clients should just not appear, but I'd like something
that will not produce noise on old kernels, and I'm happy if it
doesn't produce any log messages without a backport of a small
kernel patch.

Comments?

V2 series comment:

V1 of this series didn't update the usage() message for mountd,
and omited the required ':' after the 'T' sort-option. This
series fixes those two omissions.

Original series comment:

When NFSv3 is used mountd provides logs of successful and failed mount
attempts which can be used for auditing.
When NFSv4 is used there are no such logs as NFSv4 does not have a
distinct "mount" request.

However mountd still knows about which filesysytems are being accessed
from which clients, and can actually provide more reliable logs than it
currently does, though they must be more verbose - with periodic "is
being accessed" message replacing a single "was mounted" message.

This series adds support for that logging, and adds some related
improvements to make the logs as useful as possible.

NeilBrown

---

NeilBrown (7):
mountd: reject unknown client IP when !use_ipaddr.
mountd: Don't proactively add export info when fh info is requested.
mountd/exports: update man page
mountd: add logging for authentication results for accesses.
mountd: add --cache-use-ipaddr option to force use_ipaddr
mountd: make default ttl settable by option
mountd: add logging of NFSv4 clients attaching and detaching.


nfs.conf | 4 +
support/export/Makefile.am | 3 +-
support/export/auth.c | 4 +
support/export/cache.c | 41 +++++----
support/export/export.h | 9 +-
support/export/v4clients.c | 177 +++++++++++++++++++++++++++++++++++++
support/export/v4root.c | 3 +-
support/include/exportfs.h | 3 +-
support/nfs/exports.c | 4 +-
systemd/nfs.conf.man | 20 +++++
utils/exportd/exportd.c | 42 +++++++--
utils/exportd/exportd.man | 94 ++++++++++++++------
utils/mountd/mountd.c | 32 ++++++-
utils/mountd/mountd.h | 5 --
utils/mountd/mountd.man | 98 ++++++++++++++++----
utils/mountd/svc_run.c | 5 +-
16 files changed, 464 insertions(+), 80 deletions(-)
create mode 100644 support/export/v4clients.c

--
Signature


2021-03-05 01:04:16

by NeilBrown

[permalink] [raw]
Subject: [PATCH 6/7] mountd: make default ttl settable by option

From: NeilBrown <[email protected]>

The DEFAULT_TTL affects the rate at which authentication messages are
logged. So it is useful to make it settable.

Add "-ttl" and "-T", and add clear statement in the documentation of
both the benefits and the possible negative effects of choosing a larger
value

Signed-off-by: NeilBrown <[email protected]>
---
nfs.conf | 2 ++
support/export/cache.c | 6 +++---
support/export/v4root.c | 3 ++-
support/include/exportfs.h | 3 ++-
support/nfs/exports.c | 4 +++-
systemd/nfs.conf.man | 2 ++
utils/exportd/exportd.c | 24 ++++++++++++++++++++----
utils/exportd/exportd.man | 19 ++++++++++++++++---
utils/mountd/mountd.c | 20 ++++++++++++++++++--
utils/mountd/mountd.man | 19 ++++++++++++++++---
10 files changed, 84 insertions(+), 18 deletions(-)

diff --git a/nfs.conf b/nfs.conf
index 0c32eed1a5be..8b0da7aa47da 100644
--- a/nfs.conf
+++ b/nfs.conf
@@ -35,6 +35,7 @@
# state-directory-path=/var/lib/nfs
# threads=1
# cache-use-ipaddr=n
+# ttl=1800
[mountd]
# debug="all|auth|call|general|parse"
# manage-gids=n
@@ -45,6 +46,7 @@
# state-directory-path=/var/lib/nfs
# ha-callout=
# cache-use-ipaddr=n
+# ttl=1800
#
[nfsdcld]
# debug=0
diff --git a/support/export/cache.c b/support/export/cache.c
index 50f7c7a15ceb..c0848c3e437b 100644
--- a/support/export/cache.c
+++ b/support/export/cache.c
@@ -157,7 +157,7 @@ static void auth_unix_ip(int f)
bp = buf; blen = sizeof(buf);
qword_add(&bp, &blen, "nfsd");
qword_add(&bp, &blen, ipaddr);
- qword_adduint(&bp, &blen, time(0) + DEFAULT_TTL);
+ qword_adduint(&bp, &blen, time(0) + default_ttl);
if (use_ipaddr && client) {
memmove(ipaddr + 1, ipaddr, strlen(ipaddr) + 1);
ipaddr[0] = '$';
@@ -230,7 +230,7 @@ static void auth_unix_gid(int f)

bp = buf; blen = sizeof(buf);
qword_adduint(&bp, &blen, uid);
- qword_adduint(&bp, &blen, time(0) + DEFAULT_TTL);
+ qword_adduint(&bp, &blen, time(0) + default_ttl);
if (rv >= 0) {
qword_adduint(&bp, &blen, ngroups);
for (i=0; i<ngroups; i++)
@@ -968,7 +968,7 @@ static int dump_to_cache(int f, char *buf, int blen, char *domain,
ssize_t err;

if (ttl <= 1)
- ttl = DEFAULT_TTL;
+ ttl = default_ttl;

qword_add(&bp, &blen, domain);
qword_add(&bp, &blen, path);
diff --git a/support/export/v4root.c b/support/export/v4root.c
index 6f640aa9aa3f..3654bd7c10c0 100644
--- a/support/export/v4root.c
+++ b/support/export/v4root.c
@@ -45,7 +45,7 @@ static nfs_export pseudo_root = {
.e_nsqgids = 0,
.e_fsid = 0,
.e_mountpoint = NULL,
- .e_ttl = DEFAULT_TTL,
+ .e_ttl = 0,
},
.m_exported = 0,
.m_xtabent = 1,
@@ -84,6 +84,7 @@ v4root_create(char *path, nfs_export *export)
struct exportent *curexp = &export->m_export;

dupexportent(&eep, &pseudo_root.m_export);
+ eep.e_ttl = default_ttl;
eep.e_hostname = curexp->e_hostname;
strncpy(eep.e_path, path, sizeof(eep.e_path)-1);
if (strcmp(path, "/") != 0)
diff --git a/support/include/exportfs.h b/support/include/exportfs.h
index daa7e2a06d82..81d137210862 100644
--- a/support/include/exportfs.h
+++ b/support/include/exportfs.h
@@ -105,7 +105,8 @@ typedef struct mexport {
} nfs_export;

#define HASH_TABLE_SIZE 1021
-#define DEFAULT_TTL (30 * 60)
+
+extern int default_ttl;

typedef struct _exp_hash_entry {
nfs_export * p_first;
diff --git a/support/nfs/exports.c b/support/nfs/exports.c
index 037febd08d9b..2c8f0752ad9d 100644
--- a/support/nfs/exports.c
+++ b/support/nfs/exports.c
@@ -47,6 +47,8 @@ struct flav_info flav_map[] = {

const int flav_map_size = sizeof(flav_map)/sizeof(flav_map[0]);

+int default_ttl = 30 * 60;
+
static char *efname = NULL;
static XFILE *efp = NULL;
static int first;
@@ -100,7 +102,7 @@ static void init_exportent (struct exportent *ee, int fromkernel)
ee->e_nsquids = 0;
ee->e_nsqgids = 0;
ee->e_uuid = NULL;
- ee->e_ttl = DEFAULT_TTL;
+ ee->e_ttl = default_ttl;
}

struct exportent *
diff --git a/systemd/nfs.conf.man b/systemd/nfs.conf.man
index 8af4445d49c9..4dfb9293ca37 100644
--- a/systemd/nfs.conf.man
+++ b/systemd/nfs.conf.man
@@ -133,6 +133,7 @@ but on the server, this will resolve to the path
Recognized values:
.BR threads ,
.BR cache-use-upaddr ,
+.BR ttl ,
.BR state-directory-path

See
@@ -198,6 +199,7 @@ Recognized values:
.BR threads ,
.BR reverse-lookup ,
.BR cache-use-upaddr ,
+.BR ttl ,
.BR state-directory-path ,
.BR ha-callout .

diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c
index f2f209028284..76aad97375dc 100644
--- a/utils/exportd/exportd.c
+++ b/utils/exportd/exportd.c
@@ -46,9 +46,10 @@ static struct option longopts[] =
{ "num-threads", 1, 0, 't' },
{ "log-auth", 0, 0, 'l' },
{ "cache-use-ipaddr", 0, 0, 'i'},
+ { "ttl", 1, 0, 'T'},
{ NULL, 0, 0, 0 }
};
-static char shortopts[] = "d:fghs:t:li"
+static char shortopts[] = "d:fghs:t:liT:"

/*
* Signal handlers.
@@ -178,7 +179,7 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n"
-" [-g|--manage-gids] [-l|--log-auth] [-i|--cache-use-ipaddr]\n"
+" [-g|--manage-gids] [-l|--log-auth] [-i|--cache-use-ipaddr] [-T|--ttl ttl]\n"
" [-s|--state-directory-path path]\n"
" [-t num|--num-threads=num]\n", prog);
exit(n);
@@ -188,6 +189,7 @@ inline static void
read_exportd_conf(char *progname, char **argv)
{
char *s;
+ int ttl;

conf_init_file(NFS_CONFFILE);

@@ -201,14 +203,19 @@ read_exportd_conf(char *progname, char **argv)
s = conf_get_str("exportd", "state-directory-path");
if (s && !state_setup_basedir(argv[0], s))
exit(1);
+
+ ttl = conf_get_num("mountd", "ttl", default_ttl);
+ if (ttl > 0)
+ default_ttl = ttl;
}

int
main(int argc, char **argv)
{
char *progname;
- int foreground = 0;
- int c;
+ int foreground = 0;
+ int c;
+ int ttl;

/* Set the basename */
if ((progname = strrchr(argv[0], '/')) != NULL)
@@ -242,6 +249,15 @@ main(int argc, char **argv)
case 'i':
use_ipaddr = 2;
break;
+ case 'T':
+ ttl = atoi(optarg);
+ if (ttl <= 0) {
+ fprintf(stderr, "%s: bad ttl number of seconds: %s\n",
+ argv[0], optarg);
+ usage(argv[0], 1);
+ }
+ default_ttl = ttl;
+ break;
case 's':
if (!state_setup_basedir(argv[0], optarg))
exit(1);
diff --git a/utils/exportd/exportd.man b/utils/exportd/exportd.man
index a4e659f5fa4a..b238ff053272 100644
--- a/utils/exportd/exportd.man
+++ b/utils/exportd/exportd.man
@@ -34,9 +34,10 @@ Turn on debugging. Valid kinds are: all, auth, call, general and parse.
.TP
.BR \-l " or " \-\-log\-auth
Enable logging of responses to authentication and access requests from
-nfsd. Each response is then cached by the kernel for 30 minutes, and
-will be refreshed after 15 minutes if the relevant client remains
-active.
+nfsd. Each response is then cached by the kernel for 30 minutes (or as set by
+.B \-\-ttl
+below), and will be refreshed after 15 minutes (half the ttl time) if
+the relevant client remains active.
Note that
.B -l
is equivalent to
@@ -66,6 +67,17 @@ log messages produced by the
.B -l
option easier to read.
.TP
+.B \-T " or " \-\-ttl
+Provide a time-to-live (TTL) for cached information given to the kernel.
+The kernel will normally request an update if the information is needed
+after half of this time has expired. Increasing the provided number,
+which is in seconds, reduces the rate of cache update requests, and this
+is particularly noticeable when these requests are logged with
+.BR \-l .
+However increasing also means that changes to hostname to address
+mappings can take longer to be noticed.
+The default TTL is 1800 (30 minutes).
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP
@@ -107,6 +119,7 @@ Values recognized in the
.B [exportd]
section include
.B cache\-use\-ipaddr ,
+.BR ttl ,
.BR manage-gids ", and"
.B debug
which each have the same effect as the option with the same name.
diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c
index b9260aeb86a3..fce389661e7a 100644
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -76,9 +76,10 @@ static struct option longopts[] =
{ "no-udp", 0, 0, 'u' },
{ "log-auth", 0, 0, 'l'},
{ "cache-use-ipaddr", 0, 0, 'i'},
+ { "ttl", 1, 0, 'T'},
{ NULL, 0, 0, 0 }
};
-static char shortopts[] = "o:nFd:p:P:hH:N:V:vurs:t:gli";
+static char shortopts[] = "o:nFd:p:P:hH:N:V:vurs:t:gliT:";

#define NFSVERSBIT(vers) (0x1 << (vers - 1))
#define NFSVERSBIT_ALL (NFSVERSBIT(2) | NFSVERSBIT(3) | NFSVERSBIT(4))
@@ -672,6 +673,7 @@ inline static void
read_mountd_conf(char **argv)
{
char *s;
+ int ttl;

conf_init_file(NFS_CONFFILE);

@@ -706,6 +708,10 @@ read_mountd_conf(char **argv)
else
NFSCTL_VERUNSET(nfs_version, vers);
}
+
+ ttl = conf_get_num("mountd", "ttl", default_ttl);
+ if (ttl > 0)
+ default_ttl = ttl;
}

int
@@ -715,6 +721,7 @@ main(int argc, char **argv)
unsigned int listeners = 0;
int foreground = 0;
int c;
+ int ttl;
struct sigaction sa;
struct rlimit rlim;

@@ -809,6 +816,15 @@ main(int argc, char **argv)
case 'i':
use_ipaddr = 2;
break;
+ case 'T':
+ ttl = atoi(optarg);
+ if (ttl <= 0) {
+ fprintf(stderr, "%s: bad ttl number of seconds: %s\n",
+ argv[0], optarg);
+ usage(argv[0], 1);
+ }
+ default_ttl = ttl;
+ break;
case 0:
break;
case '?':
@@ -924,7 +940,7 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-F|--foreground] [-h|--help] [-v|--version] [-d kind|--debug kind]\n"
-" [-l|--log-auth] [-i|--cache-use-ipaddr]\n"
+" [-l|--log-auth] [-i|--cache-use-ipaddr] [-T|--ttl ttl]\n"
" [-o num|--descriptors num]\n"
" [-p|--port port] [-V version|--nfs-version version]\n"
" [-N version|--no-nfs-version version] [-n|--no-tcp]\n"
diff --git a/utils/mountd/mountd.man b/utils/mountd/mountd.man
index 97d4518fa2e6..1155cf94d35f 100644
--- a/utils/mountd/mountd.man
+++ b/utils/mountd/mountd.man
@@ -93,9 +93,10 @@ Turn on debugging. Valid kinds are: all, auth, call, general and parse.
.TP
.BR \-l " or " \-\-log\-auth
Enable logging of responses to authentication and access requests from
-nfsd. Each response is then cached by the kernel for 30 minutes, and
-will be refreshed after 15 minutes if the relevant client remains
-active.
+nfsd. Each response is then cached by the kernel for 30 minutes (or as set by
+.B \-\-ttl
+below), and will be refreshed after 15 minutes (half the ttl time) if
+the relevant client remains active.
Note that
.B -l
is equivalent to
@@ -129,6 +130,17 @@ log messages produced by the
.B -l
option easier to read.
.TP
+.B \-T " or " \-\-ttl
+Provide a time-to-live (TTL) for cached information given to the kernel.
+The kernel will normally request an update if the information is needed
+after half of this time has expired. Increasing the provided number,
+which is in seconds, reduces the rate of cache update requests, and this
+is particularly noticeable when these requests are logged with
+.BR \-l .
+However increasing also means that changes to hostname to address
+mappings can take longer to be noticed.
+The default TTL is 1800 (30 minutes).
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP
@@ -263,6 +275,7 @@ section include
.BR descriptors ,
.BR port ,
.BR threads ,
+.BR ttl ,
.BR reverse-lookup ", and"
.BR state-directory-path ,
.B ha-callout


2021-03-05 01:04:16

by NeilBrown

[permalink] [raw]
Subject: [PATCH 4/7] mountd: add logging for authentication results for accesses.

From: NeilBrown <[email protected]>

When NFSv3 is used to mount a filesystem, success/failure messages are
logged by mountd and can be used for auditing.
When NFSv4 is used, there is no distinct "MOUNT" request, and nothing is
logged.

We can instead log authentication requests from the kernel. These will
happen regularly - typically every 15 minutes of ongoing access - so
they may be too noisy, or might be more useful. As they might not be
wanted, make them selectable with the "AUTH" facility in xlog().

Add a "-l" to enable these logs. Alternately "debug = auth" will have
the same effect.

The same changes are made to both rpc.mountd and nfsv4.exportd.

Signed-off-by: NeilBrown <[email protected]>
---
support/export/cache.c | 18 +++++++++++++++++-
systemd/nfs.conf.man | 16 ++++++++++++++++
utils/exportd/exportd.c | 9 +++++++--
utils/exportd/exportd.man | 17 +++++++++++++++++
utils/mountd/mountd.c | 8 +++++++-
utils/mountd/mountd.man | 21 +++++++++++++++++++++
6 files changed, 85 insertions(+), 4 deletions(-)

diff --git a/support/export/cache.c b/support/export/cache.c
index 49a761749ec6..50f7c7a15ceb 100644
--- a/support/export/cache.c
+++ b/support/export/cache.c
@@ -145,6 +145,15 @@ static void auth_unix_ip(int f)
client = client_compose(ai);
nfs_freeaddrinfo(ai);
}
+ if (!client)
+ xlog(D_AUTH, "failed authentication for IP %s", ipaddr);
+ else if (!use_ipaddr)
+ xlog(D_AUTH, "successful authentication for IP %s as %s",
+ ipaddr, *client ? client : "DEFAULT");
+ else
+ xlog(D_AUTH, "successful authentication for IP %s",
+ ipaddr);
+
bp = buf; blen = sizeof(buf);
qword_add(&bp, &blen, "nfsd");
qword_add(&bp, &blen, ipaddr);
@@ -896,6 +905,8 @@ static void nfsd_fh(int f)
qword_addeol(&bp, &blen);
if (blen <= 0 || cache_write(f, buf, bp - buf) != bp - buf)
xlog(L_ERROR, "nfsd_fh: error writing reply");
+ if (!found)
+ xlog(D_AUTH, "denied access to %s", *dom == '$' ? dom+1 : dom);
out:
if (found_path)
free(found_path);
@@ -987,8 +998,13 @@ static int dump_to_cache(int f, char *buf, int blen, char *domain,
qword_add(&bp, &blen, "uuid");
qword_addhex(&bp, &blen, u, 16);
}
- } else
+ xlog(D_AUTH, "granted access to %s for %s",
+ path, *domain == '$' ? domain+1 : domain);
+ } else {
qword_adduint(&bp, &blen, now + ttl);
+ xlog(D_AUTH, "denied access to %s for %s",
+ path, *domain == '$' ? domain+1 : domain);
+ }
qword_addeol(&bp, &blen);
if (blen <= 0) {
errno = ENOBUFS;
diff --git a/systemd/nfs.conf.man b/systemd/nfs.conf.man
index d2187f8aca1a..8a02e154b1a2 100644
--- a/systemd/nfs.conf.man
+++ b/systemd/nfs.conf.man
@@ -138,6 +138,14 @@ See
.BR exportd (8)
for details.

+Note that setting
+.B "\[dq]debug = auth\[dq]"
+for
+.B exportd
+is equivalent to providing the
+.B \-\-log\-auth
+option.
+
.TP
.B nfsdcltrack
Recognized values:
@@ -197,6 +205,14 @@ section, are used to configure mountd. See
.BR rpc.mountd (8)
for details.

+Note that setting
+.B "\[dq]debug = auth\[dq]"
+for
+.B mountd
+is equivalent to providing the
+.B \-\-log\-auth
+option.
+
The
.B state-directory-path
value in the
diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c
index 0d7782becd51..8ea2f160773e 100644
--- a/utils/exportd/exportd.c
+++ b/utils/exportd/exportd.c
@@ -44,8 +44,10 @@ static struct option longopts[] =
{ "help", 0, 0, 'h' },
{ "manage-gids", 0, 0, 'g' },
{ "num-threads", 1, 0, 't' },
+ { "log-auth", 0, 0, 'l' },
{ NULL, 0, 0, 0 }
};
+static char shortopts[] = "d:fghs:t:l"

/*
* Signal handlers.
@@ -175,7 +177,7 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n"
-" [-g|--manage-gids]\n"
+" [-g|--manage-gids] [-l|--log-auth]\n"
" [-s|--state-directory-path path]\n"
" [-t num|--num-threads=num]\n", prog);
exit(n);
@@ -217,11 +219,14 @@ main(int argc, char **argv)
/* Read in config setting */
read_exportd_conf(progname, argv);

- while ((c = getopt_long(argc, argv, "d:fghs:t:", longopts, NULL)) != EOF) {
+ while ((c = getopt_long(argc, argv, shortopts, longopts, NULL)) != EOF) {
switch (c) {
case 'd':
xlog_sconfig(optarg, 1);
break;
+ case 'l':
+ xlog_sconfig("auth", 1);
+ break;
case 'f':
foreground++;
break;
diff --git a/utils/exportd/exportd.man b/utils/exportd/exportd.man
index 0dbf0c80466a..9435e98703e1 100644
--- a/utils/exportd/exportd.man
+++ b/utils/exportd/exportd.man
@@ -32,6 +32,23 @@ to respond to each request.
.B \-d kind " or " \-\-debug kind
Turn on debugging. Valid kinds are: all, auth, call, general and parse.
.TP
+.BR \-l " or " \-\-log\-auth
+Enable logging of responses to authentication and access requests from
+nfsd. Each response is then cached by the kernel for 30 minutes, and
+will be refreshed after 15 minutes if the relevant client remains
+active.
+Note that
+.B -l
+is equivalent to
+.B "-d auth"
+and so can be enabled in
+.B /etc/nfs.conf
+with
+.B "\[dq]debug = auth\[dq]"
+in the
+.B "[exportd]"
+section.
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP
diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c
index 612063ba2340..9fecf2f04c3b 100644
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -74,8 +74,10 @@ static struct option longopts[] =
{ "reverse-lookup", 0, 0, 'r' },
{ "manage-gids", 0, 0, 'g' },
{ "no-udp", 0, 0, 'u' },
+ { "log-auth", 0, 0, 'l'},
{ NULL, 0, 0, 0 }
};
+static char shortopts[] = "o:nFd:p:P:hH:N:V:vurs:t:gl";

#define NFSVERSBIT(vers) (0x1 << (vers - 1))
#define NFSVERSBIT_ALL (NFSVERSBIT(2) | NFSVERSBIT(3) | NFSVERSBIT(4))
@@ -727,7 +729,7 @@ main(int argc, char **argv)

/* Parse the command line options and arguments. */
opterr = 0;
- while ((c = getopt_long(argc, argv, "o:nFd:p:P:hH:N:V:vurs:t:g", longopts, NULL)) != EOF)
+ while ((c = getopt_long(argc, argv, shortopts, longopts, NULL)) != EOF)
switch (c) {
case 'g':
manage_gids = 1;
@@ -798,6 +800,9 @@ main(int argc, char **argv)
case 'u':
NFSCTL_UDPUNSET(_rpcprotobits);
break;
+ case 'l':
+ xlog_sconfig("auth", 1);
+ break;
case 0:
break;
case '?':
@@ -913,6 +918,7 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-F|--foreground] [-h|--help] [-v|--version] [-d kind|--debug kind]\n"
+" [-l|--log-auth]\n"
" [-o num|--descriptors num]\n"
" [-p|--port port] [-V version|--nfs-version version]\n"
" [-N version|--no-nfs-version version] [-n|--no-tcp]\n"
diff --git a/utils/mountd/mountd.man b/utils/mountd/mountd.man
index 2e191074c65f..f6d6fdddda95 100644
--- a/utils/mountd/mountd.man
+++ b/utils/mountd/mountd.man
@@ -91,6 +91,27 @@ is not updated to reflect any NFSv4 activity.
.B \-d kind " or " \-\-debug kind
Turn on debugging. Valid kinds are: all, auth, call, general and parse.
.TP
+.BR \-l " or " \-\-log\-auth
+Enable logging of responses to authentication and access requests from
+nfsd. Each response is then cached by the kernel for 30 minutes, and
+will be refreshed after 15 minutes if the relevant client remains
+active.
+Note that
+.B -l
+is equivalent to
+.B "-d auth"
+and so can be enabled in
+.B /etc/nfs.conf
+with
+.B "\[dq]debug = auth\[dq]"
+in the
+.B "[mountd]"
+section.
+.IP
+.B rpc.mountd
+will always log authentication responses to MOUNT requests when NFSv3 is
+used, but to get similar logs for NFSv4, this option is required.
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP


2021-03-05 01:04:39

by NeilBrown

[permalink] [raw]
Subject: [PATCH 1/7] mountd: reject unknown client IP when !use_ipaddr.

From: NeilBrown <[email protected]>

When use_ipaddr is not in effect, an auth_unix_ip lookup request from
the kernel for an unknown client will be rejected.
When it IS in effect, these requests are always granted with the IP
address being mapped to a string form of the address, preceded by a '$'.

This is inconsistent behaviour and could present a small information
leak.
It means that, for example, a SETCLIENT NFSv4 request may or may not
succeed depending on an internal setting in rpc.mountd.

This is easily rectified by always checking if the client is known.

Signed-off-by: NeilBrown <[email protected]>
---
support/export/cache.c | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/support/export/cache.c b/support/export/cache.c
index f1569afb558c..156ebfd4087c 100644
--- a/support/export/cache.c
+++ b/support/export/cache.c
@@ -114,6 +114,7 @@ static void auth_unix_ip(int f)
char class[20];
char ipaddr[INET6_ADDRSTRLEN + 1];
char *client = NULL;
+ struct addrinfo *ai = NULL;
struct addrinfo *tmp = NULL;
char buf[RPC_CHAN_BUF_SIZE], *bp;
int blen;
@@ -139,21 +140,17 @@ static void auth_unix_ip(int f)

auth_reload();

- /* addr is a valid, interesting address, find the domain name... */
- if (!use_ipaddr) {
- struct addrinfo *ai = NULL;
-
- ai = client_resolve(tmp->ai_addr);
- if (ai) {
- client = client_compose(ai);
- nfs_freeaddrinfo(ai);
- }
+ /* addr is a valid address, find the domain name... */
+ ai = client_resolve(tmp->ai_addr);
+ if (ai) {
+ client = client_compose(ai);
+ nfs_freeaddrinfo(ai);
}
bp = buf; blen = sizeof(buf);
qword_add(&bp, &blen, "nfsd");
qword_add(&bp, &blen, ipaddr);
qword_adduint(&bp, &blen, time(0) + DEFAULT_TTL);
- if (use_ipaddr) {
+ if (use_ipaddr && client) {
memmove(ipaddr + 1, ipaddr, strlen(ipaddr) + 1);
ipaddr[0] = '$';
qword_add(&bp, &blen, ipaddr);


2021-03-05 01:04:39

by NeilBrown

[permalink] [raw]
Subject: [PATCH 5/7] mountd: add --cache-use-ipaddr option to force use_ipaddr

From: NeilBrown <[email protected]>

When logging authentication requests, it can be easier to read the logs
if clients are always identified by IP address, not intermediate names
like netgroups or subnets.

To allow this, add --cache-use-ipaddr or -i which tell mountd to always
enable use_ipaddr.

Signed-off-by: NeilBrown <[email protected]>
---
nfs.conf | 2 ++
support/export/auth.c | 4 ++++
systemd/nfs.conf.man | 2 ++
utils/exportd/exportd.c | 16 +++++++++++-----
utils/exportd/exportd.man | 18 ++++++++++++++++++
utils/mountd/mountd.c | 10 ++++++++--
utils/mountd/mountd.man | 18 ++++++++++++++++++
7 files changed, 63 insertions(+), 7 deletions(-)

diff --git a/nfs.conf b/nfs.conf
index e69ec16d9c19..0c32eed1a5be 100644
--- a/nfs.conf
+++ b/nfs.conf
@@ -34,6 +34,7 @@
# manage-gids=n
# state-directory-path=/var/lib/nfs
# threads=1
+# cache-use-ipaddr=n
[mountd]
# debug="all|auth|call|general|parse"
# manage-gids=n
@@ -43,6 +44,7 @@
# reverse-lookup=n
# state-directory-path=/var/lib/nfs
# ha-callout=
+# cache-use-ipaddr=n
#
[nfsdcld]
# debug=0
diff --git a/support/export/auth.c b/support/export/auth.c
index 0bfa77d18469..cea376300d01 100644
--- a/support/export/auth.c
+++ b/support/export/auth.c
@@ -66,6 +66,10 @@ check_useipaddr(void)
int old_use_ipaddr = use_ipaddr;
unsigned int len = 0;

+ if (use_ipaddr > 1)
+ /* fixed - don't check */
+ return;
+
/* add length of m_hostname + 1 for the comma */
for (clp = clientlist[MCL_NETGROUP]; clp; clp = clp->m_next)
len += (strlen(clp->m_hostname) + 1);
diff --git a/systemd/nfs.conf.man b/systemd/nfs.conf.man
index 8a02e154b1a2..8af4445d49c9 100644
--- a/systemd/nfs.conf.man
+++ b/systemd/nfs.conf.man
@@ -132,6 +132,7 @@ but on the server, this will resolve to the path
.B exportd
Recognized values:
.BR threads ,
+.BR cache-use-upaddr ,
.BR state-directory-path

See
@@ -196,6 +197,7 @@ Recognized values:
.BR port ,
.BR threads ,
.BR reverse-lookup ,
+.BR cache-use-upaddr ,
.BR state-directory-path ,
.BR ha-callout .

diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c
index 8ea2f160773e..f2f209028284 100644
--- a/utils/exportd/exportd.c
+++ b/utils/exportd/exportd.c
@@ -45,9 +45,10 @@ static struct option longopts[] =
{ "manage-gids", 0, 0, 'g' },
{ "num-threads", 1, 0, 't' },
{ "log-auth", 0, 0, 'l' },
+ { "cache-use-ipaddr", 0, 0, 'i'},
{ NULL, 0, 0, 0 }
};
-static char shortopts[] = "d:fghs:t:l"
+static char shortopts[] = "d:fghs:t:li"

/*
* Signal handlers.
@@ -177,13 +178,13 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-f|--foreground] [-h|--help] [-d kind|--debug kind]\n"
-" [-g|--manage-gids] [-l|--log-auth]\n"
+" [-g|--manage-gids] [-l|--log-auth] [-i|--cache-use-ipaddr]\n"
" [-s|--state-directory-path path]\n"
" [-t num|--num-threads=num]\n", prog);
exit(n);
}

-inline static void
+inline static void
read_exportd_conf(char *progname, char **argv)
{
char *s;
@@ -194,6 +195,8 @@ read_exportd_conf(char *progname, char **argv)

manage_gids = conf_get_bool("exportd", "manage-gids", manage_gids);
num_threads = conf_get_num("exportd", "threads", num_threads);
+ if (conf_get_bool("mountd", "cache-use-ipaddr", 0))
+ use_ipaddr = 2;

s = conf_get_str("exportd", "state-directory-path");
if (s && !state_setup_basedir(argv[0], s))
@@ -236,6 +239,9 @@ main(int argc, char **argv)
case 'h':
usage(progname, 0);
break;
+ case 'i':
+ use_ipaddr = 2;
+ break;
case 's':
if (!state_setup_basedir(argv[0], optarg))
exit(1);
@@ -252,8 +258,8 @@ main(int argc, char **argv)

if (!setup_state_path_names(progname, ETAB, ETABTMP, ETABLCK, &etab))
return 1;
-
- if (!foreground)
+
+ if (!foreground)
xlog_stderr(0);

daemon_init(foreground);
diff --git a/utils/exportd/exportd.man b/utils/exportd/exportd.man
index 9435e98703e1..a4e659f5fa4a 100644
--- a/utils/exportd/exportd.man
+++ b/utils/exportd/exportd.man
@@ -49,6 +49,23 @@ in the
.B "[exportd]"
section.
.TP
+.BR \-i " or " \-\-cache\-use\-ipaddr
+Normally each client IP address is matched against each host identifier
+(name, wildcard, netgroup etc) found in
+.B /etc/exports
+and a combined identity is formed from all matching identifiers.
+Often many clients will map to the same combined identity so performing
+this mapping reduces the number of distinct access details that the
+kernel needs to store.
+Specifying the
+.B \-i
+option suppresses this mapping so that access to each filesystem is
+requested and cached separately for each client IP address. Doing this
+can increase the burden of updating the cache slightly, but can make the
+log messages produced by the
+.B -l
+option easier to read.
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP
@@ -89,6 +106,7 @@ configuration file.
Values recognized in the
.B [exportd]
section include
+.B cache\-use\-ipaddr ,
.BR manage-gids ", and"
.B debug
which each have the same effect as the option with the same name.
diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c
index 9fecf2f04c3b..b9260aeb86a3 100644
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -75,9 +75,10 @@ static struct option longopts[] =
{ "manage-gids", 0, 0, 'g' },
{ "no-udp", 0, 0, 'u' },
{ "log-auth", 0, 0, 'l'},
+ { "cache-use-ipaddr", 0, 0, 'i'},
{ NULL, 0, 0, 0 }
};
-static char shortopts[] = "o:nFd:p:P:hH:N:V:vurs:t:gl";
+static char shortopts[] = "o:nFd:p:P:hH:N:V:vurs:t:gli";

#define NFSVERSBIT(vers) (0x1 << (vers - 1))
#define NFSVERSBIT_ALL (NFSVERSBIT(2) | NFSVERSBIT(3) | NFSVERSBIT(4))
@@ -681,6 +682,8 @@ read_mountd_conf(char **argv)
num_threads = conf_get_num("mountd", "threads", num_threads);
reverse_resolve = conf_get_bool("mountd", "reverse-lookup", reverse_resolve);
ha_callout_prog = conf_get_str("mountd", "ha-callout");
+ if (conf_get_bool("mountd", "cache-use-ipaddr", 0))
+ use_ipaddr = 2;

s = conf_get_str("mountd", "state-directory-path");
if (s && !state_setup_basedir(argv[0], s))
@@ -803,6 +806,9 @@ main(int argc, char **argv)
case 'l':
xlog_sconfig("auth", 1);
break;
+ case 'i':
+ use_ipaddr = 2;
+ break;
case 0:
break;
case '?':
@@ -918,7 +924,7 @@ usage(const char *prog, int n)
{
fprintf(stderr,
"Usage: %s [-F|--foreground] [-h|--help] [-v|--version] [-d kind|--debug kind]\n"
-" [-l|--log-auth]\n"
+" [-l|--log-auth] [-i|--cache-use-ipaddr]\n"
" [-o num|--descriptors num]\n"
" [-p|--port port] [-V version|--nfs-version version]\n"
" [-N version|--no-nfs-version version] [-n|--no-tcp]\n"
diff --git a/utils/mountd/mountd.man b/utils/mountd/mountd.man
index f6d6fdddda95..97d4518fa2e6 100644
--- a/utils/mountd/mountd.man
+++ b/utils/mountd/mountd.man
@@ -112,6 +112,23 @@ section.
will always log authentication responses to MOUNT requests when NFSv3 is
used, but to get similar logs for NFSv4, this option is required.
.TP
+.BR \-i " or " \-\-cache\-use\-ipaddr
+Normally each client IP address is matched against each host identifier
+(name, wildcard, netgroup etc) found in
+.B /etc/exports
+and a combined identity is formed from all matching identifiers.
+Often many clients will map to the same combined identity so performing
+this mapping reduces the number of distinct access details that the
+kernel needs to store.
+Specifying the
+.B \-i
+option suppresses this mapping so that access to each filesystem is
+requested and cached separately for each client IP address. Doing this
+can increase the burden of updating the cache slightly, but can make the
+log messages produced by the
+.B -l
+option easier to read.
+.TP
.B \-F " or " \-\-foreground
Run in foreground (do not daemonize)
.TP
@@ -242,6 +259,7 @@ Values recognized in the
.B [mountd]
section include
.BR manage-gids ,
+.BR cache\-use\-ipaddr ,
.BR descriptors ,
.BR port ,
.BR threads ,


2021-03-05 01:04:39

by NeilBrown

[permalink] [raw]
Subject: [PATCH 7/7] mountd: add logging of NFSv4 clients attaching and detaching.

NFSv4 does not have a MOUNT request like NFSv3 does (via the MOUNT
protocol). So these cannot be logged.
NFSv4 does have SETCLIENTID and EXCHANGE_ID. These are indirectly
visible though changes in /proc/fs/nfsd/clients.
When a new client attaches, a directory appears. When the client
detaches, through a timeout (v4.0) or DESTROY_SESSION (v4.1+)
the directory disappears.

This patch adds tracking of these changes using inotify, with log
messages when a client attaches or detaches.

Unfortuantely clients are created in two steps, the second being a
confirmation. This results in a temporary client appearing and
disappearing. It is not possible (in Linux 5.10) to detect the
unconfirmed client, so extra attach/detach messages are generated.

This patch also moves some cache* function declarations into a header
file, and makes a few related changes to #includes.

Signed-off-by: NeilBrown <[email protected]>
---
support/export/Makefile.am | 3 -
support/export/cache.c | 9 --
support/export/export.h | 9 ++
support/export/v4clients.c | 177 ++++++++++++++++++++++++++++++++++++++++++++
utils/exportd/exportd.c | 1
utils/mountd/mountd.c | 2
utils/mountd/mountd.h | 5 -
utils/mountd/svc_run.c | 5 +
8 files changed, 195 insertions(+), 16 deletions(-)
create mode 100644 support/export/v4clients.c

diff --git a/support/export/Makefile.am b/support/export/Makefile.am
index a9e710c00ae8..eec737f66149 100644
--- a/support/export/Makefile.am
+++ b/support/export/Makefile.am
@@ -12,7 +12,8 @@ EXTRA_DIST = mount.x
noinst_LIBRARIES = libexport.a
libexport_a_SOURCES = client.c export.c hostname.c \
xtab.c mount_clnt.c mount_xdr.c \
- cache.c auth.c v4root.c fsloc.c
+ cache.c auth.c v4root.c fsloc.c \
+ v4clients.c
BUILT_SOURCES = $(GENFILES)

noinst_HEADERS = mount.h
diff --git a/support/export/cache.c b/support/export/cache.c
index c0848c3e437b..3e4f53c0a32e 100644
--- a/support/export/cache.c
+++ b/support/export/cache.c
@@ -42,13 +42,6 @@
#include "blkid/blkid.h"
#endif

-/*
- * Invoked by RPC service loop
- */
-void cache_set_fds(fd_set *fdset);
-int cache_process_req(fd_set *readfds);
-void cache_process_loop(void);
-
enum nfsd_fsid {
FSID_DEV = 0,
FSID_NUM,
@@ -1537,6 +1530,7 @@ void cache_process_loop(void)
for (;;) {

cache_set_fds(&readfds);
+ v4clients_set_fds(&readfds);

selret = select(FD_SETSIZE, &readfds,
(void *) 0, (void *) 0, (struct timeval *) 0);
@@ -1552,6 +1546,7 @@ void cache_process_loop(void)

default:
cache_process_req(&readfds);
+ v4clients_process(&readfds);
}
}
}
diff --git a/support/export/export.h b/support/export/export.h
index 4296db1ad9b1..8d5a0d3004ef 100644
--- a/support/export/export.h
+++ b/support/export/export.h
@@ -3,13 +3,14 @@
*
* support/export/export.h
*
- * Declarations for export support
+ * Declarations for export support
*/

#ifndef EXPORT_H
#define EXPORT_H

#include "nfslib.h"
+#include "exportfs.h"

unsigned int auth_reload(void);
nfs_export * auth_authenticate(const char *what,
@@ -17,8 +18,14 @@ nfs_export * auth_authenticate(const char *what,
const char *path);

void cache_open(void);
+void cache_set_fds(fd_set *fdset);
+int cache_process_req(fd_set *readfds);
void cache_process_loop(void);

+void v4clients_init(void);
+void v4clients_set_fds(fd_set *fdset);
+int v4clients_process(fd_set *fdset);
+
struct nfs_fh_len *
cache_get_filehandle(nfs_export *exp, int len, char *p);
int cache_export(nfs_export *exp, char *path);
diff --git a/support/export/v4clients.c b/support/export/v4clients.c
new file mode 100644
index 000000000000..ab1454e4c34e
--- /dev/null
+++ b/support/export/v4clients.c
@@ -0,0 +1,177 @@
+/*
+ * support/export/v4clients.c
+ *
+ * Montior clients appearing in, and disappearing from, /proc/fs/nfsd/clients
+ * and log relevant information.
+ */
+
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/inotify.h>
+#include <errno.h>
+#include "export.h"
+
+/* search.h declares 'struct entry' and nfs_prot.h
+ * does too. Easiest fix is to trick search.h into
+ * calling its struct "struct Entry".
+ */
+#define entry Entry
+#include <search.h>
+#undef entry
+
+static int clients_fd = -1;
+
+void v4clients_init(void)
+{
+ if (clients_fd >= 0)
+ return;
+ clients_fd = inotify_init1(IN_NONBLOCK);
+ if (clients_fd < 0) {
+ xlog_err("Unable to initialise v4clients watcher: %s\n",
+ strerror(errno));
+ return;
+ }
+ if (inotify_add_watch(clients_fd, "/proc/fs/nfsd/clients",
+ IN_CREATE | IN_DELETE) < 0) {
+ xlog_err("Unable to watch /proc/fs/nfsd/clients: %s\n",
+ strerror(errno));
+ close(clients_fd);
+ clients_fd = -1;
+ return;
+ }
+}
+
+void v4clients_set_fds(fd_set *fdset)
+{
+ if (clients_fd >= 0)
+ FD_SET(clients_fd, fdset);
+}
+
+static void *tree_root;
+
+struct ent {
+ unsigned long num;
+ char *clientid;
+ char *addr;
+ int vers;
+};
+
+static int ent_cmp(const void *av, const void *bv)
+{
+ const struct ent *a = av;
+ const struct ent *b = bv;
+
+ if (a->num < b->num)
+ return -1;
+ if (a->num > b->num)
+ return 1;
+ return 0;
+}
+
+static void free_ent(struct ent *ent)
+{
+ free(ent->clientid);
+ free(ent->addr);
+ free(ent);
+}
+
+static char *dup_line(char *line)
+{
+ char *ret;
+ char *e = strchr(line, '\n');
+ if (!e)
+ e = line + strlen(line);
+ ret = malloc(e - line + 1);
+ if (ret) {
+ memcpy(ret, line, e - line);
+ ret[e-line] = 0;
+ }
+ return ret;
+}
+
+static void add_id(int id)
+{
+ char buf[2048];
+ struct ent **ent;
+ struct ent *key;
+ char *path;
+ FILE *f;
+
+ asprintf(&path, "/proc/fs/nfsd/clients/%d/info", id);
+ f = fopen(path, "r");
+ if (!f) {
+ free(path);
+ return;
+ }
+ key = calloc(1, sizeof(*key));
+ if (!key) {
+ fclose(f);
+ free(path);
+ return;
+ }
+ key->num = id;
+ while (fgets(buf, sizeof(buf), f)) {
+ if (strncmp(buf, "clientid: ", 10) == 0)
+ key->clientid = dup_line(buf+10);
+ if (strncmp(buf, "address: ", 9) == 0)
+ key->addr = dup_line(buf+9);
+ if (strncmp(buf, "minor version: ", 15) == 0)
+ key->vers = atoi(buf+15);
+ }
+ fclose(f);
+ free(path);
+
+ xlog(L_NOTICE, "v4.%d client attached: %s from %s",
+ key->vers, key->clientid, key->addr);
+
+ ent = tsearch(key, &tree_root, ent_cmp);
+
+ if (!ent || *ent != key)
+ /* Already existed, or insertion failed */
+ free_ent(key);
+}
+
+static void del_id(int id)
+{
+ struct ent key = {.num = id};
+ struct ent **e, *ent;
+
+ e = tfind(&key, &tree_root, ent_cmp);
+ if (!e || !*e)
+ return;
+ ent = *e;
+ tdelete(ent, &tree_root, ent_cmp);
+ xlog(L_NOTICE, "v4.%d client detached: %s from %s",
+ ent->vers, ent->clientid, ent->addr);
+ free_ent(ent);
+}
+
+int v4clients_process(fd_set *fdset)
+{
+ char buf[4096] __attribute__((aligned(__alignof__(struct inotify_event))));
+ const struct inotify_event *ev;
+ ssize_t len;
+ char *ptr;
+
+ if (clients_fd < 0 ||
+ !FD_ISSET(clients_fd, fdset))
+ return 0;
+
+ while ((len = read(clients_fd, buf, sizeof(buf))) > 0) {
+ for (ptr = buf; ptr < buf + len;
+ ptr += sizeof(struct inotify_event) + ev->len) {
+ int id;
+ ev = (const struct inotify_event *)ptr;
+
+ id = atoi(ev->name);
+ if (id <= 0)
+ continue;
+ if (ev->mask & IN_CREATE)
+ add_id(id);
+ if (ev->mask & IN_DELETE)
+ del_id(id);
+ }
+ }
+ return 1;
+}
+
diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c
index 76aad97375dc..25c41be6bc77 100644
--- a/utils/exportd/exportd.c
+++ b/utils/exportd/exportd.c
@@ -297,6 +297,7 @@ main(int argc, char **argv)

/* Open files now to avoid sharing descriptors among forked processes */
cache_open();
+ v4clients_init();

/* Process incoming upcalls */
cache_process_loop();
diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c
index fce389661e7a..39e85fd53a87 100644
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -31,6 +31,7 @@
#include "pseudoflavors.h"
#include "nfsd_path.h"
#include "nfslib.h"
+#include "export.h"

extern void my_svc_run(void);

@@ -924,6 +925,7 @@ main(int argc, char **argv)
nfsd_path_init();
/* Open files now to avoid sharing descriptors among forked processes */
cache_open();
+ v4clients_init();

xlog(L_NOTICE, "Version " VERSION " starting");
my_svc_run();
diff --git a/utils/mountd/mountd.h b/utils/mountd/mountd.h
index f058f01d3584..d30775313f66 100644
--- a/utils/mountd/mountd.h
+++ b/utils/mountd/mountd.h
@@ -60,9 +60,4 @@ bool ipaddr_client_matches(nfs_export *exp, struct addrinfo *ai);
bool namelist_client_matches(nfs_export *exp, char *dom);
bool client_matches(nfs_export *exp, char *dom, struct addrinfo *ai);

-static inline bool is_ipaddr_client(char *dom)
-{
- return dom[0] == '$';
-}
-
#endif /* MOUNTD_H */
diff --git a/utils/mountd/svc_run.c b/utils/mountd/svc_run.c
index 41b96d7fd1c3..167b9757bde2 100644
--- a/utils/mountd/svc_run.c
+++ b/utils/mountd/svc_run.c
@@ -56,10 +56,9 @@
#ifdef HAVE_LIBTIRPC
#include <rpc/rpc_com.h>
#endif
+#include "export.h"

void my_svc_run(void);
-void cache_set_fds(fd_set *fdset);
-int cache_process_req(fd_set *readfds);

#if defined(__GLIBC__) && LONG_MAX != INT_MAX
/* bug in glibc 2.3.6 and earlier, we need
@@ -101,6 +100,7 @@ my_svc_run(void)

readfds = svc_fdset;
cache_set_fds(&readfds);
+ v4clients_set_fds(&readfds);

selret = select(FD_SETSIZE, &readfds,
(void *) 0, (void *) 0, (struct timeval *) 0);
@@ -116,6 +116,7 @@ my_svc_run(void)

default:
selret -= cache_process_req(&readfds);
+ selret -= v4clients_process(&readfds);
if (selret)
svc_getreqset(&readfds);
}


2021-03-05 01:04:39

by NeilBrown

[permalink] [raw]
Subject: [PATCH 3/7] mountd/exports: update man page

The text in the manpages about the export table is a bit outdated, and
doesn't mention the in-kernel cache which is an import part of
that table.

As a future patch will enable logging of updates to that cache, it is
important to have the caching behaviour documented. So update that
section of both man pages, and make a few other minor improvements.

Signed-off-by: NeilBrown <[email protected]>
---
utils/exportd/exportd.man | 46 +++++++++++++++++++++------------------------
utils/mountd/mountd.man | 46 ++++++++++++++++++++++++++++-----------------
2 files changed, 50 insertions(+), 42 deletions(-)

diff --git a/utils/exportd/exportd.man b/utils/exportd/exportd.man
index d788456244b2..0dbf0c80466a 100644
--- a/utils/exportd/exportd.man
+++ b/utils/exportd/exportd.man
@@ -10,30 +10,23 @@ nfsv4.exportd \- NFSv4 Server Mount Daemon
.SH DESCRIPTION
The
.B nfsv4.exportd
-is used to manage NFSv4 exports. The NFSv4 server
-receives a mount request from a client and pass it up to
-.B nfsv4.exportd.
-.B nfsv4.exportd
-then uses the exports(5) export
-table to verify the validity of the mount request.
-.PP
-An NFS server maintains a table of local physical file systems
-that are accessible to NFS clients.
-Each file system in this table is referred to as an
-.IR "exported file system" ,
-or
-.IR export ,
-for short.
-.PP
-Each file system in the export table has an access control list.
+is used to manage NFSv4 exports.
+The NFS server
+.RI ( nfsd )
+maintains a cache of authentication and authorization information which
+is used to identify the source of each requent, and then what access
+permissions that source has to any local filesystem. When required
+information is not found in the cache, the server sends a request to
.B nfsv4.exportd
-uses these access control lists to determine
-whether an NFS client is permitted to access a given file system.
-For details on how to manage your NFS server's export table, see the
-.BR exports (5)
-and
-.BR exportfs (8)
-man pages.
+to fill in the missing information.
+.B nfsv4.exportd
+uses a table of information stored in
+.B /var/lib/nfs/etab
+and maintained by
+.BR exportfs (8),
+possibly based on the contents of
+.BR exports (5),
+to respond to each request.
.SH OPTIONS
.TP
.B \-d kind " or " \-\-debug kind
@@ -46,7 +39,8 @@ Run in foreground (do not daemonize)
Display usage message.
.TP
.BR "\-t N" " or " "\-\-num\-threads=N " or " \-\-num\-threads N "
-This option specifies the number of worker threads that rpc.mountd
+This option specifies the number of worker threads that
+.B nfsv4.exports
spawns. The default is 1 thread, which is probably enough. More
threads are usually only needed for NFS servers which need to handle
mount storms of hundreds of NFS mounts in a few seconds, or when
@@ -94,4 +88,6 @@ listing exports, export options, and access control lists
.BR nfs.conf (5),
.BR firwall-cmd (1),
.sp
-RFC 3530 - "Network File System (NFS) version 4 Protocol"
+RFC 7530 - "Network File System (NFS) Version 4 Protocol"
+.br
+RFC 8881 - "Network File System (NFS) Version 4 Minor Version 1 Protocol"
diff --git a/utils/mountd/mountd.man b/utils/mountd/mountd.man
index 9978afcdb4cc..2e191074c65f 100644
--- a/utils/mountd/mountd.man
+++ b/utils/mountd/mountd.man
@@ -13,24 +13,24 @@ The
.B rpc.mountd
daemon implements the server side of the NFS MOUNT protocol,
an NFS side protocol used by NFS version 2 [RFC1094] and NFS version 3 [RFC1813].
+It also responds to requests from the Linux kernel to authenticate
+clients and provides details of access permissions.
.PP
-An NFS server maintains a table of local physical file systems
-that are accessible to NFS clients.
-Each file system in this table is referred to as an
-.IR "exported file system" ,
-or
-.IR export ,
-for short.
-.PP
-Each file system in the export table has an access control list.
-.B rpc.mountd
-uses these access control lists to determine
-whether an NFS client is permitted to access a given file system.
-For details on how to manage your NFS server's export table, see the
-.BR exports (5)
-and
-.BR exportfs (8)
-man pages.
+The NFS server
+.RI ( nfsd )
+maintains a cache of authentication and authorization information which
+is used to identify the source of each requent, and then what access
+permissions that source has to any local filesystem. When required
+information is not found in the cache, the server sends a request to
+.B mountd
+to fill in the missing information. Mountd uses a table of information
+stored in
+.B /var/lib/nfs/etab
+and maintained by
+.BR exportfs (8),
+possibly based on the contents of
+.BR exports (5),
+to respond to each request.
.SS Mounting exported NFS File Systems
The NFS MOUNT protocol has several procedures.
The most important of these are
@@ -78,6 +78,14 @@ A client may continue accessing an export even after invoking UMNT.
If the client reboots without sending a UMNT request, stale entries
remain for that client in
.IR /var/lib/nfs/rmtab .
+.SS Mounting File Systems with NFSv4
+Version 4 (and later) of NFS does not use a separate NFS MOUNT
+protocol. Instead mounting is performed using regular NFS requests
+handled by the NFS server in the Linux kernel
+.RI ( nfsd ).
+Consequently
+.I /var/lib/nfs/rmtab
+is not updated to reflect any NFSv4 activity.
.SH OPTIONS
.TP
.B \-d kind " or " \-\-debug kind
@@ -295,5 +303,9 @@ table of clients accessing server's exports
RFC 1094 - "NFS: Network File System Protocol Specification"
.br
RFC 1813 - "NFS Version 3 Protocol Specification"
+.br
+RFC 7530 - "Network File System (NFS) Version 4 Protocol"
+.br
+RFC 8881 - "Network File System (NFS) Version 4 Minor Version 1 Protocol"
.SH AUTHOR
Olaf Kirch, H. J. Lu, G. Allan Morris III, and a host of others.


2021-03-15 21:26:59

by Steve Dickson

[permalink] [raw]
Subject: Re: [PATCH 0/7 v2] nfs-utils: provide audit-logging of NFSv4 access



On 3/4/21 7:43 PM, NeilBrown wrote:
> This version improves the man-pages, usage messages, etc and
> adds a new feature: logging client attach/detach based on
> /proc/fs/nfsd/clients/
> That feature is in patch 7 and should be considered RFC at this point.
> i.e. patches 1-6 are ready to apply, patch 7 isn't.
>
> A problem with patch7 is double-reporting due to the extra transient
> entries in the clients directory. I'm wondering if we could add a
> "Confirmed: Y" line for confirmed clients, so that the code could
> ignore any clients which are not confirmed.
> Maybe those clients should just not appear, but I'd like something
> that will not produce noise on old kernels, and I'm happy if it
> doesn't produce any log messages without a backport of a small
> kernel patch.
>
> Comments?
Thank you for going the extra mile!

Series Committed... (tag: nfs-utils-2-5-4-rc1)

steved.

>
> V2 series comment:
>
> V1 of this series didn't update the usage() message for mountd,
> and omited the required ':' after the 'T' sort-option. This
> series fixes those two omissions.
>
> Original series comment:
>
> When NFSv3 is used mountd provides logs of successful and failed mount
> attempts which can be used for auditing.
> When NFSv4 is used there are no such logs as NFSv4 does not have a
> distinct "mount" request.
>
> However mountd still knows about which filesysytems are being accessed
> from which clients, and can actually provide more reliable logs than it
> currently does, though they must be more verbose - with periodic "is
> being accessed" message replacing a single "was mounted" message.
>
> This series adds support for that logging, and adds some related
> improvements to make the logs as useful as possible.
>
> NeilBrown
>
> ---
>
> NeilBrown (7):
> mountd: reject unknown client IP when !use_ipaddr.
> mountd: Don't proactively add export info when fh info is requested.
> mountd/exports: update man page
> mountd: add logging for authentication results for accesses.
> mountd: add --cache-use-ipaddr option to force use_ipaddr
> mountd: make default ttl settable by option
> mountd: add logging of NFSv4 clients attaching and detaching.
>
>
> nfs.conf | 4 +
> support/export/Makefile.am | 3 +-
> support/export/auth.c | 4 +
> support/export/cache.c | 41 +++++----
> support/export/export.h | 9 +-
> support/export/v4clients.c | 177 +++++++++++++++++++++++++++++++++++++
> support/export/v4root.c | 3 +-
> support/include/exportfs.h | 3 +-
> support/nfs/exports.c | 4 +-
> systemd/nfs.conf.man | 20 +++++
> utils/exportd/exportd.c | 42 +++++++--
> utils/exportd/exportd.man | 94 ++++++++++++++------
> utils/mountd/mountd.c | 32 ++++++-
> utils/mountd/mountd.h | 5 --
> utils/mountd/mountd.man | 98 ++++++++++++++++----
> utils/mountd/svc_run.c | 5 +-
> 16 files changed, 464 insertions(+), 80 deletions(-)
> create mode 100644 support/export/v4clients.c
>
> --
> Signature
>