2016-04-10 13:13:45

by James Simmons

[permalink] [raw]
Subject: [PATCH 00/17] staging : lustre : rest of missing patches from 2.5.0 release

This is the last collection of missing fixes present in the
Lustre 2.5.0 release. Once these are merged the upstream
client will be equal to the pre-2.6 lustre version since the
major of clio cleanups from that time frame have already
landed.

Andrew Perepechko (3):
staging: lustre: llite: variable rename in namei.c
staging: lustre: llite: speedup in unlink/rmdir
staging: lustre: lprocfs: implement log2 using bitops

Andriy Skulysh (1):
staging: lustre: lov: Don't wait for active target with OBD_STATFS_NODELAY

Bobi Jam (1):
staging: lustre: obd: MDT mount fails on MDS w/o MGS on it

Bruno Faccini (3):
staging: lustre: ldlm: Fix a race during FLock handling
staging: lustre: ldlm: refine LU-2665 patch for POSIX compliance
staging: lustre: lov: return minimal FIEMAP for released files

Jeff Mahoney (1):
staging: lustre: lloop: Fix build failure on ppc64

Jinshan Xiong (3):
staging: lustre: llite: error setting max_cache_mb at mount time
staging: lustre: llite: Truncate to restore file
staging: lustre: osc: osc_extent_wait() shouldn't be interruptible

John L. Hammond (2):
staging: lustre: hsm: permission checks for HSM ioctl operations
staging: lustre: hsm: don't use real suppgid

Niu Yawei (1):
staging: lustre: clio: incorrect assertions in 'enable-invariants'

Sebastien Buisson (1):
staging: lustre: osc: fix race issues thanks to oap_lock

Swapnil Pimpale (1):
staging: lustre: ptlrpc: return a meaningful status from ptlrpcd_init()

drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 15 +++--
drivers/staging/lustre/lustre/llite/dir.c | 3 +
drivers/staging/lustre/lustre/llite/file.c | 5 +-
.../staging/lustre/lustre/llite/llite_internal.h | 2 +-
drivers/staging/lustre/lustre/llite/llite_lib.c | 60 +++++++++----------
drivers/staging/lustre/lustre/llite/lloop.c | 3 -
drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 +-
drivers/staging/lustre/lustre/llite/namei.c | 65 ++++++++++----------
drivers/staging/lustre/lustre/llite/vvp_io.c | 5 +-
drivers/staging/lustre/lustre/lov/lov_obd.c | 21 ++++++
drivers/staging/lustre/lustre/lov/lov_request.c | 7 ++-
drivers/staging/lustre/lustre/mdc/mdc_locks.c | 4 +-
drivers/staging/lustre/lustre/mdc/mdc_request.c | 17 +++--
drivers/staging/lustre/lustre/mgc/mgc_request.c | 5 +-
drivers/staging/lustre/lustre/obdclass/cl_page.c | 5 +-
.../lustre/lustre/obdclass/lprocfs_status.c | 6 +-
drivers/staging/lustre/lustre/obdclass/obd_mount.c | 13 +++--
drivers/staging/lustre/lustre/osc/osc_cache.c | 10 +++-
drivers/staging/lustre/lustre/osc/osc_io.c | 2 +
drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 5 +-
20 files changed, 155 insertions(+), 102 deletions(-)


2016-04-10 13:13:48

by James Simmons

[permalink] [raw]
Subject: [PATCH 02/17] staging: lustre: clio: incorrect assertions in 'enable-invariants'

From: Niu Yawei <[email protected]>

Fixed several incorrect assumptions in 'enable-invariants'.

Signed-off-by: Niu Yawei <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3521
Reviewed-on: http://review.whamcloud.com/6832
Reviewed-by: Bobi Jam <[email protected]>
Reviewed-by: Jinshan Xiong <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/obdclass/cl_page.c | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/cl_page.c b/drivers/staging/lustre/lustre/obdclass/cl_page.c
index 8df39ce..03aab85 100644
--- a/drivers/staging/lustre/lustre/obdclass/cl_page.c
+++ b/drivers/staging/lustre/lustre/obdclass/cl_page.c
@@ -498,7 +498,7 @@ void cl_page_disown0(const struct lu_env *env,

state = pg->cp_state;
PINVRNT(env, pg, state == CPS_OWNED || state == CPS_FREEING);
- PINVRNT(env, pg, cl_page_invariant(pg));
+ PINVRNT(env, pg, cl_page_invariant(pg) || state == CPS_FREEING);
cl_page_owner_clear(pg);

if (state == CPS_OWNED)
@@ -670,7 +670,8 @@ EXPORT_SYMBOL(cl_page_unassume);
void cl_page_disown(const struct lu_env *env,
struct cl_io *io, struct cl_page *pg)
{
- PINVRNT(env, pg, cl_page_is_owned(pg, io));
+ PINVRNT(env, pg, cl_page_is_owned(pg, io) ||
+ pg->cp_state == CPS_FREEING);

io = cl_io_top(io);
cl_page_disown0(env, io, pg);
--
1.7.1

2016-04-10 13:13:55

by James Simmons

[permalink] [raw]
Subject: [PATCH 03/17] staging: lustre: ldlm: Fix a race during FLock handling

From: Bruno Faccini <[email protected]>

Protect against race where lock could have been just destroyed
due to overlap, in ldlm_process_flock_lock().
Easy reproducer is BULL's NFS Locktests in pthread mode.
(http://nfsv4.bullopensource.org/tools/tests/locktest.php)

Signed-off-by: Bruno Faccini <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-1126
Reviewed-on: http://review.whamcloud.com/7134
Reviewed-by: Oleg Drokin <[email protected]>
Reviewed-by: John L. Hammond <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/ldlm/ldlm_flock.c | 15 ++++++++++-----
1 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
index 3f97e1c..5102d78 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
@@ -520,11 +520,6 @@ ldlm_flock_completion_ast(struct ldlm_lock *lock, __u64 flags, void *data)
granted:
OBD_FAIL_TIMEOUT(OBD_FAIL_LDLM_CP_CB_WAIT, 10);

- if (lock->l_flags & LDLM_FL_DESTROYED) {
- LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
- return 0;
- }
-
if (lock->l_flags & LDLM_FL_FAILED) {
LDLM_DEBUG(lock, "client-side enqueue waking up: failed");
return -EIO;
@@ -534,6 +529,16 @@ granted:

lock_res_and_lock(lock);

+ /*
+ * Protect against race where lock could have been just destroyed
+ * due to overlap in ldlm_process_flock_lock().
+ */
+ if (lock->l_flags & LDLM_FL_DESTROYED) {
+ unlock_res_and_lock(lock);
+ LDLM_DEBUG(lock, "client-side enqueue waking up: destroyed");
+ return 0;
+ }
+
/* ldlm_lock_enqueue() has already placed lock on the granted list. */
list_del_init(&lock->l_res_link);

--
1.7.1

2016-04-10 13:14:02

by James Simmons

[permalink] [raw]
Subject: [PATCH 04/17] staging: lustre: ldlm: refine LU-2665 patch for POSIX compliance

From: Bruno Faccini <[email protected]>

Follow-on to patch introduced to fix LU-2665 ticket (Gerrit
Change at http://review.whamcloud.com/6415 with Change-Id:
I8faa331712abeadee46eabe111ee1c23a05840d5).
Original patch introduced regressions against POSIX test suite
(fcntl.18/fcntl.35 tests in LSB-VSX POSIX test suite at
http://www.opengroup.org/testing/linux-test/lsb-vsx.html),
so the idea is to only resend F_UNLCKs to have both LU-2665 bug
and POSIX test suite happy.

Signed-off-by: Bruno Faccini <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3701
Reviewed-on: http://review.whamcloud.com/7453
Reviewed-by: Bobi Jam <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/mdc/mdc_locks.c | 4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/drivers/staging/lustre/lustre/mdc/mdc_locks.c b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
index 958a164..01b6d77 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_locks.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_locks.c
@@ -869,7 +869,9 @@ resend:
* (explicits or automatically generated by Kernel to clean
* current FLocks upon exit) that can't be trashed
*/
- if ((rc == -EINTR) || (rc == -ETIMEDOUT))
+ if (((rc == -EINTR) || (rc == -ETIMEDOUT)) &&
+ (einfo->ei_type == LDLM_FLOCK) &&
+ (einfo->ei_mode == LCK_NL))
goto resend;
return rc;
}
--
1.7.1

2016-04-10 13:14:16

by James Simmons

[permalink] [raw]
Subject: [PATCH 10/17] staging: lustre: llite: Truncate to restore file

From: Jinshan Xiong <[email protected]>

Truncate up is safe so it won't trigger restore.

Copy optimization for truncate down - only copy the part under
truncate length. If a file is truncated to zero usually it'll be
followed by write so I choose to restore the file and set correct
stripe information.

Signed-off-by: Jinshan Xiong <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3817
Reviewed-on: http://review.whamcloud.com/7505
Reviewed-by: jacques-Charles Lafoucriere <[email protected]>
Reviewed-by: Henri Doreau <[email protected]>
Reviewed-by: Aurelien Degremont <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/file.c | 5 +-
.../staging/lustre/lustre/llite/llite_internal.h | 2 +-
drivers/staging/lustre/lustre/llite/llite_lib.c | 60 +++++++++----------
drivers/staging/lustre/lustre/llite/vvp_io.c | 5 +-
4 files changed, 35 insertions(+), 37 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 9b553d2..24fa24b 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -3621,7 +3621,7 @@ again:
/**
* This function send a restore request to the MDT
*/
-int ll_layout_restore(struct inode *inode)
+int ll_layout_restore(struct inode *inode, loff_t offset, __u64 length)
{
struct hsm_user_request *hur;
int len, rc;
@@ -3637,7 +3637,8 @@ int ll_layout_restore(struct inode *inode)
hur->hur_request.hr_flags = 0;
memcpy(&hur->hur_user_item[0].hui_fid, &ll_i2info(inode)->lli_fid,
sizeof(hur->hur_user_item[0].hui_fid));
- hur->hur_user_item[0].hui_extent.length = -1;
+ hur->hur_user_item[0].hui_extent.offset = offset;
+ hur->hur_user_item[0].hui_extent.length = length;
hur->hur_request.hr_itemcount = 1;
rc = obd_iocontrol(LL_IOC_HSM_REQUEST, ll_i2sbi(inode)->ll_md_exp,
len, hur, NULL);
diff --git a/drivers/staging/lustre/lustre/llite/llite_internal.h b/drivers/staging/lustre/lustre/llite/llite_internal.h
index d67c8c0..bb582f1 100644
--- a/drivers/staging/lustre/lustre/llite/llite_internal.h
+++ b/drivers/staging/lustre/lustre/llite/llite_internal.h
@@ -1377,7 +1377,7 @@ enum {

int ll_layout_conf(struct inode *inode, const struct cl_object_conf *conf);
int ll_layout_refresh(struct inode *inode, __u32 *gen);
-int ll_layout_restore(struct inode *inode);
+int ll_layout_restore(struct inode *inode, loff_t start, __u64 length);

int ll_xattr_init(void);
void ll_xattr_fini(void);
diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 9571742..49bfbbd 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -1266,14 +1266,6 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
LTIME_S(attr->ia_mtime), LTIME_S(attr->ia_ctime),
(s64)ktime_get_real_seconds());

- /* If we are changing file size, file content is modified, flag it. */
- if (attr->ia_valid & ATTR_SIZE) {
- attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
- spin_lock(&lli->lli_lock);
- lli->lli_flags |= LLIF_DATA_MODIFIED;
- spin_unlock(&lli->lli_lock);
- }
-
/* We always do an MDS RPC, even if we're only changing the size;
* only the MDS knows whether truncate() should fail with -ETXTBUSY
*/
@@ -1285,13 +1277,6 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
if (!S_ISDIR(inode->i_mode))
inode_unlock(inode);

- memcpy(&op_data->op_attr, attr, sizeof(*attr));
-
- /* Open epoch for truncate. */
- if (exp_connect_som(ll_i2mdexp(inode)) &&
- (attr->ia_valid & (ATTR_SIZE | ATTR_MTIME | ATTR_MTIME_SET)))
- op_data->op_flags = MF_EPOCH_OPEN;
-
/* truncate on a released file must failed with -ENODATA,
* so size must not be set on MDS for released file
* but other attributes must be set
@@ -1305,29 +1290,40 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
if (lsm && lsm->lsm_pattern & LOV_PATTERN_F_RELEASED)
file_is_released = true;
ccc_inode_lsm_put(inode, lsm);
+
+ if (!hsm_import && attr->ia_valid & ATTR_SIZE) {
+ if (file_is_released) {
+ rc = ll_layout_restore(inode, 0, attr->ia_size);
+ if (rc < 0)
+ goto out;
+
+ file_is_released = false;
+ ll_layout_refresh(inode, &gen);
+ }
+
+ /*
+ * If we are changing file size, file content is
+ * modified, flag it.
+ */
+ attr->ia_valid |= MDS_OPEN_OWNEROVERRIDE;
+ spin_lock(&lli->lli_lock);
+ lli->lli_flags |= LLIF_DATA_MODIFIED;
+ spin_unlock(&lli->lli_lock);
+ op_data->op_bias |= MDS_DATA_MODIFIED;
+ }
}

- /* if not in HSM import mode, clear size attr for released file
- * we clear the attribute send to MDT in op_data, not the original
- * received from caller in attr which is used later to
- * decide return code
- */
- if (file_is_released && (attr->ia_valid & ATTR_SIZE) && !hsm_import)
- op_data->op_attr.ia_valid &= ~ATTR_SIZE;
+ memcpy(&op_data->op_attr, attr, sizeof(*attr));
+
+ /* Open epoch for truncate. */
+ if (exp_connect_som(ll_i2mdexp(inode)) && !hsm_import &&
+ (attr->ia_valid & (ATTR_SIZE | ATTR_MTIME | ATTR_MTIME_SET)))
+ op_data->op_flags = MF_EPOCH_OPEN;

rc = ll_md_setattr(dentry, op_data, &mod);
if (rc)
goto out;

- /* truncate failed (only when non HSM import), others succeed */
- if (file_is_released) {
- if ((attr->ia_valid & ATTR_SIZE) && !hsm_import)
- rc = -ENODATA;
- else
- rc = 0;
- goto out;
- }
-
/* RPC to MDT is sent, cancel data modification flag */
if (op_data->op_bias & MDS_DATA_MODIFIED) {
spin_lock(&lli->lli_lock);
@@ -1336,7 +1332,7 @@ int ll_setattr_raw(struct dentry *dentry, struct iattr *attr, bool hsm_import)
}

ll_ioepoch_open(lli, op_data->op_ioepoch);
- if (!S_ISREG(inode->i_mode)) {
+ if (!S_ISREG(inode->i_mode) || file_is_released) {
rc = 0;
goto out;
}
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index aed7b8e..65d4637 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -295,6 +295,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)
struct cl_io *io = ios->cis_io;
struct cl_object *obj = io->ci_obj;
struct vvp_io *vio = cl2vvp_io(env, ios);
+ struct inode *inode = vvp_object_inode(obj);

CLOBINVRNT(env, obj, vvp_object_invariant(obj));

@@ -310,7 +311,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)
/* file was detected release, we need to restore it
* before finishing the io
*/
- rc = ll_layout_restore(vvp_object_inode(obj));
+ rc = ll_layout_restore(inode, 0, OBD_OBJECT_EOF);
/* if restore registration failed, no restart,
* we will return -ENODATA
*/
@@ -336,7 +337,7 @@ static void vvp_io_fini(const struct lu_env *env, const struct cl_io_slice *ios)
__u32 gen = 0;

/* check layout version */
- ll_layout_refresh(vvp_object_inode(obj), &gen);
+ ll_layout_refresh(inode, &gen);
io->ci_need_restart = vio->vui_layout_gen != gen;
if (io->ci_need_restart) {
CDEBUG(D_VFSTRACE,
--
1.7.1

2016-04-10 13:14:09

by James Simmons

[permalink] [raw]
Subject: [PATCH 09/17] staging: lustre: ptlrpc: return a meaningful status from ptlrpcd_init()

From: Swapnil Pimpale <[email protected]>

This patch has the following:
1) Fix for the return value from ptlrpcd_init(). It will now return a
correct status instead of returning zero always.
2) ptlrpcd_addref() should not increment ptlrpcd_users on error.
3) Added code in a mdc_setup() and mgc_setup() to test the return
value of ptlrpcd_addref() and return on error.

Signed-off-by: Swapnil Pimpale <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3808
Reviewed-on: http://review.whamcloud.com/7522
Reviewed-by: John L. Hammond <[email protected]>
Reviewed-by: Andreas Dilger <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/mdc/mdc_request.c | 9 ++++++---
drivers/staging/lustre/lustre/mgc/mgc_request.c | 5 ++++-
drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c | 5 ++++-
3 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index a089237..97f8669 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -2314,12 +2314,14 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg)
return -ENOMEM;
mdc_init_rpc_lock(cli->cl_rpc_lock);

- ptlrpcd_addref();
+ rc = ptlrpcd_addref();
+ if (rc < 0)
+ goto err_rpc_lock;

cli->cl_close_lock = kzalloc(sizeof(*cli->cl_close_lock), GFP_NOFS);
if (!cli->cl_close_lock) {
rc = -ENOMEM;
- goto err_rpc_lock;
+ goto err_ptlrpcd_decref;
}
mdc_init_rpc_lock(cli->cl_close_lock);

@@ -2345,9 +2347,10 @@ static int mdc_setup(struct obd_device *obd, struct lustre_cfg *cfg)

err_close_lock:
kfree(cli->cl_close_lock);
+err_ptlrpcd_decref:
+ ptlrpcd_decref();
err_rpc_lock:
kfree(cli->cl_rpc_lock);
- ptlrpcd_decref();
return rc;
}

diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index b7dc872..3f5f884 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -734,7 +734,9 @@ static int mgc_setup(struct obd_device *obd, struct lustre_cfg *lcfg)
struct task_struct *task;
int rc;

- ptlrpcd_addref();
+ rc = ptlrpcd_addref();
+ if (rc < 0)
+ goto err_noref;

rc = client_obd_setup(obd, lcfg);
if (rc)
@@ -773,6 +775,7 @@ err_cleanup:
client_obd_cleanup(obd);
err_decref:
ptlrpcd_decref();
+err_noref:
return rc;
}

diff --git a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
index dbc3376..76a355a 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c
@@ -909,8 +909,11 @@ int ptlrpcd_addref(void)
int rc = 0;

mutex_lock(&ptlrpcd_mutex);
- if (++ptlrpcd_users == 1)
+ if (++ptlrpcd_users == 1) {
rc = ptlrpcd_init();
+ if (rc < 0)
+ ptlrpcd_users--;
+ }
mutex_unlock(&ptlrpcd_mutex);
return rc;
}
--
1.7.1

2016-04-10 13:14:18

by James Simmons

[permalink] [raw]
Subject: [PATCH 11/17] staging: lustre: osc: osc_extent_wait() shouldn't be interruptible

From: Jinshan Xiong <[email protected]>

Otherwise it will hit the assertion at cl_lock.c:

cl_lock.c:1967:discard_cb())
ASSERTION( (!(page->cp_type == CPT_CACHEABLE) ||
(!PageWriteback(cl_page_vmpage(env, page)))) ) failed:

This is because in osc_lock_flush() we have to make sure the IO
is finished before discarding the pages.

Signed-off-by: Jinshan Xiong <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2779
Reviewed-on: http://review.whamcloud.com/5419
Reviewed-by: Bobi Jam <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/osc/osc_cache.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 846be76..a341160 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -965,7 +965,7 @@ static int osc_extent_wait(const struct lu_env *env, struct osc_extent *ext,
"%s: wait ext to %d timedout, recovery in progress?\n",
osc_export(obj)->exp_obd->obd_name, state);

- lwi = LWI_INTR(LWI_ON_SIGNAL_NOOP, NULL);
+ lwi = LWI_INTR(NULL, NULL);
rc = l_wait_event(ext->oe_waitq, extent_wait_cb(ext, state),
&lwi);
}
--
1.7.1

2016-04-10 13:14:25

by James Simmons

[permalink] [raw]
Subject: [PATCH 14/17] staging: lustre: lov: return minimal FIEMAP for released files

From: Bruno Faccini <[email protected]>

Since st_blocks = NULL is returned for released files, FIEMAP
should at least return a minimal mapping to make users aware
that file contains data but it is not immediately available.
This will make coreutils and tools such tar happy and have
them presume file is sparse.

Also, add a new test_228 in sanity-hsm to verify it works
for "[cp,tar] --sparse" commands.

Also fix a LBUG ("lov_fiemap()) ASSERTION( fm_local ) failed")
likely to occur when no-object/ENOMEM conditions and also now
when released.

Signed-off-by: Bruno Faccini <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3864
Reviewed-on: http://review.whamcloud.com/7584
Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by: Aurelien Degremont <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/lov/lov_obd.c | 21 +++++++++++++++++++++
1 files changed, 21 insertions(+), 0 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lov/lov_obd.c b/drivers/staging/lustre/lustre/lov/lov_obd.c
index 1a9e3e8..9b72671 100644
--- a/drivers/staging/lustre/lustre/lov/lov_obd.c
+++ b/drivers/staging/lustre/lustre/lov/lov_obd.c
@@ -1733,6 +1733,27 @@ static int lov_fiemap(struct lov_obd *lov, __u32 keylen, void *key,
unsigned int buffer_size = FIEMAP_BUFFER_SIZE;

if (!lsm_has_objects(lsm)) {
+ if (lsm && lsm_is_released(lsm) && (fm_key->fiemap.fm_start <
+ fm_key->oa.o_size)) {
+ /*
+ * released file, return a minimal FIEMAP if
+ * request fits in file-size.
+ */
+ fiemap->fm_mapped_extents = 1;
+ fiemap->fm_extents[0].fe_logical =
+ fm_key->fiemap.fm_start;
+ if (fm_key->fiemap.fm_start + fm_key->fiemap.fm_length <
+ fm_key->oa.o_size) {
+ fiemap->fm_extents[0].fe_length =
+ fm_key->fiemap.fm_length;
+ } else {
+ fiemap->fm_extents[0].fe_length =
+ fm_key->oa.o_size - fm_key->fiemap.fm_start;
+ fiemap->fm_extents[0].fe_flags |=
+ (FIEMAP_EXTENT_UNKNOWN |
+ FIEMAP_EXTENT_LAST);
+ }
+ }
rc = 0;
goto out;
}
--
1.7.1

2016-04-10 13:14:23

by James Simmons

[permalink] [raw]
Subject: [PATCH 15/17] staging: lustre: lov: Don't wait for active target with OBD_STATFS_NODELAY

From: Andriy Skulysh <[email protected]>

Patch for LU-631 which was landed before the upstream merge broke
OBD_STATFS_NODELAY behaviour. It adds unnecessary delay while
running df command with inactive OSTs. We shouldn't try to recover
connection to OST in this case.

Signed-off-by: Andriy Skulysh <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4010
Reviewed-on: http://review.whamcloud.com/7762
Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/lov/lov_request.c | 7 +++++--
1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
index 7178a02..475ca2f 100644
--- a/drivers/staging/lustre/lustre/lov/lov_request.c
+++ b/drivers/staging/lustre/lustre/lov/lov_request.c
@@ -716,12 +716,15 @@ int lov_prep_statfs_set(struct obd_device *obd, struct obd_info *oinfo,
struct lov_request *req;

if (!lov->lov_tgts[i] ||
- (!lov_check_and_wait_active(lov, i) &&
- (oinfo->oi_flags & OBD_STATFS_NODELAY))) {
+ (oinfo->oi_flags & OBD_STATFS_NODELAY &&
+ !lov->lov_tgts[i]->ltd_active)) {
CDEBUG(D_HA, "lov idx %d inactive\n", i);
continue;
}

+ if (!lov->lov_tgts[i]->ltd_active)
+ lov_check_and_wait_active(lov, i);
+
/* skip targets that have been explicitly disabled by the
* administrator
*/
--
1.7.1

2016-04-10 13:14:52

by James Simmons

[permalink] [raw]
Subject: [PATCH 17/17] staging: lustre: hsm: don't use real suppgid

From: John L. Hammond <[email protected]>

In the MDC HSM handlers that do not pack a real suppgid, use -1 rather
than 0 for the suppgid in mdt_body.

Signed-off-by: John L. Hammond <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3866
Reviewed-on: http://review.whamcloud.com/7565
Reviewed-by: Aurelien Degremont <[email protected]>
Reviewed-by: Faccini Bruno <[email protected]>
Reviewed-by: Jinshan Xiong <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/mdc/mdc_request.c | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lustre/mdc/mdc_request.c b/drivers/staging/lustre/lustre/mdc/mdc_request.c
index 97f8669..513290f 100644
--- a/drivers/staging/lustre/lustre/mdc/mdc_request.c
+++ b/drivers/staging/lustre/lustre/mdc/mdc_request.c
@@ -1169,7 +1169,7 @@ static int mdc_ioc_hsm_progress(struct obd_export *exp,
goto out;
}

- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);

/* Copy hsm_progress struct */
req_hpk = req_capsule_client_get(&req->rq_pill, &RMF_MDS_HSM_PROGRESS);
@@ -1203,7 +1203,7 @@ static int mdc_ioc_hsm_ct_register(struct obd_import *imp, __u32 archives)
goto out;
}

- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);

/* Copy hsm_progress struct */
archive_mask = req_capsule_client_get(&req->rq_pill,
@@ -1278,7 +1278,7 @@ static int mdc_ioc_hsm_ct_unregister(struct obd_import *imp)
goto out;
}

- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);

ptlrpc_request_set_replen(req);

@@ -1395,7 +1395,7 @@ static int mdc_ioc_hsm_request(struct obd_export *exp,
return rc;
}

- mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, 0, 0);
+ mdc_pack_body(req, NULL, OBD_MD_FLRMTPERM, 0, -1, 0);

/* Copy hsm_request struct */
req_hr = req_capsule_client_get(&req->rq_pill, &RMF_MDS_HSM_REQUEST);
--
1.7.1

2016-04-10 13:15:27

by James Simmons

[permalink] [raw]
Subject: [PATCH 16/17] staging: lustre: hsm: permission checks for HSM ioctl operations

From: John L. Hammond <[email protected]>

In the LL_IOC_HSM_CT_START case of ll_dir_ioctl() require
CAP_SYS_ADMIN, since the local handler for this ioctl may modify the
global KUC table.

Signed-off-by: John L. Hammond <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3866
Reviewed-on: http://review.whamcloud.com/7565
Reviewed-by: Aurelien Degremont <[email protected]>
Reviewed-by: Faccini Bruno <[email protected]>
Reviewed-by: Jinshan Xiong <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/dir.c | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 45bdf8f..94f3800 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -1844,6 +1844,9 @@ out_quotactl:
return rc;
}
case LL_IOC_HSM_CT_START:
+ if (!capable(CFS_CAP_SYS_ADMIN))
+ return -EPERM;
+
rc = copy_and_ioctl(cmd, sbi->ll_md_exp, (void __user *)arg,
sizeof(struct lustre_kernelcomm));
return rc;
--
1.7.1

2016-04-10 13:15:50

by James Simmons

[permalink] [raw]
Subject: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64

From: Jeff Mahoney <[email protected]>

On ppc64 with 64k pages, we get a build failure in lloop:

drivers/staging/lustre/lustre/llite/lloop.c:527:2:
note: in expansion of macro 'CLASSERT'
CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));

There's no need to change the queue's logical block size. Even if it could
accept a 64k value, that would result in any file system on top of it
needing to also use 64k blocks. It'd be safe to set it to 4k, but there's
no actual need for it. It's not used to split requests except for WRITE_SAME,
which lloop doesn't implement anyway.

Signed-off-by: Jeff Mahoney <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4000
Reviewed-on: http://review.whamcloud.com/7745
Reviewed-by: Jinshan Xiong <[email protected]>
Reviewed-by: Minh Diep <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/lloop.c | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
index b725fc1..f396753 100644
--- a/drivers/staging/lustre/lustre/llite/lloop.c
+++ b/drivers/staging/lustre/lustre/llite/lloop.c
@@ -525,9 +525,6 @@ static int loop_set_fd(struct lloop_device *lo, struct file *unused,
lo->lo_queue->queuedata = lo;

/* queue parameters */
- CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
- blk_queue_logical_block_size(lo->lo_queue,
- (unsigned short)PAGE_CACHE_SIZE);
blk_queue_max_hw_sectors(lo->lo_queue,
LLOOP_MAX_SEGMENTS << (PAGE_CACHE_SHIFT - 9));
blk_queue_max_segments(lo->lo_queue, LLOOP_MAX_SEGMENTS);
--
1.7.1

2016-04-10 13:16:10

by James Simmons

[permalink] [raw]
Subject: [PATCH 12/17] staging: lustre: lprocfs: implement log2 using bitops

From: Andrew Perepechko <[email protected]>

This patch implements log2 using fls.

Signed-off-by: Andrew Perepechko <[email protected]>
Reviewed-by: Alexander Boyko <[email protected]>
Reviewed-by: [email protected]
Reviewed-by: Vitaly Fertman <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3496
Xyratex-bug-id: MRP-999
Reviewed-on: http://review.whamcloud.com/6757
Reviewed-by: John L. Hammond <[email protected]>
Reviewed-by: Bob Glossman <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
.../lustre/lustre/obdclass/lprocfs_status.c | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
index d93f42f..172fc91 100644
--- a/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
+++ b/drivers/staging/lustre/lustre/obdclass/lprocfs_status.c
@@ -1471,10 +1471,10 @@ EXPORT_SYMBOL(lprocfs_oh_tally);

void lprocfs_oh_tally_log2(struct obd_histogram *oh, unsigned int value)
{
- unsigned int val;
+ unsigned int val = 0;

- for (val = 0; ((1 << val) < value) && (val <= OBD_HIST_MAX); val++)
- ;
+ if (likely(value != 0))
+ val = min(fls(value - 1), OBD_HIST_MAX);

lprocfs_oh_tally(oh, val);
}
--
1.7.1

2016-04-10 13:14:05

by James Simmons

[permalink] [raw]
Subject: [PATCH 07/17] staging: lustre: llite: error setting max_cache_mb at mount time

From: Jinshan Xiong <[email protected]>

The root cause is that when max_cache_mb conf parameter is applied,
the client isn't connected to the OST yet so that sbi->ll_dt_exp is
NULL. However, it's not necessary to shrink the cache memory in this
case so success should be returned.

Signed-off-by: Jinshan Xiong <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3676
Reviewed-on: http://review.whamcloud.com/7194
Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by: Bobi Jam <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/lproc_llite.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/lproc_llite.c b/drivers/staging/lustre/lustre/llite/lproc_llite.c
index 091144f..6947913 100644
--- a/drivers/staging/lustre/lustre/llite/lproc_llite.c
+++ b/drivers/staging/lustre/lustre/llite/lproc_llite.c
@@ -461,8 +461,8 @@ static ssize_t ll_max_cached_mb_seq_write(struct file *file,
break;

if (!sbi->ll_dt_exp) { /* being initialized */
- rc = -ENODEV;
- break;
+ rc = 0;
+ goto out;
}

/* difficult - have to ask OSCs to drop LRU slots. */
--
1.7.1

2016-04-10 13:16:59

by James Simmons

[permalink] [raw]
Subject: [PATCH 08/17] staging: lustre: obd: MDT mount fails on MDS w/o MGS on it

From: Bobi Jam <[email protected]>

If we specify multiple --mgsnode for a MDT, when we start MDS upon
it while MGS is no the other node, the MGC import connection will
always select the local nid (which is one of the candidate mgsnode)
since it think its the closest connection.

This patch treats further --mgsnode nids as failover nids, so that
multiple import connections are added for the MGC import.

Signed-off-by: Bobi Jam <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3829
Reviewed-on: http://review.whamcloud.com/7509
Reviewed-by: Liang Zhen <[email protected]>
Reviewed-by: Lai Siyao <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/obdclass/obd_mount.c | 13 ++++++++-----
1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/obdclass/obd_mount.c b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
index d3e28a3..9474aa9 100644
--- a/drivers/staging/lustre/lustre/obdclass/obd_mount.c
+++ b/drivers/staging/lustre/lustre/obdclass/obd_mount.c
@@ -307,7 +307,8 @@ int lustre_start_mgc(struct super_block *sb)
while (class_parse_nid(ptr, &nid, &ptr) == 0) {
rc = do_lcfg(mgcname, nid,
LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
- i++;
+ if (!rc)
+ i++;
/* Stop at the first failover nid */
if (*ptr == ':')
break;
@@ -345,16 +346,18 @@ int lustre_start_mgc(struct super_block *sb)
sprintf(niduuid, "%s_%x", mgcname, i);
j = 0;
while (class_parse_nid_quiet(ptr, &nid, &ptr) == 0) {
- j++;
- rc = do_lcfg(mgcname, nid,
- LCFG_ADD_UUID, niduuid, NULL, NULL, NULL);
+ rc = do_lcfg(mgcname, nid, LCFG_ADD_UUID, niduuid,
+ NULL, NULL, NULL);
+ if (!rc)
+ ++j;
if (*ptr == ':')
break;
}
if (j > 0) {
rc = do_lcfg(mgcname, 0, LCFG_ADD_CONN,
niduuid, NULL, NULL, NULL);
- i++;
+ if (!rc)
+ i++;
} else {
/* at ":/fsname" */
break;
--
1.7.1

2016-04-10 13:17:18

by James Simmons

[permalink] [raw]
Subject: [PATCH 06/17] staging: lustre: llite: speedup in unlink/rmdir

From: Andrew Perepechko <[email protected]>

Assume dchild argument is fully initialized in ->unlink and
->rmdir callbacks, so additional lookup for ELC is not needed.

Signed-off-by: Andrew Perepechko <[email protected]>
Reviewed-by: Alexander Boyko <[email protected]>
Reviewed-by: Vitaly Fertman <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3473
Xyratex-bug-id: MRP-1027
Reviewed-on: http://review.whamcloud.com/6648
Reviewed-by: Fan Yong <[email protected]>
Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/namei.c | 21 +++++++++++----------
1 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index 6445bc1..7e4247a 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -845,12 +845,6 @@ static int ll_create_nd(struct inode *dir, struct dentry *dentry,
return rc;
}

-static inline void ll_get_child_fid(struct dentry *child, struct lu_fid *fid)
-{
- if (d_really_is_positive(child))
- *fid = *ll_inode2fid(d_inode(child));
-}
-
int ll_objects_destroy(struct ptlrpc_request *request, struct inode *dir)
{
struct mdt_body *body;
@@ -943,7 +937,9 @@ static int ll_unlink(struct inode *dir, struct dentry *dchild)
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(dchild, &op_data->op_fid3);
+ if (dchild && dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(dchild->d_inode);
+
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -993,7 +989,9 @@ static int ll_rmdir(struct inode *dir, struct dentry *dchild)
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(dchild, &op_data->op_fid3);
+ if (dchild && dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(dchild->d_inode);
+
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -1074,8 +1072,11 @@ static int ll_rename(struct inode *src, struct dentry *src_dchild,
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(src_dchild, &op_data->op_fid3);
- ll_get_child_fid(tgt_dchild, &op_data->op_fid4);
+ if (src_dchild && src_dchild->d_inode)
+ op_data->op_fid3 = *ll_inode2fid(src_dchild->d_inode);
+ if (tgt_dchild && tgt_dchild->d_inode)
+ op_data->op_fid4 = *ll_inode2fid(tgt_dchild->d_inode);
+
err = md_rename(sbi->ll_md_exp, op_data,
src_dchild->d_name.name,
src_dchild->d_name.len,
--
1.7.1

2016-04-10 13:17:52

by James Simmons

[permalink] [raw]
Subject: [PATCH 05/17] staging: lustre: llite: variable rename in namei.c

From: Andrew Perepechko <[email protected]>

With the patch 6648 a fee variables were renamed. We do these
renames in broken out patch to the fix obvious.

Signed-off-by: Andrew Perepechko <[email protected]>
Reviewed-by: Alexander Boyko <[email protected]>
Reviewed-by: Vitaly Fertman <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-3473
Xyratex-bug-id: MRP-1027
Reviewed-on: http://review.whamcloud.com/6648
Reviewed-by: Fan Yong <[email protected]>
Reviewed-by: Andreas Dilger <[email protected]>
Reviewed-by: Oleg Drokin <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/llite/namei.c | 52 +++++++++++++-------------
1 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/namei.c b/drivers/staging/lustre/lustre/llite/namei.c
index f8f98e4..6445bc1 100644
--- a/drivers/staging/lustre/lustre/llite/namei.c
+++ b/drivers/staging/lustre/lustre/llite/namei.c
@@ -927,23 +927,23 @@ out:
* is any lock existing. They will recycle dentries and inodes based upon locks
* too. b=20433
*/
-static int ll_unlink(struct inode *dir, struct dentry *dentry)
+static int ll_unlink(struct inode *dir, struct dentry *dchild)
{
struct ptlrpc_request *request = NULL;
struct md_op_data *op_data;
int rc;

CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p)\n",
- dentry, dir->i_ino, dir->i_generation, dir);
+ dchild, dir->i_ino, dir->i_generation, dir);

op_data = ll_prep_md_op_data(NULL, dir, NULL,
- dentry->d_name.name,
- dentry->d_name.len,
+ dchild->d_name.name,
+ dchild->d_name.len,
0, LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(dentry, &op_data->op_fid3);
+ ll_get_child_fid(dchild, &op_data->op_fid3);
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -977,23 +977,23 @@ static int ll_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
return err;
}

-static int ll_rmdir(struct inode *dir, struct dentry *dentry)
+static int ll_rmdir(struct inode *dir, struct dentry *dchild)
{
struct ptlrpc_request *request = NULL;
struct md_op_data *op_data;
int rc;

CDEBUG(D_VFSTRACE, "VFS Op:name=%pd,dir=%lu/%u(%p)\n",
- dentry, dir->i_ino, dir->i_generation, dir);
+ dchild, dir->i_ino, dir->i_generation, dir);

op_data = ll_prep_md_op_data(NULL, dir, NULL,
- dentry->d_name.name,
- dentry->d_name.len,
+ dchild->d_name.name,
+ dchild->d_name.len,
S_IFDIR, LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(dentry, &op_data->op_fid3);
+ ll_get_child_fid(dchild, &op_data->op_fid3);
op_data->op_fid2 = op_data->op_fid3;
rc = md_unlink(ll_i2sbi(dir)->ll_md_exp, op_data, &request);
ll_finish_md_op_data(op_data);
@@ -1056,42 +1056,42 @@ out:
return err;
}

-static int ll_rename(struct inode *old_dir, struct dentry *old_dentry,
- struct inode *new_dir, struct dentry *new_dentry)
+static int ll_rename(struct inode *src, struct dentry *src_dchild,
+ struct inode *tgt, struct dentry *tgt_dchild)
{
struct ptlrpc_request *request = NULL;
- struct ll_sb_info *sbi = ll_i2sbi(old_dir);
+ struct ll_sb_info *sbi = ll_i2sbi(src);
struct md_op_data *op_data;
int err;

CDEBUG(D_VFSTRACE,
"VFS Op:oldname=%pd,src_dir=%lu/%u(%p),newname=%pd,tgt_dir=%lu/%u(%p)\n",
- old_dentry, old_dir->i_ino, old_dir->i_generation, old_dir,
- new_dentry, new_dir->i_ino, new_dir->i_generation, new_dir);
+ src_dchild, src->i_ino, src->i_generation, src,
+ tgt_dchild, tgt->i_ino, tgt->i_generation, tgt);

- op_data = ll_prep_md_op_data(NULL, old_dir, new_dir, NULL, 0, 0,
+ op_data = ll_prep_md_op_data(NULL, src, tgt, NULL, 0, 0,
LUSTRE_OPC_ANY, NULL);
if (IS_ERR(op_data))
return PTR_ERR(op_data);

- ll_get_child_fid(old_dentry, &op_data->op_fid3);
- ll_get_child_fid(new_dentry, &op_data->op_fid4);
+ ll_get_child_fid(src_dchild, &op_data->op_fid3);
+ ll_get_child_fid(tgt_dchild, &op_data->op_fid4);
err = md_rename(sbi->ll_md_exp, op_data,
- old_dentry->d_name.name,
- old_dentry->d_name.len,
- new_dentry->d_name.name,
- new_dentry->d_name.len, &request);
+ src_dchild->d_name.name,
+ src_dchild->d_name.len,
+ tgt_dchild->d_name.name,
+ tgt_dchild->d_name.len, &request);
ll_finish_md_op_data(op_data);
if (!err) {
- ll_update_times(request, old_dir);
- ll_update_times(request, new_dir);
+ ll_update_times(request, src);
+ ll_update_times(request, tgt);
ll_stats_ops_tally(sbi, LPROC_LL_RENAME, 1);
- err = ll_objects_destroy(request, old_dir);
+ err = ll_objects_destroy(request, src);
}

ptlrpc_req_finished(request);
if (!err)
- d_move(old_dentry, new_dentry);
+ d_move(src_dchild, tgt_dchild);
return err;
}

--
1.7.1

2016-04-10 13:18:15

by James Simmons

[permalink] [raw]
Subject: [PATCH 01/17] staging: lustre: osc: fix race issues thanks to oap_lock

From: Sebastien Buisson <[email protected]>

Fix 'data race condition' defects found by Coverity version 6.5.0:

Data race condition (MISSING_LOCK)

Accessing variable without holding lock. Elsewhere,
this variable is accessed with lock held.

This patch is dedicated to code fragments involving oap_lock.

Signed-off-by: Sebastien Buisson <[email protected]>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2744
Reviewed-on: http://review.whamcloud.com/6572
Reviewed-by: Oleg Drokin <[email protected]>
Reviewed-by: Andreas Dilger <[email protected]>
Signed-off-by: James Simmons <[email protected]>
---
drivers/staging/lustre/lustre/osc/osc_cache.c | 8 ++++++++
drivers/staging/lustre/lustre/osc/osc_io.c | 2 ++
2 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 956d57b..846be76 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -1147,7 +1147,9 @@ static int osc_extent_make_ready(const struct lu_env *env,
last->oap_count = osc_refresh_count(env, last, OBD_BRW_WRITE);
LASSERT(last->oap_count > 0);
LASSERT(last->oap_page_off + last->oap_count <= PAGE_CACHE_SIZE);
+ spin_lock(&last->oap_lock);
last->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&last->oap_lock);
}

/* for the rest of pages, we don't need to call osf_refresh_count()
@@ -1156,7 +1158,9 @@ static int osc_extent_make_ready(const struct lu_env *env,
list_for_each_entry(oap, &ext->oe_pages, oap_pending_item) {
if (!(oap->oap_async_flags & ASYNC_COUNT_STABLE)) {
oap->oap_count = PAGE_CACHE_SIZE - oap->oap_page_off;
+ spin_lock(&last->oap_lock);
oap->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&last->oap_lock);
}
}

@@ -2350,6 +2354,10 @@ int osc_queue_async_io(const struct lu_env *env, struct cl_io *io,
oap->oap_cmd = cmd;
oap->oap_page_off = ops->ops_from;
oap->oap_count = ops->ops_to - ops->ops_from;
+ /*
+ * No need to hold a lock here,
+ * since this page is not in any list yet.
+ */
oap->oap_async_flags = 0;
oap->oap_brw_flags = brw_flags;

diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
index cf7743d..0f50465 100644
--- a/drivers/staging/lustre/lustre/osc/osc_io.c
+++ b/drivers/staging/lustre/lustre/osc/osc_io.c
@@ -168,8 +168,10 @@ static int osc_io_submit(const struct lu_env *env,
}

cl_page_list_move(qout, qin, page);
+ spin_lock(&oap->oap_lock);
oap->oap_async_flags = ASYNC_URGENT|ASYNC_READY;
oap->oap_async_flags |= ASYNC_COUNT_STABLE;
+ spin_unlock(&oap->oap_lock);

osc_page_submit(env, opg, crt, brw_flags);
list_add_tail(&oap->oap_pending_item, &list);
--
1.7.1

2016-04-10 13:23:53

by Jeff Mahoney

[permalink] [raw]
Subject: Re: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64

This patch was shown not to work. I just haven't removed it from opensuse yet.

-Jeff

--
Jeff Mahoney
(apologies for the top post -- from my mobile)

> On Apr 10, 2016, at 9:13 AM, James Simmons <[email protected]> wrote:
>
> From: Jeff Mahoney <[email protected]>
>
> On ppc64 with 64k pages, we get a build failure in lloop:
>
> drivers/staging/lustre/lustre/llite/lloop.c:527:2:
> note: in expansion of macro 'CLASSERT'
> CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
>
> There's no need to change the queue's logical block size. Even if it could
> accept a 64k value, that would result in any file system on top of it
> needing to also use 64k blocks. It'd be safe to set it to 4k, but there's
> no actual need for it. It's not used to split requests except for WRITE_SAME,
> which lloop doesn't implement anyway.
>
> Signed-off-by: Jeff Mahoney <[email protected]>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4000
> Reviewed-on: http://review.whamcloud.com/7745
> Reviewed-by: Jinshan Xiong <[email protected]>
> Reviewed-by: Minh Diep <[email protected]>
> Reviewed-by: Oleg Drokin <[email protected]>
> Signed-off-by: James Simmons <[email protected]>
> ---
> drivers/staging/lustre/lustre/llite/lloop.c | 3 ---
> 1 files changed, 0 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
> index b725fc1..f396753 100644
> --- a/drivers/staging/lustre/lustre/llite/lloop.c
> +++ b/drivers/staging/lustre/lustre/llite/lloop.c
> @@ -525,9 +525,6 @@ static int loop_set_fd(struct lloop_device *lo, struct file *unused,
> lo->lo_queue->queuedata = lo;
>
> /* queue parameters */
> - CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
> - blk_queue_logical_block_size(lo->lo_queue,
> - (unsigned short)PAGE_CACHE_SIZE);
> blk_queue_max_hw_sectors(lo->lo_queue,
> LLOOP_MAX_SEGMENTS << (PAGE_CACHE_SHIFT - 9));
> blk_queue_max_segments(lo->lo_queue, LLOOP_MAX_SEGMENTS);
> --
> 1.7.1
>
>

2016-04-10 14:04:58

by James Simmons

[permalink] [raw]
Subject: Re: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64


> This patch was shown not to work. I just haven't removed it from opensuse yet.

Its been running in our production tree as well for some time. Guess that
change is a noop. In any case we have been discussing redoing the lloop
driver anyways. Just need to find the cycles.

> --
> Jeff Mahoney
> (apologies for the top post -- from my mobile)
>
> > On Apr 10, 2016, at 9:13 AM, James Simmons <[email protected]> wrote:
> >
> > From: Jeff Mahoney <[email protected]>
> >
> > On ppc64 with 64k pages, we get a build failure in lloop:
> >
> > drivers/staging/lustre/lustre/llite/lloop.c:527:2:
> > note: in expansion of macro 'CLASSERT'
> > CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
> >
> > There's no need to change the queue's logical block size. Even if it could
> > accept a 64k value, that would result in any file system on top of it
> > needing to also use 64k blocks. It'd be safe to set it to 4k, but there's
> > no actual need for it. It's not used to split requests except for WRITE_SAME,
> > which lloop doesn't implement anyway.
> >
> > Signed-off-by: Jeff Mahoney <[email protected]>
> > Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4000
> > Reviewed-on: http://review.whamcloud.com/7745
> > Reviewed-by: Jinshan Xiong <[email protected]>
> > Reviewed-by: Minh Diep <[email protected]>
> > Reviewed-by: Oleg Drokin <[email protected]>
> > Signed-off-by: James Simmons <[email protected]>
> > ---
> > drivers/staging/lustre/lustre/llite/lloop.c | 3 ---
> > 1 files changed, 0 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
> > index b725fc1..f396753 100644
> > --- a/drivers/staging/lustre/lustre/llite/lloop.c
> > +++ b/drivers/staging/lustre/lustre/llite/lloop.c
> > @@ -525,9 +525,6 @@ static int loop_set_fd(struct lloop_device *lo, struct file *unused,
> > lo->lo_queue->queuedata = lo;
> >
> > /* queue parameters */
> > - CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
> > - blk_queue_logical_block_size(lo->lo_queue,
> > - (unsigned short)PAGE_CACHE_SIZE);
> > blk_queue_max_hw_sectors(lo->lo_queue,
> > LLOOP_MAX_SEGMENTS << (PAGE_CACHE_SHIFT - 9));
> > blk_queue_max_segments(lo->lo_queue, LLOOP_MAX_SEGMENTS);
> > --
> > 1.7.1
> >
> >
>
>

2016-04-10 15:35:11

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64

On Sun, Apr 10, 2016 at 03:04:51PM +0100, James Simmons wrote:
>
> > This patch was shown not to work. I just haven't removed it from opensuse yet.
>
> Its been running in our production tree as well for some time. Guess that
> change is a noop. In any case we have been discussing redoing the lloop
> driver anyways. Just need to find the cycles.

Why do you need this driver? I know Christoph has pointed out many
times that it should just be dropped as it's not needed.

And should I drop this patch from the series?

thanks,

greg k-h

2016-04-10 15:52:51

by James Simmons

[permalink] [raw]
Subject: Re: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64


> > > This patch was shown not to work. I just haven't removed it from opensuse yet.
> >
> > Its been running in our production tree as well for some time. Guess that
> > change is a noop. In any case we have been discussing redoing the lloop
> > driver anyways. Just need to find the cycles.
>
> Why do you need this driver? I know Christoph has pointed out many
> times that it should just be dropped as it's not needed.

Please don't drop this driver just yet. The reason we kept it was for the DIO
support that the loop back device lacked earlier. We need to do some
testing to make sure the loop can completely replace this driver.

> And should I drop this patch from the series?

Yes since it is a noop. No other patches are dependent on it.

2016-04-10 21:13:49

by Jeff Mahoney

[permalink] [raw]
Subject: Re: [PATCH 13/17] staging: lustre: lloop: Fix build failure on ppc64

On 4/10/16 10:04 AM, James Simmons wrote:
>
>> This patch was shown not to work. I just haven't removed it from opensuse yet.
>
> Its been running in our production tree as well for some time. Guess that
> change is a noop. In any case we have been discussing redoing the lloop
> driver anyways. Just need to find the cycles.

I guess my memory was flakey and I was recalling the first comments in
LU-4000. The updated version should be ok.

-Jeff

>> --
>> Jeff Mahoney
>> (apologies for the top post -- from my mobile)
>>
>>> On Apr 10, 2016, at 9:13 AM, James Simmons <[email protected]> wrote:
>>>
>>> From: Jeff Mahoney <[email protected]>
>>>
>>> On ppc64 with 64k pages, we get a build failure in lloop:
>>>
>>> drivers/staging/lustre/lustre/llite/lloop.c:527:2:
>>> note: in expansion of macro 'CLASSERT'
>>> CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
>>>
>>> There's no need to change the queue's logical block size. Even if it could
>>> accept a 64k value, that would result in any file system on top of it
>>> needing to also use 64k blocks. It'd be safe to set it to 4k, but there's
>>> no actual need for it. It's not used to split requests except for WRITE_SAME,
>>> which lloop doesn't implement anyway.
>>>
>>> Signed-off-by: Jeff Mahoney <[email protected]>
>>> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-4000
>>> Reviewed-on: http://review.whamcloud.com/7745
>>> Reviewed-by: Jinshan Xiong <[email protected]>
>>> Reviewed-by: Minh Diep <[email protected]>
>>> Reviewed-by: Oleg Drokin <[email protected]>
>>> Signed-off-by: James Simmons <[email protected]>
>>> ---
>>> drivers/staging/lustre/lustre/llite/lloop.c | 3 ---
>>> 1 files changed, 0 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/staging/lustre/lustre/llite/lloop.c b/drivers/staging/lustre/lustre/llite/lloop.c
>>> index b725fc1..f396753 100644
>>> --- a/drivers/staging/lustre/lustre/llite/lloop.c
>>> +++ b/drivers/staging/lustre/lustre/llite/lloop.c
>>> @@ -525,9 +525,6 @@ static int loop_set_fd(struct lloop_device *lo, struct file *unused,
>>> lo->lo_queue->queuedata = lo;
>>>
>>> /* queue parameters */
>>> - CLASSERT(PAGE_CACHE_SIZE < (1 << (sizeof(unsigned short) * 8)));
>>> - blk_queue_logical_block_size(lo->lo_queue,
>>> - (unsigned short)PAGE_CACHE_SIZE);
>>> blk_queue_max_hw_sectors(lo->lo_queue,
>>> LLOOP_MAX_SEGMENTS << (PAGE_CACHE_SHIFT - 9));
>>> blk_queue_max_segments(lo->lo_queue, LLOOP_MAX_SEGMENTS);
>>> --
>>> 1.7.1
>>>
>>>
>>
>>
>


--
Jeff Mahoney
SUSE Labs


Attachments:
signature.asc (827.00 B)
OpenPGP digital signature

2016-04-12 04:18:59

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 01/17] staging: lustre: osc: fix race issues thanks to oap_lock

On Sun, Apr 10, 2016 at 09:13:24AM -0400, James Simmons wrote:
> From: Sebastien Buisson <[email protected]>
>
> Fix 'data race condition' defects found by Coverity version 6.5.0:
>
> Data race condition (MISSING_LOCK)
>
> Accessing variable without holding lock. Elsewhere,
> this variable is accessed with lock held.
>
> This patch is dedicated to code fragments involving oap_lock.
>
> Signed-off-by: Sebastien Buisson <[email protected]>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2744
> Reviewed-on: http://review.whamcloud.com/6572
> Reviewed-by: Oleg Drokin <[email protected]>
> Reviewed-by: Andreas Dilger <[email protected]>
> Signed-off-by: James Simmons <[email protected]>
> ---
> drivers/staging/lustre/lustre/osc/osc_cache.c | 8 ++++++++
> drivers/staging/lustre/lustre/osc/osc_io.c | 2 ++
> 2 files changed, 10 insertions(+), 0 deletions(-)

This patch doesn't apply at all. Can you rebase the whole series and
resend them?

thanks,

greg k-h