This is the first rebased version of the tree over 2.6.36-rc*.
The tree's branches are structured the same (see details below).
Cumulative patches can be generated from
git://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.36-rc3 pnfs-all-2.6.36-rc3-2010-08-30
git diff v2.6.35 pnfs-all-2.6.35-2010-08-24
Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
Latest patches (since pnfs-all-2.6.35-2010-08-24):
pnfs-submit:
Benny Halevy 32fb90a SQUASHME: pnfs-submit: do not use NIPQUAD in nfs4_callback_layoutrecall dprintk
Benny Halevy 133b759 SQUASHME: pnfs-submit: get machince creds for getdeviceinfo
pnfs:
Benny Halevy 247fe09 SQUASHME: pnfs: get machince creds for getdevicelist
pnfs-obj-all:
Benny Halevy c384327 SQUASHME: pnfs-obj use REQ flags rather than BIO flags
The full tree structure is:
master (v2.6.36-rc3)
nfsd-upstream
nfsd41-all
nfs-upstream
nfs41-all
nfs41-all-latest
pnfs-submit
pnfs
pnfs-block
pnfsd-files
pnfsd
spnfs
spnfs-block
pnfsd-lexp
pnfs-submit-all
pnfs-all
pnfs-obj-all
pnfs-exofs-all
pnfs-block-all
spnfs-all
spnfs-block-all
pnfs-all-latest
Benny
Not available in 2.6.26
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/callback_proc.c | 5 +----
1 files changed, 1 insertions(+), 4 deletions(-)
diff --git a/fs/nfs/callback_proc.c b/fs/nfs/callback_proc.c
index 6107b2e..09991d2 100644
--- a/fs/nfs/callback_proc.c
+++ b/fs/nfs/callback_proc.c
@@ -362,11 +362,8 @@ __be32 nfs4_callback_layoutrecall(struct cb_layoutrecallargs *args,
res = cpu_to_be32(NFS4ERR_OP_NOT_IN_SESSION);
clp = nfs_find_client(args->cbl_addr, 4);
- if (clp == NULL) {
- dprintk("%s: no client for addr %u.%u.%u.%u\n",
- __func__, NIPQUAD(args->cbl_addr));
+ if (clp == NULL)
goto out;
- }
res = cpu_to_be32(NFS4ERR_NOMATCHING_LAYOUT);
do {
--
1.7.2.2
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/nfs4proc.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 33c4057..48a763e 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5784,11 +5784,13 @@ static int _nfs4_getdevicelist(struct nfs_server *server,
.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_GETDEVICELIST],
.rpc_argp = &args,
.rpc_resp = &res,
+ .rpc_cred = nfs4_get_machine_cred(server->nfs_client),
};
int status;
dprintk("--> %s\n", __func__);
status = nfs4_call_sync(server, &msg, &args, &res, 0);
+ put_rpccred(msg.rpc_cred);
dprintk("<-- %s status=%d\n", __func__, status);
return status;
}
--
1.7.2.2
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/nfs4_fs.h | 13 ++++++++++++-
fs/nfs/nfs4proc.c | 2 ++
2 files changed, 14 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/nfs4_fs.h b/fs/nfs/nfs4_fs.h
index 9743326..ced4a20 100644
--- a/fs/nfs/nfs4_fs.h
+++ b/fs/nfs/nfs4_fs.h
@@ -303,13 +303,24 @@ extern void nfs4_kill_renewd(struct nfs_client *);
extern void nfs4_renew_state(struct work_struct *);
/* nfs4state.c */
+struct rpc_cred *nfs4_get_machine_cred_locked(struct nfs_client *clp);
struct rpc_cred *nfs4_get_setclientid_cred(struct nfs_client *clp);
struct rpc_cred *nfs4_get_renew_cred_locked(struct nfs_client *clp);
#if defined(CONFIG_NFS_V4_1)
-struct rpc_cred *nfs4_get_machine_cred_locked(struct nfs_client *clp);
struct rpc_cred *nfs4_get_exchange_id_cred(struct nfs_client *clp);
#endif /* CONFIG_NFS_V4_1 */
+static inline struct rpc_cred *
+nfs4_get_machine_cred(struct nfs_client *clp)
+{
+ struct rpc_cred *cred;
+
+ spin_lock(&clp->cl_lock);
+ cred = nfs4_get_machine_cred_locked(clp);
+ spin_unlock(&clp->cl_lock);
+ return cred;
+}
+
extern struct nfs4_state_owner * nfs4_get_state_owner(struct nfs_server *, struct rpc_cred *);
extern void nfs4_put_state_owner(struct nfs4_state_owner *);
extern struct nfs4_state * nfs4_get_open_state(struct inode *, struct nfs4_state_owner *);
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 78717a7..33c4057 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5824,11 +5824,13 @@ int nfs4_proc_getdeviceinfo(struct nfs_server *server, struct pnfs_device *pdev)
.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_GETDEVICEINFO],
.rpc_argp = &args,
.rpc_resp = &res,
+ .rpc_cred = nfs4_get_machine_cred(server->nfs_client),
};
int status;
dprintk("--> %s\n", __func__);
status = nfs4_call_sync(server, &msg, &args, &res, 0);
+ put_rpccred(msg.rpc_cred);
dprintk("<-- %s status=%d\n", __func__, status);
return status;
--
1.7.2.2
As they were converged starting 2.6.36
Cc: Boaz Harrosh <[email protected]>
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/objlayout/objio_osd.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/objlayout/objio_osd.c b/fs/nfs/objlayout/objio_osd.c
index 316f9c5..223980b 100644
--- a/fs/nfs/objlayout/objio_osd.c
+++ b/fs/nfs/objlayout/objio_osd.c
@@ -910,7 +910,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)
} else {
bio = master_dev->bio;
/* FIXME: bio_set_dir() */
- bio->bi_rw |= (1 << BIO_RW);
+ bio->bi_rw |= (1 << REQ_WRITE);
}
osd_req_write(or, &obj, per_dev->offset, bio, per_dev->length);
--
1.7.2.2
On 2010-09-29 16:36, Tigran Mkrtchyan wrote:
> On 09/29/2010 04:22 PM, Benny Halevy wrote:
>> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>> I am support to publish first kernel crash.....
>>> sorry to publish....
>>>
>>>> this is
>>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>>
>>>> Regards,
>>>> Tigran.
>>>>
>>>>
>> Thanks for testing!
>> Does the following patch help?
>>
>
> No :(
>
Tigran, do you have a new trace maybe?
I cross referenced the dump you sent to the
disassembled code and it seems like it hit NULL dereference on
size = fl->stripe_unit;
I'm going to commit this patch anyway but I want to
understand what else I missed...
Benny
>> git diff --stat -p -M
>> fs/nfs/nfs4filelayout.c | 2 ++
>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>> index 75e07c8..8fbb0db 100644
>> --- a/fs/nfs/nfs4filelayout.c
>> +++ b/fs/nfs/nfs4filelayout.c
>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>
>> /* Horrible hack...ideally upper layer would send lseg */
>> lseg = pnfs_has_layout(lo,&range);
>> + if (!lseg)
>> + return 0;
>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>> size = fl->stripe_unit;
>> put_lseg_locked(lseg);
>
On 2010-09-30 09:58, Tigran Mkrtchyan wrote:
> I recompiled kernel with debug symbols and problem is gone.
> Crap!
> Or, as Fred suspected, I did not apply you patch second time.
Yeah, this was my inclination too...
Benny
>
> Regards,
> Tigran.
>
> On 09/29/2010 05:25 PM, Benny Halevy wrote:
>> On 2010-09-29 16:36, Tigran Mkrtchyan wrote:
>>> On 09/29/2010 04:22 PM, Benny Halevy wrote:
>>>> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>>>>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>>>> I am support to publish first kernel crash.....
>>>>> sorry to publish....
>>>>>
>>>>>> this is
>>>>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>>>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>>>>
>>>>>> Regards,
>>>>>> Tigran.
>>>>>>
>>>>>>
>>>> Thanks for testing!
>>>> Does the following patch help?
>>>>
>>> No :(
>>>
>> Tigran, do you have a new trace maybe?
>> I cross referenced the dump you sent to the
>> disassembled code and it seems like it hit NULL dereference on
>> size = fl->stripe_unit;
>>
>> I'm going to commit this patch anyway but I want to
>> understand what else I missed...
>>
>> Benny
>>
>>>> git diff --stat -p -M
>>>> fs/nfs/nfs4filelayout.c | 2 ++
>>>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>>>> index 75e07c8..8fbb0db 100644
>>>> --- a/fs/nfs/nfs4filelayout.c
>>>> +++ b/fs/nfs/nfs4filelayout.c
>>>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>>>
>>>> /* Horrible hack...ideally upper layer would send lseg */
>>>> lseg = pnfs_has_layout(lo,&range);
>>>> + if (!lseg)
>>>> + return 0;
>>>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>>>> size = fl->stripe_unit;
>>>> put_lseg_locked(lseg);
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/pnfs.c | 21 ++++++++++++++++++---
fs/nfs/pnfs.h | 4 ++++
2 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index b3e1e8a..8339444 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -233,6 +233,21 @@ EXPORT_SYMBOL_GPL(pnfs_unregister_layoutdriver);
* pNFS client layout cache
*/
+static struct pnfs_layout_hdr *
+pnfs_alloc_layout_hdr(struct inode *ino)
+{
+ struct pnfs_layoutdriver_type *ld = NFS_SERVER(ino)->pnfs_curr_ld;
+ return ld->alloc_layout_hdr ? ld->alloc_layout_hdr(ino) :
+ kzalloc(sizeof(struct pnfs_layout_hdr), GFP_KERNEL);
+}
+
+static void
+pnfs_free_layout_hdr(struct pnfs_layout_hdr *lo)
+{
+ struct pnfs_layoutdriver_type *ld = NFS_SERVER(lo->inode)->pnfs_curr_ld;
+ return ld->alloc_layout_hdr ? ld->free_layout_hdr(lo) : kfree(lo);
+}
+
static void
get_layout_hdr_locked(struct pnfs_layout_hdr *lo)
{
@@ -251,7 +266,7 @@ put_layout_hdr_locked(struct pnfs_layout_hdr *lo)
dprintk("%s: freeing layout cache %p\n", __func__, lo);
BUG_ON(!list_empty(&lo->layouts));
NFS_I(lo->inode)->layout = NULL;
- kfree(lo);
+ pnfs_free_layout_hdr(lo);
}
}
@@ -839,7 +854,7 @@ alloc_init_layout_hdr(struct inode *ino)
{
struct pnfs_layout_hdr *lo;
- lo = kzalloc(sizeof(struct pnfs_layout_hdr), GFP_KERNEL);
+ lo = pnfs_alloc_layout_hdr(ino);
if (!lo)
return NULL;
lo->refcount = 1;
@@ -869,7 +884,7 @@ pnfs_find_alloc_layout(struct inode *ino)
if (likely(nfsi->layout == NULL)) /* Won the race? */
nfsi->layout = new;
else
- kfree(new);
+ pnfs_free_layout_hdr(new);
return nfsi->layout;
}
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index af29b02..36ce9fa 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -76,6 +76,10 @@ struct pnfs_layoutdriver_type {
unsigned flags;
int (*initialize_mountpoint) (struct nfs_server *, const struct nfs_fh *);
int (*uninitialize_mountpoint) (struct nfs_server *);
+
+ struct pnfs_layout_hdr * (*alloc_layout_hdr) (struct inode *inode);
+ void (*free_layout_hdr) (struct pnfs_layout_hdr *);
+
struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_hdr *layoutid, struct nfs4_layoutget_res *lgr);
void (*free_lseg) (struct pnfs_layout_segment *lseg);
--
1.7.2.3
On 2010-09-30 11:12, Tigran Mkrtchyan wrote:
> On 09/30/2010 10:30 AM, Benny Halevy wrote:
>> On 2010-09-29 16:22, Benny Halevy wrote:
>>> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>>>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>>> I am support to publish first kernel crash.....
>>>> sorry to publish....
>>>>
>>>>> this is
>>>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>>>
>>>>> Regards,
>>>>> Tigran.
>>>>>
>>>>>
>>> Thanks for testing!
>>> Does the following patch help?
>>>
>> Applied.
>>
> did you applied to pnfs-submit as well?
Not yet. It's in pnfs-all-latest until the next rebase.
I planned to do it tomorrow before I leave for Boston in case more patches
are required, unless folks want to test pnfs-submit / pnfs-submit-all with
these fixes right now.
Benny
>> Benny
>>
>>> git diff --stat -p -M
>>> fs/nfs/nfs4filelayout.c | 2 ++
>>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>>> index 75e07c8..8fbb0db 100644
>>> --- a/fs/nfs/nfs4filelayout.c
>>> +++ b/fs/nfs/nfs4filelayout.c
>>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>>
>>> /* Horrible hack...ideally upper layer would send lseg */
>>> lseg = pnfs_has_layout(lo,&range);
>>> + if (!lseg)
>>> + return 0;
>>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>>> size = fl->stripe_unit;
>>> put_lseg_locked(lseg);
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Introduce lo_rpcwaitq and sleep on it while there's an lseg marked invalid,
i.e. it is being returned (in the future, it could also be layoutget in progress).
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/inode.c | 1 +
fs/nfs/nfs4proc.c | 25 ++++++++++++++++++++++---
fs/nfs/pnfs.c | 26 ++++++++++++++++----------
include/linux/nfs_fs.h | 1 +
4 files changed, 40 insertions(+), 13 deletions(-)
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 8cdea1a..e3d44a9 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1459,6 +1459,7 @@ static inline void nfs4_init_once(struct nfs_inode *nfsi)
nfsi->delegation_state = 0;
init_rwsem(&nfsi->rwsem);
init_waitqueue_head(&nfsi->lo_waitq);
+ rpc_init_wait_queue(&nfsi->lo_rpcwaitq, "pNFS Layout");
nfsi->layout = NULL;
#endif
}
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index cd87704..ebf5127 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5426,13 +5426,32 @@ nfs4_layoutget_prepare(struct rpc_task *task, void *calldata)
{
struct nfs4_layoutget *lgp = calldata;
struct inode *ino = lgp->args.inode;
+ struct nfs_inode *nfsi = NFS_I(ino);
struct nfs_server *server = NFS_SERVER(ino);
+ struct pnfs_layout_segment *lseg;
dprintk("--> %s\n", __func__);
- if (nfs4_setup_sequence(server, NULL, &lgp->args.seq_args,
- &lgp->res.seq_res, 0, task))
+ spin_lock(&ino->i_lock);
+ lseg = pnfs_has_layout(nfsi->layout, &lgp->args.range);
+ if (likely(!lseg)) {
+ spin_unlock(&ino->i_lock);
+ dprintk("%s: no lseg found, proceeding\n", __func__);
+ if (!nfs4_setup_sequence(server, NULL, &lgp->args.seq_args,
+ &lgp->res.seq_res, 0, task))
+ rpc_call_start(task);
return;
- rpc_call_start(task);
+ }
+ if (!lseg->valid) {
+ put_lseg_locked(lseg);
+ spin_unlock(&ino->i_lock);
+ dprintk("%s: invalid lseg found, waiting\n", __func__);
+ rpc_sleep_on(&nfsi->lo_rpcwaitq, task, NULL);
+ return;
+ }
+ *lgp->lsegpp = lseg;
+ spin_unlock(&ino->i_lock);
+ dprintk("%s: valid lseg found, no rpc required\n", __func__);
+ rpc_exit(task, NFS4_OK);
}
static void nfs4_layoutget_done(struct rpc_task *task, void *calldata)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 8339444..31a703d 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -315,8 +315,10 @@ put_lseg_locked(struct pnfs_layout_segment *lseg)
do_wake_up = !lseg->valid;
nfsi = NFS_I(lseg->layout->inode);
kref_put(&lseg->kref, destroy_lseg);
- if (do_wake_up)
+ if (do_wake_up) {
wake_up(&nfsi->lo_waitq);
+ rpc_wake_up(&nfsi->lo_rpcwaitq);
+ }
}
EXPORT_SYMBOL_GPL(put_lseg_locked);
@@ -975,17 +977,21 @@ pnfs_update_layout(struct inode *ino,
/* Check to see if the layout for the given range already exists */
lseg = pnfs_has_layout(lo, &arg);
- if (lseg && !lseg->valid) {
- put_lseg_locked(lseg);
+ if (lseg) {
+ if (lseg->valid) {
+ dprintk("%s: Using cached lseg %p for %llu@%llu "
+ "iomode %d)\n",
+ __func__,
+ lseg,
+ arg.length,
+ arg.offset,
+ arg.iomode);
+
+ goto out_unlock;
+ }
/* someone is cleaning the layout */
+ put_lseg_locked(lseg);
lseg = NULL;
- goto out_unlock;
- }
-
- if (lseg) {
- dprintk("%s: Using cached lseg %p for iomode %d)\n",
- __func__, lseg, iomode);
- goto out_unlock;
}
/* if LAYOUTGET already failed once we don't try again */
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index 3184e63..1af18fd 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -191,6 +191,7 @@ struct nfs_inode {
/* pNFS layout information */
wait_queue_head_t lo_waitq;
+ struct rpc_wait_queue lo_rpcwaitq;
struct pnfs_layout_hdr *layout;
#endif /* CONFIG_NFS_V4*/
#ifdef CONFIG_NFS_FSCACHE
--
1.7.2.3
Base the LAYOUTGET arguments on the actual required byte ranges
rather than asking for the whole file layout.
Add a check in readpage_async_filler that the layout segment
retrieved in pnfs_pageio_init_read still covers the current page
and if not, try getting a new one.
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/file.c | 2 +-
fs/nfs/pnfs.c | 15 ++++++++-------
fs/nfs/pnfs.h | 4 ++--
fs/nfs/read.c | 16 +++++++++++++++-
4 files changed, 26 insertions(+), 11 deletions(-)
diff --git a/fs/nfs/file.c b/fs/nfs/file.c
index 1572817..f650ee3 100644
--- a/fs/nfs/file.c
+++ b/fs/nfs/file.c
@@ -390,7 +390,7 @@ static int nfs_write_begin(struct file *file, struct address_space *mapping,
lseg = pnfs_update_layout(mapping->host,
nfs_file_open_context(file),
- IOMODE_RW);
+ pos, len, IOMODE_RW);
start:
/*
* Prevent starvation issues if someone is doing a consistency
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index cf84f2c..4edab46 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -598,11 +598,9 @@ send_layoutget(struct pnfs_layout_hdr *lo,
pnfs_layoutget_release(lo);
return NULL;
}
- lgp->args.minlength = NFS4_MAX_UINT64;
+ lgp->args.minlength = PAGE_CACHE_SIZE;
lgp->args.maxcount = PNFS_LAYOUT_MAXSIZE;
- lgp->args.range.iomode = range->iomode;
- lgp->args.range.offset = 0;
- lgp->args.range.length = NFS4_MAX_UINT64;
+ lgp->args.range = *range;
lgp->args.type = server->pnfs_curr_ld->id;
lgp->args.inode = ino;
lgp->args.ctx = get_nfs_open_context(ctx);
@@ -937,12 +935,14 @@ EXPORT_SYMBOL_GPL(pnfs_has_layout);
struct pnfs_layout_segment *
pnfs_update_layout(struct inode *ino,
struct nfs_open_context *ctx,
+ loff_t pos,
+ u64 count,
enum pnfs_iomode iomode)
{
struct pnfs_layout_range arg = {
.iomode = iomode,
- .offset = 0,
- .length = NFS4_MAX_UINT64,
+ .offset = pos,
+ .length = count,
};
struct nfs_inode *nfsi = NFS_I(ino);
struct pnfs_layout_hdr *lo;
@@ -1124,7 +1124,8 @@ pnfs_pageio_init_read(struct nfs_pageio_descriptor *pgio,
readahead_range(inode, pages, &loff, &count);
if (count > 0) {
- pgio->pg_lseg = pnfs_update_layout(inode, ctx, IOMODE_READ);
+ pgio->pg_lseg = pnfs_update_layout(inode, ctx, loff, count,
+ IOMODE_READ);
if (!pgio->pg_lseg)
return;
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index ee43489..0aac85f 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -195,7 +195,7 @@ struct pnfs_layout_segment *
pnfs_has_layout(struct pnfs_layout_hdr *lo, struct pnfs_layout_range *range);
struct pnfs_layout_segment *
pnfs_update_layout(struct inode *ino, struct nfs_open_context *ctx,
- enum pnfs_iomode access_type);
+ loff_t pos, u64 count, enum pnfs_iomode access_type);
int _pnfs_return_layout(struct inode *, struct pnfs_layout_range *,
const nfs4_stateid *stateid, /* optional */
enum pnfs_layoutreturn_type, bool wait);
@@ -327,7 +327,7 @@ static inline void put_lseg_locked(struct pnfs_layout_segment *lseg)
static inline struct pnfs_layout_segment *
pnfs_update_layout(struct inode *ino, struct nfs_open_context *ctx,
- enum pnfs_iomode access_type)
+ loff_t pos, u64 count, enum pnfs_iomode access_type)
{
return NULL;
}
diff --git a/fs/nfs/read.c b/fs/nfs/read.c
index be29689..e4b12b5 100644
--- a/fs/nfs/read.c
+++ b/fs/nfs/read.c
@@ -122,12 +122,14 @@ int nfs_readpage_async(struct nfs_open_context *ctx, struct inode *inode,
LIST_HEAD(one_request);
struct nfs_page *new;
unsigned int len;
+ loff_t pgoffs;
struct pnfs_layout_segment *lseg;
len = nfs_page_length(page);
if (len == 0)
return nfs_return_empty_page(page);
- lseg = pnfs_update_layout(inode, ctx, IOMODE_READ);
+ pgoffs = (loff_t)page->index << PAGE_CACHE_SHIFT;
+ lseg = pnfs_update_layout(inode, ctx, pgoffs, len, IOMODE_READ);
new = nfs_create_request(ctx, inode, page, 0, len, lseg);
put_lseg(lseg);
if (IS_ERR(new)) {
@@ -601,14 +603,26 @@ readpage_async_filler(void *data, struct page *page)
{
struct nfs_readdesc *desc = (struct nfs_readdesc *)data;
struct inode *inode = page->mapping->host;
+ struct pnfs_layout_range *range;
struct nfs_page *new;
unsigned int len;
+ loff_t pgoff;
int error;
len = nfs_page_length(page);
if (len == 0)
return nfs_return_empty_page(page);
+ pgoff = (loff_t)page->index << PAGE_CACHE_SHIFT;
+ range = desc->pgio->pg_lseg ? &desc->pgio->pg_lseg->range : NULL;
+ if (!range ||
+ (range->offset > pgoff + len) ||
+ (range->offset + range->length < pgoff)) {
+ put_lseg(desc->pgio->pg_lseg);
+ desc->pgio->pg_lseg = pnfs_update_layout(inode, desc->ctx,
+ pgoff, len, IOMODE_READ);
+ }
+
new = nfs_create_request(desc->ctx, inode, page, 0, len,
desc->pgio->pg_lseg);
if (IS_ERR(new))
--
1.7.2.3
Tigran, are you sure you are running with Benny's suggested patch?
The two oopses you posted from before and after look suspiciously
similar.
Fred
On Wed, Sep 29, 2010 at 12:57 PM, Tigran Mkrtchyan
<[email protected]> wrote:
>
>> Tigran, do you have a new trace maybe?
>> I cross referenced the dump you sent to the
>> disassembled code and it seems like it hit NULL dereference on
>> size = fl->stripe_unit;
>>
>> I'm going to commit this patch anyway but I want to
>> understand what else I missed...
>>
>> Benny
>>
>
> The stack attached. ?Shall I compile the kernel with some debug options for
> better debugging?
>
> Tigran.
>>>>
>>>> git diff --stat -p -M
>>>> ? fs/nfs/nfs4filelayout.c | ? ?2 ++
>>>> ? 1 files changed, 2 insertions(+), 0 deletions(-)
>>>>
>>>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>>>> index 75e07c8..8fbb0db 100644
>>>> --- a/fs/nfs/nfs4filelayout.c
>>>> +++ b/fs/nfs/nfs4filelayout.c
>>>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr
>>>> *lo)
>>>>
>>>> ? ? ? ?/* Horrible hack...ideally upper layer would send lseg */
>>>> ? ? ? ?lseg = pnfs_has_layout(lo,&range);
>>>> + ? ? ? if (!lseg)
>>>> + ? ? ? ? ? ? ? return 0;
>>>> ? ? ? ?fl = container_of(lseg, struct nfs4_filelayout_segment,
>>>> generic_hdr);
>>>> ? ? ? ?size = fl->stripe_unit;
>>>> ? ? ? ?put_lseg_locked(lseg);
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
>
>
On 2010-09-29 13:06, Benny Halevy wrote:
> This version includes Fred's pnfs-submit RFC patches (v5).
> and is rebased onto v2.6.36-rc6
>
> Cumulative patches can be generated from
> git://linux-nfs.org/~bhalevy/linux-pnfs.git
> using
> git diff v2.6.36-rc6 pnfs-all-2.6.36-rc6-2010-09-29
>
> Or, they can be downloaded from the wiki at:
> http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
>
> Latest patches (since pnfs-all-2.6.36-rc3-2010-08-30):
>
> upstream:
> Boaz Harrosh cc59b6a um: Proper Fix for f25c80a4: remove duplicate structure field initialization
>
> pnfs-submit:
> The pNFS Team 1851dbc RFC: pnfsd, pnfs: protocol level pnfs constants
> The pNFS Team a1b60af RFC: nfs: change stateid to be a union
> The pNFS Team ff38ee4 RFC: nfs: ask for layouttypes during fsinfo call
> The pNFS Team 01f09aa RFC: nfs: set layout driver
> The pNFS Team f7893b6 RFC: pnfs: full mount/umount infrastructure
> The pNFS Team 1cef5e0 RFC: pnfs: filelayout: introduce minimal file layout driver
> The pNFS Team b7e2a3d RFC: nfs: create and destroy inode's layout cache
> The pNFS Team 34d2802 RFC: nfs: client needs to maintain list of inodes with active layouts
> The pNFS Team 292b90e RFC: pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure
> The pNFS Team bf610f6 RFC: pnfs: filelayout: add driver's LAYOUTGET and GETDEVICEINFO infrastructure
> Fred Isaman 068e761 pnfs-submit: revert range simplifications made by submission patches
> Benny Halevy 7c58c3d SQUASHME: pnfs-submit: add missing include file in nfs4filelayoutdev.c
> Benny Halevy 9a5956f SQUASHME: move nfs4_deviceid definitions to include/linux/nfs4.h
> Benny Halevy bb2afce SQUASHME: pnfs-submit: handle non-pnfs case in set_pnfs_layoutdriver
> Benny Halevy 998fe4a pnfs-submit: file needs layout commit, server attributes may be stale
> Benny Halevy f18cdf8 NFS: clear fsinfo before sendign rpc
>
> pnfs:
Hmm, one patch escaped the automatic tool.
It seems like its timestamp got changed due to squashing
while rebasing it. :-(
Benny Halevy 9f05c5a pnfs: get layout in proper segments
> Benny Halevy 52a6277 pnfs: alloc and free layout_hdr layoutdriver methods
> Benny Halevy 6dd521e pnfs: allow nfs4_proc_layoutget to sleep on invalid lsegs
> Benny Halevy ba1dd5b SQUASHME: pnfs: refactor put_lseg{_locked}
> Benny Halevy bb03ecf SQUASHME: pnfs: get rid of lo_waitq
>
> pnfs-block:
> Benny Halevy 73b1db9 SQUASHME: pnfsblock: remove obsolete include file from blocklayout.h
> Benny Halevy d9ebc26 SQUASHME: pnfsblock: use nfs4_deviceid
> Benny Halevy 90e2af1 SQUASHME: pnfsblock: no callback ops
> Benny Halevy 0953b90 SQAUSHME: pnfsblock: no PNFS_NFS_SERVER
> Benny Halevy 7ab3774 SQUASHME: pnfsblock: no dev_notify_types
> Benny Halevy f08f966 SQUASHME: pnfsblock: use new struct pnfs_layout_hdr
>
> pnfs-obj-all:
> Benny Halevy 41a70b7 SQUASHME: pnfs-obj: fix REQ flags usage
> Benny Halevy 6057701 SQUASHME: pnfs-obj: convert to new pnfs-submit changes
>
> pnfs-exofs-all:
> Benny Halevy 10d8e6e SQUASHME: pnfsd-exofs: convert to new pnfs-submit changes
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/blocklayout/blocklayout.h | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.h b/fs/nfs/blocklayout/blocklayout.h
index ab61a19..fc0fb23 100644
--- a/fs/nfs/blocklayout/blocklayout.h
+++ b/fs/nfs/blocklayout/blocklayout.h
@@ -33,8 +33,8 @@
#define FS_NFS_NFS4BLOCKLAYOUT_H
#include <linux/nfs_fs.h>
-#include <linux/nfs4_pnfs.h>
#include <linux/dm-ioctl.h> /* Needed for struct dm_ioctl*/
+#include <../pnfs.h>
#define PAGE_CACHE_SECTORS (PAGE_CACHE_SIZE >> 9)
--
1.7.2.3
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/blocklayout/blocklayout.c | 22 +++++++---------------
1 files changed, 7 insertions(+), 15 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index ac53a3f..23dbe91 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -546,7 +546,7 @@ release_inval_marks(struct pnfs_inval_markings *marks)
/* Note we are relying on caller locking to prevent nasty races. */
static void
-bl_free_layout(struct pnfs_layout_hdr *lo)
+bl_free_layout_hdr(struct pnfs_layout_hdr *lo)
{
struct pnfs_block_layout *bl = BLK_LO2EXT(lo);
@@ -557,7 +557,7 @@ bl_free_layout(struct pnfs_layout_hdr *lo)
}
static struct pnfs_layout_hdr *
-bl_alloc_layout(struct inode *inode)
+bl_alloc_layout_hdr(struct inode *inode)
{
struct pnfs_block_layout *bl;
@@ -1105,15 +1105,17 @@ bl_pg_test(struct nfs_pageio_descriptor *pgio, struct nfs_page *prev,
return 1;
}
-static struct layoutdriver_io_operations blocklayout_io_operations = {
+static struct pnfs_layoutdriver_type blocklayout_type = {
+ .id = LAYOUT_BLOCK_VOLUME,
+ .name = "LAYOUT_BLOCK_VOLUME",
.commit = bl_commit,
.read_pagelist = bl_read_pagelist,
.write_pagelist = bl_write_pagelist,
.write_begin = bl_write_begin,
.write_end = bl_write_end,
.write_end_cleanup = bl_write_end_cleanup,
- .alloc_layout = bl_alloc_layout,
- .free_layout = bl_free_layout,
+ .alloc_layout_hdr = bl_alloc_layout_hdr,
+ .free_layout_hdr = bl_free_layout_hdr,
.alloc_lseg = bl_alloc_lseg,
.free_lseg = bl_free_lseg,
.setup_layoutcommit = bl_setup_layoutcommit,
@@ -1121,20 +1123,10 @@ static struct layoutdriver_io_operations blocklayout_io_operations = {
.cleanup_layoutcommit = bl_cleanup_layoutcommit,
.initialize_mountpoint = bl_initialize_mountpoint,
.uninitialize_mountpoint = bl_uninitialize_mountpoint,
-};
-
-static struct layoutdriver_policy_operations blocklayout_policy_operations = {
.get_stripesize = bl_get_stripesize,
.pg_test = bl_pg_test,
};
-static struct pnfs_layoutdriver_type blocklayout_type = {
- .id = LAYOUT_BLOCK_VOLUME,
- .name = "LAYOUT_BLOCK_VOLUME",
- .ld_io_ops = &blocklayout_io_operations,
- .ld_policy_ops = &blocklayout_policy_operations,
-};
-
static int __init nfs4blocklayout_init(void)
{
int ret;
--
1.7.2.3
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/pnfs.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 1ccf8d8..9d53c3c 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -139,10 +139,14 @@ set_pnfs_layoutdriver(struct nfs_server *server, u32 id)
{
struct pnfs_layoutdriver_type *ld_type = NULL;
- BUG_ON(!(server->nfs_client->cl_exchange_flags &
- (EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_USE_PNFS_MDS)));
if (id == 0)
goto out_no_driver;
+ if (!(server->nfs_client->cl_exchange_flags &
+ (EXCHGID4_FLAG_USE_NON_PNFS | EXCHGID4_FLAG_USE_PNFS_MDS))) {
+ printk(KERN_ERR "%s: id %u cl_exchange_flags 0x%x\n", __func__,
+ id, server->nfs_client->cl_exchange_flags);
+ goto out_no_driver;
+ }
ld_type = find_pnfs_driver(id);
if (!ld_type) {
request_module("%s-%u", LAYOUT_NFSV4_1_MODULE_PREFIX, id);
--
1.7.2.3
---
fs/nfs/blocklayout/blocklayout.c | 2 +-
fs/nfs/blocklayout/blocklayout.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index 5dbd9c6..5e50c93 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -619,7 +619,7 @@ static int
bl_setup_layoutcommit(struct pnfs_layout_hdr *lo,
struct nfs4_layoutcommit_args *arg)
{
- struct nfs_server *nfss = PNFS_NFS_SERVER(lo);
+ struct nfs_server *nfss = NFS_SERVER(lo->inode);
struct bl_layoutupdate_data *layoutupdate_data;
dprintk("%s enter\n", __func__);
diff --git a/fs/nfs/blocklayout/blocklayout.h b/fs/nfs/blocklayout/blocklayout.h
index d62e3f9..7db7768 100644
--- a/fs/nfs/blocklayout/blocklayout.h
+++ b/fs/nfs/blocklayout/blocklayout.h
@@ -191,7 +191,7 @@ struct bl_layoutupdate_data {
struct list_head ranges;
};
-#define BLK_ID(lo) ((struct block_mount_id *)(PNFS_NFS_SERVER(lo)->pnfs_ld_data))
+#define BLK_ID(lo) ((struct block_mount_id *)(NFS_SERVER(lo->inode)->pnfs_ld_data))
static inline struct pnfs_block_layout *
BLK_LO2EXT(struct pnfs_layout_hdr *lo)
--
1.7.2.3
---
fs/nfs/blocklayout/blocklayout.c | 2 +-
fs/nfs/blocklayout/blocklayout.h | 10 +++++-----
fs/nfs/blocklayout/blocklayoutdev.c | 8 ++++----
fs/nfs/blocklayout/extents.c | 2 +-
4 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index f49c68c..071e7ef 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -683,7 +683,7 @@ static void free_blk_mountid(struct block_mount_id *mid)
*/
static struct pnfs_block_dev *
nfs4_blk_get_deviceinfo(struct nfs_server *server, const struct nfs_fh *fh,
- struct pnfs_deviceid *d_id,
+ struct nfs4_deviceid *d_id,
struct list_head *sdlist)
{
struct pnfs_device *dev;
diff --git a/fs/nfs/blocklayout/blocklayout.h b/fs/nfs/blocklayout/blocklayout.h
index fc0fb23..d62e3f9 100644
--- a/fs/nfs/blocklayout/blocklayout.h
+++ b/fs/nfs/blocklayout/blocklayout.h
@@ -55,7 +55,7 @@ struct block_mount_id {
struct pnfs_block_dev {
struct list_head bm_node;
- struct pnfs_deviceid bm_mdevid; /* associated devid */
+ struct nfs4_deviceid bm_mdevid; /* associated devid */
struct block_device *bm_mdev; /* meta device itself */
};
@@ -132,7 +132,7 @@ struct pnfs_inval_tracking {
struct pnfs_block_extent {
struct kref be_refcnt;
struct list_head be_node; /* link into lseg list */
- struct pnfs_deviceid be_devid; /* STUB - remevable??? */
+ struct nfs4_deviceid be_devid; /* STUB - remevable??? */
struct block_device *be_mdev;
sector_t be_f_offset; /* the starting offset in the file */
sector_t be_length; /* the size of the extent */
@@ -144,7 +144,7 @@ struct pnfs_block_extent {
/* Shortened extent used by LAYOUTCOMMIT */
struct pnfs_block_short_extent {
struct list_head bse_node;
- struct pnfs_deviceid bse_devid; /* STUB - removable??? */
+ struct nfs4_deviceid bse_devid; /* STUB - removable??? */
struct block_device *bse_mdev;
sector_t bse_f_offset; /* the starting offset in the file */
sector_t bse_length; /* the size of the extent */
@@ -226,7 +226,7 @@ uint32_t *blk_overflow(uint32_t *p, uint32_t *end, size_t nbytes);
memcpy((x), p, nbytes); \
p += XDR_QUADLEN(nbytes); \
} while (0)
-#define READ_DEVID(x) COPYMEM((x)->data, NFS4_PNFS_DEVICEID4_SIZE)
+#define READ_DEVID(x) COPYMEM((x)->data, NFS4_DEVICEID4_SIZE)
#define READ_SECTOR(x) do { \
READ64(tmp); \
if (tmp & 0x1ff) { \
@@ -248,7 +248,7 @@ uint32_t *blk_overflow(uint32_t *p, uint32_t *end, size_t nbytes);
#define WRITEMEM(ptr, nbytes) do { \
p = xdr_encode_opaque_fixed(p, ptr, nbytes); \
} while (0)
-#define WRITE_DEVID(x) WRITEMEM((x)->data, NFS4_PNFS_DEVICEID4_SIZE)
+#define WRITE_DEVID(x) WRITEMEM((x)->data, NFS4_DEVICEID4_SIZE)
/* blocklayoutdev.c */
struct block_device *nfs4_blkdev_get(dev_t dev);
diff --git a/fs/nfs/blocklayout/blocklayoutdev.c b/fs/nfs/blocklayout/blocklayoutdev.c
index e9ea86a..17bd25a 100644
--- a/fs/nfs/blocklayout/blocklayoutdev.c
+++ b/fs/nfs/blocklayout/blocklayoutdev.c
@@ -133,7 +133,7 @@ nfs4_blk_decode_device(struct nfs_server *server,
goto out_err;
rv->bm_mdev = bd;
- memcpy(&rv->bm_mdevid, &dev->dev_id, sizeof(struct pnfs_deviceid));
+ memcpy(&rv->bm_mdevid, &dev->dev_id, sizeof(struct nfs4_deviceid));
dprintk("%s Created device %s with bd_block_size %u\n",
__func__,
bd->bd_disk->disk_name,
@@ -153,7 +153,7 @@ out_err:
/* Map deviceid returned by the server to constructed block_device */
static struct block_device *translate_devid(struct pnfs_layout_hdr *lo,
- struct pnfs_deviceid *id)
+ struct nfs4_deviceid *id)
{
struct block_device *rv = NULL;
struct block_mount_id *mid;
@@ -164,7 +164,7 @@ static struct block_device *translate_devid(struct pnfs_layout_hdr *lo,
spin_lock(&mid->bm_lock);
list_for_each_entry(dev, &mid->bm_devlist, bm_node) {
if (memcmp(id->data, dev->bm_mdevid.data,
- NFS4_PNFS_DEVICEID4_SIZE) == 0) {
+ NFS4_DEVICEID4_SIZE) == 0) {
rv = dev->bm_mdev;
goto out;
}
@@ -254,7 +254,7 @@ nfs4_blk_process_layoutget(struct pnfs_layout_hdr *lo,
READ32(count);
dprintk("%s enter, number of extents %i\n", __func__, count);
- BLK_READBUF(p, end, (28 + NFS4_PNFS_DEVICEID4_SIZE) * count);
+ BLK_READBUF(p, end, (28 + NFS4_DEVICEID4_SIZE) * count);
/* Decode individual extents, putting them in temporary
* staging area until whole layout is decoded to make error
diff --git a/fs/nfs/blocklayout/extents.c b/fs/nfs/blocklayout/extents.c
index 20cc863..40dff82 100644
--- a/fs/nfs/blocklayout/extents.c
+++ b/fs/nfs/blocklayout/extents.c
@@ -794,7 +794,7 @@ _prep_new_extent(struct pnfs_block_extent *new,
{
kref_init(&new->be_refcnt);
/* don't need to INIT_LIST_HEAD(&new->be_node) */
- memcpy(&new->be_devid, &orig->be_devid, sizeof(struct pnfs_deviceid));
+ memcpy(&new->be_devid, &orig->be_devid, sizeof(struct nfs4_deviceid));
new->be_mdev = orig->be_mdev;
new->be_f_offset = offset;
new->be_length = length;
--
1.7.2.3
I recompiled kernel with debug symbols and problem is gone.
Crap!
Or, as Fred suspected, I did not apply you patch second time.
Regards,
Tigran.
On 09/29/2010 05:25 PM, Benny Halevy wrote:
> On 2010-09-29 16:36, Tigran Mkrtchyan wrote:
>> On 09/29/2010 04:22 PM, Benny Halevy wrote:
>>> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>>>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>>> I am support to publish first kernel crash.....
>>>> sorry to publish....
>>>>
>>>>> this is
>>>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>>>
>>>>> Regards,
>>>>> Tigran.
>>>>>
>>>>>
>>> Thanks for testing!
>>> Does the following patch help?
>>>
>> No :(
>>
> Tigran, do you have a new trace maybe?
> I cross referenced the dump you sent to the
> disassembled code and it seems like it hit NULL dereference on
> size = fl->stripe_unit;
>
> I'm going to commit this patch anyway but I want to
> understand what else I missed...
>
> Benny
>
>>> git diff --stat -p -M
>>> fs/nfs/nfs4filelayout.c | 2 ++
>>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>>> index 75e07c8..8fbb0db 100644
>>> --- a/fs/nfs/nfs4filelayout.c
>>> +++ b/fs/nfs/nfs4filelayout.c
>>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>>
>>> /* Horrible hack...ideally upper layer would send lseg */
>>> lseg = pnfs_has_layout(lo,&range);
>>> + if (!lseg)
>>> + return 0;
>>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>>> size = fl->stripe_unit;
>>> put_lseg_locked(lseg);
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
fixes "pnfs-obj use REQ flags rather than BIO flags"
Cc: Boaz Harrosh <[email protected]>
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/objlayout/objio_osd.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/objlayout/objio_osd.c b/fs/nfs/objlayout/objio_osd.c
index 223980b..75d5df4 100644
--- a/fs/nfs/objlayout/objio_osd.c
+++ b/fs/nfs/objlayout/objio_osd.c
@@ -910,7 +910,7 @@ static int _write_mirrors(struct objio_state *ios, unsigned cur_comp)
} else {
bio = master_dev->bio;
/* FIXME: bio_set_dir() */
- bio->bi_rw |= (1 << REQ_WRITE);
+ bio->bi_rw |= REQ_WRITE;
}
osd_req_write(or, &obj, per_dev->offset, bio, per_dev->length);
--
1.7.2.3
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/nfs4filelayoutdev.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4filelayoutdev.c b/fs/nfs/nfs4filelayoutdev.c
index bd76794..e4f91f6 100644
--- a/fs/nfs/nfs4filelayoutdev.c
+++ b/fs/nfs/nfs4filelayoutdev.c
@@ -29,6 +29,7 @@
*/
#include <linux/nfs_fs.h>
+#include <linux/vmalloc.h>
#include "internal.h"
#include "nfs4filelayout.h"
--
1.7.2.3
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/blocklayout/blocklayout.c | 19 ++++++++++---------
1 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index 071e7ef..5dbd9c6 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -44,7 +44,6 @@ MODULE_AUTHOR("Andy Adamson <[email protected]>");
MODULE_DESCRIPTION("The NFSv4.1 pNFS Block layout driver");
/* Callback operations to the pNFS client */
-static struct pnfs_client_operations *pnfs_block_callback_ops;
static void print_page(struct page *page)
{
@@ -199,7 +198,7 @@ static void bl_read_cleanup(struct work_struct *work)
dprintk("%s enter\n", __func__);
task = container_of(work, struct rpc_task, u.tk_work);
rdata = container_of(task, struct nfs_read_data, task);
- pnfs_block_callback_ops->nfs_readlist_complete(rdata);
+ pnfs_read_done(rdata);
}
static void
@@ -411,7 +410,7 @@ static void bl_write_cleanup(struct work_struct *work)
mark_extents_written(BLK_LSEG2EXT(wdata->pdata.lseg),
wdata->args.offset, wdata->args.count);
}
- pnfs_block_callback_ops->nfs_writelist_complete(wdata);
+ pnfs_writeback_done(wdata);
}
/* Called when last of bios associated with a bl_write_pagelist call finishes */
@@ -733,7 +732,7 @@ nfs4_blk_get_deviceinfo(struct nfs_server *server, const struct nfs_fh *fh,
dev->mincount = 0;
dprintk("%s: dev_id: %s\n", __func__, dev->dev_id.data);
- rc = pnfs_block_callback_ops->nfs_getdeviceinfo(server, dev);
+ rc = nfs4_proc_getdeviceinfo(server, dev);
dprintk("%s getdevice info returns %d\n", __func__, rc);
if (rc)
goto out_free;
@@ -783,8 +782,7 @@ bl_initialize_mountpoint(struct nfs_server *server, const struct nfs_fh *fh)
goto out_error;
dlist->eof = 0;
while (!dlist->eof) {
- status = pnfs_block_callback_ops->nfs_getdevicelist(
- server, fh, dlist);
+ status = nfs4_proc_getdevicelist(server, fh, dlist);
if (status)
goto out_error;
dprintk("%s GETDEVICELIST numdevs=%i, eof=%i\n",
@@ -1140,11 +1138,14 @@ static struct pnfs_layoutdriver_type blocklayout_type = {
static int __init nfs4blocklayout_init(void)
{
+ int ret;
+
dprintk("%s: NFSv4 Block Layout Driver Registering...\n", __func__);
- pnfs_block_callback_ops = pnfs_register_layoutdriver(&blocklayout_type);
- bl_pipe_init();
- return 0;
+ ret = pnfs_register_layoutdriver(&blocklayout_type);
+ if (!ret)
+ bl_pipe_init();
+ return ret;
}
static void __exit nfs4blocklayout_exit(void)
--
1.7.2.3
I am support to publish first kernel crash.....
this is
pnfs-all-2.6.36-rc6-2010-09-29 git c2b3b75c6b1db1dfc3f236200de1763e14c514d8
Regards,
Tigran.
On 09/29/2010 01:06 PM, Benny Halevy wrote:
> This version includes Fred's pnfs-submit RFC patches (v5).
> and is rebased onto v2.6.36-rc6
>
> Cumulative patches can be generated from
> git://linux-nfs.org/~bhalevy/linux-pnfs.git
> using
> git diff v2.6.36-rc6 pnfs-all-2.6.36-rc6-2010-09-29
>
> Or, they can be downloaded from the wiki at:
> http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
>
> Latest patches (since pnfs-all-2.6.36-rc3-2010-08-30):
>
> upstream:
> Boaz Harrosh cc59b6a um: Proper Fix for f25c80a4: remove duplicate structure field initialization
>
> pnfs-submit:
> The pNFS Team 1851dbc RFC: pnfsd, pnfs: protocol level pnfs constants
> The pNFS Team a1b60af RFC: nfs: change stateid to be a union
> The pNFS Team ff38ee4 RFC: nfs: ask for layouttypes during fsinfo call
> The pNFS Team 01f09aa RFC: nfs: set layout driver
> The pNFS Team f7893b6 RFC: pnfs: full mount/umount infrastructure
> The pNFS Team 1cef5e0 RFC: pnfs: filelayout: introduce minimal file layout driver
> The pNFS Team b7e2a3d RFC: nfs: create and destroy inode's layout cache
> The pNFS Team 34d2802 RFC: nfs: client needs to maintain list of inodes with active layouts
> The pNFS Team 292b90e RFC: pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure
> The pNFS Team bf610f6 RFC: pnfs: filelayout: add driver's LAYOUTGET and GETDEVICEINFO infrastructure
> Fred Isaman 068e761 pnfs-submit: revert range simplifications made by submission patches
> Benny Halevy 7c58c3d SQUASHME: pnfs-submit: add missing include file in nfs4filelayoutdev.c
> Benny Halevy 9a5956f SQUASHME: move nfs4_deviceid definitions to include/linux/nfs4.h
> Benny Halevy bb2afce SQUASHME: pnfs-submit: handle non-pnfs case in set_pnfs_layoutdriver
> Benny Halevy 998fe4a pnfs-submit: file needs layout commit, server attributes may be stale
> Benny Halevy f18cdf8 NFS: clear fsinfo before sendign rpc
>
> pnfs:
> Benny Halevy 52a6277 pnfs: alloc and free layout_hdr layoutdriver methods
> Benny Halevy 6dd521e pnfs: allow nfs4_proc_layoutget to sleep on invalid lsegs
> Benny Halevy ba1dd5b SQUASHME: pnfs: refactor put_lseg{_locked}
> Benny Halevy bb03ecf SQUASHME: pnfs: get rid of lo_waitq
>
> pnfs-block:
> Benny Halevy 73b1db9 SQUASHME: pnfsblock: remove obsolete include file from blocklayout.h
> Benny Halevy d9ebc26 SQUASHME: pnfsblock: use nfs4_deviceid
> Benny Halevy 90e2af1 SQUASHME: pnfsblock: no callback ops
> Benny Halevy 0953b90 SQAUSHME: pnfsblock: no PNFS_NFS_SERVER
> Benny Halevy 7ab3774 SQUASHME: pnfsblock: no dev_notify_types
> Benny Halevy f08f966 SQUASHME: pnfsblock: use new struct pnfs_layout_hdr
>
> pnfs-obj-all:
> Benny Halevy 41a70b7 SQUASHME: pnfs-obj: fix REQ flags usage
> Benny Halevy 6057701 SQUASHME: pnfs-obj: convert to new pnfs-submit changes
>
> pnfs-exofs-all:
> Benny Halevy 10d8e6e SQUASHME: pnfsd-exofs: convert to new pnfs-submit changes
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
These are pure protocol level constants
squash into "pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure"
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/pnfs.h | 6 ------
include/linux/nfs4.h | 6 ++++++
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 97c3bc4..9500aa4 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -38,8 +38,6 @@ struct pnfs_layout_segment {
struct pnfs_layout_hdr *layout;
};
-#define NFS4_DEVICEID4_SIZE 16
-
enum pnfs_try_status {
PNFS_ATTEMPTED = 0,
PNFS_NOT_ATTEMPTED = 1,
@@ -113,10 +111,6 @@ struct pnfs_layout_hdr {
struct inode *inode;
};
-struct nfs4_deviceid {
- char data[NFS4_DEVICEID4_SIZE];
-};
-
struct pnfs_device {
struct nfs4_deviceid dev_id;
unsigned int layout_type;
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index ae99e4f..3d04746 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -613,6 +613,12 @@ enum filelayout_hint_care4 {
NFLH4_CARE_STRIPE_COUNT = 0x00000080
};
+#define NFS4_DEVICEID4_SIZE 16
+
+struct nfs4_deviceid {
+ char data[NFS4_DEVICEID4_SIZE];
+};
+
#endif
#endif
--
1.7.2.3
I've rebased the following fixes (since pnfs-all-2.6.36-rc6-2010-09-29)
and released pnfs-all-2.6.36-rc6-2010-09-30
nfsd-upstream:
Benny Halevy a7cd1d3 nfsd4: adjust buflen for encoded attrs bitmap based on actual bitmap length
Benny Halevy 960a019 nfsd41: mask out unsupported pnfs attributes
pnfs-submit:
Benny Halevy 24ed91a SQUASHME: pnfs-submit: handle NULL lseg in filelayout_get_stripesize
Bian Naimeng 1691bd5 SQUASHME: pnfs-submit: specify the layout type before initialize mountpoint
pnfs-block:
Steve Dickson 0d4e550 SQUASHME: pnfsblock: compile error in blocklayout code
pnfsd-files:
Benny Halevy 435c888 pnfsd: mask out unsupported pnfs attributes only when !CONFIG_PNFSD
pnfsd:
Benny Halevy 8abfd95 SQUASHME: pnfsd: FATTR4_WORD2_LAYOUT_BLKSIZE is supported only under CONFIG_PNFSD
Benny Halevy 04b3521 SQUASHME: pnfsd: mask out FATTR4_WORD2_LAYOUT_BLKSIZE only when !CONFIG_PNFSD
spnfs:
Benny Halevy b6231c5 SQUASHME: pnfsd: factor out pnfsd check_export crap out of check_export
pnfsd-lexp:
Benny Halevy 01cda7e SQUASHME: pnfsd: factor out pnfsd check_export crap out of check_export
> Tigran, do you have a new trace maybe?
> I cross referenced the dump you sent to the
> disassembled code and it seems like it hit NULL dereference on
> size = fl->stripe_unit;
>
> I'm going to commit this patch anyway but I want to
> understand what else I missed...
>
> Benny
>
The stack attached. Shall I compile the kernel with some debug options
for better debugging?
Tigran.
>>> git diff --stat -p -M
>>> fs/nfs/nfs4filelayout.c | 2 ++
>>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>>> index 75e07c8..8fbb0db 100644
>>> --- a/fs/nfs/nfs4filelayout.c
>>> +++ b/fs/nfs/nfs4filelayout.c
>>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>>
>>> /* Horrible hack...ideally upper layer would send lseg */
>>> lseg = pnfs_has_layout(lo,&range);
>>> + if (!lseg)
>>> + return 0;
>>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>>> size = fl->stripe_unit;
>>> put_lseg_locked(lseg);
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
it has no more users at this point.
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/inode.c | 1 -
fs/nfs/pnfs.c | 6 +-----
include/linux/nfs_fs.h | 1 -
3 files changed, 1 insertions(+), 7 deletions(-)
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index e3d44a9..437d9a6 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1458,7 +1458,6 @@ static inline void nfs4_init_once(struct nfs_inode *nfsi)
nfsi->delegation = NULL;
nfsi->delegation_state = 0;
init_rwsem(&nfsi->rwsem);
- init_waitqueue_head(&nfsi->lo_waitq);
rpc_init_wait_queue(&nfsi->lo_rpcwaitq, "pNFS Layout");
nfsi->layout = NULL;
#endif
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 4e47c2a..0796c17 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -311,10 +311,8 @@ put_lseg_common(struct nfs_inode *nfsi, struct pnfs_layout_segment *lseg)
do_wake_up = !lseg->valid;
nfsi = NFS_I(lseg->layout->inode);
kref_put(&lseg->kref, destroy_lseg);
- if (do_wake_up) {
- wake_up(&nfsi->lo_waitq);
+ if (do_wake_up)
rpc_wake_up(&nfsi->lo_rpcwaitq);
- }
}
void
@@ -476,7 +474,6 @@ pnfs_layoutget_release(struct pnfs_layout_hdr *lo)
*/
put_layout_hdr_locked(lo);
spin_unlock(&nfsi->vfs_inode.i_lock);
- wake_up_all(&nfsi->lo_waitq);
}
void
@@ -494,7 +491,6 @@ pnfs_layoutreturn_release(struct pnfs_layout_hdr *lo,
*/
put_layout_hdr_locked(lo);
spin_unlock(&nfsi->vfs_inode.i_lock);
- wake_up_all(&nfsi->lo_waitq);
}
void
diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
index 1af18fd..059e61d 100644
--- a/include/linux/nfs_fs.h
+++ b/include/linux/nfs_fs.h
@@ -190,7 +190,6 @@ struct nfs_inode {
struct rw_semaphore rwsem;
/* pNFS layout information */
- wait_queue_head_t lo_waitq;
struct rpc_wait_queue lo_rpcwaitq;
struct pnfs_layout_hdr *layout;
#endif /* CONFIG_NFS_V4*/
--
1.7.2.3
[part of "pnfs: layoutcommit"]
Signed-off-by: Andy Adamson <[email protected]>
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/inode.c | 8 ++++++++
1 files changed, 8 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index 979483f..8cdea1a 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1212,6 +1212,14 @@ static int nfs_update_inode(struct inode *inode, struct nfs_fattr *fattr)
server->fsid = fattr->fsid;
/*
+ * file needs layout commit, server attributes may be stale
+ */
+ if (layoutcommit_needed(nfsi) && nfsi->change_attr >= fattr->change_attr) {
+ dprintk("NFS: %s: layoutcommit is needed for file %s/%ld\n",
+ __func__, inode->i_sb->s_id, inode->i_ino);
+ return 0;
+ }
+ /*
* Update the read time so we don't revalidate too often.
*/
nfsi->read_cache_jiffies = fattr->time_start;
--
1.7.2.3
To initialize all values to zero, in case the server or protocol version
do no support particular attributes.
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/client.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 23d740c..beb206e 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -942,6 +942,7 @@ static int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *mntfh, str
goto out_error;
}
+ memset(&fsinfo, 0, sizeof(fsinfo));
fsinfo.fattr = fattr;
error = clp->rpc_ops->fsinfo(server, mntfh, &fsinfo);
if (error < 0)
--
1.7.2.3
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/objlayout/objio_osd.c | 57 ++++++++++++++++++++----------
fs/nfs/objlayout/objlayout.c | 77 ++++++++++++++++-------------------------
fs/nfs/objlayout/objlayout.h | 47 ++++++++++++++++++++++---
fs/nfs/objlayout/panfs_shim.c | 51 ++++++++++++++++++---------
include/linux/nfs4.h | 2 +
include/linux/pnfs_osd_xdr.h | 4 +-
6 files changed, 147 insertions(+), 91 deletions(-)
diff --git a/fs/nfs/objlayout/objio_osd.c b/fs/nfs/objlayout/objio_osd.c
index 75d5df4..696b6f5 100644
--- a/fs/nfs/objlayout/objio_osd.c
+++ b/fs/nfs/objlayout/objio_osd.c
@@ -64,7 +64,7 @@ struct objio_mount_type {
struct _dev_ent {
struct list_head list;
- struct pnfs_deviceid d_id;
+ struct nfs4_deviceid d_id;
struct osd_dev *od;
};
@@ -85,7 +85,7 @@ static void _dev_list_remove_all(struct objio_mount_type *omt)
}
static struct osd_dev *___dev_list_find(struct objio_mount_type *omt,
- struct pnfs_deviceid *d_id)
+ struct nfs4_deviceid *d_id)
{
struct list_head *le;
@@ -100,7 +100,7 @@ static struct osd_dev *___dev_list_find(struct objio_mount_type *omt,
}
static struct osd_dev *_dev_list_find(struct objio_mount_type *omt,
- struct pnfs_deviceid *d_id)
+ struct nfs4_deviceid *d_id)
{
struct osd_dev *od;
@@ -111,7 +111,7 @@ static struct osd_dev *_dev_list_find(struct objio_mount_type *omt,
}
static int _dev_list_add(struct objio_mount_type *omt,
- struct pnfs_deviceid *d_id, struct osd_dev *od)
+ struct nfs4_deviceid *d_id, struct osd_dev *od)
{
struct _dev_ent *de = kzalloc(sizeof(*de), GFP_KERNEL);
@@ -180,10 +180,10 @@ static struct osd_dev *_device_lookup(struct pnfs_layout_hdr *pnfslay,
{
struct pnfs_osd_layout *layout = objio_seg->layout;
struct pnfs_osd_deviceaddr *deviceaddr;
- struct pnfs_deviceid *d_id;
+ struct nfs4_deviceid *d_id;
struct osd_dev *od;
struct osd_dev_info odi;
- struct objio_mount_type *omt = PNFS_NFS_SERVER(pnfslay)->pnfs_ld_data;
+ struct objio_mount_type *omt = NFS_SERVER(pnfslay->inode)->pnfs_ld_data;
int err;
d_id = &layout->olo_comps[comp].oc_object_id.oid_device_id;
@@ -981,7 +981,8 @@ objlayout_get_stripesize(struct pnfs_layout_hdr *pnfslay)
list_for_each_entry(lseg, &pnfslay->segs, fi_list) {
int n;
- struct objlayout_segment *objlseg = LSEG_LD_DATA(lseg);
+ struct objlayout_segment *objlseg =
+ container_of(lseg, struct objlayout_segment, lseg);
struct pnfs_osd_layout *lo =
(struct pnfs_osd_layout *)objlseg->pnfs_osd_layout;
struct pnfs_osd_data_map *map = &lo->olo_map;
@@ -1025,23 +1026,35 @@ objlayout_get_blocksize(void)
return sz;
}
-static struct layoutdriver_policy_operations objlayout_policy_operations = {
/*
* Don't gather across stripes, but rather gather (coalesce) up to
* the stripe size.
*
* FIXME: change interface to use merge_align, merge_count
*/
- .flags = PNFS_LAYOUTRET_ON_SETATTR,
- .get_stripesize = objlayout_get_stripesize,
- .get_blocksize = objlayout_get_blocksize,
-};
-
static struct pnfs_layoutdriver_type objlayout_type = {
.id = LAYOUT_OSD2_OBJECTS,
.name = "LAYOUT_OSD2_OBJECTS",
- .ld_io_ops = &objlayout_io_operations,
- .ld_policy_ops = &objlayout_policy_operations,
+ .flags = PNFS_LAYOUTRET_ON_SETATTR,
+
+ .initialize_mountpoint = objlayout_initialize_mountpoint,
+ .uninitialize_mountpoint = objlayout_uninitialize_mountpoint,
+
+ .alloc_layout_hdr = objlayout_alloc_layout_hdr,
+ .free_layout_hdr = objlayout_free_layout_hdr,
+
+ .alloc_lseg = objlayout_alloc_lseg,
+ .free_lseg = objlayout_free_lseg,
+
+ .get_stripesize = objlayout_get_stripesize,
+ .get_blocksize = objlayout_get_blocksize,
+
+ .read_pagelist = objlayout_read_pagelist,
+ .write_pagelist = objlayout_write_pagelist,
+ .commit = objlayout_commit,
+
+ .encode_layoutcommit = objlayout_encode_layoutcommit,
+ .encode_layoutreturn = objlayout_encode_layoutreturn,
};
void *objio_init_mt(void)
@@ -1069,10 +1082,16 @@ MODULE_LICENSE("GPL");
static int __init
objlayout_init(void)
{
- pnfs_client_ops = pnfs_register_layoutdriver(&objlayout_type);
- printk(KERN_INFO "%s: Registered OSD pNFS Layout Driver\n",
- __func__);
- return 0;
+ int ret = pnfs_register_layoutdriver(&objlayout_type);
+
+ if (ret)
+ printk(KERN_INFO
+ "%s: Registering OSD pNFS Layout Driver failed: error=%d\n",
+ __func__, ret);
+ else
+ printk(KERN_INFO "%s: Registered OSD pNFS Layout Driver\n",
+ __func__);
+ return ret;
}
static void __exit
diff --git a/fs/nfs/objlayout/objlayout.c b/fs/nfs/objlayout/objlayout.c
index 259c616..b647577 100644
--- a/fs/nfs/objlayout/objlayout.c
+++ b/fs/nfs/objlayout/objlayout.c
@@ -49,8 +49,8 @@ struct pnfs_client_operations *pnfs_client_ops;
/*
* Create a objlayout layout structure for the given inode and return it.
*/
-static struct pnfs_layout_hdr *
-objlayout_alloc_layout(struct inode *inode)
+struct pnfs_layout_hdr *
+objlayout_alloc_layout_hdr(struct inode *inode)
{
struct objlayout *objlay;
@@ -66,8 +66,8 @@ objlayout_alloc_layout(struct inode *inode)
/*
* Free an objlayout layout structure
*/
-static void
-objlayout_free_layout(struct pnfs_layout_hdr *lo)
+void
+objlayout_free_layout_hdr(struct pnfs_layout_hdr *lo)
{
struct objlayout *objlay = OBJLAYOUT(lo);
@@ -80,13 +80,12 @@ objlayout_free_layout(struct pnfs_layout_hdr *lo)
/*
* Unmarshall layout and store it in pnfslay.
*/
-static struct pnfs_layout_segment *
+struct pnfs_layout_segment *
objlayout_alloc_lseg(struct pnfs_layout_hdr *pnfslay,
struct nfs4_layoutget_res *lgr)
{
int status;
void *layout = lgr->layout.buf;
- struct pnfs_layout_segment *lseg;
struct objlayout_segment *objlseg;
struct pnfs_osd_layout *pnfs_osd_layout;
@@ -95,32 +94,31 @@ objlayout_alloc_lseg(struct pnfs_layout_hdr *pnfslay,
BUG_ON(!layout);
status = -ENOMEM;
- lseg = kzalloc(sizeof(*lseg) + sizeof(*objlseg) +
- pnfs_osd_layout_incore_sz(layout), GFP_KERNEL);
- if (!lseg)
+ objlseg = kzalloc(sizeof(*objlseg) +
+ pnfs_osd_layout_incore_sz(layout), GFP_KERNEL);
+ if (!objlseg)
goto err;
- objlseg = LSEG_LD_DATA(lseg);
pnfs_osd_layout = (struct pnfs_osd_layout *)objlseg->pnfs_osd_layout;
pnfs_osd_xdr_decode_layout(pnfs_osd_layout, layout);
- status = objio_alloc_lseg(&objlseg->internal, pnfslay, lseg,
+ status = objio_alloc_lseg(&objlseg->internal, pnfslay, &objlseg->lseg,
pnfs_osd_layout);
if (status)
goto err;
- dprintk("%s: Return %p\n", __func__, lseg);
- return lseg;
+ dprintk("%s: Return %p\n", __func__, &objlseg->lseg);
+ return &objlseg->lseg;
err:
- kfree(lseg);
+ kfree(objlseg);
return ERR_PTR(status);
}
/*
* Free a layout segement
*/
-static void
+void
objlayout_free_lseg(struct pnfs_layout_segment *lseg)
{
struct objlayout_segment *objlseg;
@@ -130,9 +128,9 @@ objlayout_free_lseg(struct pnfs_layout_segment *lseg)
if (unlikely(!lseg))
return;
- objlseg = LSEG_LD_DATA(lseg);
+ objlseg = container_of(lseg, struct objlayout_segment, lseg);
objio_free_lseg(objlseg->internal);
- kfree(lseg);
+ kfree(objlseg);
}
/*
@@ -168,7 +166,8 @@ objlayout_alloc_io_state(struct pnfs_layout_hdr *pnfs_layout_type,
struct pnfs_layout_segment *lseg,
void *rpcdata)
{
- struct objlayout_segment *objlseg = LSEG_LD_DATA(lseg);
+ struct objlayout_segment *objlseg =
+ container_of(lseg, struct objlayout_segment, lseg);
struct objlayout_io_state *state;
u64 lseg_end_offset;
size_t size_nr_pages;
@@ -199,7 +198,7 @@ objlayout_alloc_io_state(struct pnfs_layout_hdr *pnfs_layout_type,
nr_pages = size_nr_pages;
INIT_LIST_HEAD(&state->err_list);
- state->lseg = lseg;
+ state->objlseg = objlseg;
state->rpcdata = rpcdata;
state->pages = pages;
state->pgbase = pgbase;
@@ -232,7 +231,7 @@ objlayout_iodone(struct objlayout_io_state *state)
if (likely(state->status >= 0)) {
objlayout_free_io_state(state);
} else {
- struct objlayout *objlay = OBJLAYOUT(state->lseg->layout);
+ struct objlayout *objlay = OBJLAYOUT(state->objlseg->lseg.layout);
spin_lock(&objlay->lock);
objlay->delta_space_valid = OBJ_DSU_INVALID;
@@ -255,9 +254,8 @@ objlayout_io_set_result(struct objlayout_io_state *state, unsigned index,
BUG_ON(index >= state->num_comps);
if (osd_error) {
- struct objlayout_segment *objlseg = LSEG_LD_DATA(state->lseg);
struct pnfs_osd_layout *layout =
- (typeof(layout))objlseg->pnfs_osd_layout;
+ (typeof(layout))state->objlseg->pnfs_osd_layout;
ioerr->oer_component = layout->olo_comps[index].oc_object_id;
ioerr->oer_comp_offset = offset;
@@ -290,7 +288,7 @@ static void _rpc_commit_complete(struct work_struct *work)
task = container_of(work, struct rpc_task, u.tk_work);
wdata = container_of(task, struct nfs_write_data, task);
- pnfs_client_ops->nfs_commit_complete(wdata);
+ pnfs_commit_done(wdata);
}
/*
@@ -320,7 +318,7 @@ static void _rpc_read_complete(struct work_struct *work)
task = container_of(work, struct rpc_task, u.tk_work);
rdata = container_of(task, struct nfs_read_data, task);
- pnfs_client_ops->nfs_readlist_complete(rdata);
+ pnfs_read_done(rdata);
}
void
@@ -341,7 +339,7 @@ objlayout_read_done(struct objlayout_io_state *state, ssize_t status, bool sync)
/* must not use state after this point */
if (sync)
- pnfs_client_ops->nfs_readlist_complete(rdata);
+ pnfs_read_done(rdata);
else {
INIT_WORK(&rdata->task.u.tk_work, _rpc_read_complete);
schedule_work(&rdata->task.u.tk_work);
@@ -405,7 +403,7 @@ static void _rpc_write_complete(struct work_struct *work)
task = container_of(work, struct rpc_task, u.tk_work);
wdata = container_of(task, struct nfs_write_data, task);
- pnfs_client_ops->nfs_writelist_complete(wdata);
+ pnfs_writeback_done(wdata);
}
void
@@ -431,7 +429,7 @@ objlayout_write_done(struct objlayout_io_state *state, ssize_t status,
/* must not use state after this point */
if (sync)
- pnfs_client_ops->nfs_writelist_complete(wdata);
+ pnfs_writeback_done(wdata);
else {
INIT_WORK(&wdata->task.u.tk_work, _rpc_write_complete);
schedule_work(&wdata->task.u.tk_work);
@@ -683,7 +681,7 @@ struct objlayout_deviceinfo {
* should be called.
*/
int objlayout_get_deviceinfo(struct pnfs_layout_hdr *pnfslay,
- struct pnfs_deviceid *d_id, struct pnfs_osd_deviceaddr **deviceaddr)
+ struct nfs4_deviceid *d_id, struct pnfs_osd_deviceaddr **deviceaddr)
{
struct objlayout_deviceinfo *odi;
struct pnfs_device pd;
@@ -701,14 +699,13 @@ int objlayout_get_deviceinfo(struct pnfs_layout_hdr *pnfslay,
memcpy(&pd.dev_id, d_id, sizeof(*d_id));
pd.layout_type = LAYOUT_OSD2_OBJECTS;
- pd.dev_notify_types = 0;
pd.pages = &page;
pd.pgbase = 0;
pd.pglen = PAGE_SIZE;
pd.mincount = 0;
- sb = PNFS_INODE(pnfslay)->i_sb;
- err = pnfs_client_ops->nfs_getdeviceinfo(PNFS_NFS_SERVER(pnfslay), &pd);
+ sb = pnfslay->inode->i_sb;
+ err = nfs4_proc_getdeviceinfo(NFS_SERVER(pnfslay->inode), &pd);
dprintk("%s nfs_getdeviceinfo returned %d\n", __func__, err);
if (err)
goto err_out;
@@ -746,7 +743,7 @@ void objlayout_put_deviceinfo(struct pnfs_osd_deviceaddr *deviceaddr)
* Return the pnfs_mount_type structure so the
* pNFS_client can refer to the mount point later on.
*/
-static int
+int
objlayout_initialize_mountpoint(struct nfs_server *server,
const struct nfs_fh *mntfh)
{
@@ -767,24 +764,10 @@ objlayout_initialize_mountpoint(struct nfs_server *server,
/*
* Uninitialize a mountpoint
*/
-static int
+int
objlayout_uninitialize_mountpoint(struct nfs_server *server)
{
dprintk("%s: Begin %p\n", __func__, server->pnfs_ld_data);
objio_fini_mt(server->pnfs_ld_data);
return 0;
}
-
-struct layoutdriver_io_operations objlayout_io_operations = {
- .commit = objlayout_commit,
- .read_pagelist = objlayout_read_pagelist,
- .write_pagelist = objlayout_write_pagelist,
- .alloc_layout = objlayout_alloc_layout,
- .free_layout = objlayout_free_layout,
- .alloc_lseg = objlayout_alloc_lseg,
- .free_lseg = objlayout_free_lseg,
- .encode_layoutcommit = objlayout_encode_layoutcommit,
- .encode_layoutreturn = objlayout_encode_layoutreturn,
- .initialize_mountpoint = objlayout_initialize_mountpoint,
- .uninitialize_mountpoint = objlayout_uninitialize_mountpoint,
-};
diff --git a/fs/nfs/objlayout/objlayout.h b/fs/nfs/objlayout/objlayout.h
index adec7ad..cad24a4 100644
--- a/fs/nfs/objlayout/objlayout.h
+++ b/fs/nfs/objlayout/objlayout.h
@@ -44,13 +44,14 @@
#define _OBJLAYOUT_H
#include <linux/nfs_fs.h>
-#include <linux/nfs4_pnfs.h>
#include <linux/pnfs_osd_xdr.h>
+#include "../pnfs.h"
/*
* in-core layout segment
*/
struct objlayout_segment {
+ struct pnfs_layout_segment lseg;
void *internal; /* for provider internal use */
u8 pnfs_osd_layout[];
};
@@ -85,7 +86,7 @@ OBJLAYOUT(struct pnfs_layout_hdr *lo)
* embedded in objects provider io_state data structure
*/
struct objlayout_io_state {
- struct pnfs_layout_segment *lseg;
+ struct objlayout_segment *objlseg;
struct page **pages;
unsigned pgbase;
@@ -139,7 +140,7 @@ extern void objlayout_io_set_result(struct objlayout_io_state *state,
static inline void
objlayout_add_delta_space_used(struct objlayout_io_state *state, s64 space_used)
{
- struct objlayout *objlay = OBJLAYOUT(state->lseg->layout);
+ struct objlayout *objlay = OBJLAYOUT(state->objlseg->lseg.layout);
/* If one of the I/Os errored out and the delta_space_used was
* invalid we render the complete report as invalid. Protocol mandate
@@ -159,13 +160,47 @@ extern void objlayout_write_done(struct objlayout_io_state *state,
ssize_t status, bool sync);
extern int objlayout_get_deviceinfo(struct pnfs_layout_hdr *pnfslay,
- struct pnfs_deviceid *d_id, struct pnfs_osd_deviceaddr **deviceaddr);
+ struct nfs4_deviceid *d_id, struct pnfs_osd_deviceaddr **deviceaddr);
extern void objlayout_put_deviceinfo(struct pnfs_osd_deviceaddr *deviceaddr);
/*
* exported generic objects function vectors
*/
-extern struct layoutdriver_io_operations objlayout_io_operations;
-extern struct pnfs_client_operations *pnfs_client_ops;
+
+extern int objlayout_initialize_mountpoint(
+ struct nfs_server *,
+ const struct nfs_fh *);
+extern int objlayout_uninitialize_mountpoint(struct nfs_server *);
+
+extern struct pnfs_layout_hdr *objlayout_alloc_layout_hdr(struct inode *);
+extern void objlayout_free_layout_hdr(struct pnfs_layout_hdr *);
+
+extern struct pnfs_layout_segment *objlayout_alloc_lseg(
+ struct pnfs_layout_hdr *,
+ struct nfs4_layoutget_res *);
+extern void objlayout_free_lseg(struct pnfs_layout_segment *);
+
+extern enum pnfs_try_status objlayout_read_pagelist(
+ struct nfs_read_data *,
+ unsigned nr_pages);
+
+extern enum pnfs_try_status objlayout_write_pagelist(
+ struct nfs_write_data *,
+ unsigned nr_pages,
+ int how);
+
+extern enum pnfs_try_status objlayout_commit(
+ struct nfs_write_data *,
+ int how);
+
+extern void objlayout_encode_layoutcommit(
+ struct pnfs_layout_hdr *,
+ struct xdr_stream *,
+ const struct nfs4_layoutcommit_args *);
+
+extern void objlayout_encode_layoutreturn(
+ struct pnfs_layout_hdr *,
+ struct xdr_stream *,
+ const struct nfs4_layoutreturn_args *);
#endif /* _OBJLAYOUT_H */
diff --git a/fs/nfs/objlayout/panfs_shim.c b/fs/nfs/objlayout/panfs_shim.c
index fd46e96..9a499a9 100644
--- a/fs/nfs/objlayout/panfs_shim.c
+++ b/fs/nfs/objlayout/panfs_shim.c
@@ -463,8 +463,7 @@ objio_read_pagelist(struct objlayout_io_state *ol_state)
{
struct panfs_shim_io_state *state = container_of(ol_state,
struct panfs_shim_io_state, ol_state);
- struct objlayout_segment *lseg = LSEG_LD_DATA(ol_state->lseg);
- pan_sm_map_cap_t *mcs = (pan_sm_map_cap_t *)lseg->internal;
+ pan_sm_map_cap_t *mcs = (pan_sm_map_cap_t *)ol_state->objlseg->internal;
ssize_t status = 0;
pan_status_t rc = PAN_SUCCESS;
@@ -544,8 +543,7 @@ objio_write_pagelist(struct objlayout_io_state *ol_state,
{
struct panfs_shim_io_state *state = container_of(ol_state,
struct panfs_shim_io_state, ol_state);
- struct objlayout_segment *lseg = LSEG_LD_DATA(ol_state->lseg);
- pan_sm_map_cap_t *mcs = (pan_sm_map_cap_t *)lseg->internal;
+ pan_sm_map_cap_t *mcs = (pan_sm_map_cap_t *)ol_state->objlseg->internal;
ssize_t status = 0;
pan_status_t rc = PAN_SUCCESS;
@@ -636,7 +634,8 @@ panlayout_get_stripesize(struct pnfs_layout_hdr *pnfslay)
list_for_each_entry(lseg, &pnfslay->segs, fi_list) {
int n;
- struct objlayout_segment *panlseg = LSEG_LD_DATA(lseg);
+ struct objlayout_segment *panlseg =
+ container_of(lseg, struct objlayout_segment, lseg);
struct pnfs_osd_layout *lo =
(struct pnfs_osd_layout *)panlseg->pnfs_osd_layout;
struct pnfs_osd_data_map *map = &lo->olo_map;
@@ -688,25 +687,37 @@ panlayout_get_blocksize(void)
return sz;
}
-static struct layoutdriver_policy_operations panlayout_policy_operations = {
/*
* Don't gather across stripes, but rather gather (coalesce) up to
* the stripe size.
*
* FIXME: change interface to use merge_align, merge_count
*/
- .flags = PNFS_LAYOUTRET_ON_SETATTR,
- .get_stripesize = panlayout_get_stripesize,
- .get_blocksize = panlayout_get_blocksize,
-};
-
#define PNFS_LAYOUT_PANOSD (NFS4_PNFS_PRIVATE_LAYOUT | LAYOUT_OSD2_OBJECTS)
static struct pnfs_layoutdriver_type panlayout_type = {
.id = PNFS_LAYOUT_PANOSD,
.name = "PNFS_LAYOUT_PANOSD",
- .ld_io_ops = &objlayout_io_operations,
- .ld_policy_ops = &panlayout_policy_operations,
+ .flags = PNFS_LAYOUTRET_ON_SETATTR,
+
+ .initialize_mountpoint = objlayout_initialize_mountpoint,
+ .uninitialize_mountpoint = objlayout_uninitialize_mountpoint,
+
+ .alloc_layout_hdr = objlayout_alloc_layout_hdr,
+ .free_layout_hdr = objlayout_free_layout_hdr,
+
+ .alloc_lseg = objlayout_alloc_lseg,
+ .free_lseg = objlayout_free_lseg,
+
+ .get_stripesize = panlayout_get_stripesize,
+ .get_blocksize = panlayout_get_blocksize,
+
+ .read_pagelist = objlayout_read_pagelist,
+ .write_pagelist = objlayout_write_pagelist,
+ .commit = objlayout_commit,
+
+ .encode_layoutcommit = objlayout_encode_layoutcommit,
+ .encode_layoutreturn = objlayout_encode_layoutreturn,
};
MODULE_DESCRIPTION("pNFS Layout Driver for Panasas OSDs");
@@ -716,10 +727,16 @@ MODULE_LICENSE("GPL");
static int __init
panlayout_init(void)
{
- pnfs_client_ops = pnfs_register_layoutdriver(&panlayout_type);
- printk(KERN_INFO "%s: Registered Panasas OSD pNFS Layout Driver\n",
- __func__);
- return 0;
+ int ret = pnfs_register_layoutdriver(&panlayout_type);
+
+ if (ret)
+ printk(KERN_INFO
+ "%s: Registering Panasas OSD pNFS Layout Driver failed: error=%d\n",
+ __func__, ret);
+ else
+ printk(KERN_INFO "%s: Registered Panasas OSD pNFS Layout Driver\n",
+ __func__);
+ return ret;
}
static void __exit
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index fef61e4..df29296 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -588,6 +588,8 @@ enum pnfs_layouttype {
LAYOUT_NFSV4_1_FILES = 1,
LAYOUT_OSD2_OBJECTS = 2,
LAYOUT_BLOCK_VOLUME = 3,
+
+ NFS4_PNFS_PRIVATE_LAYOUT = 0x80000000
};
/* used for both layout return and recall */
diff --git a/include/linux/pnfs_osd_xdr.h b/include/linux/pnfs_osd_xdr.h
index 8fc21b0..b404f33 100644
--- a/include/linux/pnfs_osd_xdr.h
+++ b/include/linux/pnfs_osd_xdr.h
@@ -118,7 +118,7 @@ pnfs_osd_data_map_incore_sz(void)
* };
*/
struct pnfs_osd_objid {
- struct pnfs_deviceid oid_device_id;
+ struct nfs4_deviceid oid_device_id;
u64 oid_partition_id;
u64 oid_object_id;
};
@@ -133,7 +133,7 @@ struct pnfs_osd_objid {
static inline int
pnfs_osd_objid_xdr_sz(void)
{
- return (NFS4_PNFS_DEVICEID4_SIZE / 4) + 2 + 2;
+ return (NFS4_DEVICEID4_SIZE / 4) + 2 + 2;
}
static inline size_t
--
1.7.2.3
This version includes Fred's pnfs-submit RFC patches (v5).
and is rebased onto v2.6.36-rc6
Cumulative patches can be generated from
git://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.36-rc6 pnfs-all-2.6.36-rc6-2010-09-29
Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
Latest patches (since pnfs-all-2.6.36-rc3-2010-08-30):
upstream:
Boaz Harrosh cc59b6a um: Proper Fix for f25c80a4: remove duplicate structure field initialization
pnfs-submit:
The pNFS Team 1851dbc RFC: pnfsd, pnfs: protocol level pnfs constants
The pNFS Team a1b60af RFC: nfs: change stateid to be a union
The pNFS Team ff38ee4 RFC: nfs: ask for layouttypes during fsinfo call
The pNFS Team 01f09aa RFC: nfs: set layout driver
The pNFS Team f7893b6 RFC: pnfs: full mount/umount infrastructure
The pNFS Team 1cef5e0 RFC: pnfs: filelayout: introduce minimal file layout driver
The pNFS Team b7e2a3d RFC: nfs: create and destroy inode's layout cache
The pNFS Team 34d2802 RFC: nfs: client needs to maintain list of inodes with active layouts
The pNFS Team 292b90e RFC: pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure
The pNFS Team bf610f6 RFC: pnfs: filelayout: add driver's LAYOUTGET and GETDEVICEINFO infrastructure
Fred Isaman 068e761 pnfs-submit: revert range simplifications made by submission patches
Benny Halevy 7c58c3d SQUASHME: pnfs-submit: add missing include file in nfs4filelayoutdev.c
Benny Halevy 9a5956f SQUASHME: move nfs4_deviceid definitions to include/linux/nfs4.h
Benny Halevy bb2afce SQUASHME: pnfs-submit: handle non-pnfs case in set_pnfs_layoutdriver
Benny Halevy 998fe4a pnfs-submit: file needs layout commit, server attributes may be stale
Benny Halevy f18cdf8 NFS: clear fsinfo before sendign rpc
pnfs:
Benny Halevy 52a6277 pnfs: alloc and free layout_hdr layoutdriver methods
Benny Halevy 6dd521e pnfs: allow nfs4_proc_layoutget to sleep on invalid lsegs
Benny Halevy ba1dd5b SQUASHME: pnfs: refactor put_lseg{_locked}
Benny Halevy bb03ecf SQUASHME: pnfs: get rid of lo_waitq
pnfs-block:
Benny Halevy 73b1db9 SQUASHME: pnfsblock: remove obsolete include file from blocklayout.h
Benny Halevy d9ebc26 SQUASHME: pnfsblock: use nfs4_deviceid
Benny Halevy 90e2af1 SQUASHME: pnfsblock: no callback ops
Benny Halevy 0953b90 SQAUSHME: pnfsblock: no PNFS_NFS_SERVER
Benny Halevy 7ab3774 SQUASHME: pnfsblock: no dev_notify_types
Benny Halevy f08f966 SQUASHME: pnfsblock: use new struct pnfs_layout_hdr
pnfs-obj-all:
Benny Halevy 41a70b7 SQUASHME: pnfs-obj: fix REQ flags usage
Benny Halevy 6057701 SQUASHME: pnfs-obj: convert to new pnfs-submit changes
pnfs-exofs-all:
Benny Halevy 10d8e6e SQUASHME: pnfsd-exofs: convert to new pnfs-submit changes
move common code into put_lseg_common
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/nfs4proc.c | 8 ++++++++
fs/nfs/pnfs.c | 36 ++++++++++++++----------------------
fs/nfs/pnfs.h | 1 +
3 files changed, 23 insertions(+), 22 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index ebf5127..52c5ddf 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5664,9 +5664,17 @@ nfs4_layoutreturn_prepare(struct rpc_task *task, void *calldata)
{
struct nfs4_layoutreturn *lrp = calldata;
struct inode *ino = lrp->args.inode;
+ struct nfs_inode *nfsi = NFS_I(ino);
struct nfs_server *server = NFS_SERVER(ino);
dprintk("--> %s\n", __func__);
+ if ((lrp->args.return_type == RETURN_FILE) &&
+ pnfs_return_layout_barrier(nfsi, &lrp->args.range)) {
+ dprintk("%s: waiting on barrier\n", __func__);
+ rpc_sleep_on(&nfsi->lo_rpcwaitq, task, NULL);
+ return;
+ }
+
if (nfs4_setup_sequence(server, NULL, &lrp->args.seq_args,
&lrp->res.seq_res, 0, task))
return;
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 31a703d..4e47c2a 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -301,14 +301,10 @@ destroy_lseg(struct kref *kref)
put_layout_hdr_locked(local);
}
-void
-put_lseg_locked(struct pnfs_layout_segment *lseg)
+static void
+put_lseg_common(struct nfs_inode *nfsi, struct pnfs_layout_segment *lseg)
{
bool do_wake_up;
- struct nfs_inode *nfsi;
-
- if (!lseg)
- return;
dprintk("%s: lseg %p ref %d valid %d\n", __func__, lseg,
atomic_read(&lseg->kref.refcount), lseg->valid);
@@ -320,12 +316,21 @@ put_lseg_locked(struct pnfs_layout_segment *lseg)
rpc_wake_up(&nfsi->lo_rpcwaitq);
}
}
+
+void
+put_lseg_locked(struct pnfs_layout_segment *lseg)
+{
+ if (!lseg)
+ return;
+
+ assert_spin_locked(&lseg->layout->inode->i_lock);
+ put_lseg_common(NFS_I(lseg->layout->inode), lseg);
+}
EXPORT_SYMBOL_GPL(put_lseg_locked);
void
put_lseg(struct pnfs_layout_segment *lseg)
{
- bool do_wake_up;
struct nfs_inode *nfsi;
if (!lseg)
@@ -333,13 +338,10 @@ put_lseg(struct pnfs_layout_segment *lseg)
dprintk("%s: lseg %p ref %d valid %d\n", __func__, lseg,
atomic_read(&lseg->kref.refcount), lseg->valid);
- do_wake_up = !lseg->valid;
nfsi = NFS_I(lseg->layout->inode);
spin_lock(&nfsi->vfs_inode.i_lock);
- kref_put(&lseg->kref, destroy_lseg);
+ put_lseg_common(nfsi, lseg);
spin_unlock(&nfsi->vfs_inode.i_lock);
- if (do_wake_up)
- wake_up(&nfsi->lo_waitq);
}
EXPORT_SYMBOL_GPL(put_lseg);
@@ -654,7 +656,7 @@ has_layout_to_return(struct pnfs_layout_hdr *lo,
return out;
}
-static bool
+bool
pnfs_return_layout_barrier(struct nfs_inode *nfsi,
struct pnfs_layout_range *range)
{
@@ -743,16 +745,6 @@ _pnfs_return_layout(struct inode *ino, struct pnfs_layout_range *range,
spin_unlock(&ino->i_lock);
- if (pnfs_return_layout_barrier(nfsi, &arg)) {
- if (stateid) { /* callback */
- status = -EAGAIN;
- goto out_put;
- }
- dprintk("%s: waiting\n", __func__);
- wait_event(nfsi->lo_waitq,
- !pnfs_return_layout_barrier(nfsi, &arg));
- }
-
if (layoutcommit_needed(nfsi)) {
if (stateid && !wait) { /* callback */
dprintk("%s: layoutcommit pending\n", __func__);
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 36ce9fa..b143390 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -239,6 +239,7 @@ pnfs_has_layout(struct pnfs_layout_hdr *lo, struct pnfs_layout_range *range);
struct pnfs_layout_segment *
pnfs_update_layout(struct inode *ino, struct nfs_open_context *ctx,
loff_t pos, u64 count, enum pnfs_iomode access_type);
+bool pnfs_return_layout_barrier(struct nfs_inode *, struct pnfs_layout_range *);
int _pnfs_return_layout(struct inode *, struct pnfs_layout_range *,
const nfs4_stateid *stateid, /* optional */
enum pnfs_layoutreturn_type, bool wait);
--
1.7.2.3
It been a while since I used pNFS, I see GETATTR requests going to the DS,
did I configure something wrong or is the same old bug that keeps coming
back.
Thanks, Marc.
Signed-off-by: Benny Halevy <[email protected]>
---
fs/exofs/export.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index fe2c3a7..69bce46 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -32,7 +32,7 @@ static int exofs_layout_type(struct super_block *sb)
return LAYOUT_OSD2_OBJECTS;
}
-static void set_dev_id(struct pnfs_deviceid *pnfs_devid, u64 sbid, u64 devid)
+static void set_dev_id(struct nfs4_deviceid *pnfs_devid, u64 sbid, u64 devid)
{
struct nfsd4_pnfs_deviceid *dev_id =
(struct nfsd4_pnfs_deviceid *)pnfs_devid;
--
1.7.2.3
not supported yet
Signed-off-by: Benny Halevy <[email protected]>
---
fs/nfs/blocklayout/blocklayout.c | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/blocklayout/blocklayout.c b/fs/nfs/blocklayout/blocklayout.c
index 5e50c93..ac53a3f 100644
--- a/fs/nfs/blocklayout/blocklayout.c
+++ b/fs/nfs/blocklayout/blocklayout.c
@@ -725,7 +725,6 @@ nfs4_blk_get_deviceinfo(struct nfs_server *server, const struct nfs_fh *fh,
memcpy(&dev->dev_id, d_id, sizeof(*d_id));
dev->layout_type = LAYOUT_BLOCK_VOLUME;
- dev->dev_notify_types = 0;
dev->pages = pages;
dev->pgbase = 0;
dev->pglen = PAGE_SIZE * max_pages;
--
1.7.2.3
On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>
> I am support to publish first kernel crash.....
sorry to publish....
>
> this is
> pnfs-all-2.6.36-rc6-2010-09-29 git
> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>
> Regards,
> Tigran.
>
>
> On 09/29/2010 01:06 PM, Benny Halevy wrote:
>> This version includes Fred's pnfs-submit RFC patches (v5).
>> and is rebased onto v2.6.36-rc6
>>
>> Cumulative patches can be generated from
>> git://linux-nfs.org/~bhalevy/linux-pnfs.git
>> using
>> git diff v2.6.36-rc6 pnfs-all-2.6.36-rc6-2010-09-29
>>
>> Or, they can be downloaded from the wiki at:
>> http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
>>
>> Latest patches (since pnfs-all-2.6.36-rc3-2010-08-30):
>>
>> upstream:
>> Boaz Harrosh cc59b6a um: Proper Fix for f25c80a4: remove
>> duplicate structure field initialization
>>
>> pnfs-submit:
>> The pNFS Team 1851dbc RFC: pnfsd, pnfs: protocol level pnfs
>> constants
>> The pNFS Team a1b60af RFC: nfs: change stateid to be a union
>> The pNFS Team ff38ee4 RFC: nfs: ask for layouttypes during
>> fsinfo call
>> The pNFS Team 01f09aa RFC: nfs: set layout driver
>> The pNFS Team f7893b6 RFC: pnfs: full mount/umount infrastructure
>> The pNFS Team 1cef5e0 RFC: pnfs: filelayout: introduce minimal
>> file layout driver
>> The pNFS Team b7e2a3d RFC: nfs: create and destroy inode's
>> layout cache
>> The pNFS Team 34d2802 RFC: nfs: client needs to maintain list
>> of inodes with active layouts
>> The pNFS Team 292b90e RFC: pnfs: add LAYOUTGET and
>> GETDEVICEINFO infrastructure
>> The pNFS Team bf610f6 RFC: pnfs: filelayout: add driver's
>> LAYOUTGET and GETDEVICEINFO infrastructure
>> Fred Isaman 068e761 pnfs-submit: revert range simplifications
>> made by submission patches
>> Benny Halevy 7c58c3d SQUASHME: pnfs-submit: add missing
>> include file in nfs4filelayoutdev.c
>> Benny Halevy 9a5956f SQUASHME: move nfs4_deviceid definitions
>> to include/linux/nfs4.h
>> Benny Halevy bb2afce SQUASHME: pnfs-submit: handle non-pnfs
>> case in set_pnfs_layoutdriver
>> Benny Halevy 998fe4a pnfs-submit: file needs layout commit,
>> server attributes may be stale
>> Benny Halevy f18cdf8 NFS: clear fsinfo before sendign rpc
>>
>> pnfs:
>> Benny Halevy 52a6277 pnfs: alloc and free layout_hdr
>> layoutdriver methods
>> Benny Halevy 6dd521e pnfs: allow nfs4_proc_layoutget to sleep
>> on invalid lsegs
>> Benny Halevy ba1dd5b SQUASHME: pnfs: refactor put_lseg{_locked}
>> Benny Halevy bb03ecf SQUASHME: pnfs: get rid of lo_waitq
>>
>> pnfs-block:
>> Benny Halevy 73b1db9 SQUASHME: pnfsblock: remove obsolete
>> include file from blocklayout.h
>> Benny Halevy d9ebc26 SQUASHME: pnfsblock: use nfs4_deviceid
>> Benny Halevy 90e2af1 SQUASHME: pnfsblock: no callback ops
>> Benny Halevy 0953b90 SQAUSHME: pnfsblock: no PNFS_NFS_SERVER
>> Benny Halevy 7ab3774 SQUASHME: pnfsblock: no dev_notify_types
>> Benny Halevy f08f966 SQUASHME: pnfsblock: use new struct
>> pnfs_layout_hdr
>>
>> pnfs-obj-all:
>> Benny Halevy 41a70b7 SQUASHME: pnfs-obj: fix REQ flags usage
>> Benny Halevy 6057701 SQUASHME: pnfs-obj: convert to new
>> pnfs-submit changes
>>
>> pnfs-exofs-all:
>> Benny Halevy 10d8e6e SQUASHME: pnfsd-exofs: convert to new
>> pnfs-submit changes
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>
>> I am support to publish first kernel crash.....
> sorry to publish....
>
>>
>> this is
>> pnfs-all-2.6.36-rc6-2010-09-29 git
>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>
>> Regards,
>> Tigran.
>>
>>
Thanks for testing!
Does the following patch help?
git diff --stat -p -M
fs/nfs/nfs4filelayout.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
index 75e07c8..8fbb0db 100644
--- a/fs/nfs/nfs4filelayout.c
+++ b/fs/nfs/nfs4filelayout.c
@@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
/* Horrible hack...ideally upper layer would send lseg */
lseg = pnfs_has_layout(lo, &range);
+ if (!lseg)
+ return 0;
fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
size = fl->stripe_unit;
put_lseg_locked(lseg);
On 09/29/2010 04:22 PM, Benny Halevy wrote:
> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>> I am support to publish first kernel crash.....
>> sorry to publish....
>>
>>> this is
>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>
>>> Regards,
>>> Tigran.
>>>
>>>
> Thanks for testing!
> Does the following patch help?
>
No :(
> git diff --stat -p -M
> fs/nfs/nfs4filelayout.c | 2 ++
> 1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
> index 75e07c8..8fbb0db 100644
> --- a/fs/nfs/nfs4filelayout.c
> +++ b/fs/nfs/nfs4filelayout.c
> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>
> /* Horrible hack...ideally upper layer would send lseg */
> lseg = pnfs_has_layout(lo,&range);
> + if (!lseg)
> + return 0;
> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
> size = fl->stripe_unit;
> put_lseg_locked(lseg);
On 2010-09-29 16:22, Benny Halevy wrote:
> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>
>>> I am support to publish first kernel crash.....
>> sorry to publish....
>>
>>>
>>> this is
>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>
>>> Regards,
>>> Tigran.
>>>
>>>
>
> Thanks for testing!
> Does the following patch help?
>
Applied.
Benny
> git diff --stat -p -M
> fs/nfs/nfs4filelayout.c | 2 ++
> 1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
> index 75e07c8..8fbb0db 100644
> --- a/fs/nfs/nfs4filelayout.c
> +++ b/fs/nfs/nfs4filelayout.c
> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>
> /* Horrible hack...ideally upper layer would send lseg */
> lseg = pnfs_has_layout(lo, &range);
> + if (!lseg)
> + return 0;
> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
> size = fl->stripe_unit;
> put_lseg_locked(lseg);
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On 09/30/2010 10:30 AM, Benny Halevy wrote:
> On 2010-09-29 16:22, Benny Halevy wrote:
>> On 2010-09-29 16:09, Tigran Mkrtchyan wrote:
>>> On 09/29/2010 04:07 PM, Tigran Mkrtchyan wrote:
>>>> I am support to publish first kernel crash.....
>>> sorry to publish....
>>>
>>>> this is
>>>> pnfs-all-2.6.36-rc6-2010-09-29 git
>>>> c2b3b75c6b1db1dfc3f236200de1763e14c514d8
>>>>
>>>> Regards,
>>>> Tigran.
>>>>
>>>>
>> Thanks for testing!
>> Does the following patch help?
>>
> Applied.
>
did you applied to pnfs-submit as well?
> Benny
>
>> git diff --stat -p -M
>> fs/nfs/nfs4filelayout.c | 2 ++
>> 1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfs/nfs4filelayout.c b/fs/nfs/nfs4filelayout.c
>> index 75e07c8..8fbb0db 100644
>> --- a/fs/nfs/nfs4filelayout.c
>> +++ b/fs/nfs/nfs4filelayout.c
>> @@ -636,6 +636,8 @@ filelayout_get_stripesize(struct pnfs_layout_hdr *lo)
>>
>> /* Horrible hack...ideally upper layer would send lseg */
>> lseg = pnfs_has_layout(lo,&range);
>> + if (!lseg)
>> + return 0;
>> fl = container_of(lseg, struct nfs4_filelayout_segment, generic_hdr);
>> size = fl->stripe_unit;
>> put_lseg_locked(lseg);
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Oct 01, 2010 at 08:40:36AM -0700, Marc Eshel wrote:
> On 10/1/2010 5:33 AM, Benny Halevy wrote:
> >On 2010-10-01 10:47, Boaz Harrosh wrote:
> >>On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
> >>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
> >>>>Hi Benny,
> >>>>
> >>>>Running connectathon I see that some times the clients decides to destroy
> >>>>the session with the DS. The test continue and the session is
> >>>>re-established. It looks like layout return reduces the hold on device
> >>>>info the reduces the hold on the client struct which decide to destroy the
> >>>>session. Is that a known problem?
> >>>>
> >>Yes, I want to emphasize on Marks words: "a known *problem*"
> >Marc, assuming the code behaves as expected, does this cause any other badness
> >like the GETATTRs you see going out to the DS?
> >
> >Benny
> >
>
> No i don't see any "badness" the test continues without errors and
> this problem is not related to the GETATTRs I see on the DS but I
> would consider destroying the session in short run of couple of
> minutes some times more than one time as something bad.
Why?
I wouldn't expect session destruction/creation to be *that* expensive.
--b.
On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
> To initialize all values to zero, in case the server or protocol version
> do no support particular attributes.
Sorry for the delayed response, but...
Zero is not an appropriate default for many of the values. Further,
decode_fsinfo sets a default for each value, even in cases where the
server or protocol version do not support particular attributes. So
this patch seems to server no purpose.
Fred
>
> Signed-off-by: Benny Halevy <[email protected]>
> ---
> ?fs/nfs/client.c | ? ?1 +
> ?1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
> index 23d740c..beb206e 100644
> --- a/fs/nfs/client.c
> +++ b/fs/nfs/client.c
> @@ -942,6 +942,7 @@ static int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *mntfh, str
> ? ? ? ? ? ? ? ? ? ? ? ?goto out_error;
> ? ? ? ?}
>
> + ? ? ? memset(&fsinfo, 0, sizeof(fsinfo));
> ? ? ? ?fsinfo.fattr = fattr;
> ? ? ? ?error = clp->rpc_ops->fsinfo(server, mntfh, &fsinfo);
> ? ? ? ?if (error < 0)
> --
> 1.7.2.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
>
On Wed, 2010-10-13 at 17:13 -0400, Benny Halevy wrote:
> On 2010-10-13 16:54, Trond Myklebust wrote:
> > On Wed, 2010-10-13 at 16:34 -0400, Benny Halevy wrote:
> >> On 2010-10-13 14:03, Fred Isaman wrote:
> >>> On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
> >>>> To initialize all values to zero, in case the server or protocol version
> >>>> do no support particular attributes.
> >>>
> >>> Sorry for the delayed response, but...
> >>>
> >>> Zero is not an appropriate default for many of the values. Further,
> >>> decode_fsinfo sets a default for each value, even in cases where the
> >>> server or protocol version do not support particular attributes. So
> >>> this patch seems to server no purpose.
> >>
> >> Note that nfs_probe_fsinfo is called also for nfs version 2 and 3
> >> and these don't know anything about nfsv4.1 attributes so they can't
> >> cannot explicitly set them to any default value.
> >
> > Err... Why would we care? Under exactly what circumstances would we want
> > generic code to be processing fsinfo attributes that are specific only
> > to NFSv4.1?
> >
> > Trond
>
> Originally, set_pnfs_layoutdriver, was called only for nfsv4.1
> but we changed it to always be called in nfs_server_set_fsinfo()
So it relies on a flag in the fsinfo structure to tell it that this is a
pNFS-capable server? That's fine, but in that case, you should perhaps
explicitly initialise that flag to the non-pNFS capable value in the
generic code.
We shouldn't need to zero out the entire structure.
Cheers
Trond
On Fri, Oct 01, 2010 at 11:00:04AM -0700, Marc Eshel wrote:
> On 10/1/2010 10:10 AM, J. Bruce Fields wrote:
> >On Fri, Oct 01, 2010 at 08:40:36AM -0700, Marc Eshel wrote:
> >> On 10/1/2010 5:33 AM, Benny Halevy wrote:
> >>>On 2010-10-01 10:47, Boaz Harrosh wrote:
> >>>>On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
> >>>>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
> >>>>>>Hi Benny,
> >>>>>>
> >>>>>>Running connectathon I see that some times the clients decides to destroy
> >>>>>>the session with the DS. The test continue and the session is
> >>>>>>re-established. It looks like layout return reduces the hold on device
> >>>>>>info the reduces the hold on the client struct which decide to destroy the
> >>>>>>session. Is that a known problem?
> >>>>>>
> >>>>Yes, I want to emphasize on Marks words: "a known *problem*"
> >>>Marc, assuming the code behaves as expected, does this cause any other badness
> >>>like the GETATTRs you see going out to the DS?
> >>>
> >>>Benny
> >>>
> >>No i don't see any "badness" the test continues without errors and
> >>this problem is not related to the GETATTRs I see on the DS but I
> >>would consider destroying the session in short run of couple of
> >>minutes some times more than one time as something bad.
> >Why?
> >
> >I wouldn't expect session destruction/creation to be *that* expensive.
>
> I assumed that it is inexpensive. We are talking about potential
> destruction/creation of session from every DS for each file IO if
> there is no overlap in holding layouts, right ?
Well, I guess the tradeoffs aren't obvious to me: if you end up having
to set up an enormous number of sessions (and tcp connections, etc.) all
at once, then I can see why it might be a problem. It would also seem
inefficient to keep around an enormous number of those when they aren't
being used for a while.
--b.
On 2010-10-13 16:54, Trond Myklebust wrote:
> On Wed, 2010-10-13 at 16:34 -0400, Benny Halevy wrote:
>> On 2010-10-13 14:03, Fred Isaman wrote:
>>> On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
>>>> To initialize all values to zero, in case the server or protocol version
>>>> do no support particular attributes.
>>>
>>> Sorry for the delayed response, but...
>>>
>>> Zero is not an appropriate default for many of the values. Further,
>>> decode_fsinfo sets a default for each value, even in cases where the
>>> server or protocol version do not support particular attributes. So
>>> this patch seems to server no purpose.
>>
>> Note that nfs_probe_fsinfo is called also for nfs version 2 and 3
>> and these don't know anything about nfsv4.1 attributes so they can't
>> cannot explicitly set them to any default value.
>
> Err... Why would we care? Under exactly what circumstances would we want
> generic code to be processing fsinfo attributes that are specific only
> to NFSv4.1?
>
> Trond
Originally, set_pnfs_layoutdriver, was called only for nfsv4.1
but we changed it to always be called in nfs_server_set_fsinfo()
Benny
Hi Benny,
Running connectathon I see that some times the clients decides to destroy
the session with the DS. The test continue and the session is
re-established. It looks like layout return reduces the hold on device
info the reduces the hold on the client struct which decide to destroy the
session. Is that a known problem?
Thanks, Marc.
On Wed, 2010-10-13 at 16:34 -0400, Benny Halevy wrote:
> On 2010-10-13 14:03, Fred Isaman wrote:
> > On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
> >> To initialize all values to zero, in case the server or protocol version
> >> do no support particular attributes.
> >
> > Sorry for the delayed response, but...
> >
> > Zero is not an appropriate default for many of the values. Further,
> > decode_fsinfo sets a default for each value, even in cases where the
> > server or protocol version do not support particular attributes. So
> > this patch seems to server no purpose.
>
> Note that nfs_probe_fsinfo is called also for nfs version 2 and 3
> and these don't know anything about nfsv4.1 attributes so they can't
> cannot explicitly set them to any default value.
Err... Why would we care? Under exactly what circumstances would we want
generic code to be processing fsinfo attributes that are specific only
to NFSv4.1?
Trond
On 10/01/2010 06:17 AM, Marc Eshel wrote:
> Hi Benny,
>
> Running connectathon I see that some times the clients decides to destroy
> the session with the DS. The test continue and the session is
> re-established. It looks like layout return reduces the hold on device
> info the reduces the hold on the client struct which decide to destroy the
> session. Is that a known problem?
>
Just wan to confirm that we see this as well (pnfs-submit and
pnfs-all-latest).
Tigran.
> Thanks, Marc.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On 10/01/2010 02:29 PM, Andy Adamson wrote:
>
> On Oct 1, 2010, at 2:14 PM, J. Bruce Fields wrote:
>
>> On Fri, Oct 01, 2010 at 11:00:04AM -0700, Marc Eshel wrote:
>>> On 10/1/2010 10:10 AM, J. Bruce Fields wrote:
>>>> On Fri, Oct 01, 2010 at 08:40:36AM -0700, Marc Eshel wrote:
>>>>> On 10/1/2010 5:33 AM, Benny Halevy wrote:
>>>>>> On 2010-10-01 10:47, Boaz Harrosh wrote:
>>>>>>> On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
>>>>>>>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>>>>>>>>> Hi Benny,
>>>>>>>>>
>>>>>>>>> Running connectathon I see that some times the clients decides to destroy
>>>>>>>>> the session with the DS. The test continue and the session is
>>>>>>>>> re-established. It looks like layout return reduces the hold on device
>>>>>>>>> info the reduces the hold on the client struct which decide to destroy the
>>>>>>>>> session. Is that a known problem?
>>>>>>>>>
>>>>>>> Yes, I want to emphasize on Marks words: "a known *problem*"
>>>>>> Marc, assuming the code behaves as expected, does this cause any other badness
>>>>>> like the GETATTRs you see going out to the DS?
>>>>>>
>>>>>> Benny
>>>>>>
>>>>> No i don't see any "badness" the test continues without errors and
>>>>> this problem is not related to the GETATTRs I see on the DS but I
>>>>> would consider destroying the session in short run of couple of
>>>>> minutes some times more than one time as something bad.
>>>> Why?
>>>>
>>>> I wouldn't expect session destruction/creation to be *that* expensive.
>>>
>>> I assumed that it is inexpensive. We are talking about potential
>>> destruction/creation of session from every DS for each file IO if
>>> there is no overlap in holding layouts, right ?
>>
>> Well, I guess the tradeoffs aren't obvious to me: if you end up having
>> to set up an enormous number of sessions (and tcp connections, etc.) all
>> at once, then I can see why it might be a problem. It would also seem
>> inefficient to keep around an enormous number of those when they aren't
>> being used for a while.
>
> The plan is to add some code that waits a lease time before destroying an un-referenced deviceid.
> The next submission patch set will include layoutreturn and the return-on-close code, so it will
> probably be added then.
Rrr No! That's not what I wont. I want a cap on resources. That is say 17 servers I'll keep
for ever, anything over I can drop when done on a: least used basis.
I don't get this Gerrila programming, all of a sudden. Why do you guys go to grate length to shoot
yourself in the foot. We all have a limited and constant number of servers in our clusters. It
is counter performance, counter memory efficient, and a call for grate instability. Why deallocate
what I know for sure that I'll need on every future IO.
Time is not an issue here. Resources is, and speed. I'm idle for a while. Then on next IO I'm
slowed to an halt. It's not like I'm not sure the same servers will be used again.
The system Bruce describes above "It would also seem inefficient to keep around an enormous
number of those" does not yet exist. You are all talking about a future, not existent system.
Mean while sacrificing stability and performance. For no gain.
I can't see why you are picking on this simple matter.
You need memory problems on your head. Clone a Linux git tree on a system with less then
128Mg over pnfs. There you have a problem! it dead locks on network out of memory.
Boaz
>
> -->Andy
>
On 10/1/2010 10:10 AM, J. Bruce Fields wrote:
> On Fri, Oct 01, 2010 at 08:40:36AM -0700, Marc Eshel wrote:
>> On 10/1/2010 5:33 AM, Benny Halevy wrote:
>>> On 2010-10-01 10:47, Boaz Harrosh wrote:
>>>> On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
>>>>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>>>>>> Hi Benny,
>>>>>>
>>>>>> Running connectathon I see that some times the clients decides to destroy
>>>>>> the session with the DS. The test continue and the session is
>>>>>> re-established. It looks like layout return reduces the hold on device
>>>>>> info the reduces the hold on the client struct which decide to destroy the
>>>>>> session. Is that a known problem?
>>>>>>
>>>> Yes, I want to emphasize on Marks words: "a known *problem*"
>>> Marc, assuming the code behaves as expected, does this cause any other badness
>>> like the GETATTRs you see going out to the DS?
>>>
>>> Benny
>>>
>> No i don't see any "badness" the test continues without errors and
>> this problem is not related to the GETATTRs I see on the DS but I
>> would consider destroying the session in short run of couple of
>> minutes some times more than one time as something bad.
> Why?
>
> I wouldn't expect session destruction/creation to be *that* expensive.
I assumed that it is inexpensive. We are talking about potential
destruction/creation of session from every DS for each file IO if there
is no overlap in holding layouts, right ?
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
On 2010-10-01 10:47, Boaz Harrosh wrote:
> On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>>> Hi Benny,
>>>
>>> Running connectathon I see that some times the clients decides to destroy
>>> the session with the DS. The test continue and the session is
>>> re-established. It looks like layout return reduces the hold on device
>>> info the reduces the hold on the client struct which decide to destroy the
>>> session. Is that a known problem?
>>>
>
> Yes, I want to emphasize on Marks words: "a known *problem*"
Marc, assuming the code behaves as expected, does this cause any other badness
like the GETATTRs you see going out to the DS?
Benny
>
> I have objected strongly to this new "fixture" by Fred and backed up by Benny.
> They decided to only hold a deviceid as long as a layout references it. And
> not like before, until unmount. I think they are totally wrong with regard
> to current servers and implementation and a setup that might enjoy what is
> done now, will not exist for at least 3 years or more.
>
> An argument was made that current code is more simple. But I have demonstrated
> that all that is needed is a one-line get_ref at device add. And that iteration
> on all devices at umount time that was there before and removed in latest code.
>
> !!!
> Boaz
>
>
>> Just wan to confirm that we see this as well (pnfs-submit and
>> pnfs-all-latest).
>>
>> Tigran.
>>> Thanks, Marc.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
This version is based on v2.6.36
Cumulative patches can be generated from
git://linux-nfs.org/~bhalevy/linux-pnfs.git
using
git diff v2.6.36-rc6 pnfs-all-2.6.36-2010-10-28
Or, they can be downloaded from the wiki at:
http://wiki.linux-nfs.org/wiki/index.php/PNFS_Development_Git_tree
Note that I'm in the process of rebasing the whole tree onto linux-2.6/master
(to be v2.6.37-rc1) and it's taking some time due to conflicts and significant
changes both on the client and on the server side.
I'll send an announcement when the 2.6.37 tree is ready to go.
In the meanwhile pnfs-all-latest == pnfs-all-2.6.36 but the rest of the branches
were not updated.
Latest patches that went in since pnfs-all-2.6.36-rc6-2010-09-29:
upstream:
Jan Kara db02183 bdi: Initialize inode->i_mapping.backing_dev_info to sb->s_bdi
nfsd-upstream:
Benny Halevy 8dcd082 nfsd4: adjust buflen for encoded attrs bitmap based on actual bitmap length
pnfsd-files:
Benny Halevy c5a841c SQUASHME: pnfsd-files: update layout stateid properly
pnfsd:
Benny Halevy 12e7883 SQUASHME: pnfsd: FATTR4_WORD2_LAYOUT_BLKSIZE is supported only under CONFIG_PNFSD
spnfs:
Benny Halevy e129405 SQUASHME: pnfsd: factor out pnfsd check_export crap out of check_export
pnfs-submit-layoutget:
Bian Naimeng 2e16f9a SQUASHME: pnfs-submit: specify the layout type before initialize mountpoint
Fred Isaman 8781381 SQUASHME: pnfs-devinfo-submit: misc small fixes
Fred Isaman 07c9c9a SQUASHME: pnfs-devinfo-submit: Use only asynch error handler for LAYOUTGET
Fred Isaman c17dfaa SQUASHME: pnfs-submit: Don't check exchange flags unless using v4.1 try 2
pnfs-submit:
Fred Isaman 43db3d4 SQUASHME: pnfs-devinfo-submit: fix comments in pnfs_layout*_release
Benny Halevy 57a08c9 SQUASHME: pnfs-submit: handle NULL lseg in filelayout_get_stripesize
Fred Isaman 2a90270 SQUASHME: pnfs-submit: misc small rebase fixes
Fred Isaman e141824 SQUASHME: pnfs_submit: remove inlines from *.c files
Fred Isaman f60c230 nfs: change end of line commas to semicolons
Fred Isaman 625834c SQUASHME: pnfs_submit: Only use asynch error handler for LAYOUTCOMMIT
Fred Isaman 9bc5857 SQUASHME: pnfs_submit: only use asynch error handler for LAYOUTRETURN
Benny Halevy 67f8ba9 SQUASHME: pnfs-submit: init pg_lseg to NULL when !CONFIG_NFS_V4_1
Fred Isaman a262d18 pnfs_submit: Remove nonsensical check
Fred Isaman 0561b05 pnfs_submit: Only update stateid if it is more recent than current
Fred Isaman 48f0c0f pnfs_submit: simplify nfs4_callback_layoutrecall
Fred Isaman dc08d69 pnfs_submit: Fix clp refcounting in layout recalls
Fred Isaman df7c832 pnfs_submit: Remove the get_stripesize function
Andy Adamson 4e97d26 pnfs_submit: move layout segment valid test
Andy Adamson a40a6ae pnfs_submit: fix deadlock in pnfs_clear_lseg_list
Andy Adamson 5155659 pnfs_submit: enforce requested DS only pNFS role
Benny Halevy 1664201 SQUASHME: pnfs-submit: fail init_clientid if a DS is not really a DS
Benny Halevy 424dea1 SQUASHME: pnfs-submit: do not get_lseg in pnfs_has_layout
Benny Halevy 86dc18d SQUASHME: pnfs: get_lseg in nfs4_layoutget_prepare rather than in pnfs_has_layout
Fred Isaman ba39b5a pnfs_submit: Restore forgetful mode io draining
Andy Adamson 6328fa0 pnfs_submit: fix layoutreturn layout stateid processing
Andy Adamson 767d1da pnfs_submit: replace pnfs_layoutget_release with put_layout_hdr
Benny Halevy 0c407c7 pnfs: do not change layout stateid when dropping layouts.
pnfs:
Benny Halevy c249cff SQUASHME: pnfs: loga_minlength must not be greater than loga_length
Benny Halevy 2ec6978 pnfs: mark page with error in readpage_async_filler when crossing lsegs
Fred Isaman 23b3e65 SQUASHME: pnfs: filelayout: call print_ds under ifdebug(FACILITY)
pnfs-obj:
Benny Halevy 456ac6e SQUASHME: pnfs-obj: deprecate get_stripesize
pnfs-block:
Steve Dickson 97127f0 SQUASHME: pnfsblock: compile error in blocklayout code
Benny Halevy 81197a6 SQUASHME: pnfs-block: deprecate get_stripesize
Jim Rees 3bf1cd1 move include lines out of include file
On 10/1/2010 5:33 AM, Benny Halevy wrote:
> On 2010-10-01 10:47, Boaz Harrosh wrote:
>> On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
>>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>>>> Hi Benny,
>>>>
>>>> Running connectathon I see that some times the clients decides to destroy
>>>> the session with the DS. The test continue and the session is
>>>> re-established. It looks like layout return reduces the hold on device
>>>> info the reduces the hold on the client struct which decide to destroy the
>>>> session. Is that a known problem?
>>>>
>> Yes, I want to emphasize on Marks words: "a known *problem*"
> Marc, assuming the code behaves as expected, does this cause any other badness
> like the GETATTRs you see going out to the DS?
>
> Benny
>
No i don't see any "badness" the test continues without errors and this
problem is not related to the GETATTRs I see on the DS but I would
consider destroying the session in short run of couple of minutes some
times more than one time as something bad.
Marc.
>> I have objected strongly to this new "fixture" by Fred and backed up by Benny.
>> They decided to only hold a deviceid as long as a layout references it. And
>> not like before, until unmount. I think they are totally wrong with regard
>> to current servers and implementation and a setup that might enjoy what is
>> done now, will not exist for at least 3 years or more.
>>
>> An argument was made that current code is more simple. But I have demonstrated
>> that all that is needed is a one-line get_ref at device add. And that iteration
>> on all devices at umount time that was there before and removed in latest code.
>>
>> !!!
>> Boaz
>>
>>
>>> Just wan to confirm that we see this as well (pnfs-submit and
>>> pnfs-all-latest).
>>>
>>> Tigran.
>>>> Thanks, Marc.
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>
On Oct 1, 2010, at 2:14 PM, J. Bruce Fields wrote:
> On Fri, Oct 01, 2010 at 11:00:04AM -0700, Marc Eshel wrote:
>> On 10/1/2010 10:10 AM, J. Bruce Fields wrote:
>>> On Fri, Oct 01, 2010 at 08:40:36AM -0700, Marc Eshel wrote:
>>>> On 10/1/2010 5:33 AM, Benny Halevy wrote:
>>>>> On 2010-10-01 10:47, Boaz Harrosh wrote:
>>>>>> On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
>>>>>>> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>>>>>>>> Hi Benny,
>>>>>>>>
>>>>>>>> Running connectathon I see that some times the clients decides to destroy
>>>>>>>> the session with the DS. The test continue and the session is
>>>>>>>> re-established. It looks like layout return reduces the hold on device
>>>>>>>> info the reduces the hold on the client struct which decide to destroy the
>>>>>>>> session. Is that a known problem?
>>>>>>>>
>>>>>> Yes, I want to emphasize on Marks words: "a known *problem*"
>>>>> Marc, assuming the code behaves as expected, does this cause any other badness
>>>>> like the GETATTRs you see going out to the DS?
>>>>>
>>>>> Benny
>>>>>
>>>> No i don't see any "badness" the test continues without errors and
>>>> this problem is not related to the GETATTRs I see on the DS but I
>>>> would consider destroying the session in short run of couple of
>>>> minutes some times more than one time as something bad.
>>> Why?
>>>
>>> I wouldn't expect session destruction/creation to be *that* expensive.
>>
>> I assumed that it is inexpensive. We are talking about potential
>> destruction/creation of session from every DS for each file IO if
>> there is no overlap in holding layouts, right ?
>
> Well, I guess the tradeoffs aren't obvious to me: if you end up having
> to set up an enormous number of sessions (and tcp connections, etc.) all
> at once, then I can see why it might be a problem. It would also seem
> inefficient to keep around an enormous number of those when they aren't
> being used for a while.
The plan is to add some code that waits a lease time before destroying an un-referenced deviceid. The next submission patch set will include layoutreturn and the return-on-close code, so it will probably be added then.
-->Andy
>
> --b.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
On 2010-10-13 14:03, Fred Isaman wrote:
> On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
>> To initialize all values to zero, in case the server or protocol version
>> do no support particular attributes.
>
> Sorry for the delayed response, but...
>
> Zero is not an appropriate default for many of the values. Further,
> decode_fsinfo sets a default for each value, even in cases where the
> server or protocol version do not support particular attributes. So
> this patch seems to server no purpose.
Note that nfs_probe_fsinfo is called also for nfs version 2 and 3
and these don't know anything about nfsv4.1 attributes so they can't
cannot explicitly set them to any default value.
Benny
>
> Fred
>
>>
>> Signed-off-by: Benny Halevy <[email protected]>
>> ---
>> fs/nfs/client.c | 1 +
>> 1 files changed, 1 insertions(+), 0 deletions(-)
>>
>> diff --git a/fs/nfs/client.c b/fs/nfs/client.c
>> index 23d740c..beb206e 100644
>> --- a/fs/nfs/client.c
>> +++ b/fs/nfs/client.c
>> @@ -942,6 +942,7 @@ static int nfs_probe_fsinfo(struct nfs_server *server, struct nfs_fh *mntfh, str
>> goto out_error;
>> }
>>
>> + memset(&fsinfo, 0, sizeof(fsinfo));
>> fsinfo.fattr = fattr;
>> error = clp->rpc_ops->fsinfo(server, mntfh, &fsinfo);
>> if (error < 0)
>> --
>> 1.7.2.3
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
On 10/01/2010 08:12 AM, Tigran Mkrtchyan wrote:
> On 10/01/2010 06:17 AM, Marc Eshel wrote:
>> Hi Benny,
>>
>> Running connectathon I see that some times the clients decides to destroy
>> the session with the DS. The test continue and the session is
>> re-established. It looks like layout return reduces the hold on device
>> info the reduces the hold on the client struct which decide to destroy the
>> session. Is that a known problem?
>>
Yes, I want to emphasize on Marks words: "a known *problem*"
I have objected strongly to this new "fixture" by Fred and backed up by Benny.
They decided to only hold a deviceid as long as a layout references it. And
not like before, until unmount. I think they are totally wrong with regard
to current servers and implementation and a setup that might enjoy what is
done now, will not exist for at least 3 years or more.
An argument was made that current code is more simple. But I have demonstrated
that all that is needed is a one-line get_ref at device add. And that iteration
on all devices at umount time that was there before and removed in latest code.
!!!
Boaz
> Just wan to confirm that we see this as well (pnfs-submit and
> pnfs-all-latest).
>
> Tigran.
>> Thanks, Marc.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
On 2010-10-13 17:20, Trond Myklebust wrote:
> On Wed, 2010-10-13 at 17:13 -0400, Benny Halevy wrote:
>> On 2010-10-13 16:54, Trond Myklebust wrote:
>>> On Wed, 2010-10-13 at 16:34 -0400, Benny Halevy wrote:
>>>> On 2010-10-13 14:03, Fred Isaman wrote:
>>>>> On Wed, Sep 29, 2010 at 7:10 AM, Benny Halevy <[email protected]> wrote:
>>>>>> To initialize all values to zero, in case the server or protocol version
>>>>>> do no support particular attributes.
>>>>>
>>>>> Sorry for the delayed response, but...
>>>>>
>>>>> Zero is not an appropriate default for many of the values. Further,
>>>>> decode_fsinfo sets a default for each value, even in cases where the
>>>>> server or protocol version do not support particular attributes. So
>>>>> this patch seems to server no purpose.
>>>>
>>>> Note that nfs_probe_fsinfo is called also for nfs version 2 and 3
>>>> and these don't know anything about nfsv4.1 attributes so they can't
>>>> cannot explicitly set them to any default value.
>>>
>>> Err... Why would we care? Under exactly what circumstances would we want
>>> generic code to be processing fsinfo attributes that are specific only
>>> to NFSv4.1?
>>>
>>> Trond
>>
>> Originally, set_pnfs_layoutdriver, was called only for nfsv4.1
>> but we changed it to always be called in nfs_server_set_fsinfo()
>
> So it relies on a flag in the fsinfo structure to tell it that this is a
> pNFS-capable server? That's fine, but in that case, you should perhaps
> explicitly initialise that flag to the non-pNFS capable value in the
> generic code.
> We shouldn't need to zero out the entire structure.
OK. That's possible.
Benny
>
> Cheers
> Trond