Responded to comments, added a 2 cleanup patchses
Plus some code cleanup
0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch
and some bug fixes
0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch
NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch
was replaced by:
0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch
Remove unused (by file layout) encode_layoutreturn io operation
0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch
0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch
0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch
Note: pnfs4_proc_layoutget is only called by send_layout() which prints
the status.
0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch
Add back encode_layoutreturn io operation
0008-pnfs-post-submit-restore-encode_layoutreturn.patch
New patches:
0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch
This gets rid of a frame stack warning;
0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch
Testing:
---------
CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
Passes Connectathon tests
Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the
pyNFS server and the testclient framework.
Still todo:
Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
layoutcommit do not pass nfs_stste to the error handlers.
Handle NFS4ERR_BAD_LAYOUT.
CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.
-->Andy
On Mon, Jun 28, 2010 at 3:22 PM, William A. (Andy) Adamson
<[email protected]> wrote:
> On Mon, Jun 28, 2010 at 2:53 PM, Benny Halevy <[email protected]> w=
rote:
>> On Jun. 28, 2010, 19:44 +0300, Andy Adamson <[email protected]> wrot=
e:
>>> Hi Benny
>>>
>>> I have not been able to reproduce this BUG. I've tried against the
>>> files pyNFS server with return_on_close False as well a True, and
>>> against a GFS2/pNFS cluster with write layouts turned on.
>>>
>>> Patch 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patc=
h
>>> calls put_lseg when I/O to a DS fails. I tested this using the pyNF=
S
>>> files layout server and blocking the DS with iptables. I think this=
is
>>> the only change in this patch set that would affect the refcounting=
=2E
>>>
>>> Are you able to reproduce the BUG?
>>
>> The easiest way I found to reproduce this bug is running the cthon t=
ests
>> on a locally mounted file system exported over PNFSD_LOCAL_EXPORT.
>> The test machine is a dual core SMP machine.
>> Are you testing over a VM? =A0Is it uni-processor?
>
> Its a VM with one processor, but with SMP support turned on in the
> kernel. I just added a processor and will try re-running tests.
Added a processor - all cthon tests succeeded. Just to be clear, I'm
testing the client pnfs-submit branch.
-->Andy
>
> -->Andy
>
>>
>> Benny
>>
>>>
>>> -->Andy
>>>
>>> On Jun 24, 2010, at 1:02 PM, William A. (Andy) Adamson wrote:
>>>
>>>> OK - I'll look into it.
>>>>
>>>> Sorry I missed today's pNFS call.
>>>>
>>>> -->Andy
>>>>
>>>> On Thu, Jun 24, 2010 at 9:14 AM, Benny Halevy <[email protected]=
>
>>>> wrote:
>>>>> On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:
>>>>>> Responded to comments, added a 2 cleanup patchses
>>>>>>
>>>>>> Plus some code cleanup
>>>>>> 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch
>>>>>>
>>>>>> and some bug fixes
>>>>>> 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch
>>>>>>
>>>>>> NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-t=
he-
>>>>>> MDS.patch
>>>>>> was replaced by:
>>>>>> 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch
>>>>>>
>>>>>>
>>>>>> Remove unused (by file layout) encode_layoutreturn io operation
>>>>>> 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch
>>>>>> 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch
>>>>>>
>>>>>> 0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch
>>>>>>
>>>>>> Note: pnfs4_proc_layoutget is only called by send_layout() which
>>>>>> prints
>>>>>> the status.
>>>>>> 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch
>>>>>>
>>>>>> Add back encode_layoutreturn io operation
>>>>>> 0008-pnfs-post-submit-restore-encode_layoutreturn.patch
>>>>>>
>>>>>>
>>>>>> New patches:
>>>>>> 0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch
>>>>>>
>>>>>> This gets rid of a frame stack warning;
>>>>>> 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch
>>>>>>
>>>>>> Testing:
>>>>>> ---------
>>>>>>
>>>>>> CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
>>>>>> Passes Connectathon tests
>>>>>>
>>>>>> Tested layoutget and layoutreturn recovery from
>>>>>> NFS4ERR_DEAD_SESSION with the
>>>>>> pyNFS server and the testclient framework.
>>>>>>
>>>>>> Still todo:
>>>>>>
>>>>>> Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn,
>>>>>> layoutget, and
>>>>>> layoutcommit do not pass nfs_stste to the error handlers.
>>>>>>
>>>>>> Handle NFS4ERR_BAD_LAYOUT.
>>>>>>
>>>>>> CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.
>>>>>>
>>>>>> -->Andy
>>>>>
>>>>> Andy, I've hit
>>>>> =A0 =A0 =A0 BUG_ON(lo->refcount <=3D 0);
>>>>> in put_layout() with this patchset.
>>>>> I'm not sure if it introduced it or not, still investigating...
>>>>>
>>>>> Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING:
>>>>> layout.refcount 1
>>>>> Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
>>>>> Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalev=
y/
>>>>> git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
>>>>> Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP
>>>>> DEBUG_PAGEALLOC
>>>>> Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/
>>>>> initstate
>>>>> Jun 24 12:07:26 tl2 kernel: CPU 1
>>>>> Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nf=
sd
>>>>> exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4
>>>>> crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi
>>>>> cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror
>>>>> dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm
>>>>> snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel
>>>>> snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core
>>>>> snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore
>>>>> snd_page_alloc button video output rng_core sg cdrom ata_generic
>>>>> ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd
>>>>> ohci_hcd ehci_hcd [last unloaded: microcode]
>>>>> Jun 24 12:07:26 tl2 kernel:
>>>>> Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted
>>>>> 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
>>>>> Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>]
>>>>> [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
>>>>> Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20 =A0EFLAGS:
>>>>> 00010246
>>>>> Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX:
>>>>> ffff8800704b6b78 RCX: 0000000000000066
>>>>> Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI:
>>>>> ffffea0001b931a8 RDI: ffff8800704b6b78
>>>>> Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08:
>>>>> 0000000000000000 R09: ffff88007356a500
>>>>> Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11:
>>>>> 0000000000000003 R12: ffff8800704b69a8
>>>>> Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14:
>>>>> ffff88007356a508 R15: ffff88007356a590
>>>>> Jun 24 12:07:26 tl2 kernel: FS: =A00000000000000000(0000)
>>>>> GS:ffff880001a80000(0000) knlGS:0000000000000000
>>>>> Jun 24 12:07:26 tl2 kernel: CS: =A00010 DS: 0000 ES: 0000 CR0:
>>>>> 000000008005003b
>>>>> Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3:
>>>>> 0000000001698000 CR4: 00000000000406e0
>>>>> Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1:
>>>>> 0000000000000000 DR2: 0000000000000000
>>>>> Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6:
>>>>> 00000000ffff0ff0 DR7: 0000000000000400
>>>>> Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadin=
fo
>>>>> ffff88007525c000, task ffff88007d988000)
>>>>> Jun 24 12:07:26 tl2 kernel: Stack:
>>>>> Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8
>>>>> ffff88007525dd60 ffffffffa05d203f
>>>>> Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18
>>>>> ffff880073854f00 ffffffffa05d5880
>>>>> Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c
>>>>> ffff88007525dd90 ffff88007356a500
>>>>> Jun 24 12:07:26 tl2 kernel: Call Trace:
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_rele=
ase
>>>>> +0x43/0x68 [nfs]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>]
>>>>> nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>]
>>>>> rpc_release_calldata+0x17/0x19 [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x=
5e/
>>>>> 0x66 [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task
>>>>> +0x98/0x9c [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute
>>>>> +0x205/0x212 [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedu=
le
>>>>> +0x15/0x17 [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread
>>>>> +0x1aa/0x23b
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ?
>>>>> rpc_async_schedule+0x0/0x17 [sunrpc]
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ?
>>>>> autoremove_wake_function+0x0/0x39
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ?
>>>>> spin_unlock_irqrestore+0xe/0x10
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread
>>>>> +0x0/0x23b
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x8=
7
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>]
>>>>> kernel_thread_helper+0x4/0x10
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x=
87
>>>>> Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ?
>>>>> kernel_thread_helper+0x0/0x10
>>>>> Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 2=
4
>>>>> 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04
>>>>> 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 =
67
>>>>> 48 8b 82 48 03 00 00 f6 05
>>>>> Jun 24 12:07:26 tl2 kernel: RIP =A0[<ffffffffa05d0ea4>] put_layou=
t
>>>>> +0x2f/0xa7 [nfs]
>>>>> Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
>>>>> Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-
>>>>> nfs" in
>>>>> the body of a message to [email protected]
>>>>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.h=
tml
>>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nf=
s"
>>>> in
>>>> the body of a message to [email protected]
>>>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht=
ml
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs=
" in
>>> the body of a message to [email protected]
>>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm=
l
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs"=
in
>> the body of a message to [email protected]
>> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>>
>
On 06/23/2010 10:21 PM, [email protected] wrote:
> From: Andy Adamson <[email protected]>
>
> Redo pnfs4_proc_layoutget() logic to match the other async operations that call
> nfs4_wait_for_completion_rpc_task.
>
> pnfs_get_layout_done passes session and delay errors. Handle them with the
> generic error handlers.
>
> Signed-off-by: Andy Adamson <[email protected]>
> ---
> fs/nfs/nfs4proc.c | 45 +++++++++++++++++++++++++++++----------------
> 1 files changed, 29 insertions(+), 16 deletions(-)
>
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index 35e8a56..6283996 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct rpc_task *task, void *calldata)
> return;
>
> pnfs_get_layout_done(lgp, task->tk_status);
> +
> + if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
> + nfs_restart_rpc(task, server->nfs_client);
> +
Andy high.
In pnfs_get_layout_done() it is working hard to do "lgp->status = XXX"
I never understood, what gets to look at that later on?
Thanks
Boaz
> dprintk("<-- %s\n", __func__);
> }
>
> @@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_layoutget_call_ops = {
> * and deal with retries.
> * Currently we can't since we release lgp and its contents.
> */
> -int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
> +static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
> {
> - struct inode *ino = lgp->args.inode;
> - struct nfs_server *server = NFS_SERVER(ino);
> + struct nfs_server *server = NFS_SERVER(lgp->args.inode);
> struct rpc_task *task;
> struct rpc_message msg = {
> .rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_PNFS_LAYOUTGET],
> @@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
> .callback_data = lgp,
> .flags = RPC_TASK_ASYNC,
> };
> - int status;
> + int status = 0;
>
> dprintk("--> %s\n", __func__);
>
> lgp->res.layout.buf = (void *)__get_free_page(GFP_NOFS);
> if (lgp->res.layout.buf == NULL) {
> nfs4_pnfs_layoutget_release(lgp);
> - status = -ENOMEM;
> - goto out;
> + return -ENOMEM;
> }
>
> lgp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
> task = rpc_run_task(&task_setup_data);
> - if (IS_ERR(task)) {
> - status = PTR_ERR(task);
> - goto out;
> - }
> + if (IS_ERR(task))
> + return PTR_ERR(task);
> status = nfs4_wait_for_completion_rpc_task(task);
> - if (status == 0) {
> - status = lgp->status;
> - if (status == 0)
> - status = pnfs_layout_process(lgp);
> - }
> - rpc_put_task(task);
> + if (status != 0)
> + goto out;
> + status = lgp->status;
> + if (status != 0)
> + goto out;
> + status = pnfs_layout_process(lgp);
> out:
> + rpc_put_task(task);
> dprintk("<-- %s status=%d\n", __func__, status);
> return status;
> }
>
> +int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
> +{
> + struct nfs_server *server = NFS_SERVER(lgp->args.inode);
> + struct nfs4_exception exception = { };
> + int err;
> + do {
> + err = nfs4_handle_exception(server, _pnfs4_proc_layoutget(lgp),
> + &exception);
> + } while (exception.retry);
> + return err;
> +}
> +
> static void pnfs_layoutcommit_prepare(struct rpc_task *task, void *data)
> {
> struct pnfs_layoutcommit_data *ldata =
On Wed, Jun 30, 2010 at 11:19 AM, Boaz Harrosh <[email protected]> w=
rote:
> On 06/23/2010 10:21 PM, [email protected] wrote:
>> From: Andy Adamson <[email protected]>
>>
>> Redo pnfs4_proc_layoutget() logic to match the other async operation=
s that call
>> nfs4_wait_for_completion_rpc_task.
>>
>> pnfs_get_layout_done passes session and delay errors. Handle them wi=
th the
>> generic error handlers.
>>
>> Signed-off-by: Andy Adamson <[email protected]>
>> ---
>> =A0fs/nfs/nfs4proc.c | =A0 45 +++++++++++++++++++++++++++++---------=
-------
>> =A01 files changed, 29 insertions(+), 16 deletions(-)
>>
>> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
>> index 35e8a56..6283996 100644
>> --- a/fs/nfs/nfs4proc.c
>> +++ b/fs/nfs/nfs4proc.c
>> @@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct r=
pc_task *task, void *calldata)
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return;
>>
>> =A0 =A0 =A0 pnfs_get_layout_done(lgp, task->tk_status);
>> +
>> + =A0 =A0 if (nfs4_async_handle_error(task, server, NULL, NULL) =3D=3D=
-EAGAIN)
>> + =A0 =A0 =A0 =A0 =A0 =A0 nfs_restart_rpc(task, server->nfs_client);
>> +
>
> Andy high.
>
> In pnfs_get_layout_done() it is working hard to do "lgp->status =3D X=
XX"
>
> I never understood, what gets to look at that later on?
HI Boaz
_pnfs4_proc_layoutget will return it if it is set, then send_layoutget
returns it to pnfs_update_layout.
-->Andy
>
> Thanks
> Boaz
>
>> =A0 =A0 =A0 dprintk("<-- %s\n", __func__);
>> =A0}
>>
>> @@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_la=
youtget_call_ops =3D {
>> =A0 * and deal with retries.
>> =A0 * Currently we can't since we release lgp and its contents.
>> =A0 */
>> -int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
>> +static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
>> =A0{
>> - =A0 =A0 struct inode *ino =3D lgp->args.inode;
>> - =A0 =A0 struct nfs_server *server =3D NFS_SERVER(ino);
>> + =A0 =A0 struct nfs_server *server =3D NFS_SERVER(lgp->args.inode);
>> =A0 =A0 =A0 struct rpc_task *task;
>> =A0 =A0 =A0 struct rpc_message msg =3D {
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 .rpc_proc =3D &nfs4_procedures[NFSPROC4_=
CLNT_PNFS_LAYOUTGET],
>> @@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_la=
youtget *lgp)
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 .callback_data =3D lgp,
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 .flags =3D RPC_TASK_ASYNC,
>> =A0 =A0 =A0 };
>> - =A0 =A0 int status;
>> + =A0 =A0 int status =3D 0;
>>
>> =A0 =A0 =A0 dprintk("--> %s\n", __func__);
>>
>> =A0 =A0 =A0 lgp->res.layout.buf =3D (void *)__get_free_page(GFP_NOFS=
);
>> =A0 =A0 =A0 if (lgp->res.layout.buf =3D=3D NULL) {
>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 nfs4_pnfs_layoutget_release(lgp);
>> - =A0 =A0 =A0 =A0 =A0 =A0 status =3D -ENOMEM;
>> - =A0 =A0 =A0 =A0 =A0 =A0 goto out;
>> + =A0 =A0 =A0 =A0 =A0 =A0 return -ENOMEM;
>> =A0 =A0 =A0 }
>>
>> =A0 =A0 =A0 lgp->res.seq_res.sr_slotid =3D NFS4_MAX_SLOT_TABLE;
>> =A0 =A0 =A0 task =3D rpc_run_task(&task_setup_data);
>> - =A0 =A0 if (IS_ERR(task)) {
>> - =A0 =A0 =A0 =A0 =A0 =A0 status =3D PTR_ERR(task);
>> - =A0 =A0 =A0 =A0 =A0 =A0 goto out;
>> - =A0 =A0 }
>> + =A0 =A0 if (IS_ERR(task))
>> + =A0 =A0 =A0 =A0 =A0 =A0 return PTR_ERR(task);
>> =A0 =A0 =A0 status =3D nfs4_wait_for_completion_rpc_task(task);
>> - =A0 =A0 if (status =3D=3D 0) {
>> - =A0 =A0 =A0 =A0 =A0 =A0 status =3D lgp->status;
>> - =A0 =A0 =A0 =A0 =A0 =A0 if (status =3D=3D 0)
>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 status =3D pnfs_layout_pro=
cess(lgp);
>> - =A0 =A0 }
>> - =A0 =A0 rpc_put_task(task);
>> + =A0 =A0 if (status !=3D 0)
>> + =A0 =A0 =A0 =A0 =A0 =A0 goto out;
>> + =A0 =A0 status =3D lgp->status;
>> + =A0 =A0 if (status !=3D 0)
>> + =A0 =A0 =A0 =A0 =A0 =A0 goto out;
>> + =A0 =A0 status =3D pnfs_layout_process(lgp);
>> =A0out:
>> + =A0 =A0 rpc_put_task(task);
>> =A0 =A0 =A0 dprintk("<-- %s status=3D%d\n", __func__, status);
>> =A0 =A0 =A0 return status;
>> =A0}
>>
>> +int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
>> +{
>> + =A0 =A0 struct nfs_server *server =3D NFS_SERVER(lgp->args.inode);
>> + =A0 =A0 struct nfs4_exception exception =3D { };
>> + =A0 =A0 int err;
>> + =A0 =A0 do {
>> + =A0 =A0 =A0 =A0 =A0 =A0 err =3D nfs4_handle_exception(server, _pnf=
s4_proc_layoutget(lgp),
>> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
=A0 =A0 =A0 &exception);
>> + =A0 =A0 } while (exception.retry);
>> + =A0 =A0 return err;
>> +}
>> +
>> =A0static void pnfs_layoutcommit_prepare(struct rpc_task *task, void=
*data)
>> =A0{
>> =A0 =A0 =A0 struct pnfs_layoutcommit_data *ldata =3D
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" =
in
> the body of a message to [email protected]
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>
From: Andy Adamson <[email protected]>
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4filelayout.h | 6 ------
1 files changed, 0 insertions(+), 6 deletions(-)
diff --git a/fs/nfs/nfs4filelayout.h b/fs/nfs/nfs4filelayout.h
index 29e481d..de8391f 100644
--- a/fs/nfs/nfs4filelayout.h
+++ b/fs/nfs/nfs4filelayout.h
@@ -24,8 +24,6 @@
#define NFS4_PNFS_MAX_MULTI_CNT 64 /* 256 fit into a u8 stripe_index */
#define NFS4_PNFS_MAX_MULTI_DS 2
-#define FILE_MT(inode) ((struct filelayout_mount_type *) \
- (NFS_SERVER(inode)->pnfs_mountid->mountid))
#define FILE_DSADDR(lseg) (container_of(lseg->deviceid, \
struct nfs4_file_layout_dsaddr, \
deviceid))
@@ -76,10 +74,6 @@ struct nfs4_filelayout {
u32 stripe_unit;
};
-struct filelayout_mount_type {
- struct super_block *fl_sb;
-};
-
extern struct nfs_fh *
nfs4_fl_select_ds_fh(struct pnfs_layout_segment *lseg, loff_t offset);
--
1.6.6
From: Andy Adamson <[email protected]>
Redo pnfs4_proc_layoutget() logic to match the other async operations that call
nfs4_wait_for_completion_rpc_task.
pnfs_get_layout_done passes session and delay errors. Handle them with the
generic error handlers.
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4proc.c | 45 +++++++++++++++++++++++++++++----------------
1 files changed, 29 insertions(+), 16 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 35e8a56..6283996 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5446,6 +5446,10 @@ static void nfs4_pnfs_layoutget_done(struct rpc_task *task, void *calldata)
return;
pnfs_get_layout_done(lgp, task->tk_status);
+
+ if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
+ nfs_restart_rpc(task, server->nfs_client);
+
dprintk("<-- %s\n", __func__);
}
@@ -5471,10 +5475,9 @@ static const struct rpc_call_ops nfs4_pnfs_layoutget_call_ops = {
* and deal with retries.
* Currently we can't since we release lgp and its contents.
*/
-int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+static int _pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
{
- struct inode *ino = lgp->args.inode;
- struct nfs_server *server = NFS_SERVER(ino);
+ struct nfs_server *server = NFS_SERVER(lgp->args.inode);
struct rpc_task *task;
struct rpc_message msg = {
.rpc_proc = &nfs4_procedures[NFSPROC4_CLNT_PNFS_LAYOUTGET],
@@ -5488,35 +5491,45 @@ int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
.callback_data = lgp,
.flags = RPC_TASK_ASYNC,
};
- int status;
+ int status = 0;
dprintk("--> %s\n", __func__);
lgp->res.layout.buf = (void *)__get_free_page(GFP_NOFS);
if (lgp->res.layout.buf == NULL) {
nfs4_pnfs_layoutget_release(lgp);
- status = -ENOMEM;
- goto out;
+ return -ENOMEM;
}
lgp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
task = rpc_run_task(&task_setup_data);
- if (IS_ERR(task)) {
- status = PTR_ERR(task);
- goto out;
- }
+ if (IS_ERR(task))
+ return PTR_ERR(task);
status = nfs4_wait_for_completion_rpc_task(task);
- if (status == 0) {
- status = lgp->status;
- if (status == 0)
- status = pnfs_layout_process(lgp);
- }
- rpc_put_task(task);
+ if (status != 0)
+ goto out;
+ status = lgp->status;
+ if (status != 0)
+ goto out;
+ status = pnfs_layout_process(lgp);
out:
+ rpc_put_task(task);
dprintk("<-- %s status=%d\n", __func__, status);
return status;
}
+int pnfs4_proc_layoutget(struct nfs4_pnfs_layoutget *lgp)
+{
+ struct nfs_server *server = NFS_SERVER(lgp->args.inode);
+ struct nfs4_exception exception = { };
+ int err;
+ do {
+ err = nfs4_handle_exception(server, _pnfs4_proc_layoutget(lgp),
+ &exception);
+ } while (exception.retry);
+ return err;
+}
+
static void pnfs_layoutcommit_prepare(struct rpc_task *task, void *data)
{
struct pnfs_layoutcommit_data *ldata =
--
1.6.6
From: Andy Adamson <[email protected]>
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4proc.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 0ab875f..35e8a56 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5539,6 +5539,8 @@ pnfs_layoutcommit_done(struct rpc_task *task, void *calldata)
data->status = task->tk_status;
nfs4_sequence_done(server, &data->res.seq_res, task->tk_status);
+ if (RPC_ASSASSINATED(task))
+ return;
if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
nfs_restart_rpc(task, server->nfs_client);
--
1.6.6
From: Andy Adamson <[email protected]>
The nfs_read_data or nfs_write_data is freed during rpc call.
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/pnfs.h | 9 ++++++---
1 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index 78776f3..c60eff6 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -105,13 +105,14 @@ static inline enum pnfs_try_status
pnfs_try_to_read_data(struct nfs_read_data *data,
const struct rpc_call_ops *call_ops)
{
+ struct inode *inode = data->inode;
enum pnfs_try_status ret;
if (!data->req->wb_lseg)
return PNFS_NOT_ATTEMPTED;
ret = _pnfs_try_to_read_data(data, call_ops);
if (ret == PNFS_ATTEMPTED)
- nfs_inc_stats(data->inode, NFSIOS_PNFS_READ);
+ nfs_inc_stats(inode, NFSIOS_PNFS_READ);
else
_pnfs_clear_lseg_from_pages(&data->pages);
return ret;
@@ -122,13 +123,14 @@ pnfs_try_to_write_data(struct nfs_write_data *data,
const struct rpc_call_ops *call_ops,
int how)
{
+ struct inode *inode = data->inode;
enum pnfs_try_status ret;
if (!data->req->wb_lseg)
return PNFS_NOT_ATTEMPTED;
ret = _pnfs_try_to_write_data(data, call_ops, how);
if (ret == PNFS_ATTEMPTED)
- nfs_inc_stats(data->inode, NFSIOS_PNFS_WRITE);
+ nfs_inc_stats(inode, NFSIOS_PNFS_WRITE);
else
_pnfs_clear_lseg_from_pages(&data->pages);
return ret;
@@ -139,6 +141,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
const struct rpc_call_ops *call_ops,
int how)
{
+ struct inode *inode = data->inode;
enum pnfs_try_status ret;
/* Unlike in pnfs_try_to_write_data and pnfs_try_to_read_data,
@@ -148,7 +151,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
*/
ret = _pnfs_try_to_commit(data, call_ops, how);
if (ret == PNFS_ATTEMPTED)
- nfs_inc_stats(data->inode, NFSIOS_PNFS_COMMIT);
+ nfs_inc_stats(inode, NFSIOS_PNFS_COMMIT);
else
_pnfs_clear_lseg_from_pages(&data->pages);
return ret;
--
1.6.6
From: Andy Adamson <[email protected]>
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4xdr.c | 14 ++++++++++++--
include/linux/nfs4_pnfs.h | 4 ++++
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index fb50361..eeee855 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1874,13 +1874,23 @@ encode_layoutreturn(struct xdr_stream *xdr,
*p++ = cpu_to_be32(args->lseg.iomode);
*p = cpu_to_be32(args->return_type);
if (args->return_type == RETURN_FILE) {
+ struct layoutdriver_io_operations *ld_io_ops =
+ NFS_SERVER(args->inode)->pnfs_curr_ld->ld_io_ops;
+
p = reserve_space(xdr, 16 + NFS4_STATEID_SIZE);
p = xdr_encode_hyper(p, args->lseg.offset);
p = xdr_encode_hyper(p, args->lseg.length);
p = xdr_encode_opaque_fixed(p, &args->stateid.u.data,
NFS4_STATEID_SIZE);
- p = reserve_space(xdr, 4);
- *p = cpu_to_be32(0);
+ dprintk("%s: call %pF\n", __func__,
+ ld_io_ops->encode_layoutreturn);
+ if (ld_io_ops->encode_layoutreturn) {
+ ld_io_ops->encode_layoutreturn(
+ &NFS_I(args->inode)->layout, xdr, args);
+ } else {
+ p = reserve_space(xdr, 4);
+ *p = cpu_to_be32(0);
+ }
}
hdr->nops++;
hdr->replen += decode_layoutreturn_maxsz;
diff --git a/include/linux/nfs4_pnfs.h b/include/linux/nfs4_pnfs.h
index 782fdd9..d20b5de 100644
--- a/include/linux/nfs4_pnfs.h
+++ b/include/linux/nfs4_pnfs.h
@@ -154,6 +154,10 @@ struct layoutdriver_io_operations {
struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
void (*free_lseg) (struct pnfs_layout_segment *lseg);
+ void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
+ struct xdr_stream *xdr,
+ const struct nfs4_pnfs_layoutreturn_arg *args);
+
/* Registration information for a new mounted file system
*/
int (*initialize_mountpoint) (struct nfs_client *);
--
1.6.6
From: Andy Adamson <[email protected]>
Not used by the file layout
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4xdr.c | 15 ++-------------
include/linux/nfs4_pnfs.h | 4 ----
2 files changed, 2 insertions(+), 17 deletions(-)
diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index a509bc1..fb50361 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1874,24 +1874,13 @@ encode_layoutreturn(struct xdr_stream *xdr,
*p++ = cpu_to_be32(args->lseg.iomode);
*p = cpu_to_be32(args->return_type);
if (args->return_type == RETURN_FILE) {
- struct layoutdriver_io_operations *ld_io_ops =
- NFS_SERVER(args->inode)->pnfs_curr_ld->ld_io_ops;
-
p = reserve_space(xdr, 16 + NFS4_STATEID_SIZE);
p = xdr_encode_hyper(p, args->lseg.offset);
p = xdr_encode_hyper(p, args->lseg.length);
p = xdr_encode_opaque_fixed(p, &args->stateid.u.data,
NFS4_STATEID_SIZE);
-
- dprintk("%s: call %pF\n", __func__,
- ld_io_ops->encode_layoutreturn);
- if (ld_io_ops->encode_layoutreturn) {
- ld_io_ops->encode_layoutreturn(
- &NFS_I(args->inode)->layout, xdr, args);
- } else {
- p = reserve_space(xdr, 4);
- *p = cpu_to_be32(0);
- }
+ p = reserve_space(xdr, 4);
+ *p = cpu_to_be32(0);
}
hdr->nops++;
hdr->replen += decode_layoutreturn_maxsz;
diff --git a/include/linux/nfs4_pnfs.h b/include/linux/nfs4_pnfs.h
index d20b5de..782fdd9 100644
--- a/include/linux/nfs4_pnfs.h
+++ b/include/linux/nfs4_pnfs.h
@@ -154,10 +154,6 @@ struct layoutdriver_io_operations {
struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
void (*free_lseg) (struct pnfs_layout_segment *lseg);
- void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
- struct xdr_stream *xdr,
- const struct nfs4_pnfs_layoutreturn_arg *args);
-
/* Registration information for a new mounted file system
*/
int (*initialize_mountpoint) (struct nfs_client *);
--
1.6.6
From: Andy Adamson <[email protected]>
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4proc.c | 37 +++++++++++++++++++++++++++----------
1 files changed, 27 insertions(+), 10 deletions(-)
diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 5ffdfe1..0ab875f 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5641,6 +5641,11 @@ static void nfs4_pnfs_layoutreturn_done(struct rpc_task *task, void *calldata)
dprintk("--> %s\n", __func__);
nfs4_sequence_done(server, &lrp->res.seq_res, task->tk_status);
+ if (RPC_ASSASSINATED(task))
+ return;
+
+ if (nfs4_async_handle_error(task, server, NULL, NULL) == -EAGAIN)
+ nfs_restart_rpc(task, server->nfs_client);
dprintk("<-- %s\n", __func__);
}
@@ -5667,7 +5672,7 @@ static const struct rpc_call_ops nfs4_pnfs_layoutreturn_call_ops = {
.rpc_release = nfs4_pnfs_layoutreturn_release,
};
-int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool wait)
+int _pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool issync)
{
struct inode *ino = lrp->args.inode;
struct nfs_server *server = NFS_SERVER(ino);
@@ -5689,21 +5694,33 @@ int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool wait)
dprintk("--> %s\n", __func__);
lrp->res.seq_res.sr_slotid = NFS4_MAX_SLOT_TABLE;
task = rpc_run_task(&task_setup_data);
- if (IS_ERR(task)) {
- status = PTR_ERR(task);
+ if (IS_ERR(task))
+ return PTR_ERR(task);
+ if (!issync)
goto out;
- }
- if (wait) {
- status = nfs4_wait_for_completion_rpc_task(task);
- if (status == 0)
- status = task->tk_status;
- }
- rpc_put_task(task);
+ status = nfs4_wait_for_completion_rpc_task(task);
+ if (status != 0)
+ goto out;
+ status = task->tk_status;
out:
dprintk("<-- %s\n", __func__);
+ rpc_put_task(task);
return status;
}
+int pnfs4_proc_layoutreturn(struct nfs4_pnfs_layoutreturn *lrp, bool issync)
+{
+ struct nfs_server *server = NFS_SERVER(lrp->args.inode);
+ struct nfs4_exception exception = { };
+ int err;
+ do {
+ err = nfs4_handle_exception(server,
+ _pnfs4_proc_layoutreturn(lrp, issync),
+ &exception);
+ } while (exception.retry);
+ return err;
+}
+
int nfs4_pnfs_getdeviceinfo(struct nfs_server *server, struct pnfs_device *pdev)
{
struct nfs4_pnfs_getdeviceinfo_arg args = {
--
1.6.6
From: Andy Adamson <[email protected]>
Initialized in the VFS (inode_init_always)
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/inode.c | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
diff --git a/fs/nfs/inode.c b/fs/nfs/inode.c
index d6dd6e5..7989dea 100644
--- a/fs/nfs/inode.c
+++ b/fs/nfs/inode.c
@@ -1420,7 +1420,6 @@ static void pnfs_init_once(struct nfs_inode *nfsi)
{
#ifdef CONFIG_NFS_V4_1
init_waitqueue_head(&nfsi->lo_waitq);
- spin_lock_init(&nfsi->vfs_inode.i_lock);
seqlock_init(&nfsi->layout.seqlock);
INIT_LIST_HEAD(&nfsi->layout.lo_layouts);
INIT_LIST_HEAD(&nfsi->layout.segs);
--
1.6.6
From: Andy Adamson <[email protected]>
partial reads or writes use a single page not the page list.
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/pnfs.h | 16 ++++++++++++----
1 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index c60eff6..70f1d97 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -91,10 +91,18 @@ static inline int pnfs_enabled_sb(struct nfs_server *nfss)
return nfss->pnfs_curr_ld != NULL;
}
-static inline void _pnfs_clear_lseg_from_pages(struct list_head *head)
+static inline void _pnfs_clear_lseg_from_pages(struct nfs_page *partial,
+ struct list_head *head)
{
struct nfs_page *req;
+ /* partial_read or partial_write */
+ if (list_empty(head)) {
+ put_lseg(partial->wb_lseg);
+ partial->wb_lseg = NULL;
+ return;
+ }
+ /* full_read or full_write */
list_for_each_entry(req, head, wb_list) {
put_lseg(req->wb_lseg);
req->wb_lseg = NULL;
@@ -114,7 +122,7 @@ pnfs_try_to_read_data(struct nfs_read_data *data,
if (ret == PNFS_ATTEMPTED)
nfs_inc_stats(inode, NFSIOS_PNFS_READ);
else
- _pnfs_clear_lseg_from_pages(&data->pages);
+ _pnfs_clear_lseg_from_pages(data->req, &data->pages);
return ret;
}
@@ -132,7 +140,7 @@ pnfs_try_to_write_data(struct nfs_write_data *data,
if (ret == PNFS_ATTEMPTED)
nfs_inc_stats(inode, NFSIOS_PNFS_WRITE);
else
- _pnfs_clear_lseg_from_pages(&data->pages);
+ _pnfs_clear_lseg_from_pages(data->req, &data->pages);
return ret;
}
@@ -153,7 +161,7 @@ pnfs_try_to_commit(struct nfs_write_data *data,
if (ret == PNFS_ATTEMPTED)
nfs_inc_stats(inode, NFSIOS_PNFS_COMMIT);
else
- _pnfs_clear_lseg_from_pages(&data->pages);
+ _pnfs_clear_lseg_from_pages(data->req, &data->pages);
return ret;
}
--
1.6.6
From: Andy Adamson <[email protected]>
Signed-off-by: Andy Adamson <[email protected]>
---
fs/nfs/nfs4filelayoutdev.c | 18 ++++++++++++------
1 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/fs/nfs/nfs4filelayoutdev.c b/fs/nfs/nfs4filelayoutdev.c
index 5842c09..8e9d04a 100644
--- a/fs/nfs/nfs4filelayoutdev.c
+++ b/fs/nfs/nfs4filelayoutdev.c
@@ -121,9 +121,7 @@ _data_server_lookup(u32 ip_addr, u32 port)
static int
nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
{
- struct nfs_server tmp = {
- .nfs_client = NULL,
- };
+ struct nfs_server *tmp;
struct sockaddr_in sin;
struct rpc_clnt *mds_clnt = mds_srv->client;
struct nfs_client *clp = mds_srv->nfs_client;
@@ -154,12 +152,18 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
}
goto out;
}
+
+ /* Temporay server for nfs4_set_client */
+ tmp = kzalloc(sizeof(struct nfs_server), GFP_KERNEL);
+ if (!tmp)
+ goto out;
+
/*
* Set a retrans, timeout interval, and authflavor equual to the MDS
* values. Use the MDS nfs_client cl_ipaddr field so as to use the
* same co_ownerid as the MDS.
*/
- err = nfs4_set_client(&tmp,
+ err = nfs4_set_client(tmp,
mds_srv->nfs_client->cl_hostname,
(struct sockaddr *)&sin,
sizeof(struct sockaddr),
@@ -169,9 +173,9 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
mds_clnt->cl_xprt->timeout,
1 /* minorversion */);
if (err < 0)
- goto out;
+ goto out_free;
- clp = tmp.nfs_client;
+ clp = tmp->nfs_client;
/* Ask for only the EXCHGID4_FLAG_USE_PNFS_DS pNFS role */
dprintk("%s EXCHANGE_ID for clp %p\n", __func__, clp);
@@ -211,6 +215,8 @@ nfs4_pnfs_ds_create(struct nfs_server *mds_srv, struct nfs4_pnfs_ds *ds)
dprintk("%s: ip=%x, port=%hu, rpcclient %p\n", __func__,
ntohl(ds->ds_ip_addr), ntohs(ds->ds_port),
clp->cl_rpcclient);
+out_free:
+ kfree(tmp);
out:
dprintk("%s Returns %d\n", __func__, err);
return err;
--
1.6.6
On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:
> Responded to comments, added a 2 cleanup patchses
>
> Plus some code cleanup
> 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch
>
> and some bug fixes
> 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch
>
> NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch
> was replaced by:
> 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch
>
>
> Remove unused (by file layout) encode_layoutreturn io operation
> 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch
> 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch
>
> 0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch
>
> Note: pnfs4_proc_layoutget is only called by send_layout() which prints
> the status.
> 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch
>
> Add back encode_layoutreturn io operation
> 0008-pnfs-post-submit-restore-encode_layoutreturn.patch
>
>
> New patches:
> 0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch
>
> This gets rid of a frame stack warning;
> 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch
>
> Testing:
> ---------
>
> CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
> Passes Connectathon tests
>
> Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the
> pyNFS server and the testclient framework.
>
> Still todo:
>
> Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
> layoutcommit do not pass nfs_stste to the error handlers.
>
> Handle NFS4ERR_BAD_LAYOUT.
>
> CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.
>
> -->Andy
Andy, I've hit
BUG_ON(lo->refcount <= 0);
in put_layout() with this patchset.
I'm not sure if it introduced it or not, still investigating...
Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING: layout.refcount 1
Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/initstate
Jun 24 12:07:26 tl2 kernel: CPU 1
Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
Jun 24 12:07:26 tl2 kernel:
Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>] [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20 EFLAGS: 00010246
Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX: ffff8800704b6b78 RCX: 0000000000000066
Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI: ffffea0001b931a8 RDI: ffff8800704b6b78
Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08: 0000000000000000 R09: ffff88007356a500
Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11: 0000000000000003 R12: ffff8800704b69a8
Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14: ffff88007356a508 R15: ffff88007356a590
Jun 24 12:07:26 tl2 kernel: FS: 0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
Jun 24 12:07:26 tl2 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3: 0000000001698000 CR4: 00000000000406e0
Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo ffff88007525c000, task ffff88007d988000)
Jun 24 12:07:26 tl2 kernel: Stack:
Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8 ffff88007525dd60 ffffffffa05d203f
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18 ffff880073854f00 ffffffffa05d5880
Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c ffff88007525dd90 ffff88007356a500
Jun 24 12:07:26 tl2 kernel: Call Trace:
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release+0x43/0x68 [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>] nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>] rpc_release_calldata+0x17/0x19 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/0x66 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task+0x98/0x9c [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute+0x205/0x212 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule+0x15/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread+0x1aa/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ? rpc_async_schedule+0x0/0x17 [sunrpc]
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ? autoremove_wake_function+0x0/0x39
Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ? spin_unlock_irqrestore+0xe/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread+0x0/0x23b
Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67 48 8b 82 48 03 00 00 f6 05
Jun 24 12:07:26 tl2 kernel: RIP [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
OK - I'll look into it.
Sorry I missed today's pNFS call.
-->Andy
On Thu, Jun 24, 2010 at 9:14 AM, Benny Halevy <[email protected]> wrote:
> On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:
>> Responded to comments, added a 2 cleanup patchses
>>
>> Plus some code cleanup
>> 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch
>>
>> and some bug fixes
>> 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch
>>
>> NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch
>> was replaced by:
>> 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch
>>
>>
>> Remove unused (by file layout) encode_layoutreturn io operation
>> 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch
>> 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch
>>
>> 0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch
>>
>> Note: pnfs4_proc_layoutget is only called by send_layout() which prints
>> the status.
>> 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch
>>
>> Add back encode_layoutreturn io operation
>> 0008-pnfs-post-submit-restore-encode_layoutreturn.patch
>>
>>
>> New patches:
>> 0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch
>>
>> This gets rid of a frame stack warning;
>> 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch
>>
>> Testing:
>> ---------
>>
>> CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
>> Passes Connectathon tests
>>
>> Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the
>> pyNFS server and the testclient framework.
>>
>> Still todo:
>>
>> Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
>> layoutcommit do not pass nfs_stste to the error handlers.
>>
>> Handle NFS4ERR_BAD_LAYOUT.
>>
>> CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.
>>
>> -->Andy
>
> Andy, I've hit
> BUG_ON(lo->refcount <= 0);
> in put_layout() with this patchset.
> I'm not sure if it introduced it or not, still investigating...
>
> Jun 24 12:07:26 tl2 kernel: pnfs_destroy_inode: WARNING: layout.refcount 1
> Jun 24 12:07:26 tl2 kernel: ------------[ cut here ]------------
> Jun 24 12:07:26 tl2 kernel: kernel BUG at /usr0/export/dev/bhalevy/git/linux-pnfs-bh-nfs41/fs/nfs/pnfs.c:341!
> Jun 24 12:07:26 tl2 kernel: invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
> Jun 24 12:07:26 tl2 kernel: last sysfs file: /sys/module/nfs/initstate
> Jun 24 12:07:26 tl2 kernel: CPU 1
> Jun 24 12:07:26 tl2 kernel: Modules linked in: nfslayoutdriver nfsd exportfs nfs lockd nfs_acl auth_rpcgss sunrpc osd libosd autofs4 crc32c ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cpufreq_ondemand acpi_cpufreq freq_table mperf ext3 jbd dm_mirror dm_region_hash dm_log dm_multipath dm_mod kvm_intel kvm snd_hda_codec_realtek i915 drm_kms_helper drm snd_hda_intel snd_hda_codec snd_hwdep i2c_algo_bit snd_seq i2c_i801 i2c_core snd_seq_device snd_pcm r8169 mii snd_timer sr_mod snd soundcore snd_page_alloc button video output rng_core sg cdrom ata_generic ata_piix libata sd_mod scsi_mod ext4 mbcache jbd2 crc16 uhci_hcd ohci_hcd ehci_hcd [last unloaded: microcode]
> Jun 24 12:07:26 tl2 kernel:
> Jun 24 12:07:26 tl2 kernel: Pid: 1920, comm: rpciod/1 Not tainted 2.6.35-rc3-pnfs+ #54 G31M4 (MS-7527)/MS-7527
> Jun 24 12:07:26 tl2 kernel: RIP: 0010:[<ffffffffa05d0ea4>] [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
> Jun 24 12:07:26 tl2 kernel: RSP: 0018:ffff88007525dd20 EFLAGS: 00010246
> Jun 24 12:07:26 tl2 kernel: RAX: 0000000000000000 RBX: ffff8800704b6b78 RCX: 0000000000000066
> Jun 24 12:07:26 tl2 kernel: RDX: ffff8800704b69a8 RSI: ffffea0001b931a8 RDI: ffff8800704b6b78
> Jun 24 12:07:26 tl2 kernel: RBP: ffff88007525dd30 R08: 0000000000000000 R09: ffff88007356a500
> Jun 24 12:07:26 tl2 kernel: R10: ffff88007525dd80 R11: 0000000000000003 R12: ffff8800704b69a8
> Jun 24 12:07:26 tl2 kernel: R13: ffff880073854f00 R14: ffff88007356a508 R15: ffff88007356a590
> Jun 24 12:07:26 tl2 kernel: FS: 0000000000000000(0000) GS:ffff880001a80000(0000) knlGS:0000000000000000
> Jun 24 12:07:26 tl2 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Jun 24 12:07:26 tl2 kernel: CR2: 0000003944279000 CR3: 0000000001698000 CR4: 00000000000406e0
> Jun 24 12:07:26 tl2 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Jun 24 12:07:26 tl2 kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Jun 24 12:07:26 tl2 kernel: Process rpciod/1 (pid: 1920, threadinfo ffff88007525c000, task ffff88007d988000)
> Jun 24 12:07:26 tl2 kernel: Stack:
> Jun 24 12:07:26 tl2 kernel: ffff8800704b6b78 ffff8800704b69a8 ffff88007525dd60 ffffffffa05d203f
> Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd60 ffff880073854f18 ffff880073854f00 ffffffffa05d5880
> Jun 24 12:07:26 tl2 kernel: <0> ffff88007525dd80 ffffffffa05bfb5c ffff88007525dd90 ffff88007356a500
> Jun 24 12:07:26 tl2 kernel: Call Trace:
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05d203f>] pnfs_layout_release+0x43/0x68 [nfs]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05bfb5c>] nfs4_pnfs_layoutreturn_release+0x61/0x8b [nfs]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa056207d>] rpc_release_calldata+0x17/0x19 [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa05621bd>] rpc_free_task+0x5e/0x66 [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa056225d>] rpc_put_task+0x98/0x9c [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ea7>] __rpc_execute+0x205/0x212 [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562ef0>] rpc_async_schedule+0x15/0x17 [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81052cb7>] worker_thread+0x1aa/0x23b
> Jun 24 12:07:26 tl2 kernel: [<ffffffffa0562edb>] ? rpc_async_schedule+0x0/0x17 [sunrpc]
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81056ab7>] ? autoremove_wake_function+0x0/0x39
> Jun 24 12:07:26 tl2 kernel: [<ffffffff8102f96d>] ? spin_unlock_irqrestore+0xe/0x10
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81052b0d>] ? worker_thread+0x0/0x23b
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81056645>] kthread+0x7f/0x87
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a24>] kernel_thread_helper+0x4/0x10
> Jun 24 12:07:26 tl2 kernel: [<ffffffff810565c6>] ? kthread+0x0/0x87
> Jun 24 12:07:26 tl2 kernel: [<ffffffff81003a20>] ? kernel_thread_helper+0x0/0x10
> Jun 24 12:07:26 tl2 kernel: Code: 41 54 53 0f 1f 44 00 00 8b 87 24 01 00 00 48 89 fb 48 8d 97 30 fe ff ff 89 c1 c1 f9 08 38 c1 75 04 0f 0b eb fe 8b 07 85 c0 7f 04 <0f> 0b eb fe ff c8 85 c0 89 07 75 67 48 8b 82 48 03 00 00 f6 05
> Jun 24 12:07:26 tl2 kernel: RIP [<ffffffffa05d0ea4>] put_layout+0x2f/0xa7 [nfs]
> Jun 24 12:07:27 tl2 kernel: RSP <ffff88007525dd20>
> Jun 24 12:07:27 tl2 kernel: ---[ end trace 0468384c0ab45a1f ]---
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
On 2010-06-23 22:21, [email protected] wrote:On Jun. 23, 2010, 22:21 +0300, [email protected] wrote:
> Responded to comments, added a 2 cleanup patchses
>
> Plus some code cleanup
> 0001-SQUASHME-pnfs-submit-remove-unused-filelayout_mount_.patch
>
> and some bug fixes
> 0002-SQUASHME-pnfs-submit-pnfs_try_to_read-write-commit-u.patch
>
> NOTE: this patch: 0003-SQUASHME-pnfs-submit-tell-commit-to-use-the-MDS.patch
> was replaced by:
> 0003-SQUASHME-pnfs-submit-clear-page-lseg-on-partial-i-o.patch
>
All but patch 3/10 (as per your request) are merged at
pnfs-all-2.6.35-rc3-2010-07-01
Thanks!
Benny
>
> Remove unused (by file layout) encode_layoutreturn io operation
> 0004-SQUASHME-pnfs-submit-remove-encode_layoutreturn.patch
> 0005-SQUASHME-pnfs-submit-add-error-handling-to-layout-re.patch
>
> 0006-SQUASHME-pnfs-submit-handle-assassinated-layoutcommi.patch
>
> Note: pnfs4_proc_layoutget is only called by send_layout() which prints
> the status.
> 0007-SQUASHME-pnfs-submit-add-error-handlers-to-layout-ge.patch
>
> Add back encode_layoutreturn io operation
> 0008-pnfs-post-submit-restore-encode_layoutreturn.patch
>
>
> New patches:
> 0009-SQUASHME-pnfs-submit-don-t-re-initialize-i_lock.patch
>
> This gets rid of a frame stack warning;
> 0010-SQUASHME-pnfs-submit-remove-struct-nfs_server-from-s.patch
>
> Testing:
> ---------
>
> CONFIG_NFS_V4_1 set: NFSv4.0 NFSv4.1 pNFS
> Passes Connectathon tests
>
> Tested layoutget and layoutreturn recovery from NFS4ERR_DEAD_SESSION with the
> pyNFS server and the testclient framework.
>
> Still todo:
>
> Recover from NFS4ERR_BAD_STATEID. Currently layoutreturn, layoutget, and
> layoutcommit do not pass nfs_stste to the error handlers.
>
> Handle NFS4ERR_BAD_LAYOUT.
>
> CONFIG_NFS_V4_1 not set: NFSv4.o mount passes cthon tests.
>
> -->Andy
--
Benny Halevy
Software Architect
Panasas, Inc.
[email protected]
Tel/Fax: +972-3-647-8340
Mobile: +972-54-802-8340
Panasas: The Leader in Parallel Storage
http://www.panasas.com