Received-SPF: pass (google.com: domain of linux-nfs+bounces-3362-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Precedence: bulk
MIME-Version: 1.0
Date: Fri, 24 May 2024 06:39:49 +0200
From: Richard Kojedzinszky <richard+debian+bugreport@kojedz.in>
To: NeilBrown <neilb@suse.de>
Cc: linux-nfs@vger.kernel.org, 1071501@bugs.debian.org
Subject: Re: Linux NFS client hangs in nfs4_lookup_revalidate
In-Reply-To: <171650710476.27191.9102106000258626652@noble.neil.brown.name>
References: <>, <162d12087ba8374a57e2263d7ea762b5@kojedz.in>
 <171650710476.27191.9102106000258626652@noble.neil.brown.name>
Message-ID: <6e925c01180db44c5aff4475bfe72aef@kojedz.in>
Content-Type: multipart/mixed;
 boundary="=_1aee6e723aea2847a539d9257e25d5f0"

--=_1aee6e723aea2847a539d9257e25d5f0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8;
 format=flowed

Dear Neil,

I've stripped the code more, which still triggers the bug for me. On 
Bookworm, to get the binary, simply:

$ sudo apt-get install golang
$ go build .

And then give it an nfs mountpoint, e.g.:

$ ./ds /mnt/nfs

Meanwhile, I will try your patch too.

Regards,
Richard

2024-05-24 01:31 időpontban NeilBrown ezt írta:
> On Fri, 24 May 2024, Richard Kojedzinszky wrote:
>> Dear devs,
>> 
>> I am attaching a stripped down version of the little program which
>> triggers the bug very quickly, in a few minutes in my test lab. It
>> turned out that a single NFS mountpoint is enough. Just start the
>> program giving it the NFS mount as first argument. It will chdir 
>> there,
>> and do file operations, which will trigger a lockup in a few minutes.
> 
> I couldn't get the go code to run.  But then it is a long time since I
> played with go and I didn't try very hard.
> If you could provide simple instructions and a list of package
> dependencies that I need to install (on Debian), I can give it a try.
> 
> Or you could try this patch.  It might help, but I don't have high
> hopes.  It adds some memory barriers and fixes a bug which would cause 
> a
> problem if memory allocation failed (but memory allocation never 
> fails).
> 
> NeilBrown
> 
> diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
> index ac505671efbd..5bcc0d14d519 100644
> --- a/fs/nfs/dir.c
> +++ b/fs/nfs/dir.c
> @@ -1804,7 +1804,7 @@ __nfs_lookup_revalidate(struct dentry *dentry, 
> unsigned int flags,
>  	} else {
>  		/* Wait for unlink to complete */
>  		wait_var_event(&dentry->d_fsdata,
> -			       dentry->d_fsdata != NFS_FSDATA_BLOCKED);
> +			       smp_load_acquire(&dentry->d_fsdata) != NFS_FSDATA_BLOCKED);
>  		parent = dget_parent(dentry);
>  		ret = reval(d_inode(parent), dentry, flags);
>  		dput(parent);
> @@ -2508,7 +2508,7 @@ int nfs_unlink(struct inode *dir, struct dentry 
> *dentry)
>  	spin_unlock(&dentry->d_lock);
>  	error = nfs_safe_remove(dentry);
>  	nfs_dentry_remove_handle_error(dir, dentry, error);
> -	dentry->d_fsdata = NULL;
> +	smp_store_release(&dentry->d_fsdata, NULL);
>  	wake_up_var(&dentry->d_fsdata);
>  out:
>  	trace_nfs_unlink_exit(dir, dentry, error);
> @@ -2616,7 +2616,7 @@ nfs_unblock_rename(struct rpc_task *task, struct 
> nfs_renamedata *data)
>  {
>  	struct dentry *new_dentry = data->new_dentry;
> 
> -	new_dentry->d_fsdata = NULL;
> +	smp_store_release(&new_dentry->d_fsdata, NULL);
>  	wake_up_var(&new_dentry->d_fsdata);
>  }
> 
> @@ -2717,6 +2717,10 @@ int nfs_rename(struct mnt_idmap *idmap, struct 
> inode *old_dir,
>  	task = nfs_async_rename(old_dir, new_dir, old_dentry, new_dentry,
>  				must_unblock ? nfs_unblock_rename : NULL);
>  	if (IS_ERR(task)) {
> +		if (must_unlock) {
> +			smp_store_release(&new_dentry->d_fsdata, NULL);
> +			wake_up_var(&new_dentry->d_fsdata);
> +		}
>  		error = PTR_ERR(task);
>  		goto out;
>  	}

--=_1aee6e723aea2847a539d9257e25d5f0
Content-Transfer-Encoding: base64
Content-Type: application/x-tar;
 name=ds.tar
Content-Disposition: attachment;
 filename=ds.tar;
 size=10240

ZG92ZWNvdC1zaW11bGF0b3IvAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA3NTUAMDAwMTc1
MQAwMDAxNzUxADAwMDAwMDAwMDAwADE0NjI0MDE0MDcwADAxNDEwMgAgNQAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAGtyaWNoeQAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAa3JpY2h5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABk
b3ZlY290LXNpbXVsYXRvci9nby5zdW0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMDAwMDY0NAAwMDAxNzUx
ADAwMDE3NTEAMDAwMDAwMDAwMDAAMTQ2MjQwMTM3NzMAMDE1MjM0ACAwAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHVzdGFyICAAa3JpY2h5AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAABrcmljaHkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGRv
dmVjb3Qtc2ltdWxhdG9yL3JlYWRlci5nbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwMDAwNjQ0ADAwMDE3NTEA
MDAwMTc1MQAwMDAwMDAwMjY0MQAxNDYyNDAxMzc1NwAwMTU3MTEAIDAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAdXN0YXIgIABrcmljaHkAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAGtyaWNoeQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcGFj
a2FnZSBtYWluCgppbXBvcnQgKAoJImJ1ZmlvIgoJImNvbnRleHQiCgkibG9nIgoJIm1hdGgvcmFu
ZCIKCSJvcyIKCSJ0aW1lIgopCgp0eXBlIHJlYWRlciBzdHJ1Y3QgewoJbCAgICAgICAgICAgKmxv
Y2tmaWxlCglsYXN0TW9kVGltZSB0aW1lLlRpbWUKfQoKZnVuYyAociAqcmVhZGVyKSBydW4oY3R4
IGNvbnRleHQuQ29udGV4dCwgaW5zdGFuY2UgaW50KSB7CglyLmwgPSAmbG9ja2ZpbGV7ZmlsZTog
bG9ja0ZpbGV9CgoJd2QgOj0gbWFrZShjaGFuIGJvb2wpCgoJZ28gZnVuYygpIHsKCQl0aW1lciA6
PSB0aW1lLk5ld1RpbWVyKHRpbWUuU2Vjb25kKQoJCWRlZmVyIHRpbWVyLlN0b3AoKQoKCQlmb3Ig
ewoJCQlzZWxlY3QgewoJCQljYXNlIDwtY3R4LkRvbmUoKToKCQkJCXJldHVybgoJCQljYXNlIDwt
dGltZXIuQzoKCQkJCWxvZy5QcmludGYoInJlYWRlciglZCkgaGFzIGJlZW4gYmxvY2tlZCBmb3Ig
YSBzZWNvbmQiLCBpbnN0YW5jZSkKCQkJCXJldHVybgoJCQljYXNlIDwtd2Q6CgkJCQlpZiAhdGlt
ZXIuU3RvcCgpIHsKCQkJCQk8LXRpbWVyLkMKCQkJCX0KCQkJCXRpbWVyLlJlc2V0KHRpbWUuU2Vj
b25kKQoJCQl9CgkJfQoJfSgpCgoJZm9yIHsKCQlzZWxlY3QgewoJCWNhc2UgPC1jdHguRG9uZSgp
OgoJCQlyZXR1cm4KCQlkZWZhdWx0OgoJCX0KCgkJci5kb19yZWFkKCkKCgkJd2QgPC0gdHJ1ZQoJ
fQp9CgpmdW5jIChyICpyZWFkZXIpIGRvX3JlYWQoKSBlcnJvciB7CglzdGF0LCBlcnIgOj0gb3Mu
U3RhdChsaXN0RmlsZSkKCWlmIGVyciAhPSBuaWwgewoJCXJldHVybiBlcnIKCX0KCgltb2RUaW1l
IDo9IHN0YXQuTW9kVGltZSgpCglpZiAhbW9kVGltZS5BZnRlcihyLmxhc3RNb2RUaW1lKSB7CgkJ
cmV0dXJuIG5pbAoJfQoKCWlmIGVyciA9IHIubC5Mb2NrKCk7IGVyciAhPSBuaWwgewoJCXJldHVy
biBlcnIKCX0KCWRlZmVyIHIubC5VbmxvY2soKQoKCXJmaCwgZXJyIDo9IG9zLk9wZW4obGlzdEZp
bGUpCglpZiBlcnIgIT0gbmlsIHsKCQlyZXR1cm4gZXJyCgl9CglkZWZlciByZmguQ2xvc2UoKQoK
CXdmaCwgZXJyIDo9IG9zLk9wZW5GaWxlKGxpc3RGaWxlTmV3LCBvcy5PX0NSRUFURXxvcy5PX1dS
T05MWSwgMG82MDApCglpZiBlcnIgIT0gbmlsIHsKCQlyZXR1cm4gZXJyCgl9CglkZWZlciB3Zmgu
Q2xvc2UoKQoKCXJyIDo9IGJ1ZmlvLk5ld1JlYWRlcihyZmgpCgl3ciA6PSBidWZpby5OZXdXcml0
ZXIod2ZoKQoKCWZvciB7CgkJbGluZSwgXyA6PSByci5SZWFkU3RyaW5nKCdcbicpCgkJaWYgbGlu
ZSA9PSAiIiB7CgkJCWJyZWFrCgkJfQoKCQlpZiByYW5kLkludG4oMTApIDwgNCB7IC8vIGtlZXAg
ZmlsZXMgYXQgNDAlIGNoYW5jZQoJCQl3ci5Xcml0ZVN0cmluZyhsaW5lKQoJCX0KCX0KCgl3ci5G
bHVzaCgpCgl3ZmguQ2xvc2UoKQoKCW9zLlJlbmFtZShsaXN0RmlsZU5ldywgbGlzdEZpbGUpCgoJ
cmV0dXJuIG5pbAp9CgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZG92ZWNv
dC1zaW11bGF0b3IvbG9ja2ZpbGUuZ28AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA2NDQAMDAwMTc1MQAwMDAx
NzUxADAwMDAwMDAxMTM0ADE0NjIzNjQ0NjcwADAxNjIzNgAgMAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAGtyaWNoeQAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAa3JpY2h5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwYWNrYWdl
IG1haW4KCmltcG9ydCAoCgkibWF0aC9yYW5kIgoJIm9zIgoJInRpbWUiCikKCnR5cGUgbG9ja2Zp
bGUgc3RydWN0IHsKCWZpbGUgc3RyaW5nCglmaCAgICpvcy5GaWxlCn0KCmZ1bmMgKGwgKmxvY2tm
aWxlKSBMb2NrKCkgKGVyciBlcnJvcikgewoJaSA6PSAwCgoJZm9yIHsKCQlpZiBfLCBlcnIgPSBv
cy5TdGF0KGwuZmlsZSk7IGVyciAhPSBuaWwgJiYgb3MuSXNOb3RFeGlzdChlcnIpIHsKCQkJbC5m
aCwgZXJyID0gb3MuT3BlbkZpbGUobC5maWxlLCBvcy5PX0NSRUFURXxvcy5PX1JEV1J8b3MuT19F
WENMLCAwbzY0NCkKCQkJaWYgZXJyID09IG5pbCB7CgkJCQlyZXR1cm4KCQkJfQoJCX0gZWxzZSB7
CgkJCWVyciA9IG9zLkVyckV4aXN0CgkJfQoKCQlpKysKCQlpZiBpID09IDE1IHsKCQkJcmV0dXJu
CgkJfQoKCQl0aW1lLlNsZWVwKHRpbWUuTWlsbGlzZWNvbmQgKiB0aW1lLkR1cmF0aW9uKCgxICsg
cmFuZC5JbnRuKDEwKSkpKQoJfQp9CgpmdW5jIChsICpsb2NrZmlsZSkgVW5sb2NrKCkgKGVyciBl
cnJvcikgewoJaWYgbC5maCA9PSBuaWwgewoJCXJldHVybgoJfQoKCWwuZmguQ2xvc2UoKQoKCW9z
LlJlbW92ZShsLmZpbGUpCgoJcmV0dXJuCn0KAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZG92ZWNvdC1z
aW11bGF0b3IvbWFpbi5nbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA2NDQAMDAwMTc1MQAwMDAxNzUx
ADAwMDAwMDAxNTQ2ADE0NjI0MDEzNjU0ADAxNTM3MgAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAGtyaWNoeQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAa3JpY2h5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwYWNrYWdlIG1h
aW4KCmltcG9ydCAoCgkiY29udGV4dCIKCSJsb2ciCgkib3MiCgkib3Mvc2lnbmFsIgoJInN5bmMi
Cgkic3lzY2FsbCIKKQoKY29uc3QgKAoJbG9ja0ZpbGUgICAgPSAibWFpbC5sb2NrIgoJbGlzdEZp
bGUgICAgPSAibWFpbC5saXN0IgoJbGlzdEZpbGVOZXcgPSAibWFpbC5saXN0Lm5ldyIKKQoKZnVu
YyBtYWluKCkgewoKCXBhdGggOj0gb3MuQXJnc1sxXQoJaWYgZXJyIDo9IG9zLkNoZGlyKHBhdGgp
OyBlcnIgIT0gbmlsIHsKCQlsb2cuRmF0YWwoZXJyKQoJfQoKCWN0eCwgY2FuY2VsIDo9IGNvbnRl
eHQuV2l0aENhbmNlbChjb250ZXh0LkJhY2tncm91bmQoKSkKCWRlZmVyIGNhbmNlbCgpCgoJZ28g
ZnVuYygpIHsKCQlzaWdjaGFuIDo9IG1ha2UoY2hhbiBvcy5TaWduYWwsIDEpCgkJc2lnbmFsLk5v
dGlmeShzaWdjaGFuLCBzeXNjYWxsLlNJR0lOVCwgc3lzY2FsbC5TSUdURVJNKQoKCQk8LXNpZ2No
YW4KCgkJbG9nLlByaW50KCJFeGl0aW5nIikKCgkJY2FuY2VsKCkKCX0oKQoKCXdnIDo9ICZzeW5j
LldhaXRHcm91cHt9CgoJbG9nLlByaW50KCJTdGFydGluZyByZWFkZXJzIGFuZCB3cml0ZXJzIikK
Cglmb3IgaSA6PSAwOyBpIDwgNTsgaSsrIHsKCQl3Zy5BZGQoMSkKCQlnbyBmdW5jKGluc3RhbmNl
IGludCkgewoJCQlkZWZlciB3Zy5Eb25lKCkKCgkJCXcgOj0gJnJlYWRlcnt9CgoJCQl3LnJ1bihj
dHgsIGluc3RhbmNlKQoJCX0oaSkKCX0KCglmb3IgaSA6PSAwOyBpIDwgNTsgaSsrIHsKCQl3Zy5B
ZGQoMSkKCQlnbyBmdW5jKGluc3RhbmNlIGludCkgewoJCQlkZWZlciB3Zy5Eb25lKCkKCgkJCXcg
Oj0gJndyaXRlcnt9CgoJCQl3LnJ1bihjdHgsIGluc3RhbmNlKQoJCX0oaSkKCX0KCgl3Zy5XYWl0
KCkKfQoAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZG92ZWNvdC1zaW11
bGF0b3Ivd3JpdGVyLmdvAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA2NDQAMDAwMTc1MQAwMDAxNzUxADAw
MDAwMDAxNzYyADE0NjI0MDE0MTEzADAxNTc1MQAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAB1c3RhciAgAGtyaWNoeQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
a3JpY2h5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwYWNrYWdlIG1haW4K
CmltcG9ydCAoCgkiY29udGV4dCIKCSJmbXQiCgkibG9nIgoJIm1hdGgvcmFuZCIKCSJvcyIKCSJ0
aW1lIgopCgp0eXBlIHdyaXRlciBzdHJ1Y3QgewoJbCAqbG9ja2ZpbGUKfQoKZnVuYyAodyAqd3Jp
dGVyKSBydW4oY3R4IGNvbnRleHQuQ29udGV4dCwgaW5zdGFuY2UgaW50KSB7Cgl3LmwgPSAmbG9j
a2ZpbGV7ZmlsZTogbG9ja0ZpbGV9CgoJd2QgOj0gbWFrZShjaGFuIGJvb2wpCgoJZ28gZnVuYygp
IHsKCQl0aW1lciA6PSB0aW1lLk5ld1RpbWVyKHRpbWUuU2Vjb25kKQoJCWRlZmVyIHRpbWVyLlN0
b3AoKQoKCQlmb3IgewoJCQlzZWxlY3QgewoJCQljYXNlIDwtY3R4LkRvbmUoKToKCQkJCXJldHVy
bgoJCQljYXNlIDwtdGltZXIuQzoKCQkJCWxvZy5QcmludGYoIndyaXRlciglZCkgaGFzIGJlZW4g
YmxvY2tlZCBmb3IgYSBzZWNvbmQiLCBpbnN0YW5jZSkKCQkJCXJldHVybgoJCQljYXNlIDwtd2Q6
CgkJCQlpZiAhdGltZXIuU3RvcCgpIHsKCQkJCQk8LXRpbWVyLkMKCQkJCX0KCQkJCXRpbWVyLlJl
c2V0KHRpbWUuU2Vjb25kKQoJCQl9CgkJfQoJfSgpCgoJZm9yIHsKCQlzZWxlY3QgewoJCWNhc2Ug
PC1jdHguRG9uZSgpOgoJCQlyZXR1cm4KCQlkZWZhdWx0OgoJCX0KCgkJdy5kb193cml0ZSgpCgoJ
CXdkIDwtIHRydWUKCX0KfQoKZnVuYyAodyAqd3JpdGVyKSBkb193cml0ZSgpIGVycm9yIHsKCWlm
IGVyciA6PSB3LmwuTG9jaygpOyBlcnIgIT0gbmlsIHsKCQlyZXR1cm4gZXJyCgl9CglkZWZlciB3
LmwuVW5sb2NrKCkKCglmaCwgZXJyIDo9IG9zLk9wZW5GaWxlKGxpc3RGaWxlLCBvcy5PX0NSRUFU
RXxvcy5PX1dST05MWXxvcy5PX0FQUEVORCwgMG82MDApCglpZiBlcnIgIT0gbmlsIHsKCQlyZXR1
cm4gZXJyCgl9CglkZWZlciBmaC5DbG9zZSgpCgoJbGluZSA6PSBmbXQuU3ByaW50ZigiZmlsZS0l
ZC4lZFxuIiwgdGltZS5Ob3coKS5OYW5vc2Vjb25kKCksIHJhbmQuSW50KCkpCglmaC5Xcml0ZShb
XWJ5dGUobGluZSkpCgoJcmV0dXJuIG5pbAp9CgAAAAAAAAAAAAAAAAAAZG92ZWNvdC1zaW11bGF0
b3IvZ28ubW9kAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAwMDA2NDQAMDAwMTc1MQAwMDAxNzUxADAwMDAw
MDAwMDIzADE0NjI0MDEzNzczADAxNTIxNAAgMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAB1c3RhciAgAGtyaWNoeQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAa3Jp
Y2h5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABtb2R1bGUgZHMKCmdvIDEu
MTkKAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
--=_1aee6e723aea2847a539d9257e25d5f0--