2013-04-13 11:54:48

by Heinrich Schuchardt

[permalink] [raw]
Subject: Re: fanotify: fix support of large files

Dear Justin,

looking at the example at
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
the large file support is enabled by passing O_LARGEFILE to fanotify_init:

if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
O_RDONLY | O_CLOEXEC |
O_LARGEFILE)) < 0)

Could you, please, check if this solves your issue.

(I am resending this message because HTML was rejected by
[email protected]).

Best regards

Heinrich Schuchardt

On 14.02.2013 18:29, Justin Maggard wrote:
> Opening a file of>2GB in an area of the filesystem that has been marked for
> fanotify events currently results in an EOVERFLOW error. This is particularly
> problematic if you are using fanotify permissions checking, because it prevents
> large files from being opened at all. Fix this by setting the O_LARGEFILE flag
> on the new file handle.
>
> Signed-off-by: Justin Maggard<[email protected]>
>
> ---
> fs/notify/fanotify/fanotify_user.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/fs/notify/fanotify/fanotify_user.c b/fs/notify/fanotify/fanotify_user.c
> index 9ff4a5e..ef02b3d 100644
> --- a/fs/notify/fanotify/fanotify_user.c
> +++ b/fs/notify/fanotify/fanotify_user.c
> @@ -64,6 +64,7 @@ static int create_fd(struct fsnotify_group *group,
> struct file **file)
> {
> int client_fd;
> + int f_flags;
> struct file *new_file;
>
> pr_debug("%s: group=%p event=%p\n", __func__, group, event);
> @@ -82,12 +83,11 @@ static int create_fd(struct fsnotify_group *group,
> * we need a new file handle for the userspace program so it can read even if it was
> * originally opened O_WRONLY.
> */
> + f_flags = group->fanotify_data.f_flags | FMODE_NONOTIFY | O_LARGEFILE;
> /* it's possible this event was an overflow event. in that case dentry and mnt
> * are NULL; That's fine, just don't call dentry open */
> if (event->path.dentry&& event->path.mnt)
> - new_file = dentry_open(&event->path,
> - group->fanotify_data.f_flags | FMODE_NONOTIFY,
> - current_cred());
> + new_file = dentry_open(&event->path, f_flags, current_cred());
> else
> new_file = ERR_PTR(-EOVERFLOW);
> if (IS_ERR(new_file)) {
>


2013-04-19 19:23:56

by Justin Maggard

[permalink] [raw]
Subject: Re: fanotify: fix support of large files

On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt <[email protected]> wrote:
> Dear Justin,
>
> looking at the example at
> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>
> if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
> O_RDONLY | O_CLOEXEC | O_LARGEFILE)) <
> 0)
>
> Could you, please, check if this solves your issue.
>
> (I am resending this message because HTML was rejected by
> [email protected]).
>
> Best regards
>
> Heinrich Schuchardt
>

No, unfortunately that doesn't help. I slightly modifed
fanotify-example.c to call perror() when read() fails, and here's the
output:

jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example . &
[1] 7248
jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
truncate -s 2047m 2047m
Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
FAN_OPEN
FAN_CLOSE_WRITE
jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
read: Value too large for defined data type

-Justin

2013-04-22 18:49:28

by Heinrich Schuchardt

[permalink] [raw]
Subject: Re: fanotify: fix support of large files

Hello Justin,

I downloaded the example,
http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
compiled it without modification
$ gcc fanotify-example.c -o fanotify-example
and started the executable. In a separate windows I executed
truncate -s 2048m 2048m

This is the output of the first window:
$ sudo ./fanotify-example /home/user/temp/
Started monitoring directory '/home/user/temp/'...
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
FAN_OPEN
Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
FAN_CLOSE_WRITE

$ uname -a
Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux

To reproduce your problem, could you, please, provide the Linux
configuration file (look in your /boot directory) and a link to the
source of the Linux kernel version you use. Then I can use your
configuration file to compile that kernel version.

Best regards

Heinrich Schuchardt

On 19.04.2013 21:23, Justin Maggard wrote:
> On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt<[email protected]> wrote:
>> Dear Justin,
>>
>> looking at the example at
>> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
>> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>>
>> if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
>> O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
>> 0)
>>
>> Could you, please, check if this solves your issue.
>>
>> (I am resending this message because HTML was rejected by
>> [email protected]).
>>
>> Best regards
>>
>> Heinrich Schuchardt
>>
>
> No, unfortunately that doesn't help. I slightly modifed
> fanotify-example.c to call perror() when read() fails, and here's the
> output:
>
> jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
> [1] 7248
> jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
> truncate -s 2047m 2047m
> Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
> FAN_OPEN
> FAN_CLOSE_WRITE
> jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
> read: Value too large for defined data type
>
> -Justin
>

2013-07-11 22:10:38

by Justin Maggard

[permalink] [raw]
Subject: Re: fanotify: fix support of large files

On Mon, Apr 22, 2013 at 11:49 AM, Heinrich Schuchardt
<[email protected]> wrote:
>
> Hello Justin,
>
> I downloaded the example,
> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
> compiled it without modification
> $ gcc fanotify-example.c -o fanotify-example
> and started the executable. In a separate windows I executed
> truncate -s 2048m 2048m
>
> This is the output of the first window:
> $ sudo ./fanotify-example /home/user/temp/
> Started monitoring directory '/home/user/temp/'...
> Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
> FAN_OPEN
> Received event in path '/home/user/temp/2048m' pid=3659 (truncate):
> FAN_CLOSE_WRITE
>
> $ uname -a
> Linux family2 3.8.0 #1 SMP Fri Feb 22 22:07:58 CET 2013 i686 GNU/Linux
>

Ah yes, that's the reason. I'm running on x86_64. 32-bit platforms
actually work if you set O_LARGEFILE. From /usr/incude/bits/fcntl.h:

#ifdef __USE_LARGEFILE64
# if __WORDSIZE == 64
# define O_LARGEFILE 0
# else
# define O_LARGEFILE 0100000
# endif
#endif

If I include <asm/fcntl.h> instead of <fcntl.h>, it gets set to
0x8000, and my large file fanotify test works. Or, of course, I can
define that flag locally and use it. But it feels like I shouldn't
have to do that.

-Justin

>
> To reproduce your problem, could you, please, provide the Linux configuration file (look in your /boot directory) and a link to the source of the Linux kernel version you use. Then I can use your configuration file to compile that kernel version.
>
> Best regards
>
> Heinrich Schuchardt
>
>
> On 19.04.2013 21:23, Justin Maggard wrote:
>>
>> On Sat, Apr 13, 2013 at 4:54 AM, Heinrich Schuchardt<[email protected]> wrote:
>>>
>>> Dear Justin,
>>>
>>> looking at the example at
>>> http://www.lanedo.com/~aleksander/fanotify/fanotify-example.c
>>> the large file support is enabled by passing O_LARGEFILE to fanotify_init:
>>>
>>> if ((fanotify_fd = fanotify_init (FAN_CLOEXEC,
>>> O_RDONLY | O_CLOEXEC | O_LARGEFILE))<
>>> 0)
>>>
>>> Could you, please, check if this solves your issue.
>>>
>>> (I am resending this message because HTML was rejected by
>>> [email protected]).
>>>
>>> Best regards
>>>
>>> Heinrich Schuchardt
>>>
>>
>> No, unfortunately that doesn't help. I slightly modifed
>> fanotify-example.c to call perror() when read() fails, and here's the
>> output:
>>
>> jmaggard@jmaggard-W520:~/fanotify-test$ sudo ./fanotify-example .&
>> [1] 7248
>> jmaggard@jmaggard-W520:~/fanotify-test$ Started monitoring directory '.'...
>> truncate -s 2047m 2047m
>> Received event in path '/home/jmaggard/fanotify-test/2047m' pid=7250 (unknown):
>> FAN_OPEN
>> FAN_CLOSE_WRITE
>> jmaggard@jmaggard-W520:~/fanotify-test$ truncate -s 2048m 2048m
>> read: Value too large for defined data type
>>
>> -Justin
>>
>