Hello Michael,
These patches are some kind of brief highlights of the changes to the
userfaultfd pages.
The changes to userfaultfd functionality are also described at update to
Documentation/vm/userfaultfd.txt [1].
In general, there were three major additions:
* hugetlbfs support
* shmem support
* non-page fault events
I think we should add some details about using userfaultfd with different
memory types, describe meaning of each feature bits and add some text about
the new events.
I haven't updated 'struct uffd_msg' yet, and I hesitate whether it's
description belongs to userfaultfd.2 or ioctl_userfaultfd.2
As for the userfaultfd.7 we've discussed earlier, I believe it would
repeat Documentation/vm/userfaultfd.txt in way, so I'm not really sure it
is required.
--
Sincerely yours,
Mike.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5a02026d390ea1bb0c16a0e214e45613a3e3d885
Mike Rapoport (5):
userfaultfd.2: describe memory types that can be used from 4.11
ioctl_userfaultfd.2: describe memory types that can be used from 4.11
ioctl_userfaultfd.2: update UFFDIO_API description
userfaultfd.2: add Linux container migration use-case to NOTES
usefaultfd.2: add brief description of "non-cooperative" mode
man2/ioctl_userfaultfd.2 | 46 ++++++++++++++++++++++++++++++++++++++--------
man2/userfaultfd.2 | 25 ++++++++++++++++++++++---
2 files changed, 60 insertions(+), 11 deletions(-)
--
1.9.1
Signed-off-by: Mike Rapoport <[email protected]>
---
man2/userfaultfd.2 | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index c89484f..dc37319 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -279,7 +279,8 @@ signal and
It can also be used to implement lazy restore
for checkpoint/restore mechanisms,
as well as post-copy migration to allow (nearly) uninterrupted execution
-when transferring virtual machines from one host to another.
+when transferring virtual machines and Linux containers
+from one host to another.
.SH EXAMPLE
The program below demonstrates the use of the userfaultfd mechanism.
The program creates two threads, one of which acts as the
--
1.9.1
Signed-off-by: Mike Rapoport <[email protected]>
---
man2/ioctl_userfaultfd.2 | 38 ++++++++++++++++++++++++++++++++------
1 file changed, 32 insertions(+), 6 deletions(-)
diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 78abc4d..dade631 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -95,11 +95,6 @@ struct uffdio_api {
The
.I api
field denotes the API version requested by the application.
-Before the call, the
-.I features
-field must be initialized to zero.
-In the future, it is intended that this field can be used to ask
-whether particular features are supported.
The kernel verifies that it can support the requested API version,
and sets the
@@ -109,9 +104,40 @@ and
fields to bit masks representing all the available features and the generic
.BR ioctl (2)
operations available.
-Currently, zero (i.e., no feature bits) is placed in the
+
+For Linux kernel versions before 4.11, the
+.I features
+field must be initialized to zero before the call to
+.I UFFDIO_API
+, and zero (i.e., no feature bits) is placed in the
+.I features
+field by the kernel upon return from
+.BR ioctl (2).
+
+Starting from Linux 4.11, the
+.I features
+field can be used to to ask whether particular features are supported
+and explicitly enable userfaultfd features that are disabled by default.
+The kernel always reports all the available features in the
.I features
field.
+.\" FIXME add more details about feature negotiation and enablement
+
+Since Linux 4.11, the following feature bits may be set:
+.TP
+.B UFFD_FEATURE_EVENT_FORK
+.TP
+.B UFFD_FEATURE_EVENT_REMAP
+.TP
+.B UFFD_FEATURE_EVENT_REMOVE
+.TP
+.B UFFD_FEATURE_EVENT_UNMAP
+.TP
+.B UFFD_FEATURE_MISSING_HUGETLBFS
+.TP
+.B UFFD_FEATURE_MISSING_SHMEM
+.\" FIXME add feature description
+
The returned
.I ioctls
field can contain the following bits:
--
1.9.1
Signed-off-by: Mike Rapoport <[email protected]>
---
man2/userfaultfd.2 | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index 1603c20..c89484f 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -130,8 +130,12 @@ Details of the various
operations can be found in
.BR ioctl_userfaultfd (2).
-Currently, userfaultfd can be used only with anonymous private memory
-mappings.
+Up to Linux 4.11,
+userfaultfd can be used only with anonymous private memory mappings.
+
+Since Linux 4.11,
+userfaultfd can be also used with hugetlbfs and shared memory mappings.
+
.\"
.SS Reading from the userfaultfd structure
Each
--
1.9.1
Signed-off-by: Mike Rapoport <[email protected]>
---
man2/userfaultfd.2 | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
index dc37319..291dd10 100644
--- a/man2/userfaultfd.2
+++ b/man2/userfaultfd.2
@@ -89,6 +89,20 @@ them using the operations described in
.BR ioctl_userfaultfd (2).
When servicing the page fault events,
the fault-handling thread can trigger a wake-up for the sleeping thread.
+
+It is possible for the faulting threads and the fault-handling threads
+to run in the context of different processes.
+In this case, these threads may belong to different programs,
+and the program that executes the faulting threads
+will not necessarily cooperate with the program that handles the page faults.
+In such non-cooperative mode,
+the process that monitors userfaultfd and handles page faults,
+needs to be aware of the changes in the virtual memory layout
+of the faulting process to avoid memory corruption.
+.\" FIXME elaborate about non-cooperating mode, describe its limitations
+.\" for kerneles before 4.11, features added in 4.11
+.\" and limitations remaining in 4.11
+.\" Maybe it's worth adding a dedicated sub-section...
.\"
.SS Userfaultfd operation
After the userfaultfd object is created with
--
1.9.1
Signed-off-by: Mike Rapoport <[email protected]>
---
man2/ioctl_userfaultfd.2 | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
index 66fbfdc..78abc4d 100644
--- a/man2/ioctl_userfaultfd.2
+++ b/man2/ioctl_userfaultfd.2
@@ -169,11 +169,15 @@ field was not zero.
(Since Linux 4.3.)
Register a memory address range with the userfaultfd object.
The pages in the range must be "compatible".
-In the current implementation,
-.\" According to Mike Rapoport, this will change in Linux 4.11.
+
+Up to Linux kernel 4.11,
only private anonymous ranges are compatible for registering with
.BR UFFDIO_REGISTER .
+Since Linux 4.11,
+hugetlbfs and shared memory ranges are also compatible with
+.BR UFFDIO_REGISTER .
+
The
.I argp
argument is a pointer to a
--
1.9.1
On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <[email protected]>
Thanks, Mike. Applied.
Cheers,
Michael
> ---
> man2/userfaultfd.2 | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index 1603c20..c89484f 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -130,8 +130,12 @@ Details of the various
> operations can be found in
> .BR ioctl_userfaultfd (2).
>
> -Currently, userfaultfd can be used only with anonymous private memory
> -mappings.
> +Up to Linux 4.11,
> +userfaultfd can be used only with anonymous private memory mappings.
> +
> +Since Linux 4.11,
> +userfaultfd can be also used with hugetlbfs and shared memory mappings.
> +
> .\"
> .SS Reading from the userfaultfd structure
> Each
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <[email protected]>
Thanks, Mike. Applied.
Cheers,
Michael
> ---
> man2/ioctl_userfaultfd.2 | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/man2/ioctl_userfaultfd.2 b/man2/ioctl_userfaultfd.2
> index 66fbfdc..78abc4d 100644
> --- a/man2/ioctl_userfaultfd.2
> +++ b/man2/ioctl_userfaultfd.2
> @@ -169,11 +169,15 @@ field was not zero.
> (Since Linux 4.3.)
> Register a memory address range with the userfaultfd object.
> The pages in the range must be "compatible".
> -In the current implementation,
> -.\" According to Mike Rapoport, this will change in Linux 4.11.
> +
> +Up to Linux kernel 4.11,
> only private anonymous ranges are compatible for registering with
> .BR UFFDIO_REGISTER .
>
> +Since Linux 4.11,
> +hugetlbfs and shared memory ranges are also compatible with
> +.BR UFFDIO_REGISTER .
> +
> The
> .I argp
> argument is a pointer to a
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <[email protected]>
Thanks, Mike. Applied.
Cheers,
Michael
> ---
> man2/userfaultfd.2 | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index c89484f..dc37319 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -279,7 +279,8 @@ signal and
> It can also be used to implement lazy restore
> for checkpoint/restore mechanisms,
> as well as post-copy migration to allow (nearly) uninterrupted execution
> -when transferring virtual machines from one host to another.
> +when transferring virtual machines and Linux containers
> +from one host to another.
> .SH EXAMPLE
> The program below demonstrates the use of the userfaultfd mechanism.
> The program creates two threads, one of which acts as the
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> Signed-off-by: Mike Rapoport <[email protected]>
Thanks, Mike. Applied.
Cheers,
Michael
> ---
> man2/userfaultfd.2 | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/man2/userfaultfd.2 b/man2/userfaultfd.2
> index dc37319..291dd10 100644
> --- a/man2/userfaultfd.2
> +++ b/man2/userfaultfd.2
> @@ -89,6 +89,20 @@ them using the operations described in
> .BR ioctl_userfaultfd (2).
> When servicing the page fault events,
> the fault-handling thread can trigger a wake-up for the sleeping thread.
> +
> +It is possible for the faulting threads and the fault-handling threads
> +to run in the context of different processes.
> +In this case, these threads may belong to different programs,
> +and the program that executes the faulting threads
> +will not necessarily cooperate with the program that handles the page faults.
> +In such non-cooperative mode,
> +the process that monitors userfaultfd and handles page faults,
> +needs to be aware of the changes in the virtual memory layout
> +of the faulting process to avoid memory corruption.
> +.\" FIXME elaborate about non-cooperating mode, describe its limitations
> +.\" for kerneles before 4.11, features added in 4.11
> +.\" and limitations remaining in 4.11
> +.\" Maybe it's worth adding a dedicated sub-section...
> .\"
> .SS Userfaultfd operation
> After the userfaultfd object is created with
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
Hello Mike,
On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> Hello Michael,
>
> These patches are some kind of brief highlights of the changes to the
> userfaultfd pages.
Thanks for the patches. All merged. A few tweaks made,
and pushed to Git.
> The changes to userfaultfd functionality are also described at update to
> Documentation/vm/userfaultfd.txt [1].
>
> In general, there were three major additions:
> * hugetlbfs support
> * shmem support
> * non-page fault events
>
> I think we should add some details about using userfaultfd with different
> memory types, describe meaning of each feature bits and add some text about
> the new events.
Agreed.
> I haven't updated 'struct uffd_msg' yet, and I hesitate whether it's
> description belongs to userfaultfd.2 or ioctl_userfaultfd.2
My guess is userfaultfd.2. But, maybe I missed something.
What suggests to you that it could be ioctl_userfaultfd.2 instead?
> As for the userfaultfd.7 we've discussed earlier, I believe it would
> repeat Documentation/vm/userfaultfd.txt in way, so I'm not really sure it
> is required.
The thing about kernel Doc files is they are a lot less visible.
It would be best I think to have the user-space visible
API fully described in man pages...
Cheers,
Michael
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5a02026d390ea1bb0c16a0e214e45613a3e3d885
>
> Mike Rapoport (5):
> userfaultfd.2: describe memory types that can be used from 4.11
> ioctl_userfaultfd.2: describe memory types that can be used from 4.11
> ioctl_userfaultfd.2: update UFFDIO_API description
> userfaultfd.2: add Linux container migration use-case to NOTES
> usefaultfd.2: add brief description of "non-cooperative" mode
>
> man2/ioctl_userfaultfd.2 | 46 ++++++++++++++++++++++++++++++++++++++--------
> man2/userfaultfd.2 | 25 ++++++++++++++++++++++---
> 2 files changed, 60 insertions(+), 11 deletions(-)
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
On Wed, Apr 26, 2017 at 09:23:45AM +0200, Michael Kerrisk (man-pages) wrote:
> Hello Mike,
>
> On 04/25/2017 06:29 PM, Mike Rapoport wrote:
> > Hello Michael,
> >
> > These patches are some kind of brief highlights of the changes to the
> > userfaultfd pages.
>
> Thanks for the patches. All merged. A few tweaks made,
> and pushed to Git.
>
> > The changes to userfaultfd functionality are also described at update to
> > Documentation/vm/userfaultfd.txt [1].
> >
> > In general, there were three major additions:
> > * hugetlbfs support
> > * shmem support
> > * non-page fault events
> >
> > I think we should add some details about using userfaultfd with different
> > memory types, describe meaning of each feature bits and add some text about
> > the new events.
>
> Agreed.
>
> > I haven't updated 'struct uffd_msg' yet, and I hesitate whether it's
> > description belongs to userfaultfd.2 or ioctl_userfaultfd.2
>
> My guess is userfaultfd.2. But, maybe I missed something.
> What suggests to you that it could be ioctl_userfaultfd.2 instead?
I've started to add relatively elaborate descriptions of UFFD_EVENT_* to
ioctl_userfaultfd.2 and I've found that I write a lot about struct uffd_msg
fields, but the structure itself is described at userfaultfd.2.
Now, when I'm thinking about it, maybe it would be better to put the
detailed descriptions of the events in userfaultfd.2 and only brief notes
in ioctl_userfaultfd.2.
> > As for the userfaultfd.7 we've discussed earlier, I believe it would
> > repeat Documentation/vm/userfaultfd.txt in way, so I'm not really sure it
> > is required.
>
> The thing about kernel Doc files is they are a lot less visible.
> It would be best I think to have the user-space visible
> API fully described in man pages...
I agree with the point about the visibility, I just don't know if
userfaultfd.7 would be required or we'll have all the necessary bits in
{ioctl_}userfaultfd.2. I'm going to add more content to the man2 pages and
then we'll see if we need man7 as well.
> Cheers,
>
> Michael
>
>
> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5a02026d390ea1bb0c16a0e214e45613a3e3d885
> >
> > Mike Rapoport (5):
> > userfaultfd.2: describe memory types that can be used from 4.11
> > ioctl_userfaultfd.2: describe memory types that can be used from 4.11
> > ioctl_userfaultfd.2: update UFFDIO_API description
> > userfaultfd.2: add Linux container migration use-case to NOTES
> > usefaultfd.2: add brief description of "non-cooperative" mode
> >
> > man2/ioctl_userfaultfd.2 | 46 ++++++++++++++++++++++++++++++++++++++--------
> > man2/userfaultfd.2 | 25 ++++++++++++++++++++++---
> > 2 files changed, 60 insertions(+), 11 deletions(-)
> >
>
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
>
Hello Mike,
On 04/26/2017 09:43 AM, Mike Rapoport wrote:
> On Wed, Apr 26, 2017 at 09:23:45AM +0200, Michael Kerrisk (man-pages) wrote:
>> Hello Mike,
>>
>> On 04/25/2017 06:29 PM, Mike Rapoport wrote:
>>> Hello Michael,
>>>
>>> These patches are some kind of brief highlights of the changes to the
>>> userfaultfd pages.
>>
>> Thanks for the patches. All merged. A few tweaks made,
>> and pushed to Git.
>>
>>> The changes to userfaultfd functionality are also described at update to
>>> Documentation/vm/userfaultfd.txt [1].
>>>
>>> In general, there were three major additions:
>>> * hugetlbfs support
>>> * shmem support
>>> * non-page fault events
>>>
>>> I think we should add some details about using userfaultfd with different
>>> memory types, describe meaning of each feature bits and add some text about
>>> the new events.
>>
>> Agreed.
>>
>>> I haven't updated 'struct uffd_msg' yet, and I hesitate whether it's
>>> description belongs to userfaultfd.2 or ioctl_userfaultfd.2
>>
>> My guess is userfaultfd.2. But, maybe I missed something.
>> What suggests to you that it could be ioctl_userfaultfd.2 instead?
>
> I've started to add relatively elaborate descriptions of UFFD_EVENT_* to
> ioctl_userfaultfd.2 and I've found that I write a lot about struct uffd_msg
> fields, but the structure itself is described at userfaultfd.2.
> Now, when I'm thinking about it, maybe it would be better to put the
> detailed descriptions of the events in userfaultfd.2 and only brief notes
> in ioctl_userfaultfd.2.
Okay -- we can tune as we go along.
>>> As for the userfaultfd.7 we've discussed earlier, I believe it would
>>> repeat Documentation/vm/userfaultfd.txt in way, so I'm not really sure it
>>> is required.
>>
>> The thing about kernel Doc files is they are a lot less visible.
>> It would be best I think to have the user-space visible
>> API fully described in man pages...
>
> I agree with the point about the visibility, I just don't know if
> userfaultfd.7 would be required or we'll have all the necessary bits in
> {ioctl_}userfaultfd.2. I'm going to add more content to the man2 pages and
> then we'll see if we need man7 as well.
Okay.
Cheers,
Michael
>> Cheers,
>>
>> Michael
>>
>>
>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5a02026d390ea1bb0c16a0e214e45613a3e3d885
>>>
>>> Mike Rapoport (5):
>>> userfaultfd.2: describe memory types that can be used from 4.11
>>> ioctl_userfaultfd.2: describe memory types that can be used from 4.11
>>> ioctl_userfaultfd.2: update UFFDIO_API description
>>> userfaultfd.2: add Linux container migration use-case to NOTES
>>> usefaultfd.2: add brief description of "non-cooperative" mode
>>>
>>> man2/ioctl_userfaultfd.2 | 46 ++++++++++++++++++++++++++++++++++++++--------
>>> man2/userfaultfd.2 | 25 ++++++++++++++++++++++---
>>> 2 files changed, 60 insertions(+), 11 deletions(-)
>>>
>>
>>
>> --
>> Michael Kerrisk
>> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
>> Linux/UNIX System Programming Training: http://man7.org/training/
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-man" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/