Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <01d10e59-0ea6-e60f-8561-84aa5dee40d3@efficios.com>
Date:   Sat, 29 Oct 2022 09:58:26 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.0
Subject: Re: [RFC PATCH 0/2] tracing/user_events: Remote write ABI
To:     Beau Belgrave <beaub@linux.microsoft.com>
Cc:     rostedt@goodmis.org, mhiramat@kernel.org,
        dcook@linux.microsoft.com, alanau@linux.microsoft.com,
        linux-trace-devel@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20221027224011.2075-1-beaub@linux.microsoft.com>
 <96d9f066-2f39-78e6-9be7-f9c69235615e@efficios.com>
 <20221028221728.GA162@W11-BEAU-MD.localdomain>
Content-Language: en-US
From:   Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
In-Reply-To: <20221028221728.GA162@W11-BEAU-MD.localdomain>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2022-10-28 18:17, Beau Belgrave wrote:
> On Fri, Oct 28, 2022 at 05:50:04PM -0400, Mathieu Desnoyers wrote:
>> On 2022-10-27 18:40, Beau Belgrave wrote:

[...]
> 
>>>
>>> NOTE:
>>> User programs that wish to have the enable bit shared across forks
>>> either need to use a MAP_SHARED allocated address or register a new
>>> address and file descriptor. If MAP_SHARED cannot be used or new
>>> registrations cannot be done, then it's allowable to use MAP_PRIVATE
>>> as long as the forked children never update the page themselves. Once
>>> the page has been updated, the page from the parent will be copied over
>>> to the child. This new copy-on-write page will not receive updates from
>>> the kernel until another registration has been performed with this new
>>> address.
>>
>> This seems rather odd. I would expect that if a parent process registers
>> some instrumentation using private mappings for enabled state through the
>> user events ioctl, and then forks, the child process would seamlessly be
>> traced by the user events ABI while being able to also change the enabled
>> state from the userspace tracer libraries (which would trigger COW).
>> Requiring the child to re-register to user events is rather odd.
>>
> 
> It's the COW that is the problem, see below.
> 
>> What is preventing us from tracing the child without re-registration in this
>> scenario ?
>>
> 
> Largely knowing when the COW occurs on a specific page. We don't make
> the mappings, so I'm unsure if we can ask to be notified easily during
> these times or not. If we could, that would solve this. I'm glad you are
> thinking about this. The note here was exactly to trigger this
> discussion :)
> 
> I believe this is the same as a Futex, I'll take another look at that
> code to see if they've come up with anything regarding this.
> 
> Any ideas?

Based on your description of the symptoms, AFAIU, upon registration of a 
given user event associated with a mm_struct, the user events ioctl 
appears to translates the virtual address into a page pointer 
immediately, and keeps track of that page afterwards. This means it 
loses track of the page when COW occurs.

Why not keep track of the registered virtual address and struct_mm 
associated with the event rather than the page ? Whenever a state change 
is needed, the virtual-address-to-page translation will be performed 
again. If it follows a COW, it will get the new copied page. If it 
happens that no COW was done, it should map to the original page. If the 
mapping is shared, the kernel would update that shared page. If the 
mapping is private, then the kernel would COW the page before updating it.

Thoughts ?

Thanks,

Mathieu

> 
> Thanks,
> -Beau

-- 
Mathieu Desnoyers
EfficiOS Inc.
https://www.efficios.com