Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
From:   =?utf-8?Q?Lu=C3=ADs_Henriques?= <lhenriques@suse.de>
To:     Xiubo Li <xiubli@redhat.com>
Cc:     Jeff Layton <jlayton@kernel.org>,
        Ilya Dryomov <idryomov@gmail.com>, ceph-devel@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] ceph: invalidate pages when doing DIO in encrypted
 inodes
References: <20220401133243.1075-1-lhenriques@suse.de>
        <d6407dd1-b6df-4de4-fe37-71b765b2088a@redhat.com>
        <878rsia391.fsf@brahms.olymp>
        <6ba91390-83e8-8702-2729-dc432abd3cc5@redhat.com>
Date:   Wed, 06 Apr 2022 12:33:11 +0100
In-Reply-To: <6ba91390-83e8-8702-2729-dc432abd3cc5@redhat.com> (Xiubo Li's
        message of "Wed, 6 Apr 2022 19:18:04 +0800")
Message-ID: <87zgky8n0o.fsf@brahms.olymp>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Xiubo Li <xiubli@redhat.com> writes:

> On 4/6/22 6:57 PM, Lu=C3=ADs Henriques wrote:
>> Xiubo Li <xiubli@redhat.com> writes:
>>
>>> On 4/1/22 9:32 PM, Lu=C3=ADs Henriques wrote:
>>>> When doing DIO on an encrypted node, we need to invalidate the page ca=
che in
>>>> the range being written to, otherwise the cache will include invalid d=
ata.
>>>>
>>>> Signed-off-by: Lu=C3=ADs Henriques <lhenriques@suse.de>
>>>> ---
>>>>    fs/ceph/file.c | 11 ++++++++++-
>>>>    1 file changed, 10 insertions(+), 1 deletion(-)
>>>>
>>>> Changes since v1:
>>>> - Replaced truncate_inode_pages_range() by invalidate_inode_pages2_ran=
ge
>>>> - Call fscache_invalidate with FSCACHE_INVAL_DIO_WRITE if we're doing =
DIO
>>>>
>>>> Note: I'm not really sure this last change is required, it doesn't rea=
lly
>>>> affect generic/647 result, but seems to be the most correct.
>>>>
>>>> diff --git a/fs/ceph/file.c b/fs/ceph/file.c
>>>> index 5072570c2203..b2743c342305 100644
>>>> --- a/fs/ceph/file.c
>>>> +++ b/fs/ceph/file.c
>>>> @@ -1605,7 +1605,7 @@ ceph_sync_write(struct kiocb *iocb, struct iov_i=
ter *from, loff_t pos,
>>>>    	if (ret < 0)
>>>>    		return ret;
>>>>    -	ceph_fscache_invalidate(inode, false);
>>>> +	ceph_fscache_invalidate(inode, (iocb->ki_flags & IOCB_DIRECT));
>>>>    	ret =3D invalidate_inode_pages2_range(inode->i_mapping,
>>>>    					    pos >> PAGE_SHIFT,
>>>>    					    (pos + count - 1) >> PAGE_SHIFT);
>>> The above has already invalidated the pages, why doesn't it work ?
>> I suspect the reason is because later on we loop through the number of
>> pages, call copy_page_from_iter() and then ceph_fscrypt_encrypt_pages().
>
> Checked the 'copy_page_from_iter()', it will do the kmap for the pages bu=
t will
> kunmap them again later. And they shouldn't update the i_mapping if I did=
n't
> miss something important.
>
> For 'ceph_fscrypt_encrypt_pages()' it will encrypt/dencrypt the context i=
nplace,
> IMO if it needs to map the page and it should also unmap it just like in
> 'copy_page_from_iter()'.
>
> I thought it possibly be when we need to do RMW, it may will update the
> i_mapping when reading contents, but I checked the code didn't find any=20
> place is doing this. So I am wondering where tha page caches come from ? =
If that
> page caches really from reading the contents, then we should discard it i=
nstead
> of flushing it back ?
>
> BTW, what's the problem without this fixing ? xfstest fails ?

Yes, generic/647 fails if you run it with test_dummy_encryption.  And I've
also checked that the RMW code was never executed in this test.

But yeah I have assumed (perhaps wrongly) that the kmap/kunmap could
change the inode->i_mapping.  In my debugging this seemed to be the case
for the O_DIRECT path.  That's why I added this extra call here.

Cheers,
--=20
Lu=C3=ADs