Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 22/35] kvm, x86: Distinguish dmemfs page from mmio page
To:     Sean Christopherson <sean.j.christopherson@intel.com>,
        yulei.kernel@gmail.com
Cc:     akpm@linux-foundation.org, naoya.horiguchi@nec.com,
        viro@zeniv.linux.org.uk, pbonzini@redhat.com,
        linux-fsdevel@vger.kernel.org, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, xiaoguangrong.eric@gmail.com,
        kernellwp@gmail.com, lihaiwei.kernel@gmail.com,
        Yulei Zhang <yuleixzhang@tencent.com>,
        Chen Zhuo <sagazchen@tencent.com>
References: <cover.1602093760.git.yuleixzhang@tencent.com>
 <b2b6837785f6786575823c919788464373d3ee05.1602093760.git.yuleixzhang@tencent.com>
 <20201009005823.GA11151@linux.intel.com>
From:   Joao Martins <joao.m.martins@oracle.com>
Message-ID: <a6dd5fbe-ca71-cf05-ec40-ec916843e9b7@oracle.com>
Date:   Fri, 9 Oct 2020 11:28:31 +0100
MIME-Version: 1.0
In-Reply-To: <20201009005823.GA11151@linux.intel.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 10/9/20 1:58 AM, Sean Christopherson wrote:
> On Thu, Oct 08, 2020 at 03:54:12PM +0800, yulei.kernel@gmail.com wrote:
>> From: Yulei Zhang <yuleixzhang@tencent.com>
>>
>> Dmem page is pfn invalid but not mmio. Support cacheable
>> dmem page for kvm.
>>
>> Signed-off-by: Chen Zhuo <sagazchen@tencent.com>
>> Signed-off-by: Yulei Zhang <yuleixzhang@tencent.com>
>> ---
>>  arch/x86/kvm/mmu/mmu.c | 5 +++--
>>  include/linux/dmem.h   | 7 +++++++
>>  mm/dmem.c              | 7 +++++++
>>  3 files changed, 17 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
>> index 71aa3da2a0b7..0115c1767063 100644
>> --- a/arch/x86/kvm/mmu/mmu.c
>> +++ b/arch/x86/kvm/mmu/mmu.c
>> @@ -41,6 +41,7 @@
>>  #include <linux/hash.h>
>>  #include <linux/kern_levels.h>
>>  #include <linux/kthread.h>
>> +#include <linux/dmem.h>
>>  
>>  #include <asm/page.h>
>>  #include <asm/memtype.h>
>> @@ -2962,9 +2963,9 @@ static bool kvm_is_mmio_pfn(kvm_pfn_t pfn)
>>  			 */
>>  			(!pat_enabled() || pat_pfn_immune_to_uc_mtrr(pfn));
>>  
>> -	return !e820__mapped_raw_any(pfn_to_hpa(pfn),
>> +	return (!e820__mapped_raw_any(pfn_to_hpa(pfn),
>>  				     pfn_to_hpa(pfn + 1) - 1,
>> -				     E820_TYPE_RAM);
>> +				     E820_TYPE_RAM)) || (!is_dmem_pfn(pfn));
> 
> This is wrong.  As is, the logic reads "A PFN is MMIO if it is INVALID &&
> (!RAM || !DMEM)".  The obvious fix would be to change it to "INVALID &&
> !RAM && !DMEM", but that begs the question of whether or DMEM is reported
> as RAM.  I don't see any e820 related changes in the series, i.e. no evidence
> that dmem yanks its memory out of the e820 tables, which makes me think this
> change is unnecessary.
> 
Even if there would exist e820 changes, e820__mapped_raw_any() checks against
hardware-provided e820 that we are given before any changes happen i.e. not the one kernel
has changed (e820_table_firmware). So unless you're having that memory carved from an MMIO
range (which would be wrong), or the BIOS is misrepresenting its memory map... the
e820__mapped_raw_any(E820_TYPE_RAM) ought to be enough to cover RAM.

Or at least that has been my experience with similar work.

	Joao