Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 6 Mar 2020 16:36:07 -0800 (PST)
From:   David Rientjes <rientjes@google.com>
To:     Christoph Hellwig <hch@lst.de>
cc:     Tom Lendacky <thomas.lendacky@amd.com>,
        "Singh, Brijesh" <brijesh.singh@amd.com>,
        "Grimm, Jon" <jon.grimm@amd.com>, Joerg Roedel <joro@8bytes.org>,
        baekhw@google.com,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Subject: Re: [rfc 5/6] dma-direct: atomic allocations must come from unencrypted
 pools
In-Reply-To: <20200305154456.GC5332@lst.de>
Message-ID: <alpine.DEB.2.21.2003061623060.27928@chino.kir.corp.google.com>
References: <alpine.DEB.2.21.1912311738130.68206@chino.kir.corp.google.com> <b22416ec-cc28-3fd2-3a10-89840be173fa@amd.com> <alpine.DEB.2.21.2002280118461.165532@chino.kir.corp.google.com> <alpine.DEB.2.21.2003011535510.213582@chino.kir.corp.google.com>
 <alpine.DEB.2.21.2003011538040.213582@chino.kir.corp.google.com> <20200305154456.GC5332@lst.de>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 5 Mar 2020, Christoph Hellwig wrote:

> On Sun, Mar 01, 2020 at 04:05:23PM -0800, David Rientjes wrote:
> > When AMD memory encryption is enabled, all non-blocking DMA allocations
> > must originate from the atomic pools depending on the device and the gfp
> > mask of the allocation.
> > 
> > Keep all memory in these pools unencrypted.
> > 
> > Signed-off-by: David Rientjes <rientjes@google.com>
> > ---
> >  arch/x86/Kconfig    | 1 +
> >  kernel/dma/direct.c | 9 ++++-----
> >  kernel/dma/remap.c  | 2 ++
> >  3 files changed, 7 insertions(+), 5 deletions(-)
> > 
> > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > --- a/arch/x86/Kconfig
> > +++ b/arch/x86/Kconfig
> > @@ -1523,6 +1523,7 @@ config X86_CPA_STATISTICS
> >  config AMD_MEM_ENCRYPT
> >  	bool "AMD Secure Memory Encryption (SME) support"
> >  	depends on X86_64 && CPU_SUP_AMD
> > +	select DMA_DIRECT_REMAP
> 
> I think we need to split the pool from remapping so that we don't drag
> in the remap code for x86.
> 

Thanks for the review, Christoph.  I can address all the comments that you 
provided for the series but am hoping to get a clarification on this one 
depending on how elaborate the change you would prefer.

As a preliminary change to this series, I could move the atomic pools and 
coherent_pool command line to a new kernel/dma/atomic_pools.c file with a 
new CONFIG_DMA_ATOMIC_POOLS that would get "select"ed by CONFIG_DMA_REMAP 
and CONFIG_AMD_MEM_ENCRYPT and call into dma_common_contiguous_remap() if 
we have CONFIG_DMA_DIRECT_REMAP when adding pages to the pool.

I think that's what you mean by splitting the pool from remapping, 
otherwise we still have a full CONFIG_DMA_REMAP dependency here.  If you 
had something else in mind, please let me know.  Thanks!

> >  	if (IS_ENABLED(CONFIG_DMA_DIRECT_REMAP) &&
> > -	    dma_alloc_need_uncached(dev, attrs) &&
> 
> We still need a check here for either uncached or memory encryption.
> 
> > @@ -141,6 +142,7 @@ static int atomic_pool_expand(struct gen_pool *pool, size_t pool_size,
> >  	if (!addr)
> >  		goto free_page;
> >  
> > +	set_memory_decrypted((unsigned long)page_to_virt(page), nr_pages);
> 
> This probably warrants a comment.
> 
> Also I think the infrastructure changes should be split from the x86
> wire up.
>