Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [Resend RFC PATCH V2 10/12] HV/IOMMU: Add Hyper-V dma ops support
To:     Christoph Hellwig <hch@lst.de>, konrad.wilk@oracle.com
Cc:     kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com,
        wei.liu@kernel.org, tglx@linutronix.de, mingo@redhat.com,
        bp@alien8.de, x86@kernel.org, hpa@zytor.com, arnd@arndb.de,
        akpm@linux-foundation.org, gregkh@linuxfoundation.org,
        konrad.wilk@oracle.com, m.szyprowski@samsung.com,
        robin.murphy@arm.com, joro@8bytes.org, will@kernel.org,
        davem@davemloft.net, kuba@kernel.org, jejb@linux.ibm.com,
        martin.petersen@oracle.com, Tianyu Lan <Tianyu.Lan@microsoft.com>,
        iommu@lists.linux-foundation.org, linux-arch@vger.kernel.org,
        linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org, linux-scsi@vger.kernel.org,
        netdev@vger.kernel.org, vkuznets@redhat.com,
        thomas.lendacky@amd.com, brijesh.singh@amd.com,
        sunilmut@microsoft.com
References: <20210414144945.3460554-1-ltykernel@gmail.com>
 <20210414144945.3460554-11-ltykernel@gmail.com>
 <20210414154729.GD32045@lst.de>
From:   Tianyu Lan <ltykernel@gmail.com>
Message-ID: <a316af73-2c96-f307-6285-593597e05123@gmail.com>
Date:   Thu, 13 May 2021 00:01:01 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.10.1
MIME-Version: 1.0
In-Reply-To: <20210414154729.GD32045@lst.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Christoph and Konrad:
      Current Swiotlb bounce buffer uses a pool for all devices. There
is a high overhead to get or free bounce buffer during performance test.
Swiotlb code now use a global spin lock to protect bounce buffer data.
Several device queues try to acquire the spin lock and this introduce
additional overhead.

For performance and security perspective, each devices should have a
separate swiotlb bounce buffer pool and so this part needs to rework.
I want to check this is right way to resolve performance issues with 
swiotlb bounce buffer. If you have some other suggestions,welcome.

Thanks.

On 4/14/2021 11:47 PM, Christoph Hellwig wrote:
>> +static dma_addr_t hyperv_map_page(struct device *dev, struct page *page,
>> +				  unsigned long offset, size_t size,
>> +				  enum dma_data_direction dir,
>> +				  unsigned long attrs)
>> +{
>> +	phys_addr_t map, phys = (page_to_pfn(page) << PAGE_SHIFT) + offset;
>> +
>> +	if (!hv_is_isolation_supported())
>> +		return phys;
>> +
>> +	map = swiotlb_tbl_map_single(dev, phys, size, HV_HYP_PAGE_SIZE, dir,
>> +				     attrs);
>> +	if (map == (phys_addr_t)DMA_MAPPING_ERROR)
>> +		return DMA_MAPPING_ERROR;
>> +
>> +	return map;
>> +}
> 
> This largerly duplicates what dma-direct + swiotlb does.  Please use
> force_dma_unencrypted to force bounce buffering and just use the generic
> code.
> 
>> +	if (hv_isolation_type_snp()) {
>> +		ret = hv_set_mem_host_visibility(
>> +				phys_to_virt(hyperv_io_tlb_start),
>> +				hyperv_io_tlb_size,
>> +				VMBUS_PAGE_VISIBLE_READ_WRITE);
>> +		if (ret)
>> +			panic("%s: Fail to mark Hyper-v swiotlb buffer visible to host. err=%d\n",
>> +			      __func__, ret);
>> +
>> +		hyperv_io_tlb_remap = ioremap_cache(hyperv_io_tlb_start
>> +					    + ms_hyperv.shared_gpa_boundary,
>> +						    hyperv_io_tlb_size);
>> +		if (!hyperv_io_tlb_remap)
>> +			panic("%s: Fail to remap io tlb.\n", __func__);
>> +
>> +		memset(hyperv_io_tlb_remap, 0x00, hyperv_io_tlb_size);
>> +		swiotlb_set_bounce_remap(hyperv_io_tlb_remap);
> 
> And this really needs to go into a common hook where we currently just
> call set_memory_decrypted so that all the different schemes for these
> trusted VMs (we have about half a dozen now) can share code rather than
> reinventing it.
>