Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH RFC v1 5/6] xen-swiotlb: convert variables to arrays
To:     Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
        Christoph Hellwig <hch@lst.de>, jgross@suse.com
Cc:     Dongli Zhang <dongli.zhang@oracle.com>,
        dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org,
        iommu@lists.linux-foundation.org, linux-mips@vger.kernel.org,
        linux-mmc@vger.kernel.org, linux-pci@vger.kernel.org,
        linuxppc-dev@lists.ozlabs.org, nouveau@lists.freedesktop.org,
        x86@kernel.org, xen-devel@lists.xenproject.org,
        linux-kernel@vger.kernel.org, adrian.hunter@intel.com,
        akpm@linux-foundation.org, benh@kernel.crashing.org,
        bskeggs@redhat.com, bhelgaas@google.com, bp@alien8.de,
        chris@chris-wilson.co.uk, daniel@ffwll.ch, airlied@linux.ie,
        hpa@zytor.com, mingo@kernel.org, mingo@redhat.com,
        jani.nikula@linux.intel.com, joonas.lahtinen@linux.intel.com,
        m.szyprowski@samsung.com, matthew.auld@intel.com,
        mpe@ellerman.id.au, rppt@kernel.org, paulus@samba.org,
        peterz@infradead.org, robin.murphy@arm.com, rodrigo.vivi@intel.com,
        sstabellini@kernel.org, bauerman@linux.ibm.com,
        tsbogend@alpha.franken.de, tglx@linutronix.de,
        ulf.hansson@linaro.org, joe.jin@oracle.com, thomas.lendacky@amd.com
References: <20210203233709.19819-1-dongli.zhang@oracle.com>
 <20210203233709.19819-6-dongli.zhang@oracle.com>
 <20210204084023.GA32328@lst.de> <20210207155601.GA25111@lst.de>
 <YDAgT2ZIdncNwNlf@Konrads-MacBook-Pro.local>
From:   Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <e0baa2fa-0ca4-ef21-aeb0-319d9648e830@oracle.com>
Date:   Fri, 19 Feb 2021 18:59:50 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.7.1
In-Reply-To: <YDAgT2ZIdncNwNlf@Konrads-MacBook-Pro.local>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [10.74.102.113] (138.3.200.49) by SA9PR13CA0183.namprd13.prod.outlook.com (2603:10b6:806:26::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.12 via Frontend Transport; Fri, 19 Feb 2021 23:59:54 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: bb977870-6db9-4847-4298-08d8d5327889
X-MS-TrafficTypeDiagnostic: BYAPR10MB3478:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR10MB347819496D28F67CB66D38608A839@BYAPR10MB3478.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: wHw8Rtx56J3eNRtZBG9OfoZ1cOLDXUf6QDLcCWPioJ5/KOPcUj3PSCBdhUP7BePHPJ6ibTO6BrurKX3Exn4z+Ksng6aTsUlWNbm1G09qeQWyZfSwuhPHQxpmy9/yWDJAzq5qjIrtO4wYs1YIUzcOc4h6YpRoISCpv3l9eDx3QTjzUuhWuRcSlM0N6jXktGmoJ3CvjBdiZmBxzoD3cUFDmHGwsf3xTmMRvfSkZsr4wJej2OvfnF0plIxq0yfiKeiJOm038h6DGy4hz1z3Yn6GUjOUY1NmhKCHKUay95nlQce1Ae1MxZ68x1cMUzsZHGb4HssnfLTgnq5ACwU7dvu7Siyyllr7mYN7d4siXgzqtxBuNAgQZkFsTfDi1KuzKC+zKFJol1Wl18x7y/YEtCcng/cbeyFT9wO18tJ0IEEKveNT+HLyit2yadfNjZIV7kguyyoK3jgNePYYM4paLSijNM8AIEjD18LwhZ5vmILZrhXotVvTB7NWCNpYeBRs3dUjFJLwU/OV7PIfnPmvBFAB3VyZsgeB7NmvIBJZ9YsHqDmLV/A+F/cISJl2SraKa3/0/D0ni6tVIQi+OR5Q8wfZg5RFzwOQejne37B3sjBHnFU=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3288.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(136003)(346002)(39860400002)(396003)(376002)(6486002)(186003)(86362001)(53546011)(36756003)(2906002)(8936002)(66476007)(16526019)(31686004)(7416002)(478600001)(316002)(5660300002)(8676002)(26005)(31696002)(66946007)(2616005)(44832011)(7406005)(16576012)(6666004)(110136005)(956004)(66556008)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?QVNCUUpBb2NkS1p2blZLb3hFcGxJUkI3NHQvSElScElQcUNXNTVVcWErT2hQ?=
 =?utf-8?B?ckZScGRYT3UvRWM1czFuVjgrVDViTVVyY0FQQmpqNCt5eVJEMUtNcDkxYjFD?=
 =?utf-8?B?L28xVWtrRUQ2YXYrcGw3cVBIVm1CbmQ2aS9DeDJSWnBPeVc1aXN2S3FIVkdP?=
 =?utf-8?B?WmV6TkNuTjdUazdKR0hOUStHKzFWajlZWmFBdisvRnFRTmR6RVIxVGQraFpQ?=
 =?utf-8?B?T2dPUkpEY3dvMFBFT2FqOG5HWUZOR20wYnkxQ0s3cVlsMEN6OXpGaDUxTW8v?=
 =?utf-8?B?Rm54WURzTDh4RWJjb2hUWlpibkZrdDFyc29OQjduRDhwK1NKalk5bkMvWXFl?=
 =?utf-8?B?SVllM3ROWVcyZkZvNXNRd1BEM2pobm5jWUNhOWl3VC9JOFFESi82YzhLeFN0?=
 =?utf-8?B?TjVxcVJtTkhjSm45akN0TW90UU5kU2ZLQ0pXUDlJRTQ1MFAzbno5ajQ3eWxt?=
 =?utf-8?B?SlUyUWhZTDZCOTZLSEw1anJ1T0lVMmVZU2txMTdQbXI0NWhsbE82RnhobXMv?=
 =?utf-8?B?VHF3S2tESy8rYmhlSWh6OTZhN1VjUDRKbHcyckJpckVFRjdXcHVoanVnZkJy?=
 =?utf-8?B?VHdPZmczc05abzEwMkxrdW4wS1NqRTFwbzJ4L082OVZmbVp5VTk2aDFOQURJ?=
 =?utf-8?B?RWZOKzNMWnRDNGpxdHM0YkpCVjlwQzZ2d1FZcWtFYWVqR3plMzU1U0ZtSWpZ?=
 =?utf-8?B?YjFxUzhDcmZIYUJSL2JOR25scGZkM1dVWGpVLzdsMEdvN2hwMzlXVmlDa01O?=
 =?utf-8?B?QVErY3VCa3J0bzBFMThmN2xzWEtWb3VqTHFVcTdBUmlBeWE3ZWJndDhhTkUv?=
 =?utf-8?B?dE12QnNPUGhUZWVHZ01TZzdJamhBZEo1UW44NGhhM1NTUkhjbkdEeitpeHIw?=
 =?utf-8?B?d2pjRStpNFFjSFlaeUFQbWl3TUZEZWlNZ3lGaE5razNaY2lqZ2JIUCtTSUpM?=
 =?utf-8?B?YUNyRUtySjg5eVhxYzNDZkV5V3o5N3hnUUdFN3pybzYrRnRhUDdSSGxUb2x5?=
 =?utf-8?B?NFNoQnFuMVkxRk1xN3EvT0pwRUJEcTRHbmZCdUVOamRlbWNjTHJoaldTMExn?=
 =?utf-8?B?RW9kcVhyQU9qV3dPbG14S2ZwMjVHQlZzak1PU0N1RHpGcHNWYjJNRFU1TERE?=
 =?utf-8?B?K0RadlpBaGpUVklYZW9iZUl0TVpCQlpUMzFsdmU4S0tHRWxVcEphQWI1MG1i?=
 =?utf-8?B?aURDSXgwbzZBR0U0aHM3am5kZzZHL0U1OWhJVjlvem5CQ3VNR3ZRRU1YQWRJ?=
 =?utf-8?B?MkxTdlBXNkU4QWMyZnBNNFIvZzNhSHU1MGRmV3RJK2lmVU5nYTNOTmgwMHJ5?=
 =?utf-8?B?YVpKVkVXNTRQRTkzZVB3Z1pXZEVjQ3FzMDcramdETkZveXZrNjB3ODV3V0dN?=
 =?utf-8?B?Y0ZhcTF6YXZmbHFrcDlNZjdEYVBGbjA1bzNmZ050R3FPbTVIYVllY0g3VFVj?=
 =?utf-8?B?NWRJbm9tOEhmVStJQ3hzWWlyclJQQlZwVzBlYkVvVWN4RFMrc0d4SUJGamlK?=
 =?utf-8?B?S2hEb05WejZmclpKd1pvOUtXeHVteHEzNUZkS1ZOWTY3a0Rzc0dKdHNZMzRl?=
 =?utf-8?B?WUlWRVhQWEMzQ1dNUkkzQzVVOHlpeTEzbWo0Z1lYNThxSkRDUTBkajdZM3VQ?=
 =?utf-8?B?OXVkUVI0eVNTYkZqeDV2cU9ObDBEeEpoaExaSmQ1YjRhN25lZm5NMXpZSURl?=
 =?utf-8?B?S2FsVGZCS3J0YU1aZ3c3NmgvWE5rb09qNDBaeVo1Wk43NXArbU9GZFhTeEhz?=
 =?utf-8?Q?n2oelDtJvZWoXjrMy4S1QIGAPbxs5OfZ1Dx/wvW?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bb977870-6db9-4847-4298-08d8d5327889
X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3288.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2021 00:00:02.3805
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Des4+MdYQUm56M8MWeOPJBdTKtHtZDWQxPVWdvf+t3tq7lw1L/455wkojdxiIP+TV14ml/NWhpHwXjt94wgy9iCuufxjtuBu/+w3PAZeb30=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3478
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9900 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 mlxscore=0
 phishscore=0 adultscore=0 bulkscore=0 mlxlogscore=999 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2102190196
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9900 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 lowpriorityscore=0 suspectscore=0
 impostorscore=0 priorityscore=1501 clxscore=1011 spamscore=0 mlxscore=0
 phishscore=0 malwarescore=0 bulkscore=0 adultscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2102190196
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/19/21 3:32 PM, Konrad Rzeszutek Wilk wrote:
> On Sun, Feb 07, 2021 at 04:56:01PM +0100, Christoph Hellwig wrote:
>> On Thu, Feb 04, 2021 at 09:40:23AM +0100, Christoph Hellwig wrote:
>>> So one thing that has been on my mind for a while:  I'd really like
>>> to kill the separate dma ops in Xen swiotlb.  If we compare xen-swiotlb
>>> to swiotlb the main difference seems to be:
>>>
>>>  - additional reasons to bounce I/O vs the plain DMA capable
>>>  - the possibility to do a hypercall on arm/arm64
>>>  - an extra translation layer before doing the phys_to_dma and vice
>>>    versa
>>>  - an special memory allocator
>>>
>>> I wonder if inbetween a few jump labels or other no overhead enablement
>>> options and possibly better use of the dma_range_map we could kill
>>> off most of swiotlb-xen instead of maintaining all this code duplication?
>> So I looked at this a bit more.
>>
>> For x86 with XENFEAT_auto_translated_physmap (how common is that?)
> Juergen, Boris please correct me if I am wrong, but that XENFEAT_auto_translated_physmap
> only works for PVH guests?


That's both HVM and PVH (for dom0 it's only PVH).


-boris


>
>> pfn_to_gfn is a nop, so plain phys_to_dma/dma_to_phys do work as-is.
>>
>> xen_arch_need_swiotlb always returns true for x86, and
>> range_straddles_page_boundary should never be true for the
>> XENFEAT_auto_translated_physmap case.
> Correct. The kernel should have no clue of what the real MFNs are
> for PFNs.
>> So as far as I can tell the mapping fast path for the
>> XENFEAT_auto_translated_physmap can be trivially reused from swiotlb.
>>
>> That leaves us with the next more complicated case, x86 or fully cache
>> coherent arm{,64} without XENFEAT_auto_translated_physmap.  In that case
>> we need to patch in a phys_to_dma/dma_to_phys that performs the MFN
>> lookup, which could be done using alternatives or jump labels.
>> I think if that is done right we should also be able to let that cover
>> the foreign pages in is_xen_swiotlb_buffer/is_swiotlb_buffer, but
>> in that worst case that would need another alternative / jump label.
>>
>> For non-coherent arm{,64} we'd also need to use alternatives or jump
>> labels to for the cache maintainance ops, but that isn't a hard problem
>> either.
>>
>>