Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp640209pxb; Wed, 27 Jan 2021 17:42:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJzH8aq2F4/LO60pKm7pOXyPq6vC3ThUlaf5Fhuz2z4087moXoQ2cuMVwAf/6xmmMjle2NW/ X-Received: by 2002:a17:906:17d3:: with SMTP id u19mr9531986eje.316.1611798146881; Wed, 27 Jan 2021 17:42:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611798146; cv=none; d=google.com; s=arc-20160816; b=uJHuUzzEhyVayDbfAC9JSMVEI4QGBcGzWcdGunWOtuvolVuiCXR8HLyyPvOSuKHa9C 4lVpELTj4Ku2//GsJ5zWqBSSceteS5TT5pIcn8Ip7pv/lj27QWP9M6qisF8FkbVSVU5M V4qd2cCL0QFAdl6KAhNbQMEaXjniu+fgPXhVcKo+qshp5PWCatSt7jj0lnGUGPQ3/SWq fmqW7oaFnyxo3otgxMOEpS5IYR1Qr8gj/f1Dx1gKiCER5tw56osOXGrzQ/L1oIJjE4E+ RCYqNfQZWvKfd7Q7ohfHB3zmgYGN+IfnQ0qukOrTRMEPJ/oYHX3+Vs/cX0uyO1vNjPVw /p7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :content-language:accept-language:in-reply-to:references:message-id :date:thread-index:thread-topic:subject:cc:to:from; bh=Fg7LYwHeVJHTpWsRrTzw5nPqVu3NGeQ6t0y+qnx0cEE=; b=YfBIBKA8UClnylkZnvVHtriPJDp6DdnPeOy7DQLtILcNQvyfPF+YCcRi7qG5NpqTR+ QapE0Rfe0RKSfYs8PRS6I74MnOuFB5LozBSHctrlKe4SOV6g1L/Faze+oh7+L6LnOhsu VpOe0DmUrpKMvVM5SouM2YnUulLcJseDQcDDdTjfSVkQS45S5hs5DaJ6Nq/8d0InuxCT ilcuCgtd4Ox6Mo+1w9ja1w9C1iCc99VqDw3GJipwsyEAoKHlRSRT+urg6eIrw/F0u1BW 5MgcxMbI1w6ImyIcG5s18bHgoJP5jLtFNuD+8f8dT42GID0xOjwkMIESvCO+QEM9n6Vw uNqA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id be1si1958882edb.481.2021.01.27.17.42.02; Wed, 27 Jan 2021 17:42:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229808AbhA1BaL convert rfc822-to-8bit (ORCPT + 99 others); Wed, 27 Jan 2021 20:30:11 -0500 Received: from szxga03-in.huawei.com ([45.249.212.189]:2883 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229757AbhA1B3y (ORCPT ); Wed, 27 Jan 2021 20:29:54 -0500 Received: from DGGEMM403-HUB.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4DR2rv3Fv3z5LbD; Thu, 28 Jan 2021 09:27:27 +0800 (CST) Received: from dggpemm100010.china.huawei.com (7.185.36.24) by DGGEMM403-HUB.china.huawei.com (10.3.20.211) with Microsoft SMTP Server (TLS) id 14.3.498.0; Thu, 28 Jan 2021 09:28:27 +0800 Received: from dggemi761-chm.china.huawei.com (10.1.198.147) by dggpemm100010.china.huawei.com (7.185.36.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2106.2; Thu, 28 Jan 2021 09:28:27 +0800 Received: from dggemi761-chm.china.huawei.com ([10.9.49.202]) by dggemi761-chm.china.huawei.com ([10.9.49.202]) with mapi id 15.01.2106.006; Thu, 28 Jan 2021 09:28:27 +0800 From: "Song Bao Hua (Barry Song)" To: Jason Gunthorpe CC: "Wangzhou (B)" , Greg Kroah-Hartman , Arnd Bergmann , Zhangfei Gao , "linux-accelerators@lists.ozlabs.org" , "linux-kernel@vger.kernel.org" , "iommu@lists.linux-foundation.org" , "linux-mm@kvack.org" , "Liguozhu (Kenneth)" , "chensihang (A)" Subject: RE: [RFC PATCH v2] uacce: Add uacce_ctrl misc device Thread-Topic: [RFC PATCH v2] uacce: Add uacce_ctrl misc device Thread-Index: AQHW8vWniUnMS+RFOU2UJJCa8sDlvKo39q+AgADtcaD//5AFgIAAh9tQ//+YwwCAAIUv4IAAmdoAgAKKrfA= Date: Thu, 28 Jan 2021 01:28:26 +0000 Message-ID: References: <1611563696-235269-1-git-send-email-wangzhou1@hisilicon.com> <20210125154717.GW4605@ziepe.ca> <96b655ade2534a65974a378bb68383ee@hisilicon.com> <20210125231619.GY4605@ziepe.ca> <5f64a68042c64f37b5cba74028bd2189@hisilicon.com> <20210126011304.GZ4605@ziepe.ca> <20210126182024.GA4605@ziepe.ca> In-Reply-To: <20210126182024.GA4605@ziepe.ca> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.126.200.149] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 8BIT MIME-Version: 1.0 X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > -----Original Message----- > From: Jason Gunthorpe [mailto:jgg@ziepe.ca] > Sent: Wednesday, January 27, 2021 7:20 AM > To: Song Bao Hua (Barry Song) > Cc: Wangzhou (B) ; Greg Kroah-Hartman > ; Arnd Bergmann ; Zhangfei Gao > ; linux-accelerators@lists.ozlabs.org; > linux-kernel@vger.kernel.org; iommu@lists.linux-foundation.org; > linux-mm@kvack.org; Liguozhu (Kenneth) ; chensihang > (A) > Subject: Re: [RFC PATCH v2] uacce: Add uacce_ctrl misc device > > On Tue, Jan 26, 2021 at 01:26:45AM +0000, Song Bao Hua (Barry Song) wrote: > > > On Mon, Jan 25, 2021 at 11:35:22PM +0000, Song Bao Hua (Barry Song) wrote: > > > > > > > > On Mon, Jan 25, 2021 at 10:21:14PM +0000, Song Bao Hua (Barry Song) > wrote: > > > > > > mlock, while certainly be able to prevent swapping out, it won't > > > > > > be able to stop page moving due to: > > > > > > * memory compaction in alloc_pages() > > > > > > * making huge pages > > > > > > * numa balance > > > > > > * memory compaction in CMA > > > > > > > > > > Enabling those things is a major reason to have SVA device in the > > > > > first place, providing a SW API to turn it all off seems like the > > > > > wrong direction. > > > > > > > > I wouldn't say this is a major reason to have SVA. If we read the > > > > history of SVA and papers, people would think easy programming due > > > > to data struct sharing between cpu and device, and process space > > > > isolation in device would be the major reasons for SVA. SVA also > > > > declares it supports zero-copy while zero-copy doesn't necessarily > > > > depend on SVA. > > > > > > Once you have to explicitly make system calls to declare memory under > > > IO, you loose all of that. > > > > > > Since you've asked the app to be explicit about the DMAs it intends to > > > do, there is not really much reason to use SVA for those DMAs anymore. > > > > Let's see a non-SVA case. We are not using SVA, we can have > > a memory pool by hugetlb or pin, and app can allocate memory > > from this pool, and get stable I/O performance on the memory > > from the pool. But device has its separate page table which > > is not bound with this process, thus lacking the protection > > of process space isolation. Plus, CPU and device are using > > different address. > > So you are relying on the platform to do the SVA for the device? > Sorry for late response. uacce and its userspace framework UADK depend on SVA, leveraging the enhanced security by isolated process address space. This patch is mainly an extension for performance optimization to get stable high-performance I/O on pinned memory even though the hardware supports IO page fault to get pages back after swapping out or page migration. But IO page fault will cause serious latency jitter for high-speed I/O. For slow speed device, they don't need to use this extension. > This feels like it goes back to another topic where I felt the SVA > setup uAPI should be shared and not buried into every driver's unique > ioctls. > > Having something like this in a shared SVA system is somewhat less > strange. Sounds reasonable. On the other hand, uacce seems to be an common uAPI for SVA, and probably the only one for this moment. uacce is a framework not a specific driver as any accelerators can hook into this framework as long as a device provides uacce_ops and register itself by uacce_register(). Uacce, for itself, doesn't bind with any specific hardware. So uacce interfaces are kind of common uAPI :-) > > Jason Thanks Barry