Received: by 2002:a25:31c3:0:0:0:0:0 with SMTP id x186csp1461237ybx; Thu, 31 Oct 2019 10:55:02 -0700 (PDT) X-Google-Smtp-Source: APXvYqylfhEXPY1vf+FsNcXPKTOzhvnL7Hl79wEdvJYMVXxFHIIFv34/xGoxETFn2scCzQmwoCQ6 X-Received: by 2002:a17:906:4ec9:: with SMTP id i9mr5423450ejv.8.1572544502379; Thu, 31 Oct 2019 10:55:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572544502; cv=none; d=google.com; s=arc-20160816; b=ua7xm5klt/gc3VUQX+4yPfWOsi73cHqLmt8rbeqNUs3Lr5I2DA8vkGrnuZI6jEp8qB jlRF4XxBu2nwRiUA4ipq1d07ADTAtMOSfuvKQVjIySP8HtfIMZOpwS0NC7ulRv1A3Mn6 qgJEK3ZxMixCqjH+2l89RCCbYj57wsPGb2+7kM97piAsi9hsBu3FpTpEITGv4NRSvwrG bSQAoc+7J1S5EY3fCUs5g5idOXRcy/vErIVkXnIMm++E8uQmAtbbhMRt25nbsQDaVaeI jkCgNNIWGWWi8UyU3a2vixzvJ4ttOF4yIGtHbYer/956Yi3HMln1N1Ce6l+a76QOEGmK 7AJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :organization:references:in-reply-to:message-id:subject:cc:to:from :date; bh=00JVwNeW86D1AbCkGHU0MNxh0X8yu+W0lL4GeAnlpNg=; b=U4p1NY0Pm7kJ4cTj377iYz7wNT5aA03XY4FSzwvDbV+XdU4J7L2T7kRPfLw5YcUI06 YN9NqP0NiRGALiROi++g/Hb+c9zKqE19EHyb3NPkPJzaYR2+HU3aqhmPDYc3OIex5d2a M8EwA/6s9tpxoC06x9Rd9ce0S1k+s15E74R0y7CtAgU9pWadPV2Juats+JHCaHpC7lM4 UCxW1MutPTfehBnJVlNHTO0AUpOHtS/iHBi+olmBb1wd+GQoJaTuFMqkYZwlq86Vc7KR +v/t0JtRTmN52Is/vE/HujePeup8Yl+ina+lfxQt40RLraZy9NGHwsXMqHfOjs7UWL9u uGXQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id cf19si3871803ejb.14.2019.10.31.10.54.37; Thu, 31 Oct 2019 10:55:02 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729117AbfJaRyB (ORCPT + 99 others); Thu, 31 Oct 2019 13:54:01 -0400 Received: from szxga07-in.huawei.com ([45.249.212.35]:51318 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726602AbfJaRyA (ORCPT ); Thu, 31 Oct 2019 13:54:00 -0400 Received: from DGGEMS405-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 9F120B745E310E6913F0; Fri, 1 Nov 2019 01:53:58 +0800 (CST) Received: from localhost (10.202.226.61) by DGGEMS405-HUB.china.huawei.com (10.3.19.205) with Microsoft SMTP Server id 14.3.439.0; Fri, 1 Nov 2019 01:53:51 +0800 Date: Thu, 31 Oct 2019 15:37:11 +0000 From: Jonathan Cameron To: Zhangfei Gao CC: Greg Kroah-Hartman , Arnd Bergmann , Herbert Xu , , jean-philippe , "Jerome Glisse" , , , , Wangzhou , "haojian . zhuang" , , , , , , Kenneth Lee , Zaibo Xu Subject: Re: [PATCH v7 1/3] uacce: Add documents for uacce Message-ID: <20191031153711.00001708@huawei.com> In-Reply-To: <1572331216-9503-2-git-send-email-zhangfei.gao@linaro.org> References: <1572331216-9503-1-git-send-email-zhangfei.gao@linaro.org> <1572331216-9503-2-git-send-email-zhangfei.gao@linaro.org> Organization: Huawei X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; i686-w64-mingw32) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.202.226.61] X-CFilter-Loop: Reflected Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org On Tue, 29 Oct 2019 14:40:14 +0800 Zhangfei Gao wrote: > From: Kenneth Lee > > Uacce (Unified/User-space-access-intended Accelerator Framework) is > a kernel module targets to provide Shared Virtual Addressing (SVA) > between the accelerator and process. > > This patch add document to explain how it works. > > Signed-off-by: Kenneth Lee > Signed-off-by: Zaibo Xu > Signed-off-by: Zhou Wang > Signed-off-by: Zhangfei Gao > --- > Documentation/misc-devices/uacce.rst | 160 +++++++++++++++++++++++++++++++++++ > 1 file changed, 160 insertions(+) > create mode 100644 Documentation/misc-devices/uacce.rst > > diff --git a/Documentation/misc-devices/uacce.rst b/Documentation/misc-devices/uacce.rst > new file mode 100644 > index 0000000..ecd5d8b > --- /dev/null > +++ b/Documentation/misc-devices/uacce.rst > @@ -0,0 +1,160 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +Introduction of Uacce > +========================= Fix the underline length to match the title. > + > +Uacce (Unified/User-space-access-intended Accelerator Framework) targets to > +provide Shared Virtual Addressing (SVA) between accelerators and processes. > +So accelerator can access any data structure of the main cpu. > +This differs from the data sharing between cpu and io device, which share > +data content rather than address. which share only data content rather than address. > +Because of the unified address, hardware and user space of process can > +share the same virtual address in the communication. > +Uacce takes the hardware accelerator as a heterogeneous processor, while > +IOMMU share the same CPU page tables and as a result the same translation > +from va to pa. > + > + __________________________ __________________________ > + | | | | > + | User application (CPU) | | Hardware Accelerator | > + |__________________________| |__________________________| > + > + | | > + | va | va > + V V > + __________ __________ > + | | | | > + | MMU | | IOMMU | > + |__________| |__________| > + | | > + | | > + V pa V pa > + _______________________________________ > + | | > + | Memory | > + |_______________________________________| > + > + > + > +Architecture > +------------ > + > +Uacce is the kernel module, taking charge of iommu and address sharing. > +The user drivers and libraries are called WarpDrive. > + > +The uacce device, built around the IOMMU SVA API, can access multiple > +address spaces, including the one without PASID. > + > +A virtual concept, queue, is used for the communication. It provides a > +FIFO-like interface. And it maintains a unified address space between the > +application and all involved hardware. > + > + ___________________ ________________ > + | | user API | | > + | WarpDrive library | ------------> | user driver | > + |___________________| |________________| > + | | > + | | > + | queue fd | > + | | > + | | > + v | > + ___________________ _________ | > + | | | | | mmap memory > + | Other framework | | uacce | | r/w interface > + | crypto/nic/others | |_________| | > + |___________________| | > + | | | > + | register | register | > + | | | > + | | | > + | _________________ __________ | > + | | | | | | > + ------------- | Device Driver | | IOMMU | | > + |_________________| |__________| | > + | | > + | V > + | ___________________ > + | | | > + -------------------------- | Device(Hardware) | > + |___________________| > + > + > +How does it work > +================ > + > +Uacce uses mmap and IOMMU to play the trick. > + > +Uacce create a chrdev for every device registered to it. New queue is > +created when user application open the chrdev. The file descriptor is used > +as the user handle of the queue. > +The accelerator device present itself as an Uacce object, which exports as > +chrdev to the user space. The user application communicates with the > +hardware by ioctl (as control path) or share memory (as data path). > + > +The control path to the hardware is via file operation, while data path is > +via mmap space of the queue fd. > + > +The queue file address space: > +/** > + * enum uacce_qfrt: qfrt type > + * @UACCE_QFRT_MMIO: device mmio region > + * @UACCE_QFRT_DUS: device user share region > + */ > +enum uacce_qfrt { > + UACCE_QFRT_MMIO = 0, > + UACCE_QFRT_DUS = 1, > +}; > + > +All regions are optional and differ from device type to type. The > +communication protocol is wrapped by the user driver. Can you have more than one of each? If not state this clearly. > + > +The device mmio region is mapped to the hardware mmio space. It is generally > +used for doorbell or other notification to the hardware. It is not fast enough > +as data channel. > + > +The device user share region is used for share data buffer between user process > +and device. > + > + > +The Uacce register API > +----------------------- blank line here. > +The register API is defined in uacce.h. > + > +struct uacce_interface { > + char name[UACCE_MAX_NAME_SIZE]; > + enum uacce_dev_flag flags; Having an enum for a flags variable / bitmap is odd. Just use an appropriate sized integer. > + struct uacce_ops *ops; > +}; Use indenting to get literal formatting of this block. Will give formatting like the code blocks in here compiled: https://www.kernel.org/doc/html/latest/driver-api/iio/buffers.html > + > +According to the IOMMU capability, uacce_interface flags can be: > + > +/** > + * enum uacce_dev_flag: Device flags: > + * @UACCE_DEV_SVA: Shared Virtual Addresses > + * Support PASID > + * Support device page faults (PCI PRI or SMMU Stall) > + */ > +enum uacce_dev_flag { > + UACCE_DEV_SVA = BIT(0), > +}; > + > +struct uacce_device *uacce_register(struct device *parent, > + struct uacce_interface *interface); > +void uacce_unregister(struct uacce_device *uacce); > + > +uacce_register results can be: > +a. If uacce module is not compiled, ERR_PTR(-ENODEV) > +b. Succeed with the desired flags > +c. Succeed with the negotiated flags, for example > + uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA > +So user driver need check return value as well as the negotiated uacce->flags. > + > + > +The user driver > +--------------- > + > +The queue file mmap space will need a user driver to wrap the communication > +protocol. Uacce provides some attributes in sysfs for the user driver to > +match the right accelerator accordingly. > +More details in Documentation/ABI/testing/sysfs-driver-uacce.