Received: by 2002:a05:6a11:4021:0:0:0:0 with SMTP id ky33csp368119pxb; Thu, 23 Sep 2021 01:42:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyvH12ckgYS+5erO1s5A5rScNNjaUIsSEk3YhrgEBE/YaRCVrozgYzkexhTW/zgegpGeVnt X-Received: by 2002:a05:6e02:1c2e:: with SMTP id m14mr2751694ilh.23.1632386521970; Thu, 23 Sep 2021 01:42:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1632386521; cv=none; d=google.com; s=arc-20160816; b=MJaJyh7F2zJXWswq/W9cpIEB3d4saW1mXfRc6R2dCg9Yu3JDM5GvY0fMLwuszbadzB dhuSjkenlno3FQoNLBS9AUI2vm6/jIUMNJ1KuQjPHTKjZUdtMBu6iJTlKIHHbEULL4xT GldyySSkCuTo06NTgLryzevN5rGW1AxKRRCnvxKKJdjL9OAhLl6LjAeSoF88UETjDcqn /aQpu0N6NT7YHvZ1qWj+Uf/BsMwnnOTN0mn12qgA3vV2ivs1cZsrCJUdlsd5/hhiC1Dq 1syzEH+SMWxp41FEXe7zgQrsjh5/p7cLJKWY0wXBQB1Fqw4+Sr4fEiVB+hz9nO0GrWF/ H5kQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=Oj0qSIACiFDBVI58JVodM0/ikmXM6SbgZvNtZ/qAF08=; b=G2+WP4/8pTcKhFj9jnPKUWd6LnxxWk6srRAG4tfeX8fVt6ele85CJtvpitOYUlw2MG vRt8VfwyD8lVEhiNLs9uc1dQ4WMS/diws3Rxa4HAxM8TqQNFfn1ti5vaCUrB9QF7Zc7F gpdOkN7X5I+JvvOqj7sQ0tLZmNm9mLBoGsc4shS6aLJ6sGN41Iz68C97pqmxVxU2P2ad +teXM8ZTmM9GyE6lUUN4lmp90HOkwNb7ykQe0gKFmnsJ2iEG/kUMatkoZQoiGiM6i4dc V9xTdHFQaKe+gP/BUPu9dVyglXRfAyfD3pX9AYn9HLDeQZ8ir5y/EO2I3Rwz/iT0Yvxl ZjWQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l15si7290614ios.94.2021.09.23.01.41.50; Thu, 23 Sep 2021 01:42:01 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239986AbhIWImU (ORCPT + 99 others); Thu, 23 Sep 2021 04:42:20 -0400 Received: from mga06.intel.com ([134.134.136.31]:24992 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239825AbhIWImT (ORCPT ); Thu, 23 Sep 2021 04:42:19 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10115"; a="284806416" X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="284806416" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Sep 2021 01:40:48 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,316,1624345200"; d="scan'208";a="558226839" Received: from louislifei-optiplex-7050.sh.intel.com ([10.239.154.151]) by fmsmga002.fm.intel.com with ESMTP; 23 Sep 2021 01:40:46 -0700 From: Fei Li To: gregkh@linuxfoundation.org Cc: linux-kernel@vger.kernel.org, yu1.wang@intel.com, shuox.liu@gmail.com, fei1.li@intel.com Subject: [PATCH v5 1/2] virt: acrn: Introduce interfaces for MMIO device passthrough Date: Thu, 23 Sep 2021 16:41:27 +0800 Message-Id: <20210923084128.18902-2-fei1.li@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210923084128.18902-1-fei1.li@intel.com> References: <20210923084128.18902-1-fei1.li@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Shuo Liu MMIO device passthrough enables an OS in a virtual machine to directly access a MMIO device in the host. It promises almost the native performance, which is required in performance-critical scenarios of ACRN. HSM provides the following ioctls: - Assign - ACRN_IOCTL_ASSIGN_MMIODEV Pass data struct acrn_mmiodev from userspace to the hypervisor, and inform the hypervisor to assign a MMIO device to a User VM. - De-assign - ACRN_IOCTL_DEASSIGN_PCIDEV Pass data struct acrn_mmiodev from userspace to the hypervisor, and inform the hypervisor to de-assign a MMIO device from a User VM. These new APIs will be used by user space code vm_assign_mmiodev and vm_deassign_mmiodev in https://github.com/projectacrn/acrn-hypervisor/blob/master/devicemodel/core/vmmapi.c Signed-off-by: Shuo Liu Signed-off-by: Fei Li --- drivers/virt/acrn/hsm.c | 25 +++++++++++++++++++++++++ drivers/virt/acrn/hypercall.h | 26 ++++++++++++++++++++++++++ include/uapi/linux/acrn.h | 28 ++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+) diff --git a/drivers/virt/acrn/hsm.c b/drivers/virt/acrn/hsm.c index 130e12b8652a..f567ca59d7c2 100644 --- a/drivers/virt/acrn/hsm.c +++ b/drivers/virt/acrn/hsm.c @@ -114,6 +114,7 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, struct acrn_ptdev_irq *irq_info; struct acrn_ioeventfd ioeventfd; struct acrn_vm_memmap memmap; + struct acrn_mmiodev *mmiodev; struct acrn_msi_entry *msi; struct acrn_pcidev *pcidev; struct acrn_irqfd irqfd; @@ -217,6 +218,30 @@ static long acrn_dev_ioctl(struct file *filp, unsigned int cmd, ret = acrn_vm_memseg_unmap(vm, &memmap); break; + case ACRN_IOCTL_ASSIGN_MMIODEV: + mmiodev = memdup_user((void __user *)ioctl_param, + sizeof(struct acrn_mmiodev)); + if (IS_ERR(mmiodev)) + return PTR_ERR(mmiodev); + + ret = hcall_assign_mmiodev(vm->vmid, virt_to_phys(mmiodev)); + if (ret < 0) + dev_dbg(acrn_dev.this_device, + "Failed to assign MMIO device!\n"); + kfree(mmiodev); + break; + case ACRN_IOCTL_DEASSIGN_MMIODEV: + mmiodev = memdup_user((void __user *)ioctl_param, + sizeof(struct acrn_mmiodev)); + if (IS_ERR(mmiodev)) + return PTR_ERR(mmiodev); + + ret = hcall_deassign_mmiodev(vm->vmid, virt_to_phys(mmiodev)); + if (ret < 0) + dev_dbg(acrn_dev.this_device, + "Failed to deassign MMIO device!\n"); + kfree(mmiodev); + break; case ACRN_IOCTL_ASSIGN_PCIDEV: pcidev = memdup_user((void __user *)ioctl_param, sizeof(struct acrn_pcidev)); diff --git a/drivers/virt/acrn/hypercall.h b/drivers/virt/acrn/hypercall.h index 0cfad05bd1a9..f0c78e52cebb 100644 --- a/drivers/virt/acrn/hypercall.h +++ b/drivers/virt/acrn/hypercall.h @@ -41,6 +41,8 @@ #define HC_RESET_PTDEV_INTR _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x04) #define HC_ASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x05) #define HC_DEASSIGN_PCIDEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x06) +#define HC_ASSIGN_MMIODEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x07) +#define HC_DEASSIGN_MMIODEV _HC_ID(HC_ID, HC_ID_PCI_BASE + 0x08) #define HC_ID_PM_BASE 0x80UL #define HC_PM_GET_CPU_STATE _HC_ID(HC_ID, HC_ID_PM_BASE + 0x00) @@ -194,6 +196,30 @@ static inline long hcall_set_memory_regions(u64 regions_pa) return acrn_hypercall1(HC_VM_SET_MEMORY_REGIONS, regions_pa); } +/** + * hcall_assign_mmiodev() - Assign a MMIO device to a User VM + * @vmid: User VM ID + * @addr: Service VM GPA of the &struct acrn_mmiodev + * + * Return: 0 on success, <0 on failure + */ +static inline long hcall_assign_mmiodev(u64 vmid, u64 addr) +{ + return acrn_hypercall2(HC_ASSIGN_MMIODEV, vmid, addr); +} + +/** + * hcall_deassign_mmiodev() - De-assign a PCI device from a User VM + * @vmid: User VM ID + * @addr: Service VM GPA of the &struct acrn_mmiodev + * + * Return: 0 on success, <0 on failure + */ +static inline long hcall_deassign_mmiodev(u64 vmid, u64 addr) +{ + return acrn_hypercall2(HC_DEASSIGN_MMIODEV, vmid, addr); +} + /** * hcall_assign_pcidev() - Assign a PCI device to a User VM * @vmid: User VM ID diff --git a/include/uapi/linux/acrn.h b/include/uapi/linux/acrn.h index 353b2a2e4536..470036d6b1ac 100644 --- a/include/uapi/linux/acrn.h +++ b/include/uapi/linux/acrn.h @@ -396,6 +396,7 @@ struct acrn_ptdev_irq { /* Type of PCI device assignment */ #define ACRN_PTDEV_QUIRK_ASSIGN (1U << 0) +#define ACRN_MMIODEV_RES_NUM 3 #define ACRN_PCI_NUM_BARS 6 /** * struct acrn_pcidev - Info for assigning or de-assigning a PCI device @@ -417,6 +418,29 @@ struct acrn_pcidev { __u32 bar[ACRN_PCI_NUM_BARS]; }; +/** + * struct acrn_mmiodev - Info for assigning or de-assigning a MMIO device + * @name: Name of the MMIO device. + * @res[].user_vm_pa: Physical address of User VM of the MMIO region + * for the MMIO device. + * @res[].service_vm_pa: Physical address of Service VM of the MMIO + * region for the MMIO device. + * @res[].size: Size of the MMIO region for the MMIO device. + * @res[].mem_type: Memory type of the MMIO region for the MMIO + * device. + * + * This structure will be passed to hypervisor directly. + */ +struct acrn_mmiodev { + __u8 name[8]; + struct { + __u64 user_vm_pa; + __u64 service_vm_pa; + __u64 size; + __u64 mem_type; + } res[ACRN_MMIODEV_RES_NUM]; +}; + /** * struct acrn_msi_entry - Info for injecting a MSI interrupt to a VM * @msi_addr: MSI addr[19:12] with dest vCPU ID @@ -568,6 +592,10 @@ struct acrn_irqfd { _IOW(ACRN_IOCTL_TYPE, 0x55, struct acrn_pcidev) #define ACRN_IOCTL_DEASSIGN_PCIDEV \ _IOW(ACRN_IOCTL_TYPE, 0x56, struct acrn_pcidev) +#define ACRN_IOCTL_ASSIGN_MMIODEV \ + _IOW(ACRN_IOCTL_TYPE, 0x57, struct acrn_mmiodev) +#define ACRN_IOCTL_DEASSIGN_MMIODEV \ + _IOW(ACRN_IOCTL_TYPE, 0x58, struct acrn_mmiodev) #define ACRN_IOCTL_PM_GET_CPU_STATE \ _IOWR(ACRN_IOCTL_TYPE, 0x60, __u64) -- 2.17.1