Received: by 2002:ac0:aa62:0:0:0:0:0 with SMTP id w31-v6csp120453ima; Tue, 23 Oct 2018 21:01:03 -0700 (PDT) X-Google-Smtp-Source: AJdET5etHaJDW3OODIyq2lCZil77kDX45TjJruE9aWHBKB/WLojNl1xhUa/+w2rUdcvUWkgGD//D X-Received: by 2002:a17:902:728e:: with SMTP id d14-v6mr1000860pll.51.1540353663115; Tue, 23 Oct 2018 21:01:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540353663; cv=none; d=google.com; s=arc-20160816; b=wVxRRsoFq/LXXEi88MfiVsYjV7Uja95WJ+CAlniu3rbm5zp7q9eQzxmcV24annD1aK ezhTztJ/Jiws42XfOY/K3Yw3yhUZwzfzg686mR64LqD4EjZHi29Ma2TBH0070c2yTdo/ C4pbgIIjLcZUwHBWuLnw8VcfqluTi+WVHwL+xBsk54VsPwekaVF6YOwG30jjqp15e3qU bOhz7inkfzDH2zTteNWi00htL04fxU3dIbzKmVhLojVQjJ/4DxE9nG3C05mokEuGZNlH sKKUeUgFzWmqEkydV4YFQWPb4a41F+hX+oeY07vpmoiv6u3z0anbjJTCeJr9OO0rc+Yn vAew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=p152RtTqh2p4w3Wc3yKn0DYOZ1chRQT20N+0UTTGkaM=; b=LiyMzv1Alr2wSNDdph0WyvbYCUBXeR7wuCifMplPnqTx6OL58xAP6Jrsxte06cDI8m yyC7Ihdg2JrfAMJ1tHzYKscVvsa0eUeRwmQNOkCOHxW3t1RN9d9I/q+wAMLahZIVmpca C/3kUqHiEt0/B9AnSybRvAS6aef8uTHIVE7Tr1rEhhjZCgjnc1z/zDXh3LcJPSfirXlq qGE9iz11FpqStZJQrKv0bvIogn4M2kga/afigd5z6zcoIXdi9tRLDjBlmFApnHpcRiml 6co0jKPWx/tHlPtiKyk06i5VA7+ZyKZbYGT+uTL4nGtYge7Zie2+nq1VPtZDbmpJpgQR bf3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=FxxMJ3M7; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q3-v6si3442358plb.355.2018.10.23.21.00.47; Tue, 23 Oct 2018 21:01:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@ti.com header.s=ti-com-17Q1 header.b=FxxMJ3M7; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=NONE dis=NONE) header.from=ti.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726570AbeJXM0I (ORCPT + 99 others); Wed, 24 Oct 2018 08:26:08 -0400 Received: from lelv0142.ext.ti.com ([198.47.23.249]:42208 "EHLO lelv0142.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725928AbeJXM0H (ORCPT ); Wed, 24 Oct 2018 08:26:07 -0400 Received: from dlelxv90.itg.ti.com ([172.17.2.17]) by lelv0142.ext.ti.com (8.15.2/8.15.2) with ESMTP id w9O1RIWF073421; Tue, 23 Oct 2018 20:27:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1540344438; bh=p152RtTqh2p4w3Wc3yKn0DYOZ1chRQT20N+0UTTGkaM=; h=Subject:To:CC:References:From:Date:In-Reply-To; b=FxxMJ3M7wIyHG/c94d6o/wg7KXUe/JLx7qqNhSoI7uNc2oBoPqLBDAQuHbq+xOg0S uKtLmLRB2st6rgl46iBCwOrT542lMAkf7y7kU1i9jAK+f+X3TCY1kQrQ8L92AcqFok jzuzG/YeHHHD7JdOxXejE89kYejakn04d+K/x/V8= Received: from DFLE100.ent.ti.com (dfle100.ent.ti.com [10.64.6.21]) by dlelxv90.itg.ti.com (8.14.3/8.13.8) with ESMTP id w9O1RIPO014613; Tue, 23 Oct 2018 20:27:18 -0500 Received: from DFLE100.ent.ti.com (10.64.6.21) by DFLE100.ent.ti.com (10.64.6.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Tue, 23 Oct 2018 20:27:18 -0500 Received: from dlep33.itg.ti.com (157.170.170.75) by DFLE100.ent.ti.com (10.64.6.21) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_RSA_WITH_AES_256_CBC_SHA) id 15.1.1466.3 via Frontend Transport; Tue, 23 Oct 2018 20:27:18 -0500 Received: from [128.247.58.153] (ileax41-snat.itg.ti.com [10.172.224.153]) by dlep33.itg.ti.com (8.14.3/8.13.8) with ESMTP id w9O1RIsL026910; Tue, 23 Oct 2018 20:27:18 -0500 Subject: Re: [PATCH v4 13/17] remoteproc: create vdev subdevice with specific dma memory pool To: Loic PALLARDY , Bjorn Andersson CC: "ohad@wizery.com" , "linux-remoteproc@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Arnaud POULIQUEN , "benjamin.gaignard@linaro.org" , Anup Patel References: <1532697292-14272-1-git-send-email-loic.pallardy@st.com> <1532697292-14272-14-git-send-email-loic.pallardy@st.com> <20181010055823.GB20016@builder> <54cd805121f34c15a64e283bf93e5737@SFHDAG7NODE2.st.com> From: Suman Anna Message-ID: Date: Tue, 23 Oct 2018 20:27:18 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <54cd805121f34c15a64e283bf93e5737@SFHDAG7NODE2.st.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/10/18 2:17 PM, Loic PALLARDY wrote: > > >> -----Original Message----- >> From: Bjorn Andersson [mailto:bjorn.andersson@linaro.org] >> Sent: mercredi 10 octobre 2018 07:58 >> To: Loic PALLARDY >> Cc: ohad@wizery.com; linux-remoteproc@vger.kernel.org; linux- >> kernel@vger.kernel.org; Arnaud POULIQUEN ; >> benjamin.gaignard@linaro.org; s-anna@ti.com >> Subject: Re: [PATCH v4 13/17] remoteproc: create vdev subdevice with >> specific dma memory pool >> >> On Fri 27 Jul 06:14 PDT 2018, Loic Pallardy wrote: >> >>> This patch creates a dedicated vdev subdevice for each vdev declared >>> in firmware resource table and associates carveout named "vdev%dbuffer" >>> (with %d vdev index in resource table) if any as dma coherent memory >> pool. >>> >>> Then vdev subdevice is used as parent for virtio device. >>> >>> Signed-off-by: Loic Pallardy >>> --- >>> drivers/remoteproc/remoteproc_core.c | 35 >> +++++++++++++++++++++++--- >>> drivers/remoteproc/remoteproc_internal.h | 1 + >>> drivers/remoteproc/remoteproc_virtio.c | 42 >> +++++++++++++++++++++++++++++++- >>> include/linux/remoteproc.h | 1 + >>> 4 files changed, 75 insertions(+), 4 deletions(-) >>> >>> diff --git a/drivers/remoteproc/remoteproc_core.c >> b/drivers/remoteproc/remoteproc_core.c >>> index 4edc6f0..adcc66e 100644 >>> --- a/drivers/remoteproc/remoteproc_core.c >>> +++ b/drivers/remoteproc/remoteproc_core.c >>> @@ -39,6 +39,7 @@ >>> #include >>> #include >>> #include >>> +#include >>> #include >>> #include >>> #include >>> @@ -145,7 +146,7 @@ static void rproc_disable_iommu(struct rproc >> *rproc) >>> iommu_domain_free(domain); >>> } >>> >>> -static phys_addr_t rproc_va_to_pa(void *cpu_addr) >>> +phys_addr_t rproc_va_to_pa(void *cpu_addr) >>> { >>> /* >>> * Return physical address according to virtual address location >>> @@ -160,6 +161,7 @@ static phys_addr_t rproc_va_to_pa(void >> *cpu_addr) >>> WARN_ON(!virt_addr_valid(cpu_addr)); >>> return virt_to_phys(cpu_addr); >>> } >>> +EXPORT_SYMBOL(rproc_va_to_pa); >>> >>> /** >>> * rproc_da_to_va() - lookup the kernel virtual address for a remoteproc >> address >>> @@ -423,6 +425,20 @@ static void rproc_vdev_do_stop(struct >> rproc_subdev *subdev, bool crashed) >>> } >>> >>> /** >>> + * rproc_rvdev_release() - release the existence of a rvdev >>> + * >>> + * @dev: the subdevice's dev >>> + */ >>> +static void rproc_rvdev_release(struct device *dev) >>> +{ >>> + struct rproc_vdev *rvdev = container_of(dev, struct rproc_vdev, >> dev); >>> + >>> + of_reserved_mem_device_release(dev); >>> + >>> + kfree(rvdev); >>> +} >>> + >>> +/** >>> * rproc_handle_vdev() - handle a vdev fw resource >>> * @rproc: the remote processor >>> * @rsc: the vring resource descriptor >>> @@ -455,6 +471,7 @@ static int rproc_handle_vdev(struct rproc *rproc, >> struct fw_rsc_vdev *rsc, >>> struct device *dev = &rproc->dev; >>> struct rproc_vdev *rvdev; >>> int i, ret; >>> + char name[16]; >>> >>> /* make sure resource isn't truncated */ >>> if (sizeof(*rsc) + rsc->num_of_vrings * sizeof(struct >> fw_rsc_vdev_vring) >>> @@ -488,6 +505,18 @@ static int rproc_handle_vdev(struct rproc *rproc, >> struct fw_rsc_vdev *rsc, >>> rvdev->rproc = rproc; >>> rvdev->index = rproc->nb_vdev++; >>> >>> + /* Initialise vdev subdevice */ >>> + snprintf(name, sizeof(name), "vdev%dbuffer", rvdev->index); >>> + rvdev->dev.parent = rproc->dev.parent; >>> + rvdev->dev.release = rproc_rvdev_release; >>> + dev_set_name(&rvdev->dev, "%s#%s", dev_name(rvdev- >>> dev.parent), name); >>> + dev_set_drvdata(&rvdev->dev, rvdev); >>> + dma_set_coherent_mask(&rvdev->dev, DMA_BIT_MASK(32)); >>> + >>> + ret = device_register(&rvdev->dev); >>> + if (ret) >>> + goto free_rvdev; >>> + >>> /* parse the vrings */ >>> for (i = 0; i < rsc->num_of_vrings; i++) { >>> ret = rproc_parse_vring(rvdev, rsc, i); >>> @@ -518,7 +547,7 @@ static int rproc_handle_vdev(struct rproc *rproc, >> struct fw_rsc_vdev *rsc, >>> for (i--; i >= 0; i--) >>> rproc_free_vring(&rvdev->vring[i]); >>> free_rvdev: >>> - kfree(rvdev); >>> + device_unregister(&rvdev->dev); >>> return ret; >>> } >>> >>> @@ -536,7 +565,7 @@ void rproc_vdev_release(struct kref *ref) >>> >>> rproc_remove_subdev(rproc, &rvdev->subdev); >>> list_del(&rvdev->node); >>> - kfree(rvdev); >>> + device_unregister(&rvdev->dev); >>> } >>> >>> /** >>> diff --git a/drivers/remoteproc/remoteproc_internal.h >> b/drivers/remoteproc/remoteproc_internal.h >>> index f6cad24..bfeacfd 100644 >>> --- a/drivers/remoteproc/remoteproc_internal.h >>> +++ b/drivers/remoteproc/remoteproc_internal.h >>> @@ -52,6 +52,7 @@ struct dentry *rproc_create_trace_file(const char >> *name, struct rproc *rproc, >>> int rproc_alloc_vring(struct rproc_vdev *rvdev, int i); >>> >>> void *rproc_da_to_va(struct rproc *rproc, u64 da, int len); >>> +phys_addr_t rproc_va_to_pa(void *cpu_addr); >>> int rproc_trigger_recovery(struct rproc *rproc); >>> >>> int rproc_elf_sanity_check(struct rproc *rproc, const struct firmware *fw); >>> diff --git a/drivers/remoteproc/remoteproc_virtio.c >> b/drivers/remoteproc/remoteproc_virtio.c >>> index de21f62..9ee63c0 100644 >>> --- a/drivers/remoteproc/remoteproc_virtio.c >>> +++ b/drivers/remoteproc/remoteproc_virtio.c >>> @@ -17,7 +17,9 @@ >>> * GNU General Public License for more details. >>> */ >>> >>> +#include >>> #include >>> +#include >>> #include >>> #include >>> #include >>> @@ -315,10 +317,48 @@ static void rproc_virtio_dev_release(struct device >> *dev) >>> int rproc_add_virtio_dev(struct rproc_vdev *rvdev, int id) >>> { >>> struct rproc *rproc = rvdev->rproc; >>> - struct device *dev = &rproc->dev; >>> + struct device *dev = &rvdev->dev; >>> struct virtio_device *vdev = &rvdev->vdev; >>> + struct rproc_mem_entry *mem; >>> int ret; >>> >>> + /* Try to find dedicated vdev buffer carveout */ >>> + mem = rproc_find_carveout_by_name(rproc, "vdev%dbuffer", >> rvdev->index); >>> + if (mem) { >>> + phys_addr_t pa; >>> + >>> + if (mem->of_resm_idx != -1) { >>> + struct device_node *np = rproc->dev.parent- >>> of_node; >>> + >>> + /* Associate reserved memory to vdev device */ >>> + ret = of_reserved_mem_device_init_by_idx(dev, np, >>> + mem- >>> of_resm_idx); >>> + if (ret) { >>> + dev_err(dev, "Can't associate reserved >> memory\n"); >>> + goto out; >>> + } >>> + } else { >>> + if (mem->va) { >>> + dev_warn(dev, "vdev %d buffer already >> mapped\n", >>> + rvdev->index); >>> + pa = rproc_va_to_pa(mem->va); >>> + } else { >>> + /* Use dma address as carveout no >> memmapped yet */ >>> + pa = (phys_addr_t)mem->dma; >>> + } >>> + >>> + /* Associate vdev buffer memory pool to vdev >> subdev */ >>> + ret = dmam_declare_coherent_memory(dev, pa, >>> + mem->da, >>> + mem->len, >>> + >> DMA_MEMORY_EXCLUSIVE); >> >> Is it not possible to associate the dma_mem with vdev->dev directly, >> instead of injecting yet another device in-between the remoteproc >> platform device and the virtio device? > > Thank you for the review. > I'll check how to use virtio device. An additional ops for memory registration may be needed at virtio level. > >> >> That way the resulting allocation in e.g. virtio_rpmsg would be on the >> virtio device itself, not its parent. > Yes agree, it will simplify allocation/inheritance... + Anup. Anup, Since you are currently interested in patch 17 (dma allocations from vdev parent), may we get some information on how your device hierarchy is looking like inside the VM/Guest? regards Suman > > Regards, > Loic >> >> Regards, >> Bjorn >> >>> + if (ret < 0) { >>> + dev_err(dev, "Failed to associate buffer\n"); >>> + goto out; >>> + } >>> + } >>> + } >>> + >>> vdev->id.device = id, >>> vdev->config = &rproc_virtio_config_ops, >>> vdev->dev.parent = dev; >>> diff --git a/include/linux/remoteproc.h b/include/linux/remoteproc.h >>> index 6b3a234..2921dd2 100644 >>> --- a/include/linux/remoteproc.h >>> +++ b/include/linux/remoteproc.h >>> @@ -547,6 +547,7 @@ struct rproc_vdev { >>> struct kref refcount; >>> >>> struct rproc_subdev subdev; >>> + struct device dev; >>> >>> unsigned int id; >>> struct list_head node; >>> -- >>> 1.9.1 >>>