Received: by 10.223.185.116 with SMTP id b49csp4345354wrg; Mon, 26 Feb 2018 16:06:03 -0800 (PST) X-Google-Smtp-Source: AH8x225r2ZwBiLSycCjZR2HsKIP+HmUNAAf1i6lHOvBe2raT3C36XX/JFGKmf6Nuv13H1lw6j/lG X-Received: by 2002:a17:902:bb89:: with SMTP id m9-v6mr12270212pls.423.1519689963211; Mon, 26 Feb 2018 16:06:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519689963; cv=none; d=google.com; s=arc-20160816; b=C69IMsO9jVZFXBgPpw/vYLHTFRKalVytiPVor35DSFNvMzM4wKuTFS1aJEeARRuCea NYpFlvub4qN3GN1QyBwGTQ58gGvIz4i23CWR5kVDA4eoZl3YCtV9mbQ1xYZFGXeMwjiT WokSyhwIGDCMVBTMDW4+5e4VQltujhRuIIW4FQyepBQjIjfXp+E5in2N6SIQeP4MHz7K uBAq9w6zvpShoAZ9S7Na0zyDmkU/BP0MytpDYkmpl/LNGa1f7lQnJ+tC7pzVNYgWp1Km uPdA9n+EwtRy5CPvmOCOpLhl+eSdyeLqlXNGBm3C/dmyAKrRPnZvSDYrB3RoE1dsi7Qc Wh1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:in-reply-to:content-transfer-encoding :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:arc-authentication-results; bh=jMCJKdH1g7u/bLgOjL7wTI3tRYvHey+bZwIlRYTrQm0=; b=YqhkcCb0nmnbWU77MG6BWmugEXEqGOhVRq8Db5GPiKKMw1E8ISs/Cgpi1H7di8Tt3t f1yjhZHl2+c0zmn5Z2L/CzVpduDpvKqolXddhKlsT0Q8aHfvZf3DpS2BvfE0QgosG+8g zZV0MuOo+ZD/Sf3SLohfBWyNi8ja+1Yha+CYD0IP6X8ASFtg868itW46WfWRa1GFXL6d 9IlCoGDeYXEb/qe2GMqQQpGW3k+yZ2WF2xNcpd/mJxE57KQbF1PF809aViCpVxauA7Er C0RmVtVsNM8RuWY+8boQfcNrMbwGzX99+p+QZcN8hIaF3mfdFl36aRcquBx9OAa/ysXp QkSQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 70-v6si7572663ple.465.2018.02.26.16.05.47; Mon, 26 Feb 2018 16:06:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751814AbeB0AE4 (ORCPT + 99 others); Mon, 26 Feb 2018 19:04:56 -0500 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:53972 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751797AbeB0AEy (ORCPT ); Mon, 26 Feb 2018 19:04:54 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C15D2402242D; Tue, 27 Feb 2018 00:04:53 +0000 (UTC) Received: from redhat.com (ovpn-125-210.rdu2.redhat.com [10.10.125.210]) by smtp.corp.redhat.com (Postfix) with ESMTP id C623D111DCFC; Tue, 27 Feb 2018 00:04:49 +0000 (UTC) Date: Tue, 27 Feb 2018 02:04:49 +0200 From: "Michael S. Tsirkin" To: =?iso-8859-1?Q?Marc-Andr=E9?= Lureau Cc: linux-kernel@vger.kernel.org, slp@redhat.com, bhe@redhat.com, somlo@cmu.edu, xiaolong.ye@intel.com Subject: Re: [PATCH v15 11/11] RFC: fw_cfg: do DMA read operation Message-ID: <20180227020104-mutt-send-email-mst@kernel.org> References: <20180215213312.29234-1-marcandre.lureau@redhat.com> <20180215213312.29234-12-marcandre.lureau@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180215213312.29234-12-marcandre.lureau@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.3 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 27 Feb 2018 00:04:53 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Tue, 27 Feb 2018 00:04:53 +0000 (UTC) for IP:'10.11.54.3' DOMAIN:'int-mx03.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'mst@redhat.com' RCPT:'' Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Feb 15, 2018 at 10:33:12PM +0100, Marc-Andr? Lureau wrote: > Modify fw_cfg_read_blob() to use DMA if the device supports it. > Return errors, because the operation may fail. > > So far, only one call in fw_cfg_register_dir_entries() is using > kmalloc'ed buf and is thus clearly eligible to DMA read. > > Initially, I didn't implement DMA read to speed up boot time, but as a > first step before introducing DMA write (since read operations were > already presents). Even more, I didn't realize fw-cfg entries were > being read by the kernel during boot by default. But actally fw-cfg > entries are being populated during module probe. I knew DMA improved a > lot bios boot time (the main reason the DMA interface was added > afaik). Let see the time it would take to read the whole ACPI > tables (128kb allocated) > > # time cat /sys/firmware/qemu_fw_cfg/by_name/etc/acpi/tables/raw > - with DMA: sys 0m0.003s > - without DMA (-global fw_cfg.dma_enabled=off): sys 0m7.674s > > FW_CFG_FILE_DIR (0x19) is the only "file" that is read during kernel > boot to populate sysfs qemu_fw_cfg directory, and it is quite > small (1-2kb). Since it does not expose itself, in order to measure > the time it takes to read such small file, I took a comparable sized > file of 2048 bytes and exposed it (-fw_cfg test,file=file with a > modified read_raw enabling DMA) > > # perf stat -r 100 cat /sys/firmware/qemu_fw_cfg/by_name/test/raw >/dev/null > - with DMA: > 0.636037 task-clock (msec) # 0.141 CPUs utilized ( +- 1.19% ) > - without DMA: > 6.430128 task-clock (msec) # 0.622 CPUs utilized ( +- 0.22% ) > > That's a few msec saved during boot by enabling DMA read (the gain > would be more substantial if other & bigger fw-cfg entries are read by > others from sysfs, unfortunately, it's not clear if we can always > enable DMA there) > > Signed-off-by: Marc-Andr? Lureau > --- > drivers/firmware/qemu_fw_cfg.c | 61 ++++++++++++++++++++++++++++++++++-------- > 1 file changed, 50 insertions(+), 11 deletions(-) > > diff --git a/drivers/firmware/qemu_fw_cfg.c b/drivers/firmware/qemu_fw_cfg.c > index 3015e77aebca..94df57e9be66 100644 > --- a/drivers/firmware/qemu_fw_cfg.c > +++ b/drivers/firmware/qemu_fw_cfg.c > @@ -124,12 +124,47 @@ static ssize_t fw_cfg_dma_transfer(void *address, u32 length, u32 control) > return ret; > } > > +/* with acpi & dev locks taken */ > +static ssize_t fw_cfg_read_blob_dma(u16 key, > + void *buf, loff_t pos, size_t count) > +{ > + ssize_t ret; > + > + if (pos == 0) { > + ret = fw_cfg_dma_transfer(buf, count, key << 16 > + | FW_CFG_DMA_CTL_SELECT > + | FW_CFG_DMA_CTL_READ); > + } else { > + fw_cfg_sel_endianness(key); > + ret = fw_cfg_dma_transfer(NULL, pos, FW_CFG_DMA_CTL_SKIP); > + if (ret < 0) > + return ret; > + ret = fw_cfg_dma_transfer(buf, count, > + FW_CFG_DMA_CTL_READ); > + } > + > + return ret; > +} > + > +/* with acpi & dev locks taken */ > +static ssize_t fw_cfg_read_blob_io(u16 key, > + void *buf, loff_t pos, size_t count) > +{ > + fw_cfg_sel_endianness(key); > + while (pos-- > 0) > + ioread8(fw_cfg_reg_data); > + ioread8_rep(fw_cfg_reg_data, buf, count); > + return count; > +} > + > /* read chunk of given fw_cfg blob (caller responsible for sanity-check) */ > static ssize_t fw_cfg_read_blob(u16 key, > - void *buf, loff_t pos, size_t count) > + void *buf, loff_t pos, size_t count, > + bool dma) > { > u32 glk = -1U; > acpi_status status; > + ssize_t ret; > > /* If we have ACPI, ensure mutual exclusion against any potential > * device access by the firmware, e.g. via AML methods: so this adds a dma flag to fw_cfg_read_blob. > @@ -143,14 +178,17 @@ static ssize_t fw_cfg_read_blob(u16 key, > } > > mutex_lock(&fw_cfg_dev_lock); > - fw_cfg_sel_endianness(key); > - while (pos-- > 0) > - ioread8(fw_cfg_reg_data); > - ioread8_rep(fw_cfg_reg_data, buf, count); > + if (dma && fw_cfg_dma_enabled()) { > + ret = fw_cfg_read_blob_dma(key, buf, pos, count); > + } else { > + ret = fw_cfg_read_blob_io(key, buf, pos, count); > + } > + > mutex_unlock(&fw_cfg_dev_lock); > > acpi_release_global_lock(glk); > - return count; > + > + return ret; > } > > #ifdef CONFIG_CRASH_CORE If set to false it does io, if set to true it does dma. I would prefer passing an accessor function pointer since that's clearer than true/false. > @@ -284,7 +322,7 @@ static int fw_cfg_do_platform_probe(struct platform_device *pdev) > > /* verify fw_cfg device signature */ > if (fw_cfg_read_blob(FW_CFG_SIGNATURE, sig, > - 0, FW_CFG_SIG_SIZE) < 0 || > + 0, FW_CFG_SIG_SIZE, false) < 0 || > memcmp(sig, "QEMU", FW_CFG_SIG_SIZE) != 0) { > fw_cfg_io_cleanup(); > return -ENODEV; > @@ -468,7 +506,8 @@ static ssize_t fw_cfg_sysfs_read_raw(struct file *filp, struct kobject *kobj, > if (count > entry->size - pos) > count = entry->size - pos; > > - return fw_cfg_read_blob(entry->select, buf, pos, count); > + /* do not use DMA, virt_to_phys(buf) might not be ok */ > + return fw_cfg_read_blob(entry->select, buf, pos, count, false); > } > > static struct bin_attribute fw_cfg_sysfs_attr_raw = { > @@ -634,7 +673,7 @@ static int fw_cfg_register_dir_entries(void) > size_t dir_size; > > ret = fw_cfg_read_blob(FW_CFG_FILE_DIR, &files_count, > - 0, sizeof(files_count)); > + 0, sizeof(files_count), false); > if (ret < 0) > return ret; > > @@ -646,7 +685,7 @@ static int fw_cfg_register_dir_entries(void) > return -ENOMEM; > > ret = fw_cfg_read_blob(FW_CFG_FILE_DIR, dir, > - sizeof(files_count), dir_size); > + sizeof(files_count), dir_size, false); > if (ret < 0) > goto end; > > @@ -697,7 +736,7 @@ static int fw_cfg_sysfs_probe(struct platform_device *pdev) > goto err_probe; > > /* get revision number, add matching top-level attribute */ > - err = fw_cfg_read_blob(FW_CFG_ID, &rev, 0, sizeof(rev)); > + err = fw_cfg_read_blob(FW_CFG_ID, &rev, 0, sizeof(rev), false); > if (err < 0) > goto err_probe; Looks like all callers pass in false as parameter. Given this, how can this speed up any operations? Are you sure you tested this properly? > -- > 2.16.1.73.g5832b7e9f2