Received: by 10.223.176.5 with SMTP id f5csp389301wra; Sat, 27 Jan 2018 01:57:43 -0800 (PST) X-Google-Smtp-Source: AH8x225q113ufPJKFZWxfr4KvLYV8/0eB2a3vlRFgRhZ7JwUEMNZIx5WUXmHi+sQZFjuIwOH+i+5 X-Received: by 10.101.76.199 with SMTP id n7mr17390876pgt.388.1517047063092; Sat, 27 Jan 2018 01:57:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517047063; cv=none; d=google.com; s=arc-20160816; b=MoMIVcHOgAVWK4mf5tj2C3XGnMRaq7PnCceqPUR2aKa+9AL55qnGVuSW/RQB6xXJZg CCr0Jd6kJF21VrSuRQ8fUvasQvaXiRctzg1SHY0R7ibntOcMeZXamMr0NNBbWOYnrgIH WQWdgC2K6A+AqY5dKclNGQvbc/woqRbzPXXcvJFkhLuBraSsLarbXO8lai/NjxOly3mk pBN85mRpoIu0wc4jsnWGuo/QincmcJtNMQGrklvCQcExcv/SQyplHHJEwG99ufpJcnwL NfUZTwLnqy4Ns1SzDotVS/eyssIYHjqn1Yq2hkgmZynAwgszbfO/C9u/qRgkacBAkMoI St9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:dkim-signature :arc-authentication-results; bh=UE3Z+98zSFRVfkYHYyqoBLh2H4pTkQ5XjqWcwKLNfG0=; b=Y6n74gi8jChyLOjmErO9YrguJitKgi6vd3D4pHOASmedl4wH7+W74y6mveMzDu45wZ ivu7ykbhizKUr5/ZvxDjNiiz+WNPhQD9+GZQhtCIvN3yU6O7h6KH2InMqBgfg/H8FWEX ocFwFyXsvpWZMUlFwTxQVfVt5vvp5fKyPWDU156XcnfN+2KBRdfKFUUj727glrPny0Gk PLK8j3e26v3uW7LXiumQwUcggi4VEC5ZtUYt2xRWIDfsmYKXZAUuiiOEB+6SxqfNZuS0 CZ0ac38p/lNAUtVjlnloQJ03Eijtyh6ChJQi2X17fX7fAYfET0gctWqc8Z6bZ/IviNl7 psfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=GZYOBYIp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j6si4212826pgn.350.2018.01.27.01.57.28; Sat, 27 Jan 2018 01:57:43 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=GZYOBYIp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752243AbeA0J5I (ORCPT + 99 others); Sat, 27 Jan 2018 04:57:08 -0500 Received: from mail-wm0-f67.google.com ([74.125.82.67]:39617 "EHLO mail-wm0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751824AbeA0J5F (ORCPT ); Sat, 27 Jan 2018 04:57:05 -0500 Received: by mail-wm0-f67.google.com with SMTP id b21so25340009wme.4 for ; Sat, 27 Jan 2018 01:57:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=UE3Z+98zSFRVfkYHYyqoBLh2H4pTkQ5XjqWcwKLNfG0=; b=GZYOBYIpRPo4BjvidRGmQ50x+I8Q2QO22+jt12r5xAaBSCrBjxjBfW/MIO1mwk3XCI 0If2e0RIMxLpdoT8JUu2H1MIbDXtQQsInjeU2O1RX7JupMzENjqt1n8F2jWhSXelhPYO Dpnd94RHBnUsN1ZnpG+xZ/uFPfU0JBgbRDrL9aPs2hUiZ2WIdwN38ARKNvtwkLcnTJ8W SfZsBRQgm5bvarzV1rGCu1I6/iFDKke5hfKqfewmOmiOcIpa27bc1i0m0an812nJVb6b nOa9CpULMZ0TnKVChxW6a7xnSlAywB4C8KoiftFmQO6+JJRD64eupjs4KeOnLcd6PHnC axhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=UE3Z+98zSFRVfkYHYyqoBLh2H4pTkQ5XjqWcwKLNfG0=; b=nr0ywRQPIT/fcN+GaiKmk4xTRPVNM4PO3NgKSpwIkK/GXSqhLltBBjb9tQmHmX+54/ dXLAvvrJcaHuyEYcBFNKX4JRX+DiJf3L1VUOis3H8KsB5ltZs8V4K//RFLcoAj+JWM0+ jETtmRqLqBWdFvGVrTzxCk430xNqyKWRZI19ynvfP579wueORarOWPoZRzCoGcQVgFAx MayvDNeGVD+eL4hwX+iex4YZawJzJkrQ3FUXOlaKvSxJOkxDbCQRsMG6/Oyoo3D22+mN 5JG4lNPBOkN3ESncetGQemXH7/P8n5KMccfYF4uuc12/hifBDcE3j8GRIQgudh4oLVMW YR/w== X-Gm-Message-State: AKwxyteisHiyQDESPn+SukmG5dAaSJ3BQEiWC3QZGYojFkjyaL6Lcu3B 6tipojaBnx5YvsP5eJJpmzMcLBK5cUa4liF7IAE= X-Received: by 10.28.87.20 with SMTP id l20mr12355122wmb.48.1517047023668; Sat, 27 Jan 2018 01:57:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.175.53 with HTTP; Sat, 27 Jan 2018 01:57:03 -0800 (PST) In-Reply-To: <20180127075744.GK3741@linux.vnet.ibm.com> References: <1516694381-20333-1-git-send-email-lianglihao@huawei.com> <20180125055356.GT3741@linux.vnet.ibm.com> <20180127075744.GK3741@linux.vnet.ibm.com> From: Lihao Liang Date: Sat, 27 Jan 2018 09:57:03 +0000 Message-ID: Subject: Re: [PATCH RFC 00/16] A new RCU implementation based on a fast consensus protocol To: Paul McKenney Cc: "Guohanjun (Hanjun Guo)" , heng.z@huawei.com, hb.chen@huawei.com, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Jan 27, 2018 at 7:57 AM, Paul E. McKenney wrote: > On Sat, Jan 27, 2018 at 07:22:27AM +0000, Lihao Liang wrote: >> On Thu, Jan 25, 2018 at 5:53 AM, Paul E. McKenney >> wrote: >> > On Tue, Jan 23, 2018 at 03:59:25PM +0800, lianglihao@huawei.com wrote: >> >> From: Lihao Liang >> >> >> >> Dear Paul, >> >> >> >> This patch set implements a preemptive version of RCU (PRCU) based on the following paper: >> >> >> >> Fast Consensus Using Bounded Staleness for Scalable Read-mostly Synchronization. >> >> Haibo Chen, Heng Zhang, Ran Liu, Binyu Zang, and Haibing Guan. >> >> IEEE Transactions on Parallel and Distributed Systems (TPDS), 2016. >> >> https://dl.acm.org/citation.cfm?id=3024114.3024143 >> >> >> >> We have also added preliminary callback-handling support. Thus, the current version >> >> provides APIs prcu_read_lock(), prcu_read_unlock(), synchronize_prcu(), call_prcu(), >> >> and prcu_barrier(). >> >> >> >> This is an experimental patch, so it would be good to have some feedback. >> >> >> >> Known shortcoming is that the grace-period version is incremented in synchronize_prcu(). >> >> If call_prcu() or prcu_barrier() is called but there is no synchronized_prcu() invoked, >> >> callbacks cannot be invoked. Later version should address this issue, e.g. adding a >> >> grace-period expedition mechanism. Others include to use a a hierarchical structure, >> >> taking into account the NUMA topology, to send IPI in synchronize_prcu(). >> >> >> >> We have tested the implementation using rcutorture on both an x86 and ARM64 machine. >> >> PRCU passed 1h and 3h tests on all the newly added config files except PRCU07 reported BUG >> >> in a 1h run. >> >> >> >> [ 1593.604201] ---[ end trace b3bae911bec86152 ]--- >> >> [ 1594.629450] prcu-torture:torture_onoff task: offlining 14 >> >> [ 1594.755553] smpboot: CPU 14 is now offline >> >> [ 1594.757732] prcu-torture:torture_onoff task: offlined 14 >> >> [ 1597.765149] prcu-torture:torture_onoff task: onlining 11 >> >> [ 1597.766795] smpboot: Booting Node 0 Processor 11 APIC 0xb >> >> [ 1597.804102] prcu-torture:torture_onoff task: onlined 11 >> >> [ 1599.365098] prcu-torture: rtc: ffffffffb0277b90 ver: 66358 tfle: 0 rta: 66358 rtaf: 0 >> >> rtf: 66349 rtmbe: 0 rtbe: 1 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 2233418 >> >> onoff: 191/191:199/199 34,199:59,5102 10403:0 (HZ=1000) barrier: 188/189:1 cbflood: 225 >> >> [ 1599.367946] prcu-torture: !!! >> >> [ 1599.367966] ------------[ cut here ]------------ >> > >> > The "rtbe: 1" indicates that your implementation of prcu_barrier() >> > failed to wait for all preceding call_prcu() callbacks to be invoked. >> > >> > Does the immediately following "Reader Pipe:" list have any but the >> > first two numbers non-zero? >> >> Yes. > > If the third or subsequent numbers are non-zero, that would indicate > too-short grace periods. This would be a critical bug in PRCU. > >> >> We have also compared PRCU with TREE RCU using rcuperf with gp_exp set to true, that is >> >> synchronize_rcu_expedited was tested. >> >> >> >> The rcuperf results are as follows (average grace-period duration in ms of ten 10min runs): >> >> >> >> 16*Intel Xeon CPU@2.4GHz, 16GB memory, Ubuntu Linux 3.13.0-47-generic >> >> >> >> CPUs 2 4 8 12 15 16 >> >> PRCU 0.14 1.07 4.15 8.02 10.79 15.16 >> >> TREE 49.30 104.75 277.55 390.82 620.82 1381.54 >> >> >> >> 64*Cortex-A72 CPU@2.4GHz, 130GB memory, Ubuntu Linux 4.10.0-21.23-generic >> >> >> >> CPUs 2 4 8 16 32 48 63 64 >> >> PRCU 0.23 19.69 38.28 63.21 95.41 167.18 252.01 1841.44 >> >> TREE 416.73 901.89 1060.86 743.00 920.66 1325.21 1646.20 23806.27 >> > >> > Well, at the very least, this is a bug report on either expedited RCU >> > grace-period latency or on rcuperf's measurements, and thank you for that. >> > I will look into this. In the meantime, could you please let me know >> > exactly how you invoked rcuperf? >> >> We used the following command to invoke rcuperf: >> >> sudo ./kvm.sh --torture rcuperf --duration 10 --configs 10*TREE >> >> The actual script run-rcuperf.sh to run the experiments can be found >> in the following email of this patch series: >> >> [PATCH RFC 15/16] rcutorture: Add scripts to run experiments >> >> Please let us know how it goes. > > Will do! > > As I said before, at the very least you have identified a performance bug > in RCU expedited grace periods. > I should add that we also tested the normal synchronize_rcu() on the same x86 machine, and the rcuperf figures were about 10 times slower than those of synchronize_rcu_expedited(). Is this expected for synchronize_rcu()? Best, Lihao. > Thanx, Paul > >> Many thanks, >> Lihao. >> >> > I have a few comments on some of your patches based on a quick scan >> > through them. >> > >> > Thanx, Paul >> > >> >> Best wishes, >> >> Lihao. >> >> >> >> >> >> Lihao Liang (15): >> >> rcutorture: Add PRCU rcu_torture_ops >> >> rcutorture: Add PRCU test config files >> >> rcuperf: Add PRCU rcu_perf_ops >> >> rcuperf: Add PRCU test config files >> >> rcuperf: Set gp_exp to true for tests to run >> >> prcu: Implement call_prcu() API >> >> prcu: Implement PRCU callback processing >> >> prcu: Implement prcu_barrier() API >> >> rcutorture: Test call_prcu() and prcu_barrier() >> >> rcutorture: Add basic ARM64 support to run scripts >> >> prcu: Add PRCU Kconfig parameter >> >> prcu: Comment source code >> >> rcuperf: Add config files with various CONFIG_NR_CPUS >> >> rcutorture: Add scripts to run experiments >> >> Add GPLv2 license >> >> >> >> Heng Zhang (1): >> >> prcu: Add PRCU implementation >> >> >> >> include/linux/interrupt.h | 3 + >> >> include/linux/prcu.h | 122 +++++ >> >> include/linux/rcupdate.h | 1 + >> >> init/Kconfig | 7 + >> >> init/main.c | 2 + >> >> kernel/rcu/Makefile | 1 + >> >> kernel/rcu/prcu.c | 497 +++++++++++++++++++++ >> >> kernel/rcu/rcuperf.c | 33 +- >> >> kernel/rcu/rcutorture.c | 40 +- >> >> kernel/rcu/tree.c | 1 + >> >> kernel/sched/core.c | 2 + >> >> kernel/time/timer.c | 2 + >> >> kvm.sh | 452 +++++++++++++++++++ >> >> run-rcuperf.sh | 26 ++ >> >> .../testing/selftests/rcutorture/bin/functions.sh | 17 +- >> >> .../selftests/rcutorture/configs/rcu/CFLIST | 5 + >> >> .../selftests/rcutorture/configs/rcu/PRCU02 | 27 ++ >> >> .../selftests/rcutorture/configs/rcu/PRCU02.boot | 1 + >> >> .../selftests/rcutorture/configs/rcu/PRCU03 | 23 + >> >> .../selftests/rcutorture/configs/rcu/PRCU03.boot | 2 + >> >> .../selftests/rcutorture/configs/rcu/PRCU06 | 26 ++ >> >> .../selftests/rcutorture/configs/rcu/PRCU06.boot | 5 + >> >> .../selftests/rcutorture/configs/rcu/PRCU07 | 25 ++ >> >> .../selftests/rcutorture/configs/rcu/PRCU07.boot | 2 + >> >> .../selftests/rcutorture/configs/rcu/PRCU09 | 19 + >> >> .../selftests/rcutorture/configs/rcu/PRCU09.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/CFLIST | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU | 20 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-12 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-12.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-14 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-14.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-15 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-15.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-16 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-16.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-2 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-2.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-32 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-32.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-4 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-4.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-48 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-48.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-56 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-56.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-60 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-60.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-62 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-62.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-64 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-64.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU-8 | 21 + >> >> .../rcutorture/configs/rcuperf/PRCU-8.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/PRCU.boot | 1 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-12 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-14 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-15 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-16 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-2 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-32 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-4 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-48 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-56 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-60 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-62 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-64 | 21 + >> >> .../selftests/rcutorture/configs/rcuperf/TREE-8 | 21 + >> >> 68 files changed, 1918 insertions(+), 5 deletions(-) >> >> create mode 100644 include/linux/prcu.h >> >> create mode 100644 kernel/rcu/prcu.c >> >> create mode 100755 kvm.sh >> >> create mode 100755 run-rcuperf.sh >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU02 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU02.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU03 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU03.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU06 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU06.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU07 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU07.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU09 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcu/PRCU09.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-12 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-12.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-14 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-14.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-15 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-15.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-16 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-16.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-2 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-2.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-32 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-32.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-4 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-4.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-48 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-48.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-56 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-56.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-60 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-60.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-62 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-62.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-64 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-64.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-8 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU-8.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/PRCU.boot >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-12 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-14 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-15 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-16 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-2 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-32 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-4 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-48 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-56 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-60 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-62 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-64 >> >> create mode 100644 tools/testing/selftests/rcutorture/configs/rcuperf/TREE-8 >> >> >> >> -- >> >> 2.14.1.729.g59c0ea183 >> >> >> > >> >