Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Date:   Sat, 16 Sep 2023 00:03:48 -0700
From:   Dan Williams <dan.j.williams@intel.com>
To:     Ira Weiny <ira.weiny@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Alison Schofield <alison.schofield@intel.com>,
        Vishal Verma <vishal.l.verma@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Ben Widawsky <bwidawsk@kernel.org>,
        Jonathan Cameron <Jonathan.Cameron@huawei.com>
CC:     <linux-cxl@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        Ira Weiny <ira.weiny@intel.com>
Subject: RE: [PATCH 3/3] tools/testing/cxl: Document test configurations
Message-ID: <650553545372e_d7cc829461@dwillia2-xfh.jf.intel.com.notmuch>
References: <20230426-cxl-fixes-v1-0-870c4c8b463a@intel.com>
 <20230426-cxl-fixes-v1-3-870c4c8b463a@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230426-cxl-fixes-v1-3-870c4c8b463a@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?u7qTQFM3g7uYXTw1W2sW/1RkqcfW37MCzpo67oto+OEah44FZ/UEHRJbIyGX?=
 =?us-ascii?Q?nkIxk3yLmeWcLvs/C3T9vlrtAJUqs00wR3c/GJyF95jbGMKCWnV9NfP4Iat5?=
 =?us-ascii?Q?z2zp/dMVRqlDKcZQOsY6B0+dp8tzFjqoH1fUiGT4nVWzUePMpVak1VqPWquT?=
 =?us-ascii?Q?zK2VtWr69tYIagJ2IaFx61fy9G2tidRhRyislyNrOIgY9xrJsrMWTqhKH5gD?=
 =?us-ascii?Q?yChIOkH4IgtXxAav9xtM3S/bgkufbH+GaEl5ynz0nI2582MrUKZ6M9OiKQSM?=
 =?us-ascii?Q?qQNNEIzBrEvoGIfPqJr0IDNc7mn6UXMEXjrS+V7jrAx101DCdtEfX3Gl5eUr?=
 =?us-ascii?Q?YPxiPemSon4ehm1XHo5eWCBhekXDjAhS3cnyEjDUr7A4EfY1c2IILjYVO2wE?=
 =?us-ascii?Q?4T8wzCyDsoxgb7Vpwmd0SO6AUDABVhvJWHpFhFkII/Sn/PMcEiWRB5WsrQtL?=
 =?us-ascii?Q?3p81QG16zmVUInTqGqShr8saakrM52rQpJLc7XoAvmqVU4atc73CL/2wy5F1?=
 =?us-ascii?Q?ZC4q/ZaxO7MIaFlXkWUVycVFLPkmz89uuz7HkTYWQJL+o3bOlw/KMlF1c6Pr?=
 =?us-ascii?Q?54ARDNuZLXTUceEz2CfopI35HlBOpFZ4tpNvwlLcTi1pkdSkXP/6jONYLqCI?=
 =?us-ascii?Q?x9V+L0b0KlfaOod5rJ8aq9AA8XoP6sjyNpr3E61ewkAWq9bPAtqbtFE28dpU?=
 =?us-ascii?Q?6+Kcz9JDCD2haPMTH5h7Rb3P80NbwV1el950dmqk2pZw9z0TrHL73rox3p/Y?=
 =?us-ascii?Q?e2IQNyVdmYNPolFFflwiC6Ejie3FCW0HjEsOjQeYM3qs6lczj/qTP3vhoka/?=
 =?us-ascii?Q?CkmJPO+jQbgMNP5Kgqk/dmrki4NmhWZohvoohHiOznZb1RUN5hIqf0sLRHpp?=
 =?us-ascii?Q?aIyL+i9fdpN/LiVTgtF7dNoogkGHltYQs5gbXNJfqdj6MNnScg87XyOk/CU2?=
 =?us-ascii?Q?U0YiAF1NBzRw+OoYaqn959bGf/2crPjh8J1KK2MorDI6knvR2ffk6xTBhMRD?=
 =?us-ascii?Q?LLm/vdCh8d3uvLZpiQB+7+72iSVduUEg6iofo+yFD7NuKZU+XVpT1zYNYKjd?=
 =?us-ascii?Q?uD6F3Mr1MdC5wvsMHDnIKHlvVNg5kueupC/yZ8a8jXOEJbsjuH5dnOCp6iH3?=
 =?us-ascii?Q?hWXNGueh69DQ3kLM+98clP/L1+XVZTJCICO9hYIAU3pIlAokpW2Jy9OEOp2l?=
 =?us-ascii?Q?2nZ8yige2SApcuE1N3fxLu2uQTq+qlAh5AB4tRAOLIngBpYGSaDhRZBa1ewB?=
 =?us-ascii?Q?Q9vdpEdiG0gOT3C41tU9WJsTyRxWBzOLleUTCXcJyLC0WJ2Ap7zRYx70y3C4?=
 =?us-ascii?Q?LrgLociIAKO/nYAZQhEXY5QF0UkiUpwPHmlnzKbZWz6IbldIGybHPNuExffk?=
 =?us-ascii?Q?rhdmyFirIYhvUBROhBySreTiAmZxu9DfQGLVOZqNMiKmBeOe9wShoPIVj/PG?=
 =?us-ascii?Q?uTQ8nre9/BJixN+IbEydXeL/vAtlcIL2c16kXXltSFjebMJTt+ne9XciyeUQ?=
 =?us-ascii?Q?fc5lbi/xdXN7TjgFSdW/4K4ntIY9zrPdSHAPFcprXMrlFaa3tEPFiHocqHXW?=
 =?us-ascii?Q?z57TCAITtqqJkxUlLBGibNAG7LQW2N/1w9IHm7m3k19TJH1bebk0UqaOx/7G?=
 =?us-ascii?Q?Og=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 194b5bda-2e53-4b6f-f1d3-08dbb6831534
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB8107.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Sep 2023 07:03:51.7682
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fRRrBdE1sukkeb6X0384cbzU/45J0S9wRqc7FkDoE2PaNC6SR6hOG1DOPHBlmc8bx9wX7pNZXOsB/ndk5DqyyrEljwFqWuAjEYZB5suqnOU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4845
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Sat, 16 Sep 2023 00:04:45 -0700 (PDT)

Ira Weiny wrote:
> The devices created, their relationship, and intended testing purpose is
> not extremely clear, especially for those unfamiliar with cxl-test.
> 
> Document the purpose of each hierarchy.  Add ASCII art to show the
> relationship of devices.  Group the device declarations together based
> on the hierarchies.
> 
> Signed-off-by: Ira Weiny <ira.weiny@intel.com>
> ---
>  tools/testing/cxl/test/cxl.c | 75 ++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 73 insertions(+), 2 deletions(-)
> 
> diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c
> index bf00dc52fe96..bd38a5fb60ae 100644
> --- a/tools/testing/cxl/test/cxl.c
> +++ b/tools/testing/cxl/test/cxl.c
> @@ -23,6 +23,31 @@ static int interleave_arithmetic;
>  #define NR_CXL_PORT_DECODERS 8
>  #define NR_BRIDGES (NR_CXL_HOST_BRIDGES + NR_CXL_SINGLE_HOST + NR_CXL_RCH)
>  
> +/*
> + * Interleave testing
> + *
> + *             +---------------+                            +---------------+
> + *             | host_bridge[0]|                            | host_bridge[1]|
> + *             +-/---------\---+                            +--/---------\--+
> + *             /-           -\                               /-           -\
> + *           /-               -\                           /-               -\
> + *   +-------------+         +-------------+       +-------------+      +-------------+
> + *   |root_port[0] |         |root_port[1] |       |root_port[2] |      |root_port[3] |
> + *   +------|------+         +------|------+       +------|------+      +------|------+
> + *          |                       |                     |                    |
> + *  +-------|-------+       +-------|-------+     +-------|-------+    +-------|-------+
> + *  |switch_uport[0]|       |switch_uport[1]|     |switch_uport[2]|    |switch_uport[3]|
> + *  +---|-------|---+       +---/-------|---+     +---/-------|---+    +---|-------\---+
> + *      |       \              /        \            /        \            /        \
> + * +----|----++--|------++---------++----|----++---------++----|----++----|----++---------+
> + * |switch   ||switch   ||switch   ||switch   ||switch   ||switch   ||switch   ||switch   |
> + * |_dport[0]||_dport[1]||_dport[2]||_dport[3]||_dport[4]||_dport[5]||_dport[6]||_dport[7]|
> + * +----|----++--|------++----|----++----|----++----|----++----|----++----|----++----|----+
> + *      |        |            |          |          |          |          |          |
> + *  +---|--+   +-|----+   +---|--+   +---|--+    +--|---+  +---|--+   +---|--+   +---|--+
> + *  |mem[0]|   |mem[1]|   |mem[2]|   |mem[3]|    |mem[4]|  |mem[5]|   |mem[6]|   |mem[7]|
> + *  +------+   +------+   +------+   +------+    +------+  +------+   +------+   +------+
> + */

Circling back to merge this I realize that the numbering is off. For
example a snippet from "cxl list -BPT -b cxl_test"

    "ports:root3":[
      {
        "port":"port5",
        "host":"cxl_host_bridge.1",
        "depth":1,
        "nr_dports":2,
        "dports":[
          {
            "dport":"cxl_root_port.1",
            "id":1
          },
          {
            "dport":"cxl_root_port.3",
            "id":3
          }
        ],

This is due to the modulo math at setup time. I only noticed this
because I wanted a diagram to refer to when doing some recent
extensions.

I wonder if we could just use "cxl list" to maintain this diagram, or
maybe circle back and use this to keep an image up to date on a web page
somewhere:

https://lore.kernel.org/linux-cxl/cover.1660895649.git.sunfishho12@gmail.com/