From:   "Chen, Mike Ximing" <mike.ximing.chen@intel.com>
To:     Andrew Lunn <andrew@lunn.ch>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "arnd@arndb.de" <arnd@arndb.de>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "Williams, Dan J" <dan.j.williams@intel.com>,
        "pierre-louis.bossart@linux.intel.com" 
        <pierre-louis.bossart@linux.intel.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "kuba@kernel.org" <kuba@kernel.org>
Subject: RE: [RFC PATCH v12 01/17] dlb: add skeleton for DLB driver
Thread-Topic: [RFC PATCH v12 01/17] dlb: add skeleton for DLB driver
Thread-Index: AQHX9jcKOP1y1O92CEC3pIeygh1pI6w8tNAAgACRSSA=
Date:   Tue, 21 Dec 2021 20:56:42 +0000
Message-ID: <CO1PR11MB51705AE8B072576F31FEC18CD97C9@CO1PR11MB5170.namprd11.prod.outlook.com>
References: <20211221065047.290182-1-mike.ximing.chen@intel.com>
 <20211221065047.290182-2-mike.ximing.chen@intel.com>
 <YcGkILZxGLEUVVgU@lunn.ch>
In-Reply-To: <YcGkILZxGLEUVVgU@lunn.ch>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.6.200.16
dlp-reaction: no-action
dlp-product: dlpe-windows
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?tOdBbiGzPPNC6SDcHaAYVp1oUKZDahwuz5RMjF+P8nrNjpIazKx7HtBggfyh?=
 =?us-ascii?Q?BOFRzBznlSGeQ1blUADV1OTmG7/lHgyKCy+A8gJeoBUDxTJOZyvr3DDHkjW4?=
 =?us-ascii?Q?4LA0Qvu3IwKHE70AJDPSeJZpp6YVtzsAX+1LTDa0InxvrFc8YfkqhHhL3a0i?=
 =?us-ascii?Q?oKX9QcckrVQaV3BjeJurMUbfDY2fziAMvZQsVYW535U8rfUk0u0nHDZvjWJe?=
 =?us-ascii?Q?nDvskxBHZ78lIONBpCc19zK1dYrBj8N/icpaJXuGyVI0dFvBTtlxWiMqSXal?=
 =?us-ascii?Q?XM75QUuY9tKxGzvvNiDc7KBof9f6TKbrk2zctYDvk7PsEnkzuNArxNz/Ekci?=
 =?us-ascii?Q?1XDvOhvqF/kd4t9qZnKjcjJxMtNt4yH5rkqHHd8+hjXGcXO+rESALr4cGu/k?=
 =?us-ascii?Q?hj1pLz7HA/+YuGLPLtMQ9mB9qxdItMNCtyWauMym0XoinjQsJB4yAynjsqH7?=
 =?us-ascii?Q?DKIEpnZp0yMui1Z7iC9Q93CtA/Qo7eOllf9tL4IrWyKf0dFwv0LjyPdomi91?=
 =?us-ascii?Q?4pDcs6m93EiOw4KQQ2/1QbR0EcNvUaPHfmV1rRq8CEO8WXrtv27T91iJg+bO?=
 =?us-ascii?Q?8Q7sumJovGX+Of2l7HP6dAniS0i909GuMEk52N+gP0dtZycm/a+JSRnRjRW8?=
 =?us-ascii?Q?jMlUkrlS2+8f+izhIYkrXnDjHeBQp6qBr6rF7cEFVOiutjQpAgzZ1wXfpM1K?=
 =?us-ascii?Q?gCksjdk9VynyO4YiC6fnUEzLBwqeygwZJ3ofT0cHNJKDCaLKxmPo05hVJ+mK?=
 =?us-ascii?Q?3+JOwOS7Ov7VbiXBwBnUIvL0DyDiZ3eZAinof3l7hHud9HSFaJVcRPJHpUBz?=
 =?us-ascii?Q?ILB8sJM+vlov3I3/UOJTVF6bVvui+HL0Vgnz3LaK2YyIuePEwqg7MO47B7YS?=
 =?us-ascii?Q?7iHDpXwBe8jPsGHAi6eNMjuBbpz3piduk53ZXvfkRzt9DAMgmN+CU0DV68wu?=
 =?us-ascii?Q?yePsg+99ZlBpMFDaXShQJ6GO5wmhJMOEl8Xd5PDdX/oN9/FpYwhGMQIV8bm2?=
 =?us-ascii?Q?tIOmaZ//7bh3joco2M3D0gpF60vvrBlWi5ij9ToggO3wPP8+4Rd0RG6tn3ek?=
 =?us-ascii?Q?0QGL9kPQIqdbV14LbxQK54qxjSNjvDio44KVcy4GfJbDROq0KNPfshZesqcJ?=
 =?us-ascii?Q?Tx5QzXUJgMU9H+wS/iV8Wn+EaGvOSqy+K9/25KUrkN3s+ciylL7U1JBbuSvL?=
 =?us-ascii?Q?Wl4cduJPIEmmINrhHu41r0P2qeFhK4Uh9h2u+0y9JHI5W7IVQXr0ctiByNM7?=
 =?us-ascii?Q?DIt2hIM1W+UaQX0P5YMVCoxE2UxpETq2TbEKR/kj40Y4KuNfvujhfwCV+NXq?=
 =?us-ascii?Q?wQ6nlb0ibxLtYyIG66FHkGvM56at30t2yoQIt8Anj+N2E2I2uORtkQ6a5sYy?=
 =?us-ascii?Q?gB2ytkj25TBqeKtDQoBeyd3yeYcS113+0DtTb7mE9feuxsF6elkkygPXWbN7?=
 =?us-ascii?Q?2CiivaZl5YjUJF8/CW3bNjhtahFccBR+6tDOx0u06S8lztdJPQBhgiGjmZ1N?=
 =?us-ascii?Q?z319DLr4a/vvHZaZO2wNhBXPchhMsPiKyiOQ27oA0uEKTF+7y6wO1jx03zgn?=
 =?us-ascii?Q?phbMBCJ8lHLtXKar9bdxdNnWlhlHA64Rmrkup8GrIYrAvrAqchn+cRhNHFR6?=
 =?us-ascii?Q?NiEGvBH+qjPlJRnCkVh1A4/E8CPbVovCY7AT3D/dbBzUDPLrYBm+Qq5qB48l?=
 =?us-ascii?Q?/hS9mg=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB5170.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 03120129-9d77-4e2b-967a-08d9c4c46457
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Dec 2021 20:56:42.4487
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FhS9XJrnXfFjDO4ZApj/yqWdBemg+jpa1PrgjWEy31YADhc1qkM1Q/Fi9PniDSM7McfQv7xS2AUON0FIgnpnAPMYtC6rh8chipfriblt9zY=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4804
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Tuesday, December 21, 2021 4:54 AM
> To: Chen, Mike Ximing <mike.ximing.chen@intel.com>
> Cc: linux-kernel@vger.kernel.org; arnd@arndb.de; gregkh@linuxfoundation.o=
rg; Williams, Dan J
> <dan.j.williams@intel.com>; pierre-louis.bossart@linux.intel.com; netdev@=
vger.kernel.org;
> davem@davemloft.net; kuba@kernel.org
> Subject: Re: [RFC PATCH v12 01/17] dlb: add skeleton for DLB driver
>=20
> > +The following diagram shows a typical packet processing pipeline with =
the Intel DLB.
> > +
> > +                              WC1              WC4
> > + +-----+   +----+   +---+  /      \  +---+  /      \  +---+   +----+  =
 +-----+
> > + |NIC  |   |Rx  |   |DLB| /        \ |DLB| /        \ |DLB|   |Tx  |  =
 |NIC  |
> > + |Ports|---|Core|---|   |-----WC2----|   |-----WC5----|   |---|Core|--=
-|Ports|
> > + +-----+   -----+   +---+ \        / +---+ \        / +---+   +----+  =
 ------+
> > +                           \      /         \      /
> > +                              WC3              WC6
>=20
> This is the only mention of NIC here. Does the application interface to t=
he network stack in the usual way
> to receive packets from the TCP/IP stack up into user space and then copy=
 it back down into the MMIO
> block for it to enter the DLB for the first time? And at the end of the p=
ath, does the application copy it
> from the MMIO into a standard socket for TCP/IP processing to be send out=
 the NIC?
>=20
For load balancing and distribution purposes, we do not handle packets dire=
ctly in DLB. Instead, we only
send QEs (queue events) to MMIO for DLB to process. In an network applicati=
on, QEs (64 bytes each) can
contain pointers to the actual packets. The worker cores can use these poin=
ters to process packets and
forward them to the next stage. At the end of the path, the last work core =
can send the packets out to NIC.
=20
> Do you even needs NICs here? Could the data be coming of a video camera a=
nd you are distributing image
> processing over a number of cores?
No, the diagram is just an example for packet processing applications. The =
data can come from other sources
such video cameras. The DLB can schedule up to 100 million packets/events p=
er seconds. The frame rate from
a single camera is normally much, much lower than that.

>=20
> 	 Andrew