2022-05-28 18:28:10

by Stephen Hemminger

[permalink] [raw]
Subject: RE: [PATCH v2] Drivers: hv: vmbus: Don't assign VMbus channel interrupts to isolated CPUs

Doing this will actually help DPDK applications on isolated cpus.
The history is isolated cpus came first, then cpusets and now the preferred kernel solution is cgroups.

For PCI hardware this is handled in userspace typically since it is a policy decision.

-----Original Message-----
From: Michael Kelley (LINUX) <[email protected]>
Sent: Saturday, May 28, 2022 5:56 AM
To: Stephen Hemminger <[email protected]>; Saurabh Sengar <[email protected]>; KY Srinivasan <[email protected]>; Haiyang Zhang <[email protected]>; [email protected]; Dexuan Cui <[email protected]>; [email protected]; [email protected]; Saurabh Singh Sengar <[email protected]>
Subject: RE: [PATCH v2] Drivers: hv: vmbus: Don't assign VMbus channel interrupts to isolated CPUs

From: Stephen Hemminger <[email protected]> Sent: Friday, May 27, 2022 8:41 AM
>
> Would this have impact for DPDK applications using isolated cpus?

I don't have any existing knowledge of DPDK use of isolated CPUs,
so someone with more expertise feel free to correct me.

From what I see in the DPDK documentation (Section 8.3 here:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoc.dpdk.org%2Fguides%2Flinux_gsg%2Fenable_func.html&amp;data=05%7C01%7Csthemmin%40microsoft.com%7C45f1aefca73845a7470408da40a96d81%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637893393679306569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=vT3keyehM9AWGhPJ9ItWJhhjN%2Bl7ZGB07l1KapOG0I0%3D&amp;reserved=0), there's
no impact. The example in that documentation does CPU isolation
only for the purpose of scheduling, not for interrupts. The
example kernel command line is:

isolcpus=2,4,6

which defaults to "domain" as the "flag" and is equivalent to:

isolcpus=domain,2,4,6.

VMbus channel interrupts are affected only if "managed_irq" is
specified as the flag per the commit message below.

And FWIW, cpusets provide a better way to doing scheduler
isolation than the isolcpus kernel boot option. Perhaps the
DPDK documentation should be updated. :-)

Michael