Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   "Meyer, Kyle" <kyle.meyer@hpe.com>
To:     LKML <linux-kernel@vger.kernel.org>
CC:     "rjw@rjwysocki.net" <rjw@rjwysocki.net>,
        "viresh.kumar@linaro.org" <viresh.kumar@linaro.org>,
        "srinivas.pandruvada@linux.intel.com" 
        <srinivas.pandruvada@linux.intel.com>,
        "lenb@kernel.org" <lenb@kernel.org>
Subject: Re: [RFC] cpufreq: Excessive CPUFreq driver loading
Thread-Topic: [RFC] cpufreq: Excessive CPUFreq driver loading
Thread-Index: AQHXQoF9nK9JnggdOkGbcZj4RcL07qrjDSmQ
Date:   Fri, 14 May 2021 14:06:54 +0000
Message-ID: <TU4PR8401MB05927B583A481A3F1823CD549B509@TU4PR8401MB0592.NAMPRD84.PROD.OUTLOOK.COM>
References: <DF4PR8401MB103590DB5A8656D6748B428E9B589@DF4PR8401MB1035.NAMPRD84.PROD.OUTLOOK.COM>
In-Reply-To: <DF4PR8401MB103590DB5A8656D6748B428E9B589@DF4PR8401MB1035.NAMPRD84.PROD.OUTLOOK.COM>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
X-MS-Exchange-CrossTenant-Network-Message-Id: 1823693d-cfbe-44e4-83c3-08d916e18780
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2021 14:06:54.4935
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: hTuhFKP+Nj+r/kMTyHeDeTdWsutPdmlSw22BdraYZv1uc8VCxR8YaxYJIhSlsfZ07kYHolwC2sYfxoGrJZFtvA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TU4PR8401MB0591
X-OriginatorOrg: hpe.com
X-Proofpoint-GUID: XAuphFOSKvsZ5aaIGjFb90xVVT8uoBPU
X-Proofpoint-ORIG-GUID: XAuphFOSKvsZ5aaIGjFb90xVVT8uoBPU
Content-Transfer-Encoding: quoted-printable
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-HPE-SCL: -1
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761
 definitions=2021-05-14_06:2021-05-12,2021-05-14 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 impostorscore=0
 malwarescore=0 lowpriorityscore=0 mlxscore=0 phishscore=0 mlxlogscore=999
 priorityscore=1501 clxscore=1011 bulkscore=0 adultscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000
 definitions=main-2105140113
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Adding maintainers to the CC list.

Thank you,
Kyle Meyer

________________________________________
From: Meyer, Kyle <kyle.meyer@hpe.com>
Sent: Thursday, May 6, 2021 9:25 AM
To: LKML
Subject: [RFC] cpufreq: Excessive CPUFreq driver loading

Hello,

acpi-cpufreq is mutually exclusive with intel_pstate, however, acpi-cpufreq=
 is
loaded multiple times during startup while intel_pstate is enabled.

This issue was reported to the systemd maintainers and they indicated that =
it
should be fixed in the kernel: https://github.com/systemd/systemd/issues/19=
439

During startup, the kernel triggers one uevent for each device as a result =
of
systemd-udev-trigger.service executing "udevadm trigger --type=3Dsubsystems
--action=3Dadd" and "udevadm trigger --type=3Ddevices --action=3Dadd". The =
service
exists to retrigger all devices as uevents sent by the kernel, before
systemd-udevd is running, would have been missed. When systemd-udevd receiv=
es a
uevent it matches its configured rules against the device. If a uevent's
ACTION=3D=3D"add", systemd-udevd will run "kmod load $env{MODALIAS}" from
80-drivers.rules. udev's builtin kmod will then attempt to load modules
matching the device's MODALIAS.

When systemd-udevd recieves an "add" uevent from
/devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0007:XXX it runs "kmod load cpu:type:x=
86,
...,00E8,..." as "cpu:type:x86,...,00E8,..." is that devices MODALIAS.

When systemd-udevd recieves an "add" uevent from /devices/system/cpu/cpuXXX=
 it
runs "kmod load acpi:ACPI0007:" as "acpi:ACPI0007:" is that devices MODALIA=
S.

acpi-cpufreq is loaded as it matches both devices MODALIASes.
# modinfo acpi-cpufreq | grep alias
alias:          acpi
alias:          cpu:type:x86,ven*fam*mod*:feature:*00E8*
alias:          cpu:type:x86,ven*fam*mod*:feature:*0016*
alias:          acpi*:ACPI0007:*
alias:          acpi*:LNXCPU:*

On a system with 1536 logical CPUs, systemd-udevd attempts to load acpi-cpu=
freq
3072 times.

1536 * /devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0007:XXX
1536 * /devices/system/cpu/cpuXXX

The delay, caused by systemd-udevd attempting to load the driver, has a
significant impact on the startup time. It causes some devices to be
unavailable after reaching the root login prompt as it postpones the loadin=
g of
other drivers.

Each time that the driver is loaded it returns -EEXIST from acpi_cpufreq_in=
it.
static int __init acpi_cpufreq_init(void)
{
        int ret;

        if (acpi_disabled)
                return -ENODEV;

        /* don't keep reloading if cpufreq_driver exists */
        if (cpufreq_get_current_driver())
                return -EEXIST;
...

Changing the return value from -EEXIST to 0 when another driver exists prev=
ents
the driver from being loaded multiple times as kmod won't load a "live" mod=
ule.
Alternatively, blacklisting the driver (or disabling intel_pstate) prevents=
 the
issue as well. Below are the before and after startup times.

# systemd-analyze
Startup finished in 37.939s (kernel) + 10.909s (initrd) + 3min 55.004s (use=
rspace) =3D 4min 43.852s

# systemd-analyze
Startup finished in 38.307s (kernel) + 10.205s (initrd) + 38.312s (userspac=
e) =3D 1min 26.826s

Thank you,
Kyle Meyer