Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp5256105imm; Wed, 12 Sep 2018 03:30:08 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZicNgmt71pwYgqRWgxR1wG4+iMFSFu0Y7FG1CiknnNXBAiCFB9rSoD5bXOevA4L0oH3u9n X-Received: by 2002:a65:5144:: with SMTP id g4-v6mr1413123pgq.21.1536748208713; Wed, 12 Sep 2018 03:30:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536748208; cv=none; d=google.com; s=arc-20160816; b=YNHHa10aOmBCTDaJUSKLcmElQeJJ0aPGXDeHEG3seEMO6MoS2AF+aWSf+F9eI6cnv0 GJa0D2DgEcJSxJLrzdRmpuyUH3yflw2ZglhrIwYXWQ/+rSzGzTEUmY9ealBWHjhqnCmg +7zLqwCH6p0950x9J1BSPqT5xaahUuX9uv5i5DsZBj0tYnR/dOjwttsZmR901HyJOH8C 4sg0xW+HsEZIcwXZy60BW5b/RDjMYDF/4xQINPuU7PfgnTVZyqb6eSdFUEbPvz5SHng0 uWBLz91eslB6Zqf4t351iuqdYvR6FzWQIdIh/EdW41zzt7KNQDK3sz3K30LNc3MQHfxB vFxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=89jR/zikertNJxnMLg2JjlGIki4WroSUXGBIIdgyOBE=; b=fnBi78ft+tdbxI6jRORF6pQvlcazPjJzW7+q9qxCJKVYPpps9SlMYARA7ABZOz4RUr io7RSLIXqXvz6RH8QNHoKnTJVjG33aHjelJlj6xijOaI4ZJJKxGQW3xSd3vpLk2hS3Xm XNxuR4S/0YEEINVTNcuoY8lDUEu7bGtnnrEcdRy15q+brePg1gEG27HNdht6sE4j2DFl 46LRjy9/PZO9+V5mGBdyvYDB7BkcQixE6Ye61PU+qQOcvhui0YkP5grPGt0GI7r0mn7I zUCd1VJsSiETCqi8KWHDOUbTMK/4MNTJkqWitkUKNxaaJ3dYrIUiLVsf4p2Cgy90dXWh DXQw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d4-v6si628200pfc.219.2018.09.12.03.29.53; Wed, 12 Sep 2018 03:30:08 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727673AbeILPdk (ORCPT + 99 others); Wed, 12 Sep 2018 11:33:40 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57158 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726816AbeILPdk (ORCPT ); Wed, 12 Sep 2018 11:33:40 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F2CBC7A9; Wed, 12 Sep 2018 03:29:45 -0700 (PDT) Received: from [10.4.12.81] (melchizedek.emea.arm.com [10.4.12.81]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3C8693F614; Wed, 12 Sep 2018 03:29:43 -0700 (PDT) Subject: Re: [PATCH v5 03/27] arm64: alternative: Apply alternatives early in boot process To: Julien Thierry , Suzuki K Poulose Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, daniel.thompson@linaro.org, joel@joelfernandes.org, marc.zyngier@arm.com, mark.rutland@arm.com, christoffer.dall@arm.com, catalin.marinas@arm.com, will.deacon@arm.com References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> <1535471497-38854-4-git-send-email-julien.thierry@arm.com> From: James Morse Message-ID: <3becf020-b230-beb8-b304-d8097377f234@arm.com> Date: Wed, 12 Sep 2018 11:29:41 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <1535471497-38854-4-git-send-email-julien.thierry@arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Julien, On 28/08/18 16:51, Julien Thierry wrote: > From: Daniel Thompson > > Currently alternatives are applied very late in the boot process (and > a long time after we enable scheduling). Some alternative sequences, > such as those that alter the way CPU context is stored, must be applied > much earlier in the boot sequence. > > Introduce apply_boot_alternatives() to allow some alternatives to be > applied immediately after we detect the CPU features of the boot CPU. > diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c > index b5d6039..70c2604 100644 > --- a/arch/arm64/kernel/alternative.c > +++ b/arch/arm64/kernel/alternative.c > @@ -145,7 +145,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end) > } while (cur += d_size, cur < end); > } > > -static void __apply_alternatives(void *alt_region, bool is_module) > +static void __apply_alternatives(void *alt_region, bool is_module, > + unsigned long feature_mask) Shouldn't feature_mask be a DECLARE_BITMAP() maybe-array like cpu_hwcaps? This means it keeps working when NR_CAPS grows over 64, which might happen sooner than we think for backported errata... > @@ -155,6 +156,9 @@ static void __apply_alternatives(void *alt_region, bool is_module) > for (alt = region->begin; alt < region->end; alt++) { > int nr_inst; > > + if ((BIT(alt->cpufeature) & feature_mask) == 0) > + continue; > + > /* Use ARM64_CB_PATCH as an unconditional patch */ > if (alt->cpufeature < ARM64_CB_PATCH && > !cpus_have_cap(alt->cpufeature)) > @@ -213,7 +217,7 @@ static int __apply_alternatives_multi_stop(void *unused) > isb(); > } else { > BUG_ON(alternatives_applied); > - __apply_alternatives(®ion, false); > + __apply_alternatives(®ion, false, ~boot_capabilities); Ah, this is tricky. There is a bitmap_complement() for the DECLARE_BITMAP() stuff, but we'd need a second array... We could pass the scope around, but then __apply_alternatives() would need to lookup the struct arm64_cpu_capabilities up every time. This is only a problem as we have one cap-number-space for errata/features, but separate sparse lists. (I think applying the alternatives one cap at a time is a bad idea as we would need to walk the alternative region NR_CAPS times) > @@ -227,6 +231,24 @@ void __init apply_alternatives_all(void) > stop_machine(__apply_alternatives_multi_stop, NULL, cpu_online_mask); > } > > +/* > + * This is called very early in the boot process (directly after we run > + * a feature detect on the boot CPU). No need to worry about other CPUs > + * here. > + */ > +void __init apply_boot_alternatives(void) > +{ > + struct alt_region region = { > + .begin = (struct alt_instr *)__alt_instructions, > + .end = (struct alt_instr *)__alt_instructions_end, > + }; > + > + /* If called on non-boot cpu things could go wrong */ > + WARN_ON(smp_processor_id() != 0); Isn't the problem if there are multiple CPUs online? > + __apply_alternatives(®ion, false, boot_capabilities); > +} > + > #ifdef CONFIG_MODULES > void apply_alternatives_module(void *start, size_t length) > { > diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c > index 3bc1c8b..0d1e41e 100644 > --- a/arch/arm64/kernel/cpufeature.c > +++ b/arch/arm64/kernel/cpufeature.c > @@ -52,6 +52,8 @@ > DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS); > EXPORT_SYMBOL(cpu_hwcaps); > > +unsigned long boot_capabilities; > + > /* > * Flag to indicate if we have computed the system wide > * capabilities based on the boot time active CPUs. This > @@ -1375,6 +1377,9 @@ static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps, > if (!cpus_have_cap(caps->capability) && caps->desc) > pr_info("%s %s\n", info, caps->desc); > cpus_set_cap(caps->capability); Hmm, the bitmap behind cpus_set_cap() is what cpus_have_cap() in __apply_alternatives() looks at. If you had a call to __apply_alternatives after update_cpu_capabilities(SCOPE_BOOT_CPU), but before any others, it would only apply those alternatives... (I don't think there is a problem re-applying the same alternative, but I haven't checked). > + > + if (caps->type & SCOPE_BOOT_CPU) > + __set_bit(caps->capability, &boot_capabilities); > } > } Thanks, James