Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220511164514.2741934-1-masahiroy@kernel.org> <20220511164514.2741934-4-masahiroy@kernel.org>
In-Reply-To: <20220511164514.2741934-4-masahiroy@kernel.org>
From:   Masahiro Yamada <masahiroy@kernel.org>
Date:   Fri, 13 May 2022 01:28:26 +0900
Message-ID: <CAK7LNATt7iFAHqxqW=5GggJJ_k9mXuDmSEkNyyJKG3c4MkxK_w@mail.gmail.com>
Subject: Re: [PATCH v5 03/12] modpost: extract symbol versions from *.cmd files
To:     Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Nathan Chancellor <nathan@kernel.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Nicolas Schier <nicolas@fjasle.eu>,
        Peter Zijlstra <peterz@infradead.org>,
        linux-modules <linux-modules@vger.kernel.org>,
        clang-built-linux <llvm@lists.linux.dev>,
        Ard Biesheuvel <ardb@kernel.org>,
        Sami Tolvanen <samitolvanen@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, May 12, 2022 at 1:49 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Currently, CONFIG_MODVERSIONS needs extra link to embed the symbol
> versions into ELF objects. Then, modpost extracts the version CRCs
> from them.
>
> The following figures show how it currently works, and how I am trying
> to change it.
>
> Current implementation
> ======================
>                                                            |----------|
>                  embed CRC      -------------------------->| final    |
>        $(CC)       $(LD)       /  |---------|              | link for |
>        -----> *.o -------> *.o -->| modpost |              | vmlinux  |
>       /              /            |         |-- *.mod.c -->| or       |
>      / genksyms     /             |---------|              | module   |
>   *.c ------> *.symversions                                |----------|
>
> Genksyms outputs the calculated CRCs in the form of linker script
> (*.symversions), which is used by $(LD) to update the object.
>
> If CONFIG_LTO_CLANG=y, the build process is much more complex. Embedding
> the CRCs is postponed until the LLVM bitcode is converted into ELF,
> creating another intermediate *.prelink.o.
>
> However, this complexity is unneeded. There is no reason why we must
> embed version CRCs in objects so early.
>
> There is final link stage for vmlinux (scripts/link-vmlinux.sh) and
> modules (scripts/Makefile.modfinal). We can link CRCs at the very last
> moment.
>
> New implementation
> ==================
>                                                            |----------|
>                    --------------------------------------->| final    |
>        $(CC)      /    |---------|                         | link for |
>        -----> *.o ---->|         |                         | vmlinux  |
>       /                | modpost |--- .vmlinux.export.c -->| or       |
>      / genksyms        |         |--- *.mod.c ------------>| module   |
>   *.c ------> *.cmd -->|---------|                         |----------|
>
> Pass the symbol versions to modpost as separate text data, which are
> available in *.cmd files.
>
> This commit changes modpost to extract CRCs from *.cmd files instead of
> from ELF objects.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
> Tested-by: Nathan Chancellor <nathan@kernel.org>
> Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
> ---
>
> (no changes since v2)
>
> Changes in v2:
>   - Simplify the implementation (parse .cmd files after ELF)
>
>  scripts/mod/modpost.c | 177 ++++++++++++++++++++++++++++++------------
>  1 file changed, 129 insertions(+), 48 deletions(-)


> +/*
> + * The CRCs are recorded in .*.cmd files in the form of:
> + * #SYMVER <name> <crc>
> + */
> +static void extract_crcs_for_object(const char *object, struct module *mod)
> +{
> +       char cmd_file[PATH_MAX];
> +       char *buf, *p;
> +       const char *base;
> +       int dirlen, ret;
> +
> +       base = strrchr(object, '/');
> +       if (base) {
> +               base++;
> +               dirlen = base - object;
> +       } else {
> +               dirlen = 0;
> +               base = object;
> +       }
> +
> +       ret = snprintf(cmd_file, sizeof(cmd_file), "%.*s.%s.cmd",
> +                      dirlen, object, base);
> +       if (ret >= sizeof(cmd_file)) {
> +               error("%s: too long path was truncated\n", cmd_file);
> +               return;
> +       }
> +
> +       buf = read_text_file(cmd_file);
> +       p = buf;
> +
> +       while ((p = strstr(p, "\n#SYMVER "))) {
> +               char *name;
> +               size_t namelen;
> +               unsigned int crc;
> +               struct symbol *sym;
> +
> +               name = p + strlen("\n#SYMVER ");
> +
> +               p = strchr(name, ' ');
> +               if (!p)
> +                       break;
> +
> +               namelen = p - name;
> +               p++;
> +
> +               if (!isdigit(*p))
> +                       continue;       /* skip this line */
> +
> +               crc = strtol(p, &p, 0);
> +               if (*p != '\n')
> +                       continue;       /* skip this line */
> +
> +               name[namelen] = '\0';
> +
> +               sym = sym_find_with_module(name, mod);
> +               if (!sym) {
> +                       warn("Skip the version for unexported symbol \"%s\" [%s%s]\n",
> +                            name, mod->name, mod->is_vmlinux ? "" : ".ko");
> +                       continue;
> +               }


A lot of warnings are displayed when CONFIG_TRIM_UNUSED_KSYMS=y.

I will fix this in v6.


-- 
Best Regards
Masahiro Yamada