MIME-Version: 1.0
References: <20230217202234.32260-1-quic_johmoo@quicinc.com>
In-Reply-To: <20230217202234.32260-1-quic_johmoo@quicinc.com>
From:   Masahiro Yamada <masahiroy@kernel.org>
Date:   Sun, 19 Feb 2023 18:47:44 +0900
Message-ID: <CAK7LNASwzvsGPLWJ_QJFw+KjHQM9ZpaWqZFQNS60K9Kb9TU8Rw@mail.gmail.com>
Subject: Re: [PATCH RESEND 0/1] Validating UAPI backwards compatibility
To:     John Moon <quic_johmoo@quicinc.com>
Cc:     Nathan Chancellor <nathan@kernel.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Nicolas Schier <nicolas@fjasle.eu>,
        linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org,
        Trilok Soni <quic_tsoni@quicinc.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Bjorn Andersson <andersson@kernel.org>,
        Todd Kjos <tkjos@google.com>,
        Matthias Maennich <maennich@google.com>,
        Giuliano Procida <gprocida@google.com>,
        kernel-team@android.com, Jordan Crouse <jorcrous@amazon.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sat, Feb 18, 2023 at 5:23 AM John Moon <quic_johmoo@quicinc.com> wrote:
>
> + linux-arm-kernel and Greg KH
>
> Hi all,
>
> The kernel community has rigorously enforced a policy of backwards
> compatibility in its UAPI headers for a long time. This has allowed user
> applications to enjoy stability across kernel upgrades without
> recompiling.
>
> In the vendor driver community (out-of-tree modules), there's been a
> lack of discipline when it comes to maintaining UAPI backwards
> compatibility. This has been a maintenance burden and limits our options
> for long-term support of older devices.
>
> Our goal is to add tooling for vendor driver developers because the
> upstream model of expert maintainer code review can be difficult to
> replicate in-house. Tools may help developers catch simple UAPI
> incompatibilities that could be easily overlooked by in-house review.
>
> We see in the kernel documentation:
> "Kernel headers are backwards compatible, but not forwards compatible.
> This means that a program built against a C library using older kernel
> headers should run on a newer kernel (although it may not have access
> to new features), but a program built against newer kernel headers may
> not work on an older kernel."[1]
>
> How does the kernel enforce this guarantee? We would be interested to
> learn about any tools or methods used by kernel developers to make sure
> the above statement remains true.
>
> Could the documentation on UAPI maintenance (from a developer's point of
> view) be expanded? Internally, we have a set of guidelines for our kernel
> developers regarding UAPI compatibility techniques. If there's interest
> in supplying a document on this topic with the kernel, we'd be happy to
> submit a draft detailing what we have so far as a jumping off point.
>
> Additionally, I've attached a shell script we've been using internally
> to validate changes to our UAPI headers are backwards compatible. The
> script uses libabigail's[2] tool abidiff[3] to compare a modified
> header's ABI before and after a patch is applied. If an existing UAPI is
> modified, the script exits non-zero. We use this script in our CI system
> to block changes that fail the check.
>
> Currently, the script works with gcc. It generates output like this when
> a backwards-incompatible change is made to a UAPI header:
>
>  !!! ABI differences detected in include/uapi/linux/acct.h (compared to
>  file at HEAD^1) !!!
>
>      [C] 'struct acct' changed:
>        type size changed from 512 to 544 (in bits)
>        1 data member insertion:
>          '__u32 new_val', at offset 512 (in bits) at acct.h:71:1
>
>  0/1 UAPI header file changes are backwards compatible
>  UAPI header ABI check failed
>
> However, we have not had success with clang. It seems clang is more
> aggressive in optimizing dead code away (no matter which options we
> pass). Therefore, no ABI differences are found.
>
> We wanted to share with the community to receive feedback and any advice
> when it comes to tooling/policy surrounding this issue. Our hope is that
> the script will help all kernel UAPI authors (even those that haven't
> upstreamed yet) maintain good discipline and avoid breaking userspace.
>
> [1] Documentation/kbuild/headers_install.rst
> [2] https://sourceware.org/libabigail/manual/libabigail-overview.html
> [3] https://sourceware.org/libabigail/manual/abidiff.html
>
> P.S. While at Qualcomm, Jordan Crouse <jorcrous@amazon.com> authored the
> original version of the UAPI checker script. Thanks Jordan!


If you want to express the authorship of the original,
it is possible to add the "Co-developed-by" tag,
which is mentioned in
Documentation/translations/sp_SP/process/submitting-patches.rst


It depends on how much code you rewrote, though.


>
> John Moon (1):
>   check-uapi: Introduce check-uapi.sh
>
>  scripts/check-uapi.sh | 245 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 245 insertions(+)
>  create mode 100755 scripts/check-uapi.sh
>
>
> base-commit: 033c40a89f55525139fd5b6342281b09b97d05bf
> --
> 2.17.1
>


--
Best Regards
Masahiro Yamada