Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8de014c1-aa63-5783-e5fd-53b7fdece805@nvidia.com>
Date:   Mon, 18 Jul 2022 14:52:52 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH bpf-next v9 3/4] bpf, arm64: Implement
 bpf_arch_text_poke() for arm64
Content-Language: en-US
To:     Xu Kuohai <xukuohai@huawei.com>, bpf@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        netdev@vger.kernel.org,
        Jean-Philippe Brucker <jean-philippe@linaro.org>,
        Will Deacon <will@kernel.org>, KP Singh <kpsingh@kernel.org>
Cc:     Mark Rutland <mark.rutland@arm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Alexei Starovoitov <ast@kernel.org>,
        Andrii Nakryiko <andrii@kernel.org>,
        Zi Shen Lim <zlim.lnx@gmail.com>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        "David S . Miller" <davem@davemloft.net>,
        Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
        David Ahern <dsahern@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H . Peter Anvin" <hpa@zytor.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        Russell King <rmk+kernel@armlinux.org.uk>,
        James Morse <james.morse@arm.com>,
        Hou Tao <houtao1@huawei.com>,
        Jason Wang <wangborong@cdjrlc.com>
References: <20220711150823.2128542-1-xukuohai@huawei.com>
 <20220711150823.2128542-4-xukuohai@huawei.com>
From:   Jon Hunter <jonathanh@nvidia.com>
In-Reply-To: <20220711150823.2128542-4-xukuohai@huawei.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QTBtK1lyRUFOVDFkZ3RCQnlNMm5BUXJyS2R1Rkcwc1NYUWpGSGppTWhtYndz?=
 =?utf-8?B?eGpkSVB2WDFVZGxBRXF0UVFVdU5UUmxYOGRwOXMxRDNxeENPUFJTaXdNWE5i?=
 =?utf-8?B?eWVlKzNsd0lDRXA1UWJGR0ZsWjJ4eUR6dEZ5VEF0MDVUTnIrZW9vajhjSHlR?=
 =?utf-8?B?TkFWQU5uK2ZaNFNoVWxzRE5IKys5RHNMMTZnbTJCQnl6VkhmODJBU2pLVE1m?=
 =?utf-8?B?eXdHWHJROUVJbDYzZ0RkUEZ0Qk9TTGxFRTFKNWFxZkNPYXFSUXk1VUlveDdr?=
 =?utf-8?B?UFN3QXFYWHhLa2NSb0JvSFJCQlU3MkVpRkhSRzljRU51VDN5Qno4OHozOEx1?=
 =?utf-8?B?Q0ROM0xYRTVxNi9rU0EyMnJjRC9YMUdiVDBrK3ZQUElvU0VoSXhrT0VXT1Fk?=
 =?utf-8?B?NnNwbk1ZQWpnVjRDV0d1THBrT01HTUVWVGpXM3VMd1hKdXkweUt5aTJIbTlV?=
 =?utf-8?B?ZWpoRkxnRHd6cWpPeDNzUWdvZ2h1czV3N3ppTWhsZWpmYmZ5OXlTc3Ira1Ra?=
 =?utf-8?B?K3d5YkdjbVZ4YjYycE5MUDZoenZucytMYi9pSENSQnVzVk1iTlBUZ2V4amtW?=
 =?utf-8?B?NXA1MHpXdVNXR1VMRUxtanJ4UVRIS3A5ZXcraEljS0s4WjdSS09kNEFwaDB2?=
 =?utf-8?B?MklvbkI1bGJHUlozcmZDanV2bHRaSlhhU2hGdWQ2eWZSTHl3VHIrNENqTXNu?=
 =?utf-8?B?b0NBSXljakxXWmt3Rjl5b2FpaE1MTFRtTHJob0RGRWIvdXd0YmFoMCtybE9K?=
 =?utf-8?B?QVhDVUZnRzV3NGNIOTRmTTIrYXc1dUJnbHN6TCtManZCa29BNXRNNTMvRjJy?=
 =?utf-8?B?TG1oZnM4cU1aRmFqRnlBNnZHMFZYRjVGVFVTdTVlMFRDVCtiQWhqWmlnRGta?=
 =?utf-8?B?S3hBLy83b1FWQWxuNU5rMzNSUi9rNDhnUlhGWHJiQ1pTNnprN3kzd2xkQU9P?=
 =?utf-8?B?SmRFUU0wY0VMcy9VYTJPcXBQWitiMExNRGZrUjJISHgydjdldDJUZkxnaXJT?=
 =?utf-8?B?MVlZZWNUdmR4dFhrcEQxQ0FQVGliRVQva0hsejJxM2srcmY3K0pnK2hCUk9K?=
 =?utf-8?B?TVg1YWo4NytCNXZBKzJtczIxZWFmclBVcGJJNGVyU0h4allVWnpMdGY1L1Ru?=
 =?utf-8?B?d01mSlR6a0NaUjI2MEJzaGNneGltWGNjWGxhdkljV1dRNHF6bldpV1N5bkRL?=
 =?utf-8?B?WDNob0FwdjladUtFY0Y2Y2QzZHRHdEJ6MUFmTUhyWkxkeTlTd3pXTzM0eElx?=
 =?utf-8?B?NDBVdkdSOVFYK05xRVZ2YnB5MXYzcHBmWlpXRGRRTmxxY1dqVitUbnRQR0dZ?=
 =?utf-8?B?TXhhUTVBamxJQ3RTRGluVVN4MUl2bnBQT0tlWWhjNGJhQk5QL2NzczEvQlBD?=
 =?utf-8?B?QnBGcUszbmZzYmlxbXdXZlJqTk5RV0hsOTlScUlmOElwMlkyTlNBdWQ2My9s?=
 =?utf-8?B?M3E2L2s3eEFacXVOSmQ3TWh6bEJrTGFZMU9IanpjbEkvMVQxMVN4aGY0Q3J3?=
 =?utf-8?B?eXNWUE1RUXVMN3ByeEJ3cmRaWEpPbXJYVXdwV1FBYjN1aElFWHc0ZmtWbVhu?=
 =?utf-8?B?TTNtQUpYemJJbnoybkhueE9rZzVxcFNCWEF5WDVIQlVoSW5RcHdDL1dobnVq?=
 =?utf-8?B?VDhUWnR6M0FvYnlXUkZ1YjRoV283WjJ2eG11UkNFeG9SL28rYzNzaWhHTldx?=
 =?utf-8?B?cThtWjZjVmxoYjh1ZERqd1hHcjZBTHQvTDNFUWplYlRDWExhRlluVnhKc054?=
 =?utf-8?B?S3Y2Z0NnOFpLMTExR1lpbU1rKzN1S29HVVlGUVYwOU1wejYrcEo5b0MySlRG?=
 =?utf-8?B?SDd0LzcrbWl3UGk4SEgwdm1MM0FZeGI3U2N3QWlIYlRnRFBYVjc2bU5PK1Bs?=
 =?utf-8?B?VUVMY1NLSS9tcTRqa0Z3bHlWZ2pnRXVyOW9qYVpFd0lsUm16TDV6VXJkQmZM?=
 =?utf-8?B?d1dpTzMxdEp5WG40KzhERkFHcHV2Qm1VOStidFVCbGQwUmNLZmprL2xkSEtH?=
 =?utf-8?B?TEhudFd3eVppN0U5M1l5MU9EZXhQMVRHM3pwbGdCbXBJTDJiOTJQOGxMc0hO?=
 =?utf-8?B?L3htZHp4THU5bnRyNCtNY2d5c1VoS00zVmp6VjZkM1lGZ0YxYnNwbWZmdEdD?=
 =?utf-8?B?YXMxNlk4ZFhtMEw5Zm1BTVJsMHYwRXMyOVNOczBRWUpKOHVmVGtRdnFhenlt?=
 =?utf-8?B?Snc9PQ==?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9e1f25c6-4e81-4f4d-2a73-08da68c4d4d4
X-MS-Exchange-CrossTenant-AuthSource: CO6PR12MB5444.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jul 2022 13:53:02.3180
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /GBVA5Z1MBbF6GnZQJ6pGv+WtRCD56HPGOHNHacOPNUYRd4c9rG43bCP9vSNeuSVGXJz2v6+jTxjK4PRKQy3eQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB2654
X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 11/07/2022 16:08, Xu Kuohai wrote:
> Implement bpf_arch_text_poke() for arm64, so bpf prog or bpf trampoline
> can be patched with it.
> 
> When the target address is NULL, the original instruction is patched to a
> NOP.
> 
> When the target address and the source address are within the branch
> range, the original instruction is patched to a bl instruction to the
> target address directly.
> 
> To support attaching bpf trampoline to both regular kernel function and
> bpf prog, we follow the ftrace patchsite way for bpf prog. That is, two
> instructions are inserted at the beginning of bpf prog, the first one
> saves the return address to x9, and the second is a nop which will be
> patched to a bl instruction when a bpf trampoline is attached.
> 
> However, when a bpf trampoline is attached to bpf prog, the distance
> between target address and source address may exceed 128MB, the maximum
> branch range, because bpf trampoline and bpf prog are allocated
> separately with vmalloc. So long jump should be handled.
> 
> When a bpf prog is constructed, a plt pointing to empty trampoline
> dummy_tramp is placed at the end:
> 
>          bpf_prog:
>                  mov x9, lr
>                  nop // patchsite
>                  ...
>                  ret
> 
>          plt:
>                  ldr x10, target
>                  br x10
>          target:
>                  .quad dummy_tramp // plt target
> 
> This is also the state when no trampoline is attached.
> 
> When a short-jump bpf trampoline is attached, the patchsite is patched to
> a bl instruction to the trampoline directly:
> 
>          bpf_prog:
>                  mov x9, lr
>                  bl <short-jump bpf trampoline address> // patchsite
>                  ...
>                  ret
> 
>          plt:
>                  ldr x10, target
>                  br x10
>          target:
>                  .quad dummy_tramp // plt target
> 
> When a long-jump bpf trampoline is attached, the plt target is filled with
> the trampoline address and the patchsite is patched to a bl instruction to
> the plt:
> 
>          bpf_prog:
>                  mov x9, lr
>                  bl plt // patchsite
>                  ...
>                  ret
> 
>          plt:
>                  ldr x10, target
>                  br x10
>          target:
>                  .quad <long-jump bpf trampoline address>
> 
> dummy_tramp is used to prevent another CPU from jumping to an unknown
> location during the patching process, making the patching process easier.
> 
> The patching process is as follows:
> 
> 1. when neither the old address or the new address is a long jump, the
>     patchsite is replaced with a bl to the new address, or nop if the new
>     address is NULL;
> 
> 2. when the old address is not long jump but the new one is, the
>     branch target address is written to plt first, then the patchsite
>     is replaced with a bl instruction to the plt;
> 
> 3. when the old address is long jump but the new one is not, the address
>     of dummy_tramp is written to plt first, then the patchsite is replaced
>     with a bl to the new address, or a nop if the new address is NULL;
> 
> 4. when both the old address and the new address are long jump, the
>     new address is written to plt and the patchsite is not changed.
> 
> Signed-off-by: Xu Kuohai <xukuohai@huawei.com>
> Acked-by: Song Liu <songliubraving@fb.com>
> Reviewed-by: Jakub Sitnicki <jakub@cloudflare.com>
> Reviewed-by: KP Singh <kpsingh@kernel.org>
> Reviewed-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>   arch/arm64/net/bpf_jit.h      |   7 +
>   arch/arm64/net/bpf_jit_comp.c | 329 ++++++++++++++++++++++++++++++++--
>   2 files changed, 322 insertions(+), 14 deletions(-)


This change appears to be causing the build to fail ...

/tmp/cc52xO0c.s: Assembler messages:
/tmp/cc52xO0c.s:8: Error: operand 1 should be an integer register -- `mov lr,x9'
/tmp/cc52xO0c.s:7: Error: undefined symbol lr used as an immediate value
make[2]: *** [scripts/Makefile.build:250: arch/arm64/net/bpf_jit_comp.o] Error 1
make[1]: *** [scripts/Makefile.build:525: arch/arm64/net] Error 2

Let me know if you have any thoughts.

Cheers
Jon

-- 
nvpublic