Received: by 2002:a25:ab43:0:0:0:0:0 with SMTP id u61csp674036ybi; Fri, 24 May 2019 09:34:56 -0700 (PDT) X-Google-Smtp-Source: APXvYqyBqHXwIa66WOWU8BnGJpulinyGhenDkzKv1aHu8hGsdF77rr4aJnFMSE+ehLnk5fxGmeoB X-Received: by 2002:a63:5964:: with SMTP id j36mr105555495pgm.384.1558715695998; Fri, 24 May 2019 09:34:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558715695; cv=none; d=google.com; s=arc-20160816; b=a4N15lxAmyvYq0tDQuAMBzRjlA8AllkIRwhBx2csN51Wb0EnnzizwuFuzyvRmPySKu qCRvekSH6Lql7gyE/Umf4OpVI6fsVbvQZCx8lgI/z51vOeSBoIXGUMeqOHpGr78+NBS0 zkm45bpOJkeADXYaXHh4LGdmWM8m59wSaIkQ8BHoGrehaSdjMW+WLCzTW80Dt/7m2nlw 5sUWZ64yZXFPKONnbo+m1WViIk8+OFPsMlq/Z3mkgqx+GUcJbY9EHg58jZvTQ0BsGo0O idbkWeNf7ygp9GLFoGeW4j2Z+3fdr2fX0gMUGuk6jsjdfs4XC8BL5Mx/qwgNn/tHjWV8 an/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=LRCKBM+pCyV8v4GH3CTPRapygJGCpjV7JmJ/J98K3zo=; b=tHYXh3u1fGG0AEAzXp95jpo2a6hzRNsrTRvnaOtAfHL5YAICusQjYddKPwW1R6bbi6 ohbLztLRKifldK7vevWfwODsiiQq3W8WrFXXoih93sqck1rs/6y+SadsrW/gOf7fuYQ0 7oCEB7tnphjma47ujGAt2FVzp2N7BQwOzOpmNGmnt+zw99HXS6aq4UkpkEPq1V2tbWs3 xg4gXO7DKC21YmJjWlof4mt4lNOTX27OSd0djPb17JsAHcKnLeCqUkkcmuql5g+3ZDer YOv6Ob3+lFGc262KedDUUIm2ldyqY7rjJu9dmUGIfIc/g0FdGcXT8chJkBioToGx065l ACPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="J8z/TZmJ"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z9si2661364pgh.21.2019.05.24.09.34.39; Fri, 24 May 2019 09:34:55 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="J8z/TZmJ"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390504AbfEXQd2 (ORCPT + 99 others); Fri, 24 May 2019 12:33:28 -0400 Received: from mail-io1-f67.google.com ([209.85.166.67]:41533 "EHLO mail-io1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390021AbfEXQd2 (ORCPT ); Fri, 24 May 2019 12:33:28 -0400 Received: by mail-io1-f67.google.com with SMTP id a17so8228175iot.8 for ; Fri, 24 May 2019 09:33:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LRCKBM+pCyV8v4GH3CTPRapygJGCpjV7JmJ/J98K3zo=; b=J8z/TZmJDy9YhRyf5wi57/Kbe3Klzj5MlVvfu+WQ0Us/zFKhzhZMdYRSLiR8QfVlnM BhDMb3r2G2EioJINzsJPaiotxjiclp4s6knjqW64m7lRSr9W4kKERVv6Vt9qDWLmkZEC DFC5hKq4rIB42faqdX5EOqON2w72flSnxf4fXm6naoDbOyRqRaadxGjakJo0iiw4neqE fsE6hQQqB/qKs8lbcTBiLOA1utGAPh588nwv9vt+o1l5nguN8NTG8ytSQewhOvXP+0GA rdtkiZ/2JRRh7q47q79drEmhjNNQaLy04JdlpB1t3pkW2U6WYoU2/PjqiJb8+jLjOUJz ZBNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LRCKBM+pCyV8v4GH3CTPRapygJGCpjV7JmJ/J98K3zo=; b=UanrpAnRbn7MZBG1AGv5o2Mxx/FHy56hfpXMUIXz+eDeDVopvUBL8YzdFYVIv+CWLd toLa86d8KKMPbCzUwk1NRztDWG5JuPqqf7UsuC565CfkfJ4mDiiG9b/M11/f4uXf1XPZ ePiVHrfuUYmbNbceKSJslfMacgmV61OJwl/KnpMmYh5RdhhzFHHuoncwUhg/UFVRqj1t lxMc+qwuVg8HOj5zPs62e7owtnBsDvuzea8nvyMeTeZFYsDVKpU0WRA9Ftlv0V1F8Iw0 TYRfVK1j8Q/6qc/rC0sJfgAvNE65WbVO16aC97yX35sx+IRFTJ/nB3jEd+ilb8P9V1PA lC4A== X-Gm-Message-State: APjAAAWyfk1x3ChqB5/bhZ6+yuvCoIm9AmsuQZf62aUFUMIKXOfxNiKI E1wI3nIpOShZRsMGOgTI0Clt5xYb512kQgZ4HWoJlw== X-Received: by 2002:a5d:968e:: with SMTP id m14mr26693205ion.49.1558715607305; Fri, 24 May 2019 09:33:27 -0700 (PDT) MIME-Version: 1.0 References: <20190522150239.19314-1-ard.biesheuvel@arm.com> <293c9d0f-dc14-1413-e4b4-4299f0acfb9e@arm.com> <20190523091811.GA26646@fuggles.cambridge.arm.com> <907a9681-cd1d-3326-e3dd-5f6965497720@arm.com> <20190524152045.w3syntzp4bb5jb7u@treble> <20190524163104.o6xh54x4ngbihneb@treble> In-Reply-To: <20190524163104.o6xh54x4ngbihneb@treble> From: Ard Biesheuvel Date: Fri, 24 May 2019 18:33:15 +0200 Message-ID: Subject: Re: [PATCH] module/ksymtab: use 64-bit relative reference for target symbol To: Josh Poimboeuf Cc: Ard Biesheuvel , Mark Rutland , linux-arch , Arnd Bergmann , guillaume.gardet@arm.com, Marc Zyngier , "the arch/x86 maintainers" , Will Deacon , Linux Kernel Mailing List , James Morse , Jessica Yu , Ingo Molnar , linux-arm-kernel Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 24 May 2019 at 18:31, Josh Poimboeuf wrote: > > On Fri, May 24, 2019 at 05:55:37PM +0200, Ard Biesheuvel wrote: > > On Fri, 24 May 2019 at 17:21, Josh Poimboeuf wrote: > > > > > > On Thu, May 23, 2019 at 10:29:39AM +0100, Ard Biesheuvel wrote: > > > > > > > > > > > > On 5/23/19 10:18 AM, Will Deacon wrote: > > > > > On Thu, May 23, 2019 at 09:41:40AM +0100, Ard Biesheuvel wrote: > > > > > > > > > > > > > > > > > > On 5/22/19 5:28 PM, Ard Biesheuvel wrote: > > > > > > > > > > > > > > > > > > > > > On 5/22/19 4:02 PM, Ard Biesheuvel wrote: > > > > > > > > The following commit > > > > > > > > > > > > > > > > 7290d5809571 ("module: use relative references for __ksymtab entries") > > > > > > > > > > > > > > > > updated the ksymtab handling of some KASLR capable architectures > > > > > > > > so that ksymtab entries are emitted as pairs of 32-bit relative > > > > > > > > references. This reduces the size of the entries, but more > > > > > > > > importantly, it gets rid of statically assigned absolute > > > > > > > > addresses, which require fixing up at boot time if the kernel > > > > > > > > is self relocating (which takes a 24 byte RELA entry for each > > > > > > > > member of the ksymtab struct). > > > > > > > > > > > > > > > > Since ksymtab entries are always part of the same module as the > > > > > > > > symbol they export (or of the core kernel), it was assumed at the > > > > > > > > time that a 32-bit relative reference is always sufficient to > > > > > > > > capture the offset between a ksymtab entry and its target symbol. > > > > > > > > > > > > > > > > Unfortunately, this is not always true: in the case of per-CPU > > > > > > > > variables, a per-CPU variable's base address (which usually differs > > > > > > > > from the actual address of any of its per-CPU copies) could be at > > > > > > > > an arbitrary offset from the ksymtab entry, and so it may be out > > > > > > > > of range for a 32-bit relative reference. > > > > > > > > > > > > > > > > > > > > (Apologies for the 3-act monologue) > > > > > > > > > > Exposition, development and recapitulation ;) > > > > > > > > > > > This turns out to be incorrect. The symbol address of per-CPU variables > > > > > > exported by modules is always in the vicinity of __per_cpu_start, and so it > > > > > > is simply a matter of making sure that the core kernel is in range for > > > > > > module ksymtab entries containing 32-bit relative references. > > > > > > > > > > > > When running the arm64 with kaslr enabled, we currently randomize the module > > > > > > space based on the range of ADRP/ADD instruction pairs, which have a -/+ 4 > > > > > > GB range rather than the -/+ 2 GB range of 32-bit place relative data > > > > > > relocations. So we can fix this by simply reducing the randomization window > > > > > > to 2 GB. > > > > > > > > > > Makes sense. Do you see the need for an option to disable PREL relocs > > > > > altogether in case somebody wants the additional randomization range? > > > > > > > > > > > > > No, not really. To be honest, I don't think > > > > CONFIG_RANDOMIZE_MODULE_REGION_FULL is that useful to begin with, and the > > > > only reason we enabled it by default at the time was to ensure that the PLT > > > > code got some coverage after we introduced it. > > > > > > In code, percpu variables are accessed with absolute relocations, right? > > > > No, they are accessed just like ordinary symbols, so PC32 references > > on x86 or ADRP/ADD references on arm64 are both quite common. > > Ah, right, now I see some PC32 percpu references. > > So if PC32 references are sufficient for code, why aren't they > sufficient for ksymtab entries? Isn't the ksymtab data address closer > to the percpu data than the code? Do you have an example of an out of > range ksymtab reference? > Not on x86, only on arm64, which uses ADRP/ADD pairs with a -/+ 4 GB range as opposed to the -/+ 2 GB range of PC32 and PREL32 references. So when KASLR puts the modules far away from the kernel (but in range for ADRP/ADD) they may be out of range for PREL32.