Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Wed, 16 Feb 2022 09:28:49 +0000
From:   Huang Shijie <shijie@os.amperecomputing.com>
To:     Will Deacon <will@kernel.org>
Cc:     catalin.marinas@arm.com, bhe@redhat.com, vgoyal@redhat.com,
        dyoung@redhat.com, corbet@lwn.net, kexec@lists.infradead.org,
        linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
        linux-kernel@vger.kernel.org, zwang@amperecomputing.com,
        patches@amperecomputing.com, darren@os.amperecomputing.com
Subject: Re: [PATCH] arm64: crash_core: Export MODULES, VMALLOC, and VMEMMAP
 ranges
Message-ID: <YgzD0ZfZF34u6whJ@hsj>
References: <20220209092642.9181-1-shijie@os.amperecomputing.com>
 <20220215164423.GB8458@willie-the-truck>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220215164423.GB8458@willie-the-truck>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?MV701y4AdQ67KWHFRbe3yn83YVOrVWXCL8AoytidRyVAWTSHvWm86y4hzlo0?=
 =?us-ascii?Q?UnSQDKtDXLqjIJgzFcSxi3GANDn3FqhzJtqlWMwrjJZxQtEKLqvFpR+dpRkU?=
 =?us-ascii?Q?qSj1LtrNd/gB2uQvVfuF/kHSKFLHMl33IOXxgwcLJN5InCZTfrbayVOAo+eb?=
 =?us-ascii?Q?+TW5z5YLJCKLFI5ulz8W+CweZzwP1aUstiEPL9O/dnz17RqL4jlTDV/ujJdY?=
 =?us-ascii?Q?hzUNmE0+lEFUeKTBnqOzVNbiOclsDbZ5blcTeZRGAaMYNv2fS/hPnu2X27sZ?=
 =?us-ascii?Q?kFKlA+Gkrnda73DEnM3/ZvXaCNEc+eB1/20CtEgNkkq9+TAKaqCcY+FfoIxy?=
 =?us-ascii?Q?S4LORgZaTiIw65JivYV5glJhXXCQ8ANOj3kH3GC3mcuI0nP9UZc9kDsZBxkT?=
 =?us-ascii?Q?MQJBEXqpiF19WyX1Bs0FB3qNrotAhRT1kn8znUL+Bq1pDp8F8Uo5GIqvstX+?=
 =?us-ascii?Q?RKuezfhsRIG8+zqQzdkTuIlb0RyIEShQ/MXMwNtxHFz1HJykWzPO5NFbSNAl?=
 =?us-ascii?Q?0UIS0cISutQV9ZJNU6HByIHVaa+3k1T+zNszMXhB/L5X2buNhb4hVBOcYscz?=
 =?us-ascii?Q?vc+jrkEdH/eXdws010Hl2sjW6BbzAlOu5j43IVTUT7ehnNpkLiRIr3I6Wi9w?=
 =?us-ascii?Q?N/IrQUqEdq+S5SudHaJeFY5D296ebb68O6ZtMr1baDCwQTxFF4h/Iom2g/Kf?=
 =?us-ascii?Q?hTmX3KePU4BdzMatpz4rw1VKsbLujP0DFAUsOMiTvYx03so/BN2t6Z2F/mlh?=
 =?us-ascii?Q?g5V6oHFEOBRPiAD8vT7wdKcnB96g6EkG6TL9F5RjRHG/Cq/HMeHOzlD3O26p?=
 =?us-ascii?Q?3mGHsqvtcbFwKYnz36oyFA2uhLsKIP7X7BEnpQ4FwWXZps0dqQTAk5BujOnj?=
 =?us-ascii?Q?bF5QPnL4Ut+j+/QoKTBCLkIPv0PZcgwoj39VTiR/kAnCC2kHlaCIWRUr//A4?=
 =?us-ascii?Q?K8WLpVJuCfU23WIcSGZRFhxBouWHSNjzb2esEwqoC4qZ/m5Gcf27rfYYa5DM?=
 =?us-ascii?Q?8orQFVE9GD/L9rORqRMrwOy8c7l40CJyJmaeeQkG922QM9q9/A2+XKf0pFGq?=
 =?us-ascii?Q?6hBqRnwfF8gfb6eLPrkrG3jLnAwDglc+fqCy5zUSn2tAxoAEwb1DxbnHUu9f?=
 =?us-ascii?Q?kiUWg/NDEqXWFaUitI+qar10fPa4PgHE9665jsybmZadqpqOpHVvIs8T2HJT?=
 =?us-ascii?Q?64WVpxpZRHNaM+nu9rbG/6tiVTS6ygeymRC1Tz9WsFNd0rgOWJypSwjRXYGX?=
 =?us-ascii?Q?KGUM7M6ut+sB/FeLwfeaRDIS3mqsDJSzAgOVyK9Eh+XlsBAPoJ79DZT3HBfZ?=
 =?us-ascii?Q?W5O94xQypsPRwUGe5DQ0Ext8xj+s7MU8cGbmgBYg9IHh4QEMIhA2eqg1MB49?=
 =?us-ascii?Q?xZSxx9kA9DdpVqvJR6y/pojPxyLYzFZ/KNWVvaeLv6bgfLp7/UYp19z2aJ0U?=
 =?us-ascii?Q?Nc8jpJVHpVAtD1RqPbjep5DoYwUT9FCbEkny2Br0YP8EpQWT0+QS4IHvvaP2?=
 =?us-ascii?Q?eSVFzX6v2vEnI6MJOohnCf/tNn4pPmXlKolzg2CFwwBJbz91g+NEwaY2oyBq?=
 =?us-ascii?Q?WTGsUcJWwgs6Mo8ju25U88OiOKw/LgwfWZTa7rj+X9JbAxXaatG3Htd2i723?=
 =?us-ascii?Q?OagQazDY7QvhgtJrLckxm/ZYlY+7Uz/Sy1tm7cfcsk0iXii6qKLitgPtzT9q?=
 =?us-ascii?Q?nWOx+Dc2d/nZYNAPxKHtWd6HqKVwDe/up5OlkhQTna/0a2VoyFPjTmnHY8MH?=
 =?us-ascii?Q?TMthxrsi+g=3D=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 76a6a82b-4d2c-4ff0-6d94-08d9f0ec1327
X-MS-Exchange-CrossTenant-AuthSource: DM8PR01MB6824.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Feb 2022 01:31:37.9415
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GgpcYzFFqXOHrQFX6GnzvrDtAU3z2PllPwiZgrS9QkwdZYi3TNTlkc2KSD0+GgAZRUVN6Oz+WEYBrLP6zRA6jn+PQdO4icRvoPjUUJNG2GhYPdpIWqJICj1xaTpJRoMF
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR01MB4025
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Will,
On Tue, Feb 15, 2022 at 04:44:23PM +0000, Will Deacon wrote:
> On Wed, Feb 09, 2022 at 09:26:42AM +0000, Huang Shijie wrote:
> > The following interrelated ranges are needed by the kdump crash tool:
> > 	MODULES_VADDR ~ MODULES_END,
> > 	VMALLOC_START ~ VMALLOC_END,
> > 	VMEMMAP_START ~ VMEMMAP_END
> > 
> > Since these values change from time to time, it is preferable to export
> > them via vmcoreinfo than to change the crash's code frequently.
> 
> Please can you explain _why_ they are needed?

The current Crash code is still based at kernel v4.9.
   The virtual memory layout looks like this:
   +--------------------------------------------------------------------+
   |    KASAN     |   MODULE   |   VMALLOC   | .... |     VMEMMAP       |
   +--------------------------------------------------------------------+

The Crash uses MODULES range to set the VMALLOC ranges.
If the ranges are wrong, Crash will _NOT_ works well for some latest kernel
,such as v5.11 later. (Please correct me if I am wrong).
It seems the VMEMMAP range is less important.


I list all the changes since v4.9 to now:

1.) The current crash code is based at kernel v4.9.
   The virtual memory layout looks like this:
   +--------------------------------------------------------------------+
   |    KASAN     |   MODULE   |   VMALLOC   | .... |     VMEMMAP       |
   +--------------------------------------------------------------------+

    The macros are:
    #define MODULES_VADDR	(VA_START + KASAN_SHADOW_SIZE)
    #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)

    #define VMALLOC_START	(MODULES_END)
    #define VMALLOC_END		(PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K)

    #define VMEMMAP_START	(PAGE_OFFSET - VMEMMAP_SIZE)

2.) In the kernel v5.0, the patch will add a new BFP JIT region:
       "91fc957c9b1d arm64/bpf: don't allocate BPF JIT programs in module memory"

   The virtual memory layout looks like this:
   +--------------------------------------------------------------------+
   | KASAN | BPF_JIT | MODULE | VMALLOC |     ....     | VMEMMAP        |
   +--------------------------------------------------------------------+

    The macros are:
    #define MODULES_VADDR	(BPF_JIT_REGION_END)
    #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)

    #define VMALLOC_START	(MODULES_END)
    #define VMALLOC_END		(PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K)

    #define VMEMMAP_START	(PAGE_OFFSET - VMEMMAP_SIZE)

    The layout does not changed until v5.4.

3.) In the kernel v5.4, several patches changes the layout, such as:
       "ce3aaed87344 arm64: mm: Modify calculation of VMEMMAP_SIZE"
       "14c127c957c1 arm64: mm: Flip kernel VA space"
    and the virtual memory layout looks like this:

   +--------------------------------------------------------------------+
   | KASAN | BPF_JIT | MODULE | VMALLOC |     ....     | VMEMMAP        |
   +--------------------------------------------------------------------+

    The macros are:
    #define MODULES_VADDR	(BPF_JIT_REGION_END)
    #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)

    #define VMALLOC_START	(MODULES_END)
    #define VMALLOC_END		(- PUD_SIZE - VMEMMAP_SIZE - SZ_64K)

    #define VMEMMAP_START	(-VMEMMAP_SIZE - SZ_2M)

4.) In the kernel v5.11, several patches changes the layout, such as:
       "9ad7c6d5e75b arm64: mm: tidy up top of kernel VA space"
       "f4693c2716b3 arm64: mm: extend linear region for 52-bit VA configurations"
    and the virtual memory layout looks like this:

   +--------------------------------------------------------------------+
   |   BPF_JIT | MODULE  |  VMALLOC   |    ....    |     VMEMMAP        |
   +--------------------------------------------------------------------+

    The macros are:
    #define MODULES_VADDR	(BPF_JIT_REGION_END)
    #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)

    #define VMALLOC_START	(MODULES_END)
    #define VMALLOC_END		(VMEMMAP_START - SZ_256M)

    #define VMEMMAP_START	(-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
    #define VMEMMAP_END		(VMEMMAP_START + VMEMMAP_SIZE)

5.) In the kernel v5.16, after the patch
      "b89ddf4cca43 arm64/bpf: Remove 128MB limit for BPF JIT programs"
    the virtual memory layout looks like this:

   +--------------------------------------------------------------------+
   |      MODULE     |   VMALLOC   |     ....     |      VMEMMAP        |
   +--------------------------------------------------------------------+

    The macros are:
    #define MODULES_VADDR	(_PAGE_END(VA_BITS_MIN))
    #define MODULES_END		(MODULES_VADDR + MODULES_VSIZE)

    #define VMALLOC_START	(MODULES_END)
    #define VMALLOC_END		(VMEMMAP_START - SZ_256M)

    #define VMEMMAP_START	(-(UL(1) << (VA_BITS - VMEMMAP_SHIFT)))
    #define VMEMMAP_END		(VMEMMAP_START + VMEMMAP_SIZE)


BTW:I am currently coding a patch for the Crash to update all the ranges to
the latest kernel version v5.17-rc4.

Thanks
Huang Shijie