Received: by 2002:a05:6a10:6d10:0:0:0:0 with SMTP id gq16csp780520pxb; Tue, 12 Apr 2022 13:16:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwAiQpm0NLXQyTBlCTtWLbeEFkBBKPfrxH2EjFXviA4zJYpv+xAdFGQbEhAlGtQICNLkM8q X-Received: by 2002:a17:90a:bb0d:b0:1bd:3baf:c8b4 with SMTP id u13-20020a17090abb0d00b001bd3bafc8b4mr6920159pjr.15.1649794582967; Tue, 12 Apr 2022 13:16:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1649794582; cv=none; d=google.com; s=arc-20160816; b=A1iEF/pEufFzaZOpaC4pbxHDqIWW6PYSqBsX04CEmRnpym5ysAy27ieCWqoZPYp6S7 Gs9sDURdEVd4C4fENRj0g4s/hkNlxGdv36FeWya0zEz61P5GdLK/CSc/dkLC8+HwnGt5 mqVw3DbvsLnxeF28/nu+UVD5XS5zjI0ipR6RlgeHx00sTKt+mDWk4Sk3Z7UDKz9+u3cf nbgH2DqXSKjcP3siUGy0xHdORJu24CwIf5d+1BAzDrP9X4S4ubDX7+ImRZY/ulj9vppx Hfg7KcPtOs+wgl7UXgbU28g6LF1tAqdiQQ1MTvprHIPWEvlJomWXUDo8aYmfFBpaSCvn ip7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id:dkim-signature; bh=46PKW9Y5PH5pXmOmBpZQ4BZtF4FwoQVxcQUrC8+2E6E=; b=cDBvdF9E+BS0jcmCt7CWXZpSvad8a0Z42laL0t7uTEJA+l6Q2CGjA5LxJucTneo5KG TxvHuJF9huSv8RNJ8rRW/wRz04k75b/xPH/NG7Ugx721bv248kgVGg9cNVsD2CgIkT2F zQGL2ffuwdjcuYbNfo3zi3XDglkHaItMF3trA5duaGD5Ugzucr9SdmGSZxglM4lNR4sc XyxDP9OCw657qtXuMg2Qhl9aWNVyZfX8JtbbvtB7kpNPQVPSt78kuTVNTjj8k2a/ubQz +RRgFqEq88Pe0O+SGAV5A2f50hMOqulpUoI2aKQ8I2yR7jzSivtcFPejlbYiDFpOsmf+ +qAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=hGaQcfnq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id k184-20020a6384c1000000b0039d380a77f2si3429514pgd.402.2022.04.12.13.16.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Apr 2022 13:16:22 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=hGaQcfnq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 56A6676E3C; Tue, 12 Apr 2022 12:59:57 -0700 (PDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1351107AbiDLRXB (ORCPT + 99 others); Tue, 12 Apr 2022 13:23:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235984AbiDLRXA (ORCPT ); Tue, 12 Apr 2022 13:23:00 -0400 Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C6CC60CD2; Tue, 12 Apr 2022 10:20:41 -0700 (PDT) Received: by mail-ot1-x333.google.com with SMTP id w17-20020a056830111100b005b22c584b93so13806884otq.11; Tue, 12 Apr 2022 10:20:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:mime-version:user-agent:subject:content-language:to :cc:references:from:in-reply-to:content-transfer-encoding; bh=46PKW9Y5PH5pXmOmBpZQ4BZtF4FwoQVxcQUrC8+2E6E=; b=hGaQcfnqki96Ob1+8AuMEHiHTEa1K9GM7CUx/LL6749BglFYR7PLdMY5ReRq2xSGXN 6JeSR7++T85cteJIEBspTX+IXSSox1UXNASBQ3bfCCw9BkJUJpe/AZ95AxXQ6RXGG9Hh qTqHHYFdVuvHKcBFmwF1Ha6TT9hySTq3WhStDeAEOF7QtMs14Y5H22aQ0QGWxyFiljwP QZjFj6DRGvwqdo3QZzZ2y/p9CyYmnTgdI6ps7zeqFe+pLVwXX+2DjEX5uUN6GYDPVqD4 dEU1HC1SJJCfXie4oxAeuimyqMYJyJkjCHELnI633PD8qQ/PhP2YhzhG7YR7rF6OjSJR 5wFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=46PKW9Y5PH5pXmOmBpZQ4BZtF4FwoQVxcQUrC8+2E6E=; b=a7FsJqV9NESF3cov/kbafq3dsiMYNK1Z9dRhMq3W2q0hSyQevzpJnwpjK2G+I5QnAC ytVcJhhQiWZGhUgf7Jf5kN3bhXtpglmJQV9oXtJ74RtDRLBbMdvwU19x/3Pa0qpKUEYr OwvaHoIv7Lxwx4L/rcwQJrVnm4BEJ4bKobNzbdd0J4NP0OpKfxT5aOTRd5oiCcPPu0pp DeEuIKap5Nj0v1/47BiGPr3lP5zPPqpo4GbeH/Co7TS1AjH7x2HMps4umePF+xCgGzWZ UNDuWA024F3nq5aPMqu6Wd4LPsEfWX8H3ZnWEuREuG5XyfPmoPJwRL0a0zoUuWMLjPNS Tw9w== X-Gm-Message-State: AOAM530VKMGeGTODDrRw3cXs5+XZsNy7vDp1q4tr8i5r4nCl6nnieq+8 GkatxzLv1x1OQI5BE6Ra9aY= X-Received: by 2002:a9d:5913:0:b0:5cd:a050:8f55 with SMTP id t19-20020a9d5913000000b005cda0508f55mr13583590oth.44.1649784040563; Tue, 12 Apr 2022 10:20:40 -0700 (PDT) Received: from ?IPV6:2600:1700:2442:6db0:641c:88dc:7a9c:6fac? ([2600:1700:2442:6db0:641c:88dc:7a9c:6fac]) by smtp.gmail.com with ESMTPSA id 190-20020a4a0dc7000000b003244ae0bbd5sm12754029oob.7.2022.04.12.10.20.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 12 Apr 2022 10:20:39 -0700 (PDT) Message-ID: <400ab321-99fc-9669-c277-007906a9cc14@gmail.com> Date: Tue, 12 Apr 2022 12:20:38 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [PATCH v2 2/2] of: overlay: rework overlay apply and remove kfree()s Content-Language: en-US To: Rob Herring Cc: Pantelis Antoniou , Slawomir Stepien , devicetree@vger.kernel.org, "linux-kernel@vger.kernel.org" , Slawomir Stepien , Geert Uytterhoeven , Alan Tull , Jan Kiszka References: <20220410210833.441504-1-frowand.list@gmail.com> <20220410210833.441504-3-frowand.list@gmail.com> From: Frank Rowand In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A, RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 4/12/22 09:00, Rob Herring wrote: > On Sun, Apr 10, 2022 at 4:08 PM wrote: >> >> From: Frank Rowand >> >> Fix various kfree() issues related to of_overlay_apply(). >> - Double kfree() of fdt and tree when init_overlay_changeset() >> returns an error. >> - free_overlay_changeset() free the root of the unflattened >> overlay (variable tree) instead of the memory that contains >> the unflattened overlay. >> - For the case of a failure during applying an overlay, move kfree() >> of new_fdt and overlay_mem into the function that allocated them. >> For the case of removing an overlay, the kfree() remains in >> free_overlay_changeset(). >> - Check return value of of_fdt_unflatten_tree() for error instead >> of checking the returnded value of overlay_root. >> >> More clearly document policy related to lifetime of pointers into >> overlay memory. >> >> Double kfree() >> Reported-by: Slawomir Stepien >> >> Signed-off-by: Frank Rowand >> --- >> >> Changes since v1: >> - Move kfree()s from init_overlay_changeset() to of_overlay_fdt_apply() >> - Better document lifetime of pointers into overlay, both in overlay.c >> and Documentation/devicetree/overlay-notes.rst >> >> Documentation/devicetree/overlay-notes.rst | 23 +++- >> drivers/of/overlay.c | 127 ++++++++++++--------- >> 2 files changed, 91 insertions(+), 59 deletions(-) >> >> diff --git a/Documentation/devicetree/overlay-notes.rst b/Documentation/devicetree/overlay-notes.rst >> index b2b8db765b8c..7a6e85f75567 100644 >> --- a/Documentation/devicetree/overlay-notes.rst >> +++ b/Documentation/devicetree/overlay-notes.rst >> @@ -119,10 +119,25 @@ Finally, if you need to remove all overlays in one-go, just call >> of_overlay_remove_all() which will remove every single one in the correct >> order. >> >> -In addition, there is the option to register notifiers that get called on >> +There is the option to register notifiers that get called on >> overlay operations. See of_overlay_notifier_register/unregister and >> enum of_overlay_notify_action for details. >> >> -Note that a notifier callback is not supposed to store pointers to a device >> -tree node or its content beyond OF_OVERLAY_POST_REMOVE corresponding to the >> -respective node it received. >> +A notifier callback for OF_OVERLAY_PRE_APPLY, OF_OVERLAY_POST_APPLY, or >> +OF_OVERLAY_PRE_REMOVE may store pointers to a device tree node in the overlay >> +or its content but these pointers must not persist past the notifier callback >> +for OF_OVERLAY_POST_REMOVE. The memory containing the overlay will be >> +kfree()ed after OF_OVERLAY_POST_REMOVE notifiers are called. Note that the >> +memory will be kfree()ed even if the notifier for OF_OVERLAY_POST_REMOVE >> +returns an error. >> + >> +The changeset notifiers in drivers/of/dynamic.c are a second type of notifier >> +that could be triggered by applying or removing an overlay. These notifiers >> +are not allowed to store pointers to a device tree node in the overlay >> +or its content. The overlay code does not protect against such pointers >> +remaining active when the memory containing the overlay is freed as a result >> +of removing the overlay. >> + >> +Any other code that retains a pointer to the overlay nodes or data is >> +considered to be a bug because after removing the overlay the pointer >> +will refer to freed memory. >> diff --git a/drivers/of/overlay.c b/drivers/of/overlay.c >> index f74aa9ff67aa..c8e999518f2f 100644 >> --- a/drivers/of/overlay.c >> +++ b/drivers/of/overlay.c >> @@ -58,6 +58,7 @@ struct fragment { >> * @id: changeset identifier >> * @ovcs_list: list on which we are located >> * @new_fdt: Memory allocated to hold unflattened aligned FDT >> + * @overlay_mem: the memory chunk that contains @overlay_tree >> * @overlay_tree: expanded device tree that contains the fragment nodes >> * @count: count of fragment structures >> * @fragments: fragment nodes in the overlay expanded device tree >> @@ -68,6 +69,7 @@ struct overlay_changeset { >> int id; >> struct list_head ovcs_list; >> const void *new_fdt; >> + const void *overlay_mem; >> struct device_node *overlay_tree; >> int count; >> struct fragment *fragments; >> @@ -720,18 +722,20 @@ static struct device_node *find_target(struct device_node *info_node) >> * init_overlay_changeset() - initialize overlay changeset from overlay tree >> * @ovcs: Overlay changeset to build >> * @new_fdt: Memory allocated to hold unflattened aligned FDT >> + * @tree_mem: Memory that contains @overlay_tree >> * @overlay_tree: Contains the overlay fragments and overlay fixup nodes >> * >> * Initialize @ovcs. Populate @ovcs->fragments with node information from >> * the top level of @overlay_tree. The relevant top level nodes are the >> * fragment nodes and the __symbols__ node. Any other top level node will >> - * be ignored. >> + * be ignored. Populate other @ovcs fields. >> * >> * Return: 0 on success, -ENOMEM if memory allocation failure, -EINVAL if error >> * detected in @overlay_tree, or -ENOSPC if idr_alloc() error. >> */ >> static int init_overlay_changeset(struct overlay_changeset *ovcs, >> - const void *new_fdt, struct device_node *overlay_tree) >> + const void *new_fdt, const void *tree_mem, >> + struct device_node *overlay_tree) >> { >> struct device_node *node, *overlay_node; >> struct fragment *fragment; >> @@ -751,9 +755,6 @@ static int init_overlay_changeset(struct overlay_changeset *ovcs, >> if (!of_node_is_root(overlay_tree)) >> pr_debug("%s() overlay_tree is not root\n", __func__); >> >> - ovcs->overlay_tree = overlay_tree; >> - ovcs->new_fdt = new_fdt; >> - >> INIT_LIST_HEAD(&ovcs->ovcs_list); >> >> of_changeset_init(&ovcs->cset); >> @@ -832,6 +833,9 @@ static int init_overlay_changeset(struct overlay_changeset *ovcs, >> >> ovcs->id = id; >> ovcs->count = cnt; >> + ovcs->new_fdt = new_fdt; >> + ovcs->overlay_mem = tree_mem; >> + ovcs->overlay_tree = overlay_tree; >> ovcs->fragments = fragments; >> >> return 0; >> @@ -846,7 +850,7 @@ static int init_overlay_changeset(struct overlay_changeset *ovcs, >> return ret; >> } >> >> -static void free_overlay_changeset(struct overlay_changeset *ovcs) >> +static void free_overlay_changeset_contents(struct overlay_changeset *ovcs) >> { >> int i; >> >> @@ -861,12 +865,20 @@ static void free_overlay_changeset(struct overlay_changeset *ovcs) >> of_node_put(ovcs->fragments[i].overlay); >> } >> kfree(ovcs->fragments); >> +} >> +static void free_overlay_changeset(struct overlay_changeset *ovcs) >> +{ >> + >> + free_overlay_changeset_contents(ovcs); >> + >> /* >> - * There should be no live pointers into ovcs->overlay_tree and >> + * There should be no live pointers into ovcs->overlay_mem and >> * ovcs->new_fdt due to the policy that overlay notifiers are not >> - * allowed to retain pointers into the overlay devicetree. >> + * allowed to retain pointers into the overlay devicetree other >> + * than the window between OF_OVERLAY_PRE_APPLY overlay notifiers >> + * and the OF_OVERLAY_POST_REMOVE overlay notifiers. >> */ >> - kfree(ovcs->overlay_tree); >> + kfree(ovcs->overlay_mem); >> kfree(ovcs->new_fdt); >> kfree(ovcs); >> } >> @@ -876,8 +888,10 @@ static void free_overlay_changeset(struct overlay_changeset *ovcs) >> * >> * of_overlay_apply() - Create and apply an overlay changeset >> * @new_fdt: Memory allocated to hold the aligned FDT >> + * @tree_mem: Memory that contains @overlay_tree >> * @overlay_tree: Expanded overlay device tree >> * @ovcs_id: Pointer to overlay changeset id >> + * @kfree_unsafe: Pointer to flag to not kfree() @new_fdt and @overlay_tree > > This screams hack. I agree that it screams hack. > > It would be somewhat less hacky if we stored some state information in > ovcs struct that conveys the apply state of the overlay and then used > that to determine if we should do kfree or not. I tried this approach first, and thought the result was rather ugly. But it is possible if you prefer. If the state was stored in ovcs, then the kfree() code at the end of of_overlay_fdt_apply() would change from: out_free_overlay_mem: if (!kfree_unsafe) kfree(overlay_mem); out_free_new_fdt: if (!kfree_unsafe) kfree(new_fdt); return ret; to something like (untested, thrown together): out_free_overlay_mem: mutex_lock(&of_mutex); ovcs = idr_find(&ovcs_idr, *ovcs_id); if (!ovcs) { ret = -ENODEV; pr_err("remove: Could not find overlay #%d for kfree(overlay_mem)\n", *ovcs_id); goto out_unlock; } if (!ovcs->kfree_unsafe) kfree(overlay_mem); mutex_unlock(&of_mutex); out_free_new_fdt: mutex_lock(&of_mutex); ovcs = idr_find(&ovcs_idr, *ovcs_id); if (!ovcs) { ret = -ENODEV; pr_err("remove: Could not find overlay #%d for kfree(new_fdt)\n", *ovcs_id); goto out_unlock; } if (!ovcs->kfree_unsafe) kfree(new_fdt); out_unlock: mutex_unlock(&of_mutex); return ret; The upside is that new_fdt and overlay_mem would no longer be arguments to of_overlay_apply() and its children. > > Perhaps a better fix would be refcounting the overlay as a whole and > freeing everything when the refcount goes to 0. We already know when the overlay as a whole is no longer referenced -- it is when of_overlay_remove() is successful. The refcount would instead be on new_fdt and overlay_mem. But the location of incrementing the refcount is in of_overlay_apply() and its children, so the ugliness of passing new_fdt and overlay_mem into those functions remains. I am fine with the "hacky" approach or the saving more state in ovcs. I would prefer not to use the refcount approach. If the "hacky" approach is used, I would do a v3 that adds an additional comment in of_overlay_remove() that describes the window in that function where a memory leak can still occur. > >> * >> * Creates and applies an overlay changeset. >> * >> @@ -910,34 +924,25 @@ static void free_overlay_changeset(struct overlay_changeset *ovcs) >> * refused. >> * >> * Returns 0 on success, or a negative error number. Overlay changeset >> - * id is returned to *ovcs_id. >> + * id is returned to *ovcs_id. When references to @new_fdt and @overlay_tree >> + * may exist, *kfree_unsafe is set to true. >> */ >> >> -static int of_overlay_apply(const void *new_fdt, >> - struct device_node *overlay_tree, int *ovcs_id) >> +static int of_overlay_apply(const void *new_fdt, void *tree_mem, >> + struct device_node *overlay_tree, int *ovcs_id, >> + bool *kfree_unsafe) >> { >> struct overlay_changeset *ovcs; >> int ret = 0, ret_revert, ret_tmp; >> >> - /* >> - * As of this point, new_fdt and overlay_tree belong to the overlay >> - * changeset. overlay changeset code is responsible for freeing them. >> - */ >> - >> if (devicetree_corrupt()) { >> pr_err("devicetree state suspect, refuse to apply overlay\n"); >> - kfree(new_fdt); >> - kfree(overlay_tree); >> - ret = -EBUSY; >> - goto out; >> + return -EBUSY; >> } >> >> ovcs = kzalloc(sizeof(*ovcs), GFP_KERNEL); >> if (!ovcs) { >> - kfree(new_fdt); >> - kfree(overlay_tree); >> - ret = -ENOMEM; >> - goto out; >> + return -ENOMEM; >> } >> >> of_overlay_mutex_lock(); >> @@ -945,28 +950,27 @@ static int of_overlay_apply(const void *new_fdt, >> >> ret = of_resolve_phandles(overlay_tree); >> if (ret) >> - goto err_free_overlay_tree; >> + goto err_free_ovcs; >> >> - ret = init_overlay_changeset(ovcs, new_fdt, overlay_tree); >> + ret = init_overlay_changeset(ovcs, new_fdt, tree_mem, overlay_tree); >> if (ret) >> - goto err_free_overlay_tree; >> + goto err_free_ovcs_contents; >> >> /* >> - * after overlay_notify(), ovcs->overlay_tree related pointers may have >> - * leaked to drivers, so can not kfree() overlay_tree, >> - * aka ovcs->overlay_tree; and can not free memory containing aligned >> - * fdt. The aligned fdt is contained within the memory at >> - * ovcs->new_fdt, possibly at an offset from ovcs->new_fdt. >> + * After overlay_notify(), ovcs->overlay_tree related pointers may have >> + * leaked to drivers, so can not kfree() ovcs->overlay_mem and >> + * ovcs->new_fdt until after OF_OVERLAY_POST_REMOVE notifiers. >> */ >> + *kfree_unsafe = true; >> ret = overlay_notify(ovcs, OF_OVERLAY_PRE_APPLY); >> if (ret) { > > If OF_OVERLAY_PRE_APPLY failed, I would think kfree would still be okay. Not with the existing policy. If we tightened up the policy so that the OF_OVERLAY_PRE_APPLY handler is required to release all references to the overlay memory even with an error return then the kfree would be ok if OF_OVERLAY_PRE_APPLY failed. > >> pr_err("overlay changeset pre-apply notify error %d\n", ret); >> - goto err_free_overlay_changeset; >> + goto err_free_ovcs_contents; >> }