Received: by 2002:a25:ef43:0:0:0:0:0 with SMTP id w3csp1061007ybm; Wed, 27 May 2020 15:14:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz149PbKFy2tXQij5G6qz6cHuxOhle9twmUgothWB7kl0nKOVUsd9Gp0gNxmCfhpqvWtaxJ X-Received: by 2002:a17:906:6c97:: with SMTP id s23mr367538ejr.421.1590617639877; Wed, 27 May 2020 15:13:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590617639; cv=none; d=google.com; s=arc-20160816; b=MN4uCWnXFL6WM/+UU4ne8Ha51rVysfkCNEpHLgK8ffEfJouYGoNAImtU4ixgBKr50a csTImXuAGZQRex18AGBn7w0XCspVBD2Tuo9zJ7caIz8YIvPqiZ3OQHFfoYPzugueAOan iz3sOmht24aky98YAHQpgUyCWXqEbB9FRc8O3oeo5nLmX0vZqW8uyjhHVw74Tx2brIq+ nIv4wuZRw/0NNtefi+PLG52V02/DeBZIdILqFR3Pn/vB9BKLoTif2JTuMyug4d/IyOUy PVqqJJTrlSQJm3WcZjih5I8Q8T4h6gLTzF007L45IfgOP6FTRs+kow+6BhVNBkI3GLc7 HItw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=Ib++0cCLLitVx1shV2jGQa+ni4nOQcBF64cPLDTrgCc=; b=xqKHgwW3fdqgT3vsmoE9gF272sBgqHwvn7MWlbZSdGNbx0ETe7GtBO2rHJyUFO7hc6 a6x56jOx6y9+tJRQhzyU89RX9Ozo91v3gAug/DyosnL74dMUE8FCP2gIK9gdTiFf8Syg 19mKzX4sWiJLEKnFt8O5H/LapBmt9h5mEskoL7KpwBdjNOIDh0W1qS0uDs1LCVx9JKMa TP2E8sX8vHFC5ZuuTlLl9Q8Ptwn6+Wj2iD47mRndiuqTW+MDj++6DPreWlMDxR6x2ZCJ udLgm8Q+VAbmpSODlM39jT6cd+QWq/ifk6JyVM/zgVNSy8idGXCTu0kI5JzDre1XS8kD rJTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=NcXYcKIV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a2si2513862edl.517.2020.05.27.15.13.36; Wed, 27 May 2020 15:13:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=NcXYcKIV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726130AbgE0WLQ (ORCPT + 99 others); Wed, 27 May 2020 18:11:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60066 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725267AbgE0WLP (ORCPT ); Wed, 27 May 2020 18:11:15 -0400 Received: from mail-ua1-x943.google.com (mail-ua1-x943.google.com [IPv6:2607:f8b0:4864:20::943]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F3EBC05BD1E for ; Wed, 27 May 2020 15:11:15 -0700 (PDT) Received: by mail-ua1-x943.google.com with SMTP id w20so335248uaa.2 for ; Wed, 27 May 2020 15:11:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Ib++0cCLLitVx1shV2jGQa+ni4nOQcBF64cPLDTrgCc=; b=NcXYcKIVwG0ueKs966DI5d737p2jqNpsHmdvqJgR5Ck2iSd69RDowe2cbDvVSielEW 4f4lDTsaueLXnEUlskfiQPVLfIGMNLtm+QHarALLKVNc2VVcQWaft+Ihz6qri+7/WBZj pqaDs2RMNbRSOwWWEDWMuRuS01IjqvQlxSczM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Ib++0cCLLitVx1shV2jGQa+ni4nOQcBF64cPLDTrgCc=; b=Ogj4dfq3axsQPDotAMypfk1xpC1PCz1wIuuRQM8UMJvLqjSfTZOL//Tat6MPb8DPd1 ZA7lA4T1ALFKI4zzjw1uCj6DhJnICbEs74mWF/hHT/CfJtoNF+KfaetGPx5SMOp8BxG/ lVtT46b/clNw2RNL2xrlcAIyZvUWT5/9Qwpe8+TyOhcpsy4+Sn2Px4zMTLb3/jmbYrsH OQffeip2ZlW4kA/ctjLZEvsWcRnf8BJR2Mbz3B84Y8/+zG2eIyoE/4p0BvV9RYpTVNny YXLuuYP83YtxY9a07qFp6JmjuA52JBrZGUuN0jBEq9oLMoOxzO53/cnDSYbkq+J/R0l8 8e2A== X-Gm-Message-State: AOAM531FfZL4BxJ4KCDB9A5971WsTdW57zREdcs+Adtya2TbBkTkcnhx i6RT6wF6Ljo7AQ/lB/c2AqlEnCTo8y8= X-Received: by 2002:ab0:2045:: with SMTP id g5mr6449027ual.76.1590617473862; Wed, 27 May 2020 15:11:13 -0700 (PDT) Received: from mail-vs1-f47.google.com (mail-vs1-f47.google.com. [209.85.217.47]) by smtp.gmail.com with ESMTPSA id t198sm396514vkt.35.2020.05.27.15.11.12 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 27 May 2020 15:11:13 -0700 (PDT) Received: by mail-vs1-f47.google.com with SMTP id k3so1645660vsg.2 for ; Wed, 27 May 2020 15:11:12 -0700 (PDT) X-Received: by 2002:a67:e884:: with SMTP id x4mr6706217vsn.106.1590617472047; Wed, 27 May 2020 15:11:12 -0700 (PDT) MIME-Version: 1.0 References: <1588339863-1322-1-git-send-email-kalyan_t@codeaurora.org> <32c01e9a5277bdbdbab868eb71688184@codeaurora.org> In-Reply-To: From: Doug Anderson Date: Wed, 27 May 2020 15:11:00 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [Freedreno] [PATCH] drm/msm/dpu: ensure device suspend happens during PM sleep To: Kalyan Thota , Sean Paul Cc: "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" , mkrishn@codeaurora.org, linux-arm-msm , travitej@codeaurora.org, LKML , dri-devel , Rob Clark , nganji@codeaurora.org, "Kristian H. Kristensen" , freedreno , Jeykumar Sankaran Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi, On Fri, May 15, 2020 at 9:37 AM Doug Anderson wrote: > > Hi, > > On Fri, May 15, 2020 at 5:06 AM wrote: > > > > On 2020-05-14 21:47, Doug Anderson wrote: > > > Hi, > > > > > > On Fri, May 1, 2020 at 6:31 AM Kalyan Thota > > > wrote: > > >> > > >> "The PM core always increments the runtime usage counter > > >> before calling the ->suspend() callback and decrements it > > >> after calling the ->resume() callback" > > >> > > >> DPU and DSI are managed as runtime devices. When > > >> suspend is triggered, PM core adds a refcount on all the > > >> devices and calls device suspend, since usage count is > > >> already incremented, runtime suspend was not getting called > > >> and it kept the clocks on which resulted in target not > > >> entering into XO shutdown. > > >> > > >> Add changes to force suspend on runtime devices during pm sleep. > > >> > > >> Changes in v1: > > >> - Remove unnecessary checks in the function > > >> _dpu_kms_disable_dpu (Rob Clark). > > >> > > >> Changes in v2: > > >> - Avoid using suspend_late to reset the usagecount > > >> as suspend_late might not be called during suspend > > >> call failures (Doug). > > >> > > >> Changes in v3: > > >> - Use force suspend instead of managing device usage_count > > >> via runtime put and get API's to trigger callbacks (Doug). > > >> > > >> Changes in v4: > > >> - Check the return values of pm_runtime_force_suspend and > > >> pm_runtime_force_resume API's and pass appropriately (Doug). > > >> > > >> Changes in v5: > > > > > > Can you please put the version number properly in your subject? It's > > > really hard to tell one version of your patch from another. > > > > > > > > >> - With v4 patch, test cycle has uncovered issues in device resume. > > >> > > >> On bubs: cmd tx failures were seen as SW is sending panel off > > >> commands when the dsi resources are turned off. > > >> > > >> Upon suspend, DRM driver will issue a NULL composition to the > > >> dpu, followed by turning off all the HW blocks. > > >> > > >> v5 changes will serialize the NULL commit and resource unwinding > > >> by handling them under PM prepare and PM complete phases there by > > >> ensuring that clks are on when panel off commands are being > > >> processed. > > > > > > I'm still most definitely not an expert in how all the DRM pieces all > > > hook up together, but the solution you have in this patch seems wrong > > > to me. As far as I can tell the "prepare" state isn't supposed to be > > > actually doing the suspend work and here that's exactly what you're > > > doing. I think you should find a different solution to ensure > > > ordering is correct. > > > > > > -Doug > > > > > > > Hi, > > Quite honestly I'm probably not the right person to be reviewing this > code. I mostly just noticed one of your early patches and it looked > strange to me. Hopefully someone with actual experience in how all > the DRM components work together can actually review and see if this > makes sense. Maybe Sean would know better? > > That being said, let me at least look at what you're saying... > > > > Prepare and Complete are callbacks defined as part of Sleep and Resume > > sequence > > > > Entering PM SUSPEND the phases are : prepare --> suspend --> > > suspend_late --> suspend_noirq. > > While leaving PM SUSPEND the phases are: resume_noirq --> resume_early > > --> resume --> complete. > > Sure, it's part of the sequence. It's also documented in pm.h as: > > * The principal role of this callback is to prevent new children of > * the device from being registered after it has returned (the driver's > * subsystem and generally the rest of the kernel is supposed to prevent > * new calls to the probe method from being made too once @prepare() has > * succeeded). > > It does not feel like that matches your usage of this call. > > > > The reason to push drm suspend handling to PM prepare phase is that > > parent here will trigger a modeset to turn off the timing and > > subsequently the panel. > > the child devices should not turn of their clocks before parent unwinds > > the composition. Hence they are serialized as per the sequence mentioned > > above. > > So the general model in Linux is that children suspend before their > parents, right? So you're saying that, in this case, the parent needs > to act on the child before the child suspends. Is that correct? > > Rather than hijacking the prepare/complete, I'd be at least slightly > inclined to move the other driver to turn off its clocks in > suspend_late and to turn them back on in resume_early? That seems to > be what was done in "analogix_dp-rockchip.c" to solve a similar > problem. > > > > A similar approach is taken by other driver that use drm framework. In > > this driver, the device registers for prepare and complete callbacks to > > handle drm_suspend and drm_resume. > > https://gitlab.freedesktop.org/drm/msm/-/blob/msm-next/drivers/gpu/drm/exynos/exynos_drm_drv.c#L163 > > OK, if there is another driver in DRM then I guess I won't object too > strongly. Note that when searching for other drivers I noticed this > bit in todo.rst: > > * Most drivers (except i915 and nouveau) that use > * drm_atomic_helper_suspend/resume() can probably be converted to use > * drm_mode_config_helper_suspend/resume(). Also there's still open-coded version > * of the atomic suspend/resume code in older atomic modeset drivers. > > Does anything get fixed if you do that? It seems like it'd cleanup > your code a bit so maybe worth doing anyway... > > --- > > I guess the last question I'd want resolved is why you have this asymmetry: > > SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, msm_pm_resume) > > Why couldn't you use pm_runtime_force_resume()? I'm curious if you had answers to any of the questions I posed in my review. -Doug