Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org CA6BBC43383
Subject: Query: Regarding Notifier chain callback debugging or profiling
References: <82d5b63e-4ae6-fb5f-8a1c-2d5755db2638@codeaurora.org>
To:     akpm@linux-foundation.org,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Greg KH <gregkh@linuxfoundation.org>, tglx@linutronix.de
Cc:     linux-arm-msm@vger.kernel.org, neeraju@codeaurora.org
From:   Gaurav Kohli <gkohli@codeaurora.org>
Message-ID: <6e077b43-6c9e-3f4e-e079-db438e36a4eb@codeaurora.org>
Date:   Mon, 10 Feb 2020 17:26:16 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.4.2
MIME-Version: 1.0
In-Reply-To: <82d5b63e-4ae6-fb5f-8a1c-2d5755db2638@codeaurora.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi,

In Linux kernel, everywhere we are using notification chains to notify 
for any kernel events, But we don't have any debugging or profiling 
mechanism to know which callback is taking time or currently we are 
stuck on which call back(without dumps it is difficult to say for last 
problem)

Below are the few ways, which we can implement to profile callback on 
need basis:

1) Use trace event before and after callback:

static int notifier_call_chain(struct notifier_block **nl,
                                unsigned long val, void *v,
                                int nr_to_call, int *nr_calls)
{
         int ret = NOTIFY_DONE;
         struct notifier_block *nb, *next_nb;


+		trace_event for entry of callback
                 ret = nb->notifier_call(nb, val, v);
+		trace_event for exit of callback

         }
         return ret;
}

2) Or use pr_debug instead of trace_event

3) Both of the above approach has certain problems, like it will dump 
callback for each notifier chain, which might flood trace buffer or dmesg.

So we can use bool variable to control that and dump the required 
notification chain only.

Some thing like below we can use:

  struct srcu_notifier_head {
         struct mutex mutex;
         struct srcu_struct srcu;
         struct notifier_block __rcu *head;
+       bool debug_callback;
  };


  static int notifier_call_chain(struct notifier_block **nl,
                                unsigned long val, void *v,
-                              int nr_to_call, int *nr_calls)
+                              int nr_to_call, int *nr_calls, bool 
debug_callback)
  {
         int ret = NOTIFY_DONE;
         struct notifier_block *nb, *next_nb;
@@ -526,6 +526,7 @@ void srcu_init_notifier_head(struct 
srcu_notifier_head *nh)
         if (init_srcu_struct(&nh->srcu) < 0)
                 BUG();
         nh->head = NULL;
+       nh->debug_callback = false; -> by default it would be false for 
every notifier chain.

4) we can also think of something pre and post function, before and 
after each callback, And we can enable only for those who wants to profile.

Please let us what approach we can use, or please suggest some debugging 
mechanism for the same.

Regards
Gaurav

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center,
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.