Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1753508AbdCHP1C (ORCPT <rfc822;w@1wt.eu>);
        Wed, 8 Mar 2017 10:27:02 -0500
Received: from mga09.intel.com ([134.134.136.24]:63559 "EHLO mga09.intel.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1752952AbdCHP07 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 8 Mar 2017 10:26:59 -0500
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.36,264,1486454400"; 
   d="scan'208";a="942024598"
Date: Wed, 8 Mar 2017 22:46:32 +0800
From: Fengguang Wu <fengguang.wu@intel.com>
To: Chris Wilson <chris@chris-wilson.co.uk>
Cc: kernel test robot <xiaolong.ye@intel.com>,
        Boqun Feng <boqun.feng@gmail.com>, Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Nicolai =?utf-8?Q?H=C3=A4hnle?= <Nicolai.Haehnle@amd.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        LKML <linux-kernel@vger.kernel.org>, lkp@01.org
Subject: Re: [lkp-robot] [locking/ww_mutex]  857811a371:
 INFO:task_blocked_for_more_than#seconds
Message-ID: <20170308144632.ffeelkijmpqui3so@wfg-t540p.sh.intel.com>
References: <20170308010854.GA17420@yexl-desktop>
 <20170308121312.GL25284@nuc-i3427.alporthouse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <20170308121312.GL25284@nuc-i3427.alporthouse.com>
User-Agent: NeoMutt/20161104 (1.7.1)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 1565
Lines: 36

On Wed, Mar 08, 2017 at 12:13:12PM +0000, Chris Wilson wrote:
>On Wed, Mar 08, 2017 at 09:08:54AM +0800, kernel test robot wrote:
>>
>> FYI, we noticed the following commit:
>>
>> commit: 857811a37129f5d2ba162d7be3986eff44724014 ("locking/ww_mutex: Adjust the lock number for stress test")
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>
>> in testcase: boot
>>
>> on test machine: qemu-system-i386 -enable-kvm -m 320M
>>
>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>
>Now the test is running, it takes too long. :)

Sorry that's right. Up to now the 0day robot still cannot guarantee
the timely reporting of a runtime regression, nor can it guarantee
bisecting of a new regression even when some test actually triggered
the bug.

One fundamental challenge is, there are ~50,000 runtime "regressions"
queued for bisect. Obviously there is no way to bisect them all. So a
large portion of real regressions never get a chance to be bisected.
Not to mention the problem of bisect reliability and efficiency.

Most of the test "regressions" may be duplicates to each other (eg. a
bug in mainline kernel will also show up in various developer trees).
A great portion of them may also be random noises (eg. performance
fluctuations). We've tried various approaches to improve the
de-duplicate, filtering, prioritize etc. algorithms. Together with
increased test coverage, they have been reflected in our slowly
increasing report numbers. However there is still a long way to go.

Thanks,
Fengguang