Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755580AbdLVDch (ORCPT ); Thu, 21 Dec 2017 22:32:37 -0500 Received: from mail-it0-f66.google.com ([209.85.214.66]:36898 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755180AbdLVDcd (ORCPT ); Thu, 21 Dec 2017 22:32:33 -0500 X-Google-Smtp-Source: ACJfBovLKKiTr32iMEiiFKx31Yj7pStlYNDqp0dkr91RNd8eSvr0+PJ83LX5bH3sbm4Iv3rYLLltww== Date: Thu, 21 Dec 2017 21:32:29 -0600 From: Eric Biggers To: Dmitry Vyukov Cc: LKML , syzkaller , Eric Dumazet , Kostya Serebryany , Alexander Potapenko , andreyknvl , Linus Torvalds , Greg Kroah-Hartman , Andrew Morton , Tetsuo Handa , David Miller , Willem de Bruijn , Guenter Roeck , Stephan Mueller Subject: Re: [RFC] syzbot process Message-ID: <20171222033229.GB26818@zzz.localdomain> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.2 (2017-12-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3283 Lines: 62 On Thu, Dec 21, 2017 at 01:52:40PM +0100, Dmitry Vyukov wrote: > However, the cost is that it needs to understand statuses of bugs: > most importantly, what commit fixes what bug. It also has support for > marking a bug as "invalid", e.g. happened once but most likely was > caused by a previous silent memory corruption. And support for marking > bugs as duplicates of other bugs, i.e. the same root cause and will be > fixed when the target bug is fixed. These simple rules are outlined in > the footer of each report and also explained in more detail at the > referenced link: > > ---------------------------------- > This bug is generated by a dumb bot. It may contain errors. > See https://goo.gl/tpsmEJ for details. > Direct all questions to syzkaller@googlegroups.com. > Please credit me with: Reported-by: syzbot > syzbot will keep track of this bug report. > Once a fix for this bug is merged into any tree, reply to this email with: > #syz fix: exact-commit-title > If you want to test a patch for this bug, please reply with: > #syz test: git://repo/address.git branch > and provide the patch inline or as an attachment. > To mark this as a duplicate of another syzbot report, please reply with: > #syz dup: exact-subject-of-another-report > If it's a one-off invalid bug report, please reply with: > #syz invalid > Note: if the crash happens again, it will cause creation of a new bug report. > Note: all commands must start from beginning of the line in the email body. > ---------------------------------- > > Status tracking allows syzbot to (1) keep track of still unfixed bugs > (more than half actually gets lost in LKML archives if nobody keeps > track of them), (2) be able to ever report similarly looking crashes > as new bugs in future, (3) be able to test fixes. > > The problem is that these rules are mostly not followed. As others mentioned, allowing a bug ID to be in the fix's commit message, perhaps in the Reported-by line which syzbot already suggests to include, would make things a bit easier. But I think the larger problem is that people in the community don't have any visibility into the statuses of the bugs, so they don't have any motivation to manage the statuses. Are you planning to make a dashboard app publicly available for upstream kernel bugs being tracked by syzbot? I think it would be very useful for the community, especially for finding more details about a bug, e.g. when was it last seen, how often was it seen, has it been seen in multiple trees. Also for finding duplicates which may not have been sent to the correct mailing list. syzbot also should be sending out reminders for bugs that are still open if the crash is still occurring, and even moreso if there is a reproducer. However, if the crash isn't still occurring, then I expect it will become necessary to automatically invalidate the bug after some time, lest the list of bugs grow without bound due to bugs that have already been fixed that no one has time to debug to figure out exactly when/what the fix was, especially if there is no reproducer. Or perhaps the bug was only in linux-next and only existed due to a buggy patch which was dropped or modified before it reached mainline, so there is no "fix" commit. Eric