DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B90C235E1
Date: Thu, 8 Jun 2017 17:26:27 -0400
From: Steven Rostedt <rostedt@goodmis.org>
To: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Shuah Khan <shuah@kernel.org>, Masami Hiramatsu <mhiramat@kernel.org>,
        linux-kselftest@vger.kernel.org, Ingo Molnar <mingo@redhat.com>,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH 0/7] selftests/ftrace: Some improvements of ftracetest
Message-ID: <20170608172627.3dab319e@gandalf.local.home>
In-Reply-To: <9a2b4e46-a545-b862-2517-472f364a68b3@osg.samsung.com>
References: <149551934796.30231.12544755801380678045.stgit@devbox>
        <4b0b2ac8-dfba-abd9-3dba-bbe73c22f79f@kernel.org>
        <20170607185628.452a11a1@gandalf.local.home>
        <8f974157-c2e5-9b39-c55b-b637d6e7814f@osg.samsung.com>
        <20170608133743.4720e2bc@gandalf.local.home>
        <8bcf5a87-812a-6ceb-7fa4-ea2d33d4e717@osg.samsung.com>
        <20170608170844.134e1158@gandalf.local.home>
        <9a2b4e46-a545-b862-2517-472f364a68b3@osg.samsung.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2979
Lines: 85

On Thu, 8 Jun 2017 15:14:19 -0600
Shuah Khan <shuahkh@osg.samsung.com> wrote:

> On 06/08/2017 03:08 PM, Steven Rostedt wrote:
> > On Thu, 8 Jun 2017 14:06:23 -0600
> > Shuah Khan <shuahkh@osg.samsung.com> wrote:
> >   
> >>>>>>> # of passed:  41
> >>>>>>> # of failed:  1
> >>>>>>> # of unresolved:  0
> >>>>>>> # of untested:  0
> >>>>>>> # of unsupported:  3
> >>>>>>> # of xfailed:  0
> >>>>>>> # of undefined(test bug):  0    
> >>>
> >>> I booted v4.12-rc4 with your config (didn't even have to change
> >>> anything for my box), and checkout the v4.12-rc4 kernel on the test box
> >>> and ran ftracetest. Here's my result:
> >>>
> >>> # of passed:  40
> >>> # of failed:  0
> >>> # of unresolved:  0
> >>> # of untested:  0
> >>> # of unsupported:  6
> >>> # of xfailed:  0
> >>> # of undefined(test bug):  0
> >>>
> >>> The unsupported were:
> >>>
> >>> [29] event trigger - test histogram modifiers	[UNSUPPORTED]
> >>> [30] event trigger - test histogram trigger	[UNSUPPORTED]
> >>> [31] event trigger - test multiple histogram triggers	[UNSUPPORTED]
> >>>
> >>> [44] (instance)  event trigger - test histogram modifiers	[UNSUPPORTED]
> >>> [45] (instance)  event trigger - test histogram trigger	[UNSUPPORTED]
> >>> [46] (instance)  event trigger - test multiple histogram triggers	[UNSUPPORTED]
> >>>
> >>> Now why is this different? What compiler are you using to build your
> >>> kernel?
> >>>     
> >> gcc version 6.3.0 20170406 (Ubuntu 6.3.0-12ubuntu2) 
> >>
> >>
> >> I don't see any regressions. I use this system for stable release testing
> >> and haven't seen any issues. Does ftrace have a dependency on a specific
> >> gcc revision?  
> > 
> > It behaves differently with different gcc, as some support mfentry,
> > while others only use mcount. gcc 6.3.0 should be using mfentry. I'm
> > testing with 6.2, a build I made myself.
> > 
> > Just to confirm, you are running the 4.12-rc4 ftracetests on a 4.12-rc4
> > kernel and getting this error? Or are you running a different version
> > of ftracetest or a different version of the kernel?
> >   
> 
> I am running 4.12-rc4 ftracetests on a 4.12-rc4 kernel. I did a headers
> install before I built and ran kselftest.

Well, ftracetest is mainly bash scripts. So it shouldn't be affected by
any C libraries or headers.

> 
> btw tests should run on older kernels failing gracefully when feature isn't
> supported and/or enabled.

Correct, it should do that.

> 
> So the expectation is for ftracetest to run cleanly skipping tests on older
> kernels.

Note, when a bug is found, we do add the reproducer to ftracetest. So
depending on whether you have the latest stable, or the stable has a
fix, ftracetest can cause the machine to crash on older releases.

> 
> Would getting results for ftracetest on 4.11 with 4.11 ftracetest help you
> with debug?
> 

No, I'm trying to see why I get a completely different result from what
you have running the same version of the kernel, config and tests.

-- Steve