Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
To:     Eric Biggers <ebiggers@google.com>,
        "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>
CC:     Herbert Xu <herbert@gondor.apana.org.au>
Subject: RE: testmgr question
Thread-Topic: testmgr question
Thread-Index: AQHVELtKP84dFEilq0iFiqYO3+2dVaZ5E5TpgAAHQ2A=
Date:   Thu, 23 May 2019 19:44:08 +0000
Message-ID: <AM6PR09MB3523BE1CDAE2E25AC47A3FD6D2010@AM6PR09MB3523.eurprd09.prod.outlook.com>
References: <AM6PR09MB3523DA127897A981C9E0074FD2000@AM6PR09MB3523.eurprd09.prod.outlook.com>
 <20190522162737.GA132972@google.com>
 <AM6PR09MB3523F1C423891763D4B3BF7CD2010@AM6PR09MB3523.eurprd09.prod.outlook.com>
 <20190523190534.GB243994@google.com>
In-Reply-To: <20190523190534.GB243994@google.com>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: insidesecure.com does not
 designate permitted sender hosts)
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 05f5765b-c107-4cfa-70a6-08d6dfb705d0
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 May 2019 19:44:08.7600
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3c07df58-7760-4e85-afd5-84803eac70ce
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR09MB3011
Sender: linux-crypto-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

> -----Original Message-----
> From: Eric Biggers [mailto:ebiggers@google.com]
> Sent: Thursday, May 23, 2019 9:06 PM
> To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> <herbert@gondor.apana.org.au>
> Subject: Re: testmgr question
>
> On Thu, May 23, 2019 at 07:01:46AM +0000, Pascal Van Leeuwen wrote:
> > > -----Original Message-----
> > > From: Eric Biggers [mailto:ebiggers@google.com]
> > > Sent: Wednesday, May 22, 2019 6:28 PM
> > > To: Pascal Van Leeuwen <pvanleeuwen@insidesecure.com>
> > > Cc: linux-crypto-owner@vger.kernel.org; Herbert Xu
> > > <herbert@gondor.apana.org.au>
> > > Subject: Re: testmgr question
> > >
> > > On Wed, May 22, 2019 at 01:32:43PM +0000, Pascal Van Leeuwen wrote:
> > > > Ugh,
> > > >
> > > > I just synced my development branch with Linus' mainline tree (5.2-=
rc1)
> and
> > > > apparently inherited some new testmgr tests that are now failing on=
 the
> > > Inside
> > > > Secure driver. I managed to fix some trivial ones related to non-ze=
ro
> IV
> > > size
> > > > on ECB modes and error codes that differed from the expected ones, =
but
> now
> > > I'm
> > > > rather stuck with a hang case ... and I don't have a clue which
> particular
> > > test
> > > > is hanging or even which algorithm is being tested :-(
> > > >
> > > > Is there, by any chance, some magical debug switch available to mak=
e
> > > testmgr
> > > > print out which test it is actually *starting* to run?
> > > >
> > > > Regards,
> > > >
> > > > Pascal van Leeuwen
> > > > Silicon IP Architect, Multi-Protocol Engines @ Inside Secure
> > > > www.insidesecure.com
> > > >
> > >
> > > Not currently, but you can easily add some debugging messages to test=
mgr
> > > yourself.  E.g.,
> > >
> > > Print 'alg' and 'driver' at beginning of alg_test() to see which
> algorithm is
> > > starting to be tested.
> > >
> > > Print 'vec_name' and 'cfg->name' at beginning of test_hash_vec_cfg(),
> > > test_skcipher_vec_cfg(), and test_aead_vec_cfg() to see which test ve=
ctor
> is
> > > starting to be tested and under what configuration.
> > >
> > Thanks. I guess adding such debugging statements to testmgr is what I'v=
e
> been
> > doing all along. Like everyone else having to debug these issues, I gue=
ss
> ...
> > Therefore I assumed by now, there might have been some standard
> infrastructure
> > for that. Or maybe it was just a hint that such a thing might be useful=
 ;-)
> >
>
> testmgr already prints information when a test fails which is enough for =
most
> cases, and in my experience when it's not I need to add messages specific=
 to
> tracking down the particular issue anyway.  So that's why I haven't
> personally
> added more messages.  Feel free to send a patch, though.  Also, please
> continue
> any further discussion of this on linux-crypto.
>
When developing hardware drivers, when things go wrong, odds are fairly
significant that the whole thing just hangs (or is that just me? :-).
So I can imagine I'm not the only one adding these debug print statements,
which means effort is probably wasted here. But I do notice that I keep
adding and removing them/commenting them out as they're pretty annoying whe=
n
things don't actually hang ...

Usually just knowing which specific case fails is enough for me  to reason =
about
why it's failing. I rarely need more debugging information than that. Follo=
wing
a hash/HMAC operation is pretty impossible anyway unless you have a referen=
ce
implementation standing by to compare with.

Regards,
Pascal van Leeuwen
Silicon IP Architect, Multi-Protocol Engines @ Inside Secure
www.insidesecure.com