Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 08/11] selftests: net/fcnal: Replace sleep after server
 start with -k
To:     Leonard Crestez <cdleonard@gmail.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Shuah Khan <shuah@kernel.org>, David Ahern <dsahern@kernel.org>
Cc:     "David S. Miller" <davem@davemloft.net>,
        Ido Schimmel <idosch@nvidia.com>,
        Seth David Schoen <schoen@loyalty.org>,
        netdev@vger.kernel.org, linux-kselftest@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <cover.1633520807.git.cdleonard@gmail.com>
 <ec40bd7128a30e93b90ba888f3468f394617a010.1633520807.git.cdleonard@gmail.com>
 <43210038-b04b-3726-1355-d5f132f6c64e@gmail.com>
 <d6882c3f-4ecf-4b4e-c20e-09b88da4fbd6@gmail.com>
From:   David Ahern <dsahern@gmail.com>
Message-ID: <888962dc-8d55-4875-cf44-c0b8ebaa1978@gmail.com>
Date:   Wed, 6 Oct 2021 19:22:45 -0600
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <d6882c3f-4ecf-4b4e-c20e-09b88da4fbd6@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 10/6/21 3:35 PM, Leonard Crestez wrote:
> 
> I counted the [FAIL] or [ OK ] markers but not the output of nettest
> itself. I don't know what to look for, I guess I could diff the outputs?
> 
> Shouldn't it be sufficient to compare the exit codes of the nettest client?

mistakes happen. The 700+ tests that exist were verified by me when I
submitted the script - that each test passes when it should and fails
when it should. "FAIL" has many reasons. I tried to have separate exit
codes for nettest.c to capture the timeouts vs ECONNREFUSED, etc., but I
could easily have made a mistake. scanning the output is the best way.
Most of the 'supposed to fail' tests have a HINT saying why it should fail.

> 
> The output is also modified by a previous change to not capture server
> output separately and instead let it be combined with that of the
> client. That change is required for this one, doing out=$(nettest -k)
> does not return on fork unless the pipe is also closed.
> 
> I did not look at your change, mine is relatively minimal because it
> only changes who decide when the server goes into the background: the
> shell script or the server itself. This makes it work very easily even
> for tests with multiple server instances.

The logging issue is why I went with 1 binary do both server and client
after nettest.c got support for changing namespaces.