Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: need company for kernel upgrade
To:     wharms@bfs.de
Cc:     Pavel Machek <pavel@ucw.cz>, Theodore Ts'o <tytso@mit.edu>,
        Aung <aung.aungkyawsoe@gmail.com>,
        lkml <linux-kernel@vger.kernel.org>
References: <5CE53BA9.4070906@bfs.de>
 <CABC7EG8NiiPycthdfb7Ng3MsxTvmmxk_LjcosM8ZD1F0CnuDFw@mail.gmail.com>
 <5CE64BC7.4010803@bfs.de> <20190524050116.GI2532@mit.edu>
 <20190528105853.GA21111@amd> <585d6508-ace2-02d8-95ca-8e437d7cec05@metux.net>
 <5CEE3D24.8000700@bfs.de>
From:   "Enrico Weigelt, metux IT consult" <lkml@metux.net>
Organization: metux IT consult
Message-ID: <d814a076-5f83-6c54-615b-80159108e63c@metux.net>
Date:   Wed, 29 May 2019 14:31:27 +0200
User-Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <5CEE3D24.8000700@bfs.de>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 29.05.19 10:04, walter harms wrote:

Hi,

>> basic rule: don't use vendor kernels for production, unless you
>> *really* *really* have to.
> 
> we have custom hardware, so this was needed. 

Custom hardware doesn't necessarily need a vendor kernel. Most cases
should be happy w/ board specific DTS, possibly some extra drivers,
sometimes perhaps a few extra tuning patches.

But: such development, IMHO, should always happen on recent mainline
and things that aren't really customer specific should be upstreamed
as early as possible.

Vendor kernels are a different thing: usually they pick some older
base versions, do lots of strange things to get their hw somehow
working, and - one of the worst things one could do - merge down things
from mainline into their fork. After a few iterations, the code gets
pretty much unmaintainable. Such things are probably nice for a sales
demo, perhaps emc tests, but certainly not for production.

Just have a look at typical android vendor kernels. Horrible.

>> HW vendors usually don't have the capacities to offer any decent
>> kernel support. there're only few exceptions (eg. phytec) that have
>> their own kernel hackers and actively participate in upstreaming.
> 
> We have noticed that also, *active* is a big point.

Yes, but most hw vendors even never have been active in the first place.

Even if a particular company is doing that in *some* area, it doesn't
necessarily mean they're doing it for your particular product. Just
look at Intel, and what mess the produced for their flopped sofia soc.

>> by the way: should we create a separate list for commercial topics ?
>>
> 
> I used linux-arm and it worked surprisingly well (not all mails went truh the list)
> but it look very silent and i was unsure if they were still alive. 

IIRC, linux-arm is for arm specific development. Sure, the chance of
finding some consultant there is better there, as arm just has a huge
market share in embedded world.

> A big win
> for everyone would be a FAQ/Lessons-learned something you can take to check a contract.
> the customer will know what to expect and the contractor will know what to offer.

hmm, if anybody else here is interested in that, let's start with that.
maybe these discussions might be better off-list ?

> Having something a list of minimum requirements like that FAQ would improve the stand of
> the hardware developed to support linux. Not everyone has a big budget or need to
> produce in millions like the raspi guys did, and even they have sometimes troubles.

Well, one of the basic rules, IMHO, would be: if you're going to do some
linux embedded project, you should do a proper evaluation:

* check whether the board is already supported by mainline kernel
  (try to get a running mainline kernel built by yourself)
* never use vendor kernels at all
* check whether your supplier has active kernel hackers and actually
  does mainline integration
* if you run into any problems here, call in a linux embedded and kernel
  export - *soon* in the project

You spend a few extra bucks for the consultant, but he'll protect you
from the wrong choices, eg. buying unsupported / badly supported
hardware.

In recent years, I had many clients that called me in far too late.

One eg. was trying to build a medical device, using custom boards (very
high hw development costs and long timeline, due to qualifactions, etc),
with fancy HMI, but picked imx53, where no *usable* gpu driver exists.
When I brought the bad news, the project already ran for several years
and nobody dared to restart board development. (in the meantime, long
after i've gone, they indeed create a new board w/ mx6).

Such things could easily prevented if people just wouldn't trust the
hw vendor at all and ask us (kernel hackers) early.


--mtx

-- 
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
info@metux.net -- +49-151-27565287