2017-12-05 11:43:29

by Paolo Pisati

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

On Fri, Dec 1, 2017 at 12:15 AM, Jim Davis <[email protected]> wrote:
>
> If I run /snap/bin/snapcraft --version now on that Ubuntu 16.04 VM I
> am getting 2.36+git2.ae61453.

Yep, the edge channel follows the bleeding edge:

$ snap info snapcraft
...
channels:
stable: 2.35 (794) 53MB classic
candidate: ↑
beta: 2.36.1 (876) 53MB classic
edge: 2.36.1+git6.3524514 (895) 53MB classic

you better switch to beta (and then stable when 2.36.1 gets promoted there):

$ snap refresh --beta snapcraft

> I noticed that if I tried to make snap-pkg with the O=/some/dir option
> the tar step failed but the rest of the build continued, and seems to
> have finished successfully. Should the snap-pkg target stop after a
> tar failure?

That is weird, how did you do that?

My experience with the O=... option (that mimicks the other rpm-pkg
and deb-pkg targets) is that, you first create the dir, copy there the
.config and it builds fine:

~/linux $ mke defconfig
...
~/linux $ mkdir /tmp/foobar
~/linux $ mv .config /tmp/foobar
~/linux $ git clean -ffdx
~/linux $ make O=/tmp/foobar snap-pkg
...
Snapped kernel_4.14.0+_amd64.snap
~/linux $ ls -la /tmp/foobar/snap/
-rw-r--r-- 1 flag flag 14114816 dic 5 12:41 kernel_4.14.0+_amd64.snap
drwxrwxr-x 3 flag flag 4096 dic 5 12:38 parts
drwxrwxr-x 4 flag flag 4096 dic 5 12:41 prime
drwxrwxr-x 3 flag flag 4096 dic 5 12:38 snap
-rw-rw-r-- 1 flag flag 274 dic 5 12:38 snapcraft.yaml
drwxrwxr-x 3 flag flag 4096 dic 5 12:41 stage




--
bye,
p.


2017-12-05 21:23:50

by Jim Davis

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

On Tue, Dec 5, 2017 at 4:43 AM, Paolo Pisati <[email protected]> wrote:

>> I noticed that if I tried to make snap-pkg with the O=/some/dir option
>> the tar step failed but the rest of the build continued, and seems to
>> have finished successfully. Should the snap-pkg target stop after a
>> tar failure?
>
> That is weird, how did you do that?

make O=/some/dir defconfig (or your-config-of-choice)
make O=/some/dir snap-pkg

The snap-pkg target is calling the src_tar command, which bails when
$(objtree) isn't $(srctree).

Is $(objtree) = $(srctree) a necessary restriction with snap builds?
It's a restriction for rpm builds for some reason or another, but I
thought that was just a peculiarity of the rpm build process. If the
snap's tarball could be built without using src_tar, like the buildtar
script does, then Bob's your uncle.

Though if the source tree is an essential part of the snap build then
perhaps the build should just exit if the tarball can't be generated
for whatever reason, including users doing weird things :)

--
Jim

2017-12-06 10:37:43

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

2017-12-06 6:23 GMT+09:00 Jim Davis <[email protected]>:
> On Tue, Dec 5, 2017 at 4:43 AM, Paolo Pisati <[email protected]> wrote:
>
>>> I noticed that if I tried to make snap-pkg with the O=/some/dir option
>>> the tar step failed but the rest of the build continued, and seems to
>>> have finished successfully. Should the snap-pkg target stop after a
>>> tar failure?
>>
>> That is weird, how did you do that?
>
> make O=/some/dir defconfig (or your-config-of-choice)
> make O=/some/dir snap-pkg
>
> The snap-pkg target is calling the src_tar command, which bails when
> $(objtree) isn't $(srctree).
>
> Is $(objtree) = $(srctree) a necessary restriction with snap builds?
> It's a restriction for rpm builds for some reason or another,

Rather, restriction for our build system.

Working directory is changed at the top Makefile if O= is given.
This option is propagated to sub-make.


> but I
> thought that was just a peculiarity of the rpm build process. If the
> snap's tarball could be built without using src_tar, like the buildtar
> script does, then Bob's your uncle.

Probably, you must reset O=, KBUILD_OUTPUT, KBUILD_SRC, etc.
before calling rpmbuild, snapcraft, etc.

This is not a peculiarity of rpmbuild or snapcraft.

I guess things are not so simple as they look at a glance.



> Though if the source tree is an essential part of the snap build then
> perhaps the build should just exit if the tarball can't be generated
> for whatever reason, including users doing weird things :)
>

I agree, but this is not a problem specific to snap-pkg,
so Paolo's work will not be blocked.

I sent this patch:
https://patchwork.kernel.org/patch/10095377/



Just one question to Paolo:

If some change like firmware removal happens in the future,
patching for snapcraft is necessary?
(i.e. users are required to upgrade snapcraft from --edge ?)

I guess the answer is yes, because "plugin: kernel" needs to know
the kernel-build system internal.

--
Best Regards
Masahiro Yamada

2017-12-06 18:14:33

by Paolo Pisati

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

On Wed, Dec 6, 2017 at 11:36 AM, Masahiro Yamada
<[email protected]> wrote:
> Just one question to Paolo:
>
> If some change like firmware removal happens in the future,
> patching for snapcraft is necessary?
> (i.e. users are required to upgrade snapcraft from --edge ?)
>
> I guess the answer is yes, because "plugin: kernel" needs to know
> the kernel-build system internal.

Well, in this case, i had to fix the 'kernel-with-firmware' option to
skip the 'firmware_install' step since
we took it by default - let's say that in the future the
'modules_install' target is removed or change name,
then i'll have to patch snapcraft.

Wrt updating from -edge, i told you to take it from there because snap
packages land there more often and more quickly
compared to the canonicaldeb archive, so from a developer perspective
it's easier&quicker to iterate over it, but sooner
or later (e.g. once snapcraft 2.36.1 reaches the snap -stable
channel), it will be copied to the deb archive too -
IOW, you would get using apt-get only too.
--
bye,
p.

2017-12-07 15:32:20

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

2017-12-07 3:14 GMT+09:00 Paolo Pisati <[email protected]>:
> On Wed, Dec 6, 2017 at 11:36 AM, Masahiro Yamada
> <[email protected]> wrote:
>> Just one question to Paolo:
>>
>> If some change like firmware removal happens in the future,
>> patching for snapcraft is necessary?
>> (i.e. users are required to upgrade snapcraft from --edge ?)
>>
>> I guess the answer is yes, because "plugin: kernel" needs to know
>> the kernel-build system internal.
>
> Well, in this case, i had to fix the 'kernel-with-firmware' option to
> skip the 'firmware_install' step since
> we took it by default - let's say that in the future the
> 'modules_install' target is removed or change name,
> then i'll have to patch snapcraft.
>
> Wrt updating from -edge, i told you to take it from there because snap
> packages land there more often and more quickly
> compared to the canonicaldeb archive, so from a developer perspective
> it's easier&quicker to iterate over it, but sooner
> or later (e.g. once snapcraft 2.36.1 reaches the snap -stable
> channel), it will be copied to the deb archive too -
> IOW, you would get using apt-get only too.
> --


I will pick up this for v4.16-rc1.
It sounds like this target is a bit fragile, but let's see.
I hope a breakage will not happen...


--
Best Regards
Masahiro Yamada

2017-12-12 15:09:09

by Masahiro Yamada

[permalink] [raw]
Subject: Re: [PATCH v3] scripts/package: snap-pkg target

2017-12-08 0:31 GMT+09:00 Masahiro Yamada <[email protected]>:
> 2017-12-07 3:14 GMT+09:00 Paolo Pisati <[email protected]>:
>> On Wed, Dec 6, 2017 at 11:36 AM, Masahiro Yamada
>> <[email protected]> wrote:
>>> Just one question to Paolo:
>>>
>>> If some change like firmware removal happens in the future,
>>> patching for snapcraft is necessary?
>>> (i.e. users are required to upgrade snapcraft from --edge ?)
>>>
>>> I guess the answer is yes, because "plugin: kernel" needs to know
>>> the kernel-build system internal.
>>
>> Well, in this case, i had to fix the 'kernel-with-firmware' option to
>> skip the 'firmware_install' step since
>> we took it by default - let's say that in the future the
>> 'modules_install' target is removed or change name,
>> then i'll have to patch snapcraft.
>>
>> Wrt updating from -edge, i told you to take it from there because snap
>> packages land there more often and more quickly
>> compared to the canonicaldeb archive, so from a developer perspective
>> it's easier&quicker to iterate over it, but sooner
>> or later (e.g. once snapcraft 2.36.1 reaches the snap -stable
>> channel), it will be copied to the deb archive too -
>> IOW, you would get using apt-get only too.
>> --
>
>
> I will pick up this for v4.16-rc1.
> It sounds like this target is a bit fragile, but let's see.
> I hope a breakage will not happen...
>


Applied to linux-kbuild/misc. Thanks!

--
Best Regards
Masahiro Yamada