Hi,
I have a general question.
How do we normally verify linux-next tree?
I wanted to work on linux-next but I am facing some issues.
I could able to build linux-next for both x86 and arm, but I could not
verify it on any machine.
Currently I don't have a real Linux PC to boot with linux-next kernel.
So I am trying to find alternative ways, like using the virtual box or qemu-arm.
1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
kernel is not booting.
2) For qemu-arm (versatilepb), I am able to build the kernel, but I
could not figure out which rootfs to use with it.
I tried creating minimal rootfs using busybox, but it does not contain
enough interface. I am not able to open multiple terminal and also
could not setup ssh to access it using PUTTY.
So, if you know of any better rootfs to use with qemu-arm please let me know.
Or, if you know of any better option to use linux-next please tell me.
It will be really helpful.
Thank You!
Regards,
Pintu
** Re sending **
Hi,
I have a general question.
How do we normally verify linux-next tree?
I wanted to work on linux-next but I am facing some issues.
I could able to build linux-next for both x86 and arm, but I could not
verify it on any machine.
Currently I don't have a real Linux PC to boot with linux-next kernel.
So I am trying to find alternative ways, like using the virtual box or qemu-arm.
1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
kernel is not booting.
2) For qemu-arm (versatilepb), I am able to build the kernel, but I
could not figure out which rootfs to use with it.
I tried creating minimal rootfs using busybox, but it does not contain
enough interface. I am not able to open multiple terminal and also
could not setup ssh to access it using PUTTY.
So, if you know of any better rootfs to use with qemu-arm please let me know.
Or, if you know of any better option to use linux-next please tell me.
It will be really helpful.
Thank You!
Regards,
Pintu
On Fri, Sep 29, 2017 at 3:12 PM, Pintu Kumar <[email protected]> wrote:
> Hi,
>
> I have a general question.
> How do we normally verify linux-next tree?
>
> I wanted to work on linux-next but I am facing some issues.
> I could able to build linux-next for both x86 and arm, but I could not
> verify it on any machine.
> Currently I don't have a real Linux PC to boot with linux-next kernel.
> So I am trying to find alternative ways, like using the virtual box or qemu-arm.
> 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> kernel is not booting.
> 2) For qemu-arm (versatilepb), I am able to build the kernel, but I
> could not figure out which rootfs to use with it.
> I tried creating minimal rootfs using busybox, but it does not contain
> enough interface. I am not able to open multiple terminal and also
> could not setup ssh to access it using PUTTY.
>
> So, if you know of any better rootfs to use with qemu-arm please let me know.
> Or, if you know of any better option to use linux-next please tell me.
> It will be really helpful.
>
>
> Thank You!
> Regards,
> Pintu
Hi,
I have a general question.
How do we normally verify linux-next tree?
I wanted to work on linux-next but I am facing some issues.
I could able to build linux-next for both x86 and arm, but I could not
verify it on any machine.
Currently I don't have a real Linux PC to boot with linux-next kernel.
So I am trying to find alternative ways, like using the virtual box or qemu-arm.
1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
kernel is not booting.
2) For qemu-arm (versatilepb), I am able to build the kernel, but I
could not figure out which rootfs to use with it.
I tried creating minimal rootfs using busybox, but it does not contain
enough interface. I am not able to open multiple terminal and also
could not setup ssh to access it using PUTTY.
So, if you know of any better rootfs to use with qemu-arm please let me know.
Or, if you know of any better option to use linux-next please tell me.
It will be really helpful.
Thank You!
Regards,
Pintu
On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
> I have a general question.
> How do we normally verify linux-next tree?
The same exact way you "verify" any other Linux kernel, for whatever
definition of "verify" you plan to use.
> 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> kernel is not booting.
Does an Ubuntu kernel boot correctly under VirtualBox? If not, fix
that issue first. Also, "is not booting" isn't detailed enough for anybody
to make even a guess as to what's wrong.
Also, note that 5.1.28 is out.
Hello,
Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28
Host: Windows 10
Guest: Ubuntu 16.04
Best regards
Damian
Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
[email protected]:
> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>
> >
> > I have a general question.
> > How do we normally verify linux-next tree?
> The same exact way you "verify" any other Linux kernel, for whatever
> definition of "verify" you plan to use.
>
> >
> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> > kernel is not booting.
> Does an Ubuntu kernel boot correctly under VirtualBox? If not, fix
> that issue first. Also, "is not booting" isn't detailed enough for
> anybody
> to make even a guess as to what's wrong.
>
> Also, note that 5.1.28 is out.
Hello,
i can tell you ubuntu 16.04 works under virtualbox 5.1.28 with the
current linux-next kernel
My maschine:
Host: Windows 10
Guest: Ubuntu 16.04
Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
[email protected]:
> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>
> >
> > I have a general question.
> > How do we normally verify linux-next tree?
> The same exact way you "verify" any other Linux kernel, for whatever
> definition of "verify" you plan to use.
>
> >
> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> > kernel is not booting.
> Does an Ubuntu kernel boot correctly under VirtualBox? If not, fix
> that issue first. Also, "is not booting" isn't detailed enough for
> anybody
> to make even a guess as to what's wrong.
>
> Also, note that 5.1.28 is out.
On Fri, Sep 29, 2017 at 6:44 PM, Damian Tometzki
<[email protected]> wrote:
> Hello,
>
> Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28
>
> Host: Windows 10
> Guest: Ubuntu 16.04
>
> Best regards
> Damian
>
>
> Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
> [email protected]:
>> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>>
>> >
>> > I have a general question.
>> > How do we normally verify linux-next tree?
>> The same exact way you "verify" any other Linux kernel, for whatever
>> definition of "verify" you plan to use.
>>
>> >
>> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
>> > kernel is not booting.
>> Does an Ubuntu kernel boot correctly under VirtualBox? If not, fix
>> that issue first. Also, "is not booting" isn't detailed enough for
>> anybody
>> to make even a guess as to what's wrong.
>>
>> Also, note that 5.1.28 is out.
Ok, I just updated to 5.1.28. And my Ubuntu version is already 16.04.
Let me try again if it works.
Thanks all for your reply.
BTW, I am more interested in my another query about QEMU arm.
This will be much quicker and easy for me.
But the problem is I wanted to use multiple ssh shell on qemu.
Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
versatilepb machine.
It should have networking and ssh built-in so that I can connect to it
using PUTTY client.
Note that I could able to build my own minimal busybox and boot qemu
using linux-next (non graphical mode).
But I could able to get only one shell.
I need to test some client/server problem, so I need multiple shell.
I am not able to configure net/ssh on this qemu system.
So, I have 2 things to ask:
1) If you have pointers on how to setup ssh/net connection on QEMU
with busybox, do let me know.
2) Let, please point me to a pre-built qemu-arm busy box image with
full features.
Thanks,
Pintu
On Fri, 29 Sep 2017 19:56:41 +0530, Pintu Kumar said:
> 1) If you have pointers on how to setup ssh/net connection on QEMU
> with busybox, do let me know.
Busybox doesn't do that as far as I know, as it's intended as a single-user
/sbin/init replacement. You'll need a full-featured userspace with an actual
init daemon (sysvinit, systemd, etc) and an ssh daemon (openssh, or if you want
something smaller, dropbear - works well on embedded things like routers...)
And of course things like /bin/login and all the /etc files that needs in order to work...
On Fri, Sep 29, 2017 at 7:56 PM, Pintu Kumar <[email protected]> wrote:
> On Fri, Sep 29, 2017 at 6:44 PM, Damian Tometzki
> <[email protected]> wrote:
>> Hello,
>>
>> Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28
>>
>> Host: Windows 10
>> Guest: Ubuntu 16.04
>>
>> Best regards
>> Damian
>>
>>
>> Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
>> [email protected]:
>>> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>>>
>>> >
>>> > I have a general question.
>>> > How do we normally verify linux-next tree?
>>> The same exact way you "verify" any other Linux kernel, for whatever
>>> definition of "verify" you plan to use.
>>>
>>> >
>>> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
>>> > kernel is not booting.
>>> Does an Ubuntu kernel boot correctly under VirtualBox? If not, fix
>>> that issue first. Also, "is not booting" isn't detailed enough for
>>> anybody
>>> to make even a guess as to what's wrong.
>>>
>>> Also, note that 5.1.28 is out.
>
> Ok, I just updated to 5.1.28. And my Ubuntu version is already 16.04.
> Let me try again if it works.
> Thanks all for your reply.
>
Now, with vbox 5.1.28, I am getting below build failure with linux-next.
Any quick pointers on this, if anybody faced similar issue.
In file included from ./arch/x86/include/asm/atomic.h:7:0,
from ./include/linux/atomic.h:4,
from ./include/linux/mm_types_task.h:12,
from ./include/linux/mm_types.h:4,
from arch/x86/kvm/irq.h:25,
from arch/x86/kvm/vmx.c:19:
arch/x86/kvm/vmx.c: In function ‘__pi_post_block’:
./arch/x86/include/asm/cmpxchg.h:129:2: warning: ‘__ret’ is used
uninitialized in this function [-Wuninitialized]
__ret; \
^
./arch/x86/include/asm/cmpxchg.h:86:21: note: ‘__ret’ was declared here
__typeof__(*(ptr)) __ret; \
^
./arch/x86/include/asm/cmpxchg.h:133:2: note: in expansion of macro
‘__raw_cmpxchg’
__raw_cmpxchg((ptr), (old), (new), (size), LOCK_PREFIX)
^
./arch/x86/include/asm/cmpxchg.h:148:2: note: in expansion of macro ‘__cmpxchg’
__cmpxchg(ptr, old, new, sizeof(*(ptr)))
^
arch/x86/kvm/vmx.c:11732:11: note: in expansion of macro ‘cmpxchg’
} while (cmpxchg(&pi_desc->control, old.control,
^
CC kernel/trace/trace_seq.o
CC kernel/trace/trace_stat.o
In function ‘__pi_post_block’,
inlined from ‘pi_post_block’ at arch/x86/kvm/vmx.c:11831:2,
inlined from ‘vmx_post_block’ at arch/x86/kvm/vmx.c:11840:2:
./arch/x86/include/asm/cmpxchg.h:127:3: error: call to
‘__cmpxchg_wrong_size’ declared with attribute error: Bad argument
size for cmpxchg
__cmpxchg_wrong_size(); \
^
> BTW, I am more interested in my another query about QEMU arm.
> This will be much quicker and easy for me.
> But the problem is I wanted to use multiple ssh shell on qemu.
> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
> versatilepb machine.
> It should have networking and ssh built-in so that I can connect to it
> using PUTTY client.
> Note that I could able to build my own minimal busybox and boot qemu
> using linux-next (non graphical mode).
> But I could able to get only one shell.
> I need to test some client/server problem, so I need multiple shell.
> I am not able to configure net/ssh on this qemu system.
>
> So, I have 2 things to ask:
> 1) If you have pointers on how to setup ssh/net connection on QEMU
> with busybox, do let me know.
> 2) Let, please point me to a pre-built qemu-arm busy box image with
> full features.
>
>
> Thanks,
> Pintu
On Fri, Sep 29, 2017 at 07:56:41PM +0530, Pintu Kumar wrote:
> BTW, I am more interested in my another query about QEMU arm.
> This will be much quicker and easy for me.
> But the problem is I wanted to use multiple ssh shell on qemu.
> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
> versatilepb machine.
If you want to get more useful help, it might be interesting if you
were to specify exactly what kind of "verification" you are interested
in doing. What sort of kernel testing are you interested in doing?
What part of the kernel are you interested in testing? The fact that
you are trying to test both a Ubuntu x86 box as will as a virtual ARM
box makes it unclear what part of the kernel you are most intested in
testing.
In particular, why do you care about accessing the VM via ssh /
networking? What sort of testing do you plan to do after manage to
get the kernel running? And do you care what distribution you use?
I have a huge amount of test automation built for testing kernel file
systems. This includes building root_fs images for x86 for use with
kvm[1], and arm chroots for use in testing Android systems[2]. There
is also a turn-key images for running tests using the Google Cloud
Platform[3], and even a Dockerfile[4] so people can run kernel tests
using a private Kubernetes cluster.
[1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
[2] https://thunk.org/android-xfstests
[3] https://thunk.org/gce-xfstests
[4] https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/Dockerfile
If you don't have a file-system centric view of the world, and want to
do more generalized kernel testing, the above might not be as
interesting to you, although some of the utilities in the xfstests-bld
git tree for setting up and building in build chroots, using
debootstrap to create root_fs.img files, scripts for manipulating
xUnit test result files (the XML format used by Jenkins), using 9p to
communicate between the host system running qemu/kvm and the test VM,
etc.
The point is that if you really want to get serious about kernel
testing, you should really think hard about test automation. And in
that world, using networking often makes things harder, not easier.
For kvm-xfstests we just do our communications using the serial port,
which made it easy for us to adapt things for android-xfstests, where
we comunicate test runner script via "adb shell". For gce-xfstests
things _do_ get a bit more complicated, where the test summary gets
e-mail'ed back to the developer, while the full set of test artifacts
are archived on Google Cloud Storage. But one of the most powerful
things about my setup is vast majority of the test automation code
stays the same regardless of whether the kernel being tested is being
run in KVM, on a physical Android hardware, or in the Cloud using
GCE.
> 2) Let, please point me to a pre-built qemu-arm busy box image with
> full features.
Define "full features". Busy box images are generally _not_ full
featured. There is a reason why I use a minimal Debian system; a lot
of the tests I run require bash, and modern shell utilities, and
Python so I can have scripts which manipulate xUnit XML files.
Nevertheless, the complete x86 test VM is still only 87 megs, which is
still small enough that it doesn't cause me any problems.
On the other hand, since I find networking in the test VM to be
completely superfluous (and in fact, gets in the way, since a VM which
is on the corporate network can be a security problem, and may run
afoul of corporate I/T security policies --- and if you don't have
those kinds of security policies, you really should....). So my
root_fs's general have no networking support whatsoever. It keeps
$WORK's secops team *much* happier. :-)
- Ted
Thanks Mr. Tso for your reply.
Please find my reply inline.
On Sat, Sep 30, 2017 at 3:20 AM, Theodore Ts'o <[email protected]> wrote:
> On Fri, Sep 29, 2017 at 07:56:41PM +0530, Pintu Kumar wrote:
>> BTW, I am more interested in my another query about QEMU arm.
>> This will be much quicker and easy for me.
>> But the problem is I wanted to use multiple ssh shell on qemu.
>> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
>> versatilepb machine.
>
> If you want to get more useful help, it might be interesting if you
> were to specify exactly what kind of "verification" you are interested
> in doing. What sort of kernel testing are you interested in doing?
> What part of the kernel are you interested in testing? The fact that
> you are trying to test both a Ubuntu x86 box as will as a virtual ARM
> box makes it unclear what part of the kernel you are most intested in
> testing.
>
I need to submit a patch to mainline which should be verified against
linux-next tree with latest API.
My patch is already working with 4.10 LTS version but I need to upgrade.
> In particular, why do you care about accessing the VM via ssh /
> networking? What sort of testing do you plan to do after manage to
> get the kernel running? And do you care what distribution you use?
>
My patch is related to some test utility based on client/server model.
So, I need 2 terminal, one for server and one for client.
No, I really don't care about distribution, whichever works this way is
good for me. So I a trying both ways: Ububntu(x86) or qemu (arm).
The point is, I should be able to test my patch with linux-next.
> I have a huge amount of test automation built for testing kernel file
> systems. This includes building root_fs images for x86 for use with
> kvm[1], and arm chroots for use in testing Android systems[2]. There
> is also a turn-key images for running tests using the Google Cloud
> Platform[3], and even a Dockerfile[4] so people can run kernel tests
> using a private Kubernetes cluster.
>
> [1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
> [2] https://thunk.org/android-xfstests
> [3] https://thunk.org/gce-xfstests
> [4] https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/Dockerfile
>
> If you don't have a file-system centric view of the world, and want to
> do more generalized kernel testing, the above might not be as
> interesting to you, although some of the utilities in the xfstests-bld
> git tree for setting up and building in build chroots, using
> debootstrap to create root_fs.img files, scripts for manipulating
> xUnit test result files (the XML format used by Jenkins), using 9p to
> communicate between the host system running qemu/kvm and the test VM,
> etc.
>
> The point is that if you really want to get serious about kernel
> testing, you should really think hard about test automation. And in
> that world, using networking often makes things harder, not easier.
> For kvm-xfstests we just do our communications using the serial port,
> which made it easy for us to adapt things for android-xfstests, where
> we comunicate test runner script via "adb shell". For gce-xfstests
> things _do_ get a bit more complicated, where the test summary gets
> e-mail'ed back to the developer, while the full set of test artifacts
> are archived on Google Cloud Storage. But one of the most powerful
> things about my setup is vast majority of the test automation code
> stays the same regardless of whether the kernel being tested is being
> run in KVM, on a physical Android hardware, or in the Cloud using
> GCE.
>
>> 2) Let, please point me to a pre-built qemu-arm busy box image with
>> full features.
>
> Define "full features". Busy box images are generally _not_ full
> featured. There is a reason why I use a minimal Debian system; a lot
> of the tests I run require bash, and modern shell utilities, and
> Python so I can have scripts which manipulate xUnit XML files.
> Nevertheless, the complete x86 test VM is still only 87 megs, which is
> still small enough that it doesn't cause me any problems.
>
Qemu Busybox -> full feature for me means: I should be able to connect
Qemu with my PUTTY session, and open 2 terminal.
Moreover, I should be able to do scp to my qemu machine from my
ubuntu pc.
> On the other hand, since I find networking in the test VM to be
> completely superfluous (and in fact, gets in the way, since a VM which
> is on the corporate network can be a security problem, and may run
> afoul of corporate I/T security policies --- and if you don't have
> those kinds of security policies, you really should....). So my
> root_fs's general have no networking support whatsoever. It keeps
> $WORK's secops team *much* happier. :-)
>
I am really sorry for the confusion.
Ok, lets talk one by one.
1) How to resolve linux-next build error with ubuntu virtual box 5.1.28
Any quick pointers on this will really help me to quickly verify my
patch and submit.
In file included from ./arch/x86/include/asm/atomic.h:7:0,
from ./include/linux/atomic.h:4,
from ./include/linux/mm_types_task.h:12,
from ./include/linux/mm_types.h:4,
from arch/x86/kvm/irq.h:25,
from arch/x86/kvm/vmx.c:19:
arch/x86/kvm/vmx.c: In function ‘__pi_post_block’:
./arch/x86/include/asm/cmpxchg.h:129:2: warning: ‘__ret’ is used
uninitialized in this function [-Wuninitialized]
__ret; \
^
./arch/x86/include/asm/cmpxchg.h:86:21: note: ‘__ret’ was declared here
__typeof__(*(ptr)) __ret; \
^
./arch/x86/include/asm/cmpxchg.h:133:2: note: in expansion of macro
‘__raw_cmpxchg’
__raw_cmpxchg((ptr), (old), (new), (size), LOCK_PREFIX)
^
./arch/x86/include/asm/cmpxchg.h:148:2: note: in expansion of macro ‘__cmpxchg’
__cmpxchg(ptr, old, new, sizeof(*(ptr)))
^
arch/x86/kvm/vmx.c:11732:11: note: in expansion of macro ‘cmpxchg’
} while (cmpxchg(&pi_desc->control, old.control,
^
CC kernel/trace/trace_seq.o
CC kernel/trace/trace_stat.o
In function ‘__pi_post_block’,
inlined from ‘pi_post_block’ at arch/x86/kvm/vmx.c:11831:2,
inlined from ‘vmx_post_block’ at arch/x86/kvm/vmx.c:11840:2:
./arch/x86/include/asm/cmpxchg.h:127:3: error: call to
‘__cmpxchg_wrong_size’ declared with attribute error: Bad argument
size for cmpxchg
__cmpxchg_wrong_size(); \
> - Ted
On Sat, Sep 30, 2017 at 09:28:09AM +0530, Pintu Kumar wrote:
> I need to submit a patch to mainline which should be verified against
> linux-next tree with latest API.
If you want to verify a patch that you intend to submit upstream, my
suggestion is to *not* use linux-next, but rather use the latest
tagged -rc from Linus's tree. So for example, you might want to use
v4.14-rc2 as your base, and then apply your patch on top of v4.14-rc2.
And then test v4.14-rc2. That way you don't need to worry about
debugging problems that might be caused by code in other people's
development trees.
If you know which subsystem tree your commit is going to be sent to,
you might use as your base the current development branch of that
subsystem tree. But in general, it's fine to use something like
v4.14-rc2; if the subsystem maintainer you plan to be submitting your
patch has other preference, he or she will let you know, or take care
of rebasing your patch onto his subsystme tree.
> My patch is related to some test utility based on client/server model.
> So, I need 2 terminal, one for server and one for client.
That implies you're running the commands to run the test by hand. In
the ideal world, tests should be automated, even those that are using
client/server so that tests can be run unattended, over and over
again.
For example, here's an example of test involving a client and a server
in xfstests:
https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/generic/131
See? No terminal required, and certainly not two terminals!
Remember, it's important not just to run one test, because the risk is
that fixing one bug might cause a test regression somewhere else. So
when I "validate" a kernel, I'm running thousands of tests, just to
test the ext4 file system. For each bug that we fix, we try to add a
new automated test, so we can be sure that some future change doesn't
cause a bug to reappear. And if you're running hundreds or thousands
of tests, you certainly aren't going to be wanting to manually set up
each test by using putty to login to the VM using ssh!
> 1) How to resolve linux-next build error with ubuntu virtual box 5.1.28
Virtual box is not relevant. What is relevant is the kernel config
file you are using, and what compiler version / distro are you using
to build the kernel. And as I said, you're better off using something
like v4.14-rc2 instead of linux-next.
- Ted