Kernel 3.12.7 from kernel.org
With kmod-16, I'm unable to load any modules on my guest kvm machines.
The vm is booted via direct kernel boot. The modules are located on the
host and is passed to the guest via the fsdev.
I have a mountpoint on the guest filesystem located at /kernel. It is
mounted like this:
kernel /kernel 9p rw,sync,dirsync,relatime,trans=virtio,version=9p2000.L 0 0
/boot, /lib/modules, and /lib/firmware are tmpfs filesystems like this:
kboot /boot tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
kfirmware /lib/firmware tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
kmodules /lib/modules tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
/lib/modules/$(uname -r) is a symlink to /kernel/lib/modules/$(uname -r)
When trying to load any module, I get
Error: could not insert module /lib/modules/3.12.7/kernel/crypto/af_alg.ko:
Invalid module format
This module was just one I picked at random, all modules fail the same way.
Strace shows this:
open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f199aebd000
syscall_313(0x3, 0x7f199aaa2de0, 0, 0x3, 0, 0x7f199b7b2010, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = -1 (errno 8)
munmap(0x7f199aebd000, 13822) = 0
close(3) = 0
If I use kmod-9, it works w/o problems. An strace of kmod-9 shows:
open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f658ba41000
init_module(0x7f658ba41000, 13822, "") = 0
munmap(0x7f658ba41000, 13822) = 0
close(3) = 0
If I copy the module to /tmp, kmod-16 loads it just fine. It appears to be
related to the 9p filesystem.
I won't rule out a bug in kmod-16, but I was unable to figure out what the
cause of the error was. I tried to search through kmod's source and the
kernel source, but I gave up because I don't understand what's going on.
--
Microsoft has beaten Volkswagen's world record. Volkswagen only created 22
million bugs.
Wakko Warner wrote:
> Kernel 3.12.7 from kernel.org
> With kmod-16, I'm unable to load any modules on my guest kvm machines.
> The vm is booted via direct kernel boot. The modules are located on the
> host and is passed to the guest via the fsdev.
>
> I have a mountpoint on the guest filesystem located at /kernel. It is
> mounted like this:
> kernel /kernel 9p rw,sync,dirsync,relatime,trans=virtio,version=9p2000.L 0 0
>
> /boot, /lib/modules, and /lib/firmware are tmpfs filesystems like this:
> kboot /boot tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
> kfirmware /lib/firmware tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
> kmodules /lib/modules tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
>
> /lib/modules/$(uname -r) is a symlink to /kernel/lib/modules/$(uname -r)
>
> When trying to load any module, I get
> Error: could not insert module /lib/modules/3.12.7/kernel/crypto/af_alg.ko:
> Invalid module format
>
> This module was just one I picked at random, all modules fail the same way.
> Strace shows this:
> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f199aebd000
> syscall_313(0x3, 0x7f199aaa2de0, 0, 0x3, 0, 0x7f199b7b2010, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = -1 (errno 8)
> munmap(0x7f199aebd000, 13822) = 0
> close(3) = 0
>
> If I use kmod-9, it works w/o problems. An strace of kmod-9 shows:
> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f658ba41000
> init_module(0x7f658ba41000, 13822, "") = 0
> munmap(0x7f658ba41000, 13822) = 0
> close(3) = 0
>
> If I copy the module to /tmp, kmod-16 loads it just fine. It appears to be
> related to the 9p filesystem.
>
> I won't rule out a bug in kmod-16, but I was unable to figure out what the
> cause of the error was. I tried to search through kmod's source and the
> kernel source, but I gave up because I don't understand what's going on.
I did some testing with the various kmod versions between kmod-9 and 16.
The problem was introduced in 13. The change that broke this was the
addition of finit_module(). Appears that the kernel cannot handle modules
that is on the 9p filesystem. On the line that checked err == 0 (around
line libkmod-module.c:823), I added a test for ENOEXEC. The finit_module
still fails,but it uses the old method and works.
I would say from what I've seen, the problem is in the kernel for
finit_module(). I don't know enough about the kernel to go any further.
[adding Cc:s]
On 01/10/2014 03:03 PM, Wakko Warner wrote:
> Wakko Warner wrote:
>> Kernel 3.12.7 from kernel.org
>> With kmod-16, I'm unable to load any modules on my guest kvm machines.
>> The vm is booted via direct kernel boot. The modules are located on the
>> host and is passed to the guest via the fsdev.
>>
>> I have a mountpoint on the guest filesystem located at /kernel. It is
>> mounted like this:
>> kernel /kernel 9p rw,sync,dirsync,relatime,trans=virtio,version=9p2000.L 0 0
>>
>> /boot, /lib/modules, and /lib/firmware are tmpfs filesystems like this:
>> kboot /boot tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
>> kfirmware /lib/firmware tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
>> kmodules /lib/modules tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
>>
>> /lib/modules/$(uname -r) is a symlink to /kernel/lib/modules/$(uname -r)
>>
>> When trying to load any module, I get
>> Error: could not insert module /lib/modules/3.12.7/kernel/crypto/af_alg.ko:
>> Invalid module format
>>
>> This module was just one I picked at random, all modules fail the same way.
>> Strace shows this:
>> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
>> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f199aebd000
>> syscall_313(0x3, 0x7f199aaa2de0, 0, 0x3, 0, 0x7f199b7b2010, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = -1 (errno 8)
>> munmap(0x7f199aebd000, 13822) = 0
>> close(3) = 0
>>
>> If I use kmod-9, it works w/o problems. An strace of kmod-9 shows:
>> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
>> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
>> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f658ba41000
>> init_module(0x7f658ba41000, 13822, "") = 0
>> munmap(0x7f658ba41000, 13822) = 0
>> close(3) = 0
>>
>> If I copy the module to /tmp, kmod-16 loads it just fine. It appears to be
>> related to the 9p filesystem.
>>
>> I won't rule out a bug in kmod-16, but I was unable to figure out what the
>> cause of the error was. I tried to search through kmod's source and the
>> kernel source, but I gave up because I don't understand what's going on.
>
> I did some testing with the various kmod versions between kmod-9 and 16.
> The problem was introduced in 13. The change that broke this was the
> addition of finit_module(). Appears that the kernel cannot handle modules
> that is on the 9p filesystem. On the line that checked err == 0 (around
> line libkmod-module.c:823), I added a test for ENOEXEC. The finit_module
> still fails,but it uses the old method and works.
>
> I would say from what I've seen, the problem is in the kernel for
> finit_module(). I don't know enough about the kernel to go any further.
> --
--
~Randy
Hi,
Randy Dunlap wrote on Fri, Jan 10, 2014 :
> [adding Cc:s]
Thanks for the extra Cc.
> On 01/10/2014 03:03 PM, Wakko Warner wrote:
> > Wakko Warner wrote:
> >> Kernel 3.12.7 from kernel.org
> >> With kmod-16, I'm unable to load any modules on my guest kvm machines.
> >> The vm is booted via direct kernel boot. The modules are located on the
> >> host and is passed to the guest via the fsdev.
> >>
> >> I have a mountpoint on the guest filesystem located at /kernel. It is
> >> mounted like this:
> >> kernel /kernel 9p rw,sync,dirsync,relatime,trans=virtio,version=9p2000.L 0 0
> >>
> >> /boot, /lib/modules, and /lib/firmware are tmpfs filesystems like this:
> >> kboot /boot tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
> >> kfirmware /lib/firmware tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
> >> kmodules /lib/modules tmpfs rw,relatime,size=0k,nr_inodes=8 0 0
> >>
> >> /lib/modules/$(uname -r) is a symlink to /kernel/lib/modules/$(uname -r)
> >>
> >> When trying to load any module, I get
> >> Error: could not insert module /lib/modules/3.12.7/kernel/crypto/af_alg.ko:
> >> Invalid module format
> >>
> >> This module was just one I picked at random, all modules fail the same way.
> >> Strace shows this:
> >> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
> >> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f199aebd000
> >> syscall_313(0x3, 0x7f199aaa2de0, 0, 0x3, 0, 0x7f199b7b2010, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = -1 (errno 8)
> >> munmap(0x7f199aebd000, 13822) = 0
> >> close(3) = 0
I didn't look at the patch very closely because the mail apparently
never reached v9fs-developer (although it looks in copy), but I got
pointed out this patch recently:
http://www.spinics.net/lists/linux-virtualization/msg21716.html
Which quotes "This patch enables 9p-virtio to correctly handle this
case. This not only enables us to load Linux kernel modules off virtfs,"
Perhaps would it be what you need to support this syscall_313?
Good luck,
--
Dominique Martinet
Dominique Martinet wrote:
> Hi,
Hi.
The following was trimmed.
> > On 01/10/2014 03:03 PM, Wakko Warner wrote:
> > > Wakko Warner wrote:
> > >> Kernel 3.12.7 from kernel.org
> > >> With kmod-16, I'm unable to load any modules on my guest kvm machines.
> > >> open("/lib/modules/3.12.7/kernel/crypto/af_alg.ko", O_RDONLY|O_CLOEXEC) = 3
> > >> fstat(3, {st_mode=S_IFREG|0644, st_size=13822, ...}) = 0
> > >> mmap(NULL, 13822, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f199aebd000
> > >> syscall_313(0x3, 0x7f199aaa2de0, 0, 0x3, 0, 0x7f199b7b2010, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0) = -1 (errno 8)
> > >> munmap(0x7f199aebd000, 13822) = 0
> > >> close(3) = 0
>
> I didn't look at the patch very closely because the mail apparently
> never reached v9fs-developer (although it looks in copy), but I got
> pointed out this patch recently:
> http://www.spinics.net/lists/linux-virtualization/msg21716.html
I applied the patch on that page. Here's my result:
# df -T /lib/modules/3.12.7/kernel
Filesystem Type 1K-blocks Used Available Use% Mounted on
kernel 9p 2015056 721356 1172460 39% /kernel
# uname -r
3.12.7
# modprobe -V
kmod version 16
# lsmod
Module Size Used by
nfsd 217278 2
auth_rpcgss 41307 1 nfsd
oid_registry 2179 1 auth_rpcgss
exportfs 3499 1 nfsd
nfs 111109 0
...
> Which quotes "This patch enables 9p-virtio to correctly handle this
> case. This not only enables us to load Linux kernel modules off virtfs,"
>
> Perhaps would it be what you need to support this syscall_313?
Looks like that's the fix.
--
Microsoft has beaten Volkswagen's world record. Volkswagen only created 22
million bugs.