$ cat syscall.c
#include <unistd.h>
#include <sys/syscall.h>
int main(){return syscall(1000)!=-1;}
(pls see https://bugs.gentoo.org/show_bug.cgi?id=513308) gives at a 32 bit stable Gentoo Linux w/ kernel 3.15 :
Jun 16 18:29:42 n22 kernel: ------------[ cut here ]------------
Jun 16 18:29:42 n22 kernel: kernel BUG at kernel/auditsc.c:1525!
Jun 16 18:29:42 n22 kernel: invalid opcode: 0000 [#1] SMP
Jun 16 18:29:42 n22 kernel: Modules linked in: ip6t_REJECT ip6table_filter ip6_tables ipt_MASQUERADE xt_owner xt_LOG xt_limit xt_multiport ipt_REJECT xt_recent xt_conntrack xt_tcpudp nf_conntrack_ftp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables ctr ccm af_packet bridge stp llc ipv6 tun i915 cfbfillrect uvcvideo cfbimgblt i2c_algo_bit x86_pkg_temp_thermal arc4 iwldvm mac80211 coretemp fbcon bitblit softcursor font cfbcopyarea drm_kms_helper videobuf2_vmalloc videobuf2_memops usblp videobuf2_core kvm_intel videodev drm kvm iwlwifi intel_gtt psmouse evdev agpgart cfg80211 acpi_cpufreq video processor thermal sdhci_pci sdhci mmc_core fb wmi thermal_sys snd_hda_codec_conexant e1000e snd_hda_codec_generic 8250_pci battery tpm_tis tpm thinkpad_acpi nvram ac snd_hda_intel snd_hda_controller snd_hda_codec fbdev snd_pcm 8250 snd_timer i2c_i801 ptp snd serial_core rfkill hwmon button i2c_core pps_core soundcore aesni_intel xts aes
_i586 lrw gf128mul ablk_helper cryptd cbc fuse nfs lockd sunrpc dm_crypt dm_mod hid_monterey hid_microsoft hid_logitech hid_ezkey hid_cypress hid_chicony hid_cherry hid_belkin hid_apple hid_a4tech hid_generic usbhid hid sr_mod cdrom sg [last unloaded: microcode]
Jun 16 18:29:42 n22 kernel: CPU: 1 PID: 29269 Comm: a.out Not tainted 3.15.0 #3
Jun 16 18:29:42 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
Jun 16 18:29:42 n22 kernel: task: cb368aa0 ti: e4dee000 task.ti: e4dee000
Jun 16 18:29:42 n22 kernel: EIP: 0060:[<c10b6c70>] EFLAGS: 00010202 CPU: 1
Jun 16 18:29:42 n22 kernel: EIP is at __audit_syscall_entry+0xf0/0x100
Jun 16 18:29:42 n22 kernel: EAX: 40000003 EBX: f1a9a000 ECX: 00000000 EDX: 000000fc
Jun 16 18:29:42 n22 kernel: ESI: 00000001 EDI: cb368aa0 EBP: e4deffb0 ESP: e4deffa4
Jun 16 18:29:42 n22 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Jun 16 18:29:42 n22 kernel: CR0: 80050033 CR2: b75dd2c0 CR3: 22f69000 CR4: 000407f0
Jun 16 18:29:42 n22 kernel: Stack:
Jun 16 18:29:42 n22 kernel: 00000000 00000000 b76c8264 e4dee000 c14ca296 00000000 00000008 00000000
Jun 16 18:29:42 n22 kernel: b76c8264 b76c8264 000000fc 0000007b 0000007b 00000000 00000033 000000fc
Jun 16 18:29:42 n22 kernel: b76fab2c 00000073 00000246 bfcd3e1c 0000007b 807f7f7f 807f7f7f
Jun 16 18:29:42 n22 kernel: Call Trace:
Jun 16 18:29:42 n22 kernel: [<c14ca296>] sysenter_audit+0x1e/0x25
Jun 16 18:29:42 n22 kernel: Code: 7d fc 89 ec 5d c3 90 8d 74 26 00 c7 43 34 00 00 00 00 b9 b0 2a 66 c1 89 da c7 43 38 00 00 00 00 89 f8 e8 54 f6 ff ff 89 c6 eb 91 <0f> 0b 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 55 89 e5 57 56
Jun 16 18:29:42 n22 kernel: EIP: [<c10b6c70>] __audit_syscall_entry+0xf0/0x100 SS:ESP 0068:e4deffa4
Jun 16 18:29:42 n22 kernel: ---[ end trace eaa43aea29d8101e ]---
Jun 16 18:30:01 n22 crond[29299]: pam_unix(crond:session): session opened for user root by (uid=0)
Jun 16 18:30:01 n22 CROND[29303]: (root) CMD (/usr/lib/sa/sa1 60 15 )
Jun 16 18:30:01 n22 crond[29298]: pam_unix(crond:session): session opened for user root by (uid=0)
Jun 16 18:30:01 n22 CROND[29304]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
Jun 16 18:30:01 n22 CROND[29298]: pam_unix(crond:session): session closed for user root
--
Toralf
On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
> $ cat syscall.c
> #include <unistd.h>
> #include <sys/syscall.h>
> int main(){return syscall(1000)!=-1;}
>
> (pls see https://bugs.gentoo.org/show_bug.cgi?id=513308) gives at a 32 bit stable Gentoo Linux w/ kernel 3.15 :
>
> Jun 16 18:29:42 n22 kernel: ------------[ cut here ]------------
> Jun 16 18:29:42 n22 kernel: kernel BUG at kernel/auditsc.c:1525!
> Jun 16 18:29:42 n22 kernel: invalid opcode: 0000 [#1] SMP
> Jun 16 18:29:42 n22 kernel: Modules linked in: ip6t_REJECT ip6table_filter ip6_tables ipt_MASQUERADE xt_owner xt_LOG xt_limit xt_multiport ipt_REJECT xt_recent xt_conntrack xt_tcpudp nf_conntrack_ftp iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_filter ip_tables x_tables ctr ccm af_packet bridge stp llc ipv6 tun i915 cfbfillrect uvcvideo cfbimgblt i2c_algo_bit x86_pkg_temp_thermal arc4 iwldvm mac80211 coretemp fbcon bitblit softcursor font cfbcopyarea drm_kms_helper videobuf2_vmalloc videobuf2_memops usblp videobuf2_core kvm_intel videodev drm kvm iwlwifi intel_gtt psmouse evdev agpgart cfg80211 acpi_cpufreq video processor thermal sdhci_pci sdhci mmc_core fb wmi thermal_sys snd_hda_codec_conexant e1000e snd_hda_codec_generic 8250_pci battery tpm_tis tpm thinkpad_acpi nvram ac snd_hda_intel snd_hda_controller snd_hda_codec fbdev snd_pcm 8250 snd_timer i2c_i801 ptp snd serial_core rfkill hwmon button i2c_core pps_core soundcore aesni_intel xts aes
> _i586 lrw gf128mul ablk_helper cryptd cbc fuse nfs lockd sunrpc dm_crypt dm_mod hid_monterey hid_microsoft hid_logitech hid_ezkey hid_cypress hid_chicony hid_cherry hid_belkin hid_apple hid_a4tech hid_generic usbhid hid sr_mod cdrom sg [last unloaded: microcode]
> Jun 16 18:29:42 n22 kernel: CPU: 1 PID: 29269 Comm: a.out Not tainted 3.15.0 #3
> Jun 16 18:29:42 n22 kernel: Hardware name: LENOVO 4180F65/4180F65, BIOS 83ET75WW (1.45 ) 05/10/2013
> Jun 16 18:29:42 n22 kernel: task: cb368aa0 ti: e4dee000 task.ti: e4dee000
> Jun 16 18:29:42 n22 kernel: EIP: 0060:[<c10b6c70>] EFLAGS: 00010202 CPU: 1
> Jun 16 18:29:42 n22 kernel: EIP is at __audit_syscall_entry+0xf0/0x100
> Jun 16 18:29:42 n22 kernel: EAX: 40000003 EBX: f1a9a000 ECX: 00000000 EDX: 000000fc
> Jun 16 18:29:42 n22 kernel: ESI: 00000001 EDI: cb368aa0 EBP: e4deffb0 ESP: e4deffa4
> Jun 16 18:29:42 n22 kernel: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
> Jun 16 18:29:42 n22 kernel: CR0: 80050033 CR2: b75dd2c0 CR3: 22f69000 CR4: 000407f0
> Jun 16 18:29:42 n22 kernel: Stack:
> Jun 16 18:29:42 n22 kernel: 00000000 00000000 b76c8264 e4dee000 c14ca296 00000000 00000008 00000000
> Jun 16 18:29:42 n22 kernel: b76c8264 b76c8264 000000fc 0000007b 0000007b 00000000 00000033 000000fc
> Jun 16 18:29:42 n22 kernel: b76fab2c 00000073 00000246 bfcd3e1c 0000007b 807f7f7f 807f7f7f
> Jun 16 18:29:42 n22 kernel: Call Trace:
> Jun 16 18:29:42 n22 kernel: [<c14ca296>] sysenter_audit+0x1e/0x25
> Jun 16 18:29:42 n22 kernel: Code: 7d fc 89 ec 5d c3 90 8d 74 26 00 c7 43 34 00 00 00 00 b9 b0 2a 66 c1 89 da c7 43 38 00 00 00 00 89 f8 e8 54 f6 ff ff 89 c6 eb 91 <0f> 0b 8d b4 26 00 00 00 00 8d bc 27 00 00 00 00 55 89 e5 57 56
> Jun 16 18:29:42 n22 kernel: EIP: [<c10b6c70>] __audit_syscall_entry+0xf0/0x100 SS:ESP 0068:e4deffa4
> Jun 16 18:29:42 n22 kernel: ---[ end trace eaa43aea29d8101e ]---
> Jun 16 18:30:01 n22 crond[29299]: pam_unix(crond:session): session opened for user root by (uid=0)
> Jun 16 18:30:01 n22 CROND[29303]: (root) CMD (/usr/lib/sa/sa1 60 15 )
> Jun 16 18:30:01 n22 crond[29298]: pam_unix(crond:session): session opened for user root by (uid=0)
> Jun 16 18:30:01 n22 CROND[29304]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons )
> Jun 16 18:30:01 n22 CROND[29298]: pam_unix(crond:session): session closed for user root
I think this is the fix you need:
[PATCH 1/2] auditsc: audit_krule mask accesses need bounds checking
> --
> Toralf
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
--
Thanks,
//richard
On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
<[email protected]> wrote:
> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>> $ cat syscall.c
>> #include <unistd.h>
>> #include <sys/syscall.h>
>> int main(){return syscall(1000)!=-1;}
What architecture are you building for? On i386 and x86_64, 1000
shouldn't be big enough to trigger this.
--Andy
Am 16.06.2014 19:25, schrieb Andy Lutomirski:
> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
> <[email protected]> wrote:
>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>> $ cat syscall.c
>>> #include <unistd.h>
>>> #include <sys/syscall.h>
>>> int main(){return syscall(1000)!=-1;}
>
> What architecture are you building for? On i386 and x86_64, 1000
> shouldn't be big enough to trigger this.
Toralf, is this an UML kernel?
Thanks,
//richard
On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>> <[email protected]> wrote:
>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>> $ cat syscall.c
>>>> #include <unistd.h>
>>>> #include <sys/syscall.h>
>>>> int main(){return syscall(1000)!=-1;}
>>
>> What architecture are you building for? On i386 and x86_64, 1000
>> shouldn't be big enough to trigger this.
>
> Toralf, is this an UML kernel?
>
I'm also interested in the userspace architecture. If it's x32
userspace, then I'm not surprised that there's a problem.
--Andy
> Thanks,
> //richard
--
Andy Lutomirski
AMA Capital Management, LLC
On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>> <[email protected]> wrote:
>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>> $ cat syscall.c
>>>>> #include <unistd.h>
>>>>> #include <sys/syscall.h>
>>>>> int main(){return syscall(1000)!=-1;}
>>>
>>> What architecture are you building for? On i386 and x86_64, 1000
>>> shouldn't be big enough to trigger this.
>>
>> Toralf, is this an UML kernel?
>>
>
> I'm also interested in the userspace architecture. If it's x32
> userspace, then I'm not surprised that there's a problem.
It is a x86 system (ThinkPad T420) - not x32.
--
Toralf
cc: eparis. This might be a new audit bug.
On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>> <[email protected]> wrote:
>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>> $ cat syscall.c
>>>>>> #include <unistd.h>
>>>>>> #include <sys/syscall.h>
>>>>>> int main(){return syscall(1000)!=-1;}
>>>>
>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>> shouldn't be big enough to trigger this.
>>>
>>> Toralf, is this an UML kernel?
>>>
>>
>> I'm also interested in the userspace architecture. If it's x32
>> userspace, then I'm not surprised that there's a problem.
>
> It is a x86 system (ThinkPad T420) - not x32.
I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
BUG_ON(context->in_syscall || context->name_count);
Can you send the output of:
auditctl -l [run as root]
and
dmesg |grep audit
Are you using ptrace or anything like that (e.g. strace) when you
trigger this? Are you using a funny glibc version? Do you have
selinux or something like that enabled?
--Andy
On 06/16/2014 07:50 PM, Andy Lutomirski wrote:
> cc: eparis. This might be a new audit bug.
>
> On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
>> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>>> <[email protected]> wrote:
>>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>>> $ cat syscall.c
>>>>>>> #include <unistd.h>
>>>>>>> #include <sys/syscall.h>
>>>>>>> int main(){return syscall(1000)!=-1;}
>>>>>
>>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>>> shouldn't be big enough to trigger this.
>>>>
>>>> Toralf, is this an UML kernel?
>>>>
>>>
>>> I'm also interested in the userspace architecture. If it's x32
>>> userspace, then I'm not surprised that there's a problem.
>>
>> It is a x86 system (ThinkPad T420) - not x32.
>
> I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
>
> BUG_ON(context->in_syscall || context->name_count);
>
> Can you send the output of:
>
> auditctl -l [run as root]
>
> and
>
> dmesg |grep audit
>
> Are you using ptrace or anything like that (e.g. strace) when you
> trigger this? Are you using a funny glibc version? Do you have
> selinux or something like that enabled?
>
> --Andy
>
n22 ~ # auditctl -l
LIST_RULES: exit,never arch=1073741827 (0x40000003) syscall=read,write,open,close,brk,fcntl,dup2,mmap,munmap,stat,fstat,nanosleep,rt_sigaction
no ptrace/strace/SELinux, this is a stable x86 Gentoo Linux, glibc is 2.17, unstable are just KDE + Co.
(@Richard: no. it is not an UML guest, I just stumbled over this while I tried to upgrade an unstable ~x86 Gentoo UML image using chroot)
The trigger is just given by that C one-liner and kernel 3.15 (erm, I did not checked, if 3.14.x hit its too)
--
Toralf
On Mon, Jun 16, 2014 at 10:59 AM, Toralf Förster <[email protected]> wrote:
> On 06/16/2014 07:50 PM, Andy Lutomirski wrote:
>> cc: eparis. This might be a new audit bug.
>>
>> On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
>>> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>>>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>>>> <[email protected]> wrote:
>>>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>>>> $ cat syscall.c
>>>>>>>> #include <unistd.h>
>>>>>>>> #include <sys/syscall.h>
>>>>>>>> int main(){return syscall(1000)!=-1;}
>>>>>>
>>>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>>>> shouldn't be big enough to trigger this.
>>>>>
>>>>> Toralf, is this an UML kernel?
>>>>>
>>>>
>>>> I'm also interested in the userspace architecture. If it's x32
>>>> userspace, then I'm not surprised that there's a problem.
>>>
>>> It is a x86 system (ThinkPad T420) - not x32.
>>
>> I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
>>
>> BUG_ON(context->in_syscall || context->name_count);
>>
>> Can you send the output of:
>>
>> auditctl -l [run as root]
>>
>> and
>>
>> dmesg |grep audit
>>
>> Are you using ptrace or anything like that (e.g. strace) when you
>> trigger this? Are you using a funny glibc version? Do you have
>> selinux or something like that enabled?
>>
>> --Andy
>>
> n22 ~ # auditctl -l
> LIST_RULES: exit,never arch=1073741827 (0x40000003) syscall=read,write,open,close,brk,fcntl,dup2,mmap,munmap,stat,fstat,nanosleep,rt_sigaction
>
>
> no ptrace/strace/SELinux, this is a stable x86 Gentoo Linux, glibc is 2.17, unstable are just KDE + Co.
>
> (@Richard: no. it is not an UML guest, I just stumbled over this while I tried to upgrade an unstable ~x86 Gentoo UML image using chroot)
>
> The trigger is just given by that C one-liner and kernel 3.15 (erm, I did not checked, if 3.14.x hit its too)
At the very least, it looks like sysret_audit can result in invoking
the audit exit hook twice. That's not what's causing this, but it
still looks fishy.
Toralf, can you run your test program under strace, post the output,
and see whether it still crashes? There's some chance that strace
will "fix" it, since strace causes a different set of hooks to run.
Any ideas, Eric?
--Andy
On 06/16/2014 08:15 PM, Andy Lutomirski wrote:
> On Mon, Jun 16, 2014 at 10:59 AM, Toralf Förster <[email protected]> wrote:
>> On 06/16/2014 07:50 PM, Andy Lutomirski wrote:
>>> cc: eparis. This might be a new audit bug.
>>>
>>> On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
>>>> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>>>>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>>>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>>>>> <[email protected]> wrote:
>>>>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>>>>> $ cat syscall.c
>>>>>>>>> #include <unistd.h>
>>>>>>>>> #include <sys/syscall.h>
>>>>>>>>> int main(){return syscall(1000)!=-1;}
>>>>>>>
>>>>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>>>>> shouldn't be big enough to trigger this.
>>>>>>
>>>>>> Toralf, is this an UML kernel?
>>>>>>
>>>>>
>>>>> I'm also interested in the userspace architecture. If it's x32
>>>>> userspace, then I'm not surprised that there's a problem.
>>>>
>>>> It is a x86 system (ThinkPad T420) - not x32.
>>>
>>> I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
>>>
>>> BUG_ON(context->in_syscall || context->name_count);
>>>
>>> Can you send the output of:
>>>
>>> auditctl -l [run as root]
>>>
>>> and
>>>
>>> dmesg |grep audit
>>>
>>> Are you using ptrace or anything like that (e.g. strace) when you
>>> trigger this? Are you using a funny glibc version? Do you have
>>> selinux or something like that enabled?
>>>
>>> --Andy
>>>
>> n22 ~ # auditctl -l
>> LIST_RULES: exit,never arch=1073741827 (0x40000003) syscall=read,write,open,close,brk,fcntl,dup2,mmap,munmap,stat,fstat,nanosleep,rt_sigaction
>>
>>
>> no ptrace/strace/SELinux, this is a stable x86 Gentoo Linux, glibc is 2.17, unstable are just KDE + Co.
>>
>> (@Richard: no. it is not an UML guest, I just stumbled over this while I tried to upgrade an unstable ~x86 Gentoo UML image using chroot)
>>
>> The trigger is just given by that C one-liner and kernel 3.15 (erm, I did not checked, if 3.14.x hit its too)
>
> At the very least, it looks like sysret_audit can result in invoking
> the audit exit hook twice. That's not what's causing this, but it
> still looks fishy.
>
> Toralf, can you run your test program under strace, post the output,
> and see whether it still crashes? There's some chance that strace
> will "fix" it, since strace causes a different set of hooks to run.
>
tfoerste@n22 ~/tmp $ cat syscall.c
#include <unistd.h>
#include <sys/syscall.h>
int main(){return syscall(1000)!=-1;}
tfoerste@n22 ~/tmp $ gcc strcmp.c && strace ./a.out
execve("./a.out", ["./a.out"], [/* 75 vars */]) = 0
brk(0) = 0x85f3000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7750000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=164108, ...}) = 0
mmap2(NULL, 164108, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7727000
close(3) = 0
open("/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \316\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1750448, ...}) = 0
mmap2(NULL, 1759980, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7579000
mmap2(0xb7721000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a8000) = 0xb7721000
mmap2(0xb7724000, 10988, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7724000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7578000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb75786c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb7721000, 8192, PROT_READ) = 0
mprotect(0x8049000, 4096, PROT_READ) = 0
mprotect(0xb7774000, 4096, PROT_READ) = 0
munmap(0xb7727000, 164108) = 0
fstat64(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 5), ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb774f000
write(1, "1\n", 21
) = 2
exit_group(2) = ?
+++ exited with 2 +++
> Any ideas, Eric?
>
> --Andy
>
--
Toralf
On Mon, Jun 16, 2014 at 11:21 AM, Toralf Förster <[email protected]> wrote:
> On 06/16/2014 08:15 PM, Andy Lutomirski wrote:
>> On Mon, Jun 16, 2014 at 10:59 AM, Toralf Förster <[email protected]> wrote:
>>> On 06/16/2014 07:50 PM, Andy Lutomirski wrote:
>>>> cc: eparis. This might be a new audit bug.
>>>>
>>>> On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
>>>>> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>>>>>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>>>>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>>>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>>>>>> <[email protected]> wrote:
>>>>>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>>>>>> $ cat syscall.c
>>>>>>>>>> #include <unistd.h>
>>>>>>>>>> #include <sys/syscall.h>
>>>>>>>>>> int main(){return syscall(1000)!=-1;}
>>>>>>>>
>>>>>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>>>>>> shouldn't be big enough to trigger this.
>>>>>>>
>>>>>>> Toralf, is this an UML kernel?
>>>>>>>
>>>>>>
>>>>>> I'm also interested in the userspace architecture. If it's x32
>>>>>> userspace, then I'm not surprised that there's a problem.
>>>>>
>>>>> It is a x86 system (ThinkPad T420) - not x32.
>>>>
>>>> I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
>>>>
>>>> BUG_ON(context->in_syscall || context->name_count);
>>>>
>>>> Can you send the output of:
>>>>
>>>> auditctl -l [run as root]
>>>>
>>>> and
>>>>
>>>> dmesg |grep audit
>>>>
>>>> Are you using ptrace or anything like that (e.g. strace) when you
>>>> trigger this? Are you using a funny glibc version? Do you have
>>>> selinux or something like that enabled?
>>>>
>>>> --Andy
>>>>
>>> n22 ~ # auditctl -l
>>> LIST_RULES: exit,never arch=1073741827 (0x40000003) syscall=read,write,open,close,brk,fcntl,dup2,mmap,munmap,stat,fstat,nanosleep,rt_sigaction
>>>
>>>
>>> no ptrace/strace/SELinux, this is a stable x86 Gentoo Linux, glibc is 2.17, unstable are just KDE + Co.
>>>
>>> (@Richard: no. it is not an UML guest, I just stumbled over this while I tried to upgrade an unstable ~x86 Gentoo UML image using chroot)
>>>
>>> The trigger is just given by that C one-liner and kernel 3.15 (erm, I did not checked, if 3.14.x hit its too)
>>
>> At the very least, it looks like sysret_audit can result in invoking
>> the audit exit hook twice. That's not what's causing this, but it
>> still looks fishy.
>>
>> Toralf, can you run your test program under strace, post the output,
>> and see whether it still crashes? There's some chance that strace
>> will "fix" it, since strace causes a different set of hooks to run.
>>
> tfoerste@n22 ~/tmp $ cat syscall.c
> #include <unistd.h>
> #include <sys/syscall.h>
> int main(){return syscall(1000)!=-1;}
>
> tfoerste@n22 ~/tmp $ gcc strcmp.c && strace ./a.out
> execve("./a.out", ["./a.out"], [/* 75 vars */]) = 0
> brk(0) = 0x85f3000
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7750000
> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> fstat64(3, {st_mode=S_IFREG|0644, st_size=164108, ...}) = 0
> mmap2(NULL, 164108, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7727000
> close(3) = 0
> open("/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \316\1\0004\0\0\0"..., 512) = 512
> fstat64(3, {st_mode=S_IFREG|0755, st_size=1750448, ...}) = 0
> mmap2(NULL, 1759980, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7579000
> mmap2(0xb7721000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a8000) = 0xb7721000
> mmap2(0xb7724000, 10988, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7724000
> close(3) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7578000
> set_thread_area({entry_number:-1 -> 6, base_addr:0xb75786c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
> mprotect(0xb7721000, 8192, PROT_READ) = 0
> mprotect(0x8049000, 4096, PROT_READ) = 0
> mprotect(0xb7774000, 4096, PROT_READ) = 0
> munmap(0xb7727000, 164108) = 0
> fstat64(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 5), ...}) = 0
> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb774f000
> write(1, "1\n", 21
> ) = 2
Does this mean it didn't OOPS? And where's the write(1, "1\n", 2)
coming from? Are you sure you straced the right thing?
--Andy
On 06/16/2014 08:24 PM, Andy Lutomirski wrote:
> On Mon, Jun 16, 2014 at 11:21 AM, Toralf Förster <[email protected]> wrote:
>> On 06/16/2014 08:15 PM, Andy Lutomirski wrote:
>>> On Mon, Jun 16, 2014 at 10:59 AM, Toralf Förster <[email protected]> wrote:
>>>> On 06/16/2014 07:50 PM, Andy Lutomirski wrote:
>>>>> cc: eparis. This might be a new audit bug.
>>>>>
>>>>> On Mon, Jun 16, 2014 at 10:36 AM, Toralf Förster <[email protected]> wrote:
>>>>>> On 06/16/2014 07:32 PM, Andy Lutomirski wrote:
>>>>>>> On Mon, Jun 16, 2014 at 10:29 AM, Richard Weinberger <[email protected]> wrote:
>>>>>>>> Am 16.06.2014 19:25, schrieb Andy Lutomirski:
>>>>>>>>> On Mon, Jun 16, 2014 at 10:21 AM, Richard Weinberger
>>>>>>>>> <[email protected]> wrote:
>>>>>>>>>> On Mon, Jun 16, 2014 at 6:33 PM, Toralf Förster <[email protected]> wrote:
>>>>>>>>>>> $ cat syscall.c
>>>>>>>>>>> #include <unistd.h>
>>>>>>>>>>> #include <sys/syscall.h>
>>>>>>>>>>> int main(){return syscall(1000)!=-1;}
>>>>>>>>>
>>>>>>>>> What architecture are you building for? On i386 and x86_64, 1000
>>>>>>>>> shouldn't be big enough to trigger this.
>>>>>>>>
>>>>>>>> Toralf, is this an UML kernel?
>>>>>>>>
>>>>>>>
>>>>>>> I'm also interested in the userspace architecture. If it's x32
>>>>>>> userspace, then I'm not surprised that there's a problem.
>>>>>>
>>>>>> It is a x86 system (ThinkPad T420) - not x32.
>>>>>
>>>>> I don't think this is CVE-2014-3917. It looks like you're hitting this BUG:
>>>>>
>>>>> BUG_ON(context->in_syscall || context->name_count);
>>>>>
>>>>> Can you send the output of:
>>>>>
>>>>> auditctl -l [run as root]
>>>>>
>>>>> and
>>>>>
>>>>> dmesg |grep audit
>>>>>
>>>>> Are you using ptrace or anything like that (e.g. strace) when you
>>>>> trigger this? Are you using a funny glibc version? Do you have
>>>>> selinux or something like that enabled?
>>>>>
>>>>> --Andy
>>>>>
>>>> n22 ~ # auditctl -l
>>>> LIST_RULES: exit,never arch=1073741827 (0x40000003) syscall=read,write,open,close,brk,fcntl,dup2,mmap,munmap,stat,fstat,nanosleep,rt_sigaction
>>>>
>>>>
>>>> no ptrace/strace/SELinux, this is a stable x86 Gentoo Linux, glibc is 2.17, unstable are just KDE + Co.
>>>>
>>>> (@Richard: no. it is not an UML guest, I just stumbled over this while I tried to upgrade an unstable ~x86 Gentoo UML image using chroot)
>>>>
>>>> The trigger is just given by that C one-liner and kernel 3.15 (erm, I did not checked, if 3.14.x hit its too)
>>>
>>> At the very least, it looks like sysret_audit can result in invoking
>>> the audit exit hook twice. That's not what's causing this, but it
>>> still looks fishy.
>>>
>>> Toralf, can you run your test program under strace, post the output,
>>> and see whether it still crashes? There's some chance that strace
>>> will "fix" it, since strace causes a different set of hooks to run.
>>>
>> tfoerste@n22 ~/tmp $ cat syscall.c
>> #include <unistd.h>
>> #include <sys/syscall.h>
>> int main(){return syscall(1000)!=-1;}
>>
>> tfoerste@n22 ~/tmp $ gcc strcmp.c && strace ./a.out
>> execve("./a.out", ["./a.out"], [/* 75 vars */]) = 0
>> brk(0) = 0x85f3000
>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7750000
>> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
>> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
>> fstat64(3, {st_mode=S_IFREG|0644, st_size=164108, ...}) = 0
>> mmap2(NULL, 164108, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7727000
>> close(3) = 0
>> open("/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
>> read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \316\1\0004\0\0\0"..., 512) = 512
>> fstat64(3, {st_mode=S_IFREG|0755, st_size=1750448, ...}) = 0
>> mmap2(NULL, 1759980, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7579000
>> mmap2(0xb7721000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a8000) = 0xb7721000
>> mmap2(0xb7724000, 10988, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7724000
>> close(3) = 0
>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7578000
>> set_thread_area({entry_number:-1 -> 6, base_addr:0xb75786c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
>> mprotect(0xb7721000, 8192, PROT_READ) = 0
>> mprotect(0x8049000, 4096, PROT_READ) = 0
>> mprotect(0xb7774000, 4096, PROT_READ) = 0
>> munmap(0xb7727000, 164108) = 0
>> fstat64(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(136, 5), ...}) = 0
>> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb774f000
>> write(1, "1\n", 21
>> ) = 2
>
> Does this mean it didn't OOPS? And where's the write(1, "1\n", 2)
> coming from? Are you sure you straced the right thing?
>
> --Andy
>
Ick - sry ,
here is the trace of the syscall.c, but as you expected, no BUG in syslog if I use strace :
tfoerste@n22 ~/tmp $ gcc syscall.c && strace ./a.out
execve("./a.out", ["./a.out"], [/* 75 vars */]) = 0
brk(0) = 0x8d75000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb76ee000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=164108, ...}) = 0
mmap2(NULL, 164108, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb76c5000
close(3) = 0
open("/lib/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 \316\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1750448, ...}) = 0
mmap2(NULL, 1759980, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7517000
mmap2(0xb76bf000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a8000) = 0xb76bf000
mmap2(0xb76c2000, 10988, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76c2000
close(3) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7516000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb75166c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb76bf000, 8192, PROT_READ) = 0
mprotect(0x8049000, 4096, PROT_READ) = 0
mprotect(0xb7712000, 4096, PROT_READ) = 0
munmap(0xb76c5000, 164108) = 0
syscall_1000(0, 0x8048469, 0xb76c1000, 0x8048460, 0, 0) = -1 (errno 38)
exit_group(0) = ?
+++ exited with 0 +++
--
Toralf
Well, might be the mail:subject should be adapted, b/c the issue can be triggered in a 3.13.11 kernel too.
Unfortunately it does not appear within an UML guest, therefore an automated bisecting isn't possible I fear.
--
Toralf
Am 16.06.2014 22:41, schrieb Toralf Förster:
> Well, might be the mail:subject should be adapted, b/c the issue can be triggered in a 3.13.11 kernel too.
> Unfortunately it does not appear within an UML guest, therefore an automated bisecting isn't possible I fear.
You could try KVM. :)
Thanks,
//richard
[cc: hpa, x86 list]
On Mon, Jun 16, 2014 at 1:43 PM, Richard Weinberger <[email protected]> wrote:
> Am 16.06.2014 22:41, schrieb Toralf Förster:
>> Well, might be the mail:subject should be adapted, b/c the issue can be triggered in a 3.13.11 kernel too.
>> Unfortunately it does not appear within an UML guest, therefore an automated bisecting isn't possible I fear.
>
> You could try KVM. :)
Before you do that, just to clarify:
What bitness is your kernel? That is, are you on 32-bit or 64-bit kernel?
What bitness is your test case? 'file a.out' will say.
What does /proc/cpuinfo say in flags?
Can you try the attached patch? It's only compile-tested.
To hpa, etc: It appears that entry_32.S is missing any call to the
audit exit hook on the badsys path. If I'm diagnosing this bug report
correctly, this causes OOPSes.
The the world at large: it's increasingly apparent that no one (except
maybe the blackhats) has ever scrutinized the syscall auditing code.
This is two old severe bugs in the code that have probably been there
for a long time.
--Andy
--
Andy Lutomirski
AMA Capital Management, LLC
On 06/16/2014 02:35 PM, Andy Lutomirski wrote:
>
> To hpa, etc: It appears that entry_32.S is missing any call to the
> audit exit hook on the badsys path. If I'm diagnosing this bug report
> correctly, this causes OOPSes.
>
> The the world at large: it's increasingly apparent that no one (except
> maybe the blackhats) has ever scrutinized the syscall auditing code.
> This is two old severe bugs in the code that have probably been there
> for a long time.
>
Yes, the audit code is a total mess.
> The bad syscall nr paths are their own incomprehensible route
> through the entry control flow. Rearrange them to work just like
> syscalls that return -ENOSYS.
I have to admit... it sort of lends itself to a solution like this:
/* For the 64-bit case, analogous code for 32 bits */
movl $__NR_syscall_max+1,%ecx # *Not* __NR_syscall_max
cmpq %rcx,%rax
cmovae %rcx,%rax
movq %r10,%rcx
call *sys_call_table(,%rax,8)
... and having an extra (invalid) system call slot in the syscall table
beyond the end instead of branching off separately.
(Note: we could use either cmova or cmovae, and either the 32- or 64-bit
form... the reason why is left as an exercise to the reader.)
-hpa
On Mon, Jun 16, 2014 at 2:48 PM, H. Peter Anvin <[email protected]> wrote:
> On 06/16/2014 02:35 PM, Andy Lutomirski wrote:
>>
>> To hpa, etc: It appears that entry_32.S is missing any call to the
>> audit exit hook on the badsys path. If I'm diagnosing this bug report
>> correctly, this causes OOPSes.
>>
>> The the world at large: it's increasingly apparent that no one (except
>> maybe the blackhats) has ever scrutinized the syscall auditing code.
>> This is two old severe bugs in the code that have probably been there
>> for a long time.
>>
>
> Yes, the audit code is a total mess.
>
>> The bad syscall nr paths are their own incomprehensible route
>> through the entry control flow. Rearrange them to work just like
>> syscalls that return -ENOSYS.
>
> I have to admit... it sort of lends itself to a solution like this:
>
> /* For the 64-bit case, analogous code for 32 bits */
> movl $__NR_syscall_max+1,%ecx # *Not* __NR_syscall_max
> cmpq %rcx,%rax
> cmovae %rcx,%rax
> movq %r10,%rcx
> call *sys_call_table(,%rax,8)
>
> ... and having an extra (invalid) system call slot in the syscall table
> beyond the end instead of branching off separately.
>
> (Note: we could use either cmova or cmovae, and either the 32- or 64-bit
> form... the reason why is left as an exercise to the reader.)
For 64-bit, I want to do this instead:
https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/seccomp-fastpath&id=a5ec2d7af2c54b55fc7201fa662138b53fbbda39
I see no reason why the 64-bit badsys code needs its own code path at
all. I haven't sent it yet because AFAICT it doesn't fix any bug, and
the series it's a part of isn't ready.
I'm also contemplating rewriting the 64-bit syscall entry work path in C.
--Andy
>
> For 64-bit, I want to do this instead:
>
> https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/seccomp-fastpath&id=a5ec2d7af2c54b55fc7201fa662138b53fbbda39
>
> I see no reason why the 64-bit badsys code needs its own code path at
> all. I haven't sent it yet because AFAICT it doesn't fix any bug, and
> the series it's a part of isn't ready.
>
> I'm also contemplating rewriting the 64-bit syscall entry work path in C.
>
Cute... although it still leaves an extra branch for the badsys. It
might perform better than a cmov, but it might not.
-hpa
On Mon, Jun 16, 2014 at 2:58 PM, H. Peter Anvin <[email protected]> wrote:
>>
>> For 64-bit, I want to do this instead:
>>
>> https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/seccomp-fastpath&id=a5ec2d7af2c54b55fc7201fa662138b53fbbda39
>>
>> I see no reason why the 64-bit badsys code needs its own code path at
>> all. I haven't sent it yet because AFAICT it doesn't fix any bug, and
>> the series it's a part of isn't ready.
>>
>> I'm also contemplating rewriting the 64-bit syscall entry work path in C.
>>
>
> Cute... although it still leaves an extra branch for the badsys. It
> might perform better than a cmov, but it might not.
I bet that the branch is faster in most cases -- I'd be somewhat
surprised if CPUs can speculate through a cmov, and that value is
needed for control flow almost immediately.
--Andy
On 06/16/2014 11:35 PM, Andy Lutomirski wrote:
> [cc: hpa, x86 list]
>
> On Mon, Jun 16, 2014 at 1:43 PM, Richard Weinberger <[email protected]> wrote:
>> Am 16.06.2014 22:41, schrieb Toralf Förster:
>>> Well, might be the mail:subject should be adapted, b/c the issue can be triggered in a 3.13.11 kernel too.
>>> Unfortunately it does not appear within an UML guest, therefore an automated bisecting isn't possible I fear.
>>
>> You could try KVM. :)
>
> Before you do that, just to clarify:
>
> What bitness is your kernel? That is, are you on 32-bit or 64-bit kernel?
>
> What bitness is your test case? 'file a.out' will say.
>
> What does /proc/cpuinfo say in flags?
>
tfoerste@n22 ~/tmp $ uname -a
Linux n22 3.15.1 #4 SMP Tue Jun 17 17:22:22 CEST 2014 i686 Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz GenuineIntel GNU/Linux
tfoerste@n22 ~/tmp $ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.16, not stripped
tfoerste@n22 ~/tmp $ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 42
model name : Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz
stepping : 7
microcode : 0x29
cpu MHz : 800.000
cache size : 3072 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 2
apicid : 0
initial apicid : 0
fdiv_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips : 5183.33
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:
> Can you try the attached patch? It's only compile-tested.
>
applied both on top of 3.14.8 and 3.15.1 - issue solved
Thx
> To hpa, etc: It appears that entry_32.S is missing any call to the
> audit exit hook on the badsys path. If I'm diagnosing this bug report
> correctly, this causes OOPSes.
>
> The the world at large: it's increasingly apparent that no one (except
> maybe the blackhats) has ever scrutinized the syscall auditing code.
> This is two old severe bugs in the code that have probably been there
> for a long time.
and now there are 2 less than before - this is the good news, or ?
--
Toralf
On Tue, Jun 17, 2014 at 8:38 AM, Toralf Förster <[email protected]> wrote:
> On 06/16/2014 11:35 PM, Andy Lutomirski wrote:
>> [cc: hpa, x86 list]
>>
>> On Mon, Jun 16, 2014 at 1:43 PM, Richard Weinberger <[email protected]> wrote:
>>> Am 16.06.2014 22:41, schrieb Toralf Förster:
>>>> Well, might be the mail:subject should be adapted, b/c the issue can be triggered in a 3.13.11 kernel too.
>>>> Unfortunately it does not appear within an UML guest, therefore an automated bisecting isn't possible I fear.
>>>
>>> You could try KVM. :)
>>
>> Before you do that, just to clarify:
>>
>> What bitness is your kernel? That is, are you on 32-bit or 64-bit kernel?
>>
>> What bitness is your test case? 'file a.out' will say.
>>
>> What does /proc/cpuinfo say in flags?
>>
>
> tfoerste@n22 ~/tmp $ uname -a
> Linux n22 3.15.1 #4 SMP Tue Jun 17 17:22:22 CEST 2014 i686 Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz GenuineIntel GNU/Linux
So entry_32.S is in play.
> flags : ... sep ...
And sysenter is in use, barring weird boot-time options.
>
>
>> Can you try the attached patch? It's only compile-tested.
>>
> applied both on top of 3.14.8 and 3.15.1 - issue solved
So I guess I diagnosed it right. Yay for reading ugly code.
hpa, should I resend a real emailed patch or is my attached thing ok?
Presumably this, or something like it, should cc: stable and go in
3.16, too.
--Andy
On Mon, Jun 16, 2014 at 2:48 PM, H. Peter Anvin <[email protected]> wrote:
> On 06/16/2014 02:35 PM, Andy Lutomirski wrote:
>>
>> To hpa, etc: It appears that entry_32.S is missing any call to the
>> audit exit hook on the badsys path. If I'm diagnosing this bug report
>> correctly, this causes OOPSes.
>>
>> The the world at large: it's increasingly apparent that no one (except
>> maybe the blackhats) has ever scrutinized the syscall auditing code.
>> This is two old severe bugs in the code that have probably been there
>> for a long time.
>>
>
> Yes, the audit code is a total mess.
>
>> The bad syscall nr paths are their own incomprehensible route
>> through the entry control flow. Rearrange them to work just like
>> syscalls that return -ENOSYS.
>
> I have to admit... it sort of lends itself to a solution like this:
>
> /* For the 64-bit case, analogous code for 32 bits */
> movl $__NR_syscall_max+1,%ecx # *Not* __NR_syscall_max
> cmpq %rcx,%rax
> cmovae %rcx,%rax
> movq %r10,%rcx
> call *sys_call_table(,%rax,8)
>
> ... and having an extra (invalid) system call slot in the syscall table
> beyond the end instead of branching off separately.
>
> (Note: we could use either cmova or cmovae, and either the 32- or 64-bit
> form... the reason why is left as an exercise to the reader.)
This is CVE-2014-4508, and it's probably worth fixing.
Is my patch good? I can resent and cc stable if needed.
--Andy
On 06/20/2014 05:41 PM, Andy Lutomirski wrote:
> On Mon, Jun 16, 2014 at 2:48 PM, H. Peter Anvin <[email protected]> wrote:
>> On 06/16/2014 02:35 PM, Andy Lutomirski wrote:
>>>
>>> To hpa, etc: It appears that entry_32.S is missing any call to the
>>> audit exit hook on the badsys path. If I'm diagnosing this bug report
>>> correctly, this causes OOPSes.
>>>
>>> The the world at large: it's increasingly apparent that no one (except
>>> maybe the blackhats) has ever scrutinized the syscall auditing code.
>>> This is two old severe bugs in the code that have probably been there
>>> for a long time.
>>>
>>
>> Yes, the audit code is a total mess.
>>
>>> The bad syscall nr paths are their own incomprehensible route
>>> through the entry control flow. Rearrange them to work just like
>>> syscalls that return -ENOSYS.
>>
>> I have to admit... it sort of lends itself to a solution like this:
>>
>> /* For the 64-bit case, analogous code for 32 bits */
>> movl $__NR_syscall_max+1,%ecx # *Not* __NR_syscall_max
>> cmpq %rcx,%rax
>> cmovae %rcx,%rax
>> movq %r10,%rcx
>> call *sys_call_table(,%rax,8)
>>
>> ... and having an extra (invalid) system call slot in the syscall table
>> beyond the end instead of branching off separately.
>>
>> (Note: we could use either cmova or cmovae, and either the 32- or 64-bit
>> form... the reason why is left as an exercise to the reader.)
>
> This is CVE-2014-4508, and it's probably worth fixing.
>
> Is my patch good? I can resent and cc stable if needed.
>
> --Andy
>
I'm running my system since the time you sent it to me w/o any problems with that patch on top of 3.15.1 :
tfoerste@n22 ~/devel/linux $ cat ~/devel/priv/0001-x86_32-entry-Fix-badsys-paths.patch
>From 8b43bd2118d876cb3163e8f7d9cd8253da649335 Mon Sep 17 00:00:00 2001
Message-Id: <8b43bd2118d876cb3163e8f7d9cd8253da649335.1402954406.git.luto@amacapital.net>
From: Andy Lutomirski <[email protected]>
Date: Mon, 16 Jun 2014 14:28:19 -0700
Subject: [PATCH] x86_32,entry: Fix badsys paths
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
The bad syscall nr paths are their own incomprehensible route
through the entry control flow. Rearrange them to work just like
syscalls that return -ENOSYS.
This should fix an OOPS in the audit code when auditing is enabled
and bad syscall nrs are used.
Reported-by: Toralf Förster <[email protected]>
Signed-off-by: Andy Lutomirski <[email protected]>
---
--
Toralf
On Fri, Jun 20, 2014 at 11:41 AM, Andy Lutomirski <[email protected]> wrote:
> On Mon, Jun 16, 2014 at 2:48 PM, H. Peter Anvin <[email protected]> wrote:
>> On 06/16/2014 02:35 PM, Andy Lutomirski wrote:
>>>
>>> To hpa, etc: It appears that entry_32.S is missing any call to the
>>> audit exit hook on the badsys path. If I'm diagnosing this bug report
>>> correctly, this causes OOPSes.
>>>
>>> The the world at large: it's increasingly apparent that no one (except
>>> maybe the blackhats) has ever scrutinized the syscall auditing code.
>>> This is two old severe bugs in the code that have probably been there
>>> for a long time.
>>>
>>
>> Yes, the audit code is a total mess.
>>
>>> The bad syscall nr paths are their own incomprehensible route
>>> through the entry control flow. Rearrange them to work just like
>>> syscalls that return -ENOSYS.
>>
>> I have to admit... it sort of lends itself to a solution like this:
>>
>> /* For the 64-bit case, analogous code for 32 bits */
>> movl $__NR_syscall_max+1,%ecx # *Not* __NR_syscall_max
>> cmpq %rcx,%rax
>> cmovae %rcx,%rax
>> movq %r10,%rcx
>> call *sys_call_table(,%rax,8)
>>
>> ... and having an extra (invalid) system call slot in the syscall table
>> beyond the end instead of branching off separately.
>>
>> (Note: we could use either cmova or cmovae, and either the 32- or 64-bit
>> form... the reason why is left as an exercise to the reader.)
>
> This is CVE-2014-4508, and it's probably worth fixing.
>
> Is my patch good? I can resent and cc stable if needed.
I'm planning on picking this up for Fedora tomorrow unless someone
screams it's the wrong fix. Honestly though, it would be nice to get
an indication either way.
josh
The bad syscall nr paths are their own incomprehensible route
through the entry control flow. Rearrange them to work just like
syscalls that return -ENOSYS.
This fixes an OOPS in the audit code when fast-path auditing is
enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
This has probably been broken since Linux 2.6.27:
af0575bba0 i386 syscall audit fast-path
Cc: [email protected]
Cc: Roland McGrath <[email protected]>
Reported-by: Toralf Förster <[email protected]>
Signed-off-by: Andy Lutomirski <[email protected]>
---
I realize that the syscall audit fast path and badsys code, on 32-bit
x86 no less, is possibly one of the least fun things in the kernel to
review, but this is still a real security bug and should get fixed :(
So I'm cc-ing a bunch of people and maybe someone will review it.
arch/x86/kernel/entry_32.S | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index a2a4f46..f4258a5 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -431,9 +431,10 @@ sysenter_past_esp:
jnz sysenter_audit
sysenter_do_call:
cmpl $(NR_syscalls), %eax
- jae syscall_badsys
+ jae sysenter_badsys
call *sys_call_table(,%eax,4)
movl %eax,PT_EAX(%esp)
+sysenter_after_call:
LOCKDEP_SYS_EXIT
DISABLE_INTERRUPTS(CLBR_ANY)
TRACE_IRQS_OFF
@@ -688,7 +689,12 @@ END(syscall_fault)
syscall_badsys:
movl $-ENOSYS,PT_EAX(%esp)
- jmp resume_userspace
+ jmp syscall_exit
+END(syscall_badsys)
+
+sysenter_badsys:
+ movl $-ENOSYS,PT_EAX(%esp)
+ jmp sysenter_after_call
END(syscall_badsys)
CFI_ENDPROC
/*
--
1.9.3
Commit-ID: 554086d85e71f30abe46fc014fea31929a7c6a8a
Gitweb: http://git.kernel.org/tip/554086d85e71f30abe46fc014fea31929a7c6a8a
Author: Andy Lutomirski <[email protected]>
AuthorDate: Mon, 23 Jun 2014 14:22:15 -0700
Committer: H. Peter Anvin <[email protected]>
CommitDate: Mon, 23 Jun 2014 14:59:26 -0700
x86_32, entry: Do syscall exit work on badsys (CVE-2014-4508)
The bad syscall nr paths are their own incomprehensible route
through the entry control flow. Rearrange them to work just like
syscalls that return -ENOSYS.
This fixes an OOPS in the audit code when fast-path auditing is
enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
This has probably been broken since Linux 2.6.27:
af0575bba0 i386 syscall audit fast-path
Cc: [email protected]
Cc: Roland McGrath <[email protected]>
Reported-by: Toralf Förster <[email protected]>
Signed-off-by: Andy Lutomirski <[email protected]>
Link: http://lkml.kernel.org/r/e09c499eade6fc321266dd6b54da7beb28d6991c.1403558229.git.luto@amacapital.net
Signed-off-by: H. Peter Anvin <[email protected]>
---
arch/x86/kernel/entry_32.S | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
index f0da82b..dbaa23e 100644
--- a/arch/x86/kernel/entry_32.S
+++ b/arch/x86/kernel/entry_32.S
@@ -423,9 +423,10 @@ sysenter_past_esp:
jnz sysenter_audit
sysenter_do_call:
cmpl $(NR_syscalls), %eax
- jae syscall_badsys
+ jae sysenter_badsys
call *sys_call_table(,%eax,4)
movl %eax,PT_EAX(%esp)
+sysenter_after_call:
LOCKDEP_SYS_EXIT
DISABLE_INTERRUPTS(CLBR_ANY)
TRACE_IRQS_OFF
@@ -675,7 +676,12 @@ END(syscall_fault)
syscall_badsys:
movl $-ENOSYS,PT_EAX(%esp)
- jmp resume_userspace
+ jmp syscall_exit
+END(syscall_badsys)
+
+sysenter_badsys:
+ movl $-ENOSYS,PT_EAX(%esp)
+ jmp sysenter_after_call
END(syscall_badsys)
CFI_ENDPROC
On Mon, Jun 23, 2014 at 02:22:15PM -0700, Andy Lutomirski wrote:
> The bad syscall nr paths are their own incomprehensible route
> through the entry control flow. Rearrange them to work just like
> syscalls that return -ENOSYS.
>
> This fixes an OOPS in the audit code when fast-path auditing is
> enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
>
> This has probably been broken since Linux 2.6.27:
> af0575bba0 i386 syscall audit fast-path
>
> Cc: [email protected]
> Cc: Roland McGrath <[email protected]>
> Reported-by: Toralf Förster <[email protected]>
> Signed-off-by: Andy Lutomirski <[email protected]>
> ---
>
> I realize that the syscall audit fast path and badsys code, on 32-bit
> x86 no less, is possibly one of the least fun things in the kernel to
> review, but this is still a real security bug and should get fixed :(
>
> So I'm cc-ing a bunch of people and maybe someone will review it.
Well, AFAICS, you're rerouting execution so that the audit stuff gets
properly "unwound" before returning to userspace. Which makes sense to
me.
Would it really work in all possible cases and isn't it causing any
other problems?
No friggin' idea - it would need extensive hammering to confirm it is ok
IMHO.
HTH.
--
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
On Tue, Jun 24, 2014 at 3:51 AM, Borislav Petkov <[email protected]> wrote:
> On Mon, Jun 23, 2014 at 02:22:15PM -0700, Andy Lutomirski wrote:
>> The bad syscall nr paths are their own incomprehensible route
>> through the entry control flow. Rearrange them to work just like
>> syscalls that return -ENOSYS.
>>
>> This fixes an OOPS in the audit code when fast-path auditing is
>> enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
>>
>> This has probably been broken since Linux 2.6.27:
>> af0575bba0 i386 syscall audit fast-path
>>
>> Cc: [email protected]
>> Cc: Roland McGrath <[email protected]>
>> Reported-by: Toralf Förster <[email protected]>
>> Signed-off-by: Andy Lutomirski <[email protected]>
>> ---
>>
>> I realize that the syscall audit fast path and badsys code, on 32-bit
>> x86 no less, is possibly one of the least fun things in the kernel to
>> review, but this is still a real security bug and should get fixed :(
>>
>> So I'm cc-ing a bunch of people and maybe someone will review it.
>
> Well, AFAICS, you're rerouting execution so that the audit stuff gets
> properly "unwound" before returning to userspace. Which makes sense to
> me.
>
> Would it really work in all possible cases and isn't it causing any
> other problems?
>
> No friggin' idea - it would need extensive hammering to confirm it is ok
> IMHO.
>
> HTH.
It confirms my sense that no one knows how to test this stuff :-/
It's pretty clear that no one has ever extensively hammered it.
I wonder how much could be effectively rewritten in C. I'm thinking
of redoing most of the entry work in C, but that won't help here.
--Andy
On Tue, Jun 24, 2014 at 01:53:25PM -0700, Andy Lutomirski wrote:
> It confirms my sense that no one knows how to test this stuff :-/
> It's pretty clear that no one has ever extensively hammered it.
Someone might've known at some point but who knows who knew. :-)
> I wonder how much could be effectively rewritten in C. I'm thinking of
> redoing most of the entry work in C, but that won't help here.
Whatever you do, you'll have to do testing yourself, I'm afraid, here
too. And then, after the patch is long committed and forgotten, someone
would resurface complaining that it breaks some obscure use case.
Eeh, the sad life of a kernel developer. :-\
--
Regards/Gruss,
Boris.
Sent from a fat crate under my desk. Formatting is fine.
--
On Mon, Jun 23, 2014 at 02:22:15PM -0700, Andy Lutomirski wrote:
> The bad syscall nr paths are their own incomprehensible route
> through the entry control flow. Rearrange them to work just like
> syscalls that return -ENOSYS.
>
> This fixes an OOPS in the audit code when fast-path auditing is
> enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
>
> This has probably been broken since Linux 2.6.27:
> af0575bba0 i386 syscall audit fast-path
>
> Cc: [email protected]
> Cc: Roland McGrath <[email protected]>
> Reported-by: Toralf F?rster <[email protected]>
> Signed-off-by: Andy Lutomirski <[email protected]>
> ---
>
> I realize that the syscall audit fast path and badsys code, on 32-bit
> x86 no less, is possibly one of the least fun things in the kernel to
> review, but this is still a real security bug and should get fixed :(
>
> So I'm cc-ing a bunch of people and maybe someone will review it.
>
> arch/x86/kernel/entry_32.S | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
> index a2a4f46..f4258a5 100644
> --- a/arch/x86/kernel/entry_32.S
> +++ b/arch/x86/kernel/entry_32.S
> @@ -431,9 +431,10 @@ sysenter_past_esp:
> jnz sysenter_audit
> sysenter_do_call:
> cmpl $(NR_syscalls), %eax
> - jae syscall_badsys
> + jae sysenter_badsys
> call *sys_call_table(,%eax,4)
> movl %eax,PT_EAX(%esp)
> +sysenter_after_call:
> LOCKDEP_SYS_EXIT
> DISABLE_INTERRUPTS(CLBR_ANY)
> TRACE_IRQS_OFF
> @@ -688,7 +689,12 @@ END(syscall_fault)
>
> syscall_badsys:
> movl $-ENOSYS,PT_EAX(%esp)
> - jmp resume_userspace
> + jmp syscall_exit
We're workig on preparing Ksplice updates for the last Fedora20 released
kernel and stumbled on this patch.
Apologies in advance if this is obvious, but would you mind explaining why
the above change in 'syscall_badsys' is required? We're just concerned
about doing the absolute minimum changes here if we can.
> +END(syscall_badsys)
> +
> +sysenter_badsys:
> + movl $-ENOSYS,PT_EAX(%esp)
> + jmp sysenter_after_call
> END(syscall_badsys)
Although it probably doesn't hurt, you may want to change the above
'END(syscall_badsys)' to 'END(sysenter_badsys)'.
Thanks,
Quentin
On Tue, Jul 1, 2014 at 3:52 AM, Quentin Casasnovas
<[email protected]> wrote:
> On Mon, Jun 23, 2014 at 02:22:15PM -0700, Andy Lutomirski wrote:
>> The bad syscall nr paths are their own incomprehensible route
>> through the entry control flow. Rearrange them to work just like
>> syscalls that return -ENOSYS.
>>
>> This fixes an OOPS in the audit code when fast-path auditing is
>> enabled and sysenter gets a bad syscall nr (CVE-2014-4508).
>>
>> This has probably been broken since Linux 2.6.27:
>> af0575bba0 i386 syscall audit fast-path
>>
>> Cc: [email protected]
>> Cc: Roland McGrath <[email protected]>
>> Reported-by: Toralf Förster <[email protected]>
>> Signed-off-by: Andy Lutomirski <[email protected]>
>> ---
>>
>> I realize that the syscall audit fast path and badsys code, on 32-bit
>> x86 no less, is possibly one of the least fun things in the kernel to
>> review, but this is still a real security bug and should get fixed :(
>>
>> So I'm cc-ing a bunch of people and maybe someone will review it.
>>
>> arch/x86/kernel/entry_32.S | 10 ++++++++--
>> 1 file changed, 8 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/kernel/entry_32.S b/arch/x86/kernel/entry_32.S
>> index a2a4f46..f4258a5 100644
>> --- a/arch/x86/kernel/entry_32.S
>> +++ b/arch/x86/kernel/entry_32.S
>> @@ -431,9 +431,10 @@ sysenter_past_esp:
>> jnz sysenter_audit
>> sysenter_do_call:
>> cmpl $(NR_syscalls), %eax
>> - jae syscall_badsys
>> + jae sysenter_badsys
>> call *sys_call_table(,%eax,4)
>> movl %eax,PT_EAX(%esp)
>> +sysenter_after_call:
>> LOCKDEP_SYS_EXIT
>> DISABLE_INTERRUPTS(CLBR_ANY)
>> TRACE_IRQS_OFF
>> @@ -688,7 +689,12 @@ END(syscall_fault)
>>
>> syscall_badsys:
>> movl $-ENOSYS,PT_EAX(%esp)
>> - jmp resume_userspace
>> + jmp syscall_exit
>
> We're workig on preparing Ksplice updates for the last Fedora20 released
> kernel and stumbled on this patch.
>
> Apologies in advance if this is obvious, but would you mind explaining why
> the above change in 'syscall_badsys' is required? We're just concerned
> about doing the absolute minimum changes here if we can.
The audit code expects entry and exit hooks to match up correctly.
Without this fix, a bad syscall nr would call the entry hooks and then
return to userspace without calling the exit hooks, causing an oops on
the next syscall entry.
This is trivial to reproduce: just do 'auditctl -e 1' and then do syscall(1000).
>
>> +END(syscall_badsys)
>> +
>> +sysenter_badsys:
>> + movl $-ENOSYS,PT_EAX(%esp)
>> + jmp sysenter_after_call
>> END(syscall_badsys)
>
> Although it probably doesn't hurt, you may want to change the above
> 'END(syscall_badsys)' to 'END(sysenter_badsys)'.
That's embarrassing. Thanks.
--Andy
>
> Thanks,
> Quentin
--
Andy Lutomirski
AMA Capital Management, LLC