Date: Fri, 18 Mar 2016 19:46:27 +0300
From: Yury Norov <ynorov@caviumnetworks.com>
To: Alexander Graf <agraf@suse.de>
CC: "Zhangjian (Bamvor)" <bamvor.zhangjian@huawei.com>,
        Andreas Schwab <schwab@suse.de>, <arnd@arndb.de>,
        <catalin.marinas@arm.com>, <linux-arm-kernel@lists.infradead.org>,
        <linux-kernel@vger.kernel.org>, <pinskia@gmail.com>,
        <Prasun.Kapoor@caviumnetworks.com>, <broonie@kernel.org>,
        <heiko.carstens@de.ibm.com>, <klimov.linux@gmail.com>,
        <jan.dakinevich@gmail.com>, <schwidefsky@de.ibm.com>,
        <Nathan_Lynch@mentor.com>, <joseph@codesourcery.com>,
        <christoph.muellner@theobroma-systems.com>,
        Bamvor Zhang Jian <bamvor.zhangjian@linaro.org>
Subject: Re: [RFC5 PATCH v6 00/21] ILP32 for ARM64
Message-ID: <20160318164627.GA3201@yury-N73SV>
References: <1452792198-10718-1-git-send-email-ynorov@caviumnetworks.com>
 <56AB3805.1040308@huawei.com>
 <20160129170929.GA3543@yury-N73SV>
 <56AC38F1.2030608@huawei.com>
 <20160218223506.GA7816@yury-N73SV>
 <mvm8u29ukqw.fsf@hawking.suse.de>
 <20160225202855.GD16123@yury-N73SV>
 <56EBD84D.2060009@huawei.com>
 <20160318154918.GA1595@yury-N73SV>
 <56EC24EE.6020803@suse.de>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="/04w6evG8XlLl3ft"
Content-Disposition: inline
In-Reply-To: <56EC24EE.6020803@suse.de>
User-Agent: Mutt/1.5.23 (2014-03-12)
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Mar 2016 16:46:44.4280 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR07MB616
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 9177
Lines: 195

--/04w6evG8XlLl3ft
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline

On Fri, Mar 18, 2016 at 04:55:26PM +0100, Alexander Graf wrote:
> 
> 
> On 18.03.16 16:49, Yury Norov wrote:
> > On Fri, Mar 18, 2016 at 06:28:29PM +0800, Zhangjian (Bamvor) wrote:
> >>
> >> For the glibc part, I found that there are 11 patches of ilp32 in top,
> >> but the original 28 patches of ilp32 is not in the top, there are more
> >> than 900 patches between them(referece the list below). Are you
> >> willing rebase all the ilp32 relative patches. It is very useful for
> >> reviewing and debugging.  I saw andrew request the account in glibc,
> >> maybe it has already been in processs?).
> >>
> > 
> > I already told there's mess there, and I'd prefer to make things work
> > first and then do cleanup. 
> 
> So how is progress going overall? The last submission I've seen is
> already 2 months ago. Are there particular bits holding you up?
> 
> 
> Alex

Hi Alexander, 

For last time I mostly work on library, as it needs to be reworked
well. But yes, there's one serious bug puzzling me.

Tests like umount or pathconf fail but I see no major problem with
it, as it's most probably structure padding mismatch between kernel and
glibc. But there's (at least) one major problem I see.

Float tests fail due to NULL-dereferencing (0x14 actually) at
pthread_join(). It calls tgkill(), and after that child thread crashes.
See stack trace at the end.

The minimal test reproducing it is attached. The similar test where
parent forks a child and then kills it, works fine. (Attached too).

I see that in case of pthread, there's much more stuff that is cloned.
Other's looking similar.

pthread_create():
clone(child_stack=0xb953cea0, flags=CLONE_VM|CLONE_FS|CLONE_FILES
        |CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS
        |CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
        parent_tidptr=0xb953d398, tls=0xb953d7c0, child_tidptr=0xb953d398) = 1650

fork():
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
        child_tidptr=0xe5af6278) = 30537

So this most probably means that ilp32 code doesn't handle one of cloned
item properly. I have already discovered a bug where child processes
used parent TLS, so maybe this is something similar...

Except of this, I think ILP32 series is looking pretty well, at least
kernel part.

If you have any ideas/suggestions, I'll really appreciate it.

Yury.

strace -f ./trigo
[...]
clone(child_stack=0xdbbfb000,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND
|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS
|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0xdbbfb4f8, tls=0xdbbfb920, child_tidptr=0xdbbfb4f8) = 32030
rt_sigprocmask(SIG_BLOCK, [CHLD], Process 32030 attached [], 8) = 0
[pid 32029] rt_sigaction(SIGCHLD, NULL,  <unfinished ...>
[pid 32030] set_robust_list(0xdbbfb504, 12 <unfinished ...>
[pid 32029] <... rt_sigaction resumed> {SIG_DFL, [ILL ABRT SEGV URG], 0}, 8) = 0
[pid 32030] <... set_robust_list resumed> ) = 0
[pid 32029] rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 32030] write(1, "started\n", 8started
<unfinished ...>
[pid 32029] nanosleep({1, 65536},  <unfinished ...>
[pid 32030] <... write resumed> )       = 8
[pid 32030] rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
[pid 32030] rt_sigsuspend([] <unfinished ...>
[pid 32029] <... nanosleep resumed> 0xfff9fd98) = 0
[pid 32029] write(1, "stoping...\n", 11stoping...) = 11
[pid 32029] openat(AT_FDCWD, "/root/sys-root/libilp32/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
[pid 32029] read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0 \0\0004\0\0\0"..., 512) = 512
[pid 32029] fstat(3, {st_mode=S_IFREG|0644, st_size=429138, ...}) = 0
[pid 32029] mmap(NULL, 135104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xdb3db000
[pid 32029] mprotect(0xdb3ec000, 61440, PROT_NONE) = 0
[pid 32029] mmap(0xdb3fb000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0xdb3fb000
[pid 32029] close(3)                    = 0
[pid 32029] tgkill(32029, 32030, SIGRTMIN) = 0
[pid 32030] <... rt_sigsuspend resumed> ) = ? ERESTARTNOHAND (To be
restarted if no handler)
[pid 32029] write(1, "pthread_cancel == 0\n", 20pthread_cancel == 0) = 20
[pid 32030] --- SIGRTMIN {si_signo=SIGRTMIN, si_code=SI_TKILL, si_pid=32029, si_uid=0} ---
[pid 32029] write(1, "stopped\n", 8stopped
<unfinished ...>
[pid 32030] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x14} ---
[pid 32029] <... write resumed> )       = ? <unavailable>
[pid 32030] +++ killed by SIGSEGV +++
+++ killed by SIGSEGV +++
Segmentation fault

dmesg:
trigo[32246]: unhandled level 2 translation fault (11) at 0x00000014,
esr 0x90000006
pgd = ffffffc009335000
[00000014] *pgd=000000007917c003, *pud=000000007917c003,
*pmd=0000000000000000

CPU: 2 PID: 32246 Comm: trigo Not tainted 4.5.0+ #91
Hardware name: linux,dummy-virt (DT)
task: ffffffc00900e400 ti: ffffffc009078000 task.ti: ffffffc009078000
PC is at 0xda6853f0
LR is at 0xda6d5440
pc : [<00000000da6853f0>] lr : [<00000000da6d5440>] pstate: 60000000
sp : 00000000da511bc0
x29: 00000000da512e10 x28: 00000000da6a7000 
x27: 0000000000000000 x26: 00000000da513490 
x25: 0000000000000000 x24: 0000000000400820 
x23: 00000000da6a9000 x22: 00000000ff869acb 
x21: 00000000da6a9000 x20: 00000000da512e50 
x19: 0000000000000000 x18: 0000000000000001 
x17: 0000000000410bd8 x16: 00000000da691138 
x15: 0000000000000000 x14: 0000000000000000 
x13: 00000000da535970 x12: 0000000000000038 
x11: 0000000000000028 x10: 0101010101010101 
x9 : ff63647371607372 x8 : 0000000000000085 
x7 : 0000000000007df5 x6 : 00000000da512e1c 
x5 : 00000000da513518 x4 : 0000000000000002 
x3 : 00000000da513920 x2 : 0000000000000000 
x1 : 0000000000000008 x0 : 00000000da513490 


--/04w6evG8XlLl3ft
Content-Type: application/gzip
Content-Disposition: attachment; filename="mykill.tar.gz"
Content-Transfer-Encoding: base64

H4sIAHot7FYAA+1WW2/bNhTOa/grWC8DpCySbMWWkaRulzqXenDiInbQDLsECkXZXHQbJbUx
iv73HVKSK2nIuj1k7TB+DxJ5eC4fzyF1FK7vWRBYW0+JLmA4HIh3bzjo1t8Vtnq2M9zv2k7f
GW51ez0bRHjwpKxK5Gnmcoy31jlf/5Xe59b/owiL+l+499RnAX2SGKLAjtN/rP72/tCW9e86
9j4oQv1tZ3+4hbtPwqaF/3n9F8dX56cLPMLFQUDF3OlvJE4focVsNh2/Pp5c4pcjbK3ikFoi
Hdb7mN9bWRwHZOWyyMpWeeRR/mAIUWrdgch1OVk5fWOzFLAofzCWUW6g8RiC7Ggb5/qSEDQ9
kcLxWEezVz+cXF+8aSnFd795eZggND6bHp/PYdVYYmPWxcZbNwiwYaSZNwL/BwdgdvxqIrX0
Ulvu6zF9hJhPf8daZ0eLOVuyCKd6Zw93SByGbuRhoE47OsJ4vjheTAR5YexmjCAaecwH7gWl
72SMevT6SpKtOHW9huxtsGcYPHGz1cjicZxZ6To15CBgdyxI9u0/q3vryA0ZERm9p/wxOyvw
DNu0e+ZB10zjupcdrdiHjiopZEdSb81bhAtpzbrapkhI6N6xUUEYjedXIkkhHA2TIFHOMi7s
M83v0gx/a5KHhz14xXui5jc3YKHrLY1P63JV+qk4/GNPTr/hC8EROBTHS556fTNy+rDGIqiu
WIcH2k5JUlN0eXjYlIFJIUVBmtZ8mjCtK4k5QpLIoSCFtuVpF5zK9OrYIHjnOTZivPM9qD6i
2FJrBKzFF2blTQILDxtzYfSiYVPSOmzs/2/Y6cJC1ENqT0+A3K9YDCqCcTtMZVAGaJvIzRdG
b17PLn88xCSgboTkExLOQ2z4jZS3M9tS0ZuFqJZ3zRh96S/v14Gy/xeX9IlifKb/Q7fvVf1/
YHd7ov/bg77q//8GvmERCXKP4ufQB1lsrl6ghgj6SEvGlpEbNGWU87YhZ9GyKSvbSFOYRwyC
CFndep1a2Tqh6WO6n2Q+iTJJBhWdGL+LmYd3URHr1s8johUily919AFtJ8As87WOKHtGvZ+j
jn4EUjdPqSZH5XoeSVmpAB8OmuU8wpfX0+kR+ii6Q4bErdFgAB/+JXQX+AviMtC7n36RscQS
ET6Zd5theML4bDI9xbviXxsm4gUt0o8TGmkdK+ExsVIa+HAfk1T8e3DJjvlYeyZ04aNZETF6
gtX7lfCgaQS8LGlGNKml42cjfDo7E+pJngleGpG78EkQw0allphXu/XhT840zWqzQFXS4vfa
Jj7IhL96ZkUypH4aUJpovYbLNIsTcQgqr9viQ6OBlz08n5xfLS7gZ+6oqU69Godyn6c3k8Xt
/Ho8Pp3PReK/9IVRUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFD4yvEHpegEvwAoAAA=

--/04w6evG8XlLl3ft
Content-Type: application/gzip
Content-Disposition: attachment; filename="trigo.tar.gz"
Content-Transfer-Encoding: base64

H4sIAGst7FYAA+1WbXPaRhDmq+9XXKjTkVyEEGBR2yEtBhyTEvAYPHGmLx75dALFQqdKp8RM
p/+9eyewJWXctB9o2uk9H6S7vWdfbnd1Jx77C2ZWdooGoNM5FG+rc9jIv7eoWE2702o07Xa7
U2lYVqNpVfDhbsPKkCbciTGurNN4/We8z63/R8Fl/d84d9TzA7obH6LAtt1+qv7NVierf8Oy
O5bdhPo323ajghu7CaeI/3n9573LV8M57mLZCCib2u2twG4jNJ9Ox/3z3miCv+tic8lW1BTJ
MD+y+M7kjAVk6fihyZdp6NL43hCixLwFkePEZGm3jYelwA/Te2MRpgbq98HHvvZgXF8QgsYD
Kez3dTQ9fT24enNRIrHb9266ihDqn417r2awaiywMW1g460TBNgwEu52wf7REaj1TkeSpW/Y
cltP8RHyPfor1qr7GoOd+yFO9GoNVwlbrZzQxRA6reoI49m8Nx+J4IWyw32CaOj6HsSehfSN
9JH3nl+J+DKmjluQvQ1qhhFHDl92zZgxbibrxJCDwL/1g6jV/JTurkNn5ROR0TsaP6VnBq7R
rDet+lGjnrC8lX0t24eOtlLIjgy9NC8FnElz2tttioSsnFu/mwWM+rNLkaQVtEadIFHOjV/Y
Z5LeJhw/r5P7+xq8WE3U/PoaNHS9xHhcl6vSzjaGv23JbhdsIWiBY9Fesun1h5HdhjU/hOqK
dXigvYREOaITr46LMlDJpChIkpzNOkzzJDFHSAZyLIJCe7LbRUyb9OrYIHj/BTYY3v8eqE8Q
S7SCw5x/obb5kkDDxcZMKL0s6GzCOi7s/y/o6UJD1EOyxwMI7hcsBtsAWdnNVmHjoKwiN58p
XZxPJ++OMQmoEyL5hITHK2x4hZSXM1ui6MVCbJcP6gx96XP334Ls/s++0l35+Mz9b3U6rez+
b9lt27bF/Q9Tdf//E/jKD0mQuhS/gJvQZ/XlS1QQwU1SkvmL0AmKMhrHZcXYDxdF2eYiKQrT
0AcnQpbXXicmX0c0eYr7KPNIyGUwKLuL8Qfmu/gAZb5uvDQkWiZy4oWOfkN7EUTGPa0qys6p
+1NY1U9A6qQJ1eRos56GUrYhwNFBeRqHeHI1Hp+g38X9wJH4bDQYwNG/gPsF/oNi6ejDjz9n
vjZhcJwNTh5FDucxyAm0H1vJWXnRD32ufZ0j6GVGQrlLuUOWYu+0wK3hi/n55bA3uOnDcz68
eT0dTXqn46HYC9oTMRMwdzYaD/GB+POHiXjBje2xiIZa1YxiRsyEBh6cDlEifoVimSrfw9oz
wYUzfJsVwxIp+rgUFjSNgJUF5USTLB0/6+Lh9EzQo5SLJGlEptQjAYOsSxbMTVPG5bmgnsVQ
B6/Tm8vBdDJ+BwwgMM+DrMETOEFC6Z3muTXcuLearfah3fm2hmfD4Q83w8kg44tgga2LcTHY
Qh+IZq3Xt72wyTGBp0hrNq3J0tdwrrMykdxKElAaaZZetMyigmGRlwfjTkhooGWzfN8VCbjb
xc9FD0J3fWI8emzfjdJ7Bg2ZjzffusPr0fxmdtXvD2cz0cJf+uhRUFBQUFBQUFBQUFBQUFBQ
UFBQUFBQUNgR/gB/KeN/ACgAAA==

--/04w6evG8XlLl3ft--