Hi,
did anyone already happen to look into enabling CPU hotplug for the
Allwinner A20 in upstream? I'm currently running the sunxi-next branch
on Banana Pi, and echo 0 > .../cpu1/online just hangs the system. The
old 3.4 LeMaker kernel works fine in this regard. I can try to look into
details and port things over, just want to avoid duplicate efforts.
Thanks,
Jan
Hi Jan,
On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka wrote:
> did anyone already happen to look into enabling CPU hotplug for the
> Allwinner A20 in upstream? I'm currently running the sunxi-next branch
> on Banana Pi, and echo 0 > .../cpu1/online just hangs the system. The
> old 3.4 LeMaker kernel works fine in this regard. I can try to look into
> details and port things over, just want to avoid duplicate efforts.
Having hotplug support would indeed be very welcome.
However, it should be done in u-boot, through PSCI, and not in the
kernel itself.
As far as I'm aware, no one worked actively on it, beside some WIP
commit from Marc a while ago:
https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
Maxime
--
Maxime Ripard, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
On 2014-11-10 00:17, Maxime Ripard wrote:
> Hi Jan,
>
> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka wrote:
>> did anyone already happen to look into enabling CPU hotplug for the
>> Allwinner A20 in upstream? I'm currently running the sunxi-next branch
>> on Banana Pi, and echo 0 > .../cpu1/online just hangs the system. The
>> old 3.4 LeMaker kernel works fine in this regard. I can try to look into
>> details and port things over, just want to avoid duplicate efforts.
>
> Having hotplug support would indeed be very welcome.
>
> However, it should be done in u-boot, through PSCI, and not in the
> kernel itself.
>
> As far as I'm aware, no one worked actively on it, beside some WIP
> commit from Marc a while ago:
> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
OK - I guess I will need a little guidance in then: Is there a good
reference board to study and to derive from? And maybe also: What is
missing or not working in that u-boot branch? If I get this interface
right, I just takes some device tree bits to enable this for the kernel
afterward, correct?
Thanks,
Jan
PS: That the GMAC on B-Pi has some TX reliability issues, both with
kernel and u-boot, is known, right? If not, I'll open a separate thread.
On 2014-11-10 07:03, Jan Kiszka wrote:
> On 2014-11-10 00:17, Maxime Ripard wrote:
>> Hi Jan,
>>
>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka wrote:
>>> did anyone already happen to look into enabling CPU hotplug for the
>>> Allwinner A20 in upstream? I'm currently running the sunxi-next branch
>>> on Banana Pi, and echo 0 > .../cpu1/online just hangs the system. The
>>> old 3.4 LeMaker kernel works fine in this regard. I can try to look into
>>> details and port things over, just want to avoid duplicate efforts.
>>
>> Having hotplug support would indeed be very welcome.
>>
>> However, it should be done in u-boot, through PSCI, and not in the
>> kernel itself.
>>
>> As far as I'm aware, no one worked actively on it, beside some WIP
>> commit from Marc a while ago:
>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>
> OK - I guess I will need a little guidance in then: Is there a good
> reference board to study and to derive from? And maybe also: What is
> missing or not working in that u-boot branch? If I get this interface
> right, I just takes some device tree bits to enable this for the kernel
> afterward, correct?
Started to play with that patch in naive ways: CPU0 locks up when
offlining CPU1 - unless I disable the FIQ signal from CPU1. Then it
"works", both offlining and onlining again. However, I suspect that this
only parks CPU1 in wfi and does not do anything interesting to it.
Jan
On 10/11/14 08:25, Jan Kiszka wrote:
> On 2014-11-10 07:03, Jan Kiszka wrote:
>> On 2014-11-10 00:17, Maxime Ripard wrote:
>>> Hi Jan,
>>>
>>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka wrote:
>>>> did anyone already happen to look into enabling CPU hotplug
>>>> for the Allwinner A20 in upstream? I'm currently running the
>>>> sunxi-next branch on Banana Pi, and echo 0 > .../cpu1/online
>>>> just hangs the system. The old 3.4 LeMaker kernel works fine
>>>> in this regard. I can try to look into details and port
>>>> things over, just want to avoid duplicate efforts.
>>>
>>> Having hotplug support would indeed be very welcome.
>>>
>>> However, it should be done in u-boot, through PSCI, and not in
>>> the kernel itself.
>>>
>>> As far as I'm aware, no one worked actively on it, beside some
>>> WIP commit from Marc a while ago:
>>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>>
>>
>>>
OK - I guess I will need a little guidance in then: Is there a good
>> reference board to study and to derive from? And maybe also: What
>> is missing or not working in that u-boot branch? If I get this
>> interface right, I just takes some device tree bits to enable
>> this for the kernel afterward, correct?
>
> Started to play with that patch in naive ways: CPU0 locks up when
> offlining CPU1 - unless I disable the FIQ signal from CPU1. Then
> it "works", both offlining and onlining again. However, I suspect
> that this only parks CPU1 in wfi and does not do anything
> interesting to it.
Here's how this is supposed to work:
- CPU1 sends a FIQ to CPU0, bringing it into secure mode.
- CPU0 then kills CPU1 by doing the magic incantations on the power
controller
What is missing here is all the cache cleaning before signalling CPU0.
If you add that, things should look a lot better (patches welcome).
Thanks,
M.
--
Jazz is not dead. It just smells funny...
On 2014-11-10 10:17, Marc Zyngier wrote:
> On 10/11/14 08:25, Jan Kiszka wrote:
>> On 2014-11-10 07:03, Jan Kiszka wrote:
>>> On 2014-11-10 00:17, Maxime Ripard wrote:
>>>> Hi Jan,
>>>>
>>>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka wrote:
>>>>> did anyone already happen to look into enabling CPU hotplug
>>>>> for the Allwinner A20 in upstream? I'm currently running the
>>>>> sunxi-next branch on Banana Pi, and echo 0 > .../cpu1/online
>>>>> just hangs the system. The old 3.4 LeMaker kernel works fine
>>>>> in this regard. I can try to look into details and port
>>>>> things over, just want to avoid duplicate efforts.
>>>>
>>>> Having hotplug support would indeed be very welcome.
>>>>
>>>> However, it should be done in u-boot, through PSCI, and not in
>>>> the kernel itself.
>>>>
>>>> As far as I'm aware, no one worked actively on it, beside some
>>>> WIP commit from Marc a while ago:
>>>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>>>
>>>
>>>>
> OK - I guess I will need a little guidance in then: Is there a good
>>> reference board to study and to derive from? And maybe also: What
>>> is missing or not working in that u-boot branch? If I get this
>>> interface right, I just takes some device tree bits to enable
>>> this for the kernel afterward, correct?
>>
>> Started to play with that patch in naive ways: CPU0 locks up when
>> offlining CPU1 - unless I disable the FIQ signal from CPU1. Then
>> it "works", both offlining and onlining again. However, I suspect
>> that this only parks CPU1 in wfi and does not do anything
>> interesting to it.
>
> Here's how this is supposed to work:
> - CPU1 sends a FIQ to CPU0, bringing it into secure mode.
> - CPU0 then kills CPU1 by doing the magic incantations on the power
> controller
>
> What is missing here is all the cache cleaning before signalling CPU0.
> If you add that, things should look a lot better (patches welcome).
Unsure about this, or maybe this was too simplistic: I added calls to
u-boot's flush_dcache_all and invalidate_icache_all (right after
disabling the cache, just like the vendor kernel does), but CPU0 still
locks up. I suspect there is still a bug in the FIQ handling. There is
also a suspicious single "@" printed on the console. I'll play with the
FIQ handler a bit.
Jan
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 10/11/14 09:36, Jan Kiszka wrote:
> On 2014-11-10 10:17, Marc Zyngier wrote:
>> On 10/11/14 08:25, Jan Kiszka wrote:
>>> On 2014-11-10 07:03, Jan Kiszka wrote:
>>>> On 2014-11-10 00:17, Maxime Ripard wrote:
>>>>> Hi Jan,
>>>>>
>>>>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka
>>>>> wrote:
>>>>>> did anyone already happen to look into enabling CPU
>>>>>> hotplug for the Allwinner A20 in upstream? I'm currently
>>>>>> running the sunxi-next branch on Banana Pi, and echo 0 >
>>>>>> .../cpu1/online just hangs the system. The old 3.4
>>>>>> LeMaker kernel works fine in this regard. I can try to
>>>>>> look into details and port things over, just want to
>>>>>> avoid duplicate efforts.
>>>>>
>>>>> Having hotplug support would indeed be very welcome.
>>>>>
>>>>> However, it should be done in u-boot, through PSCI, and not
>>>>> in the kernel itself.
>>>>>
>>>>> As far as I'm aware, no one worked actively on it, beside
>>>>> some WIP commit from Marc a while ago:
>>>>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>>>>
>>>>
>>>>>
>>
>>>>>
OK - I guess I will need a little guidance in then: Is there a good
>>>> reference board to study and to derive from? And maybe also:
>>>> What is missing or not working in that u-boot branch? If I
>>>> get this interface right, I just takes some device tree bits
>>>> to enable this for the kernel afterward, correct?
>>>
>>> Started to play with that patch in naive ways: CPU0 locks up
>>> when offlining CPU1 - unless I disable the FIQ signal from
>>> CPU1. Then it "works", both offlining and onlining again.
>>> However, I suspect that this only parks CPU1 in wfi and does
>>> not do anything interesting to it.
>>
>> Here's how this is supposed to work: - CPU1 sends a FIQ to CPU0,
>> bringing it into secure mode. - CPU0 then kills CPU1 by doing the
>> magic incantations on the power controller
>>
>> What is missing here is all the cache cleaning before signalling
>> CPU0. If you add that, things should look a lot better (patches
>> welcome).
>
> Unsure about this, or maybe this was too simplistic: I added calls
> to u-boot's flush_dcache_all and invalidate_icache_all (right
> after disabling the cache, just like the vendor kernel does), but
> CPU0 still locks up. I suspect there is still a bug in the FIQ
> handling. There is also a suspicious single "@" printed on the
> console. I'll play with the FIQ handler a bit.
The '@' is just my own debug stuff, and might be causing issues too.
Now, you have to realise that by the time you call into this code,
u-boot itself is long gone. Only the tiny bit of code dealing with
PSCI still lives in a bank of static, secure memory. So calling into
u-boot for anything is doomed. You need to actually put the code
inside the PSCI backend.
Thanks,
M.
- --
Jazz is not dead. It just smells funny...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iQIcBAEBAgAGBQJUYIrRAAoJECPQ0LrRPXpDyjQQAMpDGLUwVODYi0lHFb20axqZ
Jjptkh6L2UJ2D2a2Ow17umrnIJqFjvVDasKtoxrBj79ya7LvHdfti4cFAjhZbh53
dOYmGqWHLP5N2Ue5KGs6zEpmW3bkUQRM2n03pTsqs907zN3XrjhfdtEqun5qAKTX
Kbk2X/CieQnGx9rYeZUAIiJm0qO4e9ijKFQCvxRDItr0Eb4voRwRCtWCiD530ZTc
34G7fpqTR4HKNRsaLeBRCUz3dJXP7KgW3VymIu35fAAyUlt0oMENrcX9qKb6AHR7
KM0JXkts+bMmxeddRPzEEikHvRfSgbFxTY0+6vsqi4DrsuihI5OG+6VqckZRrfpV
S74IkyEyBscSUhkRcXUEWsly9AcK7gvxlK0DqyR+5v+OOQd6RGfAfA8xYT6GbtpN
EZ8suyFWsl55QdolWx5Pqb8VfL0Z70AteWcMhJia6eg69hGo1YJVM6DJ93rvSwzx
x1UniBVBkp9/jZYJPgrCw0ywGXbWex7OcnTCscGAChkFMgFT/LiBCF16PKoyLhYu
oGoClNbwlLgo4xAoeB/pDLyO0eaOLQS7/w2LdLZGbPXMVqcXkYVS+vkdkfe099Hb
dYXOCkI8mcxDDisodtSg7W797bJMfjnCmfrXJuyfTbJW2nSrYgqw5yZNhhEhlWB3
n24nemun99pZ8di2HBSl
=C/8W
-----END PGP SIGNATURE-----
On 2014-11-10 10:52, Marc Zyngier wrote:
> On 10/11/14 09:36, Jan Kiszka wrote:
>> On 2014-11-10 10:17, Marc Zyngier wrote:
>>> On 10/11/14 08:25, Jan Kiszka wrote:
>>>> On 2014-11-10 07:03, Jan Kiszka wrote:
>>>>> On 2014-11-10 00:17, Maxime Ripard wrote:
>>>>>> Hi Jan,
>>>>>>
>>>>>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka
>>>>>> wrote:
>>>>>>> did anyone already happen to look into enabling CPU
>>>>>>> hotplug for the Allwinner A20 in upstream? I'm currently
>>>>>>> running the sunxi-next branch on Banana Pi, and echo 0 >
>>>>>>> .../cpu1/online just hangs the system. The old 3.4
>>>>>>> LeMaker kernel works fine in this regard. I can try to
>>>>>>> look into details and port things over, just want to
>>>>>>> avoid duplicate efforts.
>>>>>>
>>>>>> Having hotplug support would indeed be very welcome.
>>>>>>
>>>>>> However, it should be done in u-boot, through PSCI, and not
>>>>>> in the kernel itself.
>>>>>>
>>>>>> As far as I'm aware, no one worked actively on it, beside
>>>>>> some WIP commit from Marc a while ago:
>>>>>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>>>>>
>>>>>
>>>>>>
>>>
>>>>>>
> OK - I guess I will need a little guidance in then: Is there a good
>>>>> reference board to study and to derive from? And maybe also:
>>>>> What is missing or not working in that u-boot branch? If I
>>>>> get this interface right, I just takes some device tree bits
>>>>> to enable this for the kernel afterward, correct?
>>>>
>>>> Started to play with that patch in naive ways: CPU0 locks up
>>>> when offlining CPU1 - unless I disable the FIQ signal from
>>>> CPU1. Then it "works", both offlining and onlining again.
>>>> However, I suspect that this only parks CPU1 in wfi and does
>>>> not do anything interesting to it.
>>>
>>> Here's how this is supposed to work: - CPU1 sends a FIQ to CPU0,
>>> bringing it into secure mode. - CPU0 then kills CPU1 by doing the
>>> magic incantations on the power controller
>>>
>>> What is missing here is all the cache cleaning before signalling
>>> CPU0. If you add that, things should look a lot better (patches
>>> welcome).
>
>> Unsure about this, or maybe this was too simplistic: I added calls
>> to u-boot's flush_dcache_all and invalidate_icache_all (right
>> after disabling the cache, just like the vendor kernel does), but
>> CPU0 still locks up. I suspect there is still a bug in the FIQ
>> handling. There is also a suspicious single "@" printed on the
>> console. I'll play with the FIQ handler a bit.
>
> The '@' is just my own debug stuff, and might be causing issues too.
>
> Now, you have to realise that by the time you call into this code,
> u-boot itself is long gone. Only the tiny bit of code dealing with
> PSCI still lives in a bank of static, secure memory. So calling into
> u-boot for anything is doomed. You need to actually put the code
> inside the PSCI backend.
OK - seems like quite a bit of code needs to be pulled over...
Meanwhile I'm still starring at psci_fiq_enter:
...
movw r8, #(GICC_BASE & 0xffff)
movt r8, #(GICC_BASE >> 16)
ldr r9, [r8, #GICC_IAR]
movw r10, #0x3ff
movt r10, #0
cmp r9, r10
bne out1
movw r10, #0x3fe
cmp r9, r10
bne out1
...
How can GICC_IAR be 0x3ff and 0x3fe at the same time? These tests seems
bogus. What was the actual intention here?
Jan
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 10/11/14 10:33, Jan Kiszka wrote:
> On 2014-11-10 10:52, Marc Zyngier wrote:
>> On 10/11/14 09:36, Jan Kiszka wrote:
>>> On 2014-11-10 10:17, Marc Zyngier wrote:
>>>> On 10/11/14 08:25, Jan Kiszka wrote:
>>>>> On 2014-11-10 07:03, Jan Kiszka wrote:
>>>>>> On 2014-11-10 00:17, Maxime Ripard wrote:
>>>>>>> Hi Jan,
>>>>>>>
>>>>>>> On Sun, Nov 09, 2014 at 08:35:49PM +0100, Jan Kiszka
>>>>>>> wrote:
>>>>>>>> did anyone already happen to look into enabling CPU
>>>>>>>> hotplug for the Allwinner A20 in upstream? I'm
>>>>>>>> currently running the sunxi-next branch on Banana Pi,
>>>>>>>> and echo 0 > .../cpu1/online just hangs the system.
>>>>>>>> The old 3.4 LeMaker kernel works fine in this regard.
>>>>>>>> I can try to look into details and port things over,
>>>>>>>> just want to avoid duplicate efforts.
>>>>>>>
>>>>>>> Having hotplug support would indeed be very welcome.
>>>>>>>
>>>>>>> However, it should be done in u-boot, through PSCI, and
>>>>>>> not in the kernel itself.
>>>>>>>
>>>>>>> As far as I'm aware, no one worked actively on it,
>>>>>>> beside some WIP commit from Marc a while ago:
>>>>>>> https://git.kernel.org/cgit/linux/kernel/git/maz/u-boot.git/commit/?h=wip/psci&id=45379c0f9cf812f0f62722f4015ec907fa5dc144
>>>>>>
>>>>>>
>>>>>>>
>>>>
>>>>>>>
>>
>>>>>>>
OK - I guess I will need a little guidance in then: Is there a good
>>>>>> reference board to study and to derive from? And maybe
>>>>>> also: What is missing or not working in that u-boot
>>>>>> branch? If I get this interface right, I just takes some
>>>>>> device tree bits to enable this for the kernel afterward,
>>>>>> correct?
>>>>>
>>>>> Started to play with that patch in naive ways: CPU0 locks
>>>>> up when offlining CPU1 - unless I disable the FIQ signal
>>>>> from CPU1. Then it "works", both offlining and onlining
>>>>> again. However, I suspect that this only parks CPU1 in wfi
>>>>> and does not do anything interesting to it.
>>>>
>>>> Here's how this is supposed to work: - CPU1 sends a FIQ to
>>>> CPU0, bringing it into secure mode. - CPU0 then kills CPU1 by
>>>> doing the magic incantations on the power controller
>>>>
>>>> What is missing here is all the cache cleaning before
>>>> signalling CPU0. If you add that, things should look a lot
>>>> better (patches welcome).
>>
>>> Unsure about this, or maybe this was too simplistic: I added
>>> calls to u-boot's flush_dcache_all and invalidate_icache_all
>>> (right after disabling the cache, just like the vendor kernel
>>> does), but CPU0 still locks up. I suspect there is still a bug
>>> in the FIQ handling. There is also a suspicious single "@"
>>> printed on the console. I'll play with the FIQ handler a bit.
>>
>> The '@' is just my own debug stuff, and might be causing issues
>> too.
>>
>> Now, you have to realise that by the time you call into this
>> code, u-boot itself is long gone. Only the tiny bit of code
>> dealing with PSCI still lives in a bank of static, secure memory.
>> So calling into u-boot for anything is doomed. You need to
>> actually put the code inside the PSCI backend.
>
> OK - seems like quite a bit of code needs to be pulled over...
>
> Meanwhile I'm still starring at psci_fiq_enter: ... movw r8,
> #(GICC_BASE & 0xffff) movt r8, #(GICC_BASE >> 16) ldr r9, [r8,
> #GICC_IAR] movw r10, #0x3ff movt r10, #0 cmp r9, r10 bne out1 movw
> r10, #0x3fe cmp r9, r10 bne out1 ...
>
> How can GICC_IAR be 0x3ff and 0x3fe at the same time? These tests
> seems bogus. What was the actual intention here?
Yup, that looks like a massive brain fart. These two 'bne' should
really be 'beq's... You want to get out if you either read that there
is no interrupt (0x3ff) or an interrupt for non-secure (0x3fe).
M.
- --
Jazz is not dead. It just smells funny...
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/
iQIcBAEBAgAGBQJUYJx9AAoJECPQ0LrRPXpD/0kP/jwdNF2Ahtj05biOkCOTmp3V
PUPNbffo8366UZeOLMjt7Wplde+//swOiLOq0uERS5IYj+QCMOA4F9hWOSO527zn
v1j6374hAW1F8p8waGRxOGEffUgZGhXPmOxQAIsr7iqsIrvUVbE6KcNS5ofs+29R
lWEaNW08L09CxcDA7szZzhVcMfu3oUxjzVFnnh6uhHG5DMxhCWt3IgEj2cLUR+VC
ER/VAS9r0/4Bl5sZ7VO+GeToolgswIlIGgjkqifwrKFYH8XSyIxN8LDrNLAR1F4Q
RQYfo3I7UGZcOPzz+BJcOsROohx2wtDeW6VxnRsE5WkE/udx++C7Ty9IquFrvPxA
1u8pzTrra27ZdxbrV2vS8YVo8bQktTdz8IjITuvWVX/cutXBdqIeP75EXYQQXeCg
TuO26Huce4McEtn7bOEQac6SxLpTitGM+EAdrlhV9Dg7bCp1mKfzlIM4t6KuowEU
53mniGtDoLlnyUy+N4X6KyUmvqqyRNyC+c3ZWmP5xoB3VcQ1IJKuSEHSS9bJxIKb
J2FNQN7v+C9P0d/1bItVqlpz6PiLcdWFFrfvDCxOe0uqkacwzhdlMMzrYYuMgAIi
e0u8fqYunECZliT5OMaRhqBwSsniZmw2bBf6ZaUqlKgXc5MsxjncNakOKMQkQjip
FwtcX/9StOztp43Laj7o
=LZJR
-----END PGP SIGNATURE-----