OK, this took me a while to figure out.
When no undervoltage limit is configured, I can reliably trigger the initialization bug upon boot.
When the limit is set to 3.0V, it rarely occurs, but just after I send the v3 patch, I was able to reproduce...
> Am 15.07.2022 um 19:12 schrieb Christian Kohlschütter <[email protected]>:
>
> mmc/SD-card initialization may fail on NanoPi R4S with
> "mmc1: problem reading SD Status register" /
> "mmc1: error -110 whilst initialising SD card"
> either on cold boot or after a reboot.
>
> Moreover, the system would also sometimes hang upon reboot.
>
> This is prevented by setting an explicit undervoltage protection limit
> for the SD-card-specific vcc3v0_sd voltage regulator.
>
> Set the undervoltage protection limit to 2.7V, which is the minimum
> permissible SD card operating voltage.
>
> Signed-off-by: Christian Kohlschütter <[email protected]>
> ---
> arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi | 4 ++++
> 1 file changed, 4 insertions(+)
> mode change 100644 => 100755 arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>
> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
> old mode 100644
> new mode 100755
> index 8c0ff6c96e03..669c74ce4d13
> --- a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
> +++ b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
> @@ -73,6 +73,10 @@ vcc3v0_sd: vcc3v0-sd {
> regulator-always-on;
> regulator-min-microvolt = <3000000>;
> regulator-max-microvolt = <3000000>;
> +
> + // must be configured or SD card may fail to initialize occasionally
> + regulator-uv-protection-microvolt = <2700000>;
> +
> regulator-name = "vcc3v0_sd";
> vin-supply = <&vcc3v3_sys>;
> };
> --
> 2.36.1
On 2022-07-15 18:16, Christian Kohlschütter wrote:
> OK, this took me a while to figure out.
>
> When no undervoltage limit is configured, I can reliably trigger the initialization bug upon boot.
> When the limit is set to 3.0V, it rarely occurs, but just after I send the v3 patch, I was able to reproduce...
Well this has to be in the running for "weirdest placebo ever"... :/
All it actually seems to achieve is printing an error[1] (this is after
all a tiny 5-pin fixed-voltage LDO regulator, not an intelligent PMIC),
and if that makes an appreciable difference then there has to be some
kind of weird timing condition at play. Maybe regulator_register() ends
up turning it off and on again rapidly enough that the card sees a
voltage brownout and glitches, and adding more delay by printing to the
console somewhere in the middle gives it enough time to act as a proper
power cycle with no ill effect?
If you just whack something like an mdelay(500) at around that point in
set_machine_constraints(), without the DT property, does it have the
same effect?
Robin.
[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/regulator/core.c#n1521
>> Am 15.07.2022 um 19:12 schrieb Christian Kohlschütter <[email protected]>:
>>
>> mmc/SD-card initialization may fail on NanoPi R4S with
>> "mmc1: problem reading SD Status register" /
>> "mmc1: error -110 whilst initialising SD card"
>> either on cold boot or after a reboot.
>>
>> Moreover, the system would also sometimes hang upon reboot.
>>
>> This is prevented by setting an explicit undervoltage protection limit
>> for the SD-card-specific vcc3v0_sd voltage regulator.
>>
>> Set the undervoltage protection limit to 2.7V, which is the minimum
>> permissible SD card operating voltage.
>>
>> Signed-off-by: Christian Kohlschütter <[email protected]>
>> ---
>> arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi | 4 ++++
>> 1 file changed, 4 insertions(+)
>> mode change 100644 => 100755 arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>
>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>> old mode 100644
>> new mode 100755
>> index 8c0ff6c96e03..669c74ce4d13
>> --- a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>> +++ b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>> @@ -73,6 +73,10 @@ vcc3v0_sd: vcc3v0-sd {
>> regulator-always-on;
>> regulator-min-microvolt = <3000000>;
>> regulator-max-microvolt = <3000000>;
>> +
>> + // must be configured or SD card may fail to initialize occasionally
>> + regulator-uv-protection-microvolt = <2700000>;
>> +
>> regulator-name = "vcc3v0_sd";
>> vin-supply = <&vcc3v3_sys>;
>> };
>> --
>> 2.36.1
>
> Am 15.07.2022 um 20:11 schrieb Robin Murphy <[email protected]>:
>
> On 2022-07-15 18:16, Christian Kohlschütter wrote:
>> OK, this took me a while to figure out.
>> When no undervoltage limit is configured, I can reliably trigger the initialization bug upon boot.
>> When the limit is set to 3.0V, it rarely occurs, but just after I send the v3 patch, I was able to reproduce...
>
> Well this has to be in the running for "weirdest placebo ever"... :/
>
> All it actually seems to achieve is printing an error[1] (this is after all a tiny 5-pin fixed-voltage LDO regulator, not an intelligent PMIC), and if that makes an appreciable difference then there has to be some kind of weird timing condition at play. Maybe regulator_register() ends up turning it off and on again rapidly enough that the card sees a voltage brownout and glitches, and adding more delay by printing to the console somewhere in the middle gives it enough time to act as a proper power cycle with no ill effect?
That's definitely something between placebo and homeopathy :-)
I can confirm that setting a limit higher than 3.0V still works, which means that the one time incident where it still crashed means that there's indeed a timing issue at play, and adding that undervoltage statement (unlike the ramp-delay configs that I also tried) added just enough of a delay that made it work 99 out of 100 times.
> If you just whack something like an mdelay(500) at around that point in set_machine_constraints(), without the DT property, does it have the same effect?
Adding a delay for vcc3v0_sd works, which is great! (patch below)
Is there an existing path from device-tree parser to regular/core.c that we can use to specify this delay specifically for this regulator?
Also, what delay should we choose to make sure it works all the time and not just 99 out of 100 times?
Best,
Christian
diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c
index c4d844ffad7a..0e15ec2548f4 100644
--- a/drivers/regulator/core.c
+++ b/drivers/regulator/core.c
@@ -1483,6 +1483,11 @@ static int set_machine_constraints(struct regulator_dev *rdev)
"IC does not support requested over voltage limits\n");
}
+if(!strncmp(rdev_get_name(rdev),"vcc3v0_sd",sizeof("vcc3v0_sd"))) {
+ rdev_err(rdev, "DELAY: %s\n", rdev_get_name(rdev));
+ mdelay(500);
+}
+
if (rdev->constraints->under_voltage_detection)
ret = handle_notify_limits(rdev,
ops->set_under_voltage_protection,
On 2022-07-15 19:11, Robin Murphy wrote:
> On 2022-07-15 18:16, Christian Kohlschütter wrote:
>> OK, this took me a while to figure out.
>>
>> When no undervoltage limit is configured, I can reliably trigger the
>> initialization bug upon boot.
>> When the limit is set to 3.0V, it rarely occurs, but just after I send
>> the v3 patch, I was able to reproduce...
>
> Well this has to be in the running for "weirdest placebo ever"... :/
>
> All it actually seems to achieve is printing an error[1] (this is after
> all a tiny 5-pin fixed-voltage LDO regulator, not an intelligent PMIC),
> and if that makes an appreciable difference then there has to be some
> kind of weird timing condition at play. Maybe regulator_register() ends
> up turning it off and on again rapidly enough that the card sees a
> voltage brownout and glitches, and adding more delay by printing to the
> console somewhere in the middle gives it enough time to act as a proper
> power cycle with no ill effect?
...and apparently the answer is yes, it seems to be doing exactly that
(see attached). But seemingly my SD cards don't mind, or maybe my T4
board happens to have more capacitance than Christian's R4S so my
voltage dip isn't as bad, or both.
So it seems like the solution here might indeed simply be to remove the
regulator-always-on which doesn't seem to have any reason to be here
anyway. Without that, the enable stays low until the MMC driver probes
and claims it, which is then massively longer than the time it takes for
VCC3V0_SD to ramp down completely.
Robin.
>
> If you just whack something like an mdelay(500) at around that point in
> set_machine_constraints(), without the DT property, does it have the
> same effect?
>
> Robin.
>
> [1]
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/regulator/core.c#n1521
>
>
>>> Am 15.07.2022 um 19:12 schrieb Christian Kohlschütter
>>> <[email protected]>:
>>>
>>> mmc/SD-card initialization may fail on NanoPi R4S with
>>> "mmc1: problem reading SD Status register" /
>>> "mmc1: error -110 whilst initialising SD card"
>>> either on cold boot or after a reboot.
>>>
>>> Moreover, the system would also sometimes hang upon reboot.
>>>
>>> This is prevented by setting an explicit undervoltage protection limit
>>> for the SD-card-specific vcc3v0_sd voltage regulator.
>>>
>>> Set the undervoltage protection limit to 2.7V, which is the minimum
>>> permissible SD card operating voltage.
>>>
>>> Signed-off-by: Christian Kohlschütter <[email protected]>
>>> ---
>>> arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi | 4 ++++
>>> 1 file changed, 4 insertions(+)
>>> mode change 100644 => 100755
>>> arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>>
>>> diff --git a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>> b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>> old mode 100644
>>> new mode 100755
>>> index 8c0ff6c96e03..669c74ce4d13
>>> --- a/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>> +++ b/arch/arm64/boot/dts/rockchip/rk3399-nanopi4.dtsi
>>> @@ -73,6 +73,10 @@ vcc3v0_sd: vcc3v0-sd {
>>> regulator-always-on;
>>> regulator-min-microvolt = <3000000>;
>>> regulator-max-microvolt = <3000000>;
>>> +
>>> + // must be configured or SD card may fail to initialize
>>> occasionally
>>> + regulator-uv-protection-microvolt = <2700000>;
>>> +
>>> regulator-name = "vcc3v0_sd";
>>> vin-supply = <&vcc3v3_sys>;
>>> };
>>> --
>>> 2.36.1
>>
>
> _______________________________________________
> Linux-rockchip mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-rockchip