Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 1 Aug 2022 12:30:44 +0300
From:   Vadym Kochan <vadym.kochan@plvision.eu>
To:     Florian Fainelli <f.fainelli@gmail.com>
Cc:     Hu Ziji <huziji@marvell.com>,
        Adrian Hunter <adrian.hunter@intel.com>,
        Ulf Hansson <ulf.hansson@linaro.org>,
        linux-mmc@vger.kernel.org, linux-kernel@vger.kernel.org,
        Elad Nachman <enachman@marvell.com>
Subject: Re: [PATCH] mmc: sdhci-xenon: Fix 2G limitation on AC5 SoC
Message-ID: <20220801093044.GA22721@plvision.eu>
References: <20220726170711.30324-1-vadym.kochan@plvision.eu>
 <139317dc-15e2-ac63-0e04-295e715a7747@gmail.com>
 <20220727164532.GA19351@plvision.eu>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220727164532.GA19351@plvision.eu>
User-Agent: Mutt/1.9.4 (2018-02-28)
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2EVz4E7TT5ezagSiOgxpqgG782ustKP8qEZj6pNFKkE3RZJV6ittlhH19jIo?=
 =?us-ascii?Q?12NIR0HZWqsyP5AETv78Hw421kyFWcdbQmRO2hZyC5uhMmT56Z5l6G3vhEFd?=
 =?us-ascii?Q?vjlg+JA0mODRPrQa8EUKPu+cJL7ZqcGc82xKXr5VUhPOdSUcMMkc5+0xK4rz?=
 =?us-ascii?Q?xBM0lL48EuhM13A8YP+vNCs8rzI3rYQ4NjnpkcHGatxwzpQBFYx0rP1Ju8Xg?=
 =?us-ascii?Q?h3QQ/PTBsPgCO4AFr5J62cQMxrhttK2VWsIPuB+hfVzf+FItJEvWKS7844X+?=
 =?us-ascii?Q?DWvNk4a3T3QU2qHwLSt4kSvbi55Ey+jYjYgOG4LR23eAyfiIkmqFCO++AV7X?=
 =?us-ascii?Q?s4LxUJ+37sh1FHdB0f5YPv85DAxx92PQVsdWuXSGVSq/zuJcCZR/SdAjjSb1?=
 =?us-ascii?Q?weg2AR7kd6bZFLuFkfzjYjcZBDVUjx4FPIVw1k3AgmdRpojpV0Sr8jh8jQGG?=
 =?us-ascii?Q?uJN7m+ZiZB1mpNLzmFrpWxuXldFx8ct6GD4EBVAOpUIuUollhPIDq/fXePjt?=
 =?us-ascii?Q?Co3/PQTNCeliAMknLc0iCcLgE1bhtIlbYSex5BCBRKrToCmSFCRwNvuhhl+W?=
 =?us-ascii?Q?cq8hiGzpyZauE3P7lL7mMIhX2nUlgfkTOn1BcPD6uCTVCPJALLtuVgSxz023?=
 =?us-ascii?Q?SPcNht6XXUTicqOa//3NgzAFR95Wh383NIccYEX/T4rIckxjupZ7wlRhycmz?=
 =?us-ascii?Q?KKFPY9kQxkMt5o2IIfuvZQgdoG7cfp7+gc8f2TgYgTHQmdJ4Bb1wMsmtwe64?=
 =?us-ascii?Q?K4sie+V0EnK7hhk+A1qMKXHT4eah/3s2j6kOyPR+g5Ymz8gLBBuuhKZf9n8+?=
 =?us-ascii?Q?0skrh24fYxUzDzjdLv9nm/4FKbxbhx5Xe6nkran8TzZHlI17U/yqwXffQ03w?=
 =?us-ascii?Q?43WYE0FpPpc6R4Ess/zQBsOLHs+x2uGReow26MXBSW2no2E5btwvdNF2iatj?=
 =?us-ascii?Q?1VjJId4yCBKQo7g1o4VvTWXAJYBiDCK/SjM/lyjf9ojMMKNl7yAbBNMWJAUc?=
 =?us-ascii?Q?DKbMgrOvbDzUqZNhWeCPAwe52NmHbrFu0/bziBmNpcvLaVgJ6pxfEXywUlnp?=
 =?us-ascii?Q?+aGL8jD7eZ+WZTDJxRLqYX1SAdq+vkZQLLGQPtPXBRkAbAp0anA5N3vY4whc?=
 =?us-ascii?Q?l7Ay7PJY9EiwprDZcMSlDpaUkcEJHoIU91nw1gejUNDnEDG4NhEYSXeMgI//?=
 =?us-ascii?Q?3/QEDNxYFhSE85dPT5WLvmp5V2RCNP2d1XWTzzh3C7k2hHNcx29azv0+QMA4?=
 =?us-ascii?Q?Rj8j8zyTxbJHsQ00RrfSKw2aOJVerwWEqBjiymN8AWjABvSDjYoW/2Xmu8dL?=
 =?us-ascii?Q?R1cAM8Xsgn8zJ7ppdbyOX+sedps0xwMZMJgFcWQ+TMQtyHjQEzjEhU7tyuwh?=
 =?us-ascii?Q?G3C44MP0loN6aCQKvF9wPvyCaQ1C7KwW8i7UKeFdYlkDLqT0eXESrfQeW03/?=
 =?us-ascii?Q?uahnet/AnTawFZuTj37nTyrVDLdtFzUrykXtMQCX5LMS3EiW/jvM7orugPL8?=
 =?us-ascii?Q?b2JKiYZB3HDhVlSReAcEbFqP6QU3ZPq2TvBtpmof5JE7cUw0ThALstDjnN5B?=
 =?us-ascii?Q?WquZSQtn+8UPbdjnc2kQAUwN2hZafE8HMkI+kEh/+0WfZZD5uahzqBM3kzm9?=
 =?us-ascii?Q?LA=3D=3D?=
X-OriginatorOrg: plvision.eu
X-MS-Exchange-CrossTenant-Network-Message-Id: 7053eefc-7ca5-43ce-f315-08da73a083be
X-MS-Exchange-CrossTenant-AuthSource: VI1P190MB0317.EURP190.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Aug 2022 09:30:47.0043
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 03707b74-30f3-46b6-a0e0-ff0a7438c9c4
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 00lfxHDU4nYAXq7mzmLE+BzCqLv1Q4qS9jMpT/wN32DiaDk/8zZhs67+JOXUbKxZS5T8xTxxT5zAZ4wv5s5vuT2CnLi8uorWTtq82afkA9c=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PR3P190MB0988
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Florian,

On Wed, Jul 27, 2022 at 07:45:32PM +0300, Vadym Kochan wrote:
> Hi Florian,
> 
> On Tue, Jul 26, 2022 at 10:37:46AM -0700, Florian Fainelli wrote:
> > On 7/26/22 10:07, Vadym Kochan wrote:
> > > From: Elad Nachman <enachman@marvell.com>
> > > 
> > > There is a limitation on AC5 SoC that mmc controller
> > > can't have DMA access over 2G memory.
> > > 
> > > Signed-off-by: Elad Nachman <enachman@marvell.com>
> > > Signed-off-by: Vadym Kochan <vadym.kochan@plvision.eu>
> > > ---
> > >  drivers/mmc/host/sdhci-xenon.c | 13 +++++++++++++
> > >  1 file changed, 13 insertions(+)
> > > 
> > > diff --git a/drivers/mmc/host/sdhci-xenon.c b/drivers/mmc/host/sdhci-xenon.c
> > > index 08e838400b52..666d06b58564 100644
> > > --- a/drivers/mmc/host/sdhci-xenon.c
> > > +++ b/drivers/mmc/host/sdhci-xenon.c
> > > @@ -18,6 +18,7 @@
> > >  #include <linux/of.h>
> > >  #include <linux/pm.h>
> > >  #include <linux/pm_runtime.h>
> > > +#include <linux/mm.h>
> > >  
> > >  #include "sdhci-pltfm.h"
> > >  #include "sdhci-xenon.h"
> > > @@ -422,6 +423,8 @@ static int xenon_probe_params(struct platform_device *pdev)
> > >  	struct xenon_priv *priv = sdhci_pltfm_priv(pltfm_host);
> > >  	u32 sdhc_id, nr_sdhc;
> > >  	u32 tuning_count;
> > > +	struct device_node *np = pdev->dev.of_node;
> > > +	struct sysinfo si;
> > >  
> > >  	/* Disable HS200 on Armada AP806 */
> > >  	if (priv->hw_version == XENON_AP806)
> > > @@ -450,6 +453,15 @@ static int xenon_probe_params(struct platform_device *pdev)
> > >  	}
> > >  	priv->tuning_count = tuning_count;
> > >  
> > > +	si_meminfo(&si);
> > > +
> > > +	if (of_device_is_compatible(np, "marvell,ac5-sdhci") &&
> > > +	    ((si.totalram * si.mem_unit) > 0x80000000 /*2G*/)) {
> > 
> > Why not limit the DMA mask of the device and ensure, that bounce buffers get used so you can still do DMA?
> > 
> > Also, you ought to be able to describe that limitation using Device Tree (assuming this is an option) and declaring a dedicated bus node for the SDHCI controller and providing a suitable dma-ranges property, see: arch/arm/boot/dts/bcm2711.dtsi and the 'soc' node for such examples.
> > 
> > 
> 

I could use DMA only in 2 ways:

#1 Use sdhci bounce buffer with SDMA mode

    But there was the issue that SDMA requires that SDHCI v4 mode should
    be enabled, and when I enable it via sdhci_enable_v4_mode(host)
    then I got error that EXT_CSD can't be recognized.

    But if I comment this line in sdhci.c:
    
    int sdhci_setup_host(struct sdhci_host *host)
    {
        ...

	/* SDMA does not support 64-bit DMA if v4 mode not set */
	if ((host->flags & SDHCI_USE_64_BIT_DMA) && !host->v4_mode) {
		pr_info("XXX SDMA does not support 64-bit DMA if v4 mode not set\n");
		host->flags &= ~SDHCI_USE_SDMA;
	}

       ...
    }

    then everything is OK.

#2 Use restricted-dma-pool in device-tree

   But I am not sure if it is good solution compared to #1.

Setting only DMA mask did not help because after some time I got
"DMA overflow address" error stack-traces.

> Do I understand correctly that the swiotlb will be used in case DMA
> engine could not map the page in the specified range (limited by dma-ranges or
> dma-mask) ?
> 
> > 
> > > +		host->quirks |= SDHCI_QUIRK_BROKEN_DMA;
> > > +		host->quirks |= SDHCI_QUIRK_BROKEN_ADMA;
> > > +		dev_info(mmc_dev(mmc), "Disabling DMA because of 2GB DMA access limit.\n");
> > > +	}
> > > +
> > >  	return xenon_phy_parse_params(dev, host);
> > >  }
> > >  
> > > @@ -682,6 +694,7 @@ static const struct of_device_id sdhci_xenon_dt_ids[] = {
> > >  	{ .compatible = "marvell,armada-ap807-sdhci", .data = (void *)XENON_AP807},
> > >  	{ .compatible = "marvell,armada-cp110-sdhci", .data =  (void *)XENON_CP110},
> > >  	{ .compatible = "marvell,armada-3700-sdhci", .data =  (void *)XENON_A3700},
> > > +	{ .compatible = "marvell,ac5-sdhci", .data = (void *)XENON_AP806},
> > >  	{}
> > >  };
> > >  MODULE_DEVICE_TABLE(of, sdhci_xenon_dt_ids);
> > 
> > 
> > -- 
> > Florian
> 
> Thanks,
> 

Regards,