Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 28 Jul 2023 12:29:46 -0400
From:   Frank Li <Frank.li@nxp.com>
To:     Will Deacon <will@kernel.org>
Cc:     "Russell King (Oracle)" <linux@armlinux.org.uk>,
        Andrew Halaney <ahalaney@redhat.com>,
        Shenwei Wang <shenwei.wang@nxp.com>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Maxime Coquelin <mcoquelin.stm32@gmail.com>,
        Shawn Guo <shawnguo@kernel.org>,
        Sascha Hauer <s.hauer@pengutronix.de>,
        Neil Armstrong <neil.armstrong@linaro.org>,
        Kevin Hilman <khilman@baylibre.com>,
        Vinod Koul <vkoul@kernel.org>, Chen-Yu Tsai <wens@csie.org>,
        Jernej Skrabec <jernej.skrabec@gmail.com>,
        Samuel Holland <samuel@sholland.org>,
        Giuseppe Cavallaro <peppe.cavallaro@st.com>,
        Alexandre Torgue <alexandre.torgue@foss.st.com>,
        Jose Abreu <joabreu@synopsys.com>,
        Pengutronix Kernel Team <kernel@pengutronix.de>,
        Fabio Estevam <festevam@gmail.com>,
        NXP Linux Team <linux-imx@nxp.com>,
        Jerome Brunet <jbrunet@baylibre.com>,
        Martin Blumenstingl <martin.blumenstingl@googlemail.com>,
        Bhupesh Sharma <bhupesh.sharma@linaro.org>,
        Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>,
        Simon Horman <simon.horman@corigine.com>,
        Bartosz Golaszewski <bartosz.golaszewski@linaro.org>,
        Wong Vee Khee <veekhee@apple.com>,
        Revanth Kumar Uppala <ruppala@nvidia.com>,
        Jochen Henneberg <jh@henneberg-systemdesign.com>,
        netdev@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        linux-amlogic@lists.infradead.org, imx@lists.linux.dev
Subject: Re: [PATCH v2 net 2/2] net: stmmac: dwmac-imx: pause the TXC clock
 in fixed-link
Message-ID: <ZMPs+sOIzWR0LmrP@lizhi-Precision-Tower-5810>
References: <20230727152503.2199550-1-shenwei.wang@nxp.com>
 <20230727152503.2199550-3-shenwei.wang@nxp.com>
 <4govb566nypifbtqp5lcbsjhvoyble5luww3onaa2liinboguf@4kgihys6vhrg>
 <ZMPdKyOtpZKEMLsO@shell.armlinux.org.uk>
 <20230728153611.GH21718@willie-the-truck>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230728153611.GH21718@willie-the-truck>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0JCEWwfr+l4iWIHTUlfdxXjua4I7Qz6br89mX0wXGLEvKdHR6oSmZMq37vnc?=
 =?us-ascii?Q?9qRx/yWtnxeVY2aYyW0NkQXMlX7NFPioc9zOdqUQR6EM7of98z1Om6OyYbTI?=
 =?us-ascii?Q?thHwqxEdoqI3rBXQ2dVurM+pQQhaCmxIK/R9cP5003x87Y2IR8ZTEKPGhMjh?=
 =?us-ascii?Q?8eKAoMNFTC5Oh9woHt5fnNtGwEm2E2KAZmF+AcAfu1Kp8ifOMkRNwJ8Lx56E?=
 =?us-ascii?Q?sA9zz+GSQSjYBUSqFLiC19DG/+zLeuYpRmiZid2YVgUZjudI5Z8vdpp9sDP8?=
 =?us-ascii?Q?OxSQnXp5RpU/MBaJsLY70eHNeOVse0Fj8VKl/G376gNah7PZ6jTwdBdovFLe?=
 =?us-ascii?Q?8T6FsTgOSRAha9gvwV2JKgASkl7VvX4ELihaLcMv/HDFSdMZVjF8BEWLxYBs?=
 =?us-ascii?Q?wmogMkIWF1OwvqBtRAQbDO5zMeOWosxlWKBUYVSrkFHJOG9v9OvVPPg193ad?=
 =?us-ascii?Q?2UAJ614GIx+XVyLipNj0/3NL1D8t14e/c/ZKDUHtkNIEGO+E569Of0HfGSVR?=
 =?us-ascii?Q?mkBiKuG0zzxrpulThf8obQkcyI6O81GlPbQeqiBT8NDQ6cSr//FJnq2tPiHM?=
 =?us-ascii?Q?SYu6P32fQCCGkBzBp/gEEs0ATgwK8fTL22p0Y3meQDnyFPDK0sHBL1DOQL6b?=
 =?us-ascii?Q?jtZWnBXslYyXgmt8kOryMxenlwn9DpFvp9azF8JCDzlGRRhn76z8VvEKss7w?=
 =?us-ascii?Q?accjGwmG0iJDr7lQKtCP44ySVXfbKVuoaRgnvSUV2ouyqG4m6HXKBoor28tn?=
 =?us-ascii?Q?ZzrvhtvMPBUO0vGRe5wlvp9/Vo2Ker7zH495o8azJ7Rsm0A+OH506LBXvEAk?=
 =?us-ascii?Q?BS1CU6WkR/g6AQS/E44BX4YMgiBbcEVH5zXtXILwMexjC9YlT+v39GJs8kZM?=
 =?us-ascii?Q?mZxKuBhMPETds/WS88zMXo/pahoqfRgrIQdELHx3xCIrhziTlNqaUeQa7TVz?=
 =?us-ascii?Q?X8FFjOEaASadETO565WTme/LpovYmaRyb6BEhSs1h3SNQXZlsIsPYQF/l7BR?=
 =?us-ascii?Q?oGMKMVnvlB0QzdigBJUwiux7va6cWDz9eRTB6Mz4kjhcKzAcCuEcbMKiyS4w?=
 =?us-ascii?Q?2etJdXtV5WJNruY9LcYPqwFO2xpNdohQ/+GfDlADt8FbcGZnzLiphZzhB5az?=
 =?us-ascii?Q?hi3Etttz16eYmVjPw5PvkzJv84CqAQgCdyi/d0G6iKO2tFkDUbS1+xNfTjL1?=
 =?us-ascii?Q?7T26/v5tG9AbYk8Gi8h91W5JSQn93Q5ZttMnRX6a6wYyY3Bab88h8P8Gbv62?=
 =?us-ascii?Q?j53Zz70Luz6EM4ceTo23QwonTLOnxuDyIk2m/R/qAAI2ybDBavzA1/AneZDj?=
 =?us-ascii?Q?p+WRYDkbwDx4GC/BerjYlwwpvH74Rmr/IOYQQBm2HCjlwBPbr/Z44qUwH2cx?=
 =?us-ascii?Q?tfDkWhokOcOYg6/Amxw9m++B11Qr1DFTAHSGdIZVSJmAHls802ebJdCog5U/?=
 =?us-ascii?Q?UESmdOCfUCl6fOP+2kRZyWNzI1dZUf9uLUl1+UN+QnmoRG7jcsoutrw9Km7T?=
 =?us-ascii?Q?0ihFpV38oyorqd4elMQ1bgjzp+BtOtVmHrZwanec8CGgwokskkz0ef3z0fOL?=
 =?us-ascii?Q?m4l859yJFTjR4OZ+YaI=3D?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d5027379-098a-469c-d080-08db8f87e59f
X-MS-Exchange-CrossTenant-AuthSource: AM6PR04MB4838.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jul 2023 16:30:04.0288
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XacQSG8hN98OLGu58MKydZSHGIT+HPx0m2rHYUGX74mD+zYRIscKCanqxYbUCkEVgkbv/QAfg9UxtvCNfkEPaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8914
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE,
        URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jul 28, 2023 at 04:36:12PM +0100, Will Deacon wrote:
> On Fri, Jul 28, 2023 at 04:22:19PM +0100, Russell King (Oracle) wrote:
> > On Thu, Jul 27, 2023 at 01:36:45PM -0500, Andrew Halaney wrote:
> > > I don't have any documentation for the registers here, and as you can
> > > see I'm an amateur with respect to memory ordering based on my prior
> > > comment.
> > > 
> > > But you:
> > > 
> > >     1. Read intf_reg_off into variable iface
> > >     2. Write the RESET_SPEED for the appropriate mode to MAC_CTRL_REG
> > >     3. wmb() to ensure that write goes through
> > 
> > I wonder about whether that wmb() is required. If the mapping is
> > device-like rather than memory-like, the write should be committed
> > before the read that regmap_update_bits() does according to the ARM
> > memory model. Maybe a bit of information about where this barrier
> > has come from would be good, and maybe getting it reviewed by the
> > arm64 barrier specialist, Will Deacon. :)
> > 
> > wmb() is normally required to be paired with a rmb(), but we're not
> > talking about system memory here, so I also wonder whether wmb() is
> > the correct barrier to use.
> 
> Yes, I don't think wmb() is the right thing here. If you need to ensure
> that the write to MAC_CTRL_REG has taken effect, then you'll need to go
> through some device-specific sequence which probably involves reading
> something back. If you just need things to arrive in order eventually,
> the memory type already gives you that.
> 
> It's also worth pointing out that udelay() isn't necessarily ordered wrt
> MMIO writes, so that usleep_range() might need some help as well.

Hi Deacon:

Does it means below pattern will be problem?

1.writel()
2.udelay()
3.writel()

It may not wait enough time between 1 and 3. I think the above pattern
is quite common in driver code.  I am not sure if usleep_range involve
MMIO to get current counter, ARM may use cp15 to get local timer counter.

In our system, readl() is quite slow because cross some bus bridge.
even readl() can work, we don't know it is because delay by readl() itself.
Or it works logically. Suppose readl() and writel() just guarantee memory
access order. 

Frank

> Non-relaxed MMIO reads, however, _are_ ordered against a subsequent
> udelay(), so if you add the readback then this might all work out.
> 
> I gave a (slightly dated) talk about some of this at ELC a while back:
> 
> https://www.youtube.com/watch?v=i6DayghhA8Q
> 
> which might help.
> 
> Will