Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758992AbcLTQiy (ORCPT ); Tue, 20 Dec 2016 11:38:54 -0500 Received: from mail-sn1nam01on0081.outbound.protection.outlook.com ([104.47.32.81]:21472 "EHLO NAM01-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1758602AbcLTQis (ORCPT ); Tue, 20 Dec 2016 11:38:48 -0500 From: Rafal Ozieblo To: Andrei Pistirica , "netdev@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "davem@davemloft.net" , "nicolas.ferre@atmel.com" , "harinikatakamlinux@gmail.com" , "harini.katakam@xilinx.com" CC: "punnaia@xilinx.com" , "michals@xilinx.com" , "anirudh@xilinx.com" , "boris.brezillon@free-electrons.com" , "alexandre.belloni@free-electrons.com" , "tbultel@pixelsurmer.com" , "richardcochran@gmail.com" Subject: RE: [RFC PATCH net-next v4 2/2] macb: Enable 1588 support in SAMA5Dx platforms. Thread-Topic: [RFC PATCH net-next v4 2/2] macb: Enable 1588 support in SAMA5Dx platforms. Thread-Index: AQHSVfGUV4Vigz1rKk2Pt4rZw6Qu/aEQ/Bdw Date: Tue, 20 Dec 2016 16:38:45 +0000 Message-ID: References: <1481720175-12703-1-git-send-email-andrei.pistirica@microchip.com> <1481720175-12703-2-git-send-email-andrei.pistirica@microchip.com> In-Reply-To: <1481720175-12703-2-git-send-email-andrei.pistirica@microchip.com> Accept-Language: pl-PL, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rafalo@cadence.com; x-originating-ip: [213.131.238.28] x-ms-office365-filtering-correlation-id: b767f417-f2bc-491d-0019-08d428f6aaa1 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:BN3PR07MB2513; x-microsoft-exchange-diagnostics: 1;BN3PR07MB2513;7:QtVPEUCUiKgnnWNvlSrHXlqMU+aL+NiySRroFhilRnB5l+DbEC2eYqnohjyVt0iAw5DinwBYepr2WMOiNKk9oPIxJ+eMCKTxNfLBdjCnbLyiiQOtShfKPvVlcuEG+jpMMp+2a5s0ozM6aU/uR3+GGy0MH3ZISacj/kwDV2qkTDUnoYj+Z5H2vI1UKm1w7L/TpwC4pfbm6IVjNl6y0EFPqAQLy1Do+i1hDtwctct9cZetWVOeOyBh977pgexoy1shAVXF7k/k6Ac7vcwcB4vw/V0lEOQLiNjzbfMhAkY8uAFZaC9QoeQt+Id+MEjW7wxjEowYn1H6xGPiS0ErB5zvtVfMIj4nXPd3Dn92BaE26xchfPZW1XajF8l2mBW/0r1HKDSWcuSUUhzvgLivJAD5MijPFgUy7WnJP3OkhZq5MbiW+1bM9FAj1i3W3VOYASXtKl512odNfoK+T4oZMMmYSQ==;20:TGcXlqf5by1RyVg+W4Pw3bhpUgnP5eYOVtNgxfp/ZXm8Ldpx+57cl3XZ/iDVADknbgdhs05WAcudYANbwBN+20lEApuazt+ouctE5CrhxnHdLpOH6F6d9gqxRX+PshaBK0H7yY3O1bMHxG74nNt8tT9G/p+IfJiXxSuSDUplZ2Vxlq00HHe/RAk3FIvIHpUhtKt/UU7hhXYx68VVj4GUj6Defp5gO2INaH6kiOBiNCYlEDdDQpjbfZDeXVqifN6j x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(88287073810984)(72806322054110); x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(6072148);SRVR:BN3PR07MB2513;BCL:0;PCL:0;RULEID:;SRVR:BN3PR07MB2513; x-forefront-prvs: 0162ACCC24 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(6009001)(7916002)(39450400003)(36092001)(199003)(189002)(77096006)(7416002)(2501003)(76576001)(3280700002)(3660700001)(6506006)(8666005)(6116002)(25786008)(68736007)(92566002)(102836003)(4326007)(38730400001)(3846002)(6436002)(7696004)(2950100002)(66066001)(2906002)(39060400001)(99286002)(106356001)(189998001)(5001770100001)(122556002)(9686002)(101416001)(8676002)(5660300001)(105586002)(229853002)(86362001)(81166006)(8936002)(106116001)(2201001)(2900100001)(7736002)(97736004)(33656002)(305945005)(74316002)(50986999)(54356999)(81156014)(76176999)(15974865002)(7059030)(32563001)(217873001)(18886075002);DIR:OUT;SFP:1101;SCL:1;SRVR:BN3PR07MB2513;H:BN3PR07MB2516.namprd07.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-OriginatorOrg: cadence.com X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Dec 2016 16:38:45.5005 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: d36035c5-6ce6-4662-a3dc-e762e61ae4c9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR07MB2513 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by mail.home.local id uBKGd4o4019220 Content-Length: 30722 Lines: 336 From: Andrei Pistirica [mailto:andrei.pistirica@microchip.com] Sent: 14 grudnia 2016 13:56 > This patch does the following: > - Enable HW time stamp for the following platforms: SAMA5D2, SAMA5D3 and > SAMA5D4. > - HW time stamp capabilities are advertised via ethtool and macb ioctl is > updated accordingly. > - HW time stamp on the PTP Ethernet packets are received using the > SO_TIMESTAMPING API. Where timers are obtained from the PTP event/peer > registers. > > Note: Patch on net-next, on December 7th. > > Signed-off-by: Andrei Pistirica > --- > Patch history: > > Version 1: > Integration with SAMA5D2 only. This feature wasn't tested on any other platform that might use cadence/gem. > > Patch is not completely ported to the very latest version of net-next, and it will be after review. > > Version 2 modifications: > - add PTP caps for SAMA5D2/3/4 platforms > - and cosmetic changes > > Version 3 modifications: > - add support for sama5D2/3/4 platforms using GEM-PTP interface. > > Version 4 modifications: > - time stamp only PTP_V2 events > - maximum adjustment value is set based on Richard's input > > Note: Patch on net-next, on December 14th. > > drivers/net/ethernet/cadence/macb.c | 168 ++++++++++++++++++++++++++++++++++-- > 1 file changed, 163 insertions(+), 5 deletions(-) > > diff --git a/drivers/net/ethernet/cadence/macb.c b/drivers/net/ethernet/cadence/macb.c > index 538544a..8d5c976 100644 > --- a/drivers/net/ethernet/cadence/macb.c > +++ b/drivers/net/ethernet/cadence/macb.c > @@ -714,6 +714,8 @@ static void macb_tx_interrupt(struct macb_queue *queue) > > /* First, update TX stats if needed */ > if (skb) { > + gem_ptp_do_txstamp(bp, skb); > + I think, you can not do it in that way. It will hold two locks. If you enable appropriate option in kernel (as far as I remember CONFIG_DEBUG_SPINLOCK) you will get a warning here. Please look at following call-stack: 1. macb_interrupt() // spin_lock(&bp->lock) is taken 2. macb_tx_interrupt() 3. macb_handle_txtstamp() 4. skb_tstamp_tx() 5. __skb_tstamp_tx() 6. skb_may_tx_timestamp() 7. read_lock_bh() // second lock is taken I know that those are different locks and different types. But this could lead to deadlocks. This is the reason of warning I could see. And this is the reason why I get timestamp in interrupt routine but pass it to skb outside interrupt (using circular buffer). Please, refer to this: https://lkml.org/lkml/2016/11/18/168 1. macb_tx_interrupt() 2. macb_tx_timestamp_add() and schedule_work(&queue->tx_timestamp_task) Then, outside interrupt (without holding a lock) : 1. macb_tx_timestamp_flush() 2. macb_tstamp_tx() 3. skb_tstamp_tx() > netdev_vdbg(bp->dev, "skb %u (data %p) TX complete\n", > macb_tx_ring_wrap(bp, tail), > skb->data); > @@ -878,6 +880,8 @@ static int gem_rx(struct macb *bp, int budget) > GEM_BFEXT(RX_CSUM, ctrl) & GEM_RX_CSUM_CHECKED_MASK) > skb->ip_summed = CHECKSUM_UNNECESSARY; > > + gem_ptp_do_rxstamp(bp, skb); > + > bp->stats.rx_packets++; > bp->stats.rx_bytes += skb->len; > > @@ -2080,6 +2084,9 @@ static int macb_open(struct net_device *dev) > > netif_tx_start_all_queues(dev); > > + if (bp->ptp_info) > + bp->ptp_info->ptp_init(dev); > + > return 0; > } > > @@ -2101,6 +2108,9 @@ static int macb_close(struct net_device *dev) > > macb_free_consistent(bp); > > + if (bp->ptp_info) > + bp->ptp_info->ptp_remove(dev); > + > return 0; > } > > @@ -2374,6 +2384,133 @@ static int macb_set_ringparam(struct net_device *netdev, > return 0; > } > > +#ifdef CONFIG_MACB_USE_HWSTAMP > +static unsigned int gem_get_tsu_rate(struct macb *bp) { > + /* Note: TSU rate is hardwired to PCLK. */ > + return clk_get_rate(bp->pclk); > +} Not exactly. There could be separate TSU clock. In my solution I check tsu_clk in DT before I decide to take pclk. But it could be change in macb_ptp_info. > + > +static s32 gem_get_ptp_max_adj(void) > +{ > + return 3921508; > +} > + > +static int gem_get_ts_info(struct net_device *dev, > + struct ethtool_ts_info *info) > +{ > + struct macb *bp = netdev_priv(dev); > + > + ethtool_op_get_ts_info(dev, info); > + info->so_timestamping = > + SOF_TIMESTAMPING_TX_SOFTWARE | > + SOF_TIMESTAMPING_RX_SOFTWARE | > + SOF_TIMESTAMPING_SOFTWARE | > + SOF_TIMESTAMPING_TX_HARDWARE | > + SOF_TIMESTAMPING_RX_HARDWARE | > + SOF_TIMESTAMPING_RAW_HARDWARE; > + info->phc_index = -1; > + > + if (bp->ptp_clock) > + info->phc_index = ptp_clock_index(bp->ptp_clock); > + > + return 0; > +} > + > +static int gem_set_hwtst(struct net_device *netdev, > + struct ifreq *ifr, int cmd) > +{ > + struct hwtstamp_config config; > + struct macb *priv = netdev_priv(netdev); > + u32 regval; > + > + netdev_vdbg(netdev, "macb_hwtstamp_ioctl\n"); > + > + if (copy_from_user(&config, ifr->ifr_data, sizeof(config))) > + return -EFAULT; > + > + /* reserved for future extensions */ > + if (config.flags) > + return -EINVAL; > + > + switch (config.tx_type) { > + case HWTSTAMP_TX_OFF: > + priv->hwts_tx_en = false; > + break; > + case HWTSTAMP_TX_ON: > + priv->hwts_tx_en = true; > + break; > + default: > + return -ERANGE; > + } > + > + switch (config.rx_filter) { > + case HWTSTAMP_FILTER_NONE: > + if (priv->hwts_rx_en) > + priv->hwts_rx_en = false; > + break; > + case HWTSTAMP_FILTER_PTP_V2_L4_EVENT: > + case HWTSTAMP_FILTER_PTP_V2_L2_EVENT: > + case HWTSTAMP_FILTER_PTP_V2_L2_SYNC: > + case HWTSTAMP_FILTER_PTP_V2_L4_SYNC: > + case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ: > + case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ: > + case HWTSTAMP_FILTER_PTP_V2_EVENT: > + case HWTSTAMP_FILTER_PTP_V2_SYNC: > + case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ: > + config.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT; > + regval = macb_readl(priv, NCR); > + macb_writel(priv, NCR, (regval | MACB_BIT(SRTSM))); > + > + if (!priv->hwts_rx_en) > + priv->hwts_rx_en = true; > + break; > + default: > + config.rx_filter = HWTSTAMP_FILTER_NONE; > + return -ERANGE; > + } > + > + return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ? > + -EFAULT : 0; > +} > + > +static int gem_get_hwtst(struct net_device *netdev, > + struct ifreq *ifr) > +{ > + struct hwtstamp_config config; > + struct macb *priv = netdev_priv(netdev); > + > + config.flags = 0; > + config.tx_type = priv->hwts_tx_en ? HWTSTAMP_TX_ON : HWTSTAMP_TX_OFF; > + config.rx_filter = (priv->hwts_rx_en ? > + HWTSTAMP_FILTER_ALL : HWTSTAMP_FILTER_NONE); > + > + return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ? > + -EFAULT : 0; > +} > + > +static struct macb_ptp_info gem_ptp_info = { > + .ptp_init = gem_ptp_init, > + .ptp_remove = gem_ptp_remove, > + .get_ptp_max_adj = gem_get_ptp_max_adj, > + .get_tsu_rate = gem_get_tsu_rate, > + .get_ts_info = gem_get_ts_info, > + .get_hwtst = gem_get_hwtst, > + .set_hwtst = gem_set_hwtst, > +}; > +#endif > + > +static int macb_get_ts_info(struct net_device *netdev, > + struct ethtool_ts_info *info) > +{ > + struct macb *bp = netdev_priv(netdev); > + > + if (bp->ptp_info) > + return bp->ptp_info->get_ts_info(netdev, info); > + > + return ethtool_op_get_ts_info(netdev, info); } > + > static const struct ethtool_ops macb_ethtool_ops = { > .get_regs_len = macb_get_regs_len, > .get_regs = macb_get_regs, > @@ -2391,7 +2528,7 @@ static const struct ethtool_ops gem_ethtool_ops = { > .get_regs_len = macb_get_regs_len, > .get_regs = macb_get_regs, > .get_link = ethtool_op_get_link, > - .get_ts_info = ethtool_op_get_ts_info, > + .get_ts_info = macb_get_ts_info, > .get_ethtool_stats = gem_get_ethtool_stats, > .get_strings = gem_get_ethtool_strings, > .get_sset_count = gem_get_sset_count, > @@ -2404,6 +2541,7 @@ static const struct ethtool_ops gem_ethtool_ops = { static int macb_ioctl(struct net_device *dev, struct ifreq *rq, int cmd) { > struct phy_device *phydev = dev->phydev; > + struct macb *bp = netdev_priv(dev); > > if (!netif_running(dev)) > return -EINVAL; > @@ -2411,7 +2549,20 @@ static int macb_ioctl(struct net_device *dev, struct ifreq *rq, int cmd) > if (!phydev) > return -ENODEV; > > - return phy_mii_ioctl(phydev, rq, cmd); > + switch (cmd) { > + case SIOCSHWTSTAMP: > + if (bp->ptp_info) > + return bp->ptp_info->set_hwtst(dev, rq, cmd); > + > + return -EOPNOTSUPP; > + case SIOCGHWTSTAMP: > + if (bp->ptp_info) > + return bp->ptp_info->get_hwtst(dev, rq); > + > + return -EOPNOTSUPP; > + default: > + return phy_mii_ioctl(phydev, rq, cmd); > + } > } > > static int macb_set_features(struct net_device *netdev, @@ -2485,6 +2636,12 @@ static void macb_configure_caps(struct macb *bp, > dcfg = gem_readl(bp, DCFG2); > if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0) > bp->caps |= MACB_CAPS_FIFO_MODE; > + > + /* iff HWSTAMP is configure and gem has the capability */ #ifdef > +CONFIG_MACB_USE_HWSTAMP > + if (gem_has_ptp(bp)) > + bp->ptp_info = &gem_ptp_info; > +#endif > } > > dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps); @@ -3041,7 +3198,7 @@ static const struct macb_config pc302gem_config = { }; > > static const struct macb_config sama5d2_config = { > - .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII, > + .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII | MACB_CAPS_GEM_HAS_PTP, There are many IP cores with many configuration. If it is possible, capabilities should be read from IP directly. And it is possible in that case: Design Configuration Register 5 (0x290) bit 8: tsu There is now PTP hardware support without that bit. > .dma_burst_length = 16, > .clk_init = macb_clk_init, > .init = macb_init, > @@ -3049,14 +3206,15 @@ static const struct macb_config sama5d2_config = { > > static const struct macb_config sama5d3_config = { > .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE > - | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII, > + | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII > + | MACB_CAPS_GEM_HAS_PTP, > .dma_burst_length = 16, > .clk_init = macb_clk_init, > .init = macb_init, > }; > > static const struct macb_config sama5d4_config = { > - .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII, > + .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII | MACB_CAPS_GEM_HAS_PTP, > .dma_burst_length = 4, > .clk_init = macb_clk_init, > .init = macb_init, > -- > 2.7.4 In macb_start_xmit() there is also invoked skb_tx_timestamp() for software timestamping. I think, it should be disabled if you do hardware timestamping. Best regards, Rafal Ozieblo | Firmware System Engineer, www.cadence.com