Hi,
Hereby I submit a patchset that introduces various fixes and support
for new features and enhancements to the mvneta driver:
1. First three patches are minimal fixes, stable-CC'ed.
2. Suspend to ram ('s2ram') support. Due to some stability problems
Thomas Petazzoni's patches did not get merged yet, but I used them for
verification. Contrary to wfi mode ('standby' - linux does not
differentiate between them, so same routines are used) all registers'
contents are lost due to power down, so the configuration has to be
fully reconstructed during resume.
3. Optimisations - concatenating TX descriptors' flush, basing on
xmit_more support and combined approach for finalizing egress processing.
Thanks to HR timer buffers can be released with small latency, which is
good for low transfer and small queues. Along with the timer, coalescing
irqs are used, whose threshold could be increased back to 15.
4. Buffer manager (BM) support with two preparatory commits. As it is a
separate block, common for all network ports, a new driver is introduced,
which configures it and exposes API to the main network driver. It is
throughly described in binding documentation and commit log. Please note,
that enabling per-port BM usage is done using phandle and the data passed
in mvneta_bm_probe. It is designed for usage of on-demand device probe
and dev_set/get_drvdata, however it's awaiting merge to linux-next.
Therefore, deferring probe is not used - if something goes wrong (same
in case of errors during changing MTU or suspend/resume cycle) mvneta
driver falls back to software buffer management and works in a regular way.
Known issues:
- problems with obtaining all mapped buffers from internal SRAM, when
destroying the buffer pointer pool
- problems with unmapping chunk of SRAM during driver removal
Above do not have an impact on the operation, as they are called during
driver removal or in error path.
5. Enable BM on Armada XP and 38X development boards - those ones and
A370 I could check on my own. In all cases they survived night-long
linerate iperf. Also tests were performed with A388 SoC working as a
network bridge between two packet generators. They showed increase of
maximum processed 64B packets by ~20k (~555k packets with BM enabled
vs ~535 packets without BM). Also when pushing 1500B-packets with a
line rate achieved, CPU load decreased from around 25% without BM vs
18-20% with BM.
I'm looking forward to any remarks and comments.
Best regards,
Marcin Wojtas
Marcin Wojtas (12):
net: mvneta: add configuration for MBUS windows access protection
net: mvneta: enable IP checksum with jumbo frames for Armada 38x on
Port0
net: mvneta: fix bit assignment in MVNETA_RXQ_CONFIG_REG
net: mvneta: enable suspend/resume support
net: mvneta: enable mixed egress processing using HR timer
bus: mvebu-mbus: provide api for obtaining IO and DRAM window
information
ARM: mvebu: enable SRAM support in mvebu_v7_defconfig
net: mvneta: bm: add support for hardware buffer management
ARM: mvebu: add buffer manager nodes to armada-38x.dtsi
ARM: mvebu: enable buffer manager support on Armada 38x boards
ARM: mvebu: add buffer manager nodes to armada-xp.dtsi
ARM: mvebu: enable buffer manager support on Armada XP boards
Simon Guinot (1):
net: mvneta: add xmit_more support
.../bindings/net/marvell-armada-370-neta.txt | 19 +-
.../devicetree/bindings/net/marvell-neta-bm.txt | 49 ++
arch/arm/boot/dts/armada-385-db-ap.dts | 20 +-
arch/arm/boot/dts/armada-388-db.dts | 17 +-
arch/arm/boot/dts/armada-388-gp.dts | 17 +-
arch/arm/boot/dts/armada-38x.dtsi | 20 +-
arch/arm/boot/dts/armada-xp-db.dts | 19 +-
arch/arm/boot/dts/armada-xp-gp.dts | 19 +-
arch/arm/boot/dts/armada-xp.dtsi | 18 +
arch/arm/configs/mvebu_v7_defconfig | 1 +
drivers/bus/mvebu-mbus.c | 51 ++
drivers/net/ethernet/marvell/Kconfig | 14 +
drivers/net/ethernet/marvell/Makefile | 1 +
drivers/net/ethernet/marvell/mvneta.c | 660 +++++++++++++++++++--
drivers/net/ethernet/marvell/mvneta_bm.c | 642 ++++++++++++++++++++
drivers/net/ethernet/marvell/mvneta_bm.h | 171 ++++++
include/linux/mbus.h | 3 +
17 files changed, 1677 insertions(+), 64 deletions(-)
create mode 100644 Documentation/devicetree/bindings/net/marvell-neta-bm.txt
create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.c
create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.h
--
1.8.3.1
This commit adds missing configuration of MBUS windows access protection
in mvneta_conf_mbus_windows function - a dedicated variable for that
purpose remained there unused since v3.8 initial mvneta support. Because
of that the register contents were inherited from the bootloader.
Signed-off-by: Marcin Wojtas <[email protected]>
Cc: <[email protected]> # v3.8+
---
drivers/net/ethernet/marvell/mvneta.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index e84c7f2..0f30aaa 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -62,6 +62,7 @@
#define MVNETA_WIN_SIZE(w) (0x2204 + ((w) << 3))
#define MVNETA_WIN_REMAP(w) (0x2280 + ((w) << 2))
#define MVNETA_BASE_ADDR_ENABLE 0x2290
+#define MVNETA_ACCESS_PROTECT_ENABLE 0x2294
#define MVNETA_PORT_CONFIG 0x2400
#define MVNETA_UNI_PROMISC_MODE BIT(0)
#define MVNETA_DEF_RXQ(q) ((q) << 1)
@@ -3188,6 +3189,8 @@ static void mvneta_conf_mbus_windows(struct mvneta_port *pp,
win_enable &= ~(1 << i);
win_protect |= 3 << (2 * i);
+
+ mvreg_write(pp, MVNETA_ACCESS_PROTECT_ENABLE, win_protect);
}
mvreg_write(pp, MVNETA_BASE_ADDR_ENABLE, win_enable);
--
1.8.3.1
The Ethernet controller found in the Armada 38x SoC's family support
TCP/IP checksumming with frame sizes larger than 1600 bytes, however
only on port 0.
This commit enables this feature by using 'marvell,armada-xp-neta' in
'ethernet@70000' node.
Signed-off-by: Marcin Wojtas <[email protected]>
Cc: <[email protected]> # v3.18+
---
arch/arm/boot/dts/armada-38x.dtsi | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
index c6a0e9d..b7868b2 100644
--- a/arch/arm/boot/dts/armada-38x.dtsi
+++ b/arch/arm/boot/dts/armada-38x.dtsi
@@ -494,7 +494,7 @@
};
eth0: ethernet@70000 {
- compatible = "marvell,armada-370-neta";
+ compatible = "marvell,armada-xp-neta";
reg = <0x70000 0x4000>;
interrupts-extended = <&mpic 8>;
clocks = <&gateclk 4>;
--
1.8.3.1
MVNETA_RXQ_HW_BUF_ALLOC bit which controls enabling hardware buffer
allocation was mistakenly set as BIT(1). This commit fixes the assignment.
Signed-off-by: Marcin Wojtas <[email protected]>
Cc: <[email protected]> # v3.8+
---
drivers/net/ethernet/marvell/mvneta.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 0f30aaa..d12b8c6 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -36,7 +36,7 @@
/* Registers */
#define MVNETA_RXQ_CONFIG_REG(q) (0x1400 + ((q) << 2))
-#define MVNETA_RXQ_HW_BUF_ALLOC BIT(1)
+#define MVNETA_RXQ_HW_BUF_ALLOC BIT(0)
#define MVNETA_RXQ_PKT_OFFSET_ALL_MASK (0xf << 8)
#define MVNETA_RXQ_PKT_OFFSET_MASK(offs) ((offs) << 8)
#define MVNETA_RXQ_THRESHOLD_REG(q) (0x14c0 + ((q) << 2))
--
1.8.3.1
This commit introduces suspend/resume routines used for both in 'standby'
and 'mem' modes. For the latter, in which registers' contents are lost,
following steps are performed:
* in suspend - update port statistics and, if interface is running,
detach netif, clean the queues, disable cpu notifier, shutdown
interface and reset port's link status;
* in resume, for all interfaces, set default configuration of the port and
MBUS windows;
* in resume, in case the interface is running, enable accepting packets in
legacy parser, power up the port, register cpu notifier and attach netif.
Signed-off-by: Marcin Wojtas <[email protected]>
---
drivers/net/ethernet/marvell/mvneta.c | 70 +++++++++++++++++++++++++++++++++++
1 file changed, 70 insertions(+)
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index d12b8c6..f079b13 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -3442,6 +3442,72 @@ static int mvneta_remove(struct platform_device *pdev)
return 0;
}
+#ifdef CONFIG_PM_SLEEP
+static int mvneta_suspend(struct platform_device *pdev, pm_message_t state)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct mvneta_port *pp = netdev_priv(dev);
+
+ mvneta_ethtool_update_stats(pp);
+
+ if (!netif_running(dev))
+ return 0;
+
+ netif_device_detach(dev);
+
+ mvneta_stop_dev(pp);
+ unregister_cpu_notifier(&pp->cpu_notifier);
+ mvneta_cleanup_rxqs(pp);
+ mvneta_cleanup_txqs(pp);
+
+ /* Reset link status */
+ pp->link = 0;
+ pp->duplex = -1;
+ pp->speed = 0;
+
+ return 0;
+}
+
+static int mvneta_resume(struct platform_device *pdev)
+{
+ const struct mbus_dram_target_info *dram_target_info;
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct mvneta_port *pp = netdev_priv(dev);
+ int ret;
+
+ mvneta_defaults_set(pp);
+ mvneta_port_power_up(pp, pp->phy_interface);
+
+ dram_target_info = mv_mbus_dram_info();
+ if (dram_target_info)
+ mvneta_conf_mbus_windows(pp, dram_target_info);
+
+ if (!netif_running(dev))
+ return 0;
+
+ ret = mvneta_setup_rxqs(pp);
+ if (ret) {
+ netdev_err(dev, "unable to setup rxqs after resume\n");
+ return ret;
+ }
+
+ ret = mvneta_setup_txqs(pp);
+ if (ret) {
+ netdev_err(dev, "unable to setup txqs after resume\n");
+ return ret;
+ }
+
+ mvneta_set_rx_mode(dev);
+ mvneta_percpu_elect(pp);
+ register_cpu_notifier(&pp->cpu_notifier);
+ mvneta_start_dev(pp);
+
+ netif_device_attach(dev);
+
+ return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
static const struct of_device_id mvneta_match[] = {
{ .compatible = "marvell,armada-370-neta" },
{ .compatible = "marvell,armada-xp-neta" },
@@ -3452,6 +3518,10 @@ MODULE_DEVICE_TABLE(of, mvneta_match);
static struct platform_driver mvneta_driver = {
.probe = mvneta_probe,
.remove = mvneta_remove,
+#ifdef CONFIG_PM_SLEEP
+ .suspend = mvneta_suspend,
+ .resume = mvneta_resume,
+#endif
.driver = {
.name = MVNETA_DRIVER_NAME,
.of_match_table = mvneta_match,
--
1.8.3.1
From: Simon Guinot <[email protected]>
Basing on xmit_more flag of the skb, TX descriptors can be concatenated
before flushing. This commit delay Tx descriptor flush if the queue is
running and if there is more skb's to send.
Signed-off-by: Simon Guinot <[email protected]>
---
drivers/net/ethernet/marvell/mvneta.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index f079b13..9c9e858 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -467,6 +467,7 @@ struct mvneta_tx_queue {
* descriptor ring
*/
int count;
+ int pending;
int tx_stop_threshold;
int tx_wake_threshold;
@@ -751,8 +752,9 @@ static void mvneta_txq_pend_desc_add(struct mvneta_port *pp,
/* Only 255 descriptors can be added at once ; Assume caller
* process TX desriptors in quanta less than 256
*/
- val = pend_desc;
+ val = pend_desc + txq->pending;
mvreg_write(pp, MVNETA_TXQ_UPDATE_REG(txq->id), val);
+ txq->pending = 0;
}
/* Get pointer to next TX descriptor to be processed (send) by HW */
@@ -1857,11 +1859,14 @@ out:
struct netdev_queue *nq = netdev_get_tx_queue(dev, txq_id);
txq->count += frags;
- mvneta_txq_pend_desc_add(pp, txq, frags);
-
if (txq->count >= txq->tx_stop_threshold)
netif_tx_stop_queue(nq);
+ if (!skb->xmit_more || netif_xmit_stopped(nq))
+ mvneta_txq_pend_desc_add(pp, txq, frags);
+ else
+ txq->pending += frags;
+
u64_stats_update_begin(&stats->syncp);
stats->tx_packets++;
stats->tx_bytes += len;
--
1.8.3.1
Mixed approach allows using higher interrupt threshold (increased back to
15 packets), useful in high throughput. In case of small amount of data
or very short TX queues HR timer ensures releasing buffers with small
latency.
Along with existing tx_done processing by coalescing interrupts this
commit enables triggering HR timer each time the packets are sent.
Time threshold can also be configured, using ethtool.
Signed-off-by: Marcin Wojtas <[email protected]>
Signed-off-by: Simon Guinot <[email protected]>
---
drivers/net/ethernet/marvell/mvneta.c | 89 +++++++++++++++++++++++++++++++++--
1 file changed, 85 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 9c9e858..f5acaf6 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -21,6 +21,8 @@
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/if_vlan.h>
+#include <linux/hrtimer.h>
+#include <linux/ktime.h>
#include <net/ip.h>
#include <net/ipv6.h>
#include <linux/io.h>
@@ -226,7 +228,8 @@
/* Various constants */
/* Coalescing */
-#define MVNETA_TXDONE_COAL_PKTS 1
+#define MVNETA_TXDONE_COAL_PKTS 15
+#define MVNETA_TXDONE_COAL_USEC 100
#define MVNETA_RX_COAL_PKTS 32
#define MVNETA_RX_COAL_USEC 100
@@ -356,6 +359,11 @@ struct mvneta_port {
struct net_device *dev;
struct notifier_block cpu_notifier;
+ /* Egress finalization */
+ struct tasklet_struct tx_done_tasklet;
+ struct hrtimer tx_done_timer;
+ bool timer_scheduled;
+
/* Core clock */
struct clk *clk;
u8 mcast_count[256];
@@ -481,6 +489,7 @@ struct mvneta_tx_queue {
int txq_get_index;
u32 done_pkts_coal;
+ u32 done_time_coal;
/* Virtual address of the TX DMA descriptors array */
struct mvneta_tx_desc *descs;
@@ -1791,6 +1800,30 @@ error:
return -ENOMEM;
}
+/* Trigger HR timer for TX processing */
+static void mvneta_timer_set(struct mvneta_port *pp)
+{
+ ktime_t interval;
+
+ if (!pp->timer_scheduled) {
+ pp->timer_scheduled = true;
+ interval = ktime_set(0, pp->txqs[0].done_time_coal * 1000);
+ hrtimer_start(&pp->tx_done_timer, interval,
+ HRTIMER_MODE_REL_PINNED);
+ }
+}
+
+/* TX processing HR timer callback */
+static enum hrtimer_restart mvneta_hr_timer_cb(struct hrtimer *timer)
+{
+ struct mvneta_port *pp = container_of(timer, struct mvneta_port,
+ tx_done_timer);
+
+ tasklet_schedule(&pp->tx_done_tasklet);
+
+ return HRTIMER_NORESTART;
+}
+
/* Main tx processing */
static int mvneta_tx(struct sk_buff *skb, struct net_device *dev)
{
@@ -1862,10 +1895,13 @@ out:
if (txq->count >= txq->tx_stop_threshold)
netif_tx_stop_queue(nq);
- if (!skb->xmit_more || netif_xmit_stopped(nq))
+ if (!skb->xmit_more || netif_xmit_stopped(nq)) {
mvneta_txq_pend_desc_add(pp, txq, frags);
- else
+ if (txq->done_time_coal && !pp->timer_scheduled)
+ mvneta_timer_set(pp);
+ } else {
txq->pending += frags;
+ }
u64_stats_update_begin(&stats->syncp);
stats->tx_packets++;
@@ -1902,6 +1938,7 @@ static void mvneta_tx_done_gbe(struct mvneta_port *pp, u32 cause_tx_done)
{
struct mvneta_tx_queue *txq;
struct netdev_queue *nq;
+ unsigned int tx_todo = 0;
while (cause_tx_done) {
txq = mvneta_tx_done_policy(pp, cause_tx_done);
@@ -1909,12 +1946,40 @@ static void mvneta_tx_done_gbe(struct mvneta_port *pp, u32 cause_tx_done)
nq = netdev_get_tx_queue(pp->dev, txq->id);
__netif_tx_lock(nq, smp_processor_id());
- if (txq->count)
+ if (txq->count) {
mvneta_txq_done(pp, txq);
+ tx_todo += txq->count;
+ }
__netif_tx_unlock(nq);
cause_tx_done &= ~((1 << txq->id));
}
+
+ if (!pp->txqs[0].done_time_coal)
+ return;
+
+ /* Set the timer in case not all the packets were
+ * processed. Otherwise attempt to cancel timer.
+ */
+ if (tx_todo)
+ mvneta_timer_set(pp);
+ else if (pp->timer_scheduled)
+ hrtimer_cancel(&pp->tx_done_timer);
+}
+
+/* TX done processing tasklet */
+static void mvneta_tx_done_proc(unsigned long data)
+{
+ struct net_device *dev = (struct net_device *)data;
+ struct mvneta_port *pp = netdev_priv(dev);
+
+ if (!netif_running(dev))
+ return;
+
+ pp->timer_scheduled = false;
+
+ /* Process all Tx queues */
+ mvneta_tx_done_gbe(pp, (1 << txq_number) - 1);
}
/* Compute crc8 of the specified address, using a unique algorithm ,
@@ -2910,6 +2975,11 @@ static int mvneta_stop(struct net_device *dev)
for_each_present_cpu(cpu)
smp_call_function_single(cpu, mvneta_percpu_disable, pp, true);
free_percpu_irq(dev->irq, pp->ports);
+
+ hrtimer_cancel(&pp->tx_done_timer);
+ pp->timer_scheduled = false;
+ tasklet_kill(&pp->tx_done_tasklet);
+
mvneta_cleanup_rxqs(pp);
mvneta_cleanup_txqs(pp);
@@ -2967,6 +3037,7 @@ static int mvneta_ethtool_set_coalesce(struct net_device *dev,
for (queue = 0; queue < txq_number; queue++) {
struct mvneta_tx_queue *txq = &pp->txqs[queue];
+ txq->done_time_coal = c->tx_coalesce_usecs;
txq->done_pkts_coal = c->tx_max_coalesced_frames;
mvneta_tx_done_pkts_coal_set(pp, txq, txq->done_pkts_coal);
}
@@ -2983,6 +3054,7 @@ static int mvneta_ethtool_get_coalesce(struct net_device *dev,
c->rx_coalesce_usecs = pp->rxqs[0].time_coal;
c->rx_max_coalesced_frames = pp->rxqs[0].pkts_coal;
+ c->tx_coalesce_usecs = pp->txqs[0].done_time_coal;
c->tx_max_coalesced_frames = pp->txqs[0].done_pkts_coal;
return 0;
}
@@ -3146,6 +3218,7 @@ static int mvneta_init(struct device *dev, struct mvneta_port *pp)
txq->id = queue;
txq->size = pp->tx_ring_size;
txq->done_pkts_coal = MVNETA_TXDONE_COAL_PKTS;
+ txq->done_time_coal = MVNETA_TXDONE_COAL_USEC;
}
pp->rxqs = devm_kcalloc(dev, rxq_number, sizeof(struct mvneta_rx_queue),
@@ -3388,6 +3461,14 @@ static int mvneta_probe(struct platform_device *pdev)
port->pp = pp;
}
+ hrtimer_init(&pp->tx_done_timer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_PINNED);
+ pp->tx_done_timer.function = mvneta_hr_timer_cb;
+ pp->timer_scheduled = false;
+
+ tasklet_init(&pp->tx_done_tasklet, mvneta_tx_done_proc,
+ (unsigned long)dev);
+
dev->features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_TSO;
dev->hw_features |= dev->features;
dev->vlan_features |= dev->features;
--
1.8.3.1
This commit enables finding appropriate mbus window and obtaining its
target id and attribute for given physical address in two separate
routines, both for IO and DRAM windows. This functionality
is needed for Armada XP/38x Network Controller's Buffer Manager and
PnC configuration.
Signed-off-by: Marcin Wojtas <[email protected]>
[DRAM window information reference in LKv3.10]
Signed-off-by: Evan Wang <[email protected]>
---
drivers/bus/mvebu-mbus.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/mbus.h | 3 +++
2 files changed, 54 insertions(+)
diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index c43c3d2..3d1c0c3 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -948,6 +948,57 @@ void mvebu_mbus_get_pcie_io_aperture(struct resource *res)
*res = mbus_state.pcie_io_aperture;
}
+int mvebu_mbus_get_dram_win_info(phys_addr_t phyaddr, u8 *target, u8 *attr)
+{
+ const struct mbus_dram_target_info *dram;
+ int i;
+
+ /* Get dram info */
+ dram = mv_mbus_dram_info();
+ if (!dram) {
+ pr_err("missing DRAM information\n");
+ return -ENODEV;
+ }
+
+ /* Try to find matching DRAM window for phyaddr */
+ for (i = 0; i < dram->num_cs; i++) {
+ const struct mbus_dram_window *cs = dram->cs + i;
+
+ if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size)) {
+ *target = dram->mbus_dram_target_id;
+ *attr = cs->mbus_attr;
+ return 0;
+ }
+ }
+
+ pr_err("invalid dram address 0x%x\n", phyaddr);
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(mvebu_mbus_get_dram_win_info);
+
+int mvebu_mbus_get_io_win_info(phys_addr_t phyaddr, u32 *size, u8 *target,
+ u8 *attr)
+{
+ int win;
+
+ for (win = 0; win < mbus_state.soc->num_wins; win++) {
+ u64 wbase;
+ int enabled;
+
+ mvebu_mbus_read_window(&mbus_state, win, &enabled, &wbase,
+ size, target, attr, NULL);
+
+ if (!enabled)
+ continue;
+
+ if (wbase <= phyaddr && phyaddr <= wbase + *size)
+ return win;
+ }
+
+ return -EINVAL;
+}
+EXPORT_SYMBOL_GPL(mvebu_mbus_get_io_win_info);
+
static __init int mvebu_mbus_debugfs_init(void)
{
struct mvebu_mbus_state *s = &mbus_state;
diff --git a/include/linux/mbus.h b/include/linux/mbus.h
index 1f7bc63..ea34a86 100644
--- a/include/linux/mbus.h
+++ b/include/linux/mbus.h
@@ -69,6 +69,9 @@ static inline const struct mbus_dram_target_info *mv_mbus_dram_info_nooverlap(vo
int mvebu_mbus_save_cpu_target(u32 *store_addr);
void mvebu_mbus_get_pcie_mem_aperture(struct resource *res);
void mvebu_mbus_get_pcie_io_aperture(struct resource *res);
+int mvebu_mbus_get_dram_win_info(phys_addr_t phyaddr, u8 *target, u8 *attr);
+int mvebu_mbus_get_io_win_info(phys_addr_t phyaddr, u32 *size, u8 *target,
+ u8 *attr);
int mvebu_mbus_add_window_remap_by_id(unsigned int target,
unsigned int attribute,
phys_addr_t base, size_t size,
--
1.8.3.1
Signed-off-by: Marcin Wojtas <[email protected]>
---
arch/arm/configs/mvebu_v7_defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm/configs/mvebu_v7_defconfig b/arch/arm/configs/mvebu_v7_defconfig
index c6729bf..fe57e20 100644
--- a/arch/arm/configs/mvebu_v7_defconfig
+++ b/arch/arm/configs/mvebu_v7_defconfig
@@ -58,6 +58,7 @@ CONFIG_MTD_M25P80=y
CONFIG_MTD_NAND=y
CONFIG_MTD_NAND_PXA3xx=y
CONFIG_MTD_SPI_NOR=y
+CONFIG_SRAM=y
CONFIG_EEPROM_AT24=y
CONFIG_BLK_DEV_SD=y
CONFIG_ATA=y
--
1.8.3.1
Buffer manager (BM) is a dedicated hardware unit that can be used by all
ethernet ports of Armada XP and 38x SoC's. It allows to offload CPU on RX
path by sparing DRAM access on refilling buffer pool, hardware-based
filling of descriptor ring data and better memory utilization due to HW
arbitration for using 'short' pools for small packets.
Tests performed with A388 SoC working as a network bridge between two
packet generators showed increase of maximum processed 64B packets by
~20k (~555k packets with BM enabled vs ~535 packets without BM). Also
when pushing 1500B-packets with a line rate achieved, CPU load decreased
from around 25% without BM to 20% with BM.
BM comprise up to 4 buffer pointers' (BP) rings kept in DRAM, which
are called external BP pools - BPPE. Allocating and releasing buffer
pointers (BP) to/from BPPE is performed indirectly by write/read access
to a dedicated internal SRAM, where internal BP pools (BPPI) are placed.
BM hardware controls status of BPPE automatically, as well as assigning
proper buffers to RX descriptors. For more details please refer to
Functional Specification of Armada XP or 38x SoC.
In order to enable support for a separate hardware block, common for all
ports, a new driver has to be implemented ('mvneta_bm'). It provides
initialization sequence of address space, clocks, registers, SRAM,
empty pools' structures and also obtaining optional configuration
from DT (please refer to device tree binding documentation). mvneta_bm
exposes also a necessary API to mvneta driver, as well as a dedicated
structure with BM information (bm_priv), whose presence is used as a
flag notifying of BM usage by port. It has to be ensured that mvneta_bm
probe is executed prior to the ones in ports' driver. In case BM is not
used or its probe fails, mvneta falls back to use software buffer
management.
A sequence executed in mvneta_probe function is modified in order to have
an access to needed resources before possible port's BM initialization is
done. According to port-pools mapping provided by DT appropriate registers
are configured and the buffer pools are filled. RX path is modified
accordingly. Becaues the hardware allows a wide variety of configuration
options, following assumptions are made:
* using BM mechanisms can be selectively disabled/enabled basing
on DT configuration among the ports
* 'long' pool's single buffer size is tied to port's MTU
* using 'long' pool by port is obligatory and it cannot be shared
* using 'short' pool for smaller packets is optional
* one 'short' pool can be shared among all ports
This commit enables hardware buffer management operation cooperating with
existing mvneta driver. New device tree binding documentation is added and
the one of mvneta is updated accordingly.
Signed-off-by: Marcin Wojtas <[email protected]>
---
.../bindings/net/marvell-armada-370-neta.txt | 19 +-
.../devicetree/bindings/net/marvell-neta-bm.txt | 49 ++
drivers/net/ethernet/marvell/Kconfig | 14 +
drivers/net/ethernet/marvell/Makefile | 1 +
drivers/net/ethernet/marvell/mvneta.c | 493 ++++++++++++++--
drivers/net/ethernet/marvell/mvneta_bm.c | 642 +++++++++++++++++++++
drivers/net/ethernet/marvell/mvneta_bm.h | 171 ++++++
7 files changed, 1335 insertions(+), 54 deletions(-)
create mode 100644 Documentation/devicetree/bindings/net/marvell-neta-bm.txt
create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.c
create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.h
diff --git a/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt b/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt
index f5a8ca2..ea1aae2 100644
--- a/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt
+++ b/Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt
@@ -8,14 +8,29 @@ Required properties:
- phy-mode: See ethernet.txt file in the same directory
- clocks: a pointer to the reference clock for this device.
+Optional properties (valid only for Armada XP/38x):
+
+- buffer-manager: a phandle to a buffer manager node. Please refer to
+ Documentation/devicetree/bindings/net/marvell-neta-bm.txt
+- bm,pool-long: ID of a pool, that will accept all packets of a size
+ higher than 'short' pool's threshold (if set) and up to MTU value.
+ Obligatory, when the port is supposed to use hardware
+ buffer management.
+- bm,pool-short: ID of a pool, that will be used for accepting
+ packets of a size lower than given threshold. If not set, the port
+ will use a single 'long' pool for all packets, as defined above.
+
Example:
-ethernet@d0070000 {
+ethernet@70000 {
compatible = "marvell,armada-370-neta";
- reg = <0xd0070000 0x2500>;
+ reg = <0x70000 0x2500>;
interrupts = <8>;
clocks = <&gate_clk 4>;
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
+ bm,pool-short = <1>;
};
diff --git a/Documentation/devicetree/bindings/net/marvell-neta-bm.txt b/Documentation/devicetree/bindings/net/marvell-neta-bm.txt
new file mode 100644
index 0000000..c1b1d7c3
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/marvell-neta-bm.txt
@@ -0,0 +1,49 @@
+* Marvell Armada 380/XP Buffer Manager driver (BM)
+
+Required properties:
+
+- compatible: should be "marvell,armada-380-neta-bm".
+- reg: address and length of the register set for the device.
+- clocks: a pointer to the reference clock for this device.
+- internal-mem: a phandle to BM internal SRAM definition.
+
+Optional properties (port):
+
+- pool<0 : 3>,capacity: size of external buffer pointers' ring maintained
+ in DRAM. Can be set for each pool (id 0 : 3) separately. The value has
+ to be chosen between 128 and 16352 and it also has to be aligned to 32.
+ Otherwise the driver would adjust a given number or choose default if
+ not set.
+- pool<0 : 3>,pkt-size: maximum size of a packet accepted by a given buffer
+ pointers' pool (id 0 : 3). It will be taken into consideration only when pool
+ type is 'short'. For 'long' ones it would be overridden by port's MTU.
+ If not set a driver will choose a default value.
+
+In order to see how to hook the BM to a given ethernet port, please
+refer to Documentation/devicetree/bindings/net/marvell-armada-370-neta.txt.
+
+Example:
+
+- main node:
+
+bm: bm@c8000 {
+ compatible = "marvell,armada-380-neta-bm";
+ reg = <0xc8000 0xac>;
+ clocks = <&gateclk 13>;
+ internal-mem = <&bm_bppi>;
+ status = "okay";
+ pool2,capacity = <4096>;
+ pool1,pkt-size = <512>;
+};
+
+- internal SRAM node:
+
+bm_bppi: bm-bppi {
+ compatible = "mmio-sram";
+ reg = <MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ ranges = <0 MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ clocks = <&gateclk 13>;
+ status = "okay";
+};
diff --git a/drivers/net/ethernet/marvell/Kconfig b/drivers/net/ethernet/marvell/Kconfig
index a1c862b..6c8dc6d 100644
--- a/drivers/net/ethernet/marvell/Kconfig
+++ b/drivers/net/ethernet/marvell/Kconfig
@@ -40,11 +40,25 @@ config MVMDIO
This driver is used by the MV643XX_ETH and MVNETA drivers.
+config MVNETA_BM
+ tristate "Marvell Armada 38x/XP network interface BM support"
+ depends on MVNETA
+ ---help---
+ This driver supports auxiliary block of the network
+ interface units in the Marvell ARMADA XP and ARMADA 38x SoC
+ family, which is called buffer manager.
+
+ This driver, when enabled, strictly cooperates with mvneta
+ driver and is common for all network ports of the devices,
+ even for Armada 370 SoC, which doesn't support hardware
+ buffer management.
+
config MVNETA
tristate "Marvell Armada 370/38x/XP network interface support"
depends on PLAT_ORION
select MVMDIO
select FIXED_PHY
+ select MVNETA_BM
---help---
This driver supports the network interface units in the
Marvell ARMADA XP, ARMADA 370 and ARMADA 38x SoC family.
diff --git a/drivers/net/ethernet/marvell/Makefile b/drivers/net/ethernet/marvell/Makefile
index f6425bd..ff1bffa 100644
--- a/drivers/net/ethernet/marvell/Makefile
+++ b/drivers/net/ethernet/marvell/Makefile
@@ -4,6 +4,7 @@
obj-$(CONFIG_MVMDIO) += mvmdio.o
obj-$(CONFIG_MV643XX_ETH) += mv643xx_eth.o
+obj-$(CONFIG_MVNETA_BM) += mvneta_bm.o
obj-$(CONFIG_MVNETA) += mvneta.o
obj-$(CONFIG_MVPP2) += mvpp2.o
obj-$(CONFIG_PXA168_ETH) += pxa168_eth.o
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index f5acaf6..7b5d685 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -35,10 +35,15 @@
#include <linux/phy.h>
#include <linux/clk.h>
#include <linux/cpu.h>
+#include "mvneta_bm.h"
/* Registers */
#define MVNETA_RXQ_CONFIG_REG(q) (0x1400 + ((q) << 2))
#define MVNETA_RXQ_HW_BUF_ALLOC BIT(0)
+#define MVNETA_RXQ_SHORT_POOL_ID_SHIFT 4
+#define MVNETA_RXQ_SHORT_POOL_ID_MASK 0x30
+#define MVNETA_RXQ_LONG_POOL_ID_SHIFT 6
+#define MVNETA_RXQ_LONG_POOL_ID_MASK 0xc0
#define MVNETA_RXQ_PKT_OFFSET_ALL_MASK (0xf << 8)
#define MVNETA_RXQ_PKT_OFFSET_MASK(offs) ((offs) << 8)
#define MVNETA_RXQ_THRESHOLD_REG(q) (0x14c0 + ((q) << 2))
@@ -52,6 +57,9 @@
#define MVNETA_RXQ_STATUS_UPDATE_REG(q) (0x1500 + ((q) << 2))
#define MVNETA_RXQ_ADD_NON_OCCUPIED_SHIFT 16
#define MVNETA_RXQ_ADD_NON_OCCUPIED_MAX 255
+#define MVNETA_PORT_POOL_BUFFER_SZ_REG(pool) (0x1700 + ((pool) << 2))
+#define MVNETA_PORT_POOL_BUFFER_SZ_SHIFT 3
+#define MVNETA_PORT_POOL_BUFFER_SZ_MASK 0xfff8
#define MVNETA_PORT_RX_RESET 0x1cc0
#define MVNETA_PORT_RX_DMA_RESET BIT(0)
#define MVNETA_PHY_ADDR 0x2000
@@ -109,6 +117,7 @@
#define MVNETA_GMAC_CLOCK_DIVIDER 0x24f4
#define MVNETA_GMAC_1MS_CLOCK_ENABLE BIT(31)
#define MVNETA_ACC_MODE 0x2500
+#define MVNETA_BM_ADDRESS 0x2504
#define MVNETA_CPU_MAP(cpu) (0x2540 + ((cpu) << 2))
#define MVNETA_CPU_RXQ_ACCESS_ALL_MASK 0x000000ff
#define MVNETA_CPU_TXQ_ACCESS_ALL_MASK 0x0000ff00
@@ -247,7 +256,10 @@
#define MVNETA_CPU_D_CACHE_LINE_SIZE 32
#define MVNETA_TX_CSUM_MAX_SIZE 9800
-#define MVNETA_ACC_MODE_EXT 1
+#define MVNETA_ACC_MODE_EXT1 1
+#define MVNETA_ACC_MODE_EXT2 2
+
+#define MVNETA_MAX_DECODE_WIN 6
/* Timeout constants */
#define MVNETA_TX_DISABLE_TIMEOUT_MSEC 1000
@@ -282,7 +294,8 @@
((addr >= txq->tso_hdrs_phys) && \
(addr < txq->tso_hdrs_phys + txq->size * TSO_HEADER_SIZE))
-#define MVNETA_RX_BUF_SIZE(pkt_size) ((pkt_size) + NET_SKB_PAD)
+#define MVNETA_RX_GET_BM_POOL_ID(rxd) \
+ (((rxd)->status & MVNETA_RXD_BM_POOL_MASK) >> MVNETA_RXD_BM_POOL_SHIFT)
struct mvneta_statistic {
unsigned short offset;
@@ -348,6 +361,7 @@ struct mvneta_pcpu_port {
};
struct mvneta_port {
+ u8 id;
struct mvneta_pcpu_port __percpu *ports;
struct mvneta_pcpu_stats __percpu *stats;
@@ -380,6 +394,11 @@ struct mvneta_port {
unsigned int tx_csum_limit;
int use_inband_status:1;
+ struct mvneta_bm *bm_priv;
+ struct mvneta_bm_pool *pool_long;
+ struct mvneta_bm_pool *pool_short;
+ int bm_win_id;
+
u64 ethtool_stats[ARRAY_SIZE(mvneta_statistics)];
};
@@ -403,6 +422,8 @@ struct mvneta_port {
#define MVNETA_TX_L4_CSUM_NOT BIT(31)
#define MVNETA_RXD_ERR_CRC 0x0
+#define MVNETA_RXD_BM_POOL_SHIFT 13
+#define MVNETA_RXD_BM_POOL_MASK (BIT(13) | BIT(14))
#define MVNETA_RXD_ERR_SUMMARY BIT(16)
#define MVNETA_RXD_ERR_OVERRUN BIT(17)
#define MVNETA_RXD_ERR_LEN BIT(18)
@@ -813,6 +834,214 @@ static void mvneta_rxq_bm_disable(struct mvneta_port *pp,
mvreg_write(pp, MVNETA_RXQ_CONFIG_REG(rxq->id), val);
}
+/* Enable buffer management (BM) */
+static void mvneta_rxq_bm_enable(struct mvneta_port *pp,
+ struct mvneta_rx_queue *rxq)
+{
+ u32 val;
+
+ val = mvreg_read(pp, MVNETA_RXQ_CONFIG_REG(rxq->id));
+ val |= MVNETA_RXQ_HW_BUF_ALLOC;
+ mvreg_write(pp, MVNETA_RXQ_CONFIG_REG(rxq->id), val);
+}
+
+/* Notify HW about port's assignment of pool for bigger packets */
+static void mvneta_rxq_long_pool_set(struct mvneta_port *pp,
+ struct mvneta_rx_queue *rxq)
+{
+ u32 val;
+
+ val = mvreg_read(pp, MVNETA_RXQ_CONFIG_REG(rxq->id));
+ val &= ~MVNETA_RXQ_LONG_POOL_ID_MASK;
+ val |= (pp->pool_long->id << MVNETA_RXQ_LONG_POOL_ID_SHIFT);
+
+ mvreg_write(pp, MVNETA_RXQ_CONFIG_REG(rxq->id), val);
+}
+
+/* Notify HW about port's assignment of pool for smaller packets */
+static void mvneta_rxq_short_pool_set(struct mvneta_port *pp,
+ struct mvneta_rx_queue *rxq)
+{
+ u32 val;
+
+ val = mvreg_read(pp, MVNETA_RXQ_CONFIG_REG(rxq->id));
+ val &= ~MVNETA_RXQ_SHORT_POOL_ID_MASK;
+ val |= (pp->pool_short->id << MVNETA_RXQ_SHORT_POOL_ID_SHIFT);
+
+ mvreg_write(pp, MVNETA_RXQ_CONFIG_REG(rxq->id), val);
+}
+
+/* Set port's receive buffer size for assigned BM pool */
+static inline void mvneta_bm_pool_bufsize_set(struct mvneta_port *pp,
+ int buf_size,
+ u8 pool_id)
+{
+ u32 val;
+
+ if (!IS_ALIGNED(buf_size, 8)) {
+ dev_warn(pp->dev->dev.parent,
+ "illegal buf_size value %d, round to %d\n",
+ buf_size, ALIGN(buf_size, 8));
+ buf_size = ALIGN(buf_size, 8);
+ }
+
+ val = mvreg_read(pp, MVNETA_PORT_POOL_BUFFER_SZ_REG(pool_id));
+ val |= buf_size & MVNETA_PORT_POOL_BUFFER_SZ_MASK;
+ mvreg_write(pp, MVNETA_PORT_POOL_BUFFER_SZ_REG(pool_id), val);
+}
+
+/* Configure MBUS window in order to enable access BM internal SRAM */
+static int mvneta_mbus_io_win_set(struct mvneta_port *pp, u32 base, u32 wsize,
+ u8 target, u8 attr)
+{
+ u32 win_enable, win_protect;
+ int i;
+
+ win_enable = mvreg_read(pp, MVNETA_BASE_ADDR_ENABLE);
+
+ if (pp->bm_win_id < 0) {
+ /* Find first not occupied window */
+ for (i = 0; i < MVNETA_MAX_DECODE_WIN; i++) {
+ if (win_enable & (1 << i)) {
+ pp->bm_win_id = i;
+ break;
+ }
+ }
+ if (i == MVNETA_MAX_DECODE_WIN)
+ return -ENOMEM;
+ } else {
+ i = pp->bm_win_id;
+ }
+
+ mvreg_write(pp, MVNETA_WIN_BASE(i), 0);
+ mvreg_write(pp, MVNETA_WIN_SIZE(i), 0);
+
+ if (i < 4)
+ mvreg_write(pp, MVNETA_WIN_REMAP(i), 0);
+
+ mvreg_write(pp, MVNETA_WIN_BASE(i), (base & 0xffff0000) |
+ (attr << 8) | target);
+
+ mvreg_write(pp, MVNETA_WIN_SIZE(i), (wsize - 1) & 0xffff0000);
+
+ win_protect = mvreg_read(pp, MVNETA_ACCESS_PROTECT_ENABLE);
+ win_protect |= 3 << (2 * i);
+ mvreg_write(pp, MVNETA_ACCESS_PROTECT_ENABLE, win_protect);
+
+ win_enable &= ~(1 << i);
+ mvreg_write(pp, MVNETA_BASE_ADDR_ENABLE, win_enable);
+
+ return 0;
+}
+
+/* Assign and initialize pools for port. In case of fail
+ * buffer manager will remain disabled for current port.
+ */
+static int mvneta_bm_port_init(struct platform_device *pdev,
+ struct mvneta_port *pp)
+{
+ struct device_node *dn = pdev->dev.of_node;
+ u32 long_pool_id, short_pool_id, wsize;
+ u8 target, attr;
+ int err;
+
+ /* Get BM window information */
+ err = mvebu_mbus_get_io_win_info(pp->bm_priv->bppi_phys_addr, &wsize,
+ &target, &attr);
+ if (err < 0)
+ return err;
+
+ pp->bm_win_id = -1;
+
+ /* Open NETA -> BM window */
+ err = mvneta_mbus_io_win_set(pp, pp->bm_priv->bppi_phys_addr, wsize,
+ target, attr);
+ if (err < 0) {
+ netdev_info(pp->dev, "fail to configure mbus window to BM\n");
+ return err;
+ }
+
+ if (of_property_read_u32(dn, "bm,pool-long", &long_pool_id)) {
+ netdev_info(pp->dev, "missing long pool id\n");
+ return -EINVAL;
+ }
+
+ /* Create port's long pool depending on mtu */
+ pp->pool_long = mvneta_bm_pool_use(pp->bm_priv, long_pool_id,
+ MVNETA_BM_LONG, pp->id,
+ MVNETA_RX_PKT_SIZE(pp->dev->mtu));
+ if (!pp->pool_long) {
+ netdev_info(pp->dev, "fail to obtain long pool for port\n");
+ return -ENOMEM;
+ }
+
+ pp->pool_long->port_map |= 1 << pp->id;
+
+ mvneta_bm_pool_bufsize_set(pp, pp->pool_long->buf_size,
+ pp->pool_long->id);
+
+ /* If short pool id is not defined, assume using single pool */
+ if (of_property_read_u32(dn, "bm,pool-short", &short_pool_id))
+ short_pool_id = long_pool_id;
+
+ /* Create port's short pool */
+ pp->pool_short = mvneta_bm_pool_use(pp->bm_priv, short_pool_id,
+ MVNETA_BM_SHORT, pp->id,
+ MVNETA_BM_SHORT_PKT_SIZE);
+ if (!pp->pool_short) {
+ netdev_info(pp->dev, "fail to obtain short pool for port\n");
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ return -ENOMEM;
+ }
+
+ if (short_pool_id != long_pool_id) {
+ pp->pool_short->port_map |= 1 << pp->id;
+ mvneta_bm_pool_bufsize_set(pp, pp->pool_short->buf_size,
+ pp->pool_short->id);
+ }
+
+ return 0;
+}
+
+/* Update settings of a pool for bigger packets */
+static void mvneta_bm_update_mtu(struct mvneta_port *pp, int mtu)
+{
+ struct mvneta_bm_pool *bm_pool = pp->pool_long;
+ int num;
+
+ /* Release all buffers from long pool */
+ mvneta_bm_bufs_free(pp->bm_priv, bm_pool, 1 << pp->id);
+ if (bm_pool->buf_num) {
+ WARN(1, "cannot free all buffers in pool %d\n",
+ bm_pool->id);
+ goto bm_mtu_err;
+ }
+
+ bm_pool->pkt_size = MVNETA_RX_PKT_SIZE(mtu);
+ bm_pool->buf_size = MVNETA_RX_BUF_SIZE(bm_pool->pkt_size);
+ bm_pool->frag_size = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
+ SKB_DATA_ALIGN(MVNETA_RX_BUF_SIZE(bm_pool->pkt_size));
+
+ /* Fill entire long pool */
+ num = mvneta_bm_bufs_add(pp->bm_priv, bm_pool, bm_pool->size);
+ if (num != bm_pool->size) {
+ WARN(1, "pool %d: %d of %d allocated\n",
+ bm_pool->id, num, bm_pool->size);
+ goto bm_mtu_err;
+ }
+ mvneta_bm_pool_bufsize_set(pp, bm_pool->buf_size, bm_pool->id);
+
+ return;
+
+bm_mtu_err:
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_short, 1 << pp->id);
+
+ pp->bm_priv = NULL;
+ mvreg_write(pp, MVNETA_ACC_MODE, MVNETA_ACC_MODE_EXT1);
+ netdev_info(pp->dev, "fail to update MTU, fall back to software BM\n");
+}
+
/* Start the Ethernet port RX and TX activity */
static void mvneta_port_up(struct mvneta_port *pp)
{
@@ -1035,9 +1264,17 @@ static void mvneta_defaults_set(struct mvneta_port *pp)
mvreg_write(pp, MVNETA_PORT_RX_RESET, 0);
/* Set Port Acceleration Mode */
- val = MVNETA_ACC_MODE_EXT;
+ if (pp->bm_priv)
+ /* HW buffer management + legacy parser */
+ val = MVNETA_ACC_MODE_EXT2;
+ else
+ /* SW buffer management + legacy parser */
+ val = MVNETA_ACC_MODE_EXT1;
mvreg_write(pp, MVNETA_ACC_MODE, val);
+ if (pp->bm_priv)
+ mvreg_write(pp, MVNETA_BM_ADDRESS, pp->bm_priv->bppi_phys_addr);
+
/* Update val of portCfg register accordingly with all RxQueue types */
val = MVNETA_PORT_CONFIG_DEFL_VALUE(rxq_def);
mvreg_write(pp, MVNETA_PORT_CONFIG, val);
@@ -1423,23 +1660,7 @@ static void mvneta_txq_done(struct mvneta_port *pp,
}
}
-static void *mvneta_frag_alloc(const struct mvneta_port *pp)
-{
- if (likely(pp->frag_size <= PAGE_SIZE))
- return netdev_alloc_frag(pp->frag_size);
- else
- return kmalloc(pp->frag_size, GFP_ATOMIC);
-}
-
-static void mvneta_frag_free(const struct mvneta_port *pp, void *data)
-{
- if (likely(pp->frag_size <= PAGE_SIZE))
- skb_free_frag(data);
- else
- kfree(data);
-}
-
-/* Refill processing */
+/* Refill processing for SW buffer management */
static int mvneta_rx_refill(struct mvneta_port *pp,
struct mvneta_rx_desc *rx_desc)
@@ -1447,7 +1668,7 @@ static int mvneta_rx_refill(struct mvneta_port *pp,
dma_addr_t phys_addr;
void *data;
- data = mvneta_frag_alloc(pp);
+ data = mvneta_frag_alloc(pp->frag_size);
if (!data)
return -ENOMEM;
@@ -1455,7 +1676,7 @@ static int mvneta_rx_refill(struct mvneta_port *pp,
MVNETA_RX_BUF_SIZE(pp->pkt_size),
DMA_FROM_DEVICE);
if (unlikely(dma_mapping_error(pp->dev->dev.parent, phys_addr))) {
- mvneta_frag_free(pp, data);
+ mvneta_frag_free(pp->frag_size, data);
return -ENOMEM;
}
@@ -1501,17 +1722,20 @@ static void mvneta_rxq_drop_pkts(struct mvneta_port *pp,
int rx_done, i;
rx_done = mvneta_rxq_busy_desc_num_get(pp, rxq);
+ if (rx_done)
+ mvneta_rxq_desc_num_update(pp, rxq, rx_done, rx_done);
+
+ if (pp->bm_priv)
+ return;
+
for (i = 0; i < rxq->size; i++) {
struct mvneta_rx_desc *rx_desc = rxq->descs + i;
void *data = (void *)rx_desc->buf_cookie;
dma_unmap_single(pp->dev->dev.parent, rx_desc->buf_phys_addr,
MVNETA_RX_BUF_SIZE(pp->pkt_size), DMA_FROM_DEVICE);
- mvneta_frag_free(pp, data);
+ mvneta_frag_free(pp->frag_size, data);
}
-
- if (rx_done)
- mvneta_rxq_desc_num_update(pp, rxq, rx_done, rx_done);
}
/* Main rx processing */
@@ -1523,6 +1747,7 @@ static int mvneta_rx(struct mvneta_port *pp, int rx_todo,
int rx_done;
u32 rcvd_pkts = 0;
u32 rcvd_bytes = 0;
+ bool bm_in_use;
/* Get number of received packets */
rx_done = mvneta_rxq_busy_desc_num_get(pp, rxq);
@@ -1532,23 +1757,35 @@ static int mvneta_rx(struct mvneta_port *pp, int rx_todo,
rx_done = 0;
+ bm_in_use = pp->bm_priv ? true : false;
+
/* Fairness NAPI loop */
while (rx_done < rx_todo) {
struct mvneta_rx_desc *rx_desc = mvneta_rxq_next_desc_get(rxq);
+ struct mvneta_bm_pool *bm_pool = NULL;
struct sk_buff *skb;
unsigned char *data;
dma_addr_t phys_addr;
- u32 rx_status;
+ u32 rx_status, frag_size;
int rx_bytes, err;
+ u8 pool_id;
rx_done++;
rx_status = rx_desc->status;
rx_bytes = rx_desc->data_size - (ETH_FCS_LEN + MVNETA_MH_SIZE);
data = (unsigned char *)rx_desc->buf_cookie;
phys_addr = rx_desc->buf_phys_addr;
+ if (bm_in_use) {
+ pool_id = MVNETA_RX_GET_BM_POOL_ID(rx_desc);
+ bm_pool = &pp->bm_priv->bm_pools[pool_id];
+ }
if (!mvneta_rxq_desc_is_first_last(rx_status) ||
(rx_status & MVNETA_RXD_ERR_SUMMARY)) {
+ /* Return the buffer to the pool */
+ if (bm_in_use)
+ mvneta_bm_pool_put_bp(pp->bm_priv, bm_pool,
+ rx_desc->buf_phys_addr);
err_drop_frame:
dev->stats.rx_errors++;
mvneta_rx_error(pp, rx_desc);
@@ -1578,24 +1815,37 @@ static int mvneta_rx(struct mvneta_port *pp, int rx_todo,
rcvd_pkts++;
rcvd_bytes += rx_bytes;
+ /* Return the buffer to the pool */
+ if (bm_in_use)
+ mvneta_bm_pool_put_bp(pp->bm_priv, bm_pool,
+ rx_desc->buf_phys_addr);
+
/* leave the descriptor and buffer untouched */
continue;
}
/* Refill processing */
- err = mvneta_rx_refill(pp, rx_desc);
+ err = bm_in_use ? mvneta_bm_pool_refill(pp->bm_priv, bm_pool) :
+ mvneta_rx_refill(pp, rx_desc);
if (err) {
netdev_err(dev, "Linux processing - Can't refill\n");
rxq->missed++;
goto err_drop_frame;
}
- skb = build_skb(data, pp->frag_size > PAGE_SIZE ? 0 : pp->frag_size);
+ frag_size = bm_in_use ? bm_pool->frag_size : pp->frag_size;
+
+ skb = build_skb(data, frag_size > PAGE_SIZE ? 0 : frag_size);
if (!skb)
goto err_drop_frame;
- dma_unmap_single(dev->dev.parent, phys_addr,
- MVNETA_RX_BUF_SIZE(pp->pkt_size), DMA_FROM_DEVICE);
+ if (bm_in_use)
+ dma_unmap_single(&pp->bm_priv->pdev->dev, phys_addr,
+ bm_pool->buf_size, DMA_FROM_DEVICE);
+ else
+ dma_unmap_single(dev->dev.parent, phys_addr,
+ MVNETA_RX_BUF_SIZE(pp->pkt_size),
+ DMA_FROM_DEVICE);
rcvd_pkts++;
rcvd_bytes += rx_bytes;
@@ -2338,9 +2588,17 @@ static int mvneta_rxq_init(struct mvneta_port *pp,
mvneta_rx_pkts_coal_set(pp, rxq, rxq->pkts_coal);
mvneta_rx_time_coal_set(pp, rxq, rxq->time_coal);
- /* Fill RXQ with buffers from RX pool */
- mvneta_rxq_buf_size_set(pp, rxq, MVNETA_RX_BUF_SIZE(pp->pkt_size));
- mvneta_rxq_bm_disable(pp, rxq);
+ if (!pp->bm_priv) {
+ /* Fill RXQ with buffers from RX pool */
+ mvneta_rxq_buf_size_set(pp, rxq,
+ MVNETA_RX_BUF_SIZE(pp->pkt_size));
+ mvneta_rxq_bm_disable(pp, rxq);
+ } else {
+ mvneta_rxq_bm_enable(pp, rxq);
+ mvneta_rxq_long_pool_set(pp, rxq);
+ mvneta_rxq_short_pool_set(pp, rxq);
+ }
+
mvneta_rxq_fill(pp, rxq, rxq->size);
return 0;
@@ -2600,6 +2858,9 @@ static int mvneta_change_mtu(struct net_device *dev, int mtu)
dev->mtu = mtu;
if (!netif_running(dev)) {
+ if (pp->bm_priv)
+ mvneta_bm_update_mtu(pp, mtu);
+
netdev_update_features(dev);
return 0;
}
@@ -2612,6 +2873,9 @@ static int mvneta_change_mtu(struct net_device *dev, int mtu)
mvneta_cleanup_txqs(pp);
mvneta_cleanup_rxqs(pp);
+ if (pp->bm_priv)
+ mvneta_bm_update_mtu(pp, mtu);
+
pp->pkt_size = MVNETA_RX_PKT_SIZE(dev->mtu);
pp->frag_size = SKB_DATA_ALIGN(MVNETA_RX_BUF_SIZE(pp->pkt_size)) +
SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
@@ -3325,6 +3589,7 @@ static int mvneta_probe(struct platform_device *pdev)
struct resource *res;
struct device_node *dn = pdev->dev.of_node;
struct device_node *phy_node;
+ struct device_node *bm_node;
struct mvneta_port *pp;
struct net_device *dev;
const char *dt_mac_addr;
@@ -3434,26 +3699,51 @@ static int mvneta_probe(struct platform_device *pdev)
if (of_device_is_compatible(dn, "marvell,armada-370-neta"))
pp->tx_csum_limit = 1600;
+ dram_target_info = mv_mbus_dram_info();
+ if (dram_target_info)
+ mvneta_conf_mbus_windows(pp, dram_target_info);
+
pp->tx_ring_size = MVNETA_MAX_TXD;
pp->rx_ring_size = MVNETA_MAX_RXD;
pp->dev = dev;
SET_NETDEV_DEV(dev, &pdev->dev);
+ dev->features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_TSO;
+ dev->hw_features |= dev->features;
+ dev->vlan_features |= dev->features;
+ dev->priv_flags |= IFF_UNICAST_FLT;
+ dev->gso_max_segs = MVNETA_MAX_TSO_SEGS;
+
+ err = register_netdev(dev);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to register\n");
+ goto err_free_stats;
+ }
+
+ pp->id = dev->ifindex;
+
+ /* Obtain access to BM resources if enabled and already initialized */
+ bm_node = of_parse_phandle(dn, "buffer-manager", 0);
+ if (bm_node && bm_node->data) {
+ pp->bm_priv = bm_node->data;
+ err = mvneta_bm_port_init(pdev, pp);
+ if (err < 0) {
+ dev_info(&pdev->dev, "use SW buffer management\n");
+ pp->bm_priv = NULL;
+ }
+ }
+
err = mvneta_init(&pdev->dev, pp);
if (err < 0)
- goto err_free_stats;
+ goto err_netdev;
err = mvneta_port_power_up(pp, phy_mode);
if (err < 0) {
dev_err(&pdev->dev, "can't power up port\n");
- goto err_free_stats;
+ goto err_netdev;
}
- dram_target_info = mv_mbus_dram_info();
- if (dram_target_info)
- mvneta_conf_mbus_windows(pp, dram_target_info);
-
for_each_present_cpu(cpu) {
struct mvneta_pcpu_port *port = per_cpu_ptr(pp->ports, cpu);
@@ -3469,18 +3759,6 @@ static int mvneta_probe(struct platform_device *pdev)
tasklet_init(&pp->tx_done_tasklet, mvneta_tx_done_proc,
(unsigned long)dev);
- dev->features = NETIF_F_SG | NETIF_F_IP_CSUM | NETIF_F_TSO;
- dev->hw_features |= dev->features;
- dev->vlan_features |= dev->features;
- dev->priv_flags |= IFF_UNICAST_FLT;
- dev->gso_max_segs = MVNETA_MAX_TSO_SEGS;
-
- err = register_netdev(dev);
- if (err < 0) {
- dev_err(&pdev->dev, "failed to register\n");
- goto err_free_stats;
- }
-
netdev_info(dev, "Using %s mac address %pM\n", mac_from,
dev->dev_addr);
@@ -3496,6 +3774,13 @@ static int mvneta_probe(struct platform_device *pdev)
return 0;
+err_netdev:
+ unregister_netdev(dev);
+ if (pp->bm_priv) {
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_short,
+ 1 << pp->id);
+ }
err_free_stats:
free_percpu(pp->stats);
err_free_ports:
@@ -3525,10 +3810,48 @@ static int mvneta_remove(struct platform_device *pdev)
of_node_put(pp->phy_node);
free_netdev(dev);
+ if (pp->bm_priv) {
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_short,
+ 1 << pp->id);
+ }
+
return 0;
}
#ifdef CONFIG_PM_SLEEP
+static void mvneta_bm_port_suspend(struct mvneta_port *pp)
+{
+ /* Release all buffers from long pool */
+ mvneta_bm_bufs_free(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ if (pp->pool_long->buf_num) {
+ WARN(1, "cannot free all buffers in pool %d\n",
+ pp->pool_long->id);
+ goto bm_suspend_err;
+ }
+
+ if (pp->pool_short->id == pp->pool_long->id)
+ return;
+
+ /* Release all buffers from short pool in case it is not shared */
+ mvneta_bm_bufs_free(pp->bm_priv, pp->pool_short, 1 << pp->id);
+ if (pp->pool_short->buf_num && !pp->pool_short->port_map) {
+ WARN(1, "cannot free all buffers in pool %d\n",
+ pp->pool_short->id);
+ goto bm_suspend_err;
+ }
+
+ return;
+
+bm_suspend_err:
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_short, 1 << pp->id);
+
+ pp->bm_priv = NULL;
+ mvreg_write(pp, MVNETA_ACC_MODE, MVNETA_ACC_MODE_EXT1);
+ netdev_info(pp->dev, "fail to suspend, fall back to software BM\n");
+}
+
static int mvneta_suspend(struct platform_device *pdev, pm_message_t state)
{
struct net_device *dev = platform_get_drvdata(pdev);
@@ -3536,6 +3859,9 @@ static int mvneta_suspend(struct platform_device *pdev, pm_message_t state)
mvneta_ethtool_update_stats(pp);
+ if (pp->bm_priv)
+ mvneta_bm_port_suspend(pp);
+
if (!netif_running(dev))
return 0;
@@ -3554,6 +3880,61 @@ static int mvneta_suspend(struct platform_device *pdev, pm_message_t state)
return 0;
}
+static void mvneta_bm_port_resume(struct mvneta_port *pp)
+{
+ struct mvneta_bm *priv = pp->bm_priv;
+ u8 target, attr;
+ int ret, num;
+ u32 wsize;
+
+ /* Get BM window information */
+ ret = mvebu_mbus_get_io_win_info(priv->bppi_phys_addr, &wsize,
+ &target, &attr);
+ if (ret < 0)
+ goto bm_resume_err;
+
+ /* Open NETA -> BM window */
+ ret = mvneta_mbus_io_win_set(pp, priv->bppi_phys_addr, wsize,
+ target, attr);
+ if (ret < 0) {
+ netdev_info(pp->dev, "fail to configure mbus window to BM\n");
+ goto bm_resume_err;
+ }
+
+ /* Fill entire long pool */
+ num = mvneta_bm_bufs_add(priv, pp->pool_long, pp->pool_long->size);
+ if (num != pp->pool_long->size) {
+ WARN(1, "pool %d: %d of %d allocated\n",
+ pp->pool_long->id, num, pp->pool_long->size);
+ goto bm_resume_err;
+ }
+ mvneta_bm_pool_bufsize_set(pp, pp->pool_long->buf_size,
+ pp->pool_long->id);
+
+ if (pp->pool_short->id == pp->pool_long->id)
+ return;
+
+ /* Fill entire short pool */
+ num = mvneta_bm_bufs_add(priv, pp->pool_short, pp->pool_short->size);
+ if (num != pp->pool_short->size) {
+ WARN(1, "pool %d: %d of %d allocated\n",
+ pp->pool_short->id, num, pp->pool_short->size);
+ goto bm_resume_err;
+ }
+ mvneta_bm_pool_bufsize_set(pp, pp->pool_short->buf_size,
+ pp->pool_short->id);
+
+ return;
+
+bm_resume_err:
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_long, 1 << pp->id);
+ mvneta_bm_pool_destroy(pp->bm_priv, pp->pool_short, 1 << pp->id);
+
+ pp->bm_priv = NULL;
+ mvreg_write(pp, MVNETA_ACC_MODE, MVNETA_ACC_MODE_EXT1);
+ netdev_info(pp->dev, "fail to resume, fall back to software BM\n");
+}
+
static int mvneta_resume(struct platform_device *pdev)
{
const struct mbus_dram_target_info *dram_target_info;
@@ -3568,6 +3949,14 @@ static int mvneta_resume(struct platform_device *pdev)
if (dram_target_info)
mvneta_conf_mbus_windows(pp, dram_target_info);
+ if (pp->bm_priv) {
+ /* Ensure BM is resumed prior to port re-configuration */
+ if (!mvneta_bm_active(pp->bm_priv))
+ mvneta_bm_resume(pp->bm_priv->pdev);
+
+ mvneta_bm_port_resume(pp);
+ }
+
if (!netif_running(dev))
return 0;
diff --git a/drivers/net/ethernet/marvell/mvneta_bm.c b/drivers/net/ethernet/marvell/mvneta_bm.c
new file mode 100644
index 0000000..5e8ee0b
--- /dev/null
+++ b/drivers/net/ethernet/marvell/mvneta_bm.c
@@ -0,0 +1,642 @@
+/*
+ * Driver for Marvell NETA network controller Buffer Manager.
+ *
+ * Copyright (C) 2015 Marvell
+ *
+ * Marcin Wojtas <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#include <linux/kernel.h>
+#include <linux/genalloc.h>
+#include <linux/platform_device.h>
+#include <linux/netdevice.h>
+#include <linux/skbuff.h>
+#include <linux/mbus.h>
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/clk.h>
+#include "mvneta_bm.h"
+
+#define MVNETA_BM_DRIVER_NAME "mvneta_bm"
+#define MVNETA_BM_DRIVER_VERSION "1.0"
+
+static void mvneta_bm_write(struct mvneta_bm *priv, u32 offset, u32 data)
+{
+ writel(data, priv->reg_base + offset);
+}
+
+static u32 mvneta_bm_read(struct mvneta_bm *priv, u32 offset)
+{
+ return readl(priv->reg_base + offset);
+}
+
+static void mvneta_bm_pool_enable(struct mvneta_bm *priv, int pool_id)
+{
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_POOL_BASE_REG(pool_id));
+ val |= MVNETA_BM_POOL_ENABLE_MASK;
+ mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(pool_id), val);
+
+ /* Clear BM cause register */
+ mvneta_bm_write(priv, MVNETA_BM_INTR_CAUSE_REG, 0);
+}
+
+static void mvneta_bm_pool_disable(struct mvneta_bm *priv, int pool_id)
+{
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_POOL_BASE_REG(pool_id));
+ val &= ~MVNETA_BM_POOL_ENABLE_MASK;
+ mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(pool_id), val);
+}
+
+static inline void mvneta_bm_config_set(struct mvneta_bm *priv, u32 mask)
+{
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
+ val |= mask;
+ mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
+}
+
+static inline void mvneta_bm_config_clear(struct mvneta_bm *priv, u32 mask)
+{
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
+ val &= ~mask;
+ mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
+}
+
+static void mvneta_bm_pool_target_set(struct mvneta_bm *priv, int pool_id,
+ u8 target_id, u8 attr)
+{
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_XBAR_POOL_REG(pool_id));
+ val &= ~MVNETA_BM_TARGET_ID_MASK(pool_id);
+ val &= ~MVNETA_BM_XBAR_ATTR_MASK(pool_id);
+ val |= MVNETA_BM_TARGET_ID_VAL(pool_id, target_id);
+ val |= MVNETA_BM_XBAR_ATTR_VAL(pool_id, attr);
+
+ mvneta_bm_write(priv, MVNETA_BM_XBAR_POOL_REG(pool_id), val);
+}
+
+void *mvneta_frag_alloc(unsigned int frag_size)
+{
+ if (likely(frag_size <= PAGE_SIZE))
+ return netdev_alloc_frag(frag_size);
+ else
+ return kmalloc(frag_size, GFP_ATOMIC);
+}
+EXPORT_SYMBOL_GPL(mvneta_frag_alloc);
+
+void mvneta_frag_free(unsigned int frag_size, void *data)
+{
+ if (likely(frag_size <= PAGE_SIZE))
+ skb_free_frag(data);
+ else
+ kfree(data);
+}
+EXPORT_SYMBOL_GPL(mvneta_frag_free);
+
+/* Allocate skb for BM pool */
+void *mvneta_buf_alloc(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
+ dma_addr_t *buf_phys_addr)
+{
+ void *buf;
+ dma_addr_t phys_addr;
+
+ buf = mvneta_frag_alloc(bm_pool->frag_size);
+ if (!buf)
+ return NULL;
+
+ /* In order to update buf_cookie field of RX descriptor properly,
+ * BM hardware expects buf virtual address to be placed in the
+ * first four bytes of mapped buffer.
+ */
+ *(u32 *)buf = (u32)buf;
+ phys_addr = dma_map_single(&priv->pdev->dev, buf, bm_pool->buf_size,
+ DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(&priv->pdev->dev, phys_addr))) {
+ mvneta_frag_free(bm_pool->frag_size, buf);
+ return NULL;
+ }
+ *buf_phys_addr = phys_addr;
+
+ return buf;
+}
+
+/* Refill processing for HW buffer management */
+int mvneta_bm_pool_refill(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool)
+{
+ dma_addr_t buf_phys_addr;
+ void *buf;
+
+ buf = mvneta_buf_alloc(priv, bm_pool, &buf_phys_addr);
+ if (!buf)
+ return -ENOMEM;
+
+ mvneta_bm_pool_put_bp(priv, bm_pool, buf_phys_addr);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mvneta_bm_pool_refill);
+
+/* Allocate buffers for the pool */
+int mvneta_bm_bufs_add(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
+ int buf_num)
+{
+ int err, i;
+
+ if (bm_pool->buf_num == bm_pool->size) {
+ dev_dbg(&priv->pdev->dev, "pool %d already filled\n",
+ bm_pool->id);
+ return bm_pool->buf_num;
+ }
+
+ if (buf_num < 0 ||
+ (buf_num + bm_pool->buf_num > bm_pool->size)) {
+ dev_err(&priv->pdev->dev,
+ "cannot allocate %d buffers for pool %d\n",
+ buf_num, bm_pool->id);
+ return 0;
+ }
+
+ for (i = 0; i < buf_num; i++) {
+ err = mvneta_bm_pool_refill(priv, bm_pool);
+ if (err < 0)
+ break;
+ }
+
+ /* Update BM driver with number of buffers added to pool */
+ bm_pool->buf_num += i;
+
+ dev_dbg(&priv->pdev->dev,
+ "%s pool %d: pkt_size=%4d, buf_size=%4d, frag_size=%4d\n",
+ bm_pool->type == MVNETA_BM_SHORT ? "short" : "long",
+ bm_pool->id, bm_pool->pkt_size, bm_pool->buf_size,
+ bm_pool->frag_size);
+
+ dev_dbg(&priv->pdev->dev,
+ "%s pool %d: %d of %d buffers added\n",
+ bm_pool->type == MVNETA_BM_SHORT ? "short" : "long",
+ bm_pool->id, i, buf_num);
+
+ return i;
+}
+
+/* Create pool */
+static int mvneta_bm_pool_create(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool)
+{
+ struct platform_device *pdev = priv->pdev;
+ u8 target_id, attr;
+ int size_bytes, err;
+
+ size_bytes = sizeof(u32) * bm_pool->size;
+ bm_pool->virt_addr = dma_alloc_coherent(&pdev->dev, size_bytes,
+ &bm_pool->phys_addr,
+ GFP_KERNEL);
+ if (!bm_pool->virt_addr)
+ return -ENOMEM;
+
+ if (!IS_ALIGNED((u32)bm_pool->virt_addr, MVNETA_BM_POOL_PTR_ALIGN)) {
+ dma_free_coherent(&pdev->dev, size_bytes, bm_pool->virt_addr,
+ bm_pool->phys_addr);
+ dev_err(&pdev->dev, "BM pool %d is not %d bytes aligned\n",
+ bm_pool->id, MVNETA_BM_POOL_PTR_ALIGN);
+ return -ENOMEM;
+ }
+
+ err = mvebu_mbus_get_dram_win_info(bm_pool->phys_addr, &target_id,
+ &attr);
+ if (err < 0) {
+ dma_free_coherent(&pdev->dev, size_bytes, bm_pool->virt_addr,
+ bm_pool->phys_addr);
+ return err;
+ }
+
+ /* Set pool address */
+ mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(bm_pool->id),
+ bm_pool->phys_addr);
+
+ mvneta_bm_pool_target_set(priv, bm_pool->id, target_id, attr);
+ mvneta_bm_pool_enable(priv, bm_pool->id);
+
+ return 0;
+}
+
+/* Notify the driver that BM pool is being used as specific type and return the
+ * pool pointer on success
+ */
+struct mvneta_bm_pool *mvneta_bm_pool_use(struct mvneta_bm *priv, u8 pool_id,
+ enum mvneta_bm_type type, u8 port_id,
+ int pkt_size)
+{
+ struct mvneta_bm_pool *new_pool = &priv->bm_pools[pool_id];
+ int num, err;
+
+ if (new_pool->type == MVNETA_BM_LONG &&
+ new_pool->port_map != 1 << port_id) {
+ dev_err(&priv->pdev->dev,
+ "long pool cannot be shared by the ports\n");
+ return NULL;
+ }
+
+ if (new_pool->type == MVNETA_BM_SHORT && new_pool->type != type) {
+ dev_err(&priv->pdev->dev,
+ "mixing pools' types between the ports is forbidden\n");
+ return NULL;
+ }
+
+ if (new_pool->pkt_size == 0 || type != MVNETA_BM_SHORT)
+ new_pool->pkt_size = pkt_size;
+
+ /* Allocate buffers in case BM pool hasn't been used yet */
+ if (new_pool->type == MVNETA_BM_FREE) {
+ new_pool->type = type;
+ new_pool->buf_size = MVNETA_RX_BUF_SIZE(new_pool->pkt_size);
+ new_pool->frag_size =
+ SKB_DATA_ALIGN(MVNETA_RX_BUF_SIZE(new_pool->pkt_size)) +
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+
+ /* Create new pool */
+ err = mvneta_bm_pool_create(priv, new_pool);
+ if (err) {
+ dev_err(&priv->pdev->dev, "fail to create pool %d\n",
+ new_pool->id);
+ return NULL;
+ }
+
+ /* Allocate buffers for this pool */
+ num = mvneta_bm_bufs_add(priv, new_pool, new_pool->size);
+ if (num != new_pool->size) {
+ WARN(1, "pool %d: %d of %d allocated\n",
+ new_pool->id, num, new_pool->size);
+ return NULL;
+ }
+ }
+
+ return new_pool;
+}
+EXPORT_SYMBOL_GPL(mvneta_bm_pool_use);
+
+/* Free all buffers from the pool */
+void mvneta_bm_bufs_free(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
+ u8 port_map)
+{
+ int i;
+
+ bm_pool->port_map &= ~port_map;
+ if (bm_pool->port_map)
+ return;
+
+ mvneta_bm_config_set(priv, MVNETA_BM_EMPTY_LIMIT_MASK);
+
+ for (i = 0; i < bm_pool->buf_num; i++) {
+ dma_addr_t buf_phys_addr;
+ u32 *vaddr;
+
+ /* Get buffer physical address (indirect access) */
+ buf_phys_addr = mvneta_bm_pool_get_bp(priv, bm_pool);
+
+ /* Work-around to the problems when destroying the pool,
+ * when it occurs that a read access to BPPI returns 0.
+ */
+ if (buf_phys_addr == 0)
+ continue;
+
+ vaddr = phys_to_virt(buf_phys_addr);
+ if (!vaddr)
+ break;
+
+ dma_unmap_single(&priv->pdev->dev, buf_phys_addr,
+ bm_pool->buf_size, DMA_FROM_DEVICE);
+ mvneta_frag_free(bm_pool->frag_size, vaddr);
+ }
+
+ mvneta_bm_config_clear(priv, MVNETA_BM_EMPTY_LIMIT_MASK);
+
+ /* Update BM driver with number of buffers removed from pool */
+ bm_pool->buf_num -= i;
+}
+
+/* Cleanup pool */
+void mvneta_bm_pool_destroy(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool, u8 port_map)
+{
+ bm_pool->port_map &= ~port_map;
+ if (bm_pool->port_map)
+ return;
+
+ bm_pool->type = MVNETA_BM_FREE;
+
+ mvneta_bm_bufs_free(priv, bm_pool, port_map);
+ if (bm_pool->buf_num)
+ WARN(1, "cannot free all buffers in pool %d\n", bm_pool->id);
+
+ if (bm_pool->virt_addr) {
+ dma_free_coherent(&priv->pdev->dev, sizeof(u32) * bm_pool->size,
+ bm_pool->virt_addr, bm_pool->phys_addr);
+ bm_pool->virt_addr = NULL;
+ }
+
+ mvneta_bm_pool_disable(priv, bm_pool->id);
+}
+EXPORT_SYMBOL_GPL(mvneta_bm_pool_destroy);
+
+static void mvneta_bm_pools_init(struct mvneta_bm *priv)
+{
+ struct device_node *dn = priv->pdev->dev.of_node;
+ struct mvneta_bm_pool *bm_pool;
+ char prop[15];
+ u32 size;
+ int i;
+
+ /* Activate BM unit */
+ mvneta_bm_write(priv, MVNETA_BM_COMMAND_REG, MVNETA_BM_START_MASK);
+
+ /* Create all pools with maximum size */
+ for (i = 0; i < MVNETA_BM_POOLS_NUM; i++) {
+ bm_pool = &priv->bm_pools[i];
+ bm_pool->id = i;
+ bm_pool->type = MVNETA_BM_FREE;
+
+ /* Reset read pointer */
+ mvneta_bm_write(priv, MVNETA_BM_POOL_READ_PTR_REG(i), 0);
+
+ /* Reset write pointer */
+ mvneta_bm_write(priv, MVNETA_BM_POOL_WRITE_PTR_REG(i), 0);
+
+ /* Configure pool size according to DT or use default value */
+ sprintf(prop, "pool%d,capacity", i);
+ if (of_property_read_u32(dn, prop, &size)) {
+ size = MVNETA_BM_POOL_CAP_DEF;
+ } else if (size > MVNETA_BM_POOL_CAP_MAX) {
+ dev_warn(&priv->pdev->dev,
+ "Illegal pool %d capacity %d, set to %d\n",
+ i, size, MVNETA_BM_POOL_CAP_MAX);
+ size = MVNETA_BM_POOL_CAP_MAX;
+ } else if (size < MVNETA_BM_POOL_CAP_MIN) {
+ dev_warn(&priv->pdev->dev,
+ "Illegal pool %d capacity %d, set to %d\n",
+ i, size, MVNETA_BM_POOL_CAP_MIN);
+ size = MVNETA_BM_POOL_CAP_MIN;
+ } else if (!IS_ALIGNED(size, MVNETA_BM_POOL_CAP_ALIGN)) {
+ dev_warn(&priv->pdev->dev,
+ "Illegal pool %d capacity %d, round to %d\n",
+ i, size, ALIGN(size,
+ MVNETA_BM_POOL_CAP_ALIGN));
+ size = ALIGN(size, MVNETA_BM_POOL_CAP_ALIGN);
+ }
+ bm_pool->size = size;
+
+ mvneta_bm_write(priv, MVNETA_BM_POOL_SIZE_REG(i),
+ bm_pool->size);
+
+ /* Obtain custom pkt_size from DT */
+ sprintf(prop, "pool%d,pkt-size", i);
+ if (of_property_read_u32(dn, prop, &bm_pool->pkt_size))
+ bm_pool->pkt_size = 0;
+ }
+}
+
+static void mvneta_bm_default_set(struct mvneta_bm *priv)
+{
+ u32 val;
+
+ /* Mask BM all interrupts */
+ mvneta_bm_write(priv, MVNETA_BM_INTR_MASK_REG, 0);
+
+ /* Clear BM cause register */
+ mvneta_bm_write(priv, MVNETA_BM_INTR_CAUSE_REG, 0);
+
+ /* Set BM configuration register */
+ val = mvneta_bm_read(priv, MVNETA_BM_CONFIG_REG);
+
+ /* Reduce MaxInBurstSize from 32 BPs to 16 BPs */
+ val &= ~MVNETA_BM_MAX_IN_BURST_SIZE_MASK;
+ val |= MVNETA_BM_MAX_IN_BURST_SIZE_16BP;
+ mvneta_bm_write(priv, MVNETA_BM_CONFIG_REG, val);
+}
+
+static int mvneta_bm_init(struct mvneta_bm *priv)
+{
+ mvneta_bm_default_set(priv);
+
+ /* Allocate and initialize BM pools structures */
+ priv->bm_pools = devm_kcalloc(&priv->pdev->dev, MVNETA_BM_POOLS_NUM,
+ sizeof(struct mvneta_bm_pool),
+ GFP_KERNEL);
+ if (!priv->bm_pools)
+ return -ENOMEM;
+
+ mvneta_bm_pools_init(priv);
+
+ return 0;
+}
+
+static int mvneta_bm_get_sram(struct device_node *dn,
+ struct mvneta_bm *priv)
+{
+ priv->bppi_pool = of_gen_pool_get(dn, "internal-mem", 0);
+ if (!priv->bppi_pool)
+ return -ENOMEM;
+
+ priv->bppi_virt_addr = gen_pool_dma_alloc(priv->bppi_pool,
+ MVNETA_BM_BPPI_SIZE,
+ &priv->bppi_phys_addr);
+ if (!priv->bppi_virt_addr)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void mvneta_bm_put_sram(struct mvneta_bm *priv)
+{
+ gen_pool_free(priv->bppi_pool, priv->bppi_phys_addr,
+ MVNETA_BM_BPPI_SIZE);
+}
+
+static int mvneta_bm_probe(struct platform_device *pdev)
+{
+ struct device_node *dn = pdev->dev.of_node;
+ struct mvneta_bm *priv;
+ struct resource *res;
+ int err;
+
+ priv = devm_kzalloc(&pdev->dev, sizeof(struct mvneta_bm), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ priv->reg_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(priv->reg_base))
+ return PTR_ERR(priv->reg_base);
+
+ priv->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(priv->clk))
+ return PTR_ERR(priv->clk);
+ err = clk_prepare_enable(priv->clk);
+ if (err < 0)
+ return err;
+
+ err = mvneta_bm_get_sram(dn, priv);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to allocate internal memory\n");
+ goto err_clk;
+ }
+
+ priv->pdev = pdev;
+
+ /* Initialize buffer manager internals */
+ err = mvneta_bm_init(priv);
+ if (err < 0) {
+ dev_err(&pdev->dev, "failed to initialize controller\n");
+ goto err_sram;
+ }
+
+ dn->data = priv;
+ platform_set_drvdata(pdev, priv);
+
+ dev_info(&pdev->dev, "Buffer Manager for network controller enabled\n");
+
+ return 0;
+
+err_sram:
+ mvneta_bm_put_sram(priv);
+err_clk:
+ clk_disable_unprepare(priv->clk);
+ return err;
+}
+
+static int mvneta_bm_remove(struct platform_device *pdev)
+{
+ struct mvneta_bm *priv = platform_get_drvdata(pdev);
+ u8 all_ports_map = 0xff;
+ int i = 0;
+
+ for (i = 0; i < MVNETA_BM_POOLS_NUM; i++) {
+ struct mvneta_bm_pool *bm_pool = &priv->bm_pools[i];
+
+ mvneta_bm_pool_destroy(priv, bm_pool, all_ports_map);
+ }
+
+ mvneta_bm_put_sram(priv);
+
+ /* Dectivate BM unit */
+ mvneta_bm_write(priv, MVNETA_BM_COMMAND_REG, MVNETA_BM_STOP_MASK);
+
+ clk_disable_unprepare(priv->clk);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM_SLEEP
+bool mvneta_bm_active(struct mvneta_bm *priv)
+{
+ bool enabled;
+ u32 val;
+
+ val = mvneta_bm_read(priv, MVNETA_BM_COMMAND_REG);
+ val &= MVNETA_BM_STATUS_MASK;
+
+ enabled = val == MVNETA_BM_ACTIVE_MASK ? true : false;
+
+ return enabled;
+}
+
+static int mvneta_bm_pools_restore(struct mvneta_bm *priv, u8 id)
+{
+ struct mvneta_bm_pool *bm_pool = &priv->bm_pools[id];
+ u8 target_id, attr;
+ int ret;
+
+ mvneta_bm_write(priv, MVNETA_BM_POOL_SIZE_REG(id), bm_pool->size);
+
+ /* Reset read pointer */
+ mvneta_bm_write(priv, MVNETA_BM_POOL_READ_PTR_REG(id), 0);
+
+ /* Reset write pointer */
+ mvneta_bm_write(priv, MVNETA_BM_POOL_WRITE_PTR_REG(id), 0);
+
+ mvneta_bm_write(priv, MVNETA_BM_POOL_BASE_REG(bm_pool->id),
+ bm_pool->phys_addr);
+
+ ret = mvebu_mbus_get_dram_win_info(bm_pool->phys_addr, &target_id,
+ &attr);
+ if (ret < 0)
+ return ret;
+
+ mvneta_bm_pool_target_set(priv, bm_pool->id, target_id, attr);
+
+ mvneta_bm_pool_enable(priv, bm_pool->id);
+
+ return 0;
+}
+
+static int mvneta_bm_suspend(struct platform_device *pdev, pm_message_t state)
+{
+ struct mvneta_bm *priv = platform_get_drvdata(pdev);
+
+ mvneta_bm_write(priv, MVNETA_BM_COMMAND_REG, MVNETA_BM_STOP_MASK);
+
+ return 0;
+}
+
+int mvneta_bm_resume(struct platform_device *pdev)
+{
+ struct mvneta_bm *priv = platform_get_drvdata(pdev);
+ int ret, i;
+
+ if (mvneta_bm_active(priv))
+ return 0;
+
+ mvneta_bm_write(priv, MVNETA_BM_COMMAND_REG, MVNETA_BM_START_MASK);
+
+ mvneta_bm_default_set(priv);
+
+ for (i = 0; i < MVNETA_BM_POOLS_NUM; i++) {
+ ret = mvneta_bm_pools_restore(priv, i);
+ if (ret < 0) {
+ dev_err(&pdev->dev, "Fail to restore pool %d\n", i);
+ return ret;
+ }
+ }
+
+ return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
+static const struct of_device_id mvneta_bm_match[] = {
+ { .compatible = "marvell,armada-380-neta-bm" },
+ { }
+};
+MODULE_DEVICE_TABLE(of, mvneta_bm_match);
+
+static struct platform_driver mvneta_bm_driver = {
+ .probe = mvneta_bm_probe,
+ .remove = mvneta_bm_remove,
+#ifdef CONFIG_PM_SLEEP
+ .suspend = mvneta_bm_suspend,
+ .resume = mvneta_bm_resume,
+#endif
+ .driver = {
+ .name = MVNETA_BM_DRIVER_NAME,
+ .of_match_table = mvneta_bm_match,
+ },
+};
+
+module_platform_driver(mvneta_bm_driver);
+
+MODULE_DESCRIPTION("Marvell NETA Buffer Manager Driver - http://www.marvell.com");
+MODULE_AUTHOR("Marcin Wojtas <[email protected]>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/net/ethernet/marvell/mvneta_bm.h b/drivers/net/ethernet/marvell/mvneta_bm.h
new file mode 100644
index 0000000..c03ce47
--- /dev/null
+++ b/drivers/net/ethernet/marvell/mvneta_bm.h
@@ -0,0 +1,171 @@
+/*
+ * Driver for Marvell NETA network controller Buffer Manager.
+ *
+ * Copyright (C) 2015 Marvell
+ *
+ * Marcin Wojtas <[email protected]>
+ *
+ * This file is licensed under the terms of the GNU General Public
+ * License version 2. This program is licensed "as is" without any
+ * warranty of any kind, whether express or implied.
+ */
+
+#ifndef _MVNETA_BM_H_
+#define _MVNETA_BM_H_
+
+/* BM Configuration Register */
+#define MVNETA_BM_CONFIG_REG 0x0
+#define MVNETA_BM_STATUS_MASK 0x30
+#define MVNETA_BM_ACTIVE_MASK BIT(4)
+#define MVNETA_BM_MAX_IN_BURST_SIZE_MASK 0x60000
+#define MVNETA_BM_MAX_IN_BURST_SIZE_16BP BIT(18)
+#define MVNETA_BM_EMPTY_LIMIT_MASK BIT(19)
+
+/* BM Activation Register */
+#define MVNETA_BM_COMMAND_REG 0x4
+#define MVNETA_BM_START_MASK BIT(0)
+#define MVNETA_BM_STOP_MASK BIT(1)
+#define MVNETA_BM_PAUSE_MASK BIT(2)
+
+/* BM Xbar interface Register */
+#define MVNETA_BM_XBAR_01_REG 0x8
+#define MVNETA_BM_XBAR_23_REG 0xc
+#define MVNETA_BM_XBAR_POOL_REG(pool) \
+ (((pool) < 2) ? MVNETA_BM_XBAR_01_REG : MVNETA_BM_XBAR_23_REG)
+#define MVNETA_BM_TARGET_ID_OFFS(pool) (((pool) & 1) ? 16 : 0)
+#define MVNETA_BM_TARGET_ID_MASK(pool) \
+ (0xf << MVNETA_BM_TARGET_ID_OFFS(pool))
+#define MVNETA_BM_TARGET_ID_VAL(pool, id) \
+ ((id) << MVNETA_BM_TARGET_ID_OFFS(pool))
+#define MVNETA_BM_XBAR_ATTR_OFFS(pool) (((pool) & 1) ? 20 : 4)
+#define MVNETA_BM_XBAR_ATTR_MASK(pool) \
+ (0xff << MVNETA_BM_XBAR_ATTR_OFFS(pool))
+#define MVNETA_BM_XBAR_ATTR_VAL(pool, attr) \
+ ((attr) << MVNETA_BM_XBAR_ATTR_OFFS(pool))
+
+/* Address of External Buffer Pointers Pool Register */
+#define MVNETA_BM_POOL_BASE_REG(pool) (0x10 + ((pool) << 4))
+#define MVNETA_BM_POOL_ENABLE_MASK BIT(0)
+
+/* External Buffer Pointers Pool RD pointer Register */
+#define MVNETA_BM_POOL_READ_PTR_REG(pool) (0x14 + ((pool) << 4))
+#define MVNETA_BM_POOL_SET_READ_PTR_MASK 0xfffc
+#define MVNETA_BM_POOL_GET_READ_PTR_OFFS 16
+#define MVNETA_BM_POOL_GET_READ_PTR_MASK 0xfffc0000
+
+/* External Buffer Pointers Pool WR pointer */
+#define MVNETA_BM_POOL_WRITE_PTR_REG(pool) (0x18 + ((pool) << 4))
+#define MVNETA_BM_POOL_SET_WRITE_PTR_OFFS 0
+#define MVNETA_BM_POOL_SET_WRITE_PTR_MASK 0xfffc
+#define MVNETA_BM_POOL_GET_WRITE_PTR_OFFS 16
+#define MVNETA_BM_POOL_GET_WRITE_PTR_MASK 0xfffc0000
+
+/* External Buffer Pointers Pool Size Register */
+#define MVNETA_BM_POOL_SIZE_REG(pool) (0x1c + ((pool) << 4))
+#define MVNETA_BM_POOL_SIZE_MASK 0x3fff
+
+/* BM Interrupt Cause Register */
+#define MVNETA_BM_INTR_CAUSE_REG (0x50)
+
+/* BM interrupt Mask Register */
+#define MVNETA_BM_INTR_MASK_REG (0x54)
+
+/* Other definitions */
+#define MVNETA_BM_SHORT_PKT_SIZE 256
+#define MVNETA_BM_POOLS_NUM 4
+#define MVNETA_BM_POOL_CAP_MIN 128
+#define MVNETA_BM_POOL_CAP_DEF 2048
+#define MVNETA_BM_POOL_CAP_MAX \
+ (16 * 1024 - MVNETA_BM_POOL_CAP_ALIGN)
+#define MVNETA_BM_POOL_CAP_ALIGN 32
+#define MVNETA_BM_POOL_PTR_ALIGN 32
+
+#define MVNETA_BM_POOL_ACCESS_OFFS 8
+
+#define MVNETA_BM_BPPI_SIZE 0x100000
+
+#define MVNETA_RX_BUF_SIZE(pkt_size) ((pkt_size) + NET_SKB_PAD)
+
+enum mvneta_bm_type {
+ MVNETA_BM_FREE,
+ MVNETA_BM_LONG,
+ MVNETA_BM_SHORT
+};
+
+struct mvneta_bm {
+ void __iomem *reg_base;
+ struct clk *clk;
+ struct platform_device *pdev;
+
+ struct gen_pool *bppi_pool;
+ /* BPPI virtual base address */
+ void __iomem *bppi_virt_addr;
+ /* BPPI physical base address */
+ dma_addr_t bppi_phys_addr;
+
+ /* BM pools */
+ struct mvneta_bm_pool *bm_pools;
+};
+
+struct mvneta_bm_pool {
+ /* Pool number in the range 0-3 */
+ u8 id;
+ enum mvneta_bm_type type;
+
+ /* Buffer Pointers Pool External (BPPE) size in number of bytes */
+ int size;
+ /* Number of buffers used by this pool */
+ int buf_num;
+ /* Pool buffer size */
+ int buf_size;
+ /* Packet size */
+ int pkt_size;
+ /* Single frag size */
+ u32 frag_size;
+
+ /* BPPE virtual base address */
+ u32 *virt_addr;
+ /* BPPE physical base address */
+ dma_addr_t phys_addr;
+
+ /* Ports using BM pool */
+ u8 port_map;
+
+ struct mvneta_bm *priv;
+};
+
+/* Declarations and definitions */
+void *mvneta_frag_alloc(unsigned int frag_size);
+void mvneta_frag_free(unsigned int frag_size, void *data);
+void mvneta_bm_pool_destroy(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool, u8 port_map);
+void mvneta_bm_bufs_free(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
+ u8 port_map);
+int mvneta_bm_bufs_add(struct mvneta_bm *priv, struct mvneta_bm_pool *bm_pool,
+ int buf_num);
+int mvneta_bm_pool_refill(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool);
+#ifdef CONFIG_PM_SLEEP
+int mvneta_bm_resume(struct platform_device *pdev);
+bool mvneta_bm_active(struct mvneta_bm *priv);
+#endif
+struct mvneta_bm_pool *mvneta_bm_pool_use(struct mvneta_bm *priv, u8 pool_id,
+ enum mvneta_bm_type type, u8 port_id,
+ int pkt_size);
+
+static inline void mvneta_bm_pool_put_bp(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool,
+ dma_addr_t buf_phys_addr)
+{
+ writel_relaxed(buf_phys_addr, priv->bppi_virt_addr +
+ (bm_pool->id << MVNETA_BM_POOL_ACCESS_OFFS));
+}
+
+static inline u32 mvneta_bm_pool_get_bp(struct mvneta_bm *priv,
+ struct mvneta_bm_pool *bm_pool)
+{
+ return readl_relaxed(priv->bppi_virt_addr +
+ (bm_pool->id << MVNETA_BM_POOL_ACCESS_OFFS));
+}
+
+#endif
--
1.8.3.1
Armada 38x network controller supports hardware buffer management (BM).
Since it is now enabled in mvneta driver, appropriate nodes can be added
to armada-38x.dtsi - for the actual common BM unit (bm@c8000) and its
internal SRAM (bm-bppi), which is used for indirect access to buffer
pointer ring residing in DRAM.
Pools - ports mapping, bm-bppi entry in 'soc' node's ranges and optional
parameters are supposed to be set in board files.
Signed-off-by: Marcin Wojtas <[email protected]>
---
arch/arm/boot/dts/armada-38x.dtsi | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
index b7868b2..b9f4ce2 100644
--- a/arch/arm/boot/dts/armada-38x.dtsi
+++ b/arch/arm/boot/dts/armada-38x.dtsi
@@ -539,6 +539,14 @@
status = "disabled";
};
+ bm: bm@c8000 {
+ compatible = "marvell,armada-380-neta-bm";
+ reg = <0xc8000 0xac>;
+ clocks = <&gateclk 13>;
+ internal-mem = <&bm_bppi>;
+ status = "disabled";
+ };
+
sata@e0000 {
compatible = "marvell,armada-380-ahci";
reg = <0xe0000 0x2000>;
@@ -617,6 +625,16 @@
#size-cells = <1>;
ranges = <0 MBUS_ID(0x09, 0x15) 0 0x800>;
};
+
+ bm_bppi: bm-bppi {
+ compatible = "mmio-sram";
+ reg = <MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ ranges = <0 MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ clocks = <&gateclk 13>;
+ status = "disabled";
+ };
};
clocks {
--
1.8.3.1
Since mvneta driver supports using hardware buffer management (BM), in
order to use it, board files have to be adjusted accordingly. This commit
enables BM on:
* A385-DB-AP - each port has its own pool for long and common pool for
short packets,
* A388-DB - to each port unique 'short' and 'long' pools are mapped,
* A388-GP - same as above.
Moreover appropriate entry is added to 'soc' node ranges, as well as "okay"
status for 'bm' and 'bm-bppi' (internal SRAM) nodes.
Signed-off-by: Marcin Wojtas <[email protected]>
---
arch/arm/boot/dts/armada-385-db-ap.dts | 20 +++++++++++++++++++-
arch/arm/boot/dts/armada-388-db.dts | 17 ++++++++++++++++-
arch/arm/boot/dts/armada-388-gp.dts | 17 ++++++++++++++++-
3 files changed, 51 insertions(+), 3 deletions(-)
diff --git a/arch/arm/boot/dts/armada-385-db-ap.dts b/arch/arm/boot/dts/armada-385-db-ap.dts
index acd5b15..5f9451b 100644
--- a/arch/arm/boot/dts/armada-385-db-ap.dts
+++ b/arch/arm/boot/dts/armada-385-db-ap.dts
@@ -61,7 +61,8 @@
ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
MBUS_ID(0x01, 0x1d) 0 0xfff00000 0x100000
MBUS_ID(0x09, 0x19) 0 0xf1100000 0x10000
- MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000>;
+ MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0xf1200000 0x100000>;
internal-regs {
spi1: spi@10680 {
@@ -138,12 +139,18 @@
status = "okay";
phy = <&phy2>;
phy-mode = "sgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <1>;
+ bm,pool-short = <3>;
};
ethernet@34000 {
status = "okay";
phy = <&phy1>;
phy-mode = "sgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <2>;
+ bm,pool-short = <3>;
};
ethernet@70000 {
@@ -157,6 +164,13 @@
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
+ bm,pool-short = <3>;
+ };
+
+ bm@c8000 {
+ status = "okay";
};
nfc: flash@d0000 {
@@ -178,6 +192,10 @@
};
};
+ bm-bppi {
+ status = "okay";
+ };
+
pcie-controller {
status = "okay";
diff --git a/arch/arm/boot/dts/armada-388-db.dts b/arch/arm/boot/dts/armada-388-db.dts
index ff47af5..ea93ed7 100644
--- a/arch/arm/boot/dts/armada-388-db.dts
+++ b/arch/arm/boot/dts/armada-388-db.dts
@@ -66,7 +66,8 @@
ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
MBUS_ID(0x01, 0x1d) 0 0xfff00000 0x100000
MBUS_ID(0x09, 0x19) 0 0xf1100000 0x10000
- MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000>;
+ MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0xf1200000 0x100000>;
internal-regs {
spi@10600 {
@@ -99,6 +100,9 @@
status = "okay";
phy = <&phy1>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <2>;
+ bm,pool-short = <3>;
};
usb@58000 {
@@ -109,6 +113,9 @@
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
+ bm,pool-short = <1>;
};
mdio@72004 {
@@ -129,6 +136,10 @@
status = "okay";
};
+ bm@c8000 {
+ status = "okay";
+ };
+
flash@d0000 {
status = "okay";
num-cs = <1>;
@@ -169,6 +180,10 @@
};
};
+ bm-bppi {
+ status = "okay";
+ };
+
pcie-controller {
status = "okay";
/*
diff --git a/arch/arm/boot/dts/armada-388-gp.dts b/arch/arm/boot/dts/armada-388-gp.dts
index a633be3..0a3bd7f 100644
--- a/arch/arm/boot/dts/armada-388-gp.dts
+++ b/arch/arm/boot/dts/armada-388-gp.dts
@@ -60,7 +60,8 @@
ranges = <MBUS_ID(0xf0, 0x01) 0 0xf1000000 0x100000
MBUS_ID(0x01, 0x1d) 0 0xfff00000 0x100000
MBUS_ID(0x09, 0x19) 0 0xf1100000 0x10000
- MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000>;
+ MBUS_ID(0x09, 0x15) 0 0xf1110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0xf1200000 0x100000>;
internal-regs {
spi@10600 {
@@ -133,6 +134,9 @@
status = "okay";
phy = <&phy1>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <2>;
+ bm,pool-short = <3>;
};
/* CON4 */
@@ -152,6 +156,9 @@
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
+ bm,pool-short = <1>;
};
@@ -186,6 +193,10 @@
};
};
+ bm@c8000 {
+ status = "okay";
+ };
+
sata@e0000 {
pinctrl-names = "default";
pinctrl-0 = <&sata2_pins>, <&sata3_pins>;
@@ -240,6 +251,10 @@
};
};
+ bm-bppi {
+ status = "okay";
+ };
+
pcie-controller {
status = "okay";
/*
--
1.8.3.1
Armada XP network controller supports hardware buffer management (BM).
Since it is now enabled in mvneta driver, appropriate nodes can be added
to armada-xp.dtsi - for the actual common BM unit (bm@c0000) and its
internal SRAM (bm-bppi), which is used for indirect access to buffer
pointer ring residing in DRAM.
Pools - ports mapping, bm-bppi entry in 'soc' node's ranges and optional
parameters are supposed to be set in board files.
Signed-off-by: Marcin Wojtas <[email protected]>
---
arch/arm/boot/dts/armada-xp.dtsi | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/arch/arm/boot/dts/armada-xp.dtsi b/arch/arm/boot/dts/armada-xp.dtsi
index be23196..bd45936 100644
--- a/arch/arm/boot/dts/armada-xp.dtsi
+++ b/arch/arm/boot/dts/armada-xp.dtsi
@@ -253,6 +253,14 @@
marvell,crypto-sram-size = <0x800>;
};
+ bm: bm@c0000 {
+ compatible = "marvell,armada-380-neta-bm";
+ reg = <0xc0000 0xac>;
+ clocks = <&gateclk 13>;
+ internal-mem = <&bm_bppi>;
+ status = "disabled";
+ };
+
xor@f0900 {
compatible = "marvell,orion-xor";
reg = <0xF0900 0x100
@@ -291,6 +299,16 @@
#size-cells = <1>;
ranges = <0 MBUS_ID(0x09, 0x05) 0 0x800>;
};
+
+ bm_bppi: bm-bppi {
+ compatible = "mmio-sram";
+ reg = <MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ ranges = <0 MBUS_ID(0x0c, 0x04) 0 0x100000>;
+ #address-cells = <1>;
+ #size-cells = <1>;
+ clocks = <&gateclk 13>;
+ status = "disabled";
+ };
};
clocks {
--
1.8.3.1
Since mvneta driver supports using hardware buffer management (BM), in
order to use it, board files have to be adjusted accordingly. This commit
enables BM on AXP-DB and AXP-GP in same manner - because number of ports
on those boards is the same as number of possible pools, each port is
supposed to use single pool for all kind of packets.
Moreover appropriate entry is added to 'soc' node ranges, as well as "okay"
status for 'bm' and 'bm-bppi' (internal SRAM) nodes.
Signed-off-by: Marcin Wojtas <[email protected]>
---
arch/arm/boot/dts/armada-xp-db.dts | 19 ++++++++++++++++++-
arch/arm/boot/dts/armada-xp-gp.dts | 19 ++++++++++++++++++-
2 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/arch/arm/boot/dts/armada-xp-db.dts b/arch/arm/boot/dts/armada-xp-db.dts
index f774101..3065730 100644
--- a/arch/arm/boot/dts/armada-xp-db.dts
+++ b/arch/arm/boot/dts/armada-xp-db.dts
@@ -77,7 +77,8 @@
MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
MBUS_ID(0x01, 0x2f) 0 0 0xf0000000 0x1000000
MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
- MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
+ MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0 0xf1200000 0x100000>;
devbus-bootcs {
status = "okay";
@@ -181,21 +182,33 @@
status = "okay";
phy = <&phy0>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
};
ethernet@74000 {
status = "okay";
phy = <&phy1>;
phy-mode = "rgmii-id";
+ buffer-manager = <&bm>;
+ bm,pool-long = <1>;
};
ethernet@30000 {
status = "okay";
phy = <&phy2>;
phy-mode = "sgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <2>;
};
ethernet@34000 {
status = "okay";
phy = <&phy3>;
phy-mode = "sgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <3>;
+ };
+
+ bm@c0000 {
+ status = "okay";
};
mvsdio@d4000 {
@@ -230,5 +243,9 @@
};
};
};
+
+ bm-bppi {
+ status = "okay";
+ };
};
};
diff --git a/arch/arm/boot/dts/armada-xp-gp.dts b/arch/arm/boot/dts/armada-xp-gp.dts
index 4878d73..a1ded01 100644
--- a/arch/arm/boot/dts/armada-xp-gp.dts
+++ b/arch/arm/boot/dts/armada-xp-gp.dts
@@ -96,7 +96,8 @@
MBUS_ID(0x01, 0x1d) 0 0 0xfff00000 0x100000
MBUS_ID(0x01, 0x2f) 0 0 0xf0000000 0x1000000
MBUS_ID(0x09, 0x09) 0 0 0xf8100000 0x10000
- MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000>;
+ MBUS_ID(0x09, 0x05) 0 0 0xf8110000 0x10000
+ MBUS_ID(0x0c, 0x04) 0 0 0xf1200000 0x100000>;
devbus-bootcs {
status = "okay";
@@ -196,21 +197,29 @@
status = "okay";
phy = <&phy0>;
phy-mode = "qsgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <0>;
};
ethernet@74000 {
status = "okay";
phy = <&phy1>;
phy-mode = "qsgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <1>;
};
ethernet@30000 {
status = "okay";
phy = <&phy2>;
phy-mode = "qsgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <2>;
};
ethernet@34000 {
status = "okay";
phy = <&phy3>;
phy-mode = "qsgmii";
+ buffer-manager = <&bm>;
+ bm,pool-long = <3>;
};
/* Front-side USB slot */
@@ -235,6 +244,10 @@
};
};
+ bm@c0000 {
+ status = "okay";
+ };
+
nand@d0000 {
status = "okay";
num-cs = <1>;
@@ -243,5 +256,9 @@
nand-on-flash-bbt;
};
};
+
+ bm-bppi {
+ status = "okay";
+ };
};
};
--
1.8.3.1
Hello.
On 11/22/2015 10:53 AM, Marcin Wojtas wrote:
> Armada 38x network controller supports hardware buffer management (BM).
> Since it is now enabled in mvneta driver, appropriate nodes can be added
> to armada-38x.dtsi - for the actual common BM unit (bm@c8000) and its
> internal SRAM (bm-bppi), which is used for indirect access to buffer
> pointer ring residing in DRAM.
>
> Pools - ports mapping, bm-bppi entry in 'soc' node's ranges and optional
> parameters are supposed to be set in board files.
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> ---
> arch/arm/boot/dts/armada-38x.dtsi | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
> diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
> index b7868b2..b9f4ce2 100644
> --- a/arch/arm/boot/dts/armada-38x.dtsi
> +++ b/arch/arm/boot/dts/armada-38x.dtsi
> @@ -539,6 +539,14 @@
> status = "disabled";
> };
>
> + bm: bm@c8000 {
The ePAPR standard tells us to give generic names to the device nodes,
hence this should be named "buffer-manager" (?).
[...]
> @@ -617,6 +625,16 @@
> #size-cells = <1>;
> ranges = <0 MBUS_ID(0x09, 0x15) 0 0x800>;
> };
> +
> + bm_bppi: bm-bppi {
And this one "memory" (?).
[...]
MBR, Sergei
On Sunday 22 November 2015 08:53:48 Marcin Wojtas wrote:
> The Ethernet controller found in the Armada 38x SoC's family support
> TCP/IP checksumming with frame sizes larger than 1600 bytes, however
> only on port 0.
>
> This commit enables this feature by using 'marvell,armada-xp-neta' in
> 'ethernet@70000' node.
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> Cc: <[email protected]> # v3.18+
> ---
> arch/arm/boot/dts/armada-38x.dtsi | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
> index c6a0e9d..b7868b2 100644
> --- a/arch/arm/boot/dts/armada-38x.dtsi
> +++ b/arch/arm/boot/dts/armada-38x.dtsi
> @@ -494,7 +494,7 @@
> };
>
> eth0: ethernet@70000 {
> - compatible = "marvell,armada-370-neta";
> + compatible = "marvell,armada-xp-neta";
> reg = <0x70000 0x4000>;
> interrupts-extended = <&mpic 8>;
> clocks = <&gateclk 4>;
>
As it's clear that they are not 100% backwards compatible, please
add a SoC specific compatible string here as well, like
compatible = "marvell,armada-380-neta", "marvell,armada-xp-neta";
Maybe also leave the 370 string in place.
Arnd
On Sunday 22 November 2015 08:53:53 Marcin Wojtas wrote:
> This commit enables finding appropriate mbus window and obtaining its
> target id and attribute for given physical address in two separate
> routines, both for IO and DRAM windows. This functionality
> is needed for Armada XP/38x Network Controller's Buffer Manager and
> PnC configuration.
>
> Signed-off-by: Marcin Wojtas <[email protected]>
>
> [DRAM window information reference in LKv3.10]
> Signed-off-by: Evan Wang <[email protected]>
>
It's too long ago to remember all the details, but I thought we
had designed this so the configuration can just be done by
describing it in DT. What am I missing?
Arnd
On Sunday 22 November 2015 08:53:46 Marcin Wojtas wrote:
>
> 3. Optimisations - concatenating TX descriptors' flush, basing on
> xmit_more support and combined approach for finalizing egress processing.
> Thanks to HR timer buffers can be released with small latency, which is
> good for low transfer and small queues. Along with the timer, coalescing
> irqs are used, whose threshold could be increased back to 15.
>
>
If you are already reworking the TX path, it probably makes sense to
support BQL as well, see the Marvell skge and sky2 drivers for examples
using netdev_{tx_,}{sent,completed}_queue.
Arnd
Arnd,
2015-11-22 21:00 GMT+01:00 Arnd Bergmann <[email protected]>:
> On Sunday 22 November 2015 08:53:48 Marcin Wojtas wrote:
>> The Ethernet controller found in the Armada 38x SoC's family support
>> TCP/IP checksumming with frame sizes larger than 1600 bytes, however
>> only on port 0.
>>
>> This commit enables this feature by using 'marvell,armada-xp-neta' in
>> 'ethernet@70000' node.
>>
>> Signed-off-by: Marcin Wojtas <[email protected]>
>> Cc: <[email protected]> # v3.18+
>> ---
>> arch/arm/boot/dts/armada-38x.dtsi | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
>> index c6a0e9d..b7868b2 100644
>> --- a/arch/arm/boot/dts/armada-38x.dtsi
>> +++ b/arch/arm/boot/dts/armada-38x.dtsi
>> @@ -494,7 +494,7 @@
>> };
>>
>> eth0: ethernet@70000 {
>> - compatible = "marvell,armada-370-neta";
>> + compatible = "marvell,armada-xp-neta";
>> reg = <0x70000 0x4000>;
>> interrupts-extended = <&mpic 8>;
>> clocks = <&gateclk 4>;
>>
>
> As it's clear that they are not 100% backwards compatible, please
> add a SoC specific compatible string here as well, like
>
> compatible = "marvell,armada-380-neta", "marvell,armada-xp-neta";
>
Wouldn't be one sufficient ("marvell,armada-380-neta")?
> Maybe also leave the 370 string in place.
>
Now 370 string disables ip checksum for jumbo frames, so I don't think
it's appropriate to keep it for port 0.
Best regards,
Marcin
Arnd,
2015-11-22 21:02 GMT+01:00 Arnd Bergmann <[email protected]>:
> On Sunday 22 November 2015 08:53:53 Marcin Wojtas wrote:
>> This commit enables finding appropriate mbus window and obtaining its
>> target id and attribute for given physical address in two separate
>> routines, both for IO and DRAM windows. This functionality
>> is needed for Armada XP/38x Network Controller's Buffer Manager and
>> PnC configuration.
>>
>> Signed-off-by: Marcin Wojtas <[email protected]>
>>
>> [DRAM window information reference in LKv3.10]
>> Signed-off-by: Evan Wang <[email protected]>
>>
>
> It's too long ago to remember all the details, but I thought we
> had designed this so the configuration can just be done by
> describing it in DT. What am I missing?
>
And those functions do not break this approach. They just enable
finding and reading the settings of MBUS windows done during initial
configuration. Please remember that mvebu-mbus driver fills the MBUS
windows registers basing on DT, however it just configures access CPU
- DRAM/perfipheral.
In this particular case only physical adresses of buffers are known
and we have to 'open windows' between BM <-> DRAM and NETA <-> BM
internal memory. Hence instead of hardcoding size/target/attribute, we
can take information stored in CPU DRAM/IO windows registers.
Best regards,
Marcin
On Sunday 22 November 2015 22:04:38 Marcin Wojtas wrote:
> 2015-11-22 21:00 GMT+01:00 Arnd Bergmann <[email protected]>:
> > On Sunday 22 November 2015 08:53:48 Marcin Wojtas wrote:
> >> The Ethernet controller found in the Armada 38x SoC's family support
> >> TCP/IP checksumming with frame sizes larger than 1600 bytes, however
> >> only on port 0.
> >>
> >> This commit enables this feature by using 'marvell,armada-xp-neta' in
> >> 'ethernet@70000' node.
> >>
> >> Signed-off-by: Marcin Wojtas <[email protected]>
> >> Cc: <[email protected]> # v3.18+
> >> ---
> >> arch/arm/boot/dts/armada-38x.dtsi | 2 +-
> >> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
> >> index c6a0e9d..b7868b2 100644
> >> --- a/arch/arm/boot/dts/armada-38x.dtsi
> >> +++ b/arch/arm/boot/dts/armada-38x.dtsi
> >> @@ -494,7 +494,7 @@
> >> };
> >>
> >> eth0: ethernet@70000 {
> >> - compatible = "marvell,armada-370-neta";
> >> + compatible = "marvell,armada-xp-neta";
> >> reg = <0x70000 0x4000>;
> >> interrupts-extended = <&mpic 8>;
> >> clocks = <&gateclk 4>;
> >>
> >
> > As it's clear that they are not 100% backwards compatible, please
> > add a SoC specific compatible string here as well, like
> >
> > compatible = "marvell,armada-380-neta", "marvell,armada-xp-neta";
> >
>
> Wouldn't be one sufficient ("marvell,armada-380-neta")?
If they are basically compatible, you want to the original one in,
to make sure it keeps running on operating systems that only know
about the older string.
> > Maybe also leave the 370 string in place.
> >
>
> Now 370 string disables ip checksum for jumbo frames, so I don't think
> it's appropriate to keep it for port 0.
Ok, I see. We should probably have done it the other way round and
kept the default as checksum-disabled and only override it when
the newer compatible string is also present. Basically the device
*is* compatible to an Armada 370, it just has additional features that
work correctly.
If the feature set depends on the port number, we should think about
the way it gets handled again, as this is probably better not described
as something that depends (just) on the SoC, but on the way it gets
integrated. Maybe we can introduce an additional property for the
checksums on jumbo frames and use that if present but fall back to
identifying by compatible string otherwise.
Arnd
Arnd,
2015-11-22 21:06 GMT+01:00 Arnd Bergmann <[email protected]>:
> On Sunday 22 November 2015 08:53:46 Marcin Wojtas wrote:
>>
>> 3. Optimisations - concatenating TX descriptors' flush, basing on
>> xmit_more support and combined approach for finalizing egress processing.
>> Thanks to HR timer buffers can be released with small latency, which is
>> good for low transfer and small queues. Along with the timer, coalescing
>> irqs are used, whose threshold could be increased back to 15.
>>
>>
>
> If you are already reworking the TX path, it probably makes sense to
> support BQL as well, see the Marvell skge and sky2 drivers for examples
> using netdev_{tx_,}{sent,completed}_queue.
>
Good idea, I'll take a look.
Best regards,
Marcin
Arnd,
>
> If the feature set depends on the port number, we should think about
> the way it gets handled again, as this is probably better not described
> as something that depends (just) on the SoC, but on the way it gets
> integrated. Maybe we can introduce an additional property for the
> checksums on jumbo frames and use that if present but fall back to
> identifying by compatible string otherwise.
>
I think adding a property, taking also compatible strings will be the
best solution.
Best regards,
Marcin
On Sunday 22 November 2015 22:24:01 Marcin Wojtas wrote:
>
> 2015-11-22 21:02 GMT+01:00 Arnd Bergmann <[email protected]>:
> > On Sunday 22 November 2015 08:53:53 Marcin Wojtas wrote:
> >> This commit enables finding appropriate mbus window and obtaining its
> >> target id and attribute for given physical address in two separate
> >> routines, both for IO and DRAM windows. This functionality
> >> is needed for Armada XP/38x Network Controller's Buffer Manager and
> >> PnC configuration.
> >>
> >> Signed-off-by: Marcin Wojtas <[email protected]>
> >>
> >> [DRAM window information reference in LKv3.10]
> >> Signed-off-by: Evan Wang <[email protected]>
> >>
> >
> > It's too long ago to remember all the details, but I thought we
> > had designed this so the configuration can just be done by
> > describing it in DT. What am I missing?
> >
>
> And those functions do not break this approach. They just enable
> finding and reading the settings of MBUS windows done during initial
> configuration. Please remember that mvebu-mbus driver fills the MBUS
> windows registers basing on DT, however it just configures access CPU
> - DRAM/perfipheral.
>
> In this particular case only physical adresses of buffers are known
> and we have to 'open windows' between BM <-> DRAM and NETA <-> BM
> internal memory. Hence instead of hardcoding size/target/attribute, we
> can take information stored in CPU DRAM/IO windows registers.
>
>
Ok, got it. Thanks for the explanation.
Arnd
This patch series has a lot of problems.
The worst one is that you are mixing bug fixes and new
features/cleanups.
You cannot do that.
You _especially_ can't do that if you expect some of this stuff
to be submitted to -stable too.
Therefore you _must_ extract out the bug fixes, and submit those
against 'net'. And you must then wait for 'net' to be merged
into 'net-next' before submitting the new feature patches.
Furthermore, you should explicitly show what tree you are targetting
your patches at, in your Subject lines. In the form:
Subject: [PATCH $(TREE) N/M]
Where $(TREE) is either 'net' or 'net-next'.
THanks.
Hi David,
Thanks for the remark. I will do the split and re-submit.
Best regards,
Marcin
2015-11-24 17:22 GMT+01:00 David Miller <[email protected]>:
>
> This patch series has a lot of problems.
>
> The worst one is that you are mixing bug fixes and new
> features/cleanups.
>
> You cannot do that.
>
> You _especially_ can't do that if you expect some of this stuff
> to be submitted to -stable too.
>
> Therefore you _must_ extract out the bug fixes, and submit those
> against 'net'. And you must then wait for 'net' to be merged
> into 'net-next' before submitting the new feature patches.
>
> Furthermore, you should explicitly show what tree you are targetting
> your patches at, in your Subject lines. In the form:
>
> Subject: [PATCH $(TREE) N/M]
>
> Where $(TREE) is either 'net' or 'net-next'.
>
> THanks.
Hi Marcin,
On dim., nov. 22 2015, Marcin Wojtas <[email protected]> wrote:
> This commit adds missing configuration of MBUS windows access protection
> in mvneta_conf_mbus_windows function - a dedicated variable for that
> purpose remained there unused since v3.8 initial mvneta support. Because
> of that the register contents were inherited from the bootloader.
It looks OK for me and at least after applying the driver continues
working :)
I guess you find it when you tested suspend to ram.
Reviewed-by: Gregory CLEMENT <[email protected]>
Thanks,
Gregory
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> Cc: <[email protected]> # v3.8+
> ---
> drivers/net/ethernet/marvell/mvneta.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> index e84c7f2..0f30aaa 100644
> --- a/drivers/net/ethernet/marvell/mvneta.c
> +++ b/drivers/net/ethernet/marvell/mvneta.c
> @@ -62,6 +62,7 @@
> #define MVNETA_WIN_SIZE(w) (0x2204 + ((w) << 3))
> #define MVNETA_WIN_REMAP(w) (0x2280 + ((w) << 2))
> #define MVNETA_BASE_ADDR_ENABLE 0x2290
> +#define MVNETA_ACCESS_PROTECT_ENABLE 0x2294
> #define MVNETA_PORT_CONFIG 0x2400
> #define MVNETA_UNI_PROMISC_MODE BIT(0)
> #define MVNETA_DEF_RXQ(q) ((q) << 1)
> @@ -3188,6 +3189,8 @@ static void mvneta_conf_mbus_windows(struct mvneta_port *pp,
>
> win_enable &= ~(1 << i);
> win_protect |= 3 << (2 * i);
> +
> + mvreg_write(pp, MVNETA_ACCESS_PROTECT_ENABLE, win_protect);
> }
>
> mvreg_write(pp, MVNETA_BASE_ADDR_ENABLE, win_enable);
> --
> 1.8.3.1
>
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi Marcin,
On dim., nov. 22 2015, Marcin Wojtas <[email protected]> wrote:
> MVNETA_RXQ_HW_BUF_ALLOC bit which controls enabling hardware buffer
> allocation was mistakenly set as BIT(1). This commit fixes the
> assignment.
I confirm it from the datasheet I got:
Reviewed-by: Gregory CLEMENT <[email protected]>
Thanks,
Gregory
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> Cc: <[email protected]> # v3.8+
> ---
> drivers/net/ethernet/marvell/mvneta.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> index 0f30aaa..d12b8c6 100644
> --- a/drivers/net/ethernet/marvell/mvneta.c
> +++ b/drivers/net/ethernet/marvell/mvneta.c
> @@ -36,7 +36,7 @@
>
> /* Registers */
> #define MVNETA_RXQ_CONFIG_REG(q) (0x1400 + ((q) << 2))
> -#define MVNETA_RXQ_HW_BUF_ALLOC BIT(1)
> +#define MVNETA_RXQ_HW_BUF_ALLOC BIT(0)
> #define MVNETA_RXQ_PKT_OFFSET_ALL_MASK (0xf << 8)
> #define MVNETA_RXQ_PKT_OFFSET_MASK(offs) ((offs) << 8)
> #define MVNETA_RXQ_THRESHOLD_REG(q) (0x14c0 + ((q) << 2))
> --
> 1.8.3.1
>
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
On 21/11/15 23:53, Marcin Wojtas wrote:
>
> 4. Buffer manager (BM) support with two preparatory commits. As it is a
> separate block, common for all network ports, a new driver is introduced,
> which configures it and exposes API to the main network driver. It is
> throughly described in binding documentation and commit log. Please note,
> that enabling per-port BM usage is done using phandle and the data passed
> in mvneta_bm_probe. It is designed for usage of on-demand device probe
> and dev_set/get_drvdata, however it's awaiting merge to linux-next.
> Therefore, deferring probe is not used - if something goes wrong (same
> in case of errors during changing MTU or suspend/resume cycle) mvneta
> driver falls back to software buffer management and works in a regular way.
Looking at your patches, it was not entirely clear to me how the buffer
manager on these Marvell SoCs work, but other networking products have
something similar, like Broadcom's Cable Modem SoCs (BCM33xx) FPM, and
maybe Freescale's FMAN/DPAA seems to do something similar.
Does the buffer manager allocation work by giving you a reference/token
to a buffer as opposed to its address? If that is the case, it would be
good to design support for such hardware in a way that it can be used by
more drivers.
Eric Dumazet suggested a while ago to me that you could get abstract
such allocation using hardware-assisted buffer allocation by either
introducing a new mm zone (instead of ZONE_NORMAL/DMA/HIGHMEM etc.), or
using a different NUMA node id, such that SKB allocation and freeing
helpers could deal with the specifics, and your networking stack and
driver would be mostly unaware of the buffer manager underlying
implementation. The purpose would be to get a 'struct page' reference to
your buffer pool allocation object, so it becomes mostly transparent to
other areas of the kernel, and you could further specialize everything
that needs to be based on this node id or zone.
Finally, these hardware-assisted allocation schemes typically work very
well when there is a forwarding/routing workload involved, because you
can easily steal packets and SKBs from the network stack, but that does
not necessarily play nicely with host-terminated/initiated traffic which
wants to have good feedback on what's happening at the NIC level
(queueing, buffering, etc.).
>
> Known issues:
> - problems with obtaining all mapped buffers from internal SRAM, when
> destroying the buffer pointer pool
> - problems with unmapping chunk of SRAM during driver removal
> Above do not have an impact on the operation, as they are called during
> driver removal or in error path.
Humm, what is the reason for using the on-chip SRAM here, is it because
that's the only storage location the Buffer Manager can allocate from,
or is it because it is presumably faster or with constant access times
than DRAM? Would be nice to explain a bit more in details how the buffer
manager works and its interfacing with the network controllers.
Can I use the buffer manager with other peripherals as well? Like if I
wanted to do zero-copy or hardware-assisted memcpy DMA, would that be a
suitable scheme?
Thanks!
--
Florian
Hi Marcin,
On dim., nov. 22 2015, Marcin Wojtas <[email protected]> wrote:
> This commit introduces suspend/resume routines used for both in 'standby'
> and 'mem' modes. For the latter, in which registers' contents are lost,
> following steps are performed:
> * in suspend - update port statistics and, if interface is running,
> detach netif, clean the queues, disable cpu notifier, shutdown
> interface and reset port's link status;
> * in resume, for all interfaces, set default configuration of the port and
> MBUS windows;
> * in resume, in case the interface is running, enable accepting packets in
> legacy parser, power up the port, register cpu notifier and attach
> netif.
I don't have the hardware to use suspend to RAM on Armada XP but at
least in standby the network continue to work on wakeup.
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> ---
> drivers/net/ethernet/marvell/mvneta.c | 70 +++++++++++++++++++++++++++++++++++
> 1 file changed, 70 insertions(+)
>
> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> index d12b8c6..f079b13 100644
> --- a/drivers/net/ethernet/marvell/mvneta.c
> +++ b/drivers/net/ethernet/marvell/mvneta.c
> @@ -3442,6 +3442,72 @@ static int mvneta_remove(struct platform_device *pdev)
> return 0;
> }
>
> +#ifdef CONFIG_PM_SLEEP
> +static int mvneta_suspend(struct platform_device *pdev, pm_message_t state)
> +{
> + struct net_device *dev = platform_get_drvdata(pdev);
> + struct mvneta_port *pp = netdev_priv(dev);
> +
> + mvneta_ethtool_update_stats(pp);
> +
> + if (!netif_running(dev))
> + return 0;
> +
> + netif_device_detach(dev);
> +
> + mvneta_stop_dev(pp);
> + unregister_cpu_notifier(&pp->cpu_notifier);
> + mvneta_cleanup_rxqs(pp);
> + mvneta_cleanup_txqs(pp);
> +
> + /* Reset link status */
> + pp->link = 0;
> + pp->duplex = -1;
> + pp->speed = 0;
> +
Would it be possible to add a kind of mvneta_port_power_down(pp,
pp->phy_interface) here ? It would allow to save power in standby mode
too.
> + return 0;
> +}
> +
> +static int mvneta_resume(struct platform_device *pdev)
> +{
> + const struct mbus_dram_target_info *dram_target_info;
> + struct net_device *dev = platform_get_drvdata(pdev);
> + struct mvneta_port *pp = netdev_priv(dev);
> + int ret;
> +
> + mvneta_defaults_set(pp);
> + mvneta_port_power_up(pp, pp->phy_interface);
> +
> + dram_target_info = mv_mbus_dram_info();
> + if (dram_target_info)
> + mvneta_conf_mbus_windows(pp, dram_target_info);
> +
> + if (!netif_running(dev))
> + return 0;
> +
> + ret = mvneta_setup_rxqs(pp);
> + if (ret) {
> + netdev_err(dev, "unable to setup rxqs after resume\n");
> + return ret;
> + }
> +
> + ret = mvneta_setup_txqs(pp);
> + if (ret) {
> + netdev_err(dev, "unable to setup txqs after resume\n");
> + return ret;
> + }
> +
> + mvneta_set_rx_mode(dev);
> + mvneta_percpu_elect(pp);
> + register_cpu_notifier(&pp->cpu_notifier);
> + mvneta_start_dev(pp);
> +
> + netif_device_attach(dev);
> +
> + return 0;
> +}
> +#endif /* CONFIG_PM_SLEEP */
> +
> static const struct of_device_id mvneta_match[] = {
> { .compatible = "marvell,armada-370-neta" },
> { .compatible = "marvell,armada-xp-neta" },
> @@ -3452,6 +3518,10 @@ MODULE_DEVICE_TABLE(of, mvneta_match);
> static struct platform_driver mvneta_driver = {
> .probe = mvneta_probe,
> .remove = mvneta_remove,
> +#ifdef CONFIG_PM_SLEEP
> + .suspend = mvneta_suspend,
> + .resume = mvneta_resume,
> +#endif
> .driver = {
> .name = MVNETA_DRIVER_NAME,
> .of_match_table = mvneta_match,
> --
> 1.8.3.1
>
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi Marcin,
On Sun, Nov 22, 2015 at 08:53:52AM +0100, Marcin Wojtas wrote:
> Mixed approach allows using higher interrupt threshold (increased back to
> 15 packets), useful in high throughput. In case of small amount of data
> or very short TX queues HR timer ensures releasing buffers with small
> latency.
>
> Along with existing tx_done processing by coalescing interrupts this
> commit enables triggering HR timer each time the packets are sent.
> Time threshold can also be configured, using ethtool.
>
> Signed-off-by: Marcin Wojtas <[email protected]>
> Signed-off-by: Simon Guinot <[email protected]>
> ---
> drivers/net/ethernet/marvell/mvneta.c | 89 +++++++++++++++++++++++++++++++++--
> 1 file changed, 85 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> index 9c9e858..f5acaf6 100644
> --- a/drivers/net/ethernet/marvell/mvneta.c
> +++ b/drivers/net/ethernet/marvell/mvneta.c
> @@ -21,6 +21,8 @@
> #include <linux/module.h>
> #include <linux/interrupt.h>
> #include <linux/if_vlan.h>
> +#include <linux/hrtimer.h>
> +#include <linux/ktime.h>
ktime.h is already included by hrtimer.h.
> #include <net/ip.h>
> #include <net/ipv6.h>
> #include <linux/io.h>
> @@ -226,7 +228,8 @@
> /* Various constants */
>
> /* Coalescing */
> -#define MVNETA_TXDONE_COAL_PKTS 1
> +#define MVNETA_TXDONE_COAL_PKTS 15
> +#define MVNETA_TXDONE_COAL_USEC 100
Maybe we should keep the default configuration and let the user choose
to enable (or not) this feature ?
> #define MVNETA_RX_COAL_PKTS 32
> #define MVNETA_RX_COAL_USEC 100
>
> @@ -356,6 +359,11 @@ struct mvneta_port {
> struct net_device *dev;
> struct notifier_block cpu_notifier;
>
> + /* Egress finalization */
> + struct tasklet_struct tx_done_tasklet;
> + struct hrtimer tx_done_timer;
> + bool timer_scheduled;
I think we could use hrtimer_is_queued() instead of introducing a new
variable.
[...]
Simon
Hi Gregory,
>> + /* Reset link status */
>> + pp->link = 0;
>> + pp->duplex = -1;
>> + pp->speed = 0;
>> +
>
> Would it be possible to add a kind of mvneta_port_power_down(pp,
> pp->phy_interface) here ? It would allow to save power in standby mode
> too.
>
Sure, I'll try it.
Best regards,
Marcin
Hi Florian,
>
> Looking at your patches, it was not entirely clear to me how the buffer
> manager on these Marvell SoCs work, but other networking products have
> something similar, like Broadcom's Cable Modem SoCs (BCM33xx) FPM, and
> maybe Freescale's FMAN/DPAA seems to do something similar.
>
> Does the buffer manager allocation work by giving you a reference/token
> to a buffer as opposed to its address? If that is the case, it would be
> good to design support for such hardware in a way that it can be used by
> more drivers.
It does not operate on a reference/token but buffer pointers (physical
adresses). It's a ring and you cannot control which buffer will be
taken at given moment.
>
> Eric Dumazet suggested a while ago to me that you could get abstract
> such allocation using hardware-assisted buffer allocation by either
> introducing a new mm zone (instead of ZONE_NORMAL/DMA/HIGHMEM etc.), or
> using a different NUMA node id, such that SKB allocation and freeing
> helpers could deal with the specifics, and your networking stack and
> driver would be mostly unaware of the buffer manager underlying
> implementation. The purpose would be to get a 'struct page' reference to
> your buffer pool allocation object, so it becomes mostly transparent to
> other areas of the kernel, and you could further specialize everything
> that needs to be based on this node id or zone.
As this buffer manager is pretty tightly coupled with NIC (please see
below) and the solution is very platform specific, I'm not sure if it
wouldn't be an overdesign, to provide such generic, paralel to DMA
mechanism.
>
> Finally, these hardware-assisted allocation schemes typically work very
> well when there is a forwarding/routing workload involved, because you
> can easily steal packets and SKBs from the network stack, but that does
> not necessarily play nicely with host-terminated/initiated traffic which
> wants to have good feedback on what's happening at the NIC level
> (queueing, buffering, etc.).
Sure, I can imagine developing some applications that are developed on
top of the proposed patches, but I'm not sure if such things like
cutting network stack in half should be a part of original support.
>
>>
>> Known issues:
>> - problems with obtaining all mapped buffers from internal SRAM, when
>> destroying the buffer pointer pool
>> - problems with unmapping chunk of SRAM during driver removal
>> Above do not have an impact on the operation, as they are called during
>> driver removal or in error path.
>
> Humm, what is the reason for using the on-chip SRAM here, is it because
> that's the only storage location the Buffer Manager can allocate from,
> or is it because it is presumably faster or with constant access times
> than DRAM? Would be nice to explain a bit more in details how the buffer
> manager works and its interfacing with the network controllers.
Each pool of pointers is a ring maintained in DRAM (called buffer
pointers' pool external). SRAM (called buffer pointers' pool internal
memory, BPPI) ensures smaller latency, but is also the only way to
allocate/fetch buffer pointers from DRAM ring. Transfers between those
two memories are controlled by buffer manager itself.
In the beginning the external pool has to be filled with desired
amount of pointers. NIC (controlled by mvneta driver) has to be
informed, which pools it can use for longer and shorter packets, their
size and also SRAM physical address has to be written to one of NETA
registers. Moreover, in order to be able to provide direct access
between NETA and buffer manager SRAM, special, Marvell-specific
settings have to be configured (so called opening of MBUS window).
After enabling ingress, incoming packet is automatically placed in
next-to-be-used buffer from buffer manager resources and the
controller updates NIC's descriptor contents with pool's number and
buffer addresses. Once the packet is processed, a new buffer has to be
allocated and it's address written to SRAM - this way the pool of
pointers gets refilled.
>
> Can I use the buffer manager with other peripherals as well? Like if I
> wanted to do zero-copy or hardware-assisted memcpy DMA, would that be a
> suitable scheme?
Other peripherals cannot access SRAM directly - only DMA-based access
to DRAM. If one would like to access buffers using SRAM from other
drivers, it has to be done by read/write operations performed by CPU.
Moreover I see a limitation, which is a lack of control over the
current buffer index.
Best regards,
Marcin
From: Marcin Wojtas <[email protected]>
Date: Sun, 29 Nov 2015 14:21:35 +0100
>> Looking at your patches, it was not entirely clear to me how the buffer
>> manager on these Marvell SoCs work, but other networking products have
>> something similar, like Broadcom's Cable Modem SoCs (BCM33xx) FPM, and
>> maybe Freescale's FMAN/DPAA seems to do something similar.
>>
>> Does the buffer manager allocation work by giving you a reference/token
>> to a buffer as opposed to its address? If that is the case, it would be
>> good to design support for such hardware in a way that it can be used by
>> more drivers.
>
> It does not operate on a reference/token but buffer pointers (physical
> adresses). It's a ring and you cannot control which buffer will be
> taken at given moment.
He understands this, he's asking you to make an "abstraction".
FWIW, I know of at least one more chip that operates this way too and
the code I wrote for it, particularly the buffer management, took a
while to solidify. Common helpers for this kind of situation would
have helped me back when I wrote it.
Hi David and Florian,
2015-11-30 3:02 GMT+01:00 David Miller <[email protected]>:
> From: Marcin Wojtas <[email protected]>
> Date: Sun, 29 Nov 2015 14:21:35 +0100
>
>>> Looking at your patches, it was not entirely clear to me how the buffer
>>> manager on these Marvell SoCs work, but other networking products have
>>> something similar, like Broadcom's Cable Modem SoCs (BCM33xx) FPM, and
>>> maybe Freescale's FMAN/DPAA seems to do something similar.
>>>
>>> Does the buffer manager allocation work by giving you a reference/token
>>> to a buffer as opposed to its address? If that is the case, it would be
>>> good to design support for such hardware in a way that it can be used by
>>> more drivers.
>>
>> It does not operate on a reference/token but buffer pointers (physical
>> adresses). It's a ring and you cannot control which buffer will be
>> taken at given moment.
>
> He understands this, he's asking you to make an "abstraction".
I assumed that Florian is not familiar with how the HW works,
otherwise why did he ask about the details of operation and the buffer
representation in ring (token vs. address)? Nevertheless, let's talk
about the "abstraction" itself.
>
> FWIW, I know of at least one more chip that operates this way too and
> the code I wrote for it, particularly the buffer management, took a
> while to solidify. Common helpers for this kind of situation would
> have helped me back when I wrote it.
What kind of abstraction and helpers do you mean? Some kind of API
(e.g. bm_alloc_buffer, bm_initialize_ring bm_put_buffer,
bm_get_buffer), which would be used by platform drivers (and specific
aplications if one wants to develop on top of the kernel)?
In general, what is your top-view of such solution and its cooperation
with the drivers?
I'm also wondering how to satisfy different types of HW. For example
buffer managers used by mvneta and mvpp2 are similar, but the major
difference is the way of accessing the buffers (via SRAM in mvneta vs
indirectly via registers in mvpp2) - do you think some kind of
callbacks is a solution, also with other vendors taken into
consideration?
Best regards,
Marcin
Hi Simon,
2015-11-26 17:45 GMT+01:00 Simon Guinot <[email protected]>:
> Hi Marcin,
>
> On Sun, Nov 22, 2015 at 08:53:52AM +0100, Marcin Wojtas wrote:
>> Mixed approach allows using higher interrupt threshold (increased back to
>> 15 packets), useful in high throughput. In case of small amount of data
>> or very short TX queues HR timer ensures releasing buffers with small
>> latency.
>>
>> Along with existing tx_done processing by coalescing interrupts this
>> commit enables triggering HR timer each time the packets are sent.
>> Time threshold can also be configured, using ethtool.
>>
>> Signed-off-by: Marcin Wojtas <[email protected]>
>> Signed-off-by: Simon Guinot <[email protected]>
>> ---
>> drivers/net/ethernet/marvell/mvneta.c | 89 +++++++++++++++++++++++++++++++++--
>> 1 file changed, 85 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
>> index 9c9e858..f5acaf6 100644
>> --- a/drivers/net/ethernet/marvell/mvneta.c
>> +++ b/drivers/net/ethernet/marvell/mvneta.c
>> @@ -21,6 +21,8 @@
>> #include <linux/module.h>
>> #include <linux/interrupt.h>
>> #include <linux/if_vlan.h>
>> +#include <linux/hrtimer.h>
>> +#include <linux/ktime.h>
>
> ktime.h is already included by hrtimer.h.
>
>> #include <net/ip.h>
>> #include <net/ipv6.h>
>> #include <linux/io.h>
>> @@ -226,7 +228,8 @@
>> /* Various constants */
>>
>> /* Coalescing */
>> -#define MVNETA_TXDONE_COAL_PKTS 1
>> +#define MVNETA_TXDONE_COAL_PKTS 15
>> +#define MVNETA_TXDONE_COAL_USEC 100
>
> Maybe we should keep the default configuration and let the user choose
> to enable (or not) this feature ?
I think that this feature should be enabled by default, same as in RX
(which is enabled by HW in ingress). It satisfies all kinds of traffic
or queues sizes. I'd prefer a situation that if someone really wants
to disable it (even if I don't know the possible justification), then
let him use ethtool for this purpose.
>
>> #define MVNETA_RX_COAL_PKTS 32
>> #define MVNETA_RX_COAL_USEC 100
>>
>> @@ -356,6 +359,11 @@ struct mvneta_port {
>> struct net_device *dev;
>> struct notifier_block cpu_notifier;
>>
>> + /* Egress finalization */
>> + struct tasklet_struct tx_done_tasklet;
>> + struct hrtimer tx_done_timer;
>> + bool timer_scheduled;
>
> I think we could use hrtimer_is_queued() instead of introducing a new
> variable.
>
Good point, i'll try that.
Best regards,
Marcin
From: Marcin Wojtas <[email protected]>
Date: Mon, 30 Nov 2015 15:13:22 +0100
> What kind of abstraction and helpers do you mean? Some kind of API
> (e.g. bm_alloc_buffer, bm_initialize_ring bm_put_buffer,
> bm_get_buffer), which would be used by platform drivers (and specific
> aplications if one wants to develop on top of the kernel)?
>
> In general, what is your top-view of such solution and its cooperation
> with the drivers?
The tricky parts involved have to do with allocating pages for the
buffer pools and minimizing the number of atomic refcounting
operations on those pages for for the puts and gets, particularly
around buffer replenish runs.
For example, if you're allocating a page for a buffer pool the device
will chop into N (for any N < PAGE_SIZE) byte pieces, you can
eliminate many atomic operations.
Hi Marcin,
On dim., nov. 22 2015, Marcin Wojtas <[email protected]> wrote:
> Hi,
>
> Hereby I submit a patchset that introduces various fixes and support
> for new features and enhancements to the mvneta driver:
>
> 1. First three patches are minimal fixes, stable-CC'ed.
>
> 2. Suspend to ram ('s2ram') support. Due to some stability problems
> Thomas Petazzoni's patches did not get merged yet, but I used them for
> verification. Contrary to wfi mode ('standby' - linux does not
> differentiate between them, so same routines are used) all registers'
> contents are lost due to power down, so the configuration has to be
> fully reconstructed during resume.
>
> 3. Optimisations - concatenating TX descriptors' flush, basing on
> xmit_more support and combined approach for finalizing egress processing.
> Thanks to HR timer buffers can be released with small latency, which is
> good for low transfer and small queues. Along with the timer, coalescing
> irqs are used, whose threshold could be increased back to 15.
>
> 4. Buffer manager (BM) support with two preparatory commits. As it is a
> separate block, common for all network ports, a new driver is introduced,
> which configures it and exposes API to the main network driver. It is
> throughly described in binding documentation and commit log. Please note,
> that enabling per-port BM usage is done using phandle and the data passed
> in mvneta_bm_probe. It is designed for usage of on-demand device probe
> and dev_set/get_drvdata, however it's awaiting merge to linux-next.
> Therefore, deferring probe is not used - if something goes wrong (same
> in case of errors during changing MTU or suspend/resume cycle) mvneta
> driver falls back to software buffer management and works in a regular way.
>
> Known issues:
> - problems with obtaining all mapped buffers from internal SRAM, when
> destroying the buffer pointer pool
> - problems with unmapping chunk of SRAM during driver removal
> Above do not have an impact on the operation, as they are called during
> driver removal or in error path.
>
> 5. Enable BM on Armada XP and 38X development boards - those ones and
> A370 I could check on my own. In all cases they survived night-long
> linerate iperf. Also tests were performed with A388 SoC working as a
> network bridge between two packet generators. They showed increase of
> maximum processed 64B packets by ~20k (~555k packets with BM enabled
> vs ~535 packets without BM). Also when pushing 1500B-packets with a
> line rate achieved, CPU load decreased from around 25% without BM vs
> 18-20% with BM.
I was trying to test the BM part of tour series on the Armada XP GP
board. However it failed very quickly during the pool allocation. After
a first debug I found that the size of the cs used in the
mvebu_mbus_dram_info struct was 0. I have applied your series on a
v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
mbus driver, a misconfiguration on my side or something else.
Does it ring a bell for you?
How do you test test it exactly?
Especially on which kernel and with which U-Boot?
Thanks,
Gregory
>
> I'm looking forward to any remarks and comments.
>
> Best regards,
> Marcin Wojtas
>
> Marcin Wojtas (12):
> net: mvneta: add configuration for MBUS windows access protection
> net: mvneta: enable IP checksum with jumbo frames for Armada 38x on
> Port0
> net: mvneta: fix bit assignment in MVNETA_RXQ_CONFIG_REG
> net: mvneta: enable suspend/resume support
> net: mvneta: enable mixed egress processing using HR timer
> bus: mvebu-mbus: provide api for obtaining IO and DRAM window
> information
> ARM: mvebu: enable SRAM support in mvebu_v7_defconfig
> net: mvneta: bm: add support for hardware buffer management
> ARM: mvebu: add buffer manager nodes to armada-38x.dtsi
> ARM: mvebu: enable buffer manager support on Armada 38x boards
> ARM: mvebu: add buffer manager nodes to armada-xp.dtsi
> ARM: mvebu: enable buffer manager support on Armada XP boards
>
> Simon Guinot (1):
> net: mvneta: add xmit_more support
>
> .../bindings/net/marvell-armada-370-neta.txt | 19 +-
> .../devicetree/bindings/net/marvell-neta-bm.txt | 49 ++
> arch/arm/boot/dts/armada-385-db-ap.dts | 20 +-
> arch/arm/boot/dts/armada-388-db.dts | 17 +-
> arch/arm/boot/dts/armada-388-gp.dts | 17 +-
> arch/arm/boot/dts/armada-38x.dtsi | 20 +-
> arch/arm/boot/dts/armada-xp-db.dts | 19 +-
> arch/arm/boot/dts/armada-xp-gp.dts | 19 +-
> arch/arm/boot/dts/armada-xp.dtsi | 18 +
> arch/arm/configs/mvebu_v7_defconfig | 1 +
> drivers/bus/mvebu-mbus.c | 51 ++
> drivers/net/ethernet/marvell/Kconfig | 14 +
> drivers/net/ethernet/marvell/Makefile | 1 +
> drivers/net/ethernet/marvell/mvneta.c | 660 +++++++++++++++++++--
> drivers/net/ethernet/marvell/mvneta_bm.c | 642 ++++++++++++++++++++
> drivers/net/ethernet/marvell/mvneta_bm.h | 171 ++++++
> include/linux/mbus.h | 3 +
> 17 files changed, 1677 insertions(+), 64 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/net/marvell-neta-bm.txt
> create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.c
> create mode 100644 drivers/net/ethernet/marvell/mvneta_bm.h
>
> --
> 1.8.3.1
>
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi Gregory,
2015-11-30 18:16 GMT+01:00 Gregory CLEMENT <[email protected]>:
> Hi Marcin,
>
> On dim., nov. 22 2015, Marcin Wojtas <[email protected]> wrote:
>
>> Hi,
>>
>> Hereby I submit a patchset that introduces various fixes and support
>> for new features and enhancements to the mvneta driver:
>>
>> 1. First three patches are minimal fixes, stable-CC'ed.
>>
>> 2. Suspend to ram ('s2ram') support. Due to some stability problems
>> Thomas Petazzoni's patches did not get merged yet, but I used them for
>> verification. Contrary to wfi mode ('standby' - linux does not
>> differentiate between them, so same routines are used) all registers'
>> contents are lost due to power down, so the configuration has to be
>> fully reconstructed during resume.
>>
>> 3. Optimisations - concatenating TX descriptors' flush, basing on
>> xmit_more support and combined approach for finalizing egress processing.
>> Thanks to HR timer buffers can be released with small latency, which is
>> good for low transfer and small queues. Along with the timer, coalescing
>> irqs are used, whose threshold could be increased back to 15.
>>
>> 4. Buffer manager (BM) support with two preparatory commits. As it is a
>> separate block, common for all network ports, a new driver is introduced,
>> which configures it and exposes API to the main network driver. It is
>> throughly described in binding documentation and commit log. Please note,
>> that enabling per-port BM usage is done using phandle and the data passed
>> in mvneta_bm_probe. It is designed for usage of on-demand device probe
>> and dev_set/get_drvdata, however it's awaiting merge to linux-next.
>> Therefore, deferring probe is not used - if something goes wrong (same
>> in case of errors during changing MTU or suspend/resume cycle) mvneta
>> driver falls back to software buffer management and works in a regular way.
>>
>> Known issues:
>> - problems with obtaining all mapped buffers from internal SRAM, when
>> destroying the buffer pointer pool
>> - problems with unmapping chunk of SRAM during driver removal
>> Above do not have an impact on the operation, as they are called during
>> driver removal or in error path.
>>
>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>> A370 I could check on my own. In all cases they survived night-long
>> linerate iperf. Also tests were performed with A388 SoC working as a
>> network bridge between two packet generators. They showed increase of
>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>> line rate achieved, CPU load decreased from around 25% without BM vs
>> 18-20% with BM.
>
> I was trying to test the BM part of tour series on the Armada XP GP
> board. However it failed very quickly during the pool allocation. After
> a first debug I found that the size of the cs used in the
> mvebu_mbus_dram_info struct was 0. I have applied your series on a
> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
> mbus driver, a misconfiguration on my side or something else.
>
> Does it ring a bell for you?
Frankly, I'm a bit surprised, I've never seen such problems on any of
the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
function exit with an error? Can you please apply below diff:
http://pastebin.com/2ws1txWk
And send me a full log beginning from u-boot?
>
> How do you test test it exactly?
> Especially on which kernel and with which U-Boot?
>
I've just re-built the patchset I sent, which is on top of 4.4-rc1.
I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
u-boot version: v2011.12 2014_T2.0_eng_dropv2.
Best regards,
Marcin
Hi Marcin,
On lun., nov. 30 2015, Marcin Wojtas <[email protected]> wrote:
[...]
>>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>>> A370 I could check on my own. In all cases they survived night-long
>>> linerate iperf. Also tests were performed with A388 SoC working as a
>>> network bridge between two packet generators. They showed increase of
>>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>>> line rate achieved, CPU load decreased from around 25% without BM vs
>>> 18-20% with BM.
>>
>> I was trying to test the BM part of tour series on the Armada XP GP
>> board. However it failed very quickly during the pool allocation. After
>> a first debug I found that the size of the cs used in the
>> mvebu_mbus_dram_info struct was 0. I have applied your series on a
>> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
>> mbus driver, a misconfiguration on my side or something else.
>>
>> Does it ring a bell for you?
>
> Frankly, I'm a bit surprised, I've never seen such problems on any of
> the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
> function exit with an error? Can you please apply below diff:
> http://pastebin.com/2ws1txWk
Yes it exited with errors and I added the same kind traces. It was how I
knew that the size was 0!
I've just rebuild a fresh kernel using mvebu_v7_defconfig and adding
your patch, I got the same issue (see the log at the end of the email.)
But the good news is that on the same kernel on Armada 388 GP the pool
allocation does not fail. I really suspect an issue with my u-boot.
> And send me a full log beginning from u-boot?
>
>>
>> How do you test test it exactly?
>> Especially on which kernel and with which U-Boot?
>>
>
> I've just re-built the patchset I sent, which is on top of 4.4-rc1.
>
> I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
> u-boot version: v2011.12 2014_T2.0_eng_dropv2.
My config is AXP-GP, 78460 @ 1300MHz, 8GB DRAM (only 4GB are used
because I didn't activated LPAE), but the main difference is the U-Boot
version: v2011.12 2014_T2.eng_dropv1.ATAG-test02.
Thanks,
Gregory
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.4.0-rc1-00013-g76f111f9bdf8-dirty (gclement@FE-laptop) (gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu1) ) #1024 SMP Tue Dec 1 14:02:52 CET 2015
[ 0.000000] CPU: ARMv7 Processor [562f5842] revision 2 (ARMv7), cr=10c5387d
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] Machine model: Marvell Armada XP Development Board DB-MV784MP-GP
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] PERCPU: Embedded 12 pages/cpu @ee1ac000 s18752 r8192 d22208 u49152
[ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 981504
[ 0.000000] Kernel command line: console=ttyS0,115200 earlyprintk mvneta.rxq_def=2
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
[ 0.000000] log_buf_len min size: 16384 bytes
[ 0.000000] log_buf_len: 32768 bytes
[ 0.000000] early log buf free: 14924(91%)
[ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Memory: 3888204K/3932160K available (5576K kernel code, 251K rwdata, 1544K rodata, 4460K init, 207K bss, 43956K reserved, 0K cma-reserved, 3145728K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0008000 - 0xc06fc374 (7121 kB)
[ 0.000000] .init : 0xc06fd000 - 0xc0b58000 (4460 kB)
[ 0.000000] .data : 0xc0b58000 - 0xc0b96d00 ( 252 kB)
[ 0.000000] .bss : 0xc0b96d00 - 0xc0bcaa58 ( 208 kB)
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] Build-time adjustment of leaf fanout to 32.
[ 0.000000] NR_IRQS:16 nr_irqs:16 16
[ 0.000000] L2C: DT/platform modifies aux control register: 0x1a69ef10 -> 0x1a69ef12
[ 0.000000] Aurora cache controller enabled, 32 ways, 2048 kB
[ 0.000000] Aurora: CACHE_ID 0x00000100, AUX_CTRL 0x1a69ef12
[ 0.000000] Switching to timer-based delay loop, resolution 40ns
[ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
[ 0.000015] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
[ 0.000305] Console: colour dummy device 80x30
[ 0.000322] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)
[ 0.000332] pid_max: default: 32768 minimum: 301
[ 0.000417] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000425] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000737] CPU: Testing write buffer coherency: ok
[ 0.000880] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[ 0.001006] Setting up static identity map for 0x8280 - 0x82d8
[ 0.001224] mvebu-soc-id: MVEBU SoC ID=0x7846, Rev=0x2
[ 0.001321] mvebu-pmsu: Initializing Power Management Service Unit
[ 0.002062] Booting CPU 1
[ 0.004402] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[ 0.004645] Booting CPU 2
[ 0.006984] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
[ 0.007219] Booting CPU 3
[ 0.009558] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
[ 0.009605] Brought up 4 CPUs
[ 0.009615] SMP: Total of 4 processors activated (200.00 BogoMIPS).
[ 0.009620] CPU: All CPU(s) started in SVC mode.
[ 0.009957] devtmpfs: initialized
[ 0.012369] VFP support v0.3: implementor 56 architecture 2 part 20 variant 9 rev 6
[ 0.012525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.012657] pinctrl core: initialized pinctrl subsystem
[ 0.013188] NET: Registered protocol family 16
[ 0.013368] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.040064] cpuidle: using governor ladder
[ 0.070061] cpuidle: using governor menu
[ 0.075168] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 3
[ 0.075172] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 1
[ 0.075176] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 2
[ 0.075179] hw-breakpoint: CPU 1 failed to disable vector catch
[ 0.075196] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 0
[ 0.110815] vgaarb: loaded
[ 0.110981] SCSI subsystem initialized
[ 0.111306] usbcore: registered new interface driver usbfs
[ 0.111349] usbcore: registered new interface driver hub
[ 0.111398] usbcore: registered new device driver usb
[ 0.111640] Advanced Linux Sound Architecture Driver Initialized.
[ 0.111991] Bluetooth: Core ver 2.21
[ 0.112024] NET: Registered protocol family 31
[ 0.112029] Bluetooth: HCI device and connection manager initialized
[ 0.112040] Bluetooth: HCI socket layer initialized
[ 0.112048] Bluetooth: L2CAP socket layer initialized
[ 0.112067] Bluetooth: SCO socket layer initialized
[ 0.112462] clocksource: Switched to clocksource armada_370_xp_clocksource
[ 0.119267] NET: Registered protocol family 2
[ 0.119629] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
[ 0.119690] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
[ 0.119749] TCP: Hash tables configured (established 8192 bind 8192)
[ 0.119791] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 0.119814] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 0.119945] NET: Registered protocol family 1
[ 0.120143] RPC: Registered named UNIX socket transport module.
[ 0.120150] RPC: Registered udp transport module.
[ 0.120155] RPC: Registered tcp transport module.
[ 0.120160] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.290263] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
[ 0.291305] futex hash table entries: 1024 (order: 4, 65536 bytes)
[ 0.292696] bounce: pool size: 64 pages
[ 0.292746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[ 0.292758] io scheduler noop registered
[ 0.292766] io scheduler deadline registered
[ 0.292779] io scheduler cfq registered (default)
[ 0.293754] armada-xp-pinctrl f1018000.pin-ctrl: registered pinctrl driver
[ 0.294186] irq: Cannot allocate irq_descs @ IRQ48, assuming pre-allocated
[ 0.294451] irq: Cannot allocate irq_descs @ IRQ80, assuming pre-allocated
[ 0.294612] irq: Cannot allocate irq_descs @ IRQ112, assuming pre-allocated
[ 0.295033] mv_xor f1060900.xor: Marvell shared XOR driver
[ 0.332873] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.372848] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.372958] mv_xor f10f0900.xor: Marvell shared XOR driver
[ 0.412850] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.452845] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
[ 0.492634] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 0.493897] console [ttyS0] disabled
[ 0.514007] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 20, base_baud = 15625000) is a 16550A
[ 1.200947] console [ttyS0] enabled
[ 1.225041] f1012100.serial: ttyS1 at MMIO 0xf1012100 (irq = 21, base_baud = 15625000) is a 16550A
[ 1.254648] f1012200.serial: ttyS2 at MMIO 0xf1012200 (irq = 33, base_baud = 15625000) is a 16550A
[ 1.284237] f1012300.serial: ttyS3 at MMIO 0xf1012300 (irq = 34, base_baud = 15625000) is a 16550A
[ 1.294303] sata_mv f10a0000.sata: slots 32 ports 2
[ 1.300098] scsi host0: sata_mv
[ 1.303511] scsi host1: sata_mv
[ 1.306805] ata1: SATA max UDMA/133 irq 31
[ 1.310913] ata2: SATA max UDMA/133 irq 31
[ 1.315482] pxa3xx-nand f10d0000.nand: This platform can't do DMA on this device
[ 1.323280] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0x38
[ 1.329650] nand: Micron MT29F8G08ABABAWP
[ 1.333684] nand: 1024 MiB, SLC, erase size: 512 KiB, page size: 4096, OOB size: 224
[ 1.341450] pxa3xx-nand f10d0000.nand: ECC strength 16, ECC step size 2048
[ 1.349163] Bad block table found at page 261888, version 0x01
[ 1.355385] Bad block table found at page 262016, version 0x01
[ 1.364466] pxa3xx_nand-0: 'partitions' subnode not found on /soc/internal-regs/nand@d0000. Trying to parse direct subnodes as partitions.
[ 1.378215] m25p80 spi0.0: n25q128a13 (16384 Kbytes)
[ 1.385098] spi0.0: 'partitions' subnode not found on /soc/internal-regs/spi@10600/spi-flash@0. Trying to parse direct subnodes as partitions.
[ 1.399136] libphy: Fixed MDIO Bus: probed
[ 1.403396] libphy: orion_mdio_bus: probed
[ 1.415856] mvneta_bm f10c0000.bm: Buffer Manager for network controller enabled
[ 1.424232] size = 0x0, target = 0x0, attr = 0x1e
[ 1.428949] mvebu_mbus: invalid dram address 0x2d194000
[ 1.434204] mvneta_bm f10c0000.bm: fail to create pool 0
[ 1.439532] mvneta f1070000.ethernet eth0: fail to obtain long pool for port
[ 1.446607] mvneta f1070000.ethernet: use SW buffer management
[ 1.452696] mvneta f1070000.ethernet eth0: Using hardware mac address 00:50:43:68:1c:28
[ 1.461529] size = 0x0, target = 0x0, attr = 0x1e
[ 1.466258] mvebu_mbus: invalid dram address 0x2d198000
[ 1.471501] mvneta_bm f10c0000.bm: fail to create pool 1
[ 1.476836] mvneta f1074000.ethernet eth1: fail to obtain long pool for port
[ 1.483910] mvneta f1074000.ethernet: use SW buffer management
[ 1.489994] mvneta f1074000.ethernet eth1: Using random mac address be:14:de:e2:be:cf
[ 1.498661] size = 0x0, target = 0x0, attr = 0x1e
[ 1.503391] mvebu_mbus: invalid dram address 0x2d10c000
[ 1.508633] mvneta_bm f10c0000.bm: fail to create pool 2
[ 1.513968] mvneta f1030000.ethernet eth2: fail to obtain long pool for port
[ 1.521035] mvneta f1030000.ethernet: use SW buffer management
[ 1.527121] mvneta f1030000.ethernet eth2: Using random mac address 86:3c:f8:02:31:cf
[ 1.535810] size = 0x0, target = 0x0, attr = 0x1e
[ 1.540527] mvebu_mbus: invalid dram address 0x2d752000
[ 1.545781] mvneta_bm f10c0000.bm: fail to create pool 3
[ 1.551108] mvneta f1034000.ethernet eth3: fail to obtain long pool for port
[ 1.558182] mvneta f1034000.ethernet: use SW buffer management
[ 1.564269] mvneta f1034000.ethernet eth3: Using random mac address 7e:3e:74:8c:93:50
[ 1.572297] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 1.578863] ehci-pci: EHCI PCI platform driver
[ 1.583371] ehci-orion: EHCI orion driver
[ 1.587476] orion-ehci f1050000.usb: EHCI Host Controller
[ 1.592916] orion-ehci f1050000.usb: new USB bus registered, assigned bus number 1
[ 1.600567] orion-ehci f1050000.usb: irq 27, io mem 0xf1050000
[ 1.622494] orion-ehci f1050000.usb: USB 2.0 started, EHCI 1.00
[ 1.628967] hub 1-0:1.0: USB hub found
[ 1.632806] hub 1-0:1.0: 1 port detected
[ 1.637018] orion-ehci f1051000.usb: EHCI Host Controller
[ 1.642468] orion-ehci f1051000.usb: new USB bus registered, assigned bus number 2
[ 1.650111] orion-ehci f1051000.usb: irq 28, io mem 0xf1051000
[ 1.672467] orion-ehci f1051000.usb: USB 2.0 started, EHCI 1.00
[ 1.678908] hub 2-0:1.0: USB hub found
[ 1.682736] hub 2-0:1.0: 1 port detected
[ 1.687203] usbcore: registered new interface driver usb-storage
[ 1.693454] mousedev: PS/2 mouse device common for all mice
[ 1.699593] rtc-mv f1010300.rtc: rtc core: registered f1010300.rtc as rtc0
[ 1.706641] i2c /dev entries driver
[ 1.711074] orion_wdt: Initial timeout 171 sec
[ 1.715770] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:0, -19
[ 1.723985] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:1, -19
[ 1.732174] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:2, -19
[ 1.740372] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:3, -19
[ 1.748968] sdhci: Secure Digital Host Controller Interface driver
[ 1.755176] sdhci: Copyright(c) Pierre Ossman
[ 1.759723] sdhci-pltfm: SDHCI platform and OF driver helper
[ 1.765805] usbcore: registered new interface driver usbhid
[ 1.771394] usbhid: USB HID core driver
[ 1.775730] f0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x000018
[ 1.785775] Intel/Sharp Extended Query Table at 0x0031
[ 1.790936] Intel/Sharp Extended Query Table at 0x0031
[ 1.796106] Using buffer write method
[ 1.799778] cfi_cmdset_0001: Erase suspend on write enabled
[ 1.809269] f0000000.nor: 'partitions' subnode not found on /soc/devbus-bootcs/nor@0. Trying to parse direct subnodes as partitions.
[ 1.822513] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
[ 1.829037] NET: Registered protocol family 10
[ 1.834180] sit: IPv6 over IPv4 tunneling driver
[ 1.839209] NET: Registered protocol family 17
[ 1.842687] ata1.00: ATA-9: TS8GHSD310, 20111220, max UDMA/133
[ 1.842692] ata1.00: 15621984 sectors, multi 1: LBA48 NCQ (depth 31/32)
[ 1.856343] ThumbEE CPU extension supported.
[ 1.860632] Registering SWP/SWPB emulation handler
[ 1.862685] ata1.00: configured for UDMA/133
[ 1.862993] scsi 0:0:0:0: Direct-Access ATA TS8GHSD310 1220 PQ: 0 ANSI: 5
[ 1.863646] sd 0:0:0:0: [sda] 15621984 512-byte logical blocks: (7.99 GB/7.44 GiB)
[ 1.863838] sd 0:0:0:0: [sda] Write Protect is off
[ 1.863907] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 1.873801] sda: sda1 sda2 sda3 < sda5 > sda4
[ 1.875389] sd 0:0:0:0: [sda] Attached SCSI disk
[ 1.909334] rtc-mv f1010300.rtc: setting system clock to 2001-01-20 07:57:42 UTC (979977462)
[ 1.918901] ALSA device list:
[ 1.921878] No soundcards found.
[ 2.222505] ata2: SATA link down (SStatus 0 SControl F300)
[ 2.230484] Freeing unused kernel memory: 4460K (c06fd000 - c0b58000)
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi Gregory,
Thanks for the log. I think it may be an overall problem with 4GB size
representation in mvebu_mbus_dram_info structure? Maybe whole DRAM
space is associated to CS0, and the 4GB size (0x1 0000 0000) does not
fit u32 variable?
Best regards,
Marcin
2015-12-01 14:12 GMT+01:00 Gregory CLEMENT <[email protected]>:
> Hi Marcin,
>
> On lun., nov. 30 2015, Marcin Wojtas <[email protected]> wrote:
> [...]
>>>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>>>> A370 I could check on my own. In all cases they survived night-long
>>>> linerate iperf. Also tests were performed with A388 SoC working as a
>>>> network bridge between two packet generators. They showed increase of
>>>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>>>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>>>> line rate achieved, CPU load decreased from around 25% without BM vs
>>>> 18-20% with BM.
>>>
>>> I was trying to test the BM part of tour series on the Armada XP GP
>>> board. However it failed very quickly during the pool allocation. After
>>> a first debug I found that the size of the cs used in the
>>> mvebu_mbus_dram_info struct was 0. I have applied your series on a
>>> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
>>> mbus driver, a misconfiguration on my side or something else.
>>>
>>> Does it ring a bell for you?
>>
>> Frankly, I'm a bit surprised, I've never seen such problems on any of
>> the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
>> function exit with an error? Can you please apply below diff:
>> http://pastebin.com/2ws1txWk
>
> Yes it exited with errors and I added the same kind traces. It was how I
> knew that the size was 0!
>
> I've just rebuild a fresh kernel using mvebu_v7_defconfig and adding
> your patch, I got the same issue (see the log at the end of the email.)
>
>
> But the good news is that on the same kernel on Armada 388 GP the pool
> allocation does not fail. I really suspect an issue with my u-boot.
>
>
>> And send me a full log beginning from u-boot?
>>
>>>
>>> How do you test test it exactly?
>>> Especially on which kernel and with which U-Boot?
>>>
>>
>> I've just re-built the patchset I sent, which is on top of 4.4-rc1.
>>
>> I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
>> u-boot version: v2011.12 2014_T2.0_eng_dropv2.
>
> My config is AXP-GP, 78460 @ 1300MHz, 8GB DRAM (only 4GB are used
> because I didn't activated LPAE), but the main difference is the U-Boot
> version: v2011.12 2014_T2.eng_dropv1.ATAG-test02.
>
> Thanks,
>
> Gregory
>
>
> [ 0.000000] Booting Linux on physical CPU 0x0
> [ 0.000000] Linux version 4.4.0-rc1-00013-g76f111f9bdf8-dirty (gclement@FE-laptop) (gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu1) ) #1024 SMP Tue Dec 1 14:02:52 CET 2015
> [ 0.000000] CPU: ARMv7 Processor [562f5842] revision 2 (ARMv7), cr=10c5387d
> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
> [ 0.000000] Machine model: Marvell Armada XP Development Board DB-MV784MP-GP
> [ 0.000000] Memory policy: Data cache writealloc
> [ 0.000000] PERCPU: Embedded 12 pages/cpu @ee1ac000 s18752 r8192 d22208 u49152
> [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 981504
> [ 0.000000] Kernel command line: console=ttyS0,115200 earlyprintk mvneta.rxq_def=2
> [ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
> [ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
> [ 0.000000] log_buf_len min size: 16384 bytes
> [ 0.000000] log_buf_len: 32768 bytes
> [ 0.000000] early log buf free: 14924(91%)
> [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
> [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
> [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
> [ 0.000000] Memory: 3888204K/3932160K available (5576K kernel code, 251K rwdata, 1544K rodata, 4460K init, 207K bss, 43956K reserved, 0K cma-reserved, 3145728K highmem)
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
> [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
> [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
> [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
> [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
> [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
> [ 0.000000] .text : 0xc0008000 - 0xc06fc374 (7121 kB)
> [ 0.000000] .init : 0xc06fd000 - 0xc0b58000 (4460 kB)
> [ 0.000000] .data : 0xc0b58000 - 0xc0b96d00 ( 252 kB)
> [ 0.000000] .bss : 0xc0b96d00 - 0xc0bcaa58 ( 208 kB)
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] Build-time adjustment of leaf fanout to 32.
> [ 0.000000] NR_IRQS:16 nr_irqs:16 16
> [ 0.000000] L2C: DT/platform modifies aux control register: 0x1a69ef10 -> 0x1a69ef12
> [ 0.000000] Aurora cache controller enabled, 32 ways, 2048 kB
> [ 0.000000] Aurora: CACHE_ID 0x00000100, AUX_CTRL 0x1a69ef12
> [ 0.000000] Switching to timer-based delay loop, resolution 40ns
> [ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
> [ 0.000015] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
> [ 0.000305] Console: colour dummy device 80x30
> [ 0.000322] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)
> [ 0.000332] pid_max: default: 32768 minimum: 301
> [ 0.000417] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
> [ 0.000425] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
> [ 0.000737] CPU: Testing write buffer coherency: ok
> [ 0.000880] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
> [ 0.001006] Setting up static identity map for 0x8280 - 0x82d8
> [ 0.001224] mvebu-soc-id: MVEBU SoC ID=0x7846, Rev=0x2
> [ 0.001321] mvebu-pmsu: Initializing Power Management Service Unit
> [ 0.002062] Booting CPU 1
> [ 0.004402] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
> [ 0.004645] Booting CPU 2
> [ 0.006984] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
> [ 0.007219] Booting CPU 3
> [ 0.009558] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
> [ 0.009605] Brought up 4 CPUs
> [ 0.009615] SMP: Total of 4 processors activated (200.00 BogoMIPS).
> [ 0.009620] CPU: All CPU(s) started in SVC mode.
> [ 0.009957] devtmpfs: initialized
> [ 0.012369] VFP support v0.3: implementor 56 architecture 2 part 20 variant 9 rev 6
> [ 0.012525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [ 0.012657] pinctrl core: initialized pinctrl subsystem
> [ 0.013188] NET: Registered protocol family 16
> [ 0.013368] DMA: preallocated 256 KiB pool for atomic coherent allocations
> [ 0.040064] cpuidle: using governor ladder
> [ 0.070061] cpuidle: using governor menu
> [ 0.075168] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 3
> [ 0.075172] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 1
> [ 0.075176] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 2
> [ 0.075179] hw-breakpoint: CPU 1 failed to disable vector catch
> [ 0.075196] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 0
> [ 0.110815] vgaarb: loaded
> [ 0.110981] SCSI subsystem initialized
> [ 0.111306] usbcore: registered new interface driver usbfs
> [ 0.111349] usbcore: registered new interface driver hub
> [ 0.111398] usbcore: registered new device driver usb
> [ 0.111640] Advanced Linux Sound Architecture Driver Initialized.
> [ 0.111991] Bluetooth: Core ver 2.21
> [ 0.112024] NET: Registered protocol family 31
> [ 0.112029] Bluetooth: HCI device and connection manager initialized
> [ 0.112040] Bluetooth: HCI socket layer initialized
> [ 0.112048] Bluetooth: L2CAP socket layer initialized
> [ 0.112067] Bluetooth: SCO socket layer initialized
> [ 0.112462] clocksource: Switched to clocksource armada_370_xp_clocksource
> [ 0.119267] NET: Registered protocol family 2
> [ 0.119629] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
> [ 0.119690] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
> [ 0.119749] TCP: Hash tables configured (established 8192 bind 8192)
> [ 0.119791] UDP hash table entries: 512 (order: 2, 16384 bytes)
> [ 0.119814] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
> [ 0.119945] NET: Registered protocol family 1
> [ 0.120143] RPC: Registered named UNIX socket transport module.
> [ 0.120150] RPC: Registered udp transport module.
> [ 0.120155] RPC: Registered tcp transport module.
> [ 0.120160] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [ 0.290263] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
> [ 0.291305] futex hash table entries: 1024 (order: 4, 65536 bytes)
> [ 0.292696] bounce: pool size: 64 pages
> [ 0.292746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
> [ 0.292758] io scheduler noop registered
> [ 0.292766] io scheduler deadline registered
> [ 0.292779] io scheduler cfq registered (default)
> [ 0.293754] armada-xp-pinctrl f1018000.pin-ctrl: registered pinctrl driver
> [ 0.294186] irq: Cannot allocate irq_descs @ IRQ48, assuming pre-allocated
> [ 0.294451] irq: Cannot allocate irq_descs @ IRQ80, assuming pre-allocated
> [ 0.294612] irq: Cannot allocate irq_descs @ IRQ112, assuming pre-allocated
> [ 0.295033] mv_xor f1060900.xor: Marvell shared XOR driver
> [ 0.332873] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
> [ 0.372848] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
> [ 0.372958] mv_xor f10f0900.xor: Marvell shared XOR driver
> [ 0.412850] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
> [ 0.452845] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
> [ 0.492634] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
> [ 0.493897] console [ttyS0] disabled
> [ 0.514007] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 20, base_baud = 15625000) is a 16550A
> [ 1.200947] console [ttyS0] enabled
> [ 1.225041] f1012100.serial: ttyS1 at MMIO 0xf1012100 (irq = 21, base_baud = 15625000) is a 16550A
> [ 1.254648] f1012200.serial: ttyS2 at MMIO 0xf1012200 (irq = 33, base_baud = 15625000) is a 16550A
> [ 1.284237] f1012300.serial: ttyS3 at MMIO 0xf1012300 (irq = 34, base_baud = 15625000) is a 16550A
> [ 1.294303] sata_mv f10a0000.sata: slots 32 ports 2
> [ 1.300098] scsi host0: sata_mv
> [ 1.303511] scsi host1: sata_mv
> [ 1.306805] ata1: SATA max UDMA/133 irq 31
> [ 1.310913] ata2: SATA max UDMA/133 irq 31
> [ 1.315482] pxa3xx-nand f10d0000.nand: This platform can't do DMA on this device
> [ 1.323280] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0x38
> [ 1.329650] nand: Micron MT29F8G08ABABAWP
> [ 1.333684] nand: 1024 MiB, SLC, erase size: 512 KiB, page size: 4096, OOB size: 224
> [ 1.341450] pxa3xx-nand f10d0000.nand: ECC strength 16, ECC step size 2048
> [ 1.349163] Bad block table found at page 261888, version 0x01
> [ 1.355385] Bad block table found at page 262016, version 0x01
> [ 1.364466] pxa3xx_nand-0: 'partitions' subnode not found on /soc/internal-regs/nand@d0000. Trying to parse direct subnodes as partitions.
> [ 1.378215] m25p80 spi0.0: n25q128a13 (16384 Kbytes)
> [ 1.385098] spi0.0: 'partitions' subnode not found on /soc/internal-regs/spi@10600/spi-flash@0. Trying to parse direct subnodes as partitions.
> [ 1.399136] libphy: Fixed MDIO Bus: probed
> [ 1.403396] libphy: orion_mdio_bus: probed
> [ 1.415856] mvneta_bm f10c0000.bm: Buffer Manager for network controller enabled
> [ 1.424232] size = 0x0, target = 0x0, attr = 0x1e
> [ 1.428949] mvebu_mbus: invalid dram address 0x2d194000
> [ 1.434204] mvneta_bm f10c0000.bm: fail to create pool 0
> [ 1.439532] mvneta f1070000.ethernet eth0: fail to obtain long pool for port
> [ 1.446607] mvneta f1070000.ethernet: use SW buffer management
> [ 1.452696] mvneta f1070000.ethernet eth0: Using hardware mac address 00:50:43:68:1c:28
> [ 1.461529] size = 0x0, target = 0x0, attr = 0x1e
> [ 1.466258] mvebu_mbus: invalid dram address 0x2d198000
> [ 1.471501] mvneta_bm f10c0000.bm: fail to create pool 1
> [ 1.476836] mvneta f1074000.ethernet eth1: fail to obtain long pool for port
> [ 1.483910] mvneta f1074000.ethernet: use SW buffer management
> [ 1.489994] mvneta f1074000.ethernet eth1: Using random mac address be:14:de:e2:be:cf
> [ 1.498661] size = 0x0, target = 0x0, attr = 0x1e
> [ 1.503391] mvebu_mbus: invalid dram address 0x2d10c000
> [ 1.508633] mvneta_bm f10c0000.bm: fail to create pool 2
> [ 1.513968] mvneta f1030000.ethernet eth2: fail to obtain long pool for port
> [ 1.521035] mvneta f1030000.ethernet: use SW buffer management
> [ 1.527121] mvneta f1030000.ethernet eth2: Using random mac address 86:3c:f8:02:31:cf
> [ 1.535810] size = 0x0, target = 0x0, attr = 0x1e
> [ 1.540527] mvebu_mbus: invalid dram address 0x2d752000
> [ 1.545781] mvneta_bm f10c0000.bm: fail to create pool 3
> [ 1.551108] mvneta f1034000.ethernet eth3: fail to obtain long pool for port
> [ 1.558182] mvneta f1034000.ethernet: use SW buffer management
> [ 1.564269] mvneta f1034000.ethernet eth3: Using random mac address 7e:3e:74:8c:93:50
> [ 1.572297] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ 1.578863] ehci-pci: EHCI PCI platform driver
> [ 1.583371] ehci-orion: EHCI orion driver
> [ 1.587476] orion-ehci f1050000.usb: EHCI Host Controller
> [ 1.592916] orion-ehci f1050000.usb: new USB bus registered, assigned bus number 1
> [ 1.600567] orion-ehci f1050000.usb: irq 27, io mem 0xf1050000
> [ 1.622494] orion-ehci f1050000.usb: USB 2.0 started, EHCI 1.00
> [ 1.628967] hub 1-0:1.0: USB hub found
> [ 1.632806] hub 1-0:1.0: 1 port detected
> [ 1.637018] orion-ehci f1051000.usb: EHCI Host Controller
> [ 1.642468] orion-ehci f1051000.usb: new USB bus registered, assigned bus number 2
> [ 1.650111] orion-ehci f1051000.usb: irq 28, io mem 0xf1051000
> [ 1.672467] orion-ehci f1051000.usb: USB 2.0 started, EHCI 1.00
> [ 1.678908] hub 2-0:1.0: USB hub found
> [ 1.682736] hub 2-0:1.0: 1 port detected
> [ 1.687203] usbcore: registered new interface driver usb-storage
> [ 1.693454] mousedev: PS/2 mouse device common for all mice
> [ 1.699593] rtc-mv f1010300.rtc: rtc core: registered f1010300.rtc as rtc0
> [ 1.706641] i2c /dev entries driver
> [ 1.711074] orion_wdt: Initial timeout 171 sec
> [ 1.715770] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:0, -19
> [ 1.723985] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:1, -19
> [ 1.732174] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:2, -19
> [ 1.740372] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:3, -19
> [ 1.748968] sdhci: Secure Digital Host Controller Interface driver
> [ 1.755176] sdhci: Copyright(c) Pierre Ossman
> [ 1.759723] sdhci-pltfm: SDHCI platform and OF driver helper
> [ 1.765805] usbcore: registered new interface driver usbhid
> [ 1.771394] usbhid: USB HID core driver
> [ 1.775730] f0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x000018
> [ 1.785775] Intel/Sharp Extended Query Table at 0x0031
> [ 1.790936] Intel/Sharp Extended Query Table at 0x0031
> [ 1.796106] Using buffer write method
> [ 1.799778] cfi_cmdset_0001: Erase suspend on write enabled
> [ 1.809269] f0000000.nor: 'partitions' subnode not found on /soc/devbus-bootcs/nor@0. Trying to parse direct subnodes as partitions.
> [ 1.822513] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
> [ 1.829037] NET: Registered protocol family 10
> [ 1.834180] sit: IPv6 over IPv4 tunneling driver
> [ 1.839209] NET: Registered protocol family 17
> [ 1.842687] ata1.00: ATA-9: TS8GHSD310, 20111220, max UDMA/133
> [ 1.842692] ata1.00: 15621984 sectors, multi 1: LBA48 NCQ (depth 31/32)
> [ 1.856343] ThumbEE CPU extension supported.
> [ 1.860632] Registering SWP/SWPB emulation handler
> [ 1.862685] ata1.00: configured for UDMA/133
> [ 1.862993] scsi 0:0:0:0: Direct-Access ATA TS8GHSD310 1220 PQ: 0 ANSI: 5
> [ 1.863646] sd 0:0:0:0: [sda] 15621984 512-byte logical blocks: (7.99 GB/7.44 GiB)
> [ 1.863838] sd 0:0:0:0: [sda] Write Protect is off
> [ 1.863907] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
> [ 1.873801] sda: sda1 sda2 sda3 < sda5 > sda4
> [ 1.875389] sd 0:0:0:0: [sda] Attached SCSI disk
> [ 1.909334] rtc-mv f1010300.rtc: setting system clock to 2001-01-20 07:57:42 UTC (979977462)
> [ 1.918901] ALSA device list:
> [ 1.921878] No soundcards found.
> [ 2.222505] ata2: SATA link down (SStatus 0 SControl F300)
> [ 2.230484] Freeing unused kernel memory: 4460K (c06fd000 - c0b58000)
>
> --
> Gregory Clement, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
Gregory,
Please apply below patch:
http://pastebin.com/t42xyU3i
It will confirm if there's an overflow for CS0 size in your setup.
Please let know.
So far the issue may have been not noticed, because in every IO driver
using mvebu_mbus_dram_info for configuring MBUS windows, there's
following substraction:
(cs->size - 1) & 0xfffff000
I think there are two options:
1. Change size type to u64.
2. Change condition in mvebu_mbus_get_dram_win_info to:
if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size -1))
I'm looking forward to your information and opinion.
Best regards,
Marcin
2015-12-01 22:40 GMT+01:00 Marcin Wojtas <[email protected]>:
> Hi Gregory,
>
> Thanks for the log. I think it may be an overall problem with 4GB size
> representation in mvebu_mbus_dram_info structure? Maybe whole DRAM
> space is associated to CS0, and the 4GB size (0x1 0000 0000) does not
> fit u32 variable?
>
> Best regards,
> Marcin
>
> 2015-12-01 14:12 GMT+01:00 Gregory CLEMENT <[email protected]>:
>> Hi Marcin,
>>
>> On lun., nov. 30 2015, Marcin Wojtas <[email protected]> wrote:
>> [...]
>>>>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>>>>> A370 I could check on my own. In all cases they survived night-long
>>>>> linerate iperf. Also tests were performed with A388 SoC working as a
>>>>> network bridge between two packet generators. They showed increase of
>>>>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>>>>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>>>>> line rate achieved, CPU load decreased from around 25% without BM vs
>>>>> 18-20% with BM.
>>>>
>>>> I was trying to test the BM part of tour series on the Armada XP GP
>>>> board. However it failed very quickly during the pool allocation. After
>>>> a first debug I found that the size of the cs used in the
>>>> mvebu_mbus_dram_info struct was 0. I have applied your series on a
>>>> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
>>>> mbus driver, a misconfiguration on my side or something else.
>>>>
>>>> Does it ring a bell for you?
>>>
>>> Frankly, I'm a bit surprised, I've never seen such problems on any of
>>> the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
>>> function exit with an error? Can you please apply below diff:
>>> http://pastebin.com/2ws1txWk
>>
>> Yes it exited with errors and I added the same kind traces. It was how I
>> knew that the size was 0!
>>
>> I've just rebuild a fresh kernel using mvebu_v7_defconfig and adding
>> your patch, I got the same issue (see the log at the end of the email.)
>>
>>
>> But the good news is that on the same kernel on Armada 388 GP the pool
>> allocation does not fail. I really suspect an issue with my u-boot.
>>
>>
>>> And send me a full log beginning from u-boot?
>>>
>>>>
>>>> How do you test test it exactly?
>>>> Especially on which kernel and with which U-Boot?
>>>>
>>>
>>> I've just re-built the patchset I sent, which is on top of 4.4-rc1.
>>>
>>> I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
>>> u-boot version: v2011.12 2014_T2.0_eng_dropv2.
>>
>> My config is AXP-GP, 78460 @ 1300MHz, 8GB DRAM (only 4GB are used
>> because I didn't activated LPAE), but the main difference is the U-Boot
>> version: v2011.12 2014_T2.eng_dropv1.ATAG-test02.
>>
>> Thanks,
>>
>> Gregory
>>
>>
>> [ 0.000000] Booting Linux on physical CPU 0x0
>> [ 0.000000] Linux version 4.4.0-rc1-00013-g76f111f9bdf8-dirty (gclement@FE-laptop) (gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu1) ) #1024 SMP Tue Dec 1 14:02:52 CET 2015
>> [ 0.000000] CPU: ARMv7 Processor [562f5842] revision 2 (ARMv7), cr=10c5387d
>> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
>> [ 0.000000] Machine model: Marvell Armada XP Development Board DB-MV784MP-GP
>> [ 0.000000] Memory policy: Data cache writealloc
>> [ 0.000000] PERCPU: Embedded 12 pages/cpu @ee1ac000 s18752 r8192 d22208 u49152
>> [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 981504
>> [ 0.000000] Kernel command line: console=ttyS0,115200 earlyprintk mvneta.rxq_def=2
>> [ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
>> [ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
>> [ 0.000000] log_buf_len min size: 16384 bytes
>> [ 0.000000] log_buf_len: 32768 bytes
>> [ 0.000000] early log buf free: 14924(91%)
>> [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
>> [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
>> [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
>> [ 0.000000] Memory: 3888204K/3932160K available (5576K kernel code, 251K rwdata, 1544K rodata, 4460K init, 207K bss, 43956K reserved, 0K cma-reserved, 3145728K highmem)
>> [ 0.000000] Virtual kernel memory layout:
>> [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
>> [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
>> [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
>> [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
>> [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
>> [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
>> [ 0.000000] .text : 0xc0008000 - 0xc06fc374 (7121 kB)
>> [ 0.000000] .init : 0xc06fd000 - 0xc0b58000 (4460 kB)
>> [ 0.000000] .data : 0xc0b58000 - 0xc0b96d00 ( 252 kB)
>> [ 0.000000] .bss : 0xc0b96d00 - 0xc0bcaa58 ( 208 kB)
>> [ 0.000000] Hierarchical RCU implementation.
>> [ 0.000000] Build-time adjustment of leaf fanout to 32.
>> [ 0.000000] NR_IRQS:16 nr_irqs:16 16
>> [ 0.000000] L2C: DT/platform modifies aux control register: 0x1a69ef10 -> 0x1a69ef12
>> [ 0.000000] Aurora cache controller enabled, 32 ways, 2048 kB
>> [ 0.000000] Aurora: CACHE_ID 0x00000100, AUX_CTRL 0x1a69ef12
>> [ 0.000000] Switching to timer-based delay loop, resolution 40ns
>> [ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
>> [ 0.000015] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
>> [ 0.000305] Console: colour dummy device 80x30
>> [ 0.000322] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)
>> [ 0.000332] pid_max: default: 32768 minimum: 301
>> [ 0.000417] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
>> [ 0.000425] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
>> [ 0.000737] CPU: Testing write buffer coherency: ok
>> [ 0.000880] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
>> [ 0.001006] Setting up static identity map for 0x8280 - 0x82d8
>> [ 0.001224] mvebu-soc-id: MVEBU SoC ID=0x7846, Rev=0x2
>> [ 0.001321] mvebu-pmsu: Initializing Power Management Service Unit
>> [ 0.002062] Booting CPU 1
>> [ 0.004402] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
>> [ 0.004645] Booting CPU 2
>> [ 0.006984] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
>> [ 0.007219] Booting CPU 3
>> [ 0.009558] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
>> [ 0.009605] Brought up 4 CPUs
>> [ 0.009615] SMP: Total of 4 processors activated (200.00 BogoMIPS).
>> [ 0.009620] CPU: All CPU(s) started in SVC mode.
>> [ 0.009957] devtmpfs: initialized
>> [ 0.012369] VFP support v0.3: implementor 56 architecture 2 part 20 variant 9 rev 6
>> [ 0.012525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
>> [ 0.012657] pinctrl core: initialized pinctrl subsystem
>> [ 0.013188] NET: Registered protocol family 16
>> [ 0.013368] DMA: preallocated 256 KiB pool for atomic coherent allocations
>> [ 0.040064] cpuidle: using governor ladder
>> [ 0.070061] cpuidle: using governor menu
>> [ 0.075168] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 3
>> [ 0.075172] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 1
>> [ 0.075176] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 2
>> [ 0.075179] hw-breakpoint: CPU 1 failed to disable vector catch
>> [ 0.075196] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 0
>> [ 0.110815] vgaarb: loaded
>> [ 0.110981] SCSI subsystem initialized
>> [ 0.111306] usbcore: registered new interface driver usbfs
>> [ 0.111349] usbcore: registered new interface driver hub
>> [ 0.111398] usbcore: registered new device driver usb
>> [ 0.111640] Advanced Linux Sound Architecture Driver Initialized.
>> [ 0.111991] Bluetooth: Core ver 2.21
>> [ 0.112024] NET: Registered protocol family 31
>> [ 0.112029] Bluetooth: HCI device and connection manager initialized
>> [ 0.112040] Bluetooth: HCI socket layer initialized
>> [ 0.112048] Bluetooth: L2CAP socket layer initialized
>> [ 0.112067] Bluetooth: SCO socket layer initialized
>> [ 0.112462] clocksource: Switched to clocksource armada_370_xp_clocksource
>> [ 0.119267] NET: Registered protocol family 2
>> [ 0.119629] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
>> [ 0.119690] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
>> [ 0.119749] TCP: Hash tables configured (established 8192 bind 8192)
>> [ 0.119791] UDP hash table entries: 512 (order: 2, 16384 bytes)
>> [ 0.119814] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
>> [ 0.119945] NET: Registered protocol family 1
>> [ 0.120143] RPC: Registered named UNIX socket transport module.
>> [ 0.120150] RPC: Registered udp transport module.
>> [ 0.120155] RPC: Registered tcp transport module.
>> [ 0.120160] RPC: Registered tcp NFSv4.1 backchannel transport module.
>> [ 0.290263] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
>> [ 0.291305] futex hash table entries: 1024 (order: 4, 65536 bytes)
>> [ 0.292696] bounce: pool size: 64 pages
>> [ 0.292746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
>> [ 0.292758] io scheduler noop registered
>> [ 0.292766] io scheduler deadline registered
>> [ 0.292779] io scheduler cfq registered (default)
>> [ 0.293754] armada-xp-pinctrl f1018000.pin-ctrl: registered pinctrl driver
>> [ 0.294186] irq: Cannot allocate irq_descs @ IRQ48, assuming pre-allocated
>> [ 0.294451] irq: Cannot allocate irq_descs @ IRQ80, assuming pre-allocated
>> [ 0.294612] irq: Cannot allocate irq_descs @ IRQ112, assuming pre-allocated
>> [ 0.295033] mv_xor f1060900.xor: Marvell shared XOR driver
>> [ 0.332873] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>> [ 0.372848] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>> [ 0.372958] mv_xor f10f0900.xor: Marvell shared XOR driver
>> [ 0.412850] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>> [ 0.452845] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>> [ 0.492634] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
>> [ 0.493897] console [ttyS0] disabled
>> [ 0.514007] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 20, base_baud = 15625000) is a 16550A
>> [ 1.200947] console [ttyS0] enabled
>> [ 1.225041] f1012100.serial: ttyS1 at MMIO 0xf1012100 (irq = 21, base_baud = 15625000) is a 16550A
>> [ 1.254648] f1012200.serial: ttyS2 at MMIO 0xf1012200 (irq = 33, base_baud = 15625000) is a 16550A
>> [ 1.284237] f1012300.serial: ttyS3 at MMIO 0xf1012300 (irq = 34, base_baud = 15625000) is a 16550A
>> [ 1.294303] sata_mv f10a0000.sata: slots 32 ports 2
>> [ 1.300098] scsi host0: sata_mv
>> [ 1.303511] scsi host1: sata_mv
>> [ 1.306805] ata1: SATA max UDMA/133 irq 31
>> [ 1.310913] ata2: SATA max UDMA/133 irq 31
>> [ 1.315482] pxa3xx-nand f10d0000.nand: This platform can't do DMA on this device
>> [ 1.323280] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0x38
>> [ 1.329650] nand: Micron MT29F8G08ABABAWP
>> [ 1.333684] nand: 1024 MiB, SLC, erase size: 512 KiB, page size: 4096, OOB size: 224
>> [ 1.341450] pxa3xx-nand f10d0000.nand: ECC strength 16, ECC step size 2048
>> [ 1.349163] Bad block table found at page 261888, version 0x01
>> [ 1.355385] Bad block table found at page 262016, version 0x01
>> [ 1.364466] pxa3xx_nand-0: 'partitions' subnode not found on /soc/internal-regs/nand@d0000. Trying to parse direct subnodes as partitions.
>> [ 1.378215] m25p80 spi0.0: n25q128a13 (16384 Kbytes)
>> [ 1.385098] spi0.0: 'partitions' subnode not found on /soc/internal-regs/spi@10600/spi-flash@0. Trying to parse direct subnodes as partitions.
>> [ 1.399136] libphy: Fixed MDIO Bus: probed
>> [ 1.403396] libphy: orion_mdio_bus: probed
>> [ 1.415856] mvneta_bm f10c0000.bm: Buffer Manager for network controller enabled
>> [ 1.424232] size = 0x0, target = 0x0, attr = 0x1e
>> [ 1.428949] mvebu_mbus: invalid dram address 0x2d194000
>> [ 1.434204] mvneta_bm f10c0000.bm: fail to create pool 0
>> [ 1.439532] mvneta f1070000.ethernet eth0: fail to obtain long pool for port
>> [ 1.446607] mvneta f1070000.ethernet: use SW buffer management
>> [ 1.452696] mvneta f1070000.ethernet eth0: Using hardware mac address 00:50:43:68:1c:28
>> [ 1.461529] size = 0x0, target = 0x0, attr = 0x1e
>> [ 1.466258] mvebu_mbus: invalid dram address 0x2d198000
>> [ 1.471501] mvneta_bm f10c0000.bm: fail to create pool 1
>> [ 1.476836] mvneta f1074000.ethernet eth1: fail to obtain long pool for port
>> [ 1.483910] mvneta f1074000.ethernet: use SW buffer management
>> [ 1.489994] mvneta f1074000.ethernet eth1: Using random mac address be:14:de:e2:be:cf
>> [ 1.498661] size = 0x0, target = 0x0, attr = 0x1e
>> [ 1.503391] mvebu_mbus: invalid dram address 0x2d10c000
>> [ 1.508633] mvneta_bm f10c0000.bm: fail to create pool 2
>> [ 1.513968] mvneta f1030000.ethernet eth2: fail to obtain long pool for port
>> [ 1.521035] mvneta f1030000.ethernet: use SW buffer management
>> [ 1.527121] mvneta f1030000.ethernet eth2: Using random mac address 86:3c:f8:02:31:cf
>> [ 1.535810] size = 0x0, target = 0x0, attr = 0x1e
>> [ 1.540527] mvebu_mbus: invalid dram address 0x2d752000
>> [ 1.545781] mvneta_bm f10c0000.bm: fail to create pool 3
>> [ 1.551108] mvneta f1034000.ethernet eth3: fail to obtain long pool for port
>> [ 1.558182] mvneta f1034000.ethernet: use SW buffer management
>> [ 1.564269] mvneta f1034000.ethernet eth3: Using random mac address 7e:3e:74:8c:93:50
>> [ 1.572297] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
>> [ 1.578863] ehci-pci: EHCI PCI platform driver
>> [ 1.583371] ehci-orion: EHCI orion driver
>> [ 1.587476] orion-ehci f1050000.usb: EHCI Host Controller
>> [ 1.592916] orion-ehci f1050000.usb: new USB bus registered, assigned bus number 1
>> [ 1.600567] orion-ehci f1050000.usb: irq 27, io mem 0xf1050000
>> [ 1.622494] orion-ehci f1050000.usb: USB 2.0 started, EHCI 1.00
>> [ 1.628967] hub 1-0:1.0: USB hub found
>> [ 1.632806] hub 1-0:1.0: 1 port detected
>> [ 1.637018] orion-ehci f1051000.usb: EHCI Host Controller
>> [ 1.642468] orion-ehci f1051000.usb: new USB bus registered, assigned bus number 2
>> [ 1.650111] orion-ehci f1051000.usb: irq 28, io mem 0xf1051000
>> [ 1.672467] orion-ehci f1051000.usb: USB 2.0 started, EHCI 1.00
>> [ 1.678908] hub 2-0:1.0: USB hub found
>> [ 1.682736] hub 2-0:1.0: 1 port detected
>> [ 1.687203] usbcore: registered new interface driver usb-storage
>> [ 1.693454] mousedev: PS/2 mouse device common for all mice
>> [ 1.699593] rtc-mv f1010300.rtc: rtc core: registered f1010300.rtc as rtc0
>> [ 1.706641] i2c /dev entries driver
>> [ 1.711074] orion_wdt: Initial timeout 171 sec
>> [ 1.715770] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:0, -19
>> [ 1.723985] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:1, -19
>> [ 1.732174] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:2, -19
>> [ 1.740372] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:3, -19
>> [ 1.748968] sdhci: Secure Digital Host Controller Interface driver
>> [ 1.755176] sdhci: Copyright(c) Pierre Ossman
>> [ 1.759723] sdhci-pltfm: SDHCI platform and OF driver helper
>> [ 1.765805] usbcore: registered new interface driver usbhid
>> [ 1.771394] usbhid: USB HID core driver
>> [ 1.775730] f0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x000018
>> [ 1.785775] Intel/Sharp Extended Query Table at 0x0031
>> [ 1.790936] Intel/Sharp Extended Query Table at 0x0031
>> [ 1.796106] Using buffer write method
>> [ 1.799778] cfi_cmdset_0001: Erase suspend on write enabled
>> [ 1.809269] f0000000.nor: 'partitions' subnode not found on /soc/devbus-bootcs/nor@0. Trying to parse direct subnodes as partitions.
>> [ 1.822513] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
>> [ 1.829037] NET: Registered protocol family 10
>> [ 1.834180] sit: IPv6 over IPv4 tunneling driver
>> [ 1.839209] NET: Registered protocol family 17
>> [ 1.842687] ata1.00: ATA-9: TS8GHSD310, 20111220, max UDMA/133
>> [ 1.842692] ata1.00: 15621984 sectors, multi 1: LBA48 NCQ (depth 31/32)
>> [ 1.856343] ThumbEE CPU extension supported.
>> [ 1.860632] Registering SWP/SWPB emulation handler
>> [ 1.862685] ata1.00: configured for UDMA/133
>> [ 1.862993] scsi 0:0:0:0: Direct-Access ATA TS8GHSD310 1220 PQ: 0 ANSI: 5
>> [ 1.863646] sd 0:0:0:0: [sda] 15621984 512-byte logical blocks: (7.99 GB/7.44 GiB)
>> [ 1.863838] sd 0:0:0:0: [sda] Write Protect is off
>> [ 1.863907] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
>> [ 1.873801] sda: sda1 sda2 sda3 < sda5 > sda4
>> [ 1.875389] sd 0:0:0:0: [sda] Attached SCSI disk
>> [ 1.909334] rtc-mv f1010300.rtc: setting system clock to 2001-01-20 07:57:42 UTC (979977462)
>> [ 1.918901] ALSA device list:
>> [ 1.921878] No soundcards found.
>> [ 2.222505] ata2: SATA link down (SStatus 0 SControl F300)
>> [ 2.230484] Freeing unused kernel memory: 4460K (c06fd000 - c0b58000)
>>
>> --
>> Gregory Clement, Free Electrons
>> Kernel, drivers, real-time and embedded Linux
>> development, consulting, training and support.
>> http://free-electrons.com
Hi Florian,
Can you please describe in more details, what would you expect from
such special abstraction layer regarding buffer managers? I'd like to
understand more of your expectations and evaluate possible work.
Best regards,
Marcin
2015-11-30 17:25 GMT+01:00 David Miller <[email protected]>:
> From: Marcin Wojtas <[email protected]>
> Date: Mon, 30 Nov 2015 15:13:22 +0100
>
>> What kind of abstraction and helpers do you mean? Some kind of API
>> (e.g. bm_alloc_buffer, bm_initialize_ring bm_put_buffer,
>> bm_get_buffer), which would be used by platform drivers (and specific
>> aplications if one wants to develop on top of the kernel)?
>>
>> In general, what is your top-view of such solution and its cooperation
>> with the drivers?
>
> The tricky parts involved have to do with allocating pages for the
> buffer pools and minimizing the number of atomic refcounting
> operations on those pages for for the puts and gets, particularly
> around buffer replenish runs.
>
> For example, if you're allocating a page for a buffer pool the device
> will chop into N (for any N < PAGE_SIZE) byte pieces, you can
> eliminate many atomic operations.
Hi Simon,
I checked using hrtimer_is_queued instead of a custom flag and it
resulted in ~20kpps drop in my setup. timer_scheduled flag is cleared
in the tasklet, so no timer can be scheduled until the tasklet is
executed. hr_timer flags do not cover this situation, so much more
timers are enqueued. I added a counter and with maximal throughput
during 30s test and 9 times less timers were enqueued with
timer_scheduled flag (~31k vs ~281k), so it's much more efficient and
I'll leave it as is.
Best regards,
Marcin
2015-11-30 16:57 GMT+01:00 Marcin Wojtas <[email protected]>:
> Hi Simon,
>
> 2015-11-26 17:45 GMT+01:00 Simon Guinot <[email protected]>:
>> Hi Marcin,
>>
>> On Sun, Nov 22, 2015 at 08:53:52AM +0100, Marcin Wojtas wrote:
>>> Mixed approach allows using higher interrupt threshold (increased back to
>>> 15 packets), useful in high throughput. In case of small amount of data
>>> or very short TX queues HR timer ensures releasing buffers with small
>>> latency.
>>>
>>> Along with existing tx_done processing by coalescing interrupts this
>>> commit enables triggering HR timer each time the packets are sent.
>>> Time threshold can also be configured, using ethtool.
>>>
>>> Signed-off-by: Marcin Wojtas <[email protected]>
>>> Signed-off-by: Simon Guinot <[email protected]>
>>> ---
>>> drivers/net/ethernet/marvell/mvneta.c | 89 +++++++++++++++++++++++++++++++++--
>>> 1 file changed, 85 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
>>> index 9c9e858..f5acaf6 100644
>>> --- a/drivers/net/ethernet/marvell/mvneta.c
>>> +++ b/drivers/net/ethernet/marvell/mvneta.c
>>> @@ -21,6 +21,8 @@
>>> #include <linux/module.h>
>>> #include <linux/interrupt.h>
>>> #include <linux/if_vlan.h>
>>> +#include <linux/hrtimer.h>
>>> +#include <linux/ktime.h>
>>
>> ktime.h is already included by hrtimer.h.
>>
>>> #include <net/ip.h>
>>> #include <net/ipv6.h>
>>> #include <linux/io.h>
>>> @@ -226,7 +228,8 @@
>>> /* Various constants */
>>>
>>> /* Coalescing */
>>> -#define MVNETA_TXDONE_COAL_PKTS 1
>>> +#define MVNETA_TXDONE_COAL_PKTS 15
>>> +#define MVNETA_TXDONE_COAL_USEC 100
>>
>> Maybe we should keep the default configuration and let the user choose
>> to enable (or not) this feature ?
>
> I think that this feature should be enabled by default, same as in RX
> (which is enabled by HW in ingress). It satisfies all kinds of traffic
> or queues sizes. I'd prefer a situation that if someone really wants
> to disable it (even if I don't know the possible justification), then
> let him use ethtool for this purpose.
>
>>
>>> #define MVNETA_RX_COAL_PKTS 32
>>> #define MVNETA_RX_COAL_USEC 100
>>>
>>> @@ -356,6 +359,11 @@ struct mvneta_port {
>>> struct net_device *dev;
>>> struct notifier_block cpu_notifier;
>>>
>>> + /* Egress finalization */
>>> + struct tasklet_struct tx_done_tasklet;
>>> + struct hrtimer tx_done_timer;
>>> + bool timer_scheduled;
>>
>> I think we could use hrtimer_is_queued() instead of introducing a new
>> variable.
>>
>
> Good point, i'll try that.
>
> Best regards,
> Marcin
Hi Marcin,
On mer., déc. 02 2015, Marcin Wojtas <[email protected]> wrote:
> Gregory,
>
> Please apply below patch:
> http://pastebin.com/t42xyU3i
> It will confirm if there's an overflow for CS0 size in your setup.
> Please let know.
Indeed it is an overflow. In v4.0 the size was correct because it was no
longer compute as is: w->size = (size | ~DDR_SIZE_MASK) + 1;
But the commit was reverted because it caused other issues.
>
> So far the issue may have been not noticed, because in every IO driver
> using mvebu_mbus_dram_info for configuring MBUS windows, there's
> following substraction:
> (cs->size - 1) & 0xfffff000
>
> I think there are two options:
> 1. Change size type to u64.
If we switch to u64 we really must pay attention to be sure that it
won't be used to be written in a register, but the regsiter remains
32bits.
> 2. Change condition in mvebu_mbus_get_dram_win_info to:
> if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size -1))
I think it would be the best solution.
Gregory
>
> I'm looking forward to your information and opinion.
>
> Best regards,
> Marcin
>
> 2015-12-01 22:40 GMT+01:00 Marcin Wojtas <[email protected]>:
>> Hi Gregory,
>>
>> Thanks for the log. I think it may be an overall problem with 4GB size
>> representation in mvebu_mbus_dram_info structure? Maybe whole DRAM
>> space is associated to CS0, and the 4GB size (0x1 0000 0000) does not
>> fit u32 variable?
>>
>> Best regards,
>> Marcin
>>
>> 2015-12-01 14:12 GMT+01:00 Gregory CLEMENT <[email protected]>:
>>> Hi Marcin,
>>>
>>> On lun., nov. 30 2015, Marcin Wojtas <[email protected]> wrote:
>>> [...]
>>>>>> 5. Enable BM on Armada XP and 38X development boards - those ones and
>>>>>> A370 I could check on my own. In all cases they survived night-long
>>>>>> linerate iperf. Also tests were performed with A388 SoC working as a
>>>>>> network bridge between two packet generators. They showed increase of
>>>>>> maximum processed 64B packets by ~20k (~555k packets with BM enabled
>>>>>> vs ~535 packets without BM). Also when pushing 1500B-packets with a
>>>>>> line rate achieved, CPU load decreased from around 25% without BM vs
>>>>>> 18-20% with BM.
>>>>>
>>>>> I was trying to test the BM part of tour series on the Armada XP GP
>>>>> board. However it failed very quickly during the pool allocation. After
>>>>> a first debug I found that the size of the cs used in the
>>>>> mvebu_mbus_dram_info struct was 0. I have applied your series on a
>>>>> v4.4-rc1 kernel. At this stage I don't know if it is a regression in the
>>>>> mbus driver, a misconfiguration on my side or something else.
>>>>>
>>>>> Does it ring a bell for you?
>>>>
>>>> Frankly, I'm a bit surprised, I've never seen such problems on any of
>>>> the boards (AXP-GP/DB, A38X-DB/GP/AP). Did mvebu_mbus_dram_win_info
>>>> function exit with an error? Can you please apply below diff:
>>>> http://pastebin.com/2ws1txWk
>>>
>>> Yes it exited with errors and I added the same kind traces. It was how I
>>> knew that the size was 0!
>>>
>>> I've just rebuild a fresh kernel using mvebu_v7_defconfig and adding
>>> your patch, I got the same issue (see the log at the end of the email.)
>>>
>>>
>>> But the good news is that on the same kernel on Armada 388 GP the pool
>>> allocation does not fail. I really suspect an issue with my u-boot.
>>>
>>>
>>>> And send me a full log beginning from u-boot?
>>>>
>>>>>
>>>>> How do you test test it exactly?
>>>>> Especially on which kernel and with which U-Boot?
>>>>>
>>>>
>>>> I've just re-built the patchset I sent, which is on top of 4.4-rc1.
>>>>
>>>> I use AXP-GP, 78460 @ 1600MHz, 2GB DRAM, and everything works fine. My
>>>> u-boot version: v2011.12 2014_T2.0_eng_dropv2.
>>>
>>> My config is AXP-GP, 78460 @ 1300MHz, 8GB DRAM (only 4GB are used
>>> because I didn't activated LPAE), but the main difference is the U-Boot
>>> version: v2011.12 2014_T2.eng_dropv1.ATAG-test02.
>>>
>>> Thanks,
>>>
>>> Gregory
>>>
>>>
>>> [ 0.000000] Booting Linux on physical CPU 0x0
>>> [ 0.000000] Linux version 4.4.0-rc1-00013-g76f111f9bdf8-dirty (gclement@FE-laptop) (gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu1) ) #1024 SMP Tue Dec 1 14:02:52 CET 2015
>>> [ 0.000000] CPU: ARMv7 Processor [562f5842] revision 2 (ARMv7), cr=10c5387d
>>> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
>>> [ 0.000000] Machine model: Marvell Armada XP Development Board DB-MV784MP-GP
>>> [ 0.000000] Memory policy: Data cache writealloc
>>> [ 0.000000] PERCPU: Embedded 12 pages/cpu @ee1ac000 s18752 r8192 d22208 u49152
>>> [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 981504
>>> [ 0.000000] Kernel command line: console=ttyS0,115200 earlyprintk mvneta.rxq_def=2
>>> [ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
>>> [ 0.000000] log_buf_len total cpu_extra contributions: 12288 bytes
>>> [ 0.000000] log_buf_len min size: 16384 bytes
>>> [ 0.000000] log_buf_len: 32768 bytes
>>> [ 0.000000] early log buf free: 14924(91%)
>>> [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
>>> [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
>>> [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
>>> [ 0.000000] Memory: 3888204K/3932160K available (5576K kernel code, 251K rwdata, 1544K rodata, 4460K init, 207K bss, 43956K reserved, 0K cma-reserved, 3145728K highmem)
>>> [ 0.000000] Virtual kernel memory layout:
>>> [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
>>> [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
>>> [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB)
>>> [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB)
>>> [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
>>> [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
>>> [ 0.000000] .text : 0xc0008000 - 0xc06fc374 (7121 kB)
>>> [ 0.000000] .init : 0xc06fd000 - 0xc0b58000 (4460 kB)
>>> [ 0.000000] .data : 0xc0b58000 - 0xc0b96d00 ( 252 kB)
>>> [ 0.000000] .bss : 0xc0b96d00 - 0xc0bcaa58 ( 208 kB)
>>> [ 0.000000] Hierarchical RCU implementation.
>>> [ 0.000000] Build-time adjustment of leaf fanout to 32.
>>> [ 0.000000] NR_IRQS:16 nr_irqs:16 16
>>> [ 0.000000] L2C: DT/platform modifies aux control register: 0x1a69ef10 -> 0x1a69ef12
>>> [ 0.000000] Aurora cache controller enabled, 32 ways, 2048 kB
>>> [ 0.000000] Aurora: CACHE_ID 0x00000100, AUX_CTRL 0x1a69ef12
>>> [ 0.000000] Switching to timer-based delay loop, resolution 40ns
>>> [ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
>>> [ 0.000015] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
>>> [ 0.000305] Console: colour dummy device 80x30
>>> [ 0.000322] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)
>>> [ 0.000332] pid_max: default: 32768 minimum: 301
>>> [ 0.000417] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
>>> [ 0.000425] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
>>> [ 0.000737] CPU: Testing write buffer coherency: ok
>>> [ 0.000880] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
>>> [ 0.001006] Setting up static identity map for 0x8280 - 0x82d8
>>> [ 0.001224] mvebu-soc-id: MVEBU SoC ID=0x7846, Rev=0x2
>>> [ 0.001321] mvebu-pmsu: Initializing Power Management Service Unit
>>> [ 0.002062] Booting CPU 1
>>> [ 0.004402] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
>>> [ 0.004645] Booting CPU 2
>>> [ 0.006984] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
>>> [ 0.007219] Booting CPU 3
>>> [ 0.009558] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
>>> [ 0.009605] Brought up 4 CPUs
>>> [ 0.009615] SMP: Total of 4 processors activated (200.00 BogoMIPS).
>>> [ 0.009620] CPU: All CPU(s) started in SVC mode.
>>> [ 0.009957] devtmpfs: initialized
>>> [ 0.012369] VFP support v0.3: implementor 56 architecture 2 part 20 variant 9 rev 6
>>> [ 0.012525] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
>>> [ 0.012657] pinctrl core: initialized pinctrl subsystem
>>> [ 0.013188] NET: Registered protocol family 16
>>> [ 0.013368] DMA: preallocated 256 KiB pool for atomic coherent allocations
>>> [ 0.040064] cpuidle: using governor ladder
>>> [ 0.070061] cpuidle: using governor menu
>>> [ 0.075168] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 3
>>> [ 0.075172] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 1
>>> [ 0.075176] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 2
>>> [ 0.075179] hw-breakpoint: CPU 1 failed to disable vector catch
>>> [ 0.075196] hw-breakpoint: Debug register access (0xee003e17) caused undefined instruction on CPU 0
>>> [ 0.110815] vgaarb: loaded
>>> [ 0.110981] SCSI subsystem initialized
>>> [ 0.111306] usbcore: registered new interface driver usbfs
>>> [ 0.111349] usbcore: registered new interface driver hub
>>> [ 0.111398] usbcore: registered new device driver usb
>>> [ 0.111640] Advanced Linux Sound Architecture Driver Initialized.
>>> [ 0.111991] Bluetooth: Core ver 2.21
>>> [ 0.112024] NET: Registered protocol family 31
>>> [ 0.112029] Bluetooth: HCI device and connection manager initialized
>>> [ 0.112040] Bluetooth: HCI socket layer initialized
>>> [ 0.112048] Bluetooth: L2CAP socket layer initialized
>>> [ 0.112067] Bluetooth: SCO socket layer initialized
>>> [ 0.112462] clocksource: Switched to clocksource armada_370_xp_clocksource
>>> [ 0.119267] NET: Registered protocol family 2
>>> [ 0.119629] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
>>> [ 0.119690] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
>>> [ 0.119749] TCP: Hash tables configured (established 8192 bind 8192)
>>> [ 0.119791] UDP hash table entries: 512 (order: 2, 16384 bytes)
>>> [ 0.119814] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
>>> [ 0.119945] NET: Registered protocol family 1
>>> [ 0.120143] RPC: Registered named UNIX socket transport module.
>>> [ 0.120150] RPC: Registered udp transport module.
>>> [ 0.120155] RPC: Registered tcp transport module.
>>> [ 0.120160] RPC: Registered tcp NFSv4.1 backchannel transport module.
>>> [ 0.290263] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
>>> [ 0.291305] futex hash table entries: 1024 (order: 4, 65536 bytes)
>>> [ 0.292696] bounce: pool size: 64 pages
>>> [ 0.292746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
>>> [ 0.292758] io scheduler noop registered
>>> [ 0.292766] io scheduler deadline registered
>>> [ 0.292779] io scheduler cfq registered (default)
>>> [ 0.293754] armada-xp-pinctrl f1018000.pin-ctrl: registered pinctrl driver
>>> [ 0.294186] irq: Cannot allocate irq_descs @ IRQ48, assuming pre-allocated
>>> [ 0.294451] irq: Cannot allocate irq_descs @ IRQ80, assuming pre-allocated
>>> [ 0.294612] irq: Cannot allocate irq_descs @ IRQ112, assuming pre-allocated
>>> [ 0.295033] mv_xor f1060900.xor: Marvell shared XOR driver
>>> [ 0.332873] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>>> [ 0.372848] mv_xor f1060900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>>> [ 0.372958] mv_xor f10f0900.xor: Marvell shared XOR driver
>>> [ 0.412850] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>>> [ 0.452845] mv_xor f10f0900.xor: Marvell XOR (Registers Mode): ( xor cpy intr )
>>> [ 0.492634] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
>>> [ 0.493897] console [ttyS0] disabled
>>> [ 0.514007] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 20, base_baud = 15625000) is a 16550A
>>> [ 1.200947] console [ttyS0] enabled
>>> [ 1.225041] f1012100.serial: ttyS1 at MMIO 0xf1012100 (irq = 21, base_baud = 15625000) is a 16550A
>>> [ 1.254648] f1012200.serial: ttyS2 at MMIO 0xf1012200 (irq = 33, base_baud = 15625000) is a 16550A
>>> [ 1.284237] f1012300.serial: ttyS3 at MMIO 0xf1012300 (irq = 34, base_baud = 15625000) is a 16550A
>>> [ 1.294303] sata_mv f10a0000.sata: slots 32 ports 2
>>> [ 1.300098] scsi host0: sata_mv
>>> [ 1.303511] scsi host1: sata_mv
>>> [ 1.306805] ata1: SATA max UDMA/133 irq 31
>>> [ 1.310913] ata2: SATA max UDMA/133 irq 31
>>> [ 1.315482] pxa3xx-nand f10d0000.nand: This platform can't do DMA on this device
>>> [ 1.323280] nand: device found, Manufacturer ID: 0x2c, Chip ID: 0x38
>>> [ 1.329650] nand: Micron MT29F8G08ABABAWP
>>> [ 1.333684] nand: 1024 MiB, SLC, erase size: 512 KiB, page size: 4096, OOB size: 224
>>> [ 1.341450] pxa3xx-nand f10d0000.nand: ECC strength 16, ECC step size 2048
>>> [ 1.349163] Bad block table found at page 261888, version 0x01
>>> [ 1.355385] Bad block table found at page 262016, version 0x01
>>> [ 1.364466] pxa3xx_nand-0: 'partitions' subnode not found on /soc/internal-regs/nand@d0000. Trying to parse direct subnodes as partitions.
>>> [ 1.378215] m25p80 spi0.0: n25q128a13 (16384 Kbytes)
>>> [ 1.385098] spi0.0: 'partitions' subnode not found on /soc/internal-regs/spi@10600/spi-flash@0. Trying to parse direct subnodes as partitions.
>>> [ 1.399136] libphy: Fixed MDIO Bus: probed
>>> [ 1.403396] libphy: orion_mdio_bus: probed
>>> [ 1.415856] mvneta_bm f10c0000.bm: Buffer Manager for network controller enabled
>>> [ 1.424232] size = 0x0, target = 0x0, attr = 0x1e
>>> [ 1.428949] mvebu_mbus: invalid dram address 0x2d194000
>>> [ 1.434204] mvneta_bm f10c0000.bm: fail to create pool 0
>>> [ 1.439532] mvneta f1070000.ethernet eth0: fail to obtain long pool for port
>>> [ 1.446607] mvneta f1070000.ethernet: use SW buffer management
>>> [ 1.452696] mvneta f1070000.ethernet eth0: Using hardware mac address 00:50:43:68:1c:28
>>> [ 1.461529] size = 0x0, target = 0x0, attr = 0x1e
>>> [ 1.466258] mvebu_mbus: invalid dram address 0x2d198000
>>> [ 1.471501] mvneta_bm f10c0000.bm: fail to create pool 1
>>> [ 1.476836] mvneta f1074000.ethernet eth1: fail to obtain long pool for port
>>> [ 1.483910] mvneta f1074000.ethernet: use SW buffer management
>>> [ 1.489994] mvneta f1074000.ethernet eth1: Using random mac address be:14:de:e2:be:cf
>>> [ 1.498661] size = 0x0, target = 0x0, attr = 0x1e
>>> [ 1.503391] mvebu_mbus: invalid dram address 0x2d10c000
>>> [ 1.508633] mvneta_bm f10c0000.bm: fail to create pool 2
>>> [ 1.513968] mvneta f1030000.ethernet eth2: fail to obtain long pool for port
>>> [ 1.521035] mvneta f1030000.ethernet: use SW buffer management
>>> [ 1.527121] mvneta f1030000.ethernet eth2: Using random mac address 86:3c:f8:02:31:cf
>>> [ 1.535810] size = 0x0, target = 0x0, attr = 0x1e
>>> [ 1.540527] mvebu_mbus: invalid dram address 0x2d752000
>>> [ 1.545781] mvneta_bm f10c0000.bm: fail to create pool 3
>>> [ 1.551108] mvneta f1034000.ethernet eth3: fail to obtain long pool for port
>>> [ 1.558182] mvneta f1034000.ethernet: use SW buffer management
>>> [ 1.564269] mvneta f1034000.ethernet eth3: Using random mac address 7e:3e:74:8c:93:50
>>> [ 1.572297] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
>>> [ 1.578863] ehci-pci: EHCI PCI platform driver
>>> [ 1.583371] ehci-orion: EHCI orion driver
>>> [ 1.587476] orion-ehci f1050000.usb: EHCI Host Controller
>>> [ 1.592916] orion-ehci f1050000.usb: new USB bus registered, assigned bus number 1
>>> [ 1.600567] orion-ehci f1050000.usb: irq 27, io mem 0xf1050000
>>> [ 1.622494] orion-ehci f1050000.usb: USB 2.0 started, EHCI 1.00
>>> [ 1.628967] hub 1-0:1.0: USB hub found
>>> [ 1.632806] hub 1-0:1.0: 1 port detected
>>> [ 1.637018] orion-ehci f1051000.usb: EHCI Host Controller
>>> [ 1.642468] orion-ehci f1051000.usb: new USB bus registered, assigned bus number 2
>>> [ 1.650111] orion-ehci f1051000.usb: irq 28, io mem 0xf1051000
>>> [ 1.672467] orion-ehci f1051000.usb: USB 2.0 started, EHCI 1.00
>>> [ 1.678908] hub 2-0:1.0: USB hub found
>>> [ 1.682736] hub 2-0:1.0: 1 port detected
>>> [ 1.687203] usbcore: registered new interface driver usb-storage
>>> [ 1.693454] mousedev: PS/2 mouse device common for all mice
>>> [ 1.699593] rtc-mv f1010300.rtc: rtc core: registered f1010300.rtc as rtc0
>>> [ 1.706641] i2c /dev entries driver
>>> [ 1.711074] orion_wdt: Initial timeout 171 sec
>>> [ 1.715770] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:0, -19
>>> [ 1.723985] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:1, -19
>>> [ 1.732174] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:2, -19
>>> [ 1.740372] cpu: dev_pm_opp_of_cpumask_add_table: couldn't find opp table for cpu:3, -19
>>> [ 1.748968] sdhci: Secure Digital Host Controller Interface driver
>>> [ 1.755176] sdhci: Copyright(c) Pierre Ossman
>>> [ 1.759723] sdhci-pltfm: SDHCI platform and OF driver helper
>>> [ 1.765805] usbcore: registered new interface driver usbhid
>>> [ 1.771394] usbhid: USB HID core driver
>>> [ 1.775730] f0000000.nor: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000089 Chip ID 0x000018
>>> [ 1.785775] Intel/Sharp Extended Query Table at 0x0031
>>> [ 1.790936] Intel/Sharp Extended Query Table at 0x0031
>>> [ 1.796106] Using buffer write method
>>> [ 1.799778] cfi_cmdset_0001: Erase suspend on write enabled
>>> [ 1.809269] f0000000.nor: 'partitions' subnode not found on /soc/devbus-bootcs/nor@0. Trying to parse direct subnodes as partitions.
>>> [ 1.822513] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl F300)
>>> [ 1.829037] NET: Registered protocol family 10
>>> [ 1.834180] sit: IPv6 over IPv4 tunneling driver
>>> [ 1.839209] NET: Registered protocol family 17
>>> [ 1.842687] ata1.00: ATA-9: TS8GHSD310, 20111220, max UDMA/133
>>> [ 1.842692] ata1.00: 15621984 sectors, multi 1: LBA48 NCQ (depth 31/32)
>>> [ 1.856343] ThumbEE CPU extension supported.
>>> [ 1.860632] Registering SWP/SWPB emulation handler
>>> [ 1.862685] ata1.00: configured for UDMA/133
>>> [ 1.862993] scsi 0:0:0:0: Direct-Access ATA TS8GHSD310 1220 PQ: 0 ANSI: 5
>>> [ 1.863646] sd 0:0:0:0: [sda] 15621984 512-byte logical blocks: (7.99 GB/7.44 GiB)
>>> [ 1.863838] sd 0:0:0:0: [sda] Write Protect is off
>>> [ 1.863907] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
>>> [ 1.873801] sda: sda1 sda2 sda3 < sda5 > sda4
>>> [ 1.875389] sd 0:0:0:0: [sda] Attached SCSI disk
>>> [ 1.909334] rtc-mv f1010300.rtc: setting system clock to 2001-01-20 07:57:42 UTC (979977462)
>>> [ 1.918901] ALSA device list:
>>> [ 1.921878] No soundcards found.
>>> [ 2.222505] ata2: SATA link down (SStatus 0 SControl F300)
>>> [ 2.230484] Freeing unused kernel memory: 4460K (c06fd000 - c0b58000)
>>>
>>> --
>>> Gregory Clement, Free Electrons
>>> Kernel, drivers, real-time and embedded Linux
>>> development, consulting, training and support.
>>> http://free-electrons.com
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi,
On mer., déc. 02 2015, Gregory CLEMENT <[email protected]> wrote:
>>
>> So far the issue may have been not noticed, because in every IO driver
>> using mvebu_mbus_dram_info for configuring MBUS windows, there's
>> following substraction:
>> (cs->size - 1) & 0xfffff000
>>
>> I think there are two options:
>> 1. Change size type to u64.
>
> If we switch to u64 we really must pay attention to be sure that it
> won't be used to be written in a register, but the regsiter remains
> 32bits.
>
>> 2. Change condition in mvebu_mbus_get_dram_win_info to:
>> if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size -1))
>
> I think it would be the best solution.
So I applied the following patch:
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -964,7 +964,7 @@ int mvebu_mbus_get_dram_win_info(phys_addr_t phyaddr, u8 *target, u8 *attr)
for (i = 0; i < dram->num_cs; i++) {
const struct mbus_dram_window *cs = dram->cs + i;
- if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size)) {
+ if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size - 1)) {
*target = dram->mbus_dram_target_id;
*attr = cs->mbus_attr;
return 0;
I didn't get any errors during boot related to the BM. However I did not
manage to use an ethernet interface. The udhcpc never managed to get an
IP and if I set the IP manually I could not ping.
But on Armada 388 GP I didn't have any issue.
Do you have some idea about waht I could check?
Gregory
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
Hi Gregory,
2015-12-02 17:21 GMT+01:00 Gregory CLEMENT <[email protected]>:
> Hi,
>
> On mer., déc. 02 2015, Gregory CLEMENT <[email protected]> wrote:
>
>>>
>>> So far the issue may have been not noticed, because in every IO driver
>>> using mvebu_mbus_dram_info for configuring MBUS windows, there's
>>> following substraction:
>>> (cs->size - 1) & 0xfffff000
>>>
>>> I think there are two options:
>>> 1. Change size type to u64.
>>
>> If we switch to u64 we really must pay attention to be sure that it
>> won't be used to be written in a register, but the regsiter remains
>> 32bits.
>>
>>> 2. Change condition in mvebu_mbus_get_dram_win_info to:
>>> if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size -1))
>>
>> I think it would be the best solution.
>
> So I applied the following patch:
> --- a/drivers/bus/mvebu-mbus.c
> +++ b/drivers/bus/mvebu-mbus.c
> @@ -964,7 +964,7 @@ int mvebu_mbus_get_dram_win_info(phys_addr_t phyaddr, u8 *target, u8 *attr)
> for (i = 0; i < dram->num_cs; i++) {
> const struct mbus_dram_window *cs = dram->cs + i;
>
> - if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size)) {
> + if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size - 1)) {
> *target = dram->mbus_dram_target_id;
> *attr = cs->mbus_attr;
> return 0;
>
> I didn't get any errors during boot related to the BM. However I did not
> manage to use an ethernet interface. The udhcpc never managed to get an
> IP and if I set the IP manually I could not ping.
>
> But on Armada 388 GP I didn't have any issue.
>
> Do you have some idea about waht I could check?
>
I replaced 2GB with 8GB DIMM and after that, on the same kernel/board,
I can't ping either. Very strange, but as I can reproduce the issue,
don't debug it, I will.
Best regards,
Marcin
Hi Marcin,
On mer., déc. 02 2015, Marcin Wojtas <[email protected]> wrote:
>>>
>>>> 2. Change condition in mvebu_mbus_get_dram_win_info to:
>>>> if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size -1))
>>>
>>> I think it would be the best solution.
>>
>> So I applied the following patch:
>> --- a/drivers/bus/mvebu-mbus.c
>> +++ b/drivers/bus/mvebu-mbus.c
>> @@ -964,7 +964,7 @@ int mvebu_mbus_get_dram_win_info(phys_addr_t phyaddr, u8 *target, u8 *attr)
>> for (i = 0; i < dram->num_cs; i++) {
>> const struct mbus_dram_window *cs = dram->cs + i;
>>
>> - if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size)) {
>> + if (cs->base <= phyaddr && phyaddr <= (cs->base + cs->size - 1)) {
>> *target = dram->mbus_dram_target_id;
>> *attr = cs->mbus_attr;
>> return 0;
>>
>> I didn't get any errors during boot related to the BM. However I did not
>> manage to use an ethernet interface. The udhcpc never managed to get an
>> IP and if I set the IP manually I could not ping.
>>
>> But on Armada 388 GP I didn't have any issue.
>>
>> Do you have some idea about waht I could check?
>>
>
> I replaced 2GB with 8GB DIMM and after that, on the same kernel/board,
> I can't ping either. Very strange, but as I can reproduce the issue,
> don't debug it, I will.
Thanks to care about it!
Gregory
--
Gregory Clement, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
(no top posting please)
On 02/12/15 00:26, Marcin Wojtas wrote:
> Hi Florian,
>
> Can you please describe in more details, what would you expect from
> such special abstraction layer regarding buffer managers? I'd like to
> understand more of your expectations and evaluate possible work.
Well, something along these lines:
- have the ability to register a particular pool (location + number of
buffers) in a way that is relatively device agnostic (initialization
would of course be device specific)
- provide a set of buffer management APIs like those you proposed below,
and have some generic code that leverages what
drivers/net/ethernet/sun/niu.c does for instance
- introduce a netdev_alloc_skb_from_pool() or something like that which
would limit the amount of code to change in your network driver to
benefit from that feature so based
I am sure David would be able to suggest more detailed API.
>
> Best regards,
> Marcin
>
> 2015-11-30 17:25 GMT+01:00 David Miller <[email protected]>:
>> From: Marcin Wojtas <[email protected]>
>> Date: Mon, 30 Nov 2015 15:13:22 +0100
>>
>>> What kind of abstraction and helpers do you mean? Some kind of API
>>> (e.g. bm_alloc_buffer, bm_initialize_ring bm_put_buffer,
>>> bm_get_buffer), which would be used by platform drivers (and specific
>>> aplications if one wants to develop on top of the kernel)?
>>>
>>> In general, what is your top-view of such solution and its cooperation
>>> with the drivers?
>>
>> The tricky parts involved have to do with allocating pages for the
>> buffer pools and minimizing the number of atomic refcounting
>> operations on those pages for for the puts and gets, particularly
>> around buffer replenish runs.
>>
>> For example, if you're allocating a page for a buffer pool the device
>> will chop into N (for any N < PAGE_SIZE) byte pieces, you can
>> eliminate many atomic operations.
--
Florian
Hi David,
2015-12-04 21:15 GMT+01:00 Florian Fainelli <[email protected]>:
> (no top posting please)
>
> On 02/12/15 00:26, Marcin Wojtas wrote:
>> Hi Florian,
>>
>> Can you please describe in more details, what would you expect from
>> such special abstraction layer regarding buffer managers? I'd like to
>> understand more of your expectations and evaluate possible work.
>
> Well, something along these lines:
>
> - have the ability to register a particular pool (location + number of
> buffers) in a way that is relatively device agnostic (initialization
> would of course be device specific)
>
> - provide a set of buffer management APIs like those you proposed below,
> and have some generic code that leverages what
> drivers/net/ethernet/sun/niu.c does for instance
>
> - introduce a netdev_alloc_skb_from_pool() or something like that which
> would limit the amount of code to change in your network driver to
> benefit from that feature so based
>
> I am sure David would be able to suggest more detailed API.
>
As we're getting closer to what a generic BM part, could you please
share your thoughts on the possible API?
>>>
>>>> What kind of abstraction and helpers do you mean? Some kind of API
>>>> (e.g. bm_alloc_buffer, bm_initialize_ring bm_put_buffer,
>>>> bm_get_buffer), which would be used by platform drivers (and specific
>>>> aplications if one wants to develop on top of the kernel)?
>>>>
>>>> In general, what is your top-view of such solution and its cooperation
>>>> with the drivers?
>>>
>>> The tricky parts involved have to do with allocating pages for the
>>> buffer pools and minimizing the number of atomic refcounting
>>> operations on those pages for for the puts and gets, particularly
>>> around buffer replenish runs.
>>>
>>> For example, if you're allocating a page for a buffer pool the device
>>> will chop into N (for any N < PAGE_SIZE) byte pieces, you can
>>> eliminate many atomic operations.
>
Do you think you can point to anything similar that could be a sort of
reference for such solution?
Best regards,
Marcin
From: Marcin Wojtas <[email protected]>
Date: Tue, 8 Dec 2015 11:56:28 +0100
> Do you think you can point to anything similar that could be a sort
> of reference for such solution?
I don't have much time to explain because I'm constantly backlogged doing
patch review, but mostly it has to do with page management and avoiding
atomic decrement/increment of page counts wherever possible while feeding
blocks to the chip and receiving packets.
Please don't ask me for any more details as I'm really backlogged doing
patch review at the moment.
Thanks.