Set the 'parent' field of channel class devices to point to the
physical DMA device initialized by the DMA engine driver.
This allows drivers to use chan->dev.parent for syncing DMA buffers
and adds a 'device' symlink to the real device in
/sys/class/dma/dmaXchanY.
Signed-off-by: Haavard Skinnemoen <[email protected]>
---
drivers/dma/dmaengine.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 2996523..7c7cb4b 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -379,7 +379,7 @@ int dma_async_device_register(struct dma_device *device)
chan->chan_id = chancnt++;
chan->dev.class = &dma_devclass;
- chan->dev.parent = NULL;
+ chan->dev.parent = device->dev;
snprintf(chan->dev.bus_id, BUS_ID_SIZE, "dma%dchan%d",
device->dev_id, chan->chan_id);
--
1.5.3.8
Signed-off-by: Haavard Skinnemoen <[email protected]>
---
drivers/dma/Kconfig | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index 893a3f8..1a727c1 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -4,7 +4,7 @@
menuconfig DMADEVICES
bool "DMA Engine support"
- depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX
+ depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX || AVR32
depends on !HIGHMEM64G
help
DMA engines can do asynchronous data transfers without
--
1.5.3.8
This moves the code checking if a DMA channel is in use from
show_in_use() into an inline helper function, dma_is_in_use(). DMA
controllers can use this in order to give clients exclusive access to
channels (usually necessary when setting up slave DMA.)
I have to admit that I don't really understand the channel refcounting
logic at all... dma_chan_get() simply increments a per-cpu value. How
can we be sure that whatever CPU calls dma_chan_is_in_use() sees the
same value?
Signed-off-by: Haavard Skinnemoen <[email protected]>
---
drivers/dma/dmaengine.c | 12 +-----------
include/linux/dmaengine.h | 17 +++++++++++++++++
2 files changed, 18 insertions(+), 11 deletions(-)
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 9d11f06..3076d80 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -105,17 +105,7 @@ static ssize_t show_bytes_transferred(struct device *dev, struct device_attribut
static ssize_t show_in_use(struct device *dev, struct device_attribute *attr, char *buf)
{
struct dma_chan *chan = to_dma_chan(dev);
- int in_use = 0;
-
- if (unlikely(chan->slow_ref) &&
- atomic_read(&chan->refcount.refcount) > 1)
- in_use = 1;
- else {
- if (local_read(&(per_cpu_ptr(chan->local,
- get_cpu())->refcount)) > 0)
- in_use = 1;
- put_cpu();
- }
+ int in_use = dma_chan_is_in_use(chan);
return sprintf(buf, "%d\n", in_use);
}
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 9cd7ed9..5d9a3a2 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -175,6 +175,23 @@ static inline void dma_chan_put(struct dma_chan *chan)
}
}
+static inline bool dma_chan_is_in_use(struct dma_chan *chan)
+{
+ bool in_use = false;
+
+ if (unlikely(chan->slow_ref) &&
+ atomic_read(&chan->refcount.refcount) > 1)
+ in_use = true;
+ else {
+ if (local_read(&(per_cpu_ptr(chan->local,
+ get_cpu())->refcount)) > 0)
+ in_use = true;
+ put_cpu();
+ }
+
+ return in_use;
+}
+
/*
* typedef dma_event_callback - function pointer to a DMA event callback
* For each channel added to the system this routine is called for each client.
--
1.5.3.8
This patch adds the necessary interfaces to the DMA Engine framework
to use functionality found on most embedded DMA controllers: DMA from
and to I/O registers with hardware handshaking.
In this context, hardware hanshaking means that the peripheral that
owns the I/O registers in question is able to tell the DMA controller
when more data is available for reading, or when there is room for
more data to be written. This usually happens internally on the chip,
but these signals may also be exported outside the chip for things
like IDE DMA, etc.
A new struct dma_slave is introduced. This contains information that
the DMA engine driver needs to set up slave transfers to and from a
slave device. Most engines supporting DMA slave transfers will want to
extend this structure with controller-specific parameters. This
additional information is usually passed from the platform/board code
through the client driver.
A "slave" pointer is added to the dma_client struct. This must point
to a valid dma_slave structure iff the DMA_SLAVE capability is
requested. The DMA engine driver may use this information in its
device_alloc_chan_resources hook to configure the DMA controller for
slave transfers from and to the given slave device.
A new struct dma_slave_descriptor is added. This extends the standard
dma_async_tx_descriptor with a few members that are needed for doing
slave DMA from/to peripherals.
A new operation for creating such descriptors is added to struct
dma_device. Another new operation for terminating all pending
transfers is added as well. The latter is needed because there may be
errors outside the scope of the DMA Engine framework that may require
DMA operations to be terminated prematurely.
DMA Engine drivers may extend the dma_device, dma_chan and/or
dma_slave_descriptor structures to allow controller-specific
operations. The client driver can detect such extensions by looking at
the DMA Engine's struct device, or it can request a specific DMA
Engine device by setting the dma_dev field in struct dma_slave.
Signed-off-by: Haavard Skinnemoen <[email protected]>
dmaslave interface changes since v2:
* Add a dma_dev field to struct dma_slave. If set, the client can
only be bound to the DMA controller that corresponds to this
device. This allows controller-specific extensions of the
dma_slave structure; if the device matches, the controller may
safely assume its extensions are present.
* Move reg_width into struct dma_slave as there are currently no
users that need to be able to set the width on a per-transfer
basis.
dmaslave interface changes since v1:
* Drop the set_direction and set_width descriptor hooks. Pass the
direction and width to the prep function instead.
* Declare a dma_slave struct with fixed information about a slave,
i.e. register addresses, handshake interfaces and such.
* Add pointer to a dma_slave struct to dma_client. Can be NULL if
the DMA_SLAVE capability isn't requested.
* Drop the set_slave device hook since the alloc_chan_resources hook
now has enough information to set up the channel for slave
transfers.
---
drivers/dma/dmaengine.c | 16 +++++++++-
include/linux/dmaengine.h | 74 ++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 88 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 3076d80..b689140 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -159,7 +159,12 @@ static void dma_client_chan_alloc(struct dma_client *client)
enum dma_state_client ack;
/* Find a channel */
- list_for_each_entry(device, &dma_device_list, global_node)
+ list_for_each_entry(device, &dma_device_list, global_node) {
+ /* Does the client require a specific DMA controller? */
+ if (client->slave && client->slave->dma_dev
+ && client->slave->dma_dev != device->dev)
+ continue;
+
list_for_each_entry(chan, &device->channels, device_node) {
if (!dma_chan_satisfies_mask(chan, client->cap_mask))
continue;
@@ -180,6 +185,7 @@ static void dma_client_chan_alloc(struct dma_client *client)
return;
}
}
+ }
}
enum dma_status dma_sync_wait(struct dma_chan *chan, dma_cookie_t cookie)
@@ -276,6 +282,10 @@ static void dma_clients_notify_removed(struct dma_chan *chan)
*/
void dma_async_client_register(struct dma_client *client)
{
+ /* validate client data */
+ BUG_ON(dma_has_cap(DMA_SLAVE, client->cap_mask) &&
+ !client->slave);
+
mutex_lock(&dma_list_mutex);
list_add_tail(&client->global_node, &dma_client_list);
mutex_unlock(&dma_list_mutex);
@@ -350,6 +360,10 @@ int dma_async_device_register(struct dma_device *device)
!device->device_prep_dma_memset);
BUG_ON(dma_has_cap(DMA_ZERO_SUM, device->cap_mask) &&
!device->device_prep_dma_interrupt);
+ BUG_ON(dma_has_cap(DMA_SLAVE, device->cap_mask) &&
+ !device->device_prep_slave);
+ BUG_ON(dma_has_cap(DMA_SLAVE, device->cap_mask) &&
+ !device->device_terminate_all);
BUG_ON(!device->device_alloc_chan_resources);
BUG_ON(!device->device_free_chan_resources);
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 5d9a3a2..6708733 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -89,10 +89,33 @@ enum dma_transaction_type {
DMA_MEMSET,
DMA_MEMCPY_CRC32C,
DMA_INTERRUPT,
+ DMA_SLAVE,
};
/* last transaction type for creation of the capabilities mask */
-#define DMA_TX_TYPE_END (DMA_INTERRUPT + 1)
+#define DMA_TX_TYPE_END (DMA_SLAVE + 1)
+
+/**
+ * enum dma_slave_direction - direction of a DMA slave transfer
+ * @DMA_SLAVE_TO_MEMORY: Transfer data from peripheral to memory
+ * @DMA_SLAVE_FROM_MEMORY: Transfer data from memory to peripheral
+ */
+enum dma_slave_direction {
+ DMA_SLAVE_TO_MEMORY,
+ DMA_SLAVE_FROM_MEMORY,
+};
+
+/**
+ * enum dma_slave_width - DMA slave register access width.
+ * @DMA_SLAVE_WIDTH_8BIT: Do 8-bit slave register accesses
+ * @DMA_SLAVE_WIDTH_16BIT: Do 16-bit slave register accesses
+ * @DMA_SLAVE_WIDTH_32BIT: Do 32-bit slave register accesses
+ */
+enum dma_slave_width {
+ DMA_SLAVE_WIDTH_8BIT,
+ DMA_SLAVE_WIDTH_16BIT,
+ DMA_SLAVE_WIDTH_32BIT,
+};
/**
* enum dma_prep_flags - DMA flags to augment operation preparation
@@ -110,6 +133,33 @@ enum dma_prep_flags {
typedef struct { DECLARE_BITMAP(bits, DMA_TX_TYPE_END); } dma_cap_mask_t;
/**
+ * struct dma_slave - Information about a DMA slave
+ * @dev: device acting as DMA slave
+ * @dma_dev: required DMA master device. If non-NULL, the client can not be
+ * bound to other masters than this. The master driver may use
+ * this to determine whether it's safe to access
+ * @tx_reg: physical address of data register used for
+ * memory-to-peripheral transfers
+ * @rx_reg: physical address of data register used for
+ * peripheral-to-memory transfers
+ * @reg_width: peripheral register width
+ *
+ * If dma_dev is non-NULL, the client can not be bound to other DMA
+ * masters than the one corresponding to this device. The DMA master
+ * driver may use this to determine if there is controller-specific
+ * data wrapped around this struct. Drivers of platform code that sets
+ * the dma_dev field must therefore make sure to use an appropriate
+ * controller-specific dma slave structure wrapping this struct.
+ */
+struct dma_slave {
+ struct device *dev;
+ struct device *dma_dev;
+ dma_addr_t tx_reg;
+ dma_addr_t rx_reg;
+ enum dma_slave_width reg_width;
+};
+
+/**
* struct dma_chan_percpu - the per-CPU part of struct dma_chan
* @refcount: local_t used for open-coded "bigref" counting
* @memcpy_count: transaction counter
@@ -214,11 +264,14 @@ typedef enum dma_state_client (*dma_event_callback) (struct dma_client *client,
* @event_callback: func ptr to call when something happens
* @cap_mask: only return channels that satisfy the requested capabilities
* a value of zero corresponds to any capability
+ * @slave: data for preparing slave transfer. Must be non-NULL iff the
+ * DMA_SLAVE capability is requested.
* @global_node: list_head for global dma_client_list
*/
struct dma_client {
dma_event_callback event_callback;
dma_cap_mask_t cap_mask;
+ struct dma_slave *slave;
struct list_head global_node;
};
@@ -260,6 +313,17 @@ struct dma_async_tx_descriptor {
};
/**
+ * struct dma_slave_descriptor - extended DMA descriptor for slave DMA
+ * @async_tx: async transaction descriptor
+ * @client_node: for use by the client, for example when operating on
+ * scatterlists.
+ */
+struct dma_slave_descriptor {
+ struct dma_async_tx_descriptor txd;
+ struct list_head client_node;
+};
+
+/**
* struct dma_device - info on the entity supplying DMA services
* @chancnt: how many DMA channels are supported
* @channels: the list of struct dma_chan
@@ -278,6 +342,8 @@ struct dma_async_tx_descriptor {
* @device_prep_dma_zero_sum: prepares a zero_sum operation
* @device_prep_dma_memset: prepares a memset operation
* @device_prep_dma_interrupt: prepares an end of chain interrupt operation
+ * @device_prep_slave: prepares a slave dma operation
+ * @device_terminate_all: terminate all pending operations
* @device_dependency_added: async_tx notifies the channel about new deps
* @device_issue_pending: push pending transactions to hardware
*/
@@ -314,6 +380,12 @@ struct dma_device {
struct dma_async_tx_descriptor *(*device_prep_dma_interrupt)(
struct dma_chan *chan);
+ struct dma_slave_descriptor *(*device_prep_slave)(
+ struct dma_chan *chan, dma_addr_t mem_addr,
+ enum dma_slave_direction direction,
+ size_t len, unsigned long flags);
+ void (*device_terminate_all)(struct dma_chan *chan);
+
void (*device_dependency_added)(struct dma_chan *chan);
enum dma_status (*device_is_tx_complete)(struct dma_chan *chan,
dma_cookie_t cookie, dma_cookie_t *last,
--
1.5.3.8
A DMA controller capable of doing slave transfers may need to know a
few things about the slave when preparing the channel. We don't want
to add this information to struct dma_channel since the channel hasn't
yet been bound to a client at this point.
Instead, pass a reference to the client requesting the channel to the
driver's device_alloc_chan_resources hook so that it can pick the
necessary information from the dma_client struct by itself.
Signed-off-by: Haavard Skinnemoen <[email protected]>
---
drivers/dma/dmaengine.c | 3 ++-
drivers/dma/ioat_dma.c | 5 +++--
drivers/dma/iop-adma.c | 7 ++++---
include/linux/dmaengine.h | 3 ++-
4 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 7c7cb4b..9d11f06 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -174,7 +174,8 @@ static void dma_client_chan_alloc(struct dma_client *client)
if (!dma_chan_satisfies_mask(chan, client->cap_mask))
continue;
- desc = chan->device->device_alloc_chan_resources(chan);
+ desc = chan->device->device_alloc_chan_resources(
+ chan, client);
if (desc >= 0) {
ack = client->event_callback(client,
chan,
diff --git a/drivers/dma/ioat_dma.c b/drivers/dma/ioat_dma.c
index dff38ac..3e82bfb 100644
--- a/drivers/dma/ioat_dma.c
+++ b/drivers/dma/ioat_dma.c
@@ -452,7 +452,8 @@ static void ioat2_dma_massage_chan_desc(struct ioat_dma_chan *ioat_chan)
* ioat_dma_alloc_chan_resources - returns the number of allocated descriptors
* @chan: the channel to be filled out
*/
-static int ioat_dma_alloc_chan_resources(struct dma_chan *chan)
+static int ioat_dma_alloc_chan_resources(struct dma_chan *chan,
+ struct dma_client *client)
{
struct ioat_dma_chan *ioat_chan = to_ioat_chan(chan);
struct ioat_desc_sw *desc;
@@ -1058,7 +1059,7 @@ static int ioat_dma_self_test(struct ioatdma_device *device)
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (device->common.device_alloc_chan_resources(dma_chan) < 1) {
+ if (device->common.device_alloc_chan_resources(dma_chan, NULL) < 1) {
dev_err(&device->pdev->dev,
"selftest cannot allocate chan resource\n");
err = -ENODEV;
diff --git a/drivers/dma/iop-adma.c b/drivers/dma/iop-adma.c
index 3986d54..aacda12 100644
--- a/drivers/dma/iop-adma.c
+++ b/drivers/dma/iop-adma.c
@@ -447,7 +447,8 @@ static void iop_chan_start_null_memcpy(struct iop_adma_chan *iop_chan);
static void iop_chan_start_null_xor(struct iop_adma_chan *iop_chan);
/* returns the number of allocated descriptors */
-static int iop_adma_alloc_chan_resources(struct dma_chan *chan)
+static int iop_adma_alloc_chan_resources(struct dma_chan *chan,
+ struct dma_client *client)
{
char *hw_desc;
int idx;
@@ -844,7 +845,7 @@ static int __devinit iop_adma_memcpy_self_test(struct iop_adma_device *device)
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
+ if (iop_adma_alloc_chan_resources(dma_chan, NULL) < 1) {
err = -ENODEV;
goto out;
}
@@ -942,7 +943,7 @@ iop_adma_xor_zero_sum_self_test(struct iop_adma_device *device)
dma_chan = container_of(device->common.channels.next,
struct dma_chan,
device_node);
- if (iop_adma_alloc_chan_resources(dma_chan) < 1) {
+ if (iop_adma_alloc_chan_resources(dma_chan, NULL) < 1) {
err = -ENODEV;
goto out;
}
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index acbb364..9cd7ed9 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -278,7 +278,8 @@ struct dma_device {
int dev_id;
struct device *dev;
- int (*device_alloc_chan_resources)(struct dma_chan *chan);
+ int (*device_alloc_chan_resources)(struct dma_chan *chan,
+ struct dma_client *client);
void (*device_free_chan_resources)(struct dma_chan *chan);
struct dma_async_tx_descriptor *(*device_prep_dma_memcpy)(
--
1.5.3.8
This is a driver for the MMC controller on the AP7000 chips from
Atmel. It should in theory work on AT91 systems too with some
tweaking, but since the DMA interface is quite different, it's not
entirely clear if it's worth it.
This driver has been around for a while in BSPs and kernel sources
provided by Atmel, but this particular version uses the generic DMA
Engine framework (with the slave extensions) instead of an
avr32-only DMA controller framework.
This driver can also use PIO transfers when no DMA channels are
available, and for transfers where using DMA may be difficult or
impractical for some reason (e.g. the DMA setup overhead is usually
not worth it for very short transfers.)
The driver has been tested using mmc-block and ext3fs on several SD,
SDHC and MMC+ cards. Reads and writes work fine, with read transfer
rates up to 3.4MB/s with lots of debug options enabled.
The driver has also been tested using the mmc_test module on the same
cards, with somewhat less convincing results. In particular, badly
aligned multiblock writes seem to confuse some of the cards.
Signed-off-by: Haavard Skinnemoen <[email protected]>
Changes since v2:
* Reset the controller after each transfer since we're violating the
spec sometimes. This is very cheap, so we don't try to be clever.
* Turn off the MMC clock when no requests are pending.
* Implement support for PIO transfers (i.e. not using DMA.)
* Rename atmel-mci.h -> atmel-mci-regs.h
* Use controller-specific data passed from the platform code to set
up DMA slave transfers. These parameters include including physical
DMA device, peripheral handshake IDs, channel priorities, etc.
* Fix several card removal bugs
---
arch/avr32/boards/atngw100/setup.c | 7 +
arch/avr32/boards/atstk1000/atstk1002.c | 3 +
arch/avr32/mach-at32ap/at32ap700x.c | 47 +-
drivers/mmc/host/Kconfig | 10 +
drivers/mmc/host/Makefile | 1 +
drivers/mmc/host/atmel-mci-regs.h | 194 +++++
drivers/mmc/host/atmel-mci.c | 1400 +++++++++++++++++++++++++++++++
include/asm-avr32/arch-at32ap/board.h | 6 +-
include/asm-avr32/atmel-mci.h | 12 +
9 files changed, 1674 insertions(+), 6 deletions(-)
create mode 100644 drivers/mmc/host/atmel-mci-regs.h
create mode 100644 drivers/mmc/host/atmel-mci.c
create mode 100644 include/asm-avr32/atmel-mci.h
diff --git a/arch/avr32/boards/atngw100/setup.c b/arch/avr32/boards/atngw100/setup.c
index a398be2..96833bf 100644
--- a/arch/avr32/boards/atngw100/setup.c
+++ b/arch/avr32/boards/atngw100/setup.c
@@ -17,6 +17,7 @@
#include <linux/leds.h>
#include <linux/spi/spi.h>
+#include <asm/atmel-mci.h>
#include <asm/io.h>
#include <asm/setup.h>
@@ -42,6 +43,11 @@ static struct spi_board_info spi0_board_info[] __initdata = {
},
};
+static struct mci_platform_data __initdata mci0_data = {
+ .detect_pin = GPIO_PIN_PC(25),
+ .wp_pin = GPIO_PIN_PE(0),
+};
+
/*
* The next two functions should go away as the boot loader is
* supposed to initialize the macb address registers with a valid
@@ -157,6 +163,7 @@ static int __init atngw100_init(void)
set_hw_addr(at32_add_device_eth(1, ð_data[1]));
at32_add_device_spi(0, spi0_board_info, ARRAY_SIZE(spi0_board_info));
+ at32_add_device_mci(0, &mci0_data);
at32_add_device_usba(0, NULL);
for (i = 0; i < ARRAY_SIZE(ngw_leds); i++) {
diff --git a/arch/avr32/boards/atstk1000/atstk1002.c b/arch/avr32/boards/atstk1000/atstk1002.c
index 000eb42..8b92cd6 100644
--- a/arch/avr32/boards/atstk1000/atstk1002.c
+++ b/arch/avr32/boards/atstk1000/atstk1002.c
@@ -228,6 +228,9 @@ static int __init atstk1002_init(void)
#ifdef CONFIG_BOARD_ATSTK100X_SPI1
at32_add_device_spi(1, spi1_board_info, ARRAY_SIZE(spi1_board_info));
#endif
+#ifndef CONFIG_BOARD_ATSTK1002_SW2_CUSTOM
+ at32_add_device_mci(0, NULL);
+#endif
#ifdef CONFIG_BOARD_ATSTK1002_SW5_CUSTOM
set_hw_addr(at32_add_device_eth(1, ð_data[1]));
#else
diff --git a/arch/avr32/mach-at32ap/at32ap700x.c b/arch/avr32/mach-at32ap/at32ap700x.c
index 038e17b..0a0a499 100644
--- a/arch/avr32/mach-at32ap/at32ap700x.c
+++ b/arch/avr32/mach-at32ap/at32ap700x.c
@@ -6,12 +6,14 @@
* published by the Free Software Foundation.
*/
#include <linux/clk.h>
+#include <linux/dw_dmac.h>
#include <linux/fb.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/dma-mapping.h>
#include <linux/spi/spi.h>
+#include <asm/atmel-mci.h>
#include <asm/io.h>
#include <asm/irq.h>
@@ -1032,20 +1034,48 @@ static struct clk atmel_mci0_pclk = {
.index = 9,
};
-struct platform_device *__init at32_add_device_mci(unsigned int id)
+struct platform_device *__init
+at32_add_device_mci(unsigned int id, struct mci_platform_data *data)
{
- struct platform_device *pdev;
+ struct mci_platform_data _data;
+ struct platform_device *pdev;
+ struct dw_dma_slave *dws;
if (id != 0)
return NULL;
pdev = platform_device_alloc("atmel_mci", id);
if (!pdev)
- return NULL;
+ goto fail;
if (platform_device_add_resources(pdev, atmel_mci0_resource,
ARRAY_SIZE(atmel_mci0_resource)))
- goto err_add_resources;
+ goto fail;
+
+ if (!data) {
+ data = &_data;
+ memset(data, 0, sizeof(struct mci_platform_data));
+ }
+
+ if (data->dma_slave)
+ dws = kmemdup(to_dw_dma_slave(data->dma_slave),
+ sizeof(struct dw_dma_slave), GFP_KERNEL);
+ else
+ dws = kzalloc(sizeof(struct dw_dma_slave), GFP_KERNEL);
+
+ dws->slave.dev = &pdev->dev;
+ dws->slave.dma_dev = &dw_dmac0_device.dev;
+ dws->slave.reg_width = DMA_SLAVE_WIDTH_32BIT;
+ dws->cfg_hi = (DWC_CFGH_SRC_PER(0)
+ | DWC_CFGH_DST_PER(1));
+ dws->cfg_lo &= ~(DWC_CFGL_HS_DST_POL
+ | DWC_CFGL_HS_SRC_POL);
+
+ data->dma_slave = &dws->slave;
+
+ if (platform_device_add_data(pdev, data,
+ sizeof(struct mci_platform_data)))
+ goto fail;
select_peripheral(PA(10), PERIPH_A, 0); /* CLK */
select_peripheral(PA(11), PERIPH_A, 0); /* CMD */
@@ -1054,12 +1084,19 @@ struct platform_device *__init at32_add_device_mci(unsigned int id)
select_peripheral(PA(14), PERIPH_A, 0); /* DATA2 */
select_peripheral(PA(15), PERIPH_A, 0); /* DATA3 */
+ if (data) {
+ if (data->detect_pin != GPIO_PIN_NONE)
+ at32_select_gpio(data->detect_pin, 0);
+ if (data->wp_pin != GPIO_PIN_NONE)
+ at32_select_gpio(data->wp_pin, 0);
+ }
+
atmel_mci0_pclk.dev = &pdev->dev;
platform_device_add(pdev);
return pdev;
-err_add_resources:
+fail:
platform_device_put(pdev);
return NULL;
}
diff --git a/drivers/mmc/host/Kconfig b/drivers/mmc/host/Kconfig
index 3b3cd0e..f4b21de 100644
--- a/drivers/mmc/host/Kconfig
+++ b/drivers/mmc/host/Kconfig
@@ -91,6 +91,16 @@ config MMC_AT91
If unsure, say N.
+config MMC_ATMELMCI
+ tristate "Atmel Multimedia Card Interface support"
+ depends on AVR32 && DMA_ENGINE
+ help
+ This selects the Atmel Multimedia Card Interface driver. If
+ you have an AT32 (AVR32) platform with a Multimedia Card
+ slot, say Y or M here.
+
+ If unsure, say N.
+
config MMC_IMX
tristate "Motorola i.MX Multimedia Card Interface support"
depends on ARCH_IMX
diff --git a/drivers/mmc/host/Makefile b/drivers/mmc/host/Makefile
index 3877c87..e80ea72 100644
--- a/drivers/mmc/host/Makefile
+++ b/drivers/mmc/host/Makefile
@@ -15,6 +15,7 @@ obj-$(CONFIG_MMC_WBSD) += wbsd.o
obj-$(CONFIG_MMC_AU1X) += au1xmmc.o
obj-$(CONFIG_MMC_OMAP) += omap.o
obj-$(CONFIG_MMC_AT91) += at91_mci.o
+obj-$(CONFIG_MMC_ATMELMCI) += atmel-mci.o
obj-$(CONFIG_MMC_TIFM_SD) += tifm_sd.o
obj-$(CONFIG_MMC_SPI) += mmc_spi.o
diff --git a/drivers/mmc/host/atmel-mci-regs.h b/drivers/mmc/host/atmel-mci-regs.h
new file mode 100644
index 0000000..7719e37
--- /dev/null
+++ b/drivers/mmc/host/atmel-mci-regs.h
@@ -0,0 +1,194 @@
+/*
+ * Atmel MultiMedia Card Interface driver
+ *
+ * Copyright (C) 2004-2006 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#ifndef __DRIVERS_MMC_ATMEL_MCI_H__
+#define __DRIVERS_MMC_ATMEL_MCI_H__
+
+/* MCI register offsets */
+#define MCI_CR 0x0000
+#define MCI_MR 0x0004
+#define MCI_DTOR 0x0008
+#define MCI_SDCR 0x000c
+#define MCI_ARGR 0x0010
+#define MCI_CMDR 0x0014
+#define MCI_BLKR 0x0018
+#define MCI_RSPR 0x0020
+#define MCI_RSPR1 0x0024
+#define MCI_RSPR2 0x0028
+#define MCI_RSPR3 0x002c
+#define MCI_RDR 0x0030
+#define MCI_TDR 0x0034
+#define MCI_SR 0x0040
+#define MCI_IER 0x0044
+#define MCI_IDR 0x0048
+#define MCI_IMR 0x004c
+
+/* Bitfields in CR */
+#define MCI_MCIEN_OFFSET 0
+#define MCI_MCIEN_SIZE 1
+#define MCI_MCIDIS_OFFSET 1
+#define MCI_MCIDIS_SIZE 1
+#define MCI_PWSEN_OFFSET 2
+#define MCI_PWSEN_SIZE 1
+#define MCI_PWSDIS_OFFSET 3
+#define MCI_PWSDIS_SIZE 1
+#define MCI_SWRST_OFFSET 7
+#define MCI_SWRST_SIZE 1
+
+/* Bitfields in MR */
+#define MCI_CLKDIV_OFFSET 0
+#define MCI_CLKDIV_SIZE 8
+#define MCI_PWSDIV_OFFSET 8
+#define MCI_PWSDIV_SIZE 3
+#define MCI_RDPROOF_OFFSET 11
+#define MCI_RDPROOF_SIZE 1
+#define MCI_WRPROOF_OFFSET 12
+#define MCI_WRPROOF_SIZE 1
+#define MCI_PDCFBYTE_OFFSET 13
+#define MCI_PDCFBYTE_SIZE 1
+#define MCI_DMAPADV_OFFSET 14
+#define MCI_DMAPADV_SIZE 1
+#define MCI_BLKLEN_OFFSET 16
+#define MCI_BLKLEN_SIZE 16
+
+/* Bitfields in DTOR */
+#define MCI_DTOCYC_OFFSET 0
+#define MCI_DTOCYC_SIZE 4
+#define MCI_DTOMUL_OFFSET 4
+#define MCI_DTOMUL_SIZE 3
+
+/* Bitfields in SDCR */
+#define MCI_SDCSEL_OFFSET 0
+#define MCI_SDCSEL_SIZE 4
+#define MCI_SDCBUS_OFFSET 7
+#define MCI_SDCBUS_SIZE 1
+
+/* Bitfields in ARGR */
+#define MCI_ARG_OFFSET 0
+#define MCI_ARG_SIZE 32
+
+/* Bitfields in CMDR */
+#define MCI_CMDNB_OFFSET 0
+#define MCI_CMDNB_SIZE 6
+#define MCI_RSPTYP_OFFSET 6
+#define MCI_RSPTYP_SIZE 2
+#define MCI_SPCMD_OFFSET 8
+#define MCI_SPCMD_SIZE 3
+#define MCI_OPDCMD_OFFSET 11
+#define MCI_OPDCMD_SIZE 1
+#define MCI_MAXLAT_OFFSET 12
+#define MCI_MAXLAT_SIZE 1
+#define MCI_TRCMD_OFFSET 16
+#define MCI_TRCMD_SIZE 2
+#define MCI_TRDIR_OFFSET 18
+#define MCI_TRDIR_SIZE 1
+#define MCI_TRTYP_OFFSET 19
+#define MCI_TRTYP_SIZE 2
+
+/* Bitfields in BLKR */
+#define MCI_BCNT_OFFSET 0
+#define MCI_BCNT_SIZE 16
+
+/* Bitfields in RSPRn */
+#define MCI_RSP_OFFSET 0
+#define MCI_RSP_SIZE 32
+
+/* Bitfields in SR/IER/IDR/IMR */
+#define MCI_CMDRDY_OFFSET 0
+#define MCI_CMDRDY_SIZE 1
+#define MCI_RXRDY_OFFSET 1
+#define MCI_RXRDY_SIZE 1
+#define MCI_TXRDY_OFFSET 2
+#define MCI_TXRDY_SIZE 1
+#define MCI_BLKE_OFFSET 3
+#define MCI_BLKE_SIZE 1
+#define MCI_DTIP_OFFSET 4
+#define MCI_DTIP_SIZE 1
+#define MCI_NOTBUSY_OFFSET 5
+#define MCI_NOTBUSY_SIZE 1
+#define MCI_ENDRX_OFFSET 6
+#define MCI_ENDRX_SIZE 1
+#define MCI_ENDTX_OFFSET 7
+#define MCI_ENDTX_SIZE 1
+#define MCI_RXBUFF_OFFSET 14
+#define MCI_RXBUFF_SIZE 1
+#define MCI_TXBUFE_OFFSET 15
+#define MCI_TXBUFE_SIZE 1
+#define MCI_RINDE_OFFSET 16
+#define MCI_RINDE_SIZE 1
+#define MCI_RDIRE_OFFSET 17
+#define MCI_RDIRE_SIZE 1
+#define MCI_RCRCE_OFFSET 18
+#define MCI_RCRCE_SIZE 1
+#define MCI_RENDE_OFFSET 19
+#define MCI_RENDE_SIZE 1
+#define MCI_RTOE_OFFSET 20
+#define MCI_RTOE_SIZE 1
+#define MCI_DCRCE_OFFSET 21
+#define MCI_DCRCE_SIZE 1
+#define MCI_DTOE_OFFSET 22
+#define MCI_DTOE_SIZE 1
+#define MCI_OVRE_OFFSET 30
+#define MCI_OVRE_SIZE 1
+#define MCI_UNRE_OFFSET 31
+#define MCI_UNRE_SIZE 1
+
+/* Constants for DTOMUL */
+#define MCI_DTOMUL_1_CYCLE 0
+#define MCI_DTOMUL_16_CYCLES 1
+#define MCI_DTOMUL_128_CYCLES 2
+#define MCI_DTOMUL_256_CYCLES 3
+#define MCI_DTOMUL_1024_CYCLES 4
+#define MCI_DTOMUL_4096_CYCLES 5
+#define MCI_DTOMUL_65536_CYCLES 6
+#define MCI_DTOMUL_1048576_CYCLES 7
+
+/* Constants for RSPTYP */
+#define MCI_RSPTYP_NO_RESP 0
+#define MCI_RSPTYP_48_BIT 1
+#define MCI_RSPTYP_136_BIT 2
+
+/* Constants for SPCMD */
+#define MCI_SPCMD_NO_SPEC_CMD 0
+#define MCI_SPCMD_INIT_CMD 1
+#define MCI_SPCMD_SYNC_CMD 2
+#define MCI_SPCMD_INT_CMD 4
+#define MCI_SPCMD_INT_RESP 5
+
+/* Constants for TRCMD */
+#define MCI_TRCMD_NO_TRANS 0
+#define MCI_TRCMD_START_TRANS 1
+#define MCI_TRCMD_STOP_TRANS 2
+
+/* Constants for TRTYP */
+#define MCI_TRTYP_BLOCK 0
+#define MCI_TRTYP_MULTI_BLOCK 1
+#define MCI_TRTYP_STREAM 2
+
+/* Bit manipulation macros */
+#define MCI_BIT(name) \
+ (1 << MCI_##name##_OFFSET)
+#define MCI_BF(name,value) \
+ (((value) & ((1 << MCI_##name##_SIZE) - 1)) \
+ << MCI_##name##_OFFSET)
+#define MCI_BFEXT(name,value) \
+ (((value) >> MCI_##name##_OFFSET) \
+ & ((1 << MCI_##name##_SIZE) - 1))
+#define MCI_BFINS(name,value,old) \
+ (((old) & ~(((1 << MCI_##name##_SIZE) - 1) \
+ << MCI_##name##_OFFSET)) \
+ | MCI_BF(name,value))
+
+/* Register access macros */
+#define mci_readl(port,reg) \
+ __raw_readl((port)->regs + MCI_##reg)
+#define mci_writel(port,reg,value) \
+ __raw_writel((value), (port)->regs + MCI_##reg)
+
+#endif /* __DRIVERS_MMC_ATMEL_MCI_H__ */
diff --git a/drivers/mmc/host/atmel-mci.c b/drivers/mmc/host/atmel-mci.c
new file mode 100644
index 0000000..d1e29dc
--- /dev/null
+++ b/drivers/mmc/host/atmel-mci.c
@@ -0,0 +1,1400 @@
+/*
+ * Atmel MultiMedia Card Interface driver
+ *
+ * Copyright (C) 2004-2007 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#define VERBOSE_DEBUG
+#include <linux/blkdev.h>
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/ioport.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <linux/mmc/host.h>
+
+#include <asm/atmel-mci.h>
+#include <asm/io.h>
+#include <asm/unaligned.h>
+
+#include <asm/arch/board.h>
+#include <asm/arch/gpio.h>
+
+#include "atmel-mci-regs.h"
+
+#define ATMCI_DATA_ERROR_FLAGS (MCI_BIT(DCRCE) | MCI_BIT(DTOE) | \
+ MCI_BIT(OVRE) | MCI_BIT(UNRE))
+
+#define ATMCI_DMA_THRESHOLD 16
+
+enum {
+ EVENT_CMD_COMPLETE = 0,
+ EVENT_DATA_ERROR,
+ EVENT_DATA_COMPLETE,
+ EVENT_STOP_SENT,
+ EVENT_STOP_COMPLETE,
+ EVENT_DMA_COMPLETE,
+ EVENT_CARD_DETECT,
+};
+
+struct atmel_mci_dma {
+ struct dma_client client;
+ struct dma_chan *chan;
+ struct list_head data_descs;
+};
+
+struct atmel_mci {
+ struct mmc_host *mmc;
+ void __iomem *regs;
+
+ /* PIO stuff */
+ struct scatterlist *pio_sg;
+ unsigned int pio_offset;
+
+ struct mmc_request *mrq;
+ struct mmc_command *cmd;
+ struct mmc_data *data;
+
+ struct atmel_mci_dma dma;
+
+ /* DMA channel being used for the current data transfer */
+ struct dma_chan *data_chan;
+
+ u32 cmd_status;
+ u32 data_status;
+ u32 stop_status;
+ u32 stop_cmdr;
+
+ u32 mode_reg;
+ u32 sdc_reg;
+
+ struct tasklet_struct tasklet;
+ unsigned long pending_events;
+ unsigned long completed_events;
+
+ int present;
+ int detect_pin;
+ int wp_pin;
+
+ unsigned long bus_hz;
+ unsigned long mapbase;
+ struct clk *mck;
+ struct platform_device *pdev;
+
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs_root;
+ struct dentry *debugfs_regs;
+ struct dentry *debugfs_req;
+ struct dentry *debugfs_pending_events;
+ struct dentry *debugfs_completed_events;
+#endif
+};
+
+static inline struct atmel_mci *
+dma_client_to_atmel_mci(struct dma_client *client)
+{
+ return container_of(client, struct atmel_mci, dma.client);
+}
+
+/* Those printks take an awful lot of time... */
+#ifndef DEBUG
+static unsigned int fmax = 25000000U;
+#else
+static unsigned int fmax = 1000000U;
+#endif
+module_param(fmax, uint, 0444);
+MODULE_PARM_DESC(fmax, "Max frequency in Hz of the MMC bus clock");
+
+#define atmci_is_completed(host, event) \
+ test_bit(event, &host->completed_events)
+#define atmci_test_and_clear_pending(host, event) \
+ test_and_clear_bit(event, &host->pending_events)
+#define atmci_test_and_set_completed(host, event) \
+ test_and_set_bit(event, &host->completed_events)
+#define atmci_set_completed(host, event) \
+ set_bit(event, &host->completed_events)
+#define atmci_set_pending(host, event) \
+ set_bit(event, &host->pending_events)
+#define atmci_clear_pending(host, event) \
+ clear_bit(event, &host->pending_events)
+
+
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+
+#define DBG_REQ_BUF_SIZE (4096 - sizeof(unsigned int))
+
+struct req_dbg_data {
+ unsigned int nbytes;
+ char str[DBG_REQ_BUF_SIZE];
+};
+
+static int req_dbg_open(struct inode *inode, struct file *file)
+{
+ struct atmel_mci *host;
+ struct mmc_request *mrq;
+ struct mmc_command *cmd;
+ struct mmc_command *stop;
+ struct mmc_data *data;
+ struct req_dbg_data *priv;
+ char *str;
+ unsigned long n = 0;
+
+ priv = kzalloc(DBG_REQ_BUF_SIZE, GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+ str = priv->str;
+
+ mutex_lock(&inode->i_mutex);
+ host = inode->i_private;
+
+ spin_lock_irq(&host->mmc->lock);
+ mrq = host->mrq;
+ if (mrq) {
+ cmd = mrq->cmd;
+ data = mrq->data;
+ stop = mrq->stop;
+ n = snprintf(str, DBG_REQ_BUF_SIZE,
+ "CMD%u(0x%x) %x %x %x %x %x (err %u)\n",
+ cmd->opcode, cmd->arg, cmd->flags,
+ cmd->resp[0], cmd->resp[1], cmd->resp[2],
+ cmd->resp[3], cmd->error);
+ if (n < DBG_REQ_BUF_SIZE && data)
+ n += snprintf(str + n, DBG_REQ_BUF_SIZE - n,
+ "DATA %u * %u (%u) %x (err %u)\n",
+ data->blocks, data->blksz,
+ data->bytes_xfered, data->flags,
+ data->error);
+ if (n < DBG_REQ_BUF_SIZE && stop)
+ n += snprintf(str + n, DBG_REQ_BUF_SIZE - n,
+ "CMD%u(0x%x) %x %x %x %x %x (err %u)\n",
+ stop->opcode, stop->arg, stop->flags,
+ stop->resp[0], stop->resp[1],
+ stop->resp[2], stop->resp[3],
+ stop->error);
+ }
+ spin_unlock_irq(&host->mmc->lock);
+ mutex_unlock(&inode->i_mutex);
+
+ priv->nbytes = min(n, DBG_REQ_BUF_SIZE);
+ file->private_data = priv;
+
+ return 0;
+}
+
+static ssize_t req_dbg_read(struct file *file, char __user *buf,
+ size_t nbytes, loff_t *ppos)
+{
+ struct req_dbg_data *priv = file->private_data;
+
+ return simple_read_from_buffer(buf, nbytes, ppos,
+ priv->str, priv->nbytes);
+}
+
+static int req_dbg_release(struct inode *inode, struct file *file)
+{
+ kfree(file->private_data);
+ return 0;
+}
+
+static const struct file_operations req_dbg_fops = {
+ .owner = THIS_MODULE,
+ .open = req_dbg_open,
+ .llseek = no_llseek,
+ .read = req_dbg_read,
+ .release = req_dbg_release,
+};
+
+static int regs_dbg_open(struct inode *inode, struct file *file)
+{
+ struct atmel_mci *host;
+ unsigned int i;
+ u32 *data;
+ int ret;
+
+ mutex_lock(&inode->i_mutex);
+ host = inode->i_private;
+ data = kmalloc(inode->i_size, GFP_KERNEL);
+ if (!data) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ spin_lock_irq(&host->mmc->lock);
+ for (i = 0; i < inode->i_size / 4; i++)
+ data[i] = __raw_readl(host->regs + i * 4);
+ spin_unlock_irq(&host->mmc->lock);
+
+ file->private_data = data;
+ ret = 0;
+
+out:
+ mutex_unlock(&inode->i_mutex);
+
+ return ret;
+}
+
+static ssize_t regs_dbg_read(struct file *file, char __user *buf,
+ size_t nbytes, loff_t *ppos)
+{
+ struct inode *inode = file->f_dentry->d_inode;
+ int ret;
+
+ mutex_lock(&inode->i_mutex);
+ ret = simple_read_from_buffer(buf, nbytes, ppos,
+ file->private_data,
+ file->f_dentry->d_inode->i_size);
+ mutex_unlock(&inode->i_mutex);
+
+ return ret;
+}
+
+static int regs_dbg_release(struct inode *inode, struct file *file)
+{
+ kfree(file->private_data);
+ return 0;
+}
+
+static const struct file_operations regs_dbg_fops = {
+ .owner = THIS_MODULE,
+ .open = regs_dbg_open,
+ .llseek = generic_file_llseek,
+ .read = regs_dbg_read,
+ .release = regs_dbg_release,
+};
+
+static void atmci_init_debugfs(struct atmel_mci *host)
+{
+ struct mmc_host *mmc;
+ struct dentry *root;
+ struct dentry *regs;
+ struct resource *res;
+
+ mmc = host->mmc;
+ root = debugfs_create_dir(mmc_hostname(mmc), NULL);
+ if (IS_ERR(root) || !root)
+ goto err_root;
+ host->debugfs_root = root;
+
+ regs = debugfs_create_file("regs", 0400, root, host, ®s_dbg_fops);
+ if (!regs)
+ goto err_regs;
+
+ res = platform_get_resource(host->pdev, IORESOURCE_MEM, 0);
+ regs->d_inode->i_size = res->end - res->start + 1;
+ host->debugfs_regs = regs;
+
+ host->debugfs_req = debugfs_create_file("req", 0400, root,
+ host, &req_dbg_fops);
+ if (!host->debugfs_req)
+ goto err_req;
+
+ host->debugfs_pending_events
+ = debugfs_create_u32("pending_events", 0400, root,
+ (u32 *)&host->pending_events);
+ if (!host->debugfs_pending_events)
+ goto err_pending_events;
+
+ host->debugfs_completed_events
+ = debugfs_create_u32("completed_events", 0400, root,
+ (u32 *)&host->completed_events);
+ if (!host->debugfs_completed_events)
+ goto err_completed_events;
+
+ return;
+
+err_completed_events:
+ debugfs_remove(host->debugfs_pending_events);
+err_pending_events:
+ debugfs_remove(host->debugfs_req);
+err_req:
+ debugfs_remove(host->debugfs_regs);
+err_regs:
+ debugfs_remove(host->debugfs_root);
+err_root:
+ host->debugfs_root = NULL;
+ dev_err(&host->pdev->dev,
+ "failed to initialize debugfs for %s\n",
+ mmc_hostname(mmc));
+}
+
+static void atmci_cleanup_debugfs(struct atmel_mci *host)
+{
+ if (host->debugfs_root) {
+ debugfs_remove(host->debugfs_completed_events);
+ debugfs_remove(host->debugfs_pending_events);
+ debugfs_remove(host->debugfs_req);
+ debugfs_remove(host->debugfs_regs);
+ debugfs_remove(host->debugfs_root);
+ host->debugfs_root = NULL;
+ }
+}
+#else
+static inline void atmci_init_debugfs(struct atmel_mci *host)
+{
+
+}
+
+static inline void atmci_cleanup_debugfs(struct atmel_mci *host)
+{
+
+}
+#endif /* CONFIG_DEBUG_FS */
+
+static void atmci_enable(struct atmel_mci *host)
+{
+ clk_enable(host->mck);
+ mci_writel(host, CR, MCI_BIT(MCIEN));
+ mci_writel(host, MR, host->mode_reg);
+ mci_writel(host, SDCR, host->sdc_reg);
+}
+
+static void atmci_disable(struct atmel_mci *host)
+{
+ mci_writel(host, CR, MCI_BIT(SWRST));
+
+ /* Stall until write is complete, then disable the bus clock */
+ mci_readl(host, SR);
+ clk_disable(host->mck);
+}
+
+static inline unsigned int ns_to_clocks(struct atmel_mci *host,
+ unsigned int ns)
+{
+ return (ns * (host->bus_hz / 1000000) + 999) / 1000;
+}
+
+static void atmci_set_timeout(struct atmel_mci *host,
+ struct mmc_data *data)
+{
+ static unsigned dtomul_to_shift[] = {
+ 0, 4, 7, 8, 10, 12, 16, 20
+ };
+ unsigned timeout;
+ unsigned dtocyc;
+ unsigned dtomul;
+
+ timeout = ns_to_clocks(host, data->timeout_ns) + data->timeout_clks;
+
+ for (dtomul = 0; dtomul < 8; dtomul++) {
+ unsigned shift = dtomul_to_shift[dtomul];
+ dtocyc = (timeout + (1 << shift) - 1) >> shift;
+ if (dtocyc < 15)
+ break;
+ }
+
+ if (dtomul >= 8) {
+ dtomul = 7;
+ dtocyc = 15;
+ }
+
+ dev_vdbg(&host->mmc->class_dev, "setting timeout to %u cycles\n",
+ dtocyc << dtomul_to_shift[dtomul]);
+ mci_writel(host, DTOR, (MCI_BF(DTOMUL, dtomul)
+ | MCI_BF(DTOCYC, dtocyc)));
+}
+
+/*
+ * Return mask with command flags to be enabled for this command.
+ */
+static u32 atmci_prepare_command(struct mmc_host *mmc,
+ struct mmc_command *cmd)
+{
+ u32 cmdr;
+
+ cmd->error = 0;
+
+ cmdr = MCI_BF(CMDNB, cmd->opcode);
+
+ if (cmd->flags & MMC_RSP_PRESENT) {
+ if (cmd->flags & MMC_RSP_136)
+ cmdr |= MCI_BF(RSPTYP, MCI_RSPTYP_136_BIT);
+ else
+ cmdr |= MCI_BF(RSPTYP, MCI_RSPTYP_48_BIT);
+ }
+
+ /*
+ * This should really be MAXLAT_5 for CMD2 and ACMD41, but
+ * it's too difficult to determine whether this is an ACMD or
+ * not. Better make it 64.
+ */
+ cmdr |= MCI_BIT(MAXLAT);
+
+ if (mmc->ios.bus_mode == MMC_BUSMODE_OPENDRAIN)
+ cmdr |= MCI_BIT(OPDCMD);
+
+ return cmdr;
+}
+
+static void atmci_start_data(struct atmel_mci *host, struct mmc_data *data)
+{
+ struct dma_slave_descriptor *desc, *_desc;
+ struct dma_chan *chan;
+
+ chan = host->data_chan;
+
+ if (chan) {
+ dev_vdbg(&host->mmc->class_dev, "submitting descriptors...\n");
+
+ /*
+ * Use the _safe() variant here because the might
+ * complete and get deleted from the list before we
+ * get around to the next entry. No need to lock since
+ * we're not modifying the list, and only entries
+ * we've submitted can be removed.
+ */
+ list_for_each_entry_safe(desc, _desc, &host->dma.data_descs,
+ client_node)
+ desc->txd.tx_submit(&desc->txd);
+
+
+ chan->device->device_issue_pending(chan);
+ } else {
+ if (data->flags & MMC_DATA_READ)
+ mci_writel(host, IER, MCI_BIT(RXRDY));
+ else
+ mci_writel(host, IER, MCI_BIT(TXRDY));
+ }
+}
+
+static void atmci_start_command(struct atmel_mci *host,
+ struct mmc_command *cmd,
+ u32 cmd_flags)
+{
+ WARN_ON(host->cmd);
+ host->cmd = cmd;
+
+ dev_vdbg(&host->mmc->class_dev,
+ "start command: ARGR=0x%08x CMDR=0x%08x\n",
+ cmd->arg, cmd_flags);
+
+ mci_writel(host, ARGR, cmd->arg);
+ mci_writel(host, CMDR, cmd_flags);
+
+ if (cmd->data)
+ atmci_start_data(host, cmd->data);
+}
+
+static void send_stop_cmd(struct mmc_host *mmc, struct mmc_data *data)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ atmci_start_command(host, data->stop, host->stop_cmdr);
+ mci_writel(host, IER, MCI_BIT(CMDRDY));
+}
+
+static void atmci_request_end(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ WARN_ON(host->cmd || host->data);
+ host->mrq = NULL;
+
+ atmci_disable(host);
+
+ mmc_request_done(mmc, mrq);
+}
+
+static void atmci_dma_cleanup(struct atmel_mci *host)
+{
+ struct dma_slave_descriptor *desc, *_desc;
+ struct mmc_data *data = host->data;
+
+ dma_unmap_sg(&host->pdev->dev, data->sg, data->sg_len,
+ ((data->flags & MMC_DATA_WRITE)
+ ? DMA_TO_DEVICE : DMA_FROM_DEVICE));
+
+ /*
+ * REVISIT: Recycle these descriptors instead of handing them
+ * back to the controller.
+ */
+ list_for_each_entry_safe(desc, _desc, &host->dma.data_descs,
+ client_node) {
+ list_del(&desc->client_node);
+ async_tx_ack(&desc->txd);
+ }
+}
+
+static void atmci_stop_dma(struct atmel_mci *host)
+{
+ struct dma_chan *chan = host->data_chan;
+
+ if (chan) {
+ chan->device->device_terminate_all(chan);
+ atmci_dma_cleanup(host);
+ }
+}
+
+/* This function is called by the DMA driver from tasklet context. */
+static void atmci_dma_complete(void *arg)
+{
+ struct atmel_mci *host = arg;
+ struct mmc_data *data = host->data;
+
+ /*
+ * If the card was removed, data will be NULL. No point trying
+ * to send the stop command or waiting for NBUSY in this case.
+ */
+ if (data) {
+ /* A short DMA transfer may complete before the command */
+ atmci_set_completed(host, EVENT_DMA_COMPLETE);
+ if (atmci_is_completed(host, EVENT_CMD_COMPLETE)
+ && data->stop
+ && !atmci_test_and_set_completed(host,
+ EVENT_STOP_SENT))
+ send_stop_cmd(host->mmc, data);
+ }
+
+ atmci_dma_cleanup(host);
+
+ /*
+ * Regardless of what the documentation says, we have to wait
+ * for NOTBUSY even after block read operations.
+ *
+ * When the DMA transfer is complete, the controller may still
+ * be reading the CRC from the card, i.e. the data transfer is
+ * still in progress and we haven't seen all the potential
+ * error bits yet.
+ *
+ * The interrupt handler will schedule a different tasklet to
+ * finish things up when the data transfer is completely done.
+ *
+ * We may not complete the mmc request here anyway because the
+ * mmc layer may call back and cause us to violate the "don't
+ * submit new operations from the completion callback" rule of
+ * the dma engine framework.
+ */
+ if (data)
+ mci_writel(host, IER, MCI_BIT(NOTBUSY));
+}
+
+static int
+atmci_prepare_data_dma(struct atmel_mci *host, struct mmc_data *data)
+{
+ struct dma_chan *chan;
+ struct dma_slave_descriptor *desc;
+ struct scatterlist *sg;
+ unsigned long dma_flags;
+ unsigned long flags;
+ unsigned int sg_len;
+ unsigned int i;
+ enum dma_slave_direction direction;
+
+ /*
+ * We don't do DMA on "complex" transfers, i.e. with
+ * non-word-aligned buffers or lengths. Also, we don't bother
+ * with all the DMA setup overhead for short transfers.
+ */
+ if (data->blocks * data->blksz < ATMCI_DMA_THRESHOLD)
+ return -EINVAL;
+ if (data->blksz & 3)
+ return -EINVAL;
+
+ for_each_sg(data->sg, sg, data->sg_len, i) {
+ if (sg->offset & 3 || sg->length & 3)
+ return -EINVAL;
+ }
+
+ /* If we don't have a channel, we can't do DMA */
+ spin_lock_irqsave(&host->mmc->lock, flags);
+ chan = host->dma.chan;
+ if (chan) {
+ dma_chan_get(chan);
+ host->data_chan = chan;
+ }
+ spin_unlock_irqrestore(&host->mmc->lock, flags);
+
+ if (!chan)
+ return -ENODEV;
+
+ /* REVISIT: Try to cache pre-initialized descriptors */
+ dma_flags = 0;
+ dev_vdbg(&host->mmc->class_dev, "setting up descriptors (%c)...\n",
+ (data->flags & MMC_DATA_READ) ? 'r' : 'w');
+
+ if (data->flags & MMC_DATA_READ) {
+ direction = DMA_SLAVE_TO_MEMORY;
+
+#ifdef POISON_READ_BUFFER
+ for_each_sg(data->sg, sg, data->sg_len, i) {
+ void *p = kmap(sg_page(sg));
+ memset(p + sg->offset, 0x55, sg->length);
+ kunmap(p);
+ }
+#endif
+ sg_len = dma_map_sg(&host->pdev->dev, data->sg,
+ data->sg_len, DMA_FROM_DEVICE);
+ } else {
+ direction = DMA_SLAVE_FROM_MEMORY;
+
+ sg_len = dma_map_sg(&host->pdev->dev, data->sg,
+ data->sg_len, DMA_TO_DEVICE);
+ }
+
+ for_each_sg(data->sg, sg, sg_len, i) {
+ if (i == sg_len - 1)
+ dma_flags = DMA_PREP_INTERRUPT;
+
+ dev_vdbg(&host->mmc->class_dev, " addr %08x len %u\n",
+ sg_dma_address(sg), sg_dma_len(sg));
+
+ desc = chan->device->device_prep_slave(chan,
+ sg_dma_address(sg), direction,
+ sg_dma_len(sg), dma_flags);
+ desc->txd.callback = NULL;
+ list_add_tail(&desc->client_node,
+ &host->dma.data_descs);
+ }
+
+ /* Make sure we get notified when the last descriptor is done. */
+ desc = list_entry(host->dma.data_descs.prev,
+ struct dma_slave_descriptor, client_node);
+ desc->txd.callback = atmci_dma_complete;
+ desc->txd.callback_param = host;
+
+ return 0;
+}
+
+/*
+ * Returns a mask of flags to be set in the command register when the
+ * command to start the transfer is to be sent.
+ */
+static u32 atmci_prepare_data(struct mmc_host *mmc, struct mmc_data *data)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+ u32 cmd_flags;
+
+ WARN_ON(host->data);
+ host->data = data;
+
+ atmci_set_timeout(host, data);
+
+ mci_writel(host, BLKR, (MCI_BF(BCNT, data->blocks)
+ | MCI_BF(BLKLEN, data->blksz)));
+ dev_vdbg(&mmc->class_dev, "BLKR=0x%08x\n",
+ (MCI_BF(BCNT, data->blocks)
+ | MCI_BF(BLKLEN, data->blksz)));
+
+ cmd_flags = MCI_BF(TRCMD, MCI_TRCMD_START_TRANS);
+ if (data->flags & MMC_DATA_STREAM)
+ cmd_flags |= MCI_BF(TRTYP, MCI_TRTYP_STREAM);
+ else if (data->blocks > 1)
+ cmd_flags |= MCI_BF(TRTYP, MCI_TRTYP_MULTI_BLOCK);
+ else
+ cmd_flags |= MCI_BF(TRTYP, MCI_TRTYP_BLOCK);
+
+ if (data->flags & MMC_DATA_READ)
+ cmd_flags |= MCI_BIT(TRDIR);
+
+ if (atmci_prepare_data_dma(host, data)) {
+ host->data_chan = NULL;
+ host->pio_sg = data->sg;
+ host->pio_offset = 0;
+ }
+
+ return cmd_flags;
+}
+
+static void atmci_request(struct mmc_host *mmc, struct mmc_request *mrq)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+ struct mmc_data *data = mrq->data;
+ u32 iflags;
+ u32 cmdflags = 0;
+
+ iflags = mci_readl(host, IMR);
+ if (iflags)
+ dev_warn(&mmc->class_dev, "WARNING: IMR=0x%08x\n",
+ mci_readl(host, IMR));
+
+ WARN_ON(host->mrq != NULL);
+ host->mrq = mrq;
+ host->pending_events = 0;
+ host->completed_events = 0;
+
+ atmci_enable(host);
+
+ iflags = MCI_BIT(CMDRDY);
+ cmdflags = atmci_prepare_command(mmc, mrq->cmd);
+
+ if (mrq->stop) {
+ host->stop_cmdr = atmci_prepare_command(mmc, mrq->stop);
+ host->stop_cmdr |= MCI_BF(TRCMD, MCI_TRCMD_STOP_TRANS);
+ if (!(data->flags & MMC_DATA_WRITE))
+ host->stop_cmdr |= MCI_BIT(TRDIR);
+ if (data->flags & MMC_DATA_STREAM)
+ host->stop_cmdr |= MCI_BF(TRTYP, MCI_TRTYP_STREAM);
+ else
+ host->stop_cmdr |= MCI_BF(TRTYP, MCI_TRTYP_MULTI_BLOCK);
+ }
+ if (data) {
+ /* We don't support multiple blocks of weird lengths. */
+ if (data->blocks > 1 && data->blksz & 3)
+ goto fail;
+
+ cmdflags |= atmci_prepare_data(mmc, data);
+ iflags |= ATMCI_DATA_ERROR_FLAGS;
+ }
+
+ atmci_start_command(host, mrq->cmd, cmdflags);
+ mci_writel(host, IER, iflags);
+ return;
+
+fail:
+ atmci_disable(host);
+ host->mrq = NULL;
+ mrq->cmd->error = -EINVAL;
+ mmc_request_done(mmc, mrq);
+}
+
+static void atmci_set_ios(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ if (ios->clock) {
+ u32 clkdiv;
+
+ /* Set clock rate */
+ clkdiv = host->bus_hz / (2 * ios->clock) - 1;
+ if (clkdiv > 255) {
+ dev_warn(&mmc->class_dev,
+ "clock %u too slow; using %lu\n",
+ ios->clock, host->bus_hz / (2 * 256));
+ clkdiv = 255;
+ }
+
+ host->mode_reg = MCI_BF(CLKDIV, clkdiv)
+ | MCI_BIT(WRPROOF)
+ | MCI_BIT(RDPROOF);
+ }
+
+ switch (ios->bus_width) {
+ case MMC_BUS_WIDTH_1:
+ host->sdc_reg = 0;
+ break;
+ case MMC_BUS_WIDTH_4:
+ host->sdc_reg = MCI_BIT(SDCBUS);
+ break;
+ }
+
+ switch (ios->power_mode) {
+ case MMC_POWER_ON:
+ /* Send init sequence (74 clock cycles) */
+ atmci_enable(host);
+ mci_writel(host, CMDR, MCI_BF(SPCMD, MCI_SPCMD_INIT_CMD));
+ while (!(mci_readl(host, SR) & MCI_BIT(CMDRDY)))
+ cpu_relax();
+ atmci_disable(host);
+ break;
+ default:
+ /*
+ * TODO: None of the currently available AVR32-based
+ * boards allow MMC power to be turned off. Implement
+ * power control when this can be tested properly.
+ */
+ break;
+ }
+}
+
+static int atmci_get_ro(struct mmc_host *mmc)
+{
+ int read_only = 0;
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ if (host->wp_pin >= 0) {
+ read_only = gpio_get_value(host->wp_pin);
+ dev_dbg(&mmc->class_dev, "card is %s\n",
+ read_only ? "read-only" : "read-write");
+ } else {
+ dev_dbg(&mmc->class_dev,
+ "no pin for checking read-only switch."
+ " Assuming write-enable.\n");
+ }
+
+ return read_only;
+}
+
+static struct mmc_host_ops atmci_ops = {
+ .request = atmci_request,
+ .set_ios = atmci_set_ios,
+ .get_ro = atmci_get_ro,
+};
+
+static void atmci_command_complete(struct atmel_mci *host,
+ struct mmc_command *cmd, u32 status)
+{
+ /* Read the response from the card (up to 16 bytes) */
+ cmd->resp[0] = mci_readl(host, RSPR);
+ cmd->resp[1] = mci_readl(host, RSPR);
+ cmd->resp[2] = mci_readl(host, RSPR);
+ cmd->resp[3] = mci_readl(host, RSPR);
+
+ host->cmd = NULL;
+
+ if (status & MCI_BIT(RTOE))
+ cmd->error = -ETIMEDOUT;
+ else if ((cmd->flags & MMC_RSP_CRC) && (status & MCI_BIT(RCRCE)))
+ cmd->error = -EILSEQ;
+ else if (status & (MCI_BIT(RINDE) | MCI_BIT(RDIRE) | MCI_BIT(RENDE)))
+ cmd->error = -EIO;
+
+ if (cmd->error) {
+ dev_dbg(&host->mmc->class_dev,
+ "command error: status=0x%08x\n", status);
+
+ if (cmd->data) {
+ host->data = NULL;
+ atmci_stop_dma(host);
+ mci_writel(host, IDR, MCI_BIT(NOTBUSY)
+ | ATMCI_DATA_ERROR_FLAGS);
+ }
+ }
+}
+
+static void atmci_tasklet_func(unsigned long priv)
+{
+ struct mmc_host *mmc = (struct mmc_host *)priv;
+ struct atmel_mci *host = mmc_priv(mmc);
+ struct mmc_request *mrq = host->mrq;
+ struct mmc_data *data = host->data;
+
+ dev_vdbg(&mmc->class_dev,
+ "tasklet: pending/completed/mask %lx/%lx/%x\n",
+ host->pending_events, host->completed_events,
+ mci_readl(host, IMR));
+
+ if (atmci_test_and_clear_pending(host, EVENT_CMD_COMPLETE)) {
+ atmci_set_completed(host, EVENT_CMD_COMPLETE);
+ atmci_command_complete(host, mrq->cmd, host->cmd_status);
+
+ if (!mrq->cmd->error && mrq->stop
+ && atmci_is_completed(host, EVENT_DMA_COMPLETE)
+ && !atmci_test_and_set_completed(host,
+ EVENT_STOP_SENT))
+ send_stop_cmd(host->mmc, mrq->data);
+ }
+ if (atmci_test_and_clear_pending(host, EVENT_STOP_COMPLETE)) {
+ atmci_set_completed(host, EVENT_STOP_COMPLETE);
+ atmci_command_complete(host, mrq->stop, host->stop_status);
+ }
+ if (atmci_test_and_clear_pending(host, EVENT_DATA_ERROR)) {
+ u32 status = host->data_status;
+
+ atmci_set_completed(host, EVENT_DATA_ERROR);
+ atmci_stop_dma(host);
+
+ if (status & MCI_BIT(DCRCE)) {
+ dev_dbg(&mmc->class_dev, "data CRC error\n");
+ data->error = -EILSEQ;
+ } else if (status & MCI_BIT(DTOE)) {
+ dev_dbg(&mmc->class_dev,
+ "data timeout error\n");
+ data->error = -ETIMEDOUT;
+ } else {
+ dev_dbg(&mmc->class_dev, "data FIFO error\n");
+ data->error = -EIO;
+ }
+
+ mci_writel(host, IER, MCI_BIT(NOTBUSY));
+
+ if (host->present && data->stop
+ && atmci_test_and_set_completed(
+ host, EVENT_STOP_SENT))
+ send_stop_cmd(host->mmc, data);
+ }
+ if (atmci_test_and_clear_pending(host, EVENT_DATA_COMPLETE)) {
+ atmci_set_completed(host, EVENT_DATA_COMPLETE);
+
+ if (!atmci_is_completed(host, EVENT_DATA_ERROR))
+ data->bytes_xfered = data->blocks * data->blksz;
+
+ host->data = NULL;
+ }
+ if (atmci_test_and_clear_pending(host, EVENT_CARD_DETECT)) {
+ /* Clean up queue if present */
+ if (mrq) {
+ /*
+ * Reset controller to terminate any ongoing
+ * commands or data transfers.
+ */
+ mci_writel(host, CR, MCI_BIT(SWRST));
+
+ if (!atmci_is_completed(host, EVENT_CMD_COMPLETE))
+ mrq->cmd->error = -EIO;
+
+ if (mrq->data && !atmci_is_completed(host,
+ EVENT_DATA_COMPLETE)) {
+ host->data = NULL;
+ mrq->data->error = -EIO;
+ atmci_stop_dma(host);
+ }
+ if (mrq->stop && !atmci_is_completed(host,
+ EVENT_STOP_COMPLETE))
+ mrq->stop->error = -EIO;
+
+ host->cmd = NULL;
+ atmci_request_end(mmc, mrq);
+ }
+
+ mmc_detect_change(host->mmc, 0);
+ }
+
+ if (host->mrq && !host->cmd && !host->data)
+ atmci_request_end(mmc, host->mrq);
+}
+
+static void atmci_read_data_pio(struct atmel_mci *host)
+{
+ struct scatterlist *sg = host->pio_sg;
+ void *buf = sg_virt(sg);
+ unsigned int offset = host->pio_offset;
+ struct mmc_data *data = host->data;
+ u32 value;
+ u32 status;
+ unsigned int nbytes = 0;
+
+ do {
+ value = mci_readl(host, RDR);
+ if (likely(offset + 4 <= sg->length)) {
+ put_unaligned(value, (u32 *)(buf + offset));
+
+ offset += 4;
+ nbytes += 4;
+
+ if (offset == sg->length) {
+ host->pio_sg = sg = sg_next(sg);
+ if (!sg)
+ goto done;
+
+ offset = 0;
+ buf = sg_virt(sg);
+ }
+ } else {
+ unsigned int remaining = sg->length - offset;
+ memcpy(buf + offset, &value, remaining);
+ nbytes += remaining;
+
+ flush_dcache_page(sg_page(sg));
+ host->pio_sg = sg = sg_next(sg);
+ if (!sg)
+ goto done;
+
+ offset = 4 - remaining;
+ buf = sg_virt(sg);
+ memcpy(buf, (u8 *)&value + remaining, offset);
+ nbytes += offset;
+ }
+
+ status = mci_readl(host, SR);
+ if (status & ATMCI_DATA_ERROR_FLAGS) {
+ mci_writel(host, IDR, (MCI_BIT(NOTBUSY)
+ | MCI_BIT(RXRDY)
+ | ATMCI_DATA_ERROR_FLAGS));
+ host->data_status = status;
+ atmci_set_pending(host, EVENT_DATA_ERROR);
+ tasklet_schedule(&host->tasklet);
+ break;
+ }
+ } while (status & MCI_BIT(RXRDY));
+
+ host->pio_offset = offset;
+ data->bytes_xfered += nbytes;
+
+ return;
+
+done:
+ mci_writel(host, IDR, MCI_BIT(RXRDY));
+ mci_writel(host, IER, MCI_BIT(NOTBUSY));
+ data->bytes_xfered += nbytes;
+ if (data->stop && atmci_is_completed(host, EVENT_CMD_COMPLETE)
+ && !atmci_test_and_set_completed(host, EVENT_STOP_SENT))
+ send_stop_cmd(host->mmc, data);
+}
+
+static void atmci_write_data_pio(struct atmel_mci *host)
+{
+ struct scatterlist *sg = host->pio_sg;
+ void *buf = sg_virt(sg);
+ unsigned int offset = host->pio_offset;
+ struct mmc_data *data = host->data;
+ u32 value;
+ u32 status;
+ unsigned int nbytes = 0;
+
+ do {
+ if (likely(offset + 4 <= sg->length)) {
+ value = get_unaligned((u32 *)(buf + offset));
+ mci_writel(host, TDR, value);
+
+ offset += 4;
+ nbytes += 4;
+ if (offset == sg->length) {
+ host->pio_sg = sg = sg_next(sg);
+ if (!sg)
+ goto done;
+
+ offset = 0;
+ buf = sg_virt(sg);
+ }
+ } else {
+ unsigned int remaining = sg->length - offset;
+
+ value = 0;
+ memcpy(&value, buf + offset, remaining);
+ nbytes += remaining;
+
+ host->pio_sg = sg = sg_next(sg);
+ if (!sg) {
+ mci_writel(host, TDR, value);
+ goto done;
+ }
+
+ offset = 4 - remaining;
+ buf = sg_virt(sg);
+ memcpy((u8 *)&value + remaining, buf, offset);
+ mci_writel(host, TDR, value);
+ nbytes += offset;
+ }
+
+ status = mci_readl(host, SR);
+ if (status & ATMCI_DATA_ERROR_FLAGS) {
+ mci_writel(host, IDR, (MCI_BIT(NOTBUSY)
+ | MCI_BIT(TXRDY)
+ | ATMCI_DATA_ERROR_FLAGS));
+ host->data_status = status;
+ atmci_set_pending(host, EVENT_DATA_ERROR);
+ tasklet_schedule(&host->tasklet);
+ break;
+ }
+ } while (status & MCI_BIT(TXRDY));
+
+ host->pio_offset = offset;
+ data->bytes_xfered += nbytes;
+
+ return;
+
+done:
+ mci_writel(host, IDR, MCI_BIT(TXRDY));
+ mci_writel(host, IER, MCI_BIT(NOTBUSY));
+ data->bytes_xfered += nbytes;
+ if (data->stop && atmci_is_completed(host, EVENT_CMD_COMPLETE)
+ && !atmci_test_and_set_completed(host, EVENT_STOP_SENT))
+ send_stop_cmd(host->mmc, data);
+}
+
+static void atmci_cmd_interrupt(struct mmc_host *mmc, u32 status)
+{
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ mci_writel(host, IDR, MCI_BIT(CMDRDY));
+
+ if (atmci_is_completed(host, EVENT_STOP_SENT)) {
+ host->stop_status = status;
+ atmci_set_pending(host, EVENT_STOP_COMPLETE);
+ } else {
+ host->cmd_status = status;
+ atmci_set_pending(host, EVENT_CMD_COMPLETE);
+ }
+
+ tasklet_schedule(&host->tasklet);
+}
+
+static irqreturn_t atmci_interrupt(int irq, void *dev_id)
+{
+ struct mmc_host *mmc = dev_id;
+ struct atmel_mci *host = mmc_priv(mmc);
+ u32 status, mask, pending;
+ unsigned int pass_count = 0;
+
+ spin_lock(&mmc->lock);
+
+ do {
+ status = mci_readl(host, SR);
+ mask = mci_readl(host, IMR);
+ pending = status & mask;
+ if (!pending)
+ break;
+
+ if (pending & ATMCI_DATA_ERROR_FLAGS) {
+ mci_writel(host, IDR, ATMCI_DATA_ERROR_FLAGS);
+ host->data_status = status;
+ atmci_set_pending(host, EVENT_DATA_ERROR);
+ tasklet_schedule(&host->tasklet);
+ }
+ if (pending & (MCI_BIT(NOTBUSY))) {
+ mci_writel(host, IDR, (MCI_BIT(NOTBUSY)
+ | ATMCI_DATA_ERROR_FLAGS));
+ atmci_set_pending(host, EVENT_DATA_COMPLETE);
+ tasklet_schedule(&host->tasklet);
+ }
+ if (pending & MCI_BIT(RXRDY))
+ atmci_read_data_pio(host);
+ if (pending & MCI_BIT(TXRDY))
+ atmci_write_data_pio(host);
+
+ if (pending & MCI_BIT(CMDRDY))
+ atmci_cmd_interrupt(mmc, status);
+ } while (pass_count++ < 5);
+
+ spin_unlock(&mmc->lock);
+
+ return pass_count ? IRQ_HANDLED : IRQ_NONE;
+}
+
+static irqreturn_t atmci_detect_change(int irq, void *dev_id)
+{
+ struct mmc_host *mmc = dev_id;
+ struct atmel_mci *host = mmc_priv(mmc);
+
+ int present = !gpio_get_value(irq_to_gpio(irq));
+
+ if (present != host->present) {
+ dev_dbg(&mmc->class_dev, "card %s\n",
+ present ? "inserted" : "removed");
+ host->present = present;
+ atmci_set_pending(host, EVENT_CARD_DETECT);
+ tasklet_schedule(&host->tasklet);
+ }
+ return IRQ_HANDLED;
+}
+
+static enum dma_state_client atmci_dma_event(struct dma_client *client,
+ struct dma_chan *chan, enum dma_state state)
+{
+ struct atmel_mci *host;
+ enum dma_state_client ret = DMA_NAK;
+ unsigned long flags;
+
+ host = dma_client_to_atmel_mci(client);
+
+ switch (state) {
+ case DMA_RESOURCE_AVAILABLE:
+ spin_lock_irqsave(&host->mmc->lock, flags);
+ if (!host->dma.chan) {
+ host->dma.chan = chan;
+ ret = DMA_ACK;
+ }
+ spin_unlock_irqrestore(&host->mmc->lock, flags);
+
+ if (ret == DMA_ACK)
+ dev_info(&host->pdev->dev,
+ "Using %s for DMA transfers\n",
+ chan->dev.bus_id);
+ break;
+
+ case DMA_RESOURCE_REMOVED:
+ spin_lock_irqsave(&host->mmc->lock, flags);
+ if (host->dma.chan == chan) {
+ host->dma.chan = NULL;
+ ret = DMA_ACK;
+ }
+ spin_unlock_irqrestore(&host->mmc->lock, flags);
+
+ if (ret == DMA_ACK)
+ dev_info(&host->pdev->dev,
+ "Lost %s, reverting to PIO\n",
+ chan->dev.bus_id);
+ break;
+
+ default:
+ break;
+ }
+
+
+ return ret;
+}
+
+static int __init atmci_probe(struct platform_device *pdev)
+{
+ struct mci_platform_data *pdata;
+ struct atmel_mci *host;
+ struct mmc_host *mmc;
+ struct resource *regs;
+ int irq;
+ int ret;
+
+ regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!regs)
+ return -ENXIO;
+ pdata = pdev->dev.platform_data;
+ if (!pdata)
+ return -ENXIO;
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ mmc = mmc_alloc_host(sizeof(struct atmel_mci), &pdev->dev);
+ if (!mmc)
+ return -ENOMEM;
+
+ host = mmc_priv(mmc);
+ host->pdev = pdev;
+ host->mmc = mmc;
+ host->detect_pin = pdata->detect_pin;
+ host->wp_pin = pdata->wp_pin;
+ INIT_LIST_HEAD(&host->dma.data_descs);
+
+ host->mck = clk_get(&pdev->dev, "mci_clk");
+ if (IS_ERR(host->mck)) {
+ ret = PTR_ERR(host->mck);
+ goto err_clk_get;
+ }
+
+ ret = -ENOMEM;
+ host->regs = ioremap(regs->start, regs->end - regs->start + 1);
+ if (!host->regs)
+ goto err_ioremap;
+
+ clk_enable(host->mck);
+ mci_writel(host, CR, MCI_BIT(SWRST));
+ host->bus_hz = clk_get_rate(host->mck);
+ clk_disable(host->mck);
+
+ host->mapbase = regs->start;
+
+ mmc->ops = &atmci_ops;
+ mmc->f_min = (host->bus_hz + 511) / 512;
+ mmc->f_max = min((unsigned int)(host->bus_hz / 2), fmax);
+ mmc->ocr_avail = MMC_VDD_32_33 | MMC_VDD_33_34;
+ mmc->caps |= MMC_CAP_4_BIT_DATA;
+
+ tasklet_init(&host->tasklet, atmci_tasklet_func, (unsigned long)mmc);
+
+ ret = request_irq(irq, atmci_interrupt, 0, pdev->dev.bus_id, mmc);
+ if (ret)
+ goto err_request_irq;
+
+ if (pdata->dma_slave) {
+ struct dma_slave *slave = pdata->dma_slave;
+
+ slave->tx_reg = regs->start + MCI_TDR;
+ slave->rx_reg = regs->start + MCI_RDR;
+
+ /* Try to grab a DMA channel */
+ host->dma.client.event_callback = atmci_dma_event;
+ dma_cap_set(DMA_SLAVE, host->dma.client.cap_mask);
+ host->dma.client.slave = slave;
+
+ dma_async_client_register(&host->dma.client);
+ dma_async_client_chan_request(&host->dma.client);
+ }
+
+ /* Assume card is present if we don't have a detect pin */
+ host->present = 1;
+ if (host->detect_pin >= 0) {
+ if (gpio_request(host->detect_pin, "mmc_detect")) {
+ dev_dbg(&mmc->class_dev, "no detect pin available\n");
+ host->detect_pin = -1;
+ } else {
+ host->present = !gpio_get_value(host->detect_pin);
+ }
+ }
+ if (host->wp_pin >= 0) {
+ if (gpio_request(host->wp_pin, "mmc_wp")) {
+ dev_dbg(&mmc->class_dev, "no WP pin available\n");
+ host->wp_pin = -1;
+ }
+ }
+
+ platform_set_drvdata(pdev, host);
+
+ mmc_add_host(mmc);
+
+ if (host->detect_pin >= 0) {
+ ret = request_irq(gpio_to_irq(host->detect_pin),
+ atmci_detect_change,
+ IRQF_TRIGGER_FALLING | IRQF_TRIGGER_RISING,
+ "mmc-detect", mmc);
+ if (ret) {
+ dev_dbg(&mmc->class_dev,
+ "could not request IRQ %d for detect pin\n",
+ gpio_to_irq(host->detect_pin));
+ gpio_free(host->detect_pin);
+ host->detect_pin = -1;
+ }
+ }
+
+ dev_info(&mmc->class_dev,
+ "Atmel MCI controller at 0x%08lx irq %d\n",
+ host->mapbase, irq);
+
+ atmci_init_debugfs(host);
+
+ return 0;
+
+err_request_irq:
+ iounmap(host->regs);
+err_ioremap:
+ clk_put(host->mck);
+err_clk_get:
+ mmc_free_host(mmc);
+ return ret;
+}
+
+static int __exit atmci_remove(struct platform_device *pdev)
+{
+ struct atmel_mci *host = platform_get_drvdata(pdev);
+
+ platform_set_drvdata(pdev, NULL);
+
+ if (host) {
+ atmci_cleanup_debugfs(host);
+
+ if (host->detect_pin >= 0) {
+ free_irq(gpio_to_irq(host->detect_pin), host->mmc);
+ gpio_free(host->detect_pin);
+ }
+
+ mmc_remove_host(host->mmc);
+
+ clk_enable(host->mck);
+ mci_writel(host, IDR, ~0UL);
+ mci_writel(host, CR, MCI_BIT(MCIDIS));
+ mci_readl(host, SR);
+ clk_disable(host->mck);
+
+ dma_async_client_unregister(&host->dma.client);
+
+ if (host->wp_pin >= 0)
+ gpio_free(host->wp_pin);
+
+ free_irq(platform_get_irq(pdev, 0), host->mmc);
+ iounmap(host->regs);
+
+ clk_put(host->mck);
+
+ mmc_free_host(host->mmc);
+ }
+ return 0;
+}
+
+static struct platform_driver atmci_driver = {
+ .remove = __exit_p(atmci_remove),
+ .driver = {
+ .name = "atmel_mci",
+ },
+};
+
+static int __init atmci_init(void)
+{
+ return platform_driver_probe(&atmci_driver, atmci_probe);
+}
+
+static void __exit atmci_exit(void)
+{
+ platform_driver_unregister(&atmci_driver);
+}
+
+module_init(atmci_init);
+module_exit(atmci_exit);
+
+MODULE_DESCRIPTION("Atmel Multimedia Card Interface driver");
+MODULE_LICENSE("GPL");
diff --git a/include/asm-avr32/arch-at32ap/board.h b/include/asm-avr32/arch-at32ap/board.h
index 7597b0b..d90edc6 100644
--- a/include/asm-avr32/arch-at32ap/board.h
+++ b/include/asm-avr32/arch-at32ap/board.h
@@ -69,7 +69,11 @@ struct platform_device *
at32_add_device_ssc(unsigned int id, unsigned int flags);
struct platform_device *at32_add_device_twi(unsigned int id);
-struct platform_device *at32_add_device_mci(unsigned int id);
+
+struct mci_platform_data;
+struct platform_device *
+at32_add_device_mci(unsigned int id, struct mci_platform_data *data);
+
struct platform_device *at32_add_device_ac97c(unsigned int id);
struct platform_device *at32_add_device_abdac(unsigned int id);
diff --git a/include/asm-avr32/atmel-mci.h b/include/asm-avr32/atmel-mci.h
new file mode 100644
index 0000000..ea6e29d
--- /dev/null
+++ b/include/asm-avr32/atmel-mci.h
@@ -0,0 +1,12 @@
+#ifndef __ASM_AVR32_ATMEL_MCI_H
+#define __ASM_AVR32_ATMEL_MCI_H
+
+struct dma_slave;
+
+struct mci_platform_data {
+ struct dma_slave *dma_slave;
+ int detect_pin;
+ int wp_pin;
+};
+
+#endif /* __ASM_AVR32_ATMEL_MCI_H */
--
1.5.3.8
This adds a driver for the Synopsys DesignWare DMA controller (aka
DMACA on AVR32 systems.) This DMA controller can be found integrated
on the AT32AP7000 chip and is primarily meant for peripheral DMA
transfer, but can also be used for memory-to-memory transfers.
This patch is based on a driver from David Brownell which was based on
an older version of the DMA Engine framework. It also implements the
proposed extensions to the DMA Engine API for slave DMA operations.
The dmatest client shows no problems, but the performance is not as
good as it should be yet. However, DMA slave transfer performance is
quite acceptable, so while I think the memcpy performance can be
improved, it's not very high priority right now.
Signed-off-by: Haavard Skinnemoen <[email protected]>
Changes since v2:
* Dequeue all pending transfers in terminate_all()
* Rename dw_dmac.h -> dw_dmac_regs.h
* Define and use controller-specific dma_slave data
* Fix up a few outdated comments
* Define hardware registers as structs (doesn't generate better
code, unfortunately, but it looks nicer.)
* Get number of channels from platform_data instead of hardcoding it
based on CONFIG_WHATEVER_CPU.
* Give slave clients exclusive access to the channel
---
arch/avr32/mach-at32ap/at32ap700x.c | 26 +-
drivers/dma/Kconfig | 9 +
drivers/dma/Makefile | 1 +
drivers/dma/dw_dmac.c | 1170 ++++++++++++++++++++++++++++
drivers/dma/dw_dmac_regs.h | 249 ++++++
include/asm-avr32/arch-at32ap/at32ap700x.h | 16 +
include/linux/dw_dmac.h | 62 ++
7 files changed, 1520 insertions(+), 13 deletions(-)
create mode 100644 drivers/dma/dw_dmac.c
create mode 100644 drivers/dma/dw_dmac_regs.h
create mode 100644 include/linux/dw_dmac.h
diff --git a/arch/avr32/mach-at32ap/at32ap700x.c b/arch/avr32/mach-at32ap/at32ap700x.c
index 7678fee..038e17b 100644
--- a/arch/avr32/mach-at32ap/at32ap700x.c
+++ b/arch/avr32/mach-at32ap/at32ap700x.c
@@ -451,6 +451,17 @@ static void __init genclk_init_parent(struct clk *clk)
clk->parent = parent;
}
+static struct dw_dma_platform_data dw_dmac0_data = {
+ .nr_channels = 3,
+};
+
+static struct resource dw_dmac0_resource[] = {
+ PBMEM(0xff200000),
+ IRQ(2),
+};
+DEFINE_DEV_DATA(dw_dmac, 0);
+DEV_CLK(hclk, dw_dmac0, hsb, 10);
+
/* --------------------------------------------------------------------
* System peripherals
* -------------------------------------------------------------------- */
@@ -557,17 +568,6 @@ static struct clk pico_clk = {
.users = 1,
};
-static struct resource dmaca0_resource[] = {
- {
- .start = 0xff200000,
- .end = 0xff20ffff,
- .flags = IORESOURCE_MEM,
- },
- IRQ(2),
-};
-DEFINE_DEV(dmaca, 0);
-DEV_CLK(hclk, dmaca0, hsb, 10);
-
/* --------------------------------------------------------------------
* HMATRIX
* -------------------------------------------------------------------- */
@@ -667,7 +667,7 @@ void __init at32_add_system_devices(void)
platform_device_register(&at32_eic0_device);
platform_device_register(&smc0_device);
platform_device_register(&pdc_device);
- platform_device_register(&dmaca0_device);
+ platform_device_register(&dw_dmac0_device);
platform_device_register(&at32_systc0_device);
@@ -1687,7 +1687,7 @@ struct clk *at32_clock_list[] = {
&smc0_mck,
&pdc_hclk,
&pdc_pclk,
- &dmaca0_hclk,
+ &dw_dmac0_hclk,
&pico_clk,
&pio0_mck,
&pio1_mck,
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index 1a727c1..76582db 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -37,6 +37,15 @@ config INTEL_IOP_ADMA
help
Enable support for the Intel(R) IOP Series RAID engines.
+config DW_DMAC
+ tristate "Synopsys DesignWare AHB DMA support"
+ depends on AVR32
+ select DMA_ENGINE
+ default y if CPU_AT32AP7000
+ help
+ Support the Synopsys DesignWare AHB DMA controller. This
+ can be integrated in chips such as the Atmel AT32ap7000.
+
config DMA_ENGINE
bool
diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile
index cecfb60..0f3b24f 100644
--- a/drivers/dma/Makefile
+++ b/drivers/dma/Makefile
@@ -1,6 +1,7 @@
obj-$(CONFIG_DMA_ENGINE) += dmaengine.o
obj-$(CONFIG_NET_DMA) += iovlock.o
obj-$(CONFIG_INTEL_IOATDMA) += ioatdma.o
+obj-$(CONFIG_DW_DMAC) += dw_dmac.o
ioatdma-objs := ioat.o ioat_dma.o ioat_dca.o
obj-$(CONFIG_INTEL_IOP_ADMA) += iop-adma.o
obj-$(CONFIG_DMATEST) += dmatest.o
diff --git a/drivers/dma/dw_dmac.c b/drivers/dma/dw_dmac.c
new file mode 100644
index 0000000..9d63c2f
--- /dev/null
+++ b/drivers/dma/dw_dmac.c
@@ -0,0 +1,1170 @@
+/*
+ * Driver for the Synopsys DesignWare DMA Controller (aka DMACA on
+ * AVR32 systems.)
+ *
+ * Copyright (C) 2007 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+/* #define DEBUG
+#define VERBOSE_DEBUG */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/mm.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+
+/* NOTE: DMS+SMS is system-specific. We should get this information
+ * from the platform code somehow.
+ */
+#define DWC_DEFAULT_CTLLO (DWC_CTLL_DST_MSIZE(0) \
+ | DWC_CTLL_SRC_MSIZE(0) \
+ | DWC_CTLL_DMS(0) \
+ | DWC_CTLL_SMS(1) \
+ | DWC_CTLL_LLP_D_EN \
+ | DWC_CTLL_LLP_S_EN)
+
+/*
+ * This is configuration-dependent and is usually a funny size like
+ * 4095. Let's round it down to the nearest power of two.
+ */
+#define DWC_MAX_LEN 2048
+
+/*
+ * This supports the Synopsis "DesignWare AHB Central DMA Controller",
+ * (DW_ahb_dmac) which is used with various AMBA 2.0 systems (not all
+ * of which use ARM any more). See the "Databook" from Synopsis for
+ * information beyond what licensees probably provide.
+ *
+ * The driver has currently been tested only with the Atmel AT32AP7000,
+ * which does not support descriptor writeback.
+ */
+
+#define USE_DMA_POOL
+#undef USE_FREELIST
+
+#ifdef USE_DMA_POOL
+#include <linux/dmapool.h>
+#else
+#include <linux/slab.h>
+#endif
+
+#include "dw_dmac_regs.h"
+
+/*----------------------------------------------------------------------*/
+
+#define NR_DESCS_PER_CHANNEL 8
+
+/* Because we're not relying on writeback from the controller (it may not
+ * even be configured into the core!) we don't need to use dma_pool. These
+ * descriptors -- and associated data -- are cacheable. We do need to make
+ * sure their dcache entries are written back before handing them off to
+ * the controller, though.
+ *
+ * FIXME: At the moment, there are a lot of #ifdefs in this code
+ * because it's not clear yet
+ * - whether descriptors from SLAB + cache syncing is faster than
+ * coherent dma_pool allocations, or if it's the other way around.
+ * - whether or not the freelist provides any performance gain.
+ *
+ * After doing performance measurements, we should decide which
+ * strategy to use and remove the others, including the #ifdefs.
+ */
+
+#ifdef USE_FREELIST
+#define FREECNT 10 /* for fastpath allocations */
+#endif
+
+static struct dw_lli *
+dwc_lli_alloc(struct dw_dma_chan *dwc, gfp_t flags)
+{
+ struct dw_lli *lli;
+
+#ifdef USE_DMA_POOL
+ dma_addr_t phys;
+
+ lli = dma_pool_alloc(dwc->lli_pool, flags, &phys);
+ if (likely(lli))
+ lli->phys = phys;
+#else
+ lli = kmem_cache_alloc(dwc->lli_pool, flags);
+ if (unlikely(!lli))
+ return NULL;
+ lli->phys = dma_map_single(dwc->chan.dev.parent, lli,
+ sizeof *lli, DMA_TO_DEVICE);
+#endif
+
+ return lli;
+}
+
+static inline void
+dwc_lli_free(struct dw_dma_chan *dwc, struct dw_lli *lli)
+{
+#ifdef USE_DMA_POOL
+ dma_pool_free(dwc->lli_pool, lli, lli->phys);
+#else
+ dma_unmap_single(dwc->chan.dev.parent, lli->phys,
+ sizeof *lli, DMA_TO_DEVICE);
+ kmem_cache_free(dwc->lli_pool, lli);
+#endif
+}
+
+static inline void
+dwc_lli_sync_for_device(struct dw_dma_chan *dwc, struct dw_lli *lli)
+{
+#ifndef USE_DMA_POOL
+ dma_sync_single_for_device(dwc->chan.dev.parent, lli->phys,
+ sizeof(struct dw_lli), DMA_TO_DEVICE);
+#endif
+}
+
+static inline struct dw_lli *
+dwc_lli_get(struct dw_dma_chan *dwc, gfp_t flags)
+{
+ struct dw_lli *lli;
+
+#ifdef USE_FREELIST
+ lli = dwc->free;
+
+ if (lli && FREECNT) {
+ dwc->free = lli->next;
+ dwc->freecnt--;
+ } else
+#endif
+ lli = dwc_lli_alloc(dwc, flags);
+
+ return lli;
+}
+
+static inline void
+dwc_lli_put(struct dw_dma_chan *dwc, struct dw_lli *lli)
+{
+#ifdef USE_FREELIST
+ if (dwc->freecnt < FREECNT) {
+ lli->ctllo = lli->ctlhi = 0;
+ lli->next = dwc->free;
+ dwc->free = lli;
+ dwc->freecnt++;
+ } else
+#endif
+ dwc_lli_free(dwc, lli);
+}
+
+static struct dw_desc *dwc_desc_get(struct dw_dma_chan *dwc)
+{
+ struct dw_desc *desc, *_desc;
+ struct dw_desc *ret = NULL;
+
+ spin_lock_bh(&dwc->lock);
+ list_for_each_entry_safe(desc, _desc, &dwc->free_list, desc_node) {
+ if (desc->slave.txd.ack) {
+ list_del(&desc->desc_node);
+ desc->slave.txd.ack = 0;
+ ret = desc;
+ break;
+ }
+ }
+ spin_unlock_bh(&dwc->lock);
+
+ return ret;
+}
+
+static void dwc_desc_put(struct dw_dma_chan *dwc, struct dw_desc *desc)
+{
+ spin_lock_bh(&dwc->lock);
+ list_add_tail(&desc->desc_node, &dwc->free_list);
+ spin_unlock_bh(&dwc->lock);
+}
+
+/* Called with dwc->lock held and bh disabled */
+static dma_cookie_t
+dwc_assign_cookie(struct dw_dma_chan *dwc, struct dw_desc *desc)
+{
+ dma_cookie_t cookie = dwc->chan.cookie;
+
+ if (++cookie < 0)
+ cookie = 1;
+
+ dwc->chan.cookie = cookie;
+ desc->slave.txd.cookie = cookie;
+
+ return cookie;
+}
+
+/*----------------------------------------------------------------------*/
+
+/* Called with dwc->lock held and bh disabled */
+static void dwc_dostart(struct dw_dma_chan *dwc, struct dw_lli *first)
+{
+ struct dw_dma *dw = to_dw_dma(dwc->chan.device);
+
+ /* ASSERT: channel is idle */
+ if (dma_readl(dw, CH_EN) & dwc->mask) {
+ dev_err(&dwc->chan.dev,
+ "BUG: Attempted to start non-idle channel\n");
+ dev_err(&dwc->chan.dev, " new: %p last_lli: %p\n",
+ first, dwc->last_lli);
+ dev_err(&dwc->chan.dev,
+ " first_queued: %p last_queued: %p\n",
+ dwc->first_queued, dwc->last_queued);
+ dev_err(&dwc->chan.dev,
+ " LLP: 0x%x CTL: 0x%x:%08x\n",
+ channel_readl(dwc, LLP),
+ channel_readl(dwc, CTL_HI),
+ channel_readl(dwc, CTL_LO));
+
+ /* The tasklet will hopefully advance the queue... */
+ return;
+ }
+
+ channel_writel(dwc, LLP, first->phys);
+ channel_writel(dwc, CTL_LO,
+ DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN);
+ channel_writel(dwc, CTL_HI, 0);
+ channel_set_bit(dw, CH_EN, dwc->mask);
+}
+
+/*----------------------------------------------------------------------*/
+
+/*
+ * Move descriptors that have been queued up because the DMA
+ * controller was busy at the time of submission, to the "active"
+ * list. The caller must make sure that the DMA controller is
+ * kickstarted if necessary.
+ *
+ * Called with dwc->lock held and bh disabled.
+ */
+static void dwc_submit_queue(struct dw_dma_chan *dwc)
+{
+ dwc->last_lli = dwc->last_queued;
+ list_splice_init(&dwc->queue, dwc->active_list.prev);
+ dwc->first_queued = dwc->last_queued = NULL;
+}
+
+static void
+dwc_descriptor_complete(struct dw_dma_chan *dwc, struct dw_desc *desc)
+{
+ struct dw_lli *lli;
+
+ dev_vdbg(&dwc->chan.dev, "descriptor %u complete\n",
+ desc->slave.txd.cookie);
+
+ dwc->completed = desc->slave.txd.cookie;
+ for (lli = desc->first_lli; lli; lli = lli->next)
+ dwc_lli_put(dwc, lli);
+
+ desc->first_lli = NULL;
+ list_move(&desc->desc_node, &dwc->free_list);
+
+ /*
+ * The API requires that no submissions are done from a
+ * callback, so we don't need to drop the lock here
+ */
+ if (desc->slave.txd.callback)
+ desc->slave.txd.callback(desc->slave.txd.callback_param);
+}
+
+static void dwc_complete_all(struct dw_dma *dw, struct dw_dma_chan *dwc)
+{
+ struct dw_desc *desc, *_desc;
+ LIST_HEAD(list);
+
+ /*
+ * Submit queued descriptors ASAP, i.e. before we go through
+ * the completed ones.
+ */
+ list_splice_init(&dwc->active_list, &list);
+
+ if (dma_readl(dw, CH_EN) & dwc->mask) {
+ dev_err(&dwc->chan.dev,
+ "BUG: XFER bit set, but channel not idle!\n");
+
+ /* Try to continue after resetting the channel... */
+ channel_clear_bit(dw, CH_EN, dwc->mask);
+ while (dma_readl(dw, CH_EN) & dwc->mask)
+ cpu_relax();
+ }
+
+ dwc->last_lli = NULL;
+ if (dwc->first_queued) {
+ dwc_dostart(dwc, dwc->first_queued);
+ dwc_submit_queue(dwc);
+ }
+
+ list_for_each_entry_safe(desc, _desc, &list, desc_node)
+ dwc_descriptor_complete(dwc, desc);
+}
+
+static void dwc_scan_descriptors(struct dw_dma *dw, struct dw_dma_chan *dwc)
+{
+ dma_addr_t llp;
+ struct dw_desc *desc, *_desc;
+ struct dw_lli *lli, *next;
+ u32 status_xfer;
+
+ /*
+ * Clear block interrupt flag before scanning so that we don't
+ * miss any, and read LLP before RAW_XFER to ensure it is
+ * valid if we decide to scan the list.
+ */
+ dma_writel(dw, CLEAR.BLOCK, dwc->mask);
+ llp = channel_readl(dwc, LLP);
+ status_xfer = dma_readl(dw, RAW.XFER);
+
+ if (status_xfer & dwc->mask) {
+ /* Everything we've submitted is done */
+ dma_writel(dw, CLEAR.XFER, dwc->mask);
+ dwc_complete_all(dw, dwc);
+ return;
+ }
+
+ dev_vdbg(&dwc->chan.dev, "scan_descriptors: llp=0x%x\n", llp);
+
+ list_for_each_entry_safe(desc, _desc, &dwc->active_list, desc_node) {
+ for (lli = desc->first_lli ; lli; lli = next) {
+ next = lli->next;
+
+ dev_vdbg(&dwc->chan.dev, " lli 0x%x done?\n",
+ lli->phys);
+ /*
+ * The last descriptor can't be done because
+ * the controller isn't idle.
+ */
+ if (!next || next->phys == llp)
+ return;
+
+ /* Last LLI in a this descriptor? */
+ if (lli->last)
+ break;
+ }
+
+ dwc_descriptor_complete(dwc, desc);
+ }
+
+ dev_err(&dwc->chan.dev,
+ "BUG: All descriptors done, but channel not idle!\n");
+
+ /* Try to continue after resetting the channel... */
+ channel_clear_bit(dw, CH_EN, dwc->mask);
+ while (dma_readl(dw, CH_EN) & dwc->mask)
+ cpu_relax();
+
+ dwc->last_lli = NULL;
+ if (dwc->first_queued) {
+ dwc_dostart(dwc, dwc->first_queued);
+ dwc_submit_queue(dwc);
+ }
+}
+
+static void dwc_handle_error(struct dw_dma *dw, struct dw_dma_chan *dwc)
+{
+ struct dw_desc *bad_desc;
+ struct dw_desc *next_desc;
+ struct dw_lli *lli;
+
+ dwc_scan_descriptors(dw, dwc);
+
+ /*
+ * The descriptor currently at the head of the active list is
+ * borked. Since we don't have any way to report errors, we'll
+ * just have to scream loudly and try to carry on.
+ */
+ bad_desc = list_entry(dwc->active_list.next,
+ struct dw_desc, desc_node);
+ list_del_init(&bad_desc->desc_node);
+ if (dwc->first_queued)
+ dwc_submit_queue(dwc);
+
+ /* Clear the error flag and try to restart the controller */
+ dma_writel(dw, CLEAR.ERROR, dwc->mask);
+ if (!list_empty(&dwc->active_list)) {
+ next_desc = list_entry(dwc->active_list.next,
+ struct dw_desc, desc_node);
+ dwc_dostart(dwc, next_desc->first_lli);
+ }
+
+ /*
+ * KERN_CRITICAL may seem harsh, but since this only happens
+ * when someone submits a bad physical address in a
+ * descriptor, we should consider ourselves lucky that the
+ * controller flagged an error instead of scribbling over
+ * random memory locations.
+ */
+ dev_printk(KERN_CRIT, &dwc->chan.dev,
+ "Bad descriptor submitted for DMA!\n");
+ dev_printk(KERN_CRIT, &dwc->chan.dev,
+ " cookie: %d\n", bad_desc->slave.txd.cookie);
+ for (lli = bad_desc->first_lli; lli; lli = lli->next)
+ dev_printk(KERN_CRIT, &dwc->chan.dev,
+ " LLI: s/0x%x d/0x%x l/0x%x c/0x%x:%x\n",
+ lli->sar, lli->dar, lli->llp,
+ lli->ctlhi, lli->ctllo);
+
+ /* Pretend the descriptor completed successfully */
+ dwc_descriptor_complete(dwc, bad_desc);
+}
+
+static void dw_dma_tasklet(unsigned long data)
+{
+ struct dw_dma *dw = (struct dw_dma *)data;
+ struct dw_dma_chan *dwc;
+ u32 status_block;
+ u32 status_xfer;
+ u32 status_err;
+ int i;
+
+ status_block = dma_readl(dw, RAW.BLOCK);
+ status_xfer = dma_readl(dw, RAW.BLOCK);
+ status_err = dma_readl(dw, RAW.ERROR);
+
+ dev_dbg(dw->dma.dev, "tasklet: status_block=%x status_err=%x\n",
+ status_block, status_err);
+
+ for (i = 0; i < dw->dma.chancnt; i++) {
+ dwc = &dw->chan[i];
+ spin_lock(&dwc->lock);
+ if (status_err & (1 << i))
+ dwc_handle_error(dw, dwc);
+ else if ((status_block | status_xfer) & (1 << i))
+ dwc_scan_descriptors(dw, dwc);
+ spin_unlock(&dwc->lock);
+ }
+
+ /*
+ * Re-enable interrupts. Block Complete interrupts are only
+ * enabled if the INT_EN bit in the descriptor is set. This
+ * will trigger a scan before the whole list is done.
+ */
+ channel_set_bit(dw, MASK.XFER, dw->all_chan_mask);
+ channel_set_bit(dw, MASK.BLOCK, dw->all_chan_mask);
+ channel_set_bit(dw, MASK.ERROR, dw->all_chan_mask);
+}
+
+static irqreturn_t dw_dma_interrupt(int irq, void *dev_id)
+{
+ struct dw_dma *dw = dev_id;
+ u32 status;
+
+ dev_vdbg(dw->dma.dev, "interrupt: status=0x%x\n",
+ dma_readl(dw, STATUS_INT));
+
+ /*
+ * Just disable the interrupts. We'll turn them back on in the
+ * softirq handler.
+ */
+ channel_clear_bit(dw, MASK.XFER, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.BLOCK, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.ERROR, dw->all_chan_mask);
+
+ status = dma_readl(dw, STATUS_INT);
+ if (status) {
+ dev_err(dw->dma.dev,
+ "BUG: Unexpected interrupts pending: 0x%x\n",
+ status);
+
+ /* Try to recover */
+ channel_clear_bit(dw, MASK.XFER, (1 << 8) - 1);
+ channel_clear_bit(dw, MASK.BLOCK, (1 << 8) - 1);
+ channel_clear_bit(dw, MASK.SRC_TRAN, (1 << 8) - 1);
+ channel_clear_bit(dw, MASK.DST_TRAN, (1 << 8) - 1);
+ channel_clear_bit(dw, MASK.ERROR, (1 << 8) - 1);
+ }
+
+ tasklet_schedule(&dw->tasklet);
+
+ return IRQ_HANDLED;
+}
+
+/*----------------------------------------------------------------------*/
+
+static dma_cookie_t dwc_tx_submit(struct dma_async_tx_descriptor *tx)
+{
+ struct dw_desc *desc = txd_to_dw_desc(tx);
+ struct dw_dma_chan *dwc = to_dw_dma_chan(tx->chan);
+ struct dw_lli *lli;
+ dma_cookie_t cookie;
+
+ /* Make sure all descriptors are written to RAM */
+ for (lli = desc->first_lli; lli; lli = lli->next) {
+ dev_vdbg(&dwc->chan.dev,
+ "tx_submit: %x: s/%x d/%x p/%x h/%x l/%x\n",
+ lli->phys, lli->sar, lli->dar, lli->llp,
+ lli->ctlhi, lli->ctllo);
+ dwc_lli_sync_for_device(dwc, lli);
+ }
+
+ spin_lock_bh(&dwc->lock);
+ cookie = dwc_assign_cookie(dwc, desc);
+
+ /*
+ * REVISIT: We should attempt to chain as many descriptors as
+ * possible, perhaps even appending to those already submitted
+ * for DMA. But this is hard to do in a race-free manner.
+ */
+ if (dwc->last_queued || dwc->last_lli) {
+ dev_vdbg(&tx->chan->dev, "tx_submit: queued %u\n",
+ desc->slave.txd.cookie);
+
+ list_add_tail(&desc->desc_node, &dwc->queue);
+ dwc->last_queued = desc->last_lli;
+ if (!dwc->first_queued)
+ dwc->first_queued = desc->first_lli;
+ } else {
+ dev_vdbg(&tx->chan->dev, "tx_submit: started %u\n",
+ desc->slave.txd.cookie);
+
+ dwc_dostart(dwc, desc->first_lli);
+ list_add_tail(&desc->desc_node, &dwc->active_list);
+ dwc->last_lli = desc->last_lli;
+ }
+
+ spin_unlock_bh(&dwc->lock);
+
+ return cookie;
+}
+
+static struct dw_desc *dwc_prep_descriptor(struct dw_dma_chan *dwc,
+ u32 ctllo, dma_addr_t dest, dma_addr_t src,
+ size_t len, unsigned long flags)
+{
+ struct dma_chan *chan = &dwc->chan;
+ struct dw_desc *desc;
+ struct dw_lli *prev, *lli;
+ unsigned int offset;
+ size_t block_len;
+
+ if (unlikely(!len))
+ return NULL;
+
+ desc = dwc_desc_get(dwc);
+ if (!desc)
+ return NULL;
+
+ dev_vdbg(&chan->dev, " got descriptor %p\n", desc);
+
+ /*
+ * Use block chaining, and "transfer type 10" with source and
+ * destination addresses updated through LLP.
+ *
+ * IMPORTANT: here we assume the core is configured with each
+ * channel supporting dma descriptor lists!
+ */
+ prev = NULL;
+ for (offset = 0; offset < len; offset += block_len) {
+ size_t max_len = DWC_MAX_LEN;
+
+ lli = dwc_lli_get(dwc, GFP_ATOMIC);
+ if (!lli)
+ goto err_lli_get;
+
+ block_len = min(len - offset, max_len);
+
+ if (!prev) {
+ desc->first_lli = lli;
+ } else {
+ prev->last = 0;
+ prev->llp = lli->phys;
+ prev->next = lli;
+ }
+ lli->sar = src + offset;
+ lli->dar = dest + offset;
+ lli->ctllo = ctllo;
+ lli->ctlhi = block_len;
+
+ prev = lli;
+
+ dev_vdbg(&chan->dev,
+ " lli %p: src 0x%x dst 0x%x len %zu phys 0x%x\n",
+ lli, src, dest, block_len, lli->phys);
+ }
+
+ if (flags & DMA_PREP_INTERRUPT)
+ /* Trigger interrupt after last block */
+ prev->ctllo |= DWC_CTLL_INT_EN;
+
+ prev->next = NULL;
+ prev->llp = 0;
+ prev->last = 1;
+ desc->last_lli = prev;
+
+ return desc;
+
+err_lli_get:
+ for (lli = desc->first_lli; lli; lli = lli->next)
+ dwc_lli_put(dwc, lli);
+ dwc_desc_put(dwc, desc);
+ return NULL;
+}
+
+static struct dma_async_tx_descriptor *
+dwc_prep_dma_memcpy(struct dma_chan *chan, dma_addr_t dest, dma_addr_t src,
+ size_t len, unsigned long flags)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ struct dw_desc *desc;
+ u32 ctllo;
+ unsigned int src_width;
+ unsigned int dst_width;
+
+ dev_vdbg(&chan->dev, "prep_dma_memcpy\n");
+
+ if (unlikely(!len))
+ return NULL;
+
+ /* FIXME: Try to use wider transfers when possible */
+ src_width = 0;
+ dst_width = 0;
+
+ ctllo = DWC_DEFAULT_CTLLO
+ | DWC_CTLL_DST_WIDTH(dst_width)
+ | DWC_CTLL_SRC_WIDTH(src_width)
+ | DWC_CTLL_DST_INC
+ | DWC_CTLL_SRC_INC
+ | DWC_CTLL_FC_M2M;
+
+ desc = dwc_prep_descriptor(dwc, ctllo, dest, src,
+ len >> src_width, flags);
+
+ return desc ? &desc->slave.txd : NULL;
+}
+
+static struct dma_slave_descriptor *dwc_prep_slave(struct dma_chan *chan,
+ dma_addr_t mem_addr,
+ enum dma_slave_direction direction,
+ size_t len, unsigned long flags)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ struct dw_dma_slave *dws = dwc->dws;
+ struct dw_desc *desc;
+ u32 ctllo;
+ dma_addr_t dest, src;
+ unsigned int reg_width;
+ unsigned int mem_width;
+
+ dev_vdbg(&chan->dev, "prep_dma_slave\n");
+
+ reg_width = dws->slave.reg_width;
+
+ /* Sanity checks. len must be a multiple of the register width */
+ BUG_ON(!dws);
+ BUG_ON(len & ((1 << reg_width) - 1));
+
+ if (!(mem_addr & 3UL) && !(len & 3UL))
+ mem_width = 2;
+ else if (!(mem_addr & 1UL) && !(len & 1UL))
+ mem_width = 1;
+ else
+ mem_width = 0;
+
+ switch (direction) {
+ case DMA_SLAVE_TO_MEMORY:
+ ctllo = (DWC_DEFAULT_CTLLO
+ | DWC_CTLL_DST_WIDTH(mem_width)
+ | DWC_CTLL_SRC_WIDTH(reg_width)
+ | DWC_CTLL_DST_INC
+ | DWC_CTLL_SRC_FIX
+ | DWC_CTLL_FC_P2M);
+
+ src = dws->slave.rx_reg;
+ dest = mem_addr;
+ len >>= reg_width;
+ break;
+ case DMA_SLAVE_FROM_MEMORY:
+ ctllo = (DWC_DEFAULT_CTLLO
+ | DWC_CTLL_DST_WIDTH(reg_width)
+ | DWC_CTLL_SRC_WIDTH(mem_width)
+ | DWC_CTLL_DST_FIX
+ | DWC_CTLL_SRC_INC
+ | DWC_CTLL_FC_M2P);
+ src = mem_addr;
+
+ dest = dws->slave.tx_reg;
+ len >>= mem_width;
+ break;
+ default:
+ return NULL;
+ }
+
+ desc = dwc_prep_descriptor(dwc, ctllo, dest, src, len, flags);
+
+ return desc ? &desc->slave : NULL;
+}
+
+static void dwc_terminate_all(struct dma_chan *chan)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ struct dw_dma *dw = to_dw_dma(chan->device);
+ struct dw_desc *desc, *_desc;
+ LIST_HEAD(list);
+
+ /*
+ * This is only called when something went wrong elsewhere, so
+ * we don't really care about the data. Just disable the
+ * channel. We still have to poll the channel enable bit due
+ * to AHB/HSB limitations.
+ */
+ spin_lock_bh(&dwc->lock);
+
+ channel_clear_bit(dw, CH_EN, dwc->mask);
+
+ while (dma_readl(dw, CH_EN) & dwc->mask)
+ cpu_relax();
+
+ /* active_list entries will end up before queued entries */
+ list_splice_init(&dwc->queue, &list);
+ list_splice_init(&dwc->active_list, &list);
+ dwc->last_lli = NULL;
+ dwc->first_queued = NULL;
+ dwc->last_queued = NULL;
+
+ spin_unlock_bh(&dwc->lock);
+
+ /* Flush all pending and queued descriptors */
+ list_for_each_entry_safe(desc, _desc, &list, desc_node)
+ dwc_descriptor_complete(dwc, desc);
+}
+
+static void dwc_dependency_added(struct dma_chan *chan)
+{
+ /* FIXME: What is this hook supposed to do? */
+}
+
+static enum dma_status
+dwc_is_tx_complete(struct dma_chan *chan,
+ dma_cookie_t cookie,
+ dma_cookie_t *done, dma_cookie_t *used)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ dma_cookie_t last_used;
+ dma_cookie_t last_complete;
+ int ret;
+
+ last_complete = dwc->completed;
+ last_used = chan->cookie;
+
+ ret = dma_async_is_complete(cookie, last_complete, last_used);
+ if (ret != DMA_SUCCESS) {
+ dwc_scan_descriptors(to_dw_dma(chan->device), dwc);
+
+ last_complete = dwc->completed;
+ last_used = chan->cookie;
+
+ ret = dma_async_is_complete(cookie, last_complete, last_used);
+ }
+
+ if (done)
+ *done = last_complete;
+ if (used)
+ *used = last_used;
+
+ return ret;
+}
+
+static void dwc_issue_pending(struct dma_chan *chan)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+
+ spin_lock_bh(&dwc->lock);
+ if (dwc->last_queued)
+ dwc_scan_descriptors(to_dw_dma(chan->device), dwc);
+ spin_unlock_bh(&dwc->lock);
+}
+
+static int dwc_alloc_chan_resources(struct dma_chan *chan,
+ struct dma_client *client)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ struct dw_dma *dw = to_dw_dma(chan->device);
+ struct dw_desc *desc;
+ struct dma_slave *slave;
+ struct dw_dma_slave *dws;
+ int i;
+ u32 cfghi;
+ u32 cfglo;
+
+ dev_vdbg(&chan->dev, "alloc_chan_resources\n");
+
+ /* Channels doing slave DMA can only handle one client. */
+ if (dwc->dws || client->slave) {
+ if (dma_chan_is_in_use(chan))
+ return -EBUSY;
+ }
+
+ /* ASSERT: channel is idle */
+ if (dma_readl(dw, CH_EN) & dwc->mask) {
+ dev_dbg(&chan->dev, "DMA channel not idle?\n");
+ return -EIO;
+ }
+
+ dwc->completed = chan->cookie = 1;
+
+ cfghi = DWC_CFGH_FIFO_MODE;
+ cfglo = 0;
+
+ slave = client->slave;
+ if (slave) {
+ /*
+ * We need controller-specific data to set up slave
+ * transfers.
+ */
+ BUG_ON(!slave->dma_dev || slave->dma_dev != dw->dma.dev);
+
+ dws = container_of(slave, struct dw_dma_slave, slave);
+
+ dwc->dws = dws;
+ cfghi = dws->cfg_hi;
+ cfglo = dws->cfg_lo;
+ } else {
+ dwc->dws = NULL;
+ }
+
+ channel_writel(dwc, CFG_LO, cfglo);
+ channel_writel(dwc, CFG_HI, cfghi);
+
+ /*
+ * NOTE: some controllers may have additional features that we
+ * need to initialize here, like "scatter-gather" (which
+ * doesn't mean what you think it means), and status writeback.
+ */
+
+ spin_lock_bh(&dwc->lock);
+ i = dwc->descs_allocated;
+ while (dwc->descs_allocated < NR_DESCS_PER_CHANNEL) {
+ spin_unlock_bh(&dwc->lock);
+
+ desc = kzalloc(sizeof(struct dw_desc), GFP_KERNEL);
+ if (!desc) {
+ dev_info(&chan->dev,
+ "only allocated %d descriptors\n", i);
+ spin_lock_bh(&dwc->lock);
+ break;
+ }
+
+ dma_async_tx_descriptor_init(&desc->slave.txd, chan);
+ desc->slave.txd.ack = 1;
+ desc->slave.txd.tx_submit = dwc_tx_submit;
+
+ dev_vdbg(&chan->dev, " adding descriptor %p\n", desc);
+
+ spin_lock_bh(&dwc->lock);
+ i = ++dwc->descs_allocated;
+ list_add_tail(&desc->desc_node, &dwc->free_list);
+ }
+
+ /* Enable interrupts */
+ channel_set_bit(dw, MASK.XFER, dwc->mask);
+ channel_set_bit(dw, MASK.BLOCK, dwc->mask);
+ channel_set_bit(dw, MASK.ERROR, dwc->mask);
+
+ spin_unlock_bh(&dwc->lock);
+
+ dev_vdbg(&chan->dev,
+ "alloc_chan_resources allocated %d descriptors\n", i);
+
+ return i;
+}
+
+static void dwc_free_chan_resources(struct dma_chan *chan)
+{
+ struct dw_dma_chan *dwc = to_dw_dma_chan(chan);
+ struct dw_dma *dw = to_dw_dma(chan->device);
+ struct dw_desc *desc, *_desc;
+ LIST_HEAD(list);
+
+ dev_vdbg(&chan->dev, "free_chan_resources (descs allocated=%u)\n",
+ dwc->descs_allocated);
+
+ /* ASSERT: channel is idle */
+ BUG_ON(!list_empty(&dwc->active_list));
+ BUG_ON(!list_empty(&dwc->queue));
+ BUG_ON(dma_readl(to_dw_dma(chan->device), CH_EN) & dwc->mask);
+
+ spin_lock_bh(&dwc->lock);
+ list_splice_init(&dwc->free_list, &list);
+ dwc->descs_allocated = 0;
+ dwc->dws = NULL;
+
+ /* Disable interrupts */
+ channel_clear_bit(dw, MASK.XFER, dwc->mask);
+ channel_clear_bit(dw, MASK.BLOCK, dwc->mask);
+ channel_clear_bit(dw, MASK.ERROR, dwc->mask);
+
+ spin_unlock_bh(&dwc->lock);
+
+ list_for_each_entry_safe(desc, _desc, &list, desc_node) {
+ dev_vdbg(&chan->dev, " freeing descriptor %p\n", desc);
+ kfree(desc);
+ }
+
+ dev_vdbg(&chan->dev, "free_chan_resources done\n");
+}
+
+/*----------------------------------------------------------------------*/
+
+static void dw_dma_off(struct dw_dma *dw)
+{
+ dma_writel(dw, CFG, 0);
+
+ channel_clear_bit(dw, MASK.XFER, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.BLOCK, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.SRC_TRAN, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.DST_TRAN, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.ERROR, dw->all_chan_mask);
+
+ while (dma_readl(dw, CFG) & DW_CFG_DMA_EN)
+ cpu_relax();
+}
+
+static int __init dw_probe(struct platform_device *pdev)
+{
+ struct dw_dma_platform_data *pdata;
+ struct resource *io;
+ struct dw_dma *dw;
+#ifdef USE_DMA_POOL
+ struct dma_pool *lli_pool;
+#else
+ struct kmem_cache *lli_pool;
+#endif
+ size_t size;
+ int irq;
+ int err;
+ int i;
+
+ pdata = pdev->dev.platform_data;
+ if (!pdata || pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS)
+ return -EINVAL;
+
+ io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!io)
+ return -EINVAL;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq < 0)
+ return irq;
+
+ size = sizeof(struct dw_dma);
+ size += pdata->nr_channels * sizeof(struct dw_dma_chan);
+ dw = kzalloc(size, GFP_KERNEL);
+ if (!dw)
+ return -ENOMEM;
+
+ if (request_mem_region(io->start, DW_REGLEN,
+ pdev->dev.driver->name) == 0) {
+ err = -EBUSY;
+ goto err_kfree;
+ }
+
+ memset(dw, 0, sizeof *dw);
+
+ dw->regs = ioremap(io->start, DW_REGLEN);
+ if (!dw->regs) {
+ err = -ENOMEM;
+ goto err_release_r;
+ }
+
+ dw->clk = clk_get(&pdev->dev, "hclk");
+ if (IS_ERR(dw->clk)) {
+ err = PTR_ERR(dw->clk);
+ goto err_clk;
+ }
+ clk_enable(dw->clk);
+
+ /* force dma off, just in case */
+ dw_dma_off(dw);
+
+ err = request_irq(irq, dw_dma_interrupt, 0, "dw_dmac", dw);
+ if (err)
+ goto err_irq;
+
+#ifdef USE_DMA_POOL
+ lli_pool = dma_pool_create(pdev->dev.bus_id, &pdev->dev,
+ sizeof(struct dw_lli), 4, 0);
+#else
+ lli_pool = kmem_cache_create(pdev->dev.bus_id,
+ sizeof(struct dw_lli), 4, 0, NULL);
+#endif
+ if (!lli_pool) {
+ err = -ENOMEM;
+ goto err_dma_pool;
+ }
+
+ dw->lli_pool = lli_pool;
+ platform_set_drvdata(pdev, dw);
+
+ tasklet_init(&dw->tasklet, dw_dma_tasklet, (unsigned long)dw);
+
+ dw->all_chan_mask = (1 << pdata->nr_channels) - 1;
+
+ INIT_LIST_HEAD(&dw->dma.channels);
+ for (i = 0; i < pdata->nr_channels; i++, dw->dma.chancnt++) {
+ struct dw_dma_chan *dwc = &dw->chan[i];
+
+ dwc->chan.device = &dw->dma;
+ dwc->chan.cookie = dwc->completed = 1;
+ dwc->chan.chan_id = i;
+ list_add_tail(&dwc->chan.device_node, &dw->dma.channels);
+
+ dwc->ch_regs = &__dw_regs(dw)->CHAN[i];
+ dwc->lli_pool = lli_pool;
+ spin_lock_init(&dwc->lock);
+ dwc->mask = 1 << i;
+
+ INIT_LIST_HEAD(&dwc->active_list);
+ INIT_LIST_HEAD(&dwc->queue);
+ INIT_LIST_HEAD(&dwc->free_list);
+
+ channel_clear_bit(dw, CH_EN, dwc->mask);
+ }
+
+ /* Clear/disable all interrupts on all channels. */
+ dma_writel(dw, CLEAR.XFER, dw->all_chan_mask);
+ dma_writel(dw, CLEAR.BLOCK, dw->all_chan_mask);
+ dma_writel(dw, CLEAR.SRC_TRAN, dw->all_chan_mask);
+ dma_writel(dw, CLEAR.DST_TRAN, dw->all_chan_mask);
+ dma_writel(dw, CLEAR.ERROR, dw->all_chan_mask);
+
+ channel_clear_bit(dw, MASK.XFER, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.BLOCK, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.SRC_TRAN, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.DST_TRAN, dw->all_chan_mask);
+ channel_clear_bit(dw, MASK.ERROR, dw->all_chan_mask);
+
+ dma_cap_set(DMA_MEMCPY, dw->dma.cap_mask);
+ dma_cap_set(DMA_SLAVE, dw->dma.cap_mask);
+ dw->dma.dev = &pdev->dev;
+ dw->dma.device_alloc_chan_resources = dwc_alloc_chan_resources;
+ dw->dma.device_free_chan_resources = dwc_free_chan_resources;
+
+ dw->dma.device_prep_dma_memcpy = dwc_prep_dma_memcpy;
+
+ dw->dma.device_prep_slave = dwc_prep_slave;
+ dw->dma.device_terminate_all = dwc_terminate_all;
+
+ dw->dma.device_dependency_added = dwc_dependency_added;
+ dw->dma.device_is_tx_complete = dwc_is_tx_complete;
+ dw->dma.device_issue_pending = dwc_issue_pending;
+
+ dma_writel(dw, CFG, DW_CFG_DMA_EN);
+
+ printk(KERN_INFO "%s: DesignWare DMA Controller, %d channels\n",
+ pdev->dev.bus_id, dw->dma.chancnt);
+
+ dma_async_device_register(&dw->dma);
+
+ return 0;
+
+err_dma_pool:
+ free_irq(irq, dw);
+err_irq:
+ clk_disable(dw->clk);
+ clk_put(dw->clk);
+err_clk:
+ iounmap(dw->regs);
+ dw->regs = NULL;
+err_release_r:
+ release_resource(io);
+err_kfree:
+ kfree(dw);
+ return err;
+}
+
+static int __exit dw_remove(struct platform_device *pdev)
+{
+ struct dw_dma *dw = platform_get_drvdata(pdev);
+ struct dw_dma_chan *dwc, *_dwc;
+ struct resource *io;
+
+ dev_dbg(&pdev->dev, "dw_remove\n");
+
+ dw_dma_off(dw);
+ dma_async_device_unregister(&dw->dma);
+
+ free_irq(platform_get_irq(pdev, 0), dw);
+ tasklet_kill(&dw->tasklet);
+
+ list_for_each_entry_safe(dwc, _dwc, &dw->dma.channels,
+ chan.device_node) {
+ list_del(&dwc->chan.device_node);
+ channel_clear_bit(dw, CH_EN, dwc->mask);
+ }
+
+#ifdef USE_DMA_POOL
+ dma_pool_destroy(dw->lli_pool);
+#else
+ kmem_cache_destroy(dw->lli_pool);
+#endif
+
+ clk_disable(dw->clk);
+ clk_put(dw->clk);
+
+ iounmap(dw->regs);
+ dw->regs = NULL;
+
+ io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ release_mem_region(io->start, DW_REGLEN);
+
+ kfree(dw);
+
+ dev_dbg(&pdev->dev, "dw_remove done\n");
+
+ return 0;
+}
+
+static void dw_shutdown(struct platform_device *pdev)
+{
+ struct dw_dma *dw = platform_get_drvdata(pdev);
+
+ dw_dma_off(platform_get_drvdata(pdev));
+ clk_disable(dw->clk);
+}
+
+static int dw_suspend_late(struct platform_device *pdev, pm_message_t mesg)
+{
+ struct dw_dma *dw = platform_get_drvdata(pdev);
+
+ dw_dma_off(platform_get_drvdata(pdev));
+ clk_disable(dw->clk);
+ return 0;
+}
+
+static int dw_resume_early(struct platform_device *pdev)
+{
+ struct dw_dma *dw = platform_get_drvdata(pdev);
+
+ clk_enable(dw->clk);
+ dma_writel(dw, CFG, DW_CFG_DMA_EN);
+ return 0;
+
+}
+
+static struct platform_driver dw_driver = {
+ .remove = __exit_p(dw_remove),
+ .shutdown = dw_shutdown,
+ .suspend_late = dw_suspend_late,
+ .resume_early = dw_resume_early,
+ .driver = {
+ .name = "dw_dmac",
+ },
+};
+
+static int __init dw_init(void)
+{
+ return platform_driver_probe(&dw_driver, dw_probe);
+}
+module_init(dw_init);
+
+static void __exit dw_exit(void)
+{
+ platform_driver_unregister(&dw_driver);
+}
+module_exit(dw_exit);
+
+MODULE_LICENSE("GPL");
diff --git a/drivers/dma/dw_dmac_regs.h b/drivers/dma/dw_dmac_regs.h
new file mode 100644
index 0000000..0fedaee
--- /dev/null
+++ b/drivers/dma/dw_dmac_regs.h
@@ -0,0 +1,249 @@
+/*
+ * Driver for the Synopsys DesignWare AHB DMA Controller
+ *
+ * Copyright (C) 2005-2007 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/dw_dmac.h>
+
+#define DW_DMA_MAX_NR_CHANNELS 8
+
+/*
+ * Redefine this macro to handle differences between 32- and 64-bit
+ * addressing, big vs. little endian, etc.
+ */
+#define DW_REG(name) u32 name; u32 __pad_##name
+
+/* Hardware register definitions. */
+struct dw_dma_chan_regs {
+ DW_REG(SAR); /* Source Address Register */
+ DW_REG(DAR); /* Destination Address Register */
+ DW_REG(LLP); /* Linked List Pointer */
+ u32 CTL_LO; /* Control Register Low */
+ u32 CTL_HI; /* Control Register High */
+ DW_REG(SSTAT);
+ DW_REG(DSTAT);
+ DW_REG(SSTATAR);
+ DW_REG(DSTATAR);
+ u32 CFG_LO; /* Configuration Register Low */
+ u32 CFG_HI; /* Configuration Register High */
+ DW_REG(SGR);
+ DW_REG(DSR);
+};
+
+struct dw_dma_irq_regs {
+ DW_REG(XFER);
+ DW_REG(BLOCK);
+ DW_REG(SRC_TRAN);
+ DW_REG(DST_TRAN);
+ DW_REG(ERROR);
+};
+
+struct dw_dma_regs {
+ /* per-channel registers */
+ struct dw_dma_chan_regs CHAN[DW_DMA_MAX_NR_CHANNELS];
+
+ /* irq handling */
+ struct dw_dma_irq_regs RAW; /* r */
+ struct dw_dma_irq_regs STATUS; /* r (raw & mask) */
+ struct dw_dma_irq_regs MASK; /* rw (set = irq enabled) */
+ struct dw_dma_irq_regs CLEAR; /* w (ack, affects "raw") */
+
+ DW_REG(STATUS_INT); /* r */
+
+ /* software handshaking */
+ DW_REG(REQ_SRC);
+ DW_REG(REQ_DST);
+ DW_REG(SGL_REQ_SRC);
+ DW_REG(SGL_REQ_DST);
+ DW_REG(LAST_SRC);
+ DW_REG(LAST_DST);
+
+ /* miscellaneous */
+ DW_REG(CFG);
+ DW_REG(CH_EN);
+ DW_REG(ID);
+ DW_REG(TEST);
+
+ /* optional encoded params, 0x3c8..0x3 */
+};
+
+/* Bitfields in CTL_LO */
+#define DWC_CTLL_INT_EN (1 << 0) /* irqs enabled? */
+#define DWC_CTLL_DST_WIDTH(n) ((n)<<1) /* bytes per element */
+#define DWC_CTLL_SRC_WIDTH(n) ((n)<<4)
+#define DWC_CTLL_DST_INC (0<<7) /* DAR update/not */
+#define DWC_CTLL_DST_DEC (1<<7)
+#define DWC_CTLL_DST_FIX (2<<7)
+#define DWC_CTLL_SRC_INC (0<<7) /* SAR update/not */
+#define DWC_CTLL_SRC_DEC (1<<9)
+#define DWC_CTLL_SRC_FIX (2<<9)
+#define DWC_CTLL_DST_MSIZE(n) ((n)<<11) /* burst, #elements */
+#define DWC_CTLL_SRC_MSIZE(n) ((n)<<14)
+#define DWC_CTLL_S_GATH_EN (1 << 17) /* src gather, !FIX */
+#define DWC_CTLL_D_SCAT_EN (1 << 18) /* dst scatter, !FIX */
+#define DWC_CTLL_FC_M2M (0 << 20) /* mem-to-mem */
+#define DWC_CTLL_FC_M2P (1 << 20) /* mem-to-periph */
+#define DWC_CTLL_FC_P2M (2 << 20) /* periph-to-mem */
+#define DWC_CTLL_FC_P2P (3 << 20) /* periph-to-periph */
+/* plus 4 transfer types for peripheral-as-flow-controller */
+#define DWC_CTLL_DMS(n) ((n)<<23) /* dst master select */
+#define DWC_CTLL_SMS(n) ((n)<<25) /* src master select */
+#define DWC_CTLL_LLP_D_EN (1 << 27) /* dest block chain */
+#define DWC_CTLL_LLP_S_EN (1 << 28) /* src block chain */
+
+/* Bitfields in CTL_HI */
+#define DWC_CTLH_DONE 0x00001000
+#define DWC_CTLH_BLOCK_TS_MASK 0x00000fff
+
+/* Bitfields in CFG_LO. Platform-configurable bits are in <linux/dw_dmac.h> */
+#define DWC_CFGL_CH_SUSP (1 << 8) /* pause xfer */
+#define DWC_CFGL_FIFO_EMPTY (1 << 9) /* pause xfer */
+#define DWC_CFGL_HS_DST (1 << 10) /* handshake w/dst */
+#define DWC_CFGL_HS_SRC (1 << 11) /* handshake w/src */
+#define DWC_CFGL_MAX_BURST(x) ((x) << 20)
+#define DWC_CFGL_RELOAD_SAR (1 << 30)
+#define DWC_CFGL_RELOAD_DAR (1 << 31)
+
+/* Bitfields in CFG_HI. Platform-configurable bits are in <linux/dw_dmac.h> */
+#define DWC_CFGH_DS_UPD_EN (1 << 5)
+#define DWC_CFGH_SS_UPD_EN (1 << 6)
+
+/* Bitfields in SGR */
+#define DWC_SGR_SGI(x) ((x) << 0)
+#define DWC_SGR_SGC(x) ((x) << 20)
+
+/* Bitfields in DSR */
+#define DWC_DSR_DSI(x) ((x) << 0)
+#define DWC_DSR_DSC(x) ((x) << 20)
+
+/* Bitfields in CFG */
+#define DW_CFG_DMA_EN (1 << 0)
+
+#define DW_REGLEN 0x400
+
+struct dw_dma_chan {
+ struct dma_chan chan;
+ void __iomem *ch_regs;
+#ifdef USE_DMA_POOL
+ struct dma_pool *lli_pool;
+#else
+ struct kmem_cache *lli_pool;
+#endif
+ u8 mask;
+
+ spinlock_t lock;
+
+ /* these other elements are all protected by lock */
+ dma_cookie_t completed;
+ struct list_head active_list;
+ struct list_head queue;
+ struct list_head free_list;
+
+ struct dw_lli *last_lli;
+ struct dw_lli *first_queued;
+ struct dw_lli *last_queued;
+
+ struct dw_dma_slave *dws;
+
+ unsigned int descs_allocated;
+};
+
+static inline struct dw_dma_chan_regs __iomem *
+__dwc_regs(struct dw_dma_chan *dwc)
+{
+ return dwc->ch_regs;
+}
+
+#define channel_readl(dwc, name) \
+ __raw_readl(&(__dwc_regs(dwc)->name))
+#define channel_writel(dwc, name, val) \
+ __raw_writel((val), &(__dwc_regs(dwc)->name))
+
+static inline struct dw_dma_chan *to_dw_dma_chan(struct dma_chan *chan)
+{
+ return container_of(chan, struct dw_dma_chan, chan);
+}
+
+
+struct dw_dma {
+ struct dma_device dma;
+ void __iomem *regs;
+#ifdef USE_DMA_POOL
+ struct dma_pool *lli_pool;
+#else
+ struct kmem_cache *lli_pool;
+#endif
+ struct tasklet_struct tasklet;
+ struct clk *clk;
+
+ u8 all_chan_mask;
+
+ struct dw_dma_chan chan[0];
+};
+
+static inline struct dw_dma_regs __iomem *__dw_regs(struct dw_dma *dw)
+{
+ return dw->regs;
+}
+
+#define dma_readl(dw, name) \
+ __raw_readl(&(__dw_regs(dw)->name))
+#define dma_writel(dw, name, val) \
+ __raw_writel((val), &(__dw_regs(dw)->name))
+
+#define channel_set_bit(dw, reg, mask) \
+ dma_writel(dw, reg, ((mask) << 8) | (mask))
+#define channel_clear_bit(dw, reg, mask) \
+ dma_writel(dw, reg, ((mask) << 8) | 0)
+
+static inline struct dw_dma *to_dw_dma(struct dma_device *ddev)
+{
+ return container_of(ddev, struct dw_dma, dma);
+}
+
+
+/* LLI == Linked List Item; a.k.a. DMA block descriptor */
+struct dw_lli {
+ /* FIRST values the hardware uses */
+ dma_addr_t sar;
+ dma_addr_t dar;
+ dma_addr_t llp; /* chain to next lli */
+ u32 ctllo;
+ /* values that may get written back: */
+ u32 ctlhi;
+ /* sstat and dstat can snapshot peripheral register state.
+ * silicon config may discard either or both...
+ */
+ u32 sstat;
+ u32 dstat;
+
+ /* THEN values for driver housekeeping */
+ struct dw_lli *next;
+ dma_addr_t phys;
+ int last;
+};
+
+struct dw_desc {
+ struct dw_lli *first_lli;
+ struct dw_lli *last_lli;
+
+ struct dma_slave_descriptor slave;
+ struct list_head desc_node;
+};
+
+static inline struct dw_desc *
+txd_to_dw_desc(struct dma_async_tx_descriptor *txd)
+{
+ return container_of(txd, struct dw_desc, slave.txd);
+}
+
+static inline struct dw_desc *
+sd_to_dw_desc(struct dma_slave_descriptor *sd)
+{
+ return container_of(sd, struct dw_desc, slave);
+}
diff --git a/include/asm-avr32/arch-at32ap/at32ap700x.h b/include/asm-avr32/arch-at32ap/at32ap700x.h
index 31e48b0..d18a305 100644
--- a/include/asm-avr32/arch-at32ap/at32ap700x.h
+++ b/include/asm-avr32/arch-at32ap/at32ap700x.h
@@ -30,4 +30,20 @@
#define GPIO_PIN_PD(N) (GPIO_PIOD_BASE + (N))
#define GPIO_PIN_PE(N) (GPIO_PIOE_BASE + (N))
+
+/*
+ * DMAC peripheral hardware handshaking interfaces, used with dw_dmac
+ */
+#define DMAC_MCI_RX 0
+#define DMAC_MCI_TX 1
+#define DMAC_DAC_TX 2
+#define DMAC_AC97_A_RX 3
+#define DMAC_AC97_A_TX 4
+#define DMAC_AC97_B_RX 5
+#define DMAC_AC97_B_TX 6
+#define DMAC_DMAREQ_0 7
+#define DMAC_DMAREQ_1 8
+#define DMAC_DMAREQ_2 9
+#define DMAC_DMAREQ_3 10
+
#endif /* __ASM_ARCH_AT32AP700X_H__ */
diff --git a/include/linux/dw_dmac.h b/include/linux/dw_dmac.h
new file mode 100644
index 0000000..04d217b
--- /dev/null
+++ b/include/linux/dw_dmac.h
@@ -0,0 +1,62 @@
+/*
+ * Driver for the Synopsys DesignWare DMA Controller (aka DMACA on
+ * AVR32 systems.)
+ *
+ * Copyright (C) 2007 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#ifndef DW_DMAC_H
+#define DW_DMAC_H
+
+#include <linux/dmaengine.h>
+
+/**
+ * struct dw_dma_platform_data - Controller configuration parameters
+ * @nr_channels: Number of channels supported by hardware (max 8)
+ */
+struct dw_dma_platform_data {
+ unsigned int nr_channels;
+};
+
+/**
+ * struct dw_dma_slave - Controller-specific information about a slave
+ * @slave: Generic information about the slave
+ * @ctl_lo: Platform-specific initializer for the CTL_LO register
+ * @cfg_hi: Platform-specific initializer for the CFG_HI register
+ * @cfg_lo: Platform-specific initializer for the CFG_LO register
+ */
+struct dw_dma_slave {
+ struct dma_slave slave;
+ u32 cfg_hi;
+ u32 cfg_lo;
+};
+
+/* Platform-configurable bits in CFG_HI */
+#define DWC_CFGH_FCMODE (1 << 0)
+#define DWC_CFGH_FIFO_MODE (1 << 1)
+#define DWC_CFGH_PROTCTL(x) ((x) << 2)
+#define DWC_CFGH_SRC_PER(x) ((x) << 7)
+#define DWC_CFGH_DST_PER(x) ((x) << 11)
+
+/* Platform-configurable bits in CFG_LO */
+#define DWC_CFGL_PRIO(x) ((x) << 5) /* priority */
+#define DWC_CFGL_LOCK_CH_XFER (0 << 12) /* scope of LOCK_CH */
+#define DWC_CFGL_LOCK_CH_BLOCK (1 << 12)
+#define DWC_CFGL_LOCK_CH_XACT (2 << 12)
+#define DWC_CFGL_LOCK_BUS_XFER (0 << 14) /* scope of LOCK_BUS */
+#define DWC_CFGL_LOCK_BUS_BLOCK (1 << 14)
+#define DWC_CFGL_LOCK_BUS_XACT (2 << 14)
+#define DWC_CFGL_LOCK_CH (1 << 15) /* channel lockout */
+#define DWC_CFGL_LOCK_BUS (1 << 16) /* busmaster lockout */
+#define DWC_CFGL_HS_DST_POL (1 << 18) /* dst handshake active low */
+#define DWC_CFGL_HS_SRC_POL (1 << 19) /* src handshake active low */
+
+static inline struct dw_dma_slave *to_dw_dma_slave(struct dma_slave *slave)
+{
+ return container_of(slave, struct dw_dma_slave, slave);
+}
+
+#endif /* DW_DMAC_H */
--
1.5.3.8
Hi,
On Tue, Feb 12, 2008 at 05:43:56PM +0100, Haavard Skinnemoen wrote:
> Signed-off-by: Haavard Skinnemoen <[email protected]>
> ---
> drivers/dma/Kconfig | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
> index 893a3f8..1a727c1 100644
> --- a/drivers/dma/Kconfig
> +++ b/drivers/dma/Kconfig
> @@ -4,7 +4,7 @@
>
> menuconfig DMADEVICES
> bool "DMA Engine support"
> - depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX
> + depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX || AVR32
This is a slippery slope. Things should be the other way around instead,
create a HAVE_DMA_DEVICE, select it in the relevant platform code and
make DMADEVICES depend on that.
Or just let the subsystem always be available.
-Olof
On Tue, 12 Feb 2008 14:43:30 -0600
Olof Johansson <[email protected]> wrote:
> > - depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX
> > + depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX || AVR32
>
> This is a slippery slope. Things should be the other way around instead,
> create a HAVE_DMA_DEVICE, select it in the relevant platform code and
> make DMADEVICES depend on that.
Agree. I'll cook up a patch tomorrow to make it use the new HAVE_*
scheme.
> Or just let the subsystem always be available.
It used to be always available, but then it was changed. Assuming there
was a reason for this change, I guess we don't want to change it back.
Haavard
On Feb 12, 2008 3:13 PM, Haavard Skinnemoen <[email protected]> wrote:
> On Tue, 12 Feb 2008 14:43:30 -0600
> Olof Johansson <[email protected]> wrote:
>
> > > - depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX
> > > + depends on (PCI && X86) || ARCH_IOP32X || ARCH_IOP33X || ARCH_IOP13XX || AVR32
> >
> > This is a slippery slope. Things should be the other way around instead,
> > create a HAVE_DMA_DEVICE, select it in the relevant platform code and
> > make DMADEVICES depend on that.
>
> Agree. I'll cook up a patch tomorrow to make it use the new HAVE_*
> scheme.
>
I suggested something similar here:
http://marc.info/?l=linux-kernel&m=118670671806100&w=2
> > Or just let the subsystem always be available.
>
> It used to be always available, but then it was changed. Assuming there
> was a reason for this change, I guess we don't want to change it back.
>
Adrian had concerns about users enabling NET_DMA when the hardware
capability is relatively rare. So, +1 for HAVE_DMA_DEVICE
> Haavard
--
Dan
On Feb 12, 2008 9:43 AM, Haavard Skinnemoen <[email protected]> wrote:
[..]
> +enum dma_slave_direction {
> + DMA_SLAVE_TO_MEMORY,
> + DMA_SLAVE_FROM_MEMORY,
> +};
Just reuse enum dma_data_direction from the dma-mapping api.
--
Dan
On Wed, 13 Feb 2008 00:21:41 -0700
"Dan Williams" <[email protected]> wrote:
> On Feb 12, 2008 9:43 AM, Haavard Skinnemoen <[email protected]> wrote:
> [..]
> > +enum dma_slave_direction {
> > + DMA_SLAVE_TO_MEMORY,
> > + DMA_SLAVE_FROM_MEMORY,
> > +};
>
> Just reuse enum dma_data_direction from the dma-mapping api.
Hmm...ok. That will add two "directions" that are a bit useless in this
context (DMA_BIDIRECTIONAL and DMA_NONE.) But I guess it's still worth
it since we can pass the direction on unchanged when syncing the
buffers.
Haavard
On Tue, 12 Feb 2008 15:27:29 -0700
"Dan Williams" <[email protected]> wrote:
> > > Or just let the subsystem always be available.
> >
> > It used to be always available, but then it was changed. Assuming there
> > was a reason for this change, I guess we don't want to change it back.
> >
>
> Adrian had concerns about users enabling NET_DMA when the hardware
> capability is relatively rare. So, +1 for HAVE_DMA_DEVICE
Why not introduce a new hidden symbol, e.g. DMA_ENGINE_DRIVER, have all
the drivers select it and let NET_DMA depend on it?
Wait a minute...the Kconfig already does this. What was the problem again?
Haavard
On Feb 12, 2008 9:43 AM, Haavard Skinnemoen <[email protected]> wrote:
[..]
> /**
> + * struct dma_slave_descriptor - extended DMA descriptor for slave DMA
> + * @async_tx: async transaction descriptor
> + * @client_node: for use by the client, for example when operating on
> + * scatterlists.
> + */
> +struct dma_slave_descriptor {
> + struct dma_async_tx_descriptor txd;
> + struct list_head client_node;
> +};
Can you explain a bit why client_node is needed? I do not think we
need dma_slave_descriptor if dma_unmap data / control is added to
dma_async_tx_descriptor. Hmm?
On Wed, 13 Feb 2008 12:07:26 -0700
"Dan Williams" <[email protected]> wrote:
> > +struct dma_slave_descriptor {
> > + struct dma_async_tx_descriptor txd;
> > + struct list_head client_node;
> > +};
>
> Can you explain a bit why client_node is needed? I do not think we
> need dma_slave_descriptor if dma_unmap data / control is added to
> dma_async_tx_descriptor. Hmm?
Well, it's perhaps not needed for slave transfers as such. But the MMC
driver (and I suspect quite a few other users of the slave interface)
deals with scatterlists, so it needs a way to keep track of all the
descriptors it submits. Hence the list node.
But looking at your latest patch series, I guess we can use the new
"next" field instead. It's not like we really need the full
capabilities of list_head.
Haavard
On Wed, 13 Feb 2008 20:24:02 +0100
Haavard Skinnemoen <[email protected]> wrote:
> But looking at your latest patch series, I guess we can use the new
> "next" field instead. It's not like we really need the full
> capabilities of list_head.
On second thought, if we do this, we would be using the "next" field in
an undocumented way. It is currently documented as follows:
* @next: at completion submit this descriptor
But that's not how we're going to use it when doing slave transfers:
We're using it to keep track of all the descriptors that have already
been submitted.
I think it would actually be better to go the other way and have the
async_tx API extend the descriptor as well for its private fields. That
way, we get the pointer we need, but we can document it differently.
So how about we do something along the lines of the patch below? We
need to update all the users too of course, but apart from making the
dma_slave_descriptor struct smaller, I don't think it will change the
actual memory layout at all.
Haavard
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 9bb3407..abaf324 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -267,8 +267,7 @@ struct dma_client {
typedef void (*dma_async_tx_callback)(void *dma_async_param);
/**
- * struct dma_async_tx_descriptor - async transaction descriptor
- * ---dma generic offload fields---
+ * struct dma_descriptor - generic dma offload descriptor
* @cookie: tracking cookie for this transaction, set to -EBUSY if
* this tx is sitting on a dependency list
* @ack: the descriptor can not be reused until the client acknowledges
@@ -280,12 +279,8 @@ typedef void (*dma_async_tx_callback)(void *dma_async_param);
* @tx_submit: set the prepared descriptor(s) to be executed by the engine
* @callback: routine to call after this operation is complete
* @callback_param: general parameter to pass to the callback routine
- * ---async_tx api specific fields---
- * @next: at completion submit this descriptor
- * @parent: pointer to the next level up in the dependency chain
- * @lock: protect the parent and next pointers
*/
-struct dma_async_tx_descriptor {
+struct dma_descriptor {
dma_cookie_t cookie;
int ack;
dma_addr_t phys;
@@ -294,6 +289,17 @@ struct dma_async_tx_descriptor {
dma_cookie_t (*tx_submit)(struct dma_async_tx_descriptor *tx);
dma_async_tx_callback callback;
void *callback_param;
+};
+
+/**
+ * struct dma_async_tx_descriptor - async transaction descriptor
+ * @dma: generic dma descriptor
+ * @next: at completion submit this descriptor
+ * @parent: pointer to the next level up in the dependency chain
+ * @lock: protect the parent and next pointers
+ */
+struct dma_async_tx_descriptor {
+ struct dma_descriptor dma;
struct dma_async_tx_descriptor *next;
struct dma_async_tx_descriptor *parent;
spinlock_t lock;
@@ -301,13 +307,13 @@ struct dma_async_tx_descriptor {
/**
* struct dma_slave_descriptor - extended DMA descriptor for slave DMA
- * @async_tx: async transaction descriptor
- * @client_node: for use by the client, for example when operating on
- * scatterlists.
+ * @dma: generic dma descriptor
+ * @next: for use by the client, for example to keep track of
+ * submitted descriptors when dealing with scatterlists.
*/
struct dma_slave_descriptor {
- struct dma_async_tx_descriptor txd;
- struct list_head client_node;
+ struct dma_descriptor dma;
+ struct dma_slave_descriptor *next;
};
/**
>From: Haavard Skinnemoen [mailto:[email protected]]
>Sent: Friday, February 15, 2008 1:53 AM
>To: Haavard Skinnemoen
>Cc: Williams, Dan J; [email protected]; Nelson,
>Shannon; David Brownell; [email protected]; Francis
>Moreau; Paul Mundt; Vladimir A. Barinov; Pierre Ossman
>Subject: Re: [RFC v3 4/7] dmaengine: Add slave DMA interface
>
>On Wed, 13 Feb 2008 20:24:02 +0100
>Haavard Skinnemoen <[email protected]> wrote:
>
>> But looking at your latest patch series, I guess we can use the new
>> "next" field instead. It's not like we really need the full
>> capabilities of list_head.
>
>On second thought, if we do this, we would be using the "next" field in
>an undocumented way. It is currently documented as follows:
>
> * @next: at completion submit this descriptor
>
>But that's not how we're going to use it when doing slave transfers:
>We're using it to keep track of all the descriptors that have already
>been submitted.
>
>I think it would actually be better to go the other way and have the
>async_tx API extend the descriptor as well for its private fields. That
>way, we get the pointer we need, but we can document it differently.
>
>So how about we do something along the lines of the patch below? We
>need to update all the users too of course, but apart from making the
>dma_slave_descriptor struct smaller, I don't think it will change the
>actual memory layout at all.
>
>Haavard
>
>diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
>index 9bb3407..abaf324 100644
>--- a/include/linux/dmaengine.h
>+++ b/include/linux/dmaengine.h
>@@ -267,8 +267,7 @@ struct dma_client {
>
> typedef void (*dma_async_tx_callback)(void *dma_async_param);
> /**
>- * struct dma_async_tx_descriptor - async transaction descriptor
>- * ---dma generic offload fields---
>+ * struct dma_descriptor - generic dma offload descriptor
> * @cookie: tracking cookie for this transaction, set to -EBUSY if
> * this tx is sitting on a dependency list
> * @ack: the descriptor can not be reused until the client
>acknowledges
>@@ -280,12 +279,8 @@ typedef void
>(*dma_async_tx_callback)(void *dma_async_param);
> * @tx_submit: set the prepared descriptor(s) to be executed
>by the engine
> * @callback: routine to call after this operation is complete
> * @callback_param: general parameter to pass to the callback routine
>- * ---async_tx api specific fields---
>- * @next: at completion submit this descriptor
>- * @parent: pointer to the next level up in the dependency chain
>- * @lock: protect the parent and next pointers
> */
>-struct dma_async_tx_descriptor {
>+struct dma_descriptor {
> dma_cookie_t cookie;
> int ack;
> dma_addr_t phys;
>@@ -294,6 +289,17 @@ struct dma_async_tx_descriptor {
> dma_cookie_t (*tx_submit)(struct dma_async_tx_descriptor *tx);
> dma_async_tx_callback callback;
> void *callback_param;
>+};
>+
>+/**
>+ * struct dma_async_tx_descriptor - async transaction descriptor
>+ * @dma: generic dma descriptor
>+ * @next: at completion submit this descriptor
>+ * @parent: pointer to the next level up in the dependency chain
>+ * @lock: protect the parent and next pointers
>+ */
>+struct dma_async_tx_descriptor {
>+ struct dma_descriptor dma;
> struct dma_async_tx_descriptor *next;
> struct dma_async_tx_descriptor *parent;
> spinlock_t lock;
>@@ -301,13 +307,13 @@ struct dma_async_tx_descriptor {
>
> /**
> * struct dma_slave_descriptor - extended DMA descriptor for slave DMA
>- * @async_tx: async transaction descriptor
>- * @client_node: for use by the client, for example when operating on
>- * scatterlists.
>+ * @dma: generic dma descriptor
>+ * @next: for use by the client, for example to keep track of
>+ * submitted descriptors when dealing with scatterlists.
> */
> struct dma_slave_descriptor {
>- struct dma_async_tx_descriptor txd;
>- struct list_head client_node;
>+ struct dma_descriptor dma;
>+ struct dma_slave_descriptor *next;
> };
>
> /**
>
I'll jump in here briefly - I'm okay with the direction this is going,
but I want to be protective of ioatdma performance. As used in struct
ioat_desc_sw, the cookie and ack elements end up very close to the end
of a cache line and I'd like them to not get pushed out across the
boundry. I don't think this proposal changes the layout, I'm just
bringing up my concern.
Selfishly,
sln
On Feb 15, 2008 2:53 AM, Haavard Skinnemoen
<[email protected]> wrote:
> On Wed, 13 Feb 2008 20:24:02 +0100
> Haavard Skinnemoen <[email protected]> wrote:
>
> > But looking at your latest patch series, I guess we can use the new
> > "next" field instead. It's not like we really need the full
> > capabilities of list_head.
>
> On second thought, if we do this, we would be using the "next" field in
> an undocumented way. It is currently documented as follows:
>
> * @next: at completion submit this descriptor
>
> But that's not how we're going to use it when doing slave transfers:
> We're using it to keep track of all the descriptors that have already
> been submitted.
>
> I think it would actually be better to go the other way and have the
> async_tx API extend the descriptor as well for its private fields. That
> way, we get the pointer we need, but we can document it differently.
>
> So how about we do something along the lines of the patch below? We
> need to update all the users too of course, but apart from making the
> dma_slave_descriptor struct smaller, I don't think it will change the
> actual memory layout at all.
>
I like the direction of the patch, i.e. splitting out separate
functionality into separate structs. However, I do not want to break
the model of clients sourcing the operations and drivers sinking them
which dma_slave_descriptor appears to do. How about adding a
scatterlist pointer and an 'unmap_type' to the common descriptor?
Where unmap_type selects between, page, single, sg, or no-unmap.
Drivers already know the length and direction.
Regards,
Dan
On Sat, 16 Feb 2008 13:06:54 -0700
"Dan Williams" <[email protected]> wrote:
> I like the direction of the patch, i.e. splitting out separate
> functionality into separate structs. However, I do not want to break
> the model of clients sourcing the operations and drivers sinking them
> which dma_slave_descriptor appears to do. How about adding a
> scatterlist pointer and an 'unmap_type' to the common descriptor?
> Where unmap_type selects between, page, single, sg, or no-unmap.
> Drivers already know the length and direction.
But there are currently no operations available for submitting
scatterlists as a single descriptor -- the client iterates over the
scatterlist and submits one descriptor for each entry. So there's no
way you can associated a scatterlist with a single descriptor. You can
perhaps attach it to the last one, but that may get you into trouble if
the transfer is terminated early for some reason.
I don't think the pure source/sink model is very realistic -- clients
can't just submit DMA transfers and then forget about them. They must
at the very least check the status and take appropriate action if there
was an error. They must also notify the driver that the descriptor can
be reused (although I guess if a client doesn't care about the result
it can do this immediately after submission, but I really think clients
_should_ care about the result.)
And since they need to do some sort of cleanup anyway, they might as
well unmap the buffers (or call some dma_memcpy_finish() type of
function that does it for them.)
The clients certainly know the length and direction too, but they don't
necessarily know the physical address since the mapping is done by a
"middle layer". I guess that's the main problem with the model I'm
proposing.
How about we add a kind of "address cookie" struct like this (feel free
to suggest better names):
struct dma_buf_addr {
void *cpu_addr;
dma_addr_t dma_addr;
};
The client initializes the cpu_addr part and passes the struct to
dma_async_memcpy_buf_to_buf() or whatever, the middle layer sets the
dma_addr after mapping the buffer (or page), and the client passes the
same struct to dma_finish_memcpy_buf_to_buf(). Which will then be able
to unmap both buffers appropriately.
This will also eliminate the hack in crypto/async_tx/async_xor.c and
make HIGHMEM64G work again.
Scatterlists currently don't have any middle-layer support, so we can
ignore them for now and let the client take the full responsibility of
mapping and unmapping the buffers. If we were to add some middle-layer
functions for dealing with scatterlists, I don't think they would need
any special treatment -- the dma address is kept in the struct
scatterlist array passed by the client, so it would work pretty much
the same way without any special treatment.
Alternatively, we could convert the whole API to use scatterlists. But
that's probably overdoing it.
Btw, this discussion is a bit off-topic for the patch in question, but
it's an issue that needs to be resolved.
Haavard
On Fri, 15 Feb 2008 09:12:35 -0800
"Nelson, Shannon" <[email protected]> wrote:
> I'll jump in here briefly - I'm okay with the direction this is going,
> but I want to be protective of ioatdma performance. As used in struct
> ioat_desc_sw, the cookie and ack elements end up very close to the end
> of a cache line and I'd like them to not get pushed out across the
> boundry. I don't think this proposal changes the layout, I'm just
> bringing up my concern.
Sure, performance is very important, and it's good to see that you're
critical about the changes I'm proposing. That said, the memory layout
doesn't change at all with this patch -- the fields that didn't go into
the generic dma descriptor were at the end of the struct to begin with.
I can add a comment saying that cookie and ack must always come first.
Any other fields that we need to be careful about?
Haavard
On Feb 18, 2008 6:22 AM, Haavard Skinnemoen <[email protected]> wrote:
> On Sat, 16 Feb 2008 13:06:54 -0700
> "Dan Williams" <[email protected]> wrote:
>
> > I like the direction of the patch, i.e. splitting out separate
> > functionality into separate structs. However, I do not want to break
> > the model of clients sourcing the operations and drivers sinking them
> > which dma_slave_descriptor appears to do. How about adding a
> > scatterlist pointer and an 'unmap_type' to the common descriptor?
> > Where unmap_type selects between, page, single, sg, or no-unmap.
> > Drivers already know the length and direction.
>
> But there are currently no operations available for submitting
> scatterlists as a single descriptor -- the client iterates over the
> scatterlist and submits one descriptor for each entry. So there's no
> way you can associated a scatterlist with a single descriptor. You can
> perhaps attach it to the last one, but that may get you into trouble if
> the transfer is terminated early for some reason.
>
Drivers know how to treat a group of descriptors as one operation. I
see this as no different than the case where an operation exceeds the
hardware's per descriptor max transfer length. The driver internally
creates extra descriptors but the client only needs to worry about one
handle.
[..]
> Btw, this discussion is a bit off-topic for the patch in question, but
> it's an issue that needs to be resolved.
>
Agreed, more code less words :-). I'll try to brew this into a patch.
--
Dan
>From: Haavard Skinnemoen [mailto:[email protected]]
>Sent: Monday, February 18, 2008 5:30 AM
>To: Nelson, Shannon
>Cc: Haavard Skinnemoen; Williams, Dan J;
>[email protected]; David Brownell;
>[email protected]; Francis Moreau; Paul Mundt; Vladimir A.
>Barinov; Pierre Ossman
>Subject: Re: [RFC v3 4/7] dmaengine: Add slave DMA interface
>
>On Fri, 15 Feb 2008 09:12:35 -0800
>"Nelson, Shannon" <[email protected]> wrote:
>
>> I'll jump in here briefly - I'm okay with the direction this
>is going,
>> but I want to be protective of ioatdma performance. As used
>in struct
>> ioat_desc_sw, the cookie and ack elements end up very close
>to the end
>> of a cache line and I'd like them to not get pushed out across the
>> boundry. I don't think this proposal changes the layout, I'm just
>> bringing up my concern.
>
>Sure, performance is very important, and it's good to see that you're
>critical about the changes I'm proposing. That said, the memory layout
>doesn't change at all with this patch -- the fields that didn't go into
>the generic dma descriptor were at the end of the struct to begin with.
>
>I can add a comment saying that cookie and ack must always come first.
>Any other fields that we need to be careful about?
>
>Haavard
>
Those are the only two that I'm worried about at the moment. I'm just
hoping that a quirk in some compiler's struct packing doesn't push them
over that edge.
Thanks,
sln