2007-11-21 00:57:25

by Robert Hancock

[permalink] [raw]
Subject: [PATCH] sata_nv: fix ADMA ATAPI issues with memory over 4GB (v2)

This fixes some problems with ATAPI devices on nForce4 controllers in ADMA
mode on systems with memory located above 4GB. We need to delay setting the
64-bit DMA mask until the PRD table and padding buffer are allocated so that
they don't get allocated above 4GB and break legacy mode (which is needed for
ATAPI devices). Also, explicitly set a 32-bit DMA mask before allocating the
legacy buffers since setting the DMA mask affects both ports and we need to
ensure the second port's buffers are allocated properly (fixes a problem
with the previous version of this patch).

Signed-off-by: Robert Hancock <[email protected]>

--- linux-2.6.24-rc3-git1edit/drivers/ata/sata_nv.c.before2 2007-11-20 17:47:46.000000000 -0600
+++ linux-2.6.24-rc3-git1edit/drivers/ata/sata_nv.c 2007-11-20 17:50:30.000000000 -0600
@@ -247,6 +247,7 @@
void __iomem *ctl_block;
void __iomem *gen_block;
void __iomem *notifier_clear_block;
+ u64 adma_dma_mask;
u8 flags;
int last_issue_ncq;
};
@@ -748,7 +749,7 @@
adma_enable = 0;
nv_adma_register_mode(ap);
} else {
- bounce_limit = *ap->dev->dma_mask;
+ bounce_limit = pp->adma_dma_mask;
segment_boundary = NV_ADMA_DMA_BOUNDARY;
sg_tablesize = NV_ADMA_SGTBL_TOTAL_LEN;
adma_enable = 1;
@@ -1134,10 +1135,16 @@
void *mem;
dma_addr_t mem_dma;
void __iomem *mmio;
+ struct pci_dev *pdev = to_pci_dev(dev);
u16 tmp;

VPRINTK("ENTER\n");

+ /* Ensure DMA mask is set to 32-bit before allocating legacy PRD and
+ pad buffers */
+ pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+
rc = ata_port_start(ap);
if (rc)
return rc;
@@ -1153,6 +1160,14 @@
pp->notifier_clear_block = pp->gen_block +
NV_ADMA_NOTIFIER_CLEAR + (4 * ap->port_no);

+ /* Now that the legacy PRD and padding buffer are allocated we can
+ safely raise the DMA mask to allocate the CPB/APRD table. */
+ pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+ pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ /* Store the mask that was actually used so we can restore it as
+ the bounce limit later if needed */
+ pp->adma_dma_mask = *dev->dma_mask;
+
mem = dmam_alloc_coherent(dev, NV_ADMA_PORT_PRIV_DMA_SZ,
&mem_dma, GFP_KERNEL);
if (!mem)
@@ -2408,12 +2423,6 @@
hpriv->type = type;
host->private_data = hpriv;

- /* set 64bit dma masks, may fail */
- if (type == ADMA) {
- if (pci_set_dma_mask(pdev, DMA_64BIT_MASK) == 0)
- pci_set_consistent_dma_mask(pdev, DMA_64BIT_MASK);
- }
-
/* request and iomap NV_MMIO_BAR */
rc = pcim_iomap_regions(pdev, 1 << NV_MMIO_BAR, DRV_NAME);
if (rc)


2007-11-22 06:36:20

by Tejun Heo

[permalink] [raw]
Subject: Re: [PATCH] sata_nv: fix ADMA ATAPI issues with memory over 4GB (v2)

Hello, Robert.

Robert Hancock wrote:
> This fixes some problems with ATAPI devices on nForce4 controllers in ADMA
> mode on systems with memory located above 4GB. We need to delay setting the
> 64-bit DMA mask until the PRD table and padding buffer are allocated so that
> they don't get allocated above 4GB and break legacy mode (which is needed for
> ATAPI devices). Also, explicitly set a 32-bit DMA mask before allocating the
> legacy buffers since setting the DMA mask affects both ports and we need to
> ensure the second port's buffers are allocated properly (fixes a problem
> with the previous version of this patch).
>
> Signed-off-by: Robert Hancock <[email protected]>
>
> + /* Ensure DMA mask is set to 32-bit before allocating legacy PRD and
> + pad buffers */
> + pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
[--snip--]
> + pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));

I'm probably being paranoid here but please add error checks. Just
checking return value and returning error suffices.

Thanks.

--
tejun

2007-11-23 01:01:53

by Robert Hancock

[permalink] [raw]
Subject: Re: [PATCH] sata_nv: fix ADMA ATAPI issues with memory over 4GB (v2)

Tejun Heo wrote:
> Hello, Robert.
>
> Robert Hancock wrote:
>> This fixes some problems with ATAPI devices on nForce4 controllers in ADMA
>> mode on systems with memory located above 4GB. We need to delay setting the
>> 64-bit DMA mask until the PRD table and padding buffer are allocated so that
>> they don't get allocated above 4GB and break legacy mode (which is needed for
>> ATAPI devices). Also, explicitly set a 32-bit DMA mask before allocating the
>> legacy buffers since setting the DMA mask affects both ports and we need to
>> ensure the second port's buffers are allocated properly (fixes a problem
>> with the previous version of this patch).
>>
>> Signed-off-by: Robert Hancock <[email protected]>
>>
>> + /* Ensure DMA mask is set to 32-bit before allocating legacy PRD and
>> + pad buffers */
>> + pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
>> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
> [--snip--]
>> + pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
>> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
>
> I'm probably being paranoid here but please add error checks. Just
> checking return value and returning error suffices.

In the 32-bit case, I'm pretty sure those are guaranteed not to fail
because 32-bit is the default. For the 64-bit ones, we don't care if
they fail, because then we'll just use whatever mask ends up being set
(we store the actual set DMA mask in adma_dma_mask for use when we need
to reconfigure the bounce limit). We definitely don't want to fail
initialization if the 64-bit set doesn't succeed..

2007-11-23 01:17:42

by Tejun Heo

[permalink] [raw]
Subject: Re: [PATCH] sata_nv: fix ADMA ATAPI issues with memory over 4GB (v2)

Robert Hancock wrote:
>>> + /* Ensure DMA mask is set to 32-bit before allocating legacy PRD
>>> and
>>> + pad buffers */
>>> + pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
>>> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
>> [--snip--]
>>> + pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
>>> + pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
>>
>> I'm probably being paranoid here but please add error checks. Just
>> checking return value and returning error suffices.
>
> In the 32-bit case, I'm pretty sure those are guaranteed not to fail
> because 32-bit is the default. For the 64-bit ones, we don't care if
> they fail, because then we'll just use whatever mask ends up being set
> (we store the actual set DMA mask in adma_dma_mask for use when we need
> to reconfigure the bounce limit). We definitely don't want to fail
> initialization if the 64-bit set doesn't succeed..

Then please add BUG or WARN_ON after 32bit switching (but then again if
you're gonna do that why not just add if (rc) return rc?) and add
comment stating setting 64 bit dma masks is allowed to fail.

Thanks.

--
tejun