2008-06-02 16:27:43

by Brice Goglin

[permalink] [raw]
Subject: Re: howto use ioremap_wc?

Arjan van de Ven wrote:
> Brice Goglin <[email protected]> wrote
>> We're looking at using ioremap_wc() in myri10ge. No drivers seem to be
>> using it yet, so I'd like to get some clarification regarding
>> ioremap_wc failures, MTRR and so on.
>>
>> What we currently do is mtrr_add() and then ioremap. Depending on the
>> mtrr_add() success, we use the "wc_fifo" or regular PIO with fences to
>> submit requests to the NIC.
>>
>
> Ok this leads to a question: since write combining is effectively an
> extension (eg relaxation) to uncached, how much do you care if you
> actually get uncached? Eg can you just use the "WC" function even for
> the case where you get an uncached mapping ?
>

WC is strictly required for our "wcfifo" path, but this path is actually
not so important nowadays. It is disabled by default and might even be
removed in the future. So, no, myri10ge itself does not really need to
know whether the mapping is actually WC.

The only case I see where it would be helpful is to avoid calling
mtrr_add when WC got enabled through PAT. See below.

> Actually it would make sense for the ioremap_wc() implementation to try
> to add an mtrr I suppose... it's better to be done there than trying to
> do it in some driver....
>

Agreed, that would be nice!

>From what we discussed here after your reply, our plan is now to just
replace
ioremap+mtrr_add
with
ioremap_wc+mtrr_add

If mtrr_add() is moved into ioremap_wc(), we'll remove it from myri10ge.
For now, when PAT is enabled, we may have PAT + MTRR both doing WC, but
I don't think it can break anything, right?

thanks,
Brice


2008-06-02 18:46:43

by Loic Prylli

[permalink] [raw]
Subject: Re: howto use ioremap_wc?

On 06/02/2008 12:27 PM, Brice Goglin wrote:
> Arjan van de Ven wrote:
>
>> Ok this leads to a question: since write combining is effectively an
>> extension (eg relaxation) to uncached, how much do you care if you
>> actually get uncached? Eg can you just use the "WC" function even for
>> the case where you get an uncached mapping ?
>>
>>
>
> WC is strictly required for our "wcfifo" path,




To be more accurate about the above statement, that codepath will still
work correctly even if the mapping ends-up being uncached, but that
could lead to a 16X slowdown on some machines, since that path was
really designed for the WC case.


Anyway that codepath does not really matter as Brice mentioned
afterwards :-)



> but this path is actually
> not so important nowadays. It is disabled by default and might even be
> removed in the future. So, no, myri10ge itself does not really need to
> know whether the mapping is actually WC.
>




Loic

2008-06-03 02:18:46

by Arjan van de Ven

[permalink] [raw]
Subject: Re: howto use ioremap_wc?

On Mon, 02 Jun 2008 18:27:31 +0200
Brice Goglin <[email protected]> wrote:


>
> Agreed, that would be nice!
>
> >From what we discussed here after your reply, our plan is now to just
> replace
> ioremap+mtrr_add

which doesn't work as ioremap() may/will force uncached ;)


> with
> ioremap_wc+mtrr_add
>
> If mtrr_add() is moved into ioremap_wc(), we'll remove it from
> myri10ge. For now, when PAT is enabled, we may have PAT + MTRR both
> doing WC, but I don't think it can break anything, right?

Both having WC is nicely consistent and the right thing happens...

--
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings,
visit http://www.lesswatts.org