2001-07-03 09:38:50

by Stephen C. Tweedie

[permalink] [raw]
Subject: Re: about kmap_high function

Hi,

On Fri, Jun 29, 2001 at 03:06:01PM +0800, michaelc wrote:
> I found that the kmap_high function didn't call __flush_tlb_one()
> when it mapped a highmem page sucessfully, and I think it maybe
> cause the problem that TLB may store obslete page table entries, but
> the kmap_atomic() function do call the __flush_tlb_one(), someone tell
> me what's the differenc between the kmap_atomic and kmap_high except
> that kmap_atomic can be used in IRQ contexts. I appreciate in advance.

kmap_high is intended to be called routinely for access to highmem
pages. It is coded to be as fast as possible as a result. TLB
flushes are expensive, especially on SMP, so kmap_high tries hard to
avoid unnecessary flushes.

The way it does it is to do only a single, complete TLB flush of the
whole kmap VA range once every time the kmap address ring cycles.
That's what flush_all_zero_pkmaps() does --- it evicts old, unused
kmap mappings and flushes the whole TLB range, so that we are
guaranteed that there is a TLB flush between any two different uses of
any given kmap virtual address.

That way, we can avoid the cost of having to flush the TLB for every
single kmap mapping we create.

Cheers,
Stephen


2001-07-03 12:48:02

by Paul Mackerras

[permalink] [raw]
Subject: Re: about kmap_high function

Stephen C. Tweedie writes:

> kmap_high is intended to be called routinely for access to highmem
> pages. It is coded to be as fast as possible as a result. TLB
> flushes are expensive, especially on SMP, so kmap_high tries hard to
> avoid unnecessary flushes.

The code assumes that flushing a single TLB entry is expensive on SMP,
while flushing the whole TLB is relatively cheap - certainly cheaper
than flushing several individual entries. And that assumption is of
course true on i386.

On PPC it is a bit different. Flushing a single TLB entry is
relatively cheap - the hardware broadcasts the TLB invalidation on the
bus (in most implementations) so there are no cross-calls required. But
flushing the whole TLB is expensive because we (strictly speaking)
have to flush the whole of the MMU hash table as well.

The MMU gets its PTEs from a hash table (which can be very large) and
we use the hash table as a kind of level-2 cache of PTEs, which means
that the flush_tlb_* routines have to flush entries from the MMU hash
table as well. The hash table can store PTEs from many contexts, so
it can have a lot of PTEs in it at any given time. So flushing the
whole TLB would imply going through every single entry in the hash
table and clearing it. In fact, currently we cheat - flush_tlb_all
actually only flushes the kernel portion of the address space, which
is all that is required in the three places where flush_tlb_all is
called at the moment.

This is not a criticism, rather a request that we expand the
interfaces so that the architecture-specific code can make the
decisions about when and how to flush TLB entries.

For example, I would like to get rid of flush_tlb_all and define a
flush_tlb_kernel_range instead. In all the places where flush_tlb_all
is currently used, we do actually know the range of addresses which
are affected, and having that information would let us do things a lot
more efficiently on PPC. On other platforms we could define
flush_tlb_kernel_range to just flush the whole TLB, or whatever.

Note that there is already a flush_tlb_range which could be used, but
some architectures assume that it is only used on user addresses.

Regards,
Paul.

2001-07-05 02:27:01

by michael chen

[permalink] [raw]
Subject: Re[2]: about kmap_high function

Hi,

Tuesday, July 03, 2001, 5:38:09 PM, you wrote:

SCT> kmap_high is intended to be called routinely for access to highmem
SCT> pages. It is coded to be as fast as possible as a result. TLB
SCT> flushes are expensive, especially on SMP, so kmap_high tries hard to
SCT> avoid unnecessary flushes.

SCT> The way it does it is to do only a single, complete TLB flush of the
SCT> whole kmap VA range once every time the kmap address ring cycles.
SCT> That's what flush_all_zero_pkmaps() does --- it evicts old, unused
SCT> kmap mappings and flushes the whole TLB range, so that we are
SCT> guaranteed that there is a TLB flush between any two different uses of
SCT> any given kmap virtual address.

SCT> That way, we can avoid the cost of having to flush the TLB for every
SCT> single kmap mapping we create.

Thank you very much for your kindly guide, and I have two question to ask
you, One question is, Is kmap_high intended to be called merely in the user
context, so the highmem pages are mapped into user process page table, so
on SMP, other processes ( including kernel and user process) that running
on another cpu doesn't need to get that kmap virtual address.
Another question is, when kernel evicts old, unused kmap mapping and
flushes the whole TLB range( call the flush_all_zero_pkmaps), the TLB won't
keep those zero mappings, after that, when user process call kmap_high to
get a new kmap mappings, and when the process access that virtual
address, MMU component will get the page directory and page table from MEMORY
instead of TLB to translate the virtual address into physical address.

--
Best regards,
Michael Chen mailto:[email protected]