Subject: Re: A note on APIC bus latency
To: jlundell@pobox.com (Jonathan Lundell)
Date: Mon, 8 Oct 2001 23:08:13 +0100 (BST)
Cc: linux-kernel@vger.kernel.org
In-Reply-To: <p0510030bb7e7ca4c5533@[207.213.214.37]> from "Jonathan Lundell" at Oct 08, 2001 02:51:19 PM
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <E15qiZJ-00023i-00@the-village.bc.nu>
From: Alan Cox <alan@lxorguk.ukuu.org.uk>
Sender: linux-kernel-owner@vger.kernel.org

> A message exchange (IO-APIC sends an interrupt message; CPU sends 
> back an EOI message) requires from 35 to 48 APIC bus clocks, or 2-3 
> microseconds. That gets to be pretty significant compared to packet 
> times, especially at Gbit speeds, but even at 100 MHz, and is the 
> time required to burst a thousand bytes or more at faster PCI rates.
> 
> It's also likely to be significant for inter-processor interrupts, 
> though I don't know what the implications are here.

The big implication so far has been some extremely horrible to debug 
irq handling bugs where drivers such as the i810 audio assumed that the
disable of an irq on the pci device was immediate once then pci write
and a pci read to force posting completed. 

There are impacts on things like TLB shootdowns where the latency impacts
an SMP crosscall. I'm not sure how bad the impact is on the bigger numa
boxes as I notice Martin uses multiple sends for that

Alan
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/