Hi!
When trying to mount an ISO9660 CD on my USB CDRW unit, I get lots
of "sr: unaligned transfer" messages from the kernel and the mount
command fails. This message was added in kernel 2.5.1 and the
sr_scatter_pad() function was removed at the same time. The comment
above the message hints that unaligned transfers are (or were) a
normal thing.
I added a printk to get more information:
sr: unaligned transfer
sr: sector 64, s_size 2048, bufflen 1024
sr: unaligned transfer
sr: sector 68, s_size 2048, bufflen 1024
sr: unaligned transfer
sr: sector 72, s_size 2048, bufflen 1024
...
sr: unaligned transfer
sr: sector 396, s_size 2048, bufflen 1024
Unable to identify CD-ROM format.
So, what changes are needed to make CD support work?
--
Peter ?sterlund [email protected]
Sk?ndalsv?gen 35 http://w1.894.telia.com/~u89404340
S-128 66 Sk?ndal +46 8 942647
Sweden
On Sun, Dec 23, 2001 at 06:44:43PM +0100, Peter Osterlund wrote:
>
> So, what changes are needed to make CD support work?
The usb-storage driver needs some changes to get it to work properly in
the 2.5.1 kernel due to the changes in the SCSI and bio layer. I've
gotten a few other reports of problems, so you aren't alone :)
As for when the changes will be done, any volunteers?
thanks,
greg k-h
Peter Osterlund wrote:
> When trying to mount an ISO9660 CD on my USB CDRW unit, I get lots
> of "sr: unaligned transfer" messages from the kernel and the mount
> command fails. This message was added in kernel 2.5.1 and the
> sr_scatter_pad() function was removed at the same time.
I'm in the same boat except for my CDRW unit being SCSI.
> So, what changes are needed to make CD support work?
Evidently non-trivial... I tried a quick hack at putting the
sr_scatter_pad() code back into sr.c, but without success: null
pointer dereference when I tried to mount an ISO9660 CD. I think
I'll enjoy the holiday week and wait for Jens to produce the proper
fix :-).
--
-----------------------------------------------------------------------
Bob Tracy WTO + WIPO = DMCA? http://www.anti-dmca.org
[email protected]
-----------------------------------------------------------------------
[email protected] (Bob_Tracy) writes:
> Peter Osterlund wrote:
> > So, what changes are needed to make CD support work?
>
> Evidently non-trivial... I tried a quick hack at putting the
> sr_scatter_pad() code back into sr.c, but without success: null
> pointer dereference when I tried to mount an ISO9660 CD.
I also tried this, and it did get me a little further, but then I ran
into the problem with usb-storage that Greg KH mentioned.
> I think I'll enjoy the holiday week and wait for Jens to produce the
> proper fix :-).
I was hoping to be able to start testing my rewritten pktcdvd.o (CDRW
packet writing) module, but unfortunately I can't start until I get
standard USB CDROM support working.
--
Peter ?sterlund [email protected]
Sk?ndalsv?gen 35 http://w1.894.telia.com/~u89404340
S-128 66 Sk?ndal +46 8 942647
Sweden
This is not a direct question about the kernel itself, so if u don't want to
help me with this just ignore it!!!!!
well, i m about to invest in a new machine, in which i will only run one of
linux distros ( haven't decided which but probably suse 7.x or red hat
7.x ).
I talked to some guys, and came up with this machine:
Motherboard => Intel D850GB Al ( with network and sound )
Processor => Intel P4 2 GHz
Hard Drive => Seagate SCSI Cheata 10 000 rmp ( i think it is written
like this: Cheata ) with 16 megs of cache
Scsi controler => adaptec scsi ultra 160 whith two channels
Ram => 1000 mb ram 2x 512 rims
Video card => (haven't decided but probably a geforce 2 with 64 mb
ram )
Well these are the main components that i am thinking to use for building a
new machine.
I would like u to tell me if it is a good choice, or if i should buy a dual
xenon processor machine instead.....
Also if one of u kernel hackers/and any other ppl who has the knowlege and
patince to indicate me alll the main components of a good machine i would
appreciate that.
I am also trying to build a machine that won't give me problems with the
kernel itself... like hardware imcompatibilities..... and so on!!
and one last thing, what kind of computers do u guys use?? Tower ones as
this i am about to buy or laptop ones.
I find the laptop a funny toy, though i am not sure if it is a good
investment....
Plz some one... give me some advice so i can choose the best set up machine
possible ( and that i can affor ).
If u want to know, this machine set up i posted plus taxes and cd-rom, and
cd-writter, costs arround 8 500 dollars i think.
I live in Europe.. and thing s tend to be more expensive than in the states.
On Mon, Dec 24, 2001 at 02:13:35PM -0000, Astinus wrote:
> well, i m about to invest in a new machine, in which i will only run one of
> linux distros ( haven't decided which but probably suse 7.x or red hat
> 7.x ).
>
> I talked to some guys, and came up with this machine:
>
> Motherboard => Intel D850GB Al ( with network and sound )
> Processor => Intel P4 2 GHz
> Hard Drive => Seagate SCSI Cheata 10 000 rmp ( i think it is written
> like this: Cheata ) with 16 megs of cache
> Scsi controler => adaptec scsi ultra 160 whith two channels
> Ram => 1000 mb ram 2x 512 rims
> Video card => (haven't decided but probably a geforce 2 with 64 mb
> ram )
>
> Well these are the main components that i am thinking to use for building a
> new machine.
>
> I would like u to tell me if it is a good choice, or if i should buy a dual
> xenon processor machine instead.....
A single or dual Athlon also makes a nice computer, and probably gives
you much more value for money.
> Also if one of u kernel hackers/and any other ppl who has the knowlege and
> patince to indicate me alll the main components of a good machine i would
> appreciate that.
>
> I am also trying to build a machine that won't give me problems with the
> kernel itself... like hardware imcompatibilities..... and so on!!
This doesn't really look like a gamers machine to me, so I think you'd
better replace the geforce with a Matrox G450 or G550. Nvidia based
video cards have some known problems, especially when used together
with the Nvidia binary-only drivers.
> and one last thing, what kind of computers do u guys use?? Tower ones as
> this i am about to buy or laptop ones.
Desktops, laptops, servers, handhelds, embedded stuff. My main machine
is a laptop, though.
> I find the laptop a funny toy, though i am not sure if it is a good
> investment....
Ever tried to carry a desktop as hand luggage on board of a plane so
you could read your email during a transatlantic flight?
> I live in Europe.. and thing s tend to be more expensive than in the states.
Nonsense. Computers are equally expensive in Europe and the US.
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Faculty
of Information Technology and Systems, Delft University of Technology,
PO BOX 5031, 2600 GA Delft, The Netherlands Phone: +31-15-2783635
Fax: +31-15-2781843 Email: [email protected]
WWW: http://www-ict.its.tudelft.nl/~erik/
First of all, merry Christmas to everybody !!
On 20011224 Astinus wrote:
>
>I talked to some guys, and came up with this machine:
>
Disclaimer: obviously, this are just my humble ideas...
> Motherboard => Intel D850GB Al ( with network and sound )
Unless you plan to plug a ton of pci cards and run out of slots, do not
buy anything integrated. If something breaks, your entire box goes to hell,
instead of unplugging your sound card and sendin it to repair (or to trash...)
Do not know about Intel's mobos, but I like SuperMicro's. I have an oldie
(a P6DGU, 400GX, and it runs more stable than anything I have tested - well,
recently we bought some 370DLE with SeverWorks HE-LS and they look fine also).
Apart from the UDE UDMA issue with ServerWorks, nowadays I would reccommend
those, a SuperMicro with LE chipset. And I will never even look at a Via.
I have only suffered with them.
> Processor => Intel P4 2 GHz
I do not follow the MHz war, but that sounds the latest-and-greatest-and-
heatsink-burning. Do you really need it ? Think of a one of those new
1.2-1.3GHz Tualatins. They look like halfway a PIII and a Xeon. Save money
on the processor and invest it on the mobo. I will tell you to look at
the cache amount, but nowadays processor manufacturers are trading MHz
for cache. Brute force instead of cleverness...
> Hard Drive => Seagate SCSI Cheata 10 000 rmp ( i think it is written
>like this: Cheata ) with 16 megs of cache
> Scsi controler => adaptec scsi ultra 160 whith two channels
I have dealed with IBMs and Fujitsus. IBMs get my vote.
(btw, I think it's spelled 'cheeta', like the panthers...)
> Ram => 1000 mb ram 2x 512 rims
Those SuperMicro I talked about allow 2 (and perjaps 4) way interleaving
with DIMM (133). They run fast, very fast...
> Video card => (haven't decided but probably a geforce 2 with 64 mb
>ram )
>
If you do not plan a very heavy 3D, a GeForce2MX400 is the best price-feature
rate. The highest of the low-end. I think they are changing it now for some
called Titanium.
>Well these are the main components that i am thinking to use for building a
>new machine.
>
>I would like u to tell me if it is a good choice, or if i should buy a dual
>xenon processor machine instead.....
>
A dual box will feel much more responsive for interactive work, even if the
processors are not too fast. I have 2 PII@400MHz (yes, PII, did not loose
a I), and the box runs smoother than some GHz boxes (well, some cheating
here, the PII have 512KB caches, not like newest processors that even drop
it to 128KB).
Happy Christmas gift...
By.
--
J.A. Magallon # Let the source be with you...
mailto:[email protected]
Mandrake Linux release 8.2 (Cooker) for i586
Linux werewolf 2.4.17-beo #1 SMP Sun Dec 23 01:12:10 CET 2001 i686
[email protected] (Bob_Tracy) writes:
> > So, what changes are needed to make CD support work?
>
> Evidently non-trivial... I tried a quick hack at putting the
> sr_scatter_pad() code back into sr.c, but without success: null
> pointer dereference when I tried to mount an ISO9660 CD. I think
> I'll enjoy the holiday week and wait for Jens to produce the proper
> fix :-).
Sorry for bothering the list with this, but your mail server is
rejecting my emails to you, claiming they are SPAM.
----- The following addresses had permanent fatal errors -----
<[email protected]>
(reason: 555 [email protected] rejects SPAM from [email protected])
----- Transcript of session follows -----
... while talking to news.wlk.com.:
>>> RCPT To:<[email protected]>
<<< 555 [email protected] rejects SPAM from [email protected]
554 5.0.0 Service unavailable
[ Part 2: "Delivery Status" ]
Reporting-MTA: dns; mailb.telia.com
Received-From-MTA: DNS; d1o89.telia.com
Arrival-Date: Mon, 24 Dec 2001 23:28:55 +0100 (CET)
Final-Recipient: RFC822; [email protected]
Action: failed
Status: 5.0.0
Remote-MTA: DNS; news.wlk.com
Diagnostic-Code: SMTP; 555 [email protected] rejects SPAM from
[email protected]
Last-Attempt-Date: Mon, 24 Dec 2001 23:29:38 +0100 (CET)
--
Peter ?sterlund [email protected]
Sk?ndalsv?gen 35 http://w1.894.telia.com/~u89404340
S-128 66 Sk?ndal +46 8 942647
Sweden
On Monday 24. December 2001 16:37, Erik Mouw wrote:
> > I live in Europe.. and thing s tend to be more expensive than in the
> > states.
>
> Nonsense. Computers are equally expensive in Europe and the US.
>
You've obviously never been to Norway...
Of course, *everything* is expensive here, it seems.
--
Svein Ove Aas
On Tue, Dec 25, 2001 at 04:41:10PM +0100, Svein Ove Aas wrote:
> On Monday 24. December 2001 16:37, Erik Mouw wrote:
> > Nonsense. Computers are equally expensive in Europe and the US.
>
> You've obviously never been to Norway...
As a matter of fact, I have. Last time was three years ago.
> Of course, *everything* is expensive here, it seems.
That was also my observation. Sweden was already cheaper, and most
Norwegians agreeed, given the enormous number of Norwegian cars in the
parking lot of a supermarket just across the Swedish border :)
Erik
--
J.A.K. (Erik) Mouw, Information and Communication Theory Group, Faculty
of Information Technology and Systems, Delft University of Technology,
PO BOX 5031, 2600 GA Delft, The Netherlands Phone: +31-15-2783635
Fax: +31-15-2781843 Email: [email protected]
WWW: http://www-ict.its.tudelft.nl/~erik/
Peter Osterlund wrote:
> Hi!
>
> When trying to mount an ISO9660 CD on my USB CDRW unit, I get lots
> of "sr: unaligned transfer" messages from the kernel and the mount
> command fails. This message was added in kernel 2.5.1 and the
> sr_scatter_pad() function was removed at the same time. The comment
> above the message hints that unaligned transfers are (or were) a
> normal thing.
>
> I added a printk to get more information:
>
> sr: unaligned transfer
> sr: sector 64, s_size 2048, bufflen 1024
> sr: unaligned transfer
> sr: sector 68, s_size 2048, bufflen 1024
> sr: unaligned transfer
> sr: sector 72, s_size 2048, bufflen 1024
> ...
> sr: unaligned transfer
> sr: sector 396, s_size 2048, bufflen 1024
> Unable to identify CD-ROM format.
>
> So, what changes are needed to make CD support work?
>
>
Use -o block=2048 as your mount option. There is error in sr code
causing block sizes of CD-ROM drives to be incorrectly initialized to
1024 bytes.
Petr Titera
[email protected]
Greg KH <[email protected]> writes:
> On Sun, Dec 23, 2001 at 06:44:43PM +0100, Peter Osterlund wrote:
> >
> > So, what changes are needed to make CD support work?
>
> The usb-storage driver needs some changes to get it to work properly in
> the 2.5.1 kernel due to the changes in the SCSI and bio layer. I've
> gotten a few other reports of problems, so you aren't alone :)
>
> As for when the changes will be done, any volunteers?
This patch seems to work for me. I hope it is correct. The ide-scsi
driver is basically doing the same thing already.
--- linux-2.5-packet/drivers/usb/storage/scsiglue.c.old Sun Dec 30 02:10:01 2001
+++ linux-2.5-packet/drivers/usb/storage/scsiglue.c Sun Dec 30 02:09:05 2001
@@ -145,9 +145,19 @@
static int queuecommand( Scsi_Cmnd *srb , void (*done)(Scsi_Cmnd *))
{
struct us_data *us = (struct us_data *)srb->host->hostdata[0];
+ struct scatterlist *sg;
+ int i;
US_DEBUGP("queuecommand() called\n");
srb->host_scribble = (unsigned char *)us;
+
+ /* Set up address field in the scatterlist. HighMem pages have
+ * already been bounced at this point. */
+ sg = (struct scatterlist *) srb->request_buffer;
+ for (i = 0; i < srb->use_sg; i++) {
+ BUG_ON(PageHighMem(sg[i].page));
+ sg[i].address = page_address(sg[i].page) + sg[i].offset;
+ }
/* get exclusive access to the structures we want */
down(&(us->queue_exclusion));
--
Peter Osterlund - [email protected]
http://w1.894.telia.com/~u89404340
On Sun, Dec 30 2001, Peter Osterlund wrote:
> Greg KH <[email protected]> writes:
>
> > On Sun, Dec 23, 2001 at 06:44:43PM +0100, Peter Osterlund wrote:
> > >
> > > So, what changes are needed to make CD support work?
> >
> > The usb-storage driver needs some changes to get it to work properly in
> > the 2.5.1 kernel due to the changes in the SCSI and bio layer. I've
> > gotten a few other reports of problems, so you aren't alone :)
> >
> > As for when the changes will be done, any volunteers?
>
> This patch seems to work for me. I hope it is correct. The ide-scsi
> driver is basically doing the same thing already.
>
> --- linux-2.5-packet/drivers/usb/storage/scsiglue.c.old Sun Dec 30 02:10:01 2001
> +++ linux-2.5-packet/drivers/usb/storage/scsiglue.c Sun Dec 30 02:09:05 2001
> @@ -145,9 +145,19 @@
> static int queuecommand( Scsi_Cmnd *srb , void (*done)(Scsi_Cmnd *))
> {
> struct us_data *us = (struct us_data *)srb->host->hostdata[0];
> + struct scatterlist *sg;
> + int i;
>
> US_DEBUGP("queuecommand() called\n");
> srb->host_scribble = (unsigned char *)us;
> +
> + /* Set up address field in the scatterlist. HighMem pages have
> + * already been bounced at this point. */
> + sg = (struct scatterlist *) srb->request_buffer;
> + for (i = 0; i < srb->use_sg; i++) {
> + BUG_ON(PageHighMem(sg[i].page));
> + sg[i].address = page_address(sg[i].page) + sg[i].offset;
> + }
>
> /* get exclusive access to the structures we want */
> down(&(us->queue_exclusion));
That's not right, you shouldn't be using .address at all.
--
Jens Axboe
If it shouldn't be used, it should be removed from the structure to force
people to change.
This is probably why usb-storage broke, and it wasn't obvious to me what
went wrong.
So now I guess I need to either (a) compute the address for the USB layer,
or (b) figure out how to pass the memory parameters directly, so we can use
highmem.
Matt
On Sun, Dec 30, 2001 at 12:27:56PM +0100, Jens Axboe wrote:
> On Sun, Dec 30 2001, Peter Osterlund wrote:
> > Greg KH <[email protected]> writes:
> >
> > > On Sun, Dec 23, 2001 at 06:44:43PM +0100, Peter Osterlund wrote:
> > > >
> > > > So, what changes are needed to make CD support work?
> > >
> > > The usb-storage driver needs some changes to get it to work properly in
> > > the 2.5.1 kernel due to the changes in the SCSI and bio layer. I've
> > > gotten a few other reports of problems, so you aren't alone :)
> > >
> > > As for when the changes will be done, any volunteers?
> >
> > This patch seems to work for me. I hope it is correct. The ide-scsi
> > driver is basically doing the same thing already.
> >
> > --- linux-2.5-packet/drivers/usb/storage/scsiglue.c.old Sun Dec 30 02:10:01 2001
> > +++ linux-2.5-packet/drivers/usb/storage/scsiglue.c Sun Dec 30 02:09:05 2001
> > @@ -145,9 +145,19 @@
> > static int queuecommand( Scsi_Cmnd *srb , void (*done)(Scsi_Cmnd *))
> > {
> > struct us_data *us = (struct us_data *)srb->host->hostdata[0];
> > + struct scatterlist *sg;
> > + int i;
> >
> > US_DEBUGP("queuecommand() called\n");
> > srb->host_scribble = (unsigned char *)us;
> > +
> > + /* Set up address field in the scatterlist. HighMem pages have
> > + * already been bounced at this point. */
> > + sg = (struct scatterlist *) srb->request_buffer;
> > + for (i = 0; i < srb->use_sg; i++) {
> > + BUG_ON(PageHighMem(sg[i].page));
> > + sg[i].address = page_address(sg[i].page) + sg[i].offset;
> > + }
> >
> > /* get exclusive access to the structures we want */
> > down(&(us->queue_exclusion));
>
> That's not right, you shouldn't be using .address at all.
>
> --
> Jens Axboe
--
Matthew Dharm Home: [email protected]
Maintainer, Linux USB Mass Storage Driver
C: Why are you upgrading to NT?
AJ: It must be the sick, sadistic streak that runs through me.
-- Chief and A.J.
User Friendly, 5/12/1998
On Sun, Dec 30 2001, Matthew Dharm wrote:
> If it shouldn't be used, it should be removed from the structure to force
> people to change.
It will be soonish. Davem has practically finished this already.
> This is probably why usb-storage broke, and it wasn't obvious to me what
> went wrong.
It's been discussed here before, both wrt 2.5 and 2.4 with the block
highmem patches.
> So now I guess I need to either (a) compute the address for the USB layer,
> or (b) figure out how to pass the memory parameters directly, so we can use
> highmem.
If you don't set highmem_io in the scsi host structure, then you can
always do
vaddr = page_address(sg->page) + sg->offset;
--
Jens Axboe
Jens --
Thanks for the info. It may have been discussed 'here' (tho, this is
crosposted to two different lists), but I've been focused on 2.4 bugs (one
more left!) and hadn't seen this item.
I think for the first 2.5 kernels, we'll o with your 'vaddr' line, but I
think that being able to set highmem_io is a worthwhile thing. Which leads
me to two questions:
(1) Do the USB HCDs support highmem? I seem to recall they do, but I'm not
certain.
(2) How do I pass a highmem address to the HCDs? The URB structures we use
don't seem particularly well-suited for this.
Matt
On Mon, Dec 31, 2001 at 12:51:57PM +0100, Jens Axboe wrote:
> On Sun, Dec 30 2001, Matthew Dharm wrote:
> > If it shouldn't be used, it should be removed from the structure to force
> > people to change.
>
> It will be soonish. Davem has practically finished this already.
>
> > This is probably why usb-storage broke, and it wasn't obvious to me what
> > went wrong.
>
> It's been discussed here before, both wrt 2.5 and 2.4 with the block
> highmem patches.
>
> > So now I guess I need to either (a) compute the address for the USB layer,
> > or (b) figure out how to pass the memory parameters directly, so we can use
> > highmem.
>
> If you don't set highmem_io in the scsi host structure, then you can
> always do
>
> vaddr = page_address(sg->page) + sg->offset;
>
> --
> Jens Axboe
>
>
> _______________________________________________
> [email protected]
> To unsubscribe, use the last form field at:
> https://lists.sourceforge.net/lists/listinfo/linux-usb-devel
--
Matthew Dharm Home: [email protected]
Maintainer, Linux USB Mass Storage Driver
My mother not mind to die for stoppink Windows NT! She is rememberink
Stalin!
-- Pitr
User Friendly, 9/6/1998
Matt,
This was a point I tried to make but failed.
Not all SCB/SRB's are highmem tested but OEM's claim support.
This I tried to have BIO change to do highmen drop to the lowmem window
upon entry of the request and this would have prevent most form breaking,
but this did not seem to get warm acceptance.
Regards,
Andre Hedrick
Linux ATA Development
On Mon, 31 Dec 2001, Matthew Dharm wrote:
> Jens --
>
> Thanks for the info. It may have been discussed 'here' (tho, this is
> crosposted to two different lists), but I've been focused on 2.4 bugs (one
> more left!) and hadn't seen this item.
>
> I think for the first 2.5 kernels, we'll o with your 'vaddr' line, but I
> think that being able to set highmem_io is a worthwhile thing. Which leads
> me to two questions:
> (1) Do the USB HCDs support highmem? I seem to recall they do, but I'm not
> certain.
> (2) How do I pass a highmem address to the HCDs? The URB structures we use
> don't seem particularly well-suited for this.
>
> Matt
>
> On Mon, Dec 31, 2001 at 12:51:57PM +0100, Jens Axboe wrote:
> > On Sun, Dec 30 2001, Matthew Dharm wrote:
> > > If it shouldn't be used, it should be removed from the structure to force
> > > people to change.
> >
> > It will be soonish. Davem has practically finished this already.
> >
> > > This is probably why usb-storage broke, and it wasn't obvious to me what
> > > went wrong.
> >
> > It's been discussed here before, both wrt 2.5 and 2.4 with the block
> > highmem patches.
> >
> > > So now I guess I need to either (a) compute the address for the USB layer,
> > > or (b) figure out how to pass the memory parameters directly, so we can use
> > > highmem.
> >
> > If you don't set highmem_io in the scsi host structure, then you can
> > always do
> >
> > vaddr = page_address(sg->page) + sg->offset;
> >
> > --
> > Jens Axboe
> >
> >
> > _______________________________________________
> > [email protected]
> > To unsubscribe, use the last form field at:
> > https://lists.sourceforge.net/lists/listinfo/linux-usb-devel
>
> --
> Matthew Dharm Home: [email protected]
> Maintainer, Linux USB Mass Storage Driver
>
> My mother not mind to die for stoppink Windows NT! She is rememberink
> Stalin!
> -- Pitr
> User Friendly, 9/6/1998
>
On Mon, Dec 31 2001, Andre Hedrick wrote:
>
> Matt,
>
> This was a point I tried to make but failed.
> Not all SCB/SRB's are highmem tested but OEM's claim support.
> This I tried to have BIO change to do highmen drop to the lowmem window
> upon entry of the request and this would have prevent most form breaking,
> but this did not seem to get warm acceptance.
default behaviour is to bounce any highmem page, nothings changed there.
if you are talking about mapping highmem pages temporarily like
mentioned in an email a few days back, then that's clearly a bad idea.
--
Jens Axboe
On Mon, Dec 31 2001, Matthew Dharm wrote:
> Jens --
>
> Thanks for the info. It may have been discussed 'here' (tho, this is
> crosposted to two different lists), but I've been focused on 2.4 bugs (one
> more left!) and hadn't seen this item.
oh sorry, 'here' means linux-kernel to me :-)
> I think for the first 2.5 kernels, we'll o with your 'vaddr' line, but I
> think that being able to set highmem_io is a worthwhile thing. Which leads
> me to two questions:
indeed
> (1) Do the USB HCDs support highmem? I seem to recall they do, but I'm not
> certain.
most likely, I don't know though. I would imagine they support full
32-bit dma.
> (2) How do I pass a highmem address to the HCDs? The URB structures we use
> don't seem particularly well-suited for this.
you need to use the pci dma mapping interface, see
Documentation/DMA-mapping.txt
--
Jens Axboe
> (2) How do I pass a highmem address to the HCDs? The URB structures we use
> don't seem particularly well-suited for this.
The urb->transfer_buffer is required to be normal DMA-able memory,
following the rules in Documentation/DMA-mapping.txt ... that got cleared
up somewhere around the 2.4.5 timeframe, all the HCDs now use the
pci_map_single() calls with those buffers. kmalloc() is fine, not vmalloc(),
and so on.
Not that I've seen a writeup about highmem (linux/Documentation doesn't
seem to have one anyway) but if I infer correctly from that DMA-mapping.txt
writeup, URBs don't support it because there's no way to specify buffers as
a "struct page *" or an array of "struct scatterlist". That's the only way that
document identifies to access "highmem memory".
> (1) Do the USB HCDs support highmem? I seem to recall they do, but I'm not
> certain.
If URBs can't describe highmem, the HCD's won't support them per se;
you'd have to turn highmem to "lowmem" or whatever it's called, and
then let the HCDs manage the lowmem-to-dma_addr_t mappings.
Alternatively, in 2.5 we might add "highmem" support to USB. Now that
I've looked at it a few minutes, I suspect we must -- just to support block
devices (usb-storage) fully. Is there more to it than adding page+offset
as an alternative way to describe the transfer_buffer? (And making all
the "single" mapping calls in the HCDs use page mappings.)
- Dave
On Tue, Jan 01 2002, David Brownell wrote:
> Not that I've seen a writeup about highmem (linux/Documentation
> doesn't seem to have one anyway) but if I infer correctly from that
> DMA-mapping.txt writeup, URBs don't support it because there's no way
> to specify buffers as a "struct page *" or an array of "struct
> scatterlist". That's the only way that document identifies to access
> "highmem memory".
What kind of mapping does URBs require? A virtual mapping?
> > (1) Do the USB HCDs support highmem? I seem to recall they do, but
> > I'm not certain.
>
> If URBs can't describe highmem, the HCD's won't support them per se;
> you'd have to turn highmem to "lowmem" or whatever it's called, and
> then let the HCDs manage the lowmem-to-dma_addr_t mappings.
>
> Alternatively, in 2.5 we might add "highmem" support to USB. Now that
> I've looked at it a few minutes, I suspect we must -- just to support
> block devices (usb-storage) fully. Is there more to it than adding
No, you can always ask to get pages low mem bounced. Highmem is no
requirement, and if your device really can't support it there's no point
in attempting to support it.
> page+offset as an alternative way to describe the transfer_buffer?
no
> (And making all the "single" mapping calls in the HCDs use page
> mappings.)
exactly
--
Jens Axboe
On Tue, Jan 01, 2002 at 11:34:23PM +0100, Jens Axboe wrote:
> On Tue, Jan 01 2002, David Brownell wrote:
> > Not that I've seen a writeup about highmem (linux/Documentation
> > doesn't seem to have one anyway) but if I infer correctly from that
> > DMA-mapping.txt writeup, URBs don't support it because there's no way
> > to specify buffers as a "struct page *" or an array of "struct
> > scatterlist". That's the only way that document identifies to access
> > "highmem memory".
This sounds like another good reason to have URBs take scatterlists
directly, oddly enough. :)
> > > (1) Do the USB HCDs support highmem? I seem to recall they do, but
> > > I'm not certain.
> >
> > If URBs can't describe highmem, the HCD's won't support them per se;
> > you'd have to turn highmem to "lowmem" or whatever it's called, and
> > then let the HCDs manage the lowmem-to-dma_addr_t mappings.
> >
> > Alternatively, in 2.5 we might add "highmem" support to USB. Now that
> > I've looked at it a few minutes, I suspect we must -- just to support
> > block devices (usb-storage) fully. Is there more to it than adding
>
> No, you can always ask to get pages low mem bounced. Highmem is no
> requirement, and if your device really can't support it there's no point
> in attempting to support it.
I presume there is some overhead in bouncing to lowmem? I imagine that
highmem support for the HCDs wouldn't be that difficult -- they are just
PCI devices, after all.
I'd rather eliminate as much overhead as possible -- I already get
complaints from performance fanatics about the inability of usb-storage to
get past 92% bus saturation (sustained), and the problem will only get
worse on USB 2.0
> > page+offset as an alternative way to describe the transfer_buffer?
>
> no
Hrm... isn't that what one of the patches sent did? Or does that only work
for lowmem allocations described by the structure?
Matt
--
Matthew Dharm Home: [email protected]
Maintainer, Linux USB Mass Storage Driver
Umm, these aren't the droids you're looking for.
-- Bill Gates
User Friendly, 11/14/1998
> > Not that I've seen a writeup about highmem (linux/Documentation
> > doesn't seem to have one anyway) but if I infer correctly from that
> > DMA-mapping.txt writeup, URBs don't support it because there's no way
> > to specify buffers as a "struct page *" or an array of "struct
> > scatterlist". That's the only way that document identifies to access
> > "highmem memory".
>
> What kind of mapping does URBs require? A virtual mapping?
Each URB points to one transfer buffer, address plus length, where
the USB device drivers directly read/write those addresses. The
requirement for drivers is that the transfer buffers can be passed to
pci_map_single() calls by the Host Controller Drivers (HCDs). The
device drivers, and URBs, don't expose such mappings, they only
require that they can be created/destroyed.
I'd be inclined to say transfer buffers must not be "virtual" mappings,
but since terminology can differ I'll let you draw your conclusion.
> > > (1) Do the USB HCDs support highmem? I seem to recall they do, but
> > > I'm not certain.
> >
> > If URBs can't describe highmem, the HCD's won't support them per se;
> > you'd have to turn highmem to "lowmem" or whatever it's called, and
> > then let the HCDs manage the lowmem-to-dma_addr_t mappings.
> >
> > Alternatively, in 2.5 we might add "highmem" support to USB. Now that
> > I've looked at it a few minutes, I suspect we must -- just to support
> > block devices (usb-storage) fully. Is there more to it than adding
>
> No, you can always ask to get pages low mem bounced. Highmem is no
> requirement, and if your device really can't support it there's no point
> in attempting to support it.
Standard HC hardware (EHCI, OHCI, UHCI) can use 32bit addresses
for their PCI DMA. EHCI can also, in some implementations, handle 64bit
addresses.
> > page+offset as an alternative way to describe the transfer_buffer?
>
> no
>
> > (And making all the "single" mapping calls in the HCDs use page
> > mappings.)
>
> exactly
So you're saying that pci_map_page() is not necessary? And that
highmem buffers (page+offset, or scatterlist thereof) can be turned
into the form needed by URBs using some other mechanism?
I'd certainly rather that be the case for the moment, for simplicity.
Keep in mind that usb-storage seems to be the only driver that'd
run into that issue.
- Dave
> > > Not that I've seen a writeup about highmem (linux/Documentation
> > > doesn't seem to have one anyway) but if I infer correctly from that
> > > DMA-mapping.txt writeup, URBs don't support it because there's no way
> > > to specify buffers as a "struct page *" or an array of "struct
> > > scatterlist". That's the only way that document identifies to access
> > > "highmem memory".
>
> This sounds like another good reason to have URBs take scatterlists
> directly, oddly enough. :)
If it's got to be done, I'd much rather it were "page + offset", so that the
usbcore code can be simpler. We know how to turn scatterlists into
bulk queued requests, so there's no need for anything more ... :)
> > No, you can always ask to get pages low mem bounced. Highmem is no
> > requirement, and if your device really can't support it there's no point
> > in attempting to support it.
>
> I presume there is some overhead in bouncing to lowmem? I imagine that
> highmem support for the HCDs wouldn't be that difficult -- they are just
> PCI devices, after all.
I'm unclear on what "bouncing to lowmem" involves, but I'd rather avoid
teaching all three HCDs a second model for addressing transfer buffers.
At least until later in the 2.5 series, when we believe they'll share a lot
more common code and so that new model can be taught to just ONE
piece of code. Fixing bugs in one place easier than in three!
> I'd rather eliminate as much overhead as possible -- I already get
> complaints from performance fanatics about the inability of usb-storage to
> get past 92% bus saturation (sustained), and the problem will only get
> worse on USB 2.0
Well then you'll be glad to see a patch from me, soonish, that teaches
the usb-storage "transport" code to use bulk queueing. That'll get the
bandwidth utilization up as high as it can get. It won't address any of
these highmem issues though.
- Dave
> > I presume there is some overhead in bouncing to lowmem? I imagine that
> > highmem support for the HCDs wouldn't be that difficult -- they are just
> > PCI devices, after all.
>
> I'm unclear on what "bouncing to lowmem" involves, but I'd rather avoid
> teaching all three HCDs a second model for addressing transfer buffers.
AFAIK bouncing means a plain, physical copy.
Either the HCDs can do 64bit DMA or they can't.
Do you really expect there to be a significant number of 32bit machines
whose HCD can do 64bit DMA ?
If not, it's IMHO not worth doing it as you'd have either two kinds of
urbs or overhead in the common case.
On 64Bit machines we might have to deal with HCDs who can do 32Bit DMA
only. Perhaps there should be a gfp field in the usb_device struct
to export knowledge about the memory the HCD can cope with.
> > I'd rather eliminate as much overhead as possible -- I already get
> > complaints from performance fanatics about the inability of usb-storage to
> > get past 92% bus saturation (sustained), and the problem will only get
> > worse on USB 2.0
>
> Well then you'll be glad to see a patch from me, soonish, that teaches
> the usb-storage "transport" code to use bulk queueing. That'll get the
> bandwidth utilization up as high as it can get. It won't address any of
> these highmem issues though.
And there's the overhead of sleeping and waking a kernel thread. Larger io
requests might help, but I am not sure.
Regards
Oliver
On Tue, Jan 01 2002, Matthew Dharm wrote:
> On Tue, Jan 01, 2002 at 11:34:23PM +0100, Jens Axboe wrote:
> > On Tue, Jan 01 2002, David Brownell wrote:
> > > Not that I've seen a writeup about highmem (linux/Documentation
> > > doesn't seem to have one anyway) but if I infer correctly from that
> > > DMA-mapping.txt writeup, URBs don't support it because there's no way
> > > to specify buffers as a "struct page *" or an array of "struct
> > > scatterlist". That's the only way that document identifies to access
> > > "highmem memory".
>
> This sounds like another good reason to have URBs take scatterlists
> directly, oddly enough. :)
Or at least a derived structure, however it might as well just use
scatterlist (address member is soon gone). So yes.
> > > > (1) Do the USB HCDs support highmem? I seem to recall they do, but
> > > > I'm not certain.
> > >
> > > If URBs can't describe highmem, the HCD's won't support them per se;
> > > you'd have to turn highmem to "lowmem" or whatever it's called, and
> > > then let the HCDs manage the lowmem-to-dma_addr_t mappings.
> > >
> > > Alternatively, in 2.5 we might add "highmem" support to USB. Now that
> > > I've looked at it a few minutes, I suspect we must -- just to support
> > > block devices (usb-storage) fully. Is there more to it than adding
> >
> > No, you can always ask to get pages low mem bounced. Highmem is no
> > requirement, and if your device really can't support it there's no point
> > in attempting to support it.
>
> I presume there is some overhead in bouncing to lowmem? I imagine that
> highmem support for the HCDs wouldn't be that difficult -- they are just
> PCI devices, after all.
Lots of overhead required for copying data back and forth between low
and high page (both the memcpy, but also the highmem kmap). In addition
it puts added pressure on the memory allocator.
> I'd rather eliminate as much overhead as possible -- I already get
> complaints from performance fanatics about the inability of usb-storage to
> get past 92% bus saturation (sustained), and the problem will only get
> worse on USB 2.0
Then you should move the the page/len/offset construct.
>
> > > page+offset as an alternative way to describe the transfer_buffer?
> >
> > no
>
> Hrm... isn't that what one of the patches sent did? Or does that only work
> for lowmem allocations described by the structure?
Hmmm no, I don't think so. It only worked for low mem pages, since it
used page_address to 'map' the page.
--
Jens Axboe
On Tue, Jan 01 2002, David Brownell wrote:
> > > No, you can always ask to get pages low mem bounced. Highmem is no
> > > requirement, and if your device really can't support it there's no point
> > > in attempting to support it.
> >
> > I presume there is some overhead in bouncing to lowmem? I imagine that
> > highmem support for the HCDs wouldn't be that difficult -- they are just
> > PCI devices, after all.
>
> I'm unclear on what "bouncing to lowmem" involves, but I'd rather avoid
> teaching all three HCDs a second model for addressing transfer buffers.
>
> At least until later in the 2.5 series, when we believe they'll share a lot
> more common code and so that new model can be taught to just ONE
> piece of code. Fixing bugs in one place easier than in three!
Ehm I was discussing 2.5, 2.4 will always bounce the high mem pages for
you so it's moot there.
--
Jens Axboe
On Tue, Jan 01 2002, David Brownell wrote:
> > > Not that I've seen a writeup about highmem (linux/Documentation
> > > doesn't seem to have one anyway) but if I infer correctly from that
> > > DMA-mapping.txt writeup, URBs don't support it because there's no way
> > > to specify buffers as a "struct page *" or an array of "struct
> > > scatterlist". That's the only way that document identifies to access
> > > "highmem memory".
> >
> > What kind of mapping does URBs require? A virtual mapping?
>
> Each URB points to one transfer buffer, address plus length, where
> the USB device drivers directly read/write those addresses. The
> requirement for drivers is that the transfer buffers can be passed to
> pci_map_single() calls by the Host Controller Drivers (HCDs). The
> device drivers, and URBs, don't expose such mappings, they only
> require that they can be created/destroyed.
.. which is the requirement that you want to change to use pci_map_page
or pci_map_sg
> I'd be inclined to say transfer buffers must not be "virtual" mappings,
> but since terminology can differ I'll let you draw your conclusion.
They must not be, currently they are.
> Standard HC hardware (EHCI, OHCI, UHCI) can use 32bit addresses
> for their PCI DMA. EHCI can also, in some implementations, handle 64bit
> addresses.
Good
> > > (And making all the "single" mapping calls in the HCDs use page
> > > mappings.)
> >
> > exactly
>
> So you're saying that pci_map_page() is not necessary? And that
> highmem buffers (page+offset, or scatterlist thereof) can be turned
> into the form needed by URBs using some other mechanism?
Sorry if I wasn't clear, no that's not what I meant. See above.
pci_map_single requires a virtual mapping so it simply cannot support
highmem.
--
Jens Axboe
> > requirement for drivers is that the transfer buffers can be passed to
> > pci_map_single() calls by the Host Controller Drivers (HCDs). The
> > device drivers, and URBs, don't expose such mappings, they only
> > require that they can be created/destroyed.
>
> .. which is the requirement that you want to change to use pci_map_page
> or pci_map_sg
OK, I think I'm clear on this much then: in 2.5, to support block drivers
over USB (usb-storage only, for now) there needs to be an addition to
the buffer addressing model in usbcore, as exposed by URBs.
- Current "transfer_buffer" + "transfer_buffer_length" mode needs to
stay, since most drivers aren't block drivers.
- Add some kind of "page + offset" addressing model.
Discussion of details can be taken off LKML, it'd seem. Though I'm
curious when the scatterlist->address field will vanish, making these
changes a requirement. Is that a 2.5.2 thing?
Also, I noticed that include/asm-sparc/pci.h doesn't include the
standard pci_map_page() call ... what's up with that? That surely
causes portability problems.
- Dave
On Wed, Jan 02 2002, David Brownell wrote:
> > > requirement for drivers is that the transfer buffers can be passed to
> > > pci_map_single() calls by the Host Controller Drivers (HCDs). The
> > > device drivers, and URBs, don't expose such mappings, they only
> > > require that they can be created/destroyed.
> >
> > .. which is the requirement that you want to change to use pci_map_page
> > or pci_map_sg
>
> OK, I think I'm clear on this much then: in 2.5, to support block drivers
> over USB (usb-storage only, for now) there needs to be an addition to
> the buffer addressing model in usbcore, as exposed by URBs.
>
> - Current "transfer_buffer" + "transfer_buffer_length" mode needs to
> stay, since most drivers aren't block drivers.
Why? Surely USB block drivers are not the only ones that want to support
highmem.
> - Add some kind of "page + offset" addressing model.
Yes
> Discussion of details can be taken off LKML, it'd seem. Though I'm
> curious when the scatterlist->address field will vanish, making these
> changes a requirement. Is that a 2.5.2 thing?
Maybe 2.5.3, dunno for sure.
> Also, I noticed that include/asm-sparc/pci.h doesn't include the
> standard pci_map_page() call ... what's up with that? That surely
> causes portability problems.
It probably isn't up to snuff yet.
--
Jens Axboe
> > > I'd rather eliminate as much overhead as possible -- I already get
> > > complaints from performance fanatics about the inability of usb-storage to
> > > get past 92% bus saturation (sustained), and the problem will only get
> > > worse on USB 2.0
> >
> > Well then you'll be glad to see a patch from me, soonish, that teaches
> > the usb-storage "transport" code to use bulk queueing. That'll get the
> > bandwidth utilization up as high as it can get. It won't address any of
> > these highmem issues though.
>
> And there's the overhead of sleeping and waking a kernel thread. Larger io
> requests might help, but I am not sure.
Yes, it's that sleep/wake between scatterlist segments that's creating
that 92% (at 12 Mbit/sec) or about 20% (at 480 Mbit/sec :) bottleneck ...
Convert those calls to use bulk queuing, and those delays vanish.
- Dave
> > OK, I think I'm clear on this much then: in 2.5, to support block drivers
> > over USB (usb-storage only, for now) there needs to be an addition to
> > the buffer addressing model in usbcore, as exposed by URBs.
> >
> > - Current "transfer_buffer" + "transfer_buffer_length" mode needs to
> > stay, since most drivers aren't block drivers.
>
> Why? Surely USB block drivers are not the only ones that want to support
> highmem.
Once the capability is there, it'll find other uses. But allowing them is not
the same as requiring them. Getting rid of the current model would break
every USB driver, rather than just ones that want to support highmem.
> > - Add some kind of "page + offset" addressing model.
>
> Yes
>
> > Discussion of details can be taken off LKML, it'd seem. Though I'm
> > curious when the scatterlist->address field will vanish, making these
> > changes a requirement. Is that a 2.5.2 thing?
>
> Maybe 2.5.3, dunno for sure.
A bit of a delay would make things a bit easier ... :)
Of course, if scatterlist->address doesn't work any more,
it won't matter much.
- Dave
> > Also, I noticed that include/asm-sparc/pci.h doesn't include the
> > standard pci_map_page() call ... what's up with that? That surely
> > causes portability problems.
>
> It probably isn't up to snuff yet.
>
> --
> Jens Axboe
>
On Wed, Jan 02 2002, David Brownell wrote:
> > > OK, I think I'm clear on this much then: in 2.5, to support block drivers
> > > over USB (usb-storage only, for now) there needs to be an addition to
> > > the buffer addressing model in usbcore, as exposed by URBs.
> > >
> > > - Current "transfer_buffer" + "transfer_buffer_length" mode needs to
> > > stay, since most drivers aren't block drivers.
> >
> > Why? Surely USB block drivers are not the only ones that want to support
> > highmem.
>
> Once the capability is there, it'll find other uses. But allowing
> them is not the same as requiring them. Getting rid of the current
> model would break every USB driver, rather than just ones that want to
> support highmem.
So? Either you want to fix this now, or leave it that way forever. Just
IMO of course, but you might as well just make a clean break.
> > > - Add some kind of "page + offset" addressing model.
> >
> > Yes
> >
> > > Discussion of details can be taken off LKML, it'd seem. Though
> > > I'm curious when the scatterlist->address field will vanish,
> > > making these changes a requirement. Is that a 2.5.2 thing?
> >
> > Maybe 2.5.3, dunno for sure.
>
> A bit of a delay would make things a bit easier ... :) Of course, if
> scatterlist->address doesn't work any more, it won't matter much.
A bit of delay will only make things worse, afaics.
--
Jens Axboe
On Wed, 2 Jan 2002, Jens Axboe wrote:
> On Wed, Jan 02 2002, David Brownell wrote:
> > > > requirement for drivers is that the transfer buffers can be passed to
> > > > pci_map_single() calls by the Host Controller Drivers (HCDs). The
> > > > device drivers, and URBs, don't expose such mappings, they only
> > > > require that they can be created/destroyed.
> > >
> > > .. which is the requirement that you want to change to use pci_map_page
> > > or pci_map_sg
> >
> > OK, I think I'm clear on this much then: in 2.5, to support block drivers
> > over USB (usb-storage only, for now) there needs to be an addition to
> > the buffer addressing model in usbcore, as exposed by URBs.
> >
> > - Current "transfer_buffer" + "transfer_buffer_length" mode needs to
> > stay, since most drivers aren't block drivers.
>
> Why? Surely USB block drivers are not the only ones that want to support
> highmem.
Probably for a long time they'll be the only ones.
All the char drivers will mainly do a copy_to/from_user
or want memory they can manipulate directly.
Regards
Oliver