Jens, just got this crash on a testbox:
[ 37.701628] warning: process `kmodule' used the removed sysctl system call with 1.23.
[ 39.409390] BUG: unable to handle kernel paging request at virtual address 7ca76000
[ 39.416892] printing eip: 78406669 *pde = 00dda027 *pte = 04a76000
[ 39.423132] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 39.427292]
[ 39.428766] Pid: 431, comm: fsck.ext3 Not tainted (2.6.23 #45)
[ 39.434571] EIP: 0060:[<78406669>] EFLAGS: 00010006 CPU: 0
[ 39.440035] EIP is at blk_rq_map_sg+0xb9/0x170
[ 39.444450] EAX: 04aed000 EBX: 7ca1f380 ECX: 04aee000 EDX: 00095da0
[ 39.450689] ESI: 7ca75ff0 EDI: 79180da0 EBP: 00001000 ESP: 7caa1cac
[ 39.456929] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[ 39.462303] Process fsck.ext3 (pid: 431, ti=7caa0000 task=7ca80000 task.ti=7caa0000)
[ 39.469841] Stack: 00000020 7b520000 00002000 04aef000 7ca75fe0 0000001f 00000001 00000000
[ 39.478161] 7ca1f300 01000002 7ca44980 7b522b94 7ca0de00 7b520000 784c89b5 7b520000
[ 39.486480] 784c843a 7ca0de00 7b520f20 7b522b94 7ca0de00 784e54a0 784f394b 00000000
[ 39.494799] Call Trace:
[ 39.497400] [<784c89b5>] scsi_init_io+0x55/0xe0
[ 39.501992] [<784c843a>] scsi_get_cmd_from_req+0x2a/0x40
[ 39.507366] [<784e54a0>] sd_prep_fn+0x80/0x940
[ 39.511872] [<784f394b>] ata_bmdma_start+0xb/0x20
[ 39.516638] [<784ef344>] ata_qc_issue_prot+0x164/0x1e0
[ 39.521839] [<78405c63>] elv_dispatch_sort+0x23/0xe0
[ 39.526865] [<784057d0>] elv_next_request+0xa0/0x130
[ 39.531891] [<787715b8>] _spin_lock_irq+0x38/0x50
[ 39.536657] [<784c9af4>] scsi_request_fn+0x1e4/0x370
[ 39.541684] [<78120e72>] del_timer+0x62/0x70
[ 39.546016] [<78408a45>] __generic_unplug_device+0x25/0x30
[ 39.551563] [<78408d15>] generic_unplug_device+0x15/0x30
[ 39.556936] [<78406430>] blk_backing_dev_unplug+0x0/0x10
[ 39.562309] [<7840643c>] blk_backing_dev_unplug+0xc/0x10
[ 39.567682] [<7818080d>] block_sync_page+0x2d/0x40
[ 39.572535] [<78144ee9>] sync_page+0x29/0x40
[ 39.576868] [<7876fdec>] __wait_on_bit_lock+0x3c/0x70
[ 39.581981] [<78144ec0>] sync_page+0x0/0x40
[ 39.586227] [<78144ea2>] __lock_page+0x52/0x60
[ 39.590734] [<7812ae90>] wake_bit_function+0x0/0x60
[ 39.595674] [<7814566c>] do_generic_mapping_read+0x21c/0x450
[ 39.601393] [<78144c30>] file_read_actor+0x0/0x130
[ 39.606246] [<78147227>] generic_file_aio_read+0x137/0x180
[ 39.611793] [<78144c30>] file_read_actor+0x0/0x130
[ 39.616645] [<78160d95>] do_sync_read+0xd5/0x120
[ 39.621326] [<78135cd8>] mark_held_locks+0x38/0x70
[ 39.626179] [<7812ae50>] autoremove_wake_function+0x0/0x40
[ 39.631724] [<787705b4>] mutex_lock_nested+0x1a4/0x200
[ 39.636924] [<78185611>] block_llseek+0x31/0xc0
[ 39.641517] [<787703a5>] __mutex_unlock_slowpath+0xb5/0x110
[ 39.647150] [<78135e1c>] trace_hardirqs_on+0x9c/0xb0
[ 39.652177] [<78160cc0>] do_sync_read+0x0/0x120
[ 39.656770] [<7816165b>] vfs_read+0xbb/0x120
[ 39.661103] [<78161a41>] sys_read+0x41/0x70
[ 39.665349] [<781028a2>] syscall_call+0x7/0xb
[ 39.669769] =======================
[ 39.673322] Code: 05 c1 e1 0c 03 4a 08 8b 52 04 01 ca 89 54 24 0c 8b 3b 89 fa 29 c2 89 d0 c1 f8 05 c1 e0 0c 03 43 08 39 44 24 0c 0f 84 7e 00 00 00 <8b> 46 10 8d 56 10 a8 01 75 4c 89 3e 89 6e 0c 8b 43 08 89 46 04
config and full bootlog attached.
i tried another bootup with the same kernel and the crash did not
reoccur, so it seems to be spurious. This crash could be related to the
scsi or block merges done in the past few days - never saw this before.
Ingo
* Ingo Molnar <[email protected]> wrote:
> i tried another bootup with the same kernel and the crash did not
> reoccur, so it seems to be spurious. This crash could be related to
> the scsi or block merges done in the past few days - never saw this
> before.
managed to trigger it a second time, so it seems rather reproducible:
[ 328.771333] kjournald starting. Commit interval 5 seconds
[ 328.776963] EXT3 FS on sda5, internal journal
[ 328.781172] EXT3-fs: mounted filesystem with ordered data mode.
[ 329.689493] BUG: unable to handle kernel paging request at virtual address 7d516000
[ 329.696990] printing eip: 78406669 *pde = 00ddd027 *pte = 05516000
[ 329.703230] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 329.707390]
[ 329.708863] Pid: 0, comm: swapper Not tainted (2.6.23 #45)
[ 329.714321] EIP: 0060:[<78406669>] EFLAGS: 00010006 CPU: 0
[ 329.719787] EIP is at blk_rq_map_sg+0xb9/0x170
[ 329.724202] EAX: 2f6df000 EBX: 7d6af880 ECX: 34d55000 EDX: 005edbe0
[ 329.730441] ESI: 7d515ff0 EDI: 796d8be0 EBP: 00001000 ESP: 78a13db0
[ 329.736680] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 329.742054] Process swapper (pid: 0, ti=78a12000 task=7893c400 task.ti=78a12000)
[ 329.749246] Stack: 00000020 7b520000 00002000 34d56000 7d515fe0 00000007 00000001 00000000
[ 329.757565] 7d6af800 01000000 7d515e00 7b522b94 7d84f58c 7b520000 784c89b5 7b520000
[ 329.765885] 784c843a 7d84f58c 7b520f20 7b522b94 7d84f58c 784e54a0 784f394b 00000000
[ 329.774204] Call Trace:
[ 329.776804] [<784c89b5>] scsi_init_io+0x55/0xe0
[ 329.781398] [<784c843a>] scsi_get_cmd_from_req+0x2a/0x40
[ 329.786770] [<784e54a0>] sd_prep_fn+0x80/0x940
[ 329.791277] [<784f394b>] ata_bmdma_start+0xb/0x20
[ 329.796043] [<784ef344>] ata_qc_issue_prot+0x164/0x1e0
[ 329.801243] [<78405c63>] elv_dispatch_sort+0x23/0xe0
[ 329.806268] [<784057d0>] elv_next_request+0xa0/0x130
[ 329.811295] [<787715b8>] _spin_lock_irq+0x38/0x50
[ 329.816062] [<784c9af4>] scsi_request_fn+0x1e4/0x370
[ 329.821088] [<784097f6>] blk_run_queue+0x36/0x80
[ 329.825768] [<784c8370>] scsi_next_command+0x30/0x50
[ 329.830794] [<784c84fb>] scsi_end_request+0xab/0xe0
[ 329.835733] [<784c9249>] scsi_io_completion+0xa9/0x3d0
[ 329.840933] [<78135de7>] trace_hardirqs_on+0x67/0xb0
[ 329.845960] [<784068c5>] blk_done_softirq+0x45/0x80
[ 329.850900] [<784068f3>] blk_done_softirq+0x73/0x80
[ 329.855839] [<7811d3f3>] __do_softirq+0x53/0xb0
[ 329.860432] [<7811d4b8>] do_softirq+0x68/0x70
[ 329.864852] [<78105351>] do_IRQ+0x51/0x90
[ 329.868925] [<78135e1c>] trace_hardirqs_on+0x9c/0xb0
[ 329.873951] [<7810388e>] common_interrupt+0x2e/0x40
[ 329.878891] [<78100c55>] cpu_idle+0x35/0x60
[ 329.883138] [<78a14b35>] start_kernel+0x265/0x300
[ 329.887904] [<78a14380>] unknown_bootoption+0x0/0x1e0
[ 329.893016] =======================
[ 329.896570] Code: 05 c1 e1 0c 03 4a 08 8b 52 04 01 ca 89 54 24 0c 8b 3b 89 fa 29 c2 89 d0 c1 f8 05 c1 e0 0c 03 43 08 39 44 24 0c 0f 84 7e 00 00 00 <8b> 46 10 8d 56 10 a8 01 75 4c 89 3e 89 6e 0c 8b 43 08 89 46 04
[ 329.915375] EIP: [<78406669>] blk_rq_map_sg+0xb9/0x170 SS:ESP 0068:78a13db0
[ 329.922309] Kernel panic - not syncing: Fatal exception in interrupt
Ingo
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> * Ingo Molnar <[email protected]> wrote:
>
> > i tried another bootup with the same kernel and the crash did not
> > reoccur, so it seems to be spurious. This crash could be related to
> > the scsi or block merges done in the past few days - never saw this
> > before.
>
> managed to trigger it a second time, so it seems rather reproducible:
Can you pull
git://git.kernel.dk/linux-2.6-block.git for-linus
and see if it still reproduces?
--
Jens Axboe
On Wed, 17 Oct 2007, Ingo Molnar wrote:
>
> Jens, just got this crash on a testbox:
The code in question is:
mov %edx,0xc(%esp)
mov (%ebx),%edi
mov %edi,%edx
sub %eax,%edx
mov %edx,%eax
sar $0x5,%eax
shl $0xc,%eax
add 0x8(%ebx),%eax
cmp %eax,0xc(%esp)
je +126
mov 0x10(%esi),%eax <----- Oops
lea 0x10(%esi),%edx
test $0x1,%al
jne +76
mov %edi,(%esi)
mov %ebp,0xc(%esi)
mov 0x8(%ebx),%eax
mov %eax,0x4(%esi)
and it looks like %esi is overflowing from one page to the next one, ie:
BUG: unable to handle kernel paging request at virtual address 7ca76000
ESI: 7ca75ff0
and you caught this thanks to page-alloc debugging again.
I think I can match that up with the source code: that's "sg_next()". It's
doing:
sg++;
if (unlikely(sg_is_chain(sg)))
sg = sg_chain_ptr(sg);
return sg;
and the oopsing instruction is that load of "sg->page" in the assembly
code:
mov 0x10(%esi),%eax # %eax = sg->page
lea 0x10(%esi),%edx # %edx = sg+1;
test $0x1,%al # if (unlikely(sg_is_chain()))
jne +76
Jens?
Linus
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Ingo Molnar wrote:
> >
> > Jens, just got this crash on a testbox:
>
> The code in question is:
>
> mov %edx,0xc(%esp)
> mov (%ebx),%edi
> mov %edi,%edx
> sub %eax,%edx
> mov %edx,%eax
> sar $0x5,%eax
> shl $0xc,%eax
> add 0x8(%ebx),%eax
> cmp %eax,0xc(%esp)
> je +126
> mov 0x10(%esi),%eax <----- Oops
> lea 0x10(%esi),%edx
> test $0x1,%al
> jne +76
> mov %edi,(%esi)
> mov %ebp,0xc(%esi)
> mov 0x8(%ebx),%eax
> mov %eax,0x4(%esi)
>
>
> and it looks like %esi is overflowing from one page to the next one, ie:
>
> BUG: unable to handle kernel paging request at virtual address 7ca76000
> ESI: 7ca75ff0
>
> and you caught this thanks to page-alloc debugging again.
>
> I think I can match that up with the source code: that's "sg_next()". It's
> doing:
>
> sg++;
>
> if (unlikely(sg_is_chain(sg)))
> sg = sg_chain_ptr(sg);
>
> return sg;
>
> and the oopsing instruction is that load of "sg->page" in the assembly
> code:
>
> mov 0x10(%esi),%eax # %eax = sg->page
> lea 0x10(%esi),%edx # %edx = sg+1;
> test $0x1,%al # if (unlikely(sg_is_chain()))
> jne +76
>
> Jens?
Yep, that's what I came up with as well - I asked Ingo for a dump in
private, but ended up just using ksymoops to decode the line.
The way blk_rq_map_sg() operates is that it ends up doing a
next_sg = sg_next(sg);
even though sg may be the last entry. Perhaps this is crapping out,
although if sg is a valid address, then sg + 1 should be as well.
next_sg may end up being crap, in fact it will, but we'll never use that
unless there are more entries to fill. And if there is, then both sg and
next_sg were valid.
So nothing in for-linus should fix it, I'll try and come up with an
alternate way to assign next_sg so it's always valid.
--
Jens Axboe
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Linus Torvalds wrote:
> >
> >
> > On Wed, 17 Oct 2007, Ingo Molnar wrote:
> > >
> > > Jens, just got this crash on a testbox:
> >
> > The code in question is:
> >
> > mov %edx,0xc(%esp)
> > mov (%ebx),%edi
> > mov %edi,%edx
> > sub %eax,%edx
> > mov %edx,%eax
> > sar $0x5,%eax
> > shl $0xc,%eax
> > add 0x8(%ebx),%eax
> > cmp %eax,0xc(%esp)
> > je +126
> > mov 0x10(%esi),%eax <----- Oops
> > lea 0x10(%esi),%edx
> > test $0x1,%al
> > jne +76
> > mov %edi,(%esi)
> > mov %ebp,0xc(%esi)
> > mov 0x8(%ebx),%eax
> > mov %eax,0x4(%esi)
> >
> >
> > and it looks like %esi is overflowing from one page to the next one, ie:
> >
> > BUG: unable to handle kernel paging request at virtual address 7ca76000
> > ESI: 7ca75ff0
> >
> > and you caught this thanks to page-alloc debugging again.
> >
> > I think I can match that up with the source code: that's "sg_next()". It's
> > doing:
> >
> > sg++;
> >
> > if (unlikely(sg_is_chain(sg)))
> > sg = sg_chain_ptr(sg);
> >
> > return sg;
> >
> > and the oopsing instruction is that load of "sg->page" in the assembly
> > code:
> >
> > mov 0x10(%esi),%eax # %eax = sg->page
> > lea 0x10(%esi),%edx # %edx = sg+1;
> > test $0x1,%al # if (unlikely(sg_is_chain()))
> > jne +76
> >
> > Jens?
>
> Yep, that's what I came up with as well - I asked Ingo for a dump in
> private, but ended up just using ksymoops to decode the line.
>
> The way blk_rq_map_sg() operates is that it ends up doing a
>
> next_sg = sg_next(sg);
>
> even though sg may be the last entry. Perhaps this is crapping out,
> although if sg is a valid address, then sg + 1 should be as well.
> next_sg may end up being crap, in fact it will, but we'll never use that
> unless there are more entries to fill. And if there is, then both sg and
> next_sg were valid.
>
> So nothing in for-linus should fix it, I'll try and come up with an
> alternate way to assign next_sg so it's always valid.
OK, the below should actually be safe, I don't know why I talked myself
into the next_sg stuff in the beginning. It's always safe to zero sg,
since it's a valid entry - nothing to save in ->page. Ingo, does this
work for you?
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 9e3f3cc..3935469 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
struct scatterlist *sglist)
{
struct bio_vec *bvec, *bvprv;
- struct scatterlist *next_sg, *sg;
struct req_iterator iter;
+ struct scatterlist *sg;
int nsegs, cluster;
nsegs = 0;
@@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
* for each bio in rq
*/
bvprv = NULL;
- sg = next_sg = &sglist[0];
+ sg = NULL;
rq_for_each_segment(bvec, rq, iter) {
int nbytes = bvec->bv_len;
@@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
sg->length += nbytes;
} else {
new_segment:
- sg = next_sg;
- next_sg = sg_next(sg);
+ if (!sg)
+ sg = sglist;
+ else
+ sg = sg_next(sg);
memset(sg, 0, sizeof(*sg));
sg->page = bvec->bv_page;
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> > and the oopsing instruction is that load of "sg->page" in the assembly
> > code:
> >
> > mov 0x10(%esi),%eax # %eax = sg->page
> > lea 0x10(%esi),%edx # %edx = sg+1;
> > test $0x1,%al # if (unlikely(sg_is_chain()))
> > jne +76
> >
> > Jens?
>
> Yep, that's what I came up with as well - I asked Ingo for a dump in
> private, but ended up just using ksymoops to decode the line.
>
> The way blk_rq_map_sg() operates is that it ends up doing a
>
> next_sg = sg_next(sg);
>
> even though sg may be the last entry. Perhaps this is crapping out,
> although if sg is a valid address, then sg + 1 should be as well.
> next_sg may end up being crap, in fact it will, but we'll never use
> that unless there are more entries to fill. And if there is, then both
> sg and next_sg were valid.
find below the disassembled code. Here's the faulting source line
according to gdb:
(gdb) list *0x78406669
0x78406669 is in blk_rq_map_sg (include/linux/scatterlist.h:48).
43 */
44 static inline struct scatterlist *sg_next(struct scatterlist *sg)
45 {
46 sg++;
47
48 if (unlikely(sg_is_chain(sg)))
49 sg = sg_chain_ptr(sg);
50
51 return sg;
52 }
(gdb) list *0x78406673
0x78406673 is in blk_rq_map_sg (block/ll_rw_blk.c:1355).
1350 } else {
1351 new_segment:
1352 sg = next_sg;
1353 next_sg = sg_next(sg);
1354
1355 sg->page = bvec->bv_page;
1356 sg->length = nbytes;
1357 sg->offset = bvec->bv_offset;
1358 nsegs++;
1359 }
(gdb)
the compiler is gcc-4.2.2. (vanilla, built from sources)
Ingo
784065b0 <blk_rq_map_sg>:
784065b0: 55 push %ebp
784065b1: 57 push %edi
784065b2: 56 push %esi
784065b3: 53 push %ebx
784065b4: 83 ec 28 sub $0x28,%esp
784065b7: 89 44 24 04 mov %eax,0x4(%esp)
784065bb: 8b 98 08 01 00 00 mov 0x108(%eax),%ebx
784065c1: 83 e3 01 and $0x1,%ebx
784065c4: 89 5c 24 18 mov %ebx,0x18(%esp)
784065c8: 8b 52 3c mov 0x3c(%edx),%edx
784065cb: c7 44 24 14 00 00 00 movl $0x0,0x14(%esp)
784065d2: 00
784065d3: 85 d2 test %edx,%edx
784065d5: 89 54 24 20 mov %edx,0x20(%esp)
784065d9: 0f 84 fc 00 00 00 je 784066db <blk_rq_map_sg+0x12b>
784065df: 89 ce mov %ecx,%esi
784065e1: 31 d2 xor %edx,%edx
784065e3: 89 4c 24 10 mov %ecx,0x10(%esp)
784065e7: 8b 44 24 20 mov 0x20(%esp),%eax
784065eb: 0f b7 58 1a movzwl 0x1a(%eax),%ebx
784065ef: 8b 48 30 mov 0x30(%eax),%ecx
784065f2: 89 5c 24 1c mov %ebx,0x1c(%esp)
784065f6: 0f b7 40 18 movzwl 0x18(%eax),%eax
784065fa: 39 d8 cmp %ebx,%eax
784065fc: 0f 8e c6 00 00 00 jle 784066c8 <blk_rq_map_sg+0x118>
78406602: 8d 04 5b lea (%ebx,%ebx,2),%eax
78406605: 8d 1c 81 lea (%ecx,%eax,4),%ebx
78406608: 0f b6 44 24 18 movzbl 0x18(%esp),%eax
7840660d: 88 44 24 27 mov %al,0x27(%esp)
78406611: e9 8b 00 00 00 jmp 784066a1 <blk_rq_map_sg+0xf1>
78406616: 8b 4c 24 10 mov 0x10(%esp),%ecx
7840661a: 8b 41 0c mov 0xc(%ecx),%eax
7840661d: 8b 4c 24 04 mov 0x4(%esp),%ecx
78406621: 01 e8 add %ebp,%eax
78406623: 89 44 24 08 mov %eax,0x8(%esp)
78406627: 3b 81 6c 01 00 00 cmp 0x16c(%ecx),%eax
7840662d: 0f 87 80 00 00 00 ja 784066b3 <blk_rq_map_sg+0x103>
78406633: a1 18 ec d7 78 mov 0x78d7ec18,%eax
78406638: 8b 0a mov (%edx),%ecx
7840663a: 29 c1 sub %eax,%ecx
7840663c: c1 f9 05 sar $0x5,%ecx
7840663f: c1 e1 0c shl $0xc,%ecx
78406642: 03 4a 08 add 0x8(%edx),%ecx
78406645: 8b 52 04 mov 0x4(%edx),%edx
78406648: 01 ca add %ecx,%edx
7840664a: 89 54 24 0c mov %edx,0xc(%esp)
7840664e: 8b 3b mov (%ebx),%edi
78406650: 89 fa mov %edi,%edx
78406652: 29 c2 sub %eax,%edx
78406654: 89 d0 mov %edx,%eax
78406656: c1 f8 05 sar $0x5,%eax
78406659: c1 e0 0c shl $0xc,%eax
7840665c: 03 43 08 add 0x8(%ebx),%eax
7840665f: 39 44 24 0c cmp %eax,0xc(%esp)
78406663: 0f 84 7e 00 00 00 je 784066e7 <blk_rq_map_sg+0x137>
78406669: 8b 46 10 mov 0x10(%esi),%eax
7840666c: 8d 56 10 lea 0x10(%esi),%edx
7840666f: a8 01 test $0x1,%al
78406671: 75 4c jne 784066bf <blk_rq_map_sg+0x10f>
78406673: 89 3e mov %edi,(%esi)
78406675: 89 6e 0c mov %ebp,0xc(%esi)
78406678: 8b 43 08 mov 0x8(%ebx),%eax
7840667b: 89 46 04 mov %eax,0x4(%esi)
7840667e: 83 44 24 14 01 addl $0x1,0x14(%esp)
78406683: 89 74 24 10 mov %esi,0x10(%esp)
78406687: 89 d6 mov %edx,%esi
78406689: 8b 54 24 20 mov 0x20(%esp),%edx
7840668d: 83 44 24 1c 01 addl $0x1,0x1c(%esp)
78406692: 0f b7 42 18 movzwl 0x18(%edx),%eax
78406696: 3b 44 24 1c cmp 0x1c(%esp),%eax
7840669a: 7e 2a jle 784066c6 <blk_rq_map_sg+0x116>
7840669c: 89 da mov %ebx,%edx
7840669e: 83 c3 0c add $0xc,%ebx
784066a1: 85 d2 test %edx,%edx
784066a3: 8b 6b 04 mov 0x4(%ebx),%ebp
784066a6: 74 0b je 784066b3 <blk_rq_map_sg+0x103>
784066a8: 80 7c 24 27 00 cmpb $0x0,0x27(%esp)
784066ad: 0f 85 63 ff ff ff jne 78406616 <blk_rq_map_sg+0x66>
784066b3: 8b 46 10 mov 0x10(%esi),%eax
784066b6: 8d 56 10 lea 0x10(%esi),%edx
784066b9: 8b 3b mov (%ebx),%edi
784066bb: a8 01 test $0x1,%al
784066bd: 74 b4 je 78406673 <blk_rq_map_sg+0xc3>
784066bf: 89 c2 mov %eax,%edx
784066c1: 83 e2 fe and $0xfffffffe,%edx
784066c4: eb ad jmp 78406673 <blk_rq_map_sg+0xc3>
784066c6: 89 da mov %ebx,%edx
784066c8: 8b 4c 24 20 mov 0x20(%esp),%ecx
784066cc: 8b 49 08 mov 0x8(%ecx),%ecx
784066cf: 85 c9 test %ecx,%ecx
784066d1: 89 4c 24 20 mov %ecx,0x20(%esp)
784066d5: 0f 85 0c ff ff ff jne 784065e7 <blk_rq_map_sg+0x37>
784066db: 8b 44 24 14 mov 0x14(%esp),%eax
784066df: 83 c4 28 add $0x28,%esp
784066e2: 5b pop %ebx
784066e3: 5e pop %esi
784066e4: 5f pop %edi
784066e5: 5d pop %ebp
784066e6: c3 ret
784066e7: 8b 44 24 04 mov 0x4(%esp),%eax
784066eb: 8b 54 24 0c mov 0xc(%esp),%edx
784066ef: 8b 80 70 01 00 00 mov 0x170(%eax),%eax
784066f5: 89 04 24 mov %eax,(%esp)
784066f8: 09 c1 or %eax,%ecx
784066fa: 8d 44 2a ff lea 0xffffffff(%edx,%ebp,1),%eax
784066fe: 0b 04 24 or (%esp),%eax
78406701: 39 c1 cmp %eax,%ecx
78406703: 0f 85 60 ff ff ff jne 78406669 <blk_rq_map_sg+0xb9>
78406709: 8b 44 24 08 mov 0x8(%esp),%eax
7840670d: 8b 4c 24 10 mov 0x10(%esp),%ecx
78406711: 89 41 0c mov %eax,0xc(%ecx)
78406714: e9 70 ff ff ff jmp 78406689 <blk_rq_map_sg+0xd9>
78406719: 8d b4 26 00 00 00 00 lea 0x0(%esi),%esi
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Jens Axboe wrote:
> > On Wed, Oct 17 2007, Linus Torvalds wrote:
> > >
> > >
> > > On Wed, 17 Oct 2007, Ingo Molnar wrote:
> > > >
> > > > Jens, just got this crash on a testbox:
> > >
> > > The code in question is:
> > >
> > > mov %edx,0xc(%esp)
> > > mov (%ebx),%edi
> > > mov %edi,%edx
> > > sub %eax,%edx
> > > mov %edx,%eax
> > > sar $0x5,%eax
> > > shl $0xc,%eax
> > > add 0x8(%ebx),%eax
> > > cmp %eax,0xc(%esp)
> > > je +126
> > > mov 0x10(%esi),%eax <----- Oops
> > > lea 0x10(%esi),%edx
> > > test $0x1,%al
> > > jne +76
> > > mov %edi,(%esi)
> > > mov %ebp,0xc(%esi)
> > > mov 0x8(%ebx),%eax
> > > mov %eax,0x4(%esi)
> > >
> > >
> > > and it looks like %esi is overflowing from one page to the next one, ie:
> > >
> > > BUG: unable to handle kernel paging request at virtual address 7ca76000
> > > ESI: 7ca75ff0
> > >
> > > and you caught this thanks to page-alloc debugging again.
> > >
> > > I think I can match that up with the source code: that's "sg_next()". It's
> > > doing:
> > >
> > > sg++;
> > >
> > > if (unlikely(sg_is_chain(sg)))
> > > sg = sg_chain_ptr(sg);
> > >
> > > return sg;
> > >
> > > and the oopsing instruction is that load of "sg->page" in the assembly
> > > code:
> > >
> > > mov 0x10(%esi),%eax # %eax = sg->page
> > > lea 0x10(%esi),%edx # %edx = sg+1;
> > > test $0x1,%al # if (unlikely(sg_is_chain()))
> > > jne +76
> > >
> > > Jens?
> >
> > Yep, that's what I came up with as well - I asked Ingo for a dump in
> > private, but ended up just using ksymoops to decode the line.
> >
> > The way blk_rq_map_sg() operates is that it ends up doing a
> >
> > next_sg = sg_next(sg);
> >
> > even though sg may be the last entry. Perhaps this is crapping out,
> > although if sg is a valid address, then sg + 1 should be as well.
> > next_sg may end up being crap, in fact it will, but we'll never use that
> > unless there are more entries to fill. And if there is, then both sg and
> > next_sg were valid.
> >
> > So nothing in for-linus should fix it, I'll try and come up with an
> > alternate way to assign next_sg so it's always valid.
>
> OK, the below should actually be safe, I don't know why I talked myself
> into the next_sg stuff in the beginning. It's always safe to zero sg,
> since it's a valid entry - nothing to save in ->page. Ingo, does this
> work for you?
>
> diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
> index 9e3f3cc..3935469 100644
> --- a/block/ll_rw_blk.c
> +++ b/block/ll_rw_blk.c
> @@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> struct scatterlist *sglist)
> {
> struct bio_vec *bvec, *bvprv;
> - struct scatterlist *next_sg, *sg;
> struct req_iterator iter;
> + struct scatterlist *sg;
> int nsegs, cluster;
>
> nsegs = 0;
> @@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> * for each bio in rq
> */
> bvprv = NULL;
> - sg = next_sg = &sglist[0];
> + sg = NULL;
> rq_for_each_segment(bvec, rq, iter) {
> int nbytes = bvec->bv_len;
>
> @@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> sg->length += nbytes;
> } else {
> new_segment:
> - sg = next_sg;
> - next_sg = sg_next(sg);
> + if (!sg)
> + sg = sglist;
> + else
> + sg = sg_next(sg);
>
> memset(sg, 0, sizeof(*sg));
> sg->page = bvec->bv_page;
>
Scratch that, it cannot work... I'll think up a different approach.
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> [...] It's always safe to zero sg, since it's a valid entry - nothing
> to save in ->page. Ingo, does this work for you?
with that patch it not crashes on NULL dereference - see crashlog below.
Compiler bug perhaps?
Ingo
---------------->
[ 34.605614] EXT3-fs: INFO: recovery required on readonly filesystem.
[ 34.611842] EXT3-fs: write access will be enabled during recovery.
[ 34.635861] BUG: unable to handle kernel NULL pointer dereference at virtual address 00000008
[ 34.644227] printing eip: 7840840e *pde = 00000000
[ 34.649081] Oops: 0002 [#1] DEBUG_PAGEALLOC
[ 34.653239]
[ 34.654713] Pid: 1, comm: swapper Not tainted (2.6.23 #3)
[ 34.660086] EIP: 0060:[<7840840e>] EFLAGS: 00010046 CPU: 0
[ 34.665548] EIP is at blk_rq_map_sg+0x8e/0x190
[ 34.669965] EAX: 00000000 EBX: 7c885180 ECX: 00000004 EDX: 033b6000
[ 34.676205] ESI: 00001000 EDI: 00000008 EBP: 00000000 ESP: 7b4219b8
[ 34.682444] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 34.687818] Process swapper (pid: 1, ti=7b420000 task=7b416000 task.ti=7b420000)
[ 34.695010] Stack: 7b521d38 00000008 7c884000 7b520000 00002000 033b6000 7c885080 00000001
[ 34.703329] 00000001 7c885880 01000002 7c886e00 7b521d1c 7c8857c4 7b520000 784c7ae5
[ 34.711649] 7b520000 784c75ea 7c8857c4 7b524ce4 7b521d1c 7c8857c4 784e4590 7b416000
[ 34.719968] Call Trace:
[ 34.722568] [<784c7ae5>] scsi_init_io+0x55/0xe0
[ 34.727161] [<784c75ea>] scsi_get_cmd_from_req+0x2a/0x40
[ 34.732534] [<784e4590>] sd_prep_fn+0x80/0x940
[ 34.737041] [<7813661b>] __lock_acquire+0x4ab/0xe20
[ 34.741981] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 34.747007] [<78770e30>] _spin_unlock_irq+0x20/0x30
[ 34.751946] [<78404633>] elv_dispatch_sort+0x23/0xe0
[ 34.756973] [<784041a0>] elv_next_request+0xa0/0x130
[ 34.761999] [<784c8c24>] scsi_request_fn+0x1e4/0x370
[ 34.767025] [<78120f02>] del_timer+0x62/0x70
[ 34.771358] [<784072d5>] __generic_unplug_device+0x25/0x30
[ 34.776905] [<784075a5>] generic_unplug_device+0x15/0x30
[ 34.782278] [<78404e0c>] blk_backing_dev_unplug+0xc/0x10
[ 34.787650] [<78181e76>] sync_buffer+0x26/0x40
[ 34.792157] [<7876f6c2>] __wait_on_bit+0x42/0x70
[ 34.796836] [<78181e50>] sync_buffer+0x0/0x40
[ 34.801256] [<78181e50>] sync_buffer+0x0/0x40
[ 34.805676] [<7876f74a>] out_of_line_wait_on_bit+0x5a/0x70
[ 34.811223] [<7812adb0>] wake_bit_function+0x0/0x60
[ 34.816163] [<78181db4>] __wait_on_buffer+0x24/0x30
[ 34.821102] [<78207a27>] jread+0x1b7/0x250
[ 34.825262] [<78207bcd>] do_one_pass+0x10d/0x600
[ 34.829942] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 34.834968] [<7820823b>] journal_recover+0x9b/0x1a0
[ 34.839908] [<7820b811>] journal_load+0x51/0xf0
[ 34.844501] [<781e6914>] ext3_fill_super+0xdd4/0x1850
[ 34.849613] [<78411cef>] snprintf+0x1f/0x30
[ 34.853860] [<7819ce70>] disk_name+0xb0/0xc0
[ 34.858193] [<78163809>] get_sb_bdev+0x109/0x130
[ 34.862873] [<7815f300>] __kmalloc_node+0x80/0x90
[ 34.867639] [<781e3a80>] ext3_get_sb+0x20/0x30
[ 34.872146] [<781e5b40>] ext3_fill_super+0x0/0x1850
[ 34.877085] [<78163315>] vfs_kern_mount+0xb5/0x130
[ 34.881938] [<781633ed>] do_kern_mount+0x3d/0xe0
[ 34.886618] [<78177c57>] do_mount+0x5e7/0x710
[ 34.891038] [<78770fe5>] _spin_unlock_irqrestore+0x55/0x70
[ 34.896584] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 34.901610] [<78110935>] change_page_attr+0x3d5/0x400
[ 34.906724] [<7811099a>] kernel_map_pages+0x3a/0x90
[ 34.911663] [<7814972f>] get_page_from_freelist+0x1ff/0x3e0
[ 34.917296] [<7814a2cf>] __alloc_pages+0x5f/0x380
[ 34.922063] [<7814a61e>] __get_free_pages+0x2e/0x50
[ 34.927002] [<78176660>] copy_mount_options+0x40/0x140
[ 34.932202] [<78177df2>] sys_mount+0x72/0xb0
[ 34.936535] [<78a14d39>] mount_block_root+0x89/0x260
[ 34.941561] [<7816b4c7>] sys_mknod+0x27/0x30
[ 34.945894] [<78a14f56>] mount_root+0x46/0x60
[ 34.950314] [<78a1501c>] prepare_namespace+0xac/0x170
[ 34.955427] [<7816000f>] sys_access+0x1f/0x30
[ 34.959846] [<78a147ae>] kernel_init+0x15e/0x280
[ 34.964526] [<78a14650>] kernel_init+0x0/0x280
[ 34.969033] [<78103a97>] kernel_thread_helper+0x7/0x10
[ 34.974233] =======================
[ 34.977785] Code: 44 24 20 88 44 24 2b eb 54 8d 74 26 00 83 44 24 04 10 8b 7c 24 04 8b 07 a8 01 0f 85 fb 00 00 00 31 c0 b9 04 00 00 00 8b 7c 24 04 <f3> ab 8b 03 8b 54 24 04 89 02 89 72 0c 8b 43 08 89 42 04 83 44
[ 34.996590] EIP: [<7840840e>] blk_rq_map_sg+0x8e/0x190 SS:ESP 0068:7b4219b8
[ 35.003525] Kernel panic - not syncing: Attempted to kill init!
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Jens Axboe wrote:
> > On Wed, Oct 17 2007, Jens Axboe wrote:
> > > On Wed, Oct 17 2007, Linus Torvalds wrote:
> > > >
> > > >
> > > > On Wed, 17 Oct 2007, Ingo Molnar wrote:
> > > > >
> > > > > Jens, just got this crash on a testbox:
> > > >
> > > > The code in question is:
> > > >
> > > > mov %edx,0xc(%esp)
> > > > mov (%ebx),%edi
> > > > mov %edi,%edx
> > > > sub %eax,%edx
> > > > mov %edx,%eax
> > > > sar $0x5,%eax
> > > > shl $0xc,%eax
> > > > add 0x8(%ebx),%eax
> > > > cmp %eax,0xc(%esp)
> > > > je +126
> > > > mov 0x10(%esi),%eax <----- Oops
> > > > lea 0x10(%esi),%edx
> > > > test $0x1,%al
> > > > jne +76
> > > > mov %edi,(%esi)
> > > > mov %ebp,0xc(%esi)
> > > > mov 0x8(%ebx),%eax
> > > > mov %eax,0x4(%esi)
> > > >
> > > >
> > > > and it looks like %esi is overflowing from one page to the next one, ie:
> > > >
> > > > BUG: unable to handle kernel paging request at virtual address 7ca76000
> > > > ESI: 7ca75ff0
> > > >
> > > > and you caught this thanks to page-alloc debugging again.
> > > >
> > > > I think I can match that up with the source code: that's "sg_next()". It's
> > > > doing:
> > > >
> > > > sg++;
> > > >
> > > > if (unlikely(sg_is_chain(sg)))
> > > > sg = sg_chain_ptr(sg);
> > > >
> > > > return sg;
> > > >
> > > > and the oopsing instruction is that load of "sg->page" in the assembly
> > > > code:
> > > >
> > > > mov 0x10(%esi),%eax # %eax = sg->page
> > > > lea 0x10(%esi),%edx # %edx = sg+1;
> > > > test $0x1,%al # if (unlikely(sg_is_chain()))
> > > > jne +76
> > > >
> > > > Jens?
> > >
> > > Yep, that's what I came up with as well - I asked Ingo for a dump in
> > > private, but ended up just using ksymoops to decode the line.
> > >
> > > The way blk_rq_map_sg() operates is that it ends up doing a
> > >
> > > next_sg = sg_next(sg);
> > >
> > > even though sg may be the last entry. Perhaps this is crapping out,
> > > although if sg is a valid address, then sg + 1 should be as well.
> > > next_sg may end up being crap, in fact it will, but we'll never use that
> > > unless there are more entries to fill. And if there is, then both sg and
> > > next_sg were valid.
> > >
> > > So nothing in for-linus should fix it, I'll try and come up with an
> > > alternate way to assign next_sg so it's always valid.
> >
> > OK, the below should actually be safe, I don't know why I talked myself
> > into the next_sg stuff in the beginning. It's always safe to zero sg,
> > since it's a valid entry - nothing to save in ->page. Ingo, does this
> > work for you?
> >
> > diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
> > index 9e3f3cc..3935469 100644
> > --- a/block/ll_rw_blk.c
> > +++ b/block/ll_rw_blk.c
> > @@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> > struct scatterlist *sglist)
> > {
> > struct bio_vec *bvec, *bvprv;
> > - struct scatterlist *next_sg, *sg;
> > struct req_iterator iter;
> > + struct scatterlist *sg;
> > int nsegs, cluster;
> >
> > nsegs = 0;
> > @@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> > * for each bio in rq
> > */
> > bvprv = NULL;
> > - sg = next_sg = &sglist[0];
> > + sg = NULL;
> > rq_for_each_segment(bvec, rq, iter) {
> > int nbytes = bvec->bv_len;
> >
> > @@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> > sg->length += nbytes;
> > } else {
> > new_segment:
> > - sg = next_sg;
> > - next_sg = sg_next(sg);
> > + if (!sg)
> > + sg = sglist;
> > + else
> > + sg = sg_next(sg);
> >
> > memset(sg, 0, sizeof(*sg));
> > sg->page = bvec->bv_page;
> >
>
> Scratch that, it cannot work... I'll think up a different approach.
OK, it is fine, as long as the sglist is cleared initially. And I don't
think there's anyway around that, clearly I didn't think long enough
before including the memset() removal from Tomo.
Ingo, please try this rolled up version.
Linus, this should work. It would probably be best if you first did a
git revert on f5c0dde4c66421a3a2d7d6fa604a712c9b0744e5 and then applied
the ll_rw_blk.c bit alone. Do you want me to stuff that (revert + patch)
into a branch for you to pull?
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 9e3f3cc..3935469 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
struct scatterlist *sglist)
{
struct bio_vec *bvec, *bvprv;
- struct scatterlist *next_sg, *sg;
struct req_iterator iter;
+ struct scatterlist *sg;
int nsegs, cluster;
nsegs = 0;
@@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
* for each bio in rq
*/
bvprv = NULL;
- sg = next_sg = &sglist[0];
+ sg = NULL;
rq_for_each_segment(bvec, rq, iter) {
int nbytes = bvec->bv_len;
@@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
sg->length += nbytes;
} else {
new_segment:
- sg = next_sg;
- next_sg = sg_next(sg);
+ if (!sg)
+ sg = sglist;
+ else
+ sg = sg_next(sg);
memset(sg, 0, sizeof(*sg));
sg->page = bvec->bv_page;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 0c86be7..aac8a02 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -764,6 +764,8 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
+ memset(sgl, 0, sizeof(*sgl) * sgp->size);
+
/*
* first loop through, set initial index and return value
*/
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> > - sg = next_sg;
> > - next_sg = sg_next(sg);
> > + if (!sg)
> > + sg = sglist;
> > + else
> > + sg = sg_next(sg);
> >
> > memset(sg, 0, sizeof(*sg));
> > sg->page = bvec->bv_page;
> >
>
> Scratch that, it cannot work... I'll think up a different approach.
too late, crashed my box with it already :-)
Ingo
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > > - sg = next_sg;
> > > - next_sg = sg_next(sg);
> > > + if (!sg)
> > > + sg = sglist;
> > > + else
> > > + sg = sg_next(sg);
> > >
> > > memset(sg, 0, sizeof(*sg));
> > > sg->page = bvec->bv_page;
> > >
> >
> > Scratch that, it cannot work... I'll think up a different approach.
>
> too late, crashed my box with it already :-)
Sorry about that, please try the next one that includes the scsi_lib.c
one liner to clear the sg table on alloc :-)
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> OK, it is fine, as long as the sglist is cleared initially. And I
> don't think there's anyway around that, clearly I didn't think long
> enough before including the memset() removal from Tomo.
>
> Ingo, please try this rolled up version.
>
> Linus, this should work. It would probably be best if you first did a
> git revert on f5c0dde4c66421a3a2d7d6fa604a712c9b0744e5 and then
> applied the ll_rw_blk.c bit alone. Do you want me to stuff that
> (revert + patch) into a branch for you to pull?
yep, this one did the trick, it booted up fine twice in a row already!
Tested-by: Ingo Molnar <[email protected]>
thanks!
Ingo
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > OK, it is fine, as long as the sglist is cleared initially. And I
> > don't think there's anyway around that, clearly I didn't think long
> > enough before including the memset() removal from Tomo.
> >
> > Ingo, please try this rolled up version.
> >
> > Linus, this should work. It would probably be best if you first did a
> > git revert on f5c0dde4c66421a3a2d7d6fa604a712c9b0744e5 and then
> > applied the ll_rw_blk.c bit alone. Do you want me to stuff that
> > (revert + patch) into a branch for you to pull?
>
> yep, this one did the trick, it booted up fine twice in a row already!
>
> Tested-by: Ingo Molnar <[email protected]>
>
> thanks!
Great! Thanks a lot for reporting and testing... Linus, care to pull
git://git.kernel.dk/data/git/linux-2.6-block.git for-linus
Jens Axboe (2):
Revert "[SCSI] Remove full sg table memset()"
[BLOCK] blk_rq_map_sg() next_sg fixup
block/ll_rw_blk.c | 10 ++++++----
drivers/scsi/scsi_lib.c | 2 ++
2 files changed, 8 insertions(+), 4 deletions(-)
--
Jens Axboe
ok, here's a different but similar crash that triggers on the testbox:
[ 233.438890] BUG: unable to handle kernel paging request at virtual address 7d93e000
[ 233.446390] printing eip: 784e9480 *pde = 01000067 *pte = 0593e000
[ 233.452630] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 233.456790]
[ 233.458264] Pid: 0, comm: swapper Not tainted (2.6.23 #5)
[ 233.463637] EIP: 0060:[<784e9480>] EFLAGS: 00010087 CPU: 0
[ 233.469101] EIP is at ata_qc_issue+0x90/0x380
[ 233.473429] EAX: 7d93dff0 EBX: 0000001f ECX: 7d93dff0 EDX: 798daf80
[ 233.479668] ESI: 00000020 EDI: 7d93de00 EBP: 7b54007c ESP: 78a13e14
[ 233.485908] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 233.491282] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
[ 233.498473] Stack: 7d93de00 7b540000 7b540000 00000000 7d93dfe0 7b54007c 7d93db00 7b5417a4
[ 233.506793] 784c2490 784ef69e 784f21f3 7b52de98 7d93db00 7b540000 7b5417a4 7d93db00
[ 233.515112] 7b540000 7b524004 784f22e0 784ef380 784c2490 7d93db00 00000202 7b524004
[ 233.523432] Call Trace:
[ 233.526033] [<784c2490>] scsi_done+0x0/0x20
[ 233.530279] [<784ef69e>] ata_scsi_translate+0xbe/0x140
[ 233.535478] [<784f21f3>] ata_scsi_queuecmd+0x33/0x200
[ 233.540591] [<784f22e0>] ata_scsi_queuecmd+0x120/0x200
[ 233.545791] [<784ef380>] ata_scsi_rw_xlat+0x0/0x220
[ 233.550730] [<784c2490>] scsi_done+0x0/0x20
[ 233.554976] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
[ 233.560177] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 233.565202] [<784c8c7e>] scsi_request_fn+0x1be/0x370
[ 233.570229] [<78408086>] blk_run_queue+0x36/0x80
[ 233.574909] [<784c7520>] scsi_next_command+0x30/0x50
[ 233.579935] [<784c76ab>] scsi_end_request+0xab/0xe0
[ 233.584875] [<784c83f9>] scsi_io_completion+0xa9/0x3d0
[ 233.590075] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 233.595100] [<78405125>] blk_done_softirq+0x45/0x80
[ 233.600040] [<78405153>] blk_done_softirq+0x73/0x80
[ 233.604981] [<7811d4c3>] __do_softirq+0x53/0xb0
[ 233.609573] [<7811d588>] do_softirq+0x68/0x70
[ 233.613993] [<78105351>] do_IRQ+0x51/0x90
[ 233.618066] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 233.623092] [<7810f2d0>] pgd_dtor+0x0/0x50
[ 233.627252] [<7810388e>] common_interrupt+0x2e/0x40
[ 233.632192] [<7810f2d0>] pgd_dtor+0x0/0x50
[ 233.636352] [<7815f3be>] quicklist_trim+0x5e/0x90
[ 233.641118] [<7810f2cb>] check_pgt_cache+0x1b/0x20
[ 233.645971] [<78100c52>] cpu_idle+0x32/0x60
[ 233.650217] [<78a14b35>] start_kernel+0x265/0x300
[ 233.654983] [<78a14380>] unknown_bootoption+0x0/0x1e0
[ 233.660097] =======================
[ 233.663649] Code: 00 00 00 8b 45 34 a8 02 0f 84 ed 00 00 00 8b bd 88 00 00 00 31 db 89 3c 24 8b 75 3c 89 f8 c7 44 24 10 00 00 00 00 eb 1b 8d 76 00 <8b> 50 10 8d 48 10 f6 c2 01 0f 85 be 02 00 00 89 44 24 10 83 c3
[ 233.682455] EIP: [<784e9480>] ata_qc_issue+0x90/0x380 SS:ESP 0068:78a13e14
[ 233.689302] Kernel panic - not syncing: Fatal exception in interrupt
(gdb) list *0x784e9480
0x784e9480 is in ata_qc_issue (include/linux/scatterlist.h:48).
43 */
44 static inline struct scatterlist *sg_next(struct scatterlist *sg)
45 {
46 sg++;
47
48 if (unlikely(sg_is_chain(sg)))
49 sg = sg_chain_ptr(sg);
50
51 return sg;
52 }
(gdb)
so there's sg_next() involvement too. Below is the disassembly.
Ingo
------------------------->
784e93f0 <ata_qc_issue>:
784e93f0: 55 push %ebp
784e93f1: 89 c5 mov %eax,%ebp
784e93f3: 57 push %edi
784e93f4: 56 push %esi
784e93f5: 53 push %ebx
784e93f6: 83 ec 14 sub $0x14,%esp
784e93f9: 8b 00 mov (%eax),%eax
784e93fb: 89 44 24 04 mov %eax,0x4(%esp)
784e93ff: 8b 45 04 mov 0x4(%ebp),%eax
784e9402: 80 7d 14 04 cmpb $0x4,0x14(%ebp)
784e9406: 8b 10 mov (%eax),%edx
784e9408: 0f 84 d2 01 00 00 je 784e95e0 <ata_qc_issue+0x1f0>
784e940e: 8b 5c 24 04 mov 0x4(%esp),%ebx
784e9412: 83 83 84 16 00 00 01 addl $0x1,0x1684(%ebx)
784e9419: 8b 45 38 mov 0x38(%ebp),%eax
784e941c: 89 42 08 mov %eax,0x8(%edx)
784e941f: 83 4d 34 01 orl $0x1,0x34(%ebp)
784e9423: b8 01 00 00 00 mov $0x1,%eax
784e9428: 8b 4d 38 mov 0x38(%ebp),%ecx
784e942b: 89 c7 mov %eax,%edi
784e942d: 8b 54 24 04 mov 0x4(%esp),%edx
784e9431: d3 e7 shl %cl,%edi
784e9433: 09 ba 80 16 00 00 or %edi,0x1680(%edx)
784e9439: 8b 4d 00 mov 0x0(%ebp),%ecx
784e943c: 89 4c 24 08 mov %ecx,0x8(%esp)
784e9440: 80 7d 14 07 cmpb $0x7,0x14(%ebp)
784e9444: 0f 87 c6 00 00 00 ja 784e9510 <ata_qc_issue+0x120>
784e944a: 0f be 4d 14 movsbl 0x14(%ebp),%ecx
784e944e: d3 e0 shl %cl,%eax
784e9450: a8 98 test $0x98,%al
784e9452: 0f 84 ab 00 00 00 je 784e9503 <ata_qc_issue+0x113>
784e9458: 8b 45 34 mov 0x34(%ebp),%eax
784e945b: a8 02 test $0x2,%al
784e945d: 0f 84 ed 00 00 00 je 784e9550 <ata_qc_issue+0x160>
784e9463: 8b bd 88 00 00 00 mov 0x88(%ebp),%edi
784e9469: 31 db xor %ebx,%ebx
784e946b: 89 3c 24 mov %edi,(%esp)
784e946e: 8b 75 3c mov 0x3c(%ebp),%esi
784e9471: 89 f8 mov %edi,%eax
784e9473: c7 44 24 10 00 00 00 movl $0x0,0x10(%esp)
784e947a: 00
784e947b: eb 1b jmp 784e9498 <ata_qc_issue+0xa8>
784e947d: 8d 76 00 lea 0x0(%esi),%esi
784e9480: 8b 50 10 mov 0x10(%eax),%edx
784e9483: 8d 48 10 lea 0x10(%eax),%ecx
784e9486: f6 c2 01 test $0x1,%dl
784e9489: 0f 85 be 02 00 00 jne 784e974d <ata_qc_issue+0x35d>
784e948f: 89 44 24 10 mov %eax,0x10(%esp)
784e9493: 83 c3 01 add $0x1,%ebx
784e9496: 89 c8 mov %ecx,%eax
784e9498: 39 f3 cmp %esi,%ebx
784e949a: 75 e4 jne 784e9480 <ata_qc_issue+0x90>
784e949c: 8b 54 24 10 mov 0x10(%esp),%edx
784e94a0: 8b 42 0c mov 0xc(%edx),%eax
784e94a3: 83 e0 03 and $0x3,%eax
784e94a6: 85 c0 test %eax,%eax
784e94a8: 89 45 4c mov %eax,0x4c(%ebp)
784e94ab: 0f 85 53 01 00 00 jne 784e9604 <ata_qc_issue+0x214>
784e94b1: 89 f1 mov %esi,%ecx
784e94b3: 83 f9 00 cmp $0x0,%ecx
784e94b6: 0f 84 df 01 00 00 je 784e969b <ata_qc_issue+0x2ab>
784e94bc: 7e 30 jle 784e94ee <ata_qc_issue+0xfe>
784e94be: 31 d2 xor %edx,%edx
784e94c0: 8b 1c 24 mov (%esp),%ebx
784e94c3: 83 c2 01 add $0x1,%edx
784e94c6: 8b 03 mov (%ebx),%eax
784e94c8: 2b 05 f8 ec d7 78 sub 0x78d7ecf8,%eax
784e94ce: c1 f8 05 sar $0x5,%eax
784e94d1: c1 e0 0c shl $0xc,%eax
784e94d4: 03 43 04 add 0x4(%ebx),%eax
784e94d7: 89 43 08 mov %eax,0x8(%ebx)
784e94da: 83 c3 10 add $0x10,%ebx
784e94dd: 89 1c 24 mov %ebx,(%esp)
784e94e0: 8b 03 mov (%ebx),%eax
784e94e2: a8 01 test $0x1,%al
784e94e4: 0f 85 6d 02 00 00 jne 784e9757 <ata_qc_issue+0x367>
784e94ea: 39 ca cmp %ecx,%edx
784e94ec: 75 d2 jne 784e94c0 <ata_qc_issue+0xd0>
784e94ee: f0 83 44 24 00 00 lock addl $0x0,0x0(%esp)
784e94f4: 85 c9 test %ecx,%ecx
784e94f6: 89 c8 mov %ecx,%eax
784e94f8: 0f 8e 2c 02 00 00 jle 784e972a <ata_qc_issue+0x33a>
784e94fe: 89 45 3c mov %eax,0x3c(%ebp)
784e9501: eb 11 jmp 784e9514 <ata_qc_issue+0x124>
784e9503: a8 24 test $0x24,%al
784e9505: 0f 85 b5 00 00 00 jne 784e95c0 <ata_qc_issue+0x1d0>
784e950b: 90 nop
784e950c: 8d 74 26 00 lea 0x0(%esi),%esi
784e9510: 83 65 34 f9 andl $0xfffffff9,0x34(%ebp)
784e9514: 8b 5c 24 04 mov 0x4(%esp),%ebx
784e9518: 89 e8 mov %ebp,%eax
784e951a: 8b 53 04 mov 0x4(%ebx),%edx
784e951d: ff 52 48 call *0x48(%edx)
784e9520: 8b 9d 8c 00 00 00 mov 0x8c(%ebp),%ebx
784e9526: 89 e8 mov %ebp,%eax
784e9528: 8b 7c 24 04 mov 0x4(%esp),%edi
784e952c: 8b 57 04 mov 0x4(%edi),%edx
784e952f: ff 52 4c call *0x4c(%edx)
784e9532: 09 d8 or %ebx,%eax
784e9534: 85 c0 test %eax,%eax
784e9536: 89 85 8c 00 00 00 mov %eax,0x8c(%ebp)
784e953c: 0f 85 fd 01 00 00 jne 784e973f <ata_qc_issue+0x34f>
784e9542: 83 c4 14 add $0x14,%esp
784e9545: 5b pop %ebx
784e9546: 5e pop %esi
784e9547: 5f pop %edi
784e9548: 5d pop %ebp
784e9549: c3 ret
784e954a: 8d b6 00 00 00 00 lea 0x0(%esi),%esi
784e9550: a8 04 test $0x4,%al
784e9552: 74 c0 je 784e9514 <ata_qc_issue+0x124>
784e9554: 8b 95 88 00 00 00 mov 0x88(%ebp),%edx
784e955a: 8b 42 0c mov 0xc(%edx),%eax
784e955d: 83 e0 03 and $0x3,%eax
784e9560: 85 c0 test %eax,%eax
784e9562: 89 45 4c mov %eax,0x4c(%ebp)
784e9565: 0f 84 3a 01 00 00 je 784e96a5 <ata_qc_issue+0x2b5>
784e956b: 8b 45 38 mov 0x38(%ebp),%eax
784e956e: 8d 3c 85 00 00 00 00 lea 0x0(,%eax,4),%edi
784e9575: 8b 44 24 08 mov 0x8(%esp),%eax
784e9579: 03 78 24 add 0x24(%eax),%edi
784e957c: c7 07 00 00 00 00 movl $0x0,(%edi)
784e9582: f6 45 10 08 testb $0x8,0x10(%ebp)
784e9586: 0f 85 3a 01 00 00 jne 784e96c6 <ata_qc_issue+0x2d6>
784e958c: 8b 4c 24 08 mov 0x8(%esp),%ecx
784e9590: 8b 45 38 mov 0x38(%ebp),%eax
784e9593: c1 e0 02 shl $0x2,%eax
784e9596: 03 41 28 add 0x28(%ecx),%eax
784e9599: c7 85 80 00 00 00 04 movl $0x4,0x80(%ebp)
784e95a0: 00 00 00
784e95a3: 89 45 7c mov %eax,0x7c(%ebp)
784e95a6: 8b 42 0c mov 0xc(%edx),%eax
784e95a9: 2b 45 4c sub 0x4c(%ebp),%eax
784e95ac: 85 c0 test %eax,%eax
784e95ae: 89 42 0c mov %eax,0xc(%edx)
784e95b1: 0f 85 ee 00 00 00 jne 784e96a5 <ata_qc_issue+0x2b5>
784e95b7: 83 6d 3c 01 subl $0x1,0x3c(%ebp)
784e95bb: e9 54 ff ff ff jmp 784e9514 <ata_qc_issue+0x124>
784e95c0: 8b 5c 24 08 mov 0x8(%esp),%ebx
784e95c4: 80 7b 0c 00 cmpb $0x0,0xc(%ebx)
784e95c8: 0f 89 42 ff ff ff jns 784e9510 <ata_qc_issue+0x120>
784e95ce: 8b 45 34 mov 0x34(%ebp),%eax
784e95d1: a8 02 test $0x2,%al
784e95d3: 0f 84 77 ff ff ff je 784e9550 <ata_qc_issue+0x160>
784e95d9: e9 85 fe ff ff jmp 784e9463 <ata_qc_issue+0x73>
784e95de: 89 f6 mov %esi,%esi
784e95e0: 8b 42 0c mov 0xc(%edx),%eax
784e95e3: 85 c0 test %eax,%eax
784e95e5: 75 0b jne 784e95f2 <ata_qc_issue+0x202>
784e95e7: 8b 4c 24 04 mov 0x4(%esp),%ecx
784e95eb: 83 81 84 16 00 00 01 addl $0x1,0x1684(%ecx)
784e95f2: 8b 4d 38 mov 0x38(%ebp),%ecx
784e95f5: b8 01 00 00 00 mov $0x1,%eax
784e95fa: d3 e0 shl %cl,%eax
784e95fc: 09 42 0c or %eax,0xc(%edx)
784e95ff: e9 1b fe ff ff jmp 784e941f <ata_qc_issue+0x2f>
784e9604: 8b 45 38 mov 0x38(%ebp),%eax
784e9607: 8b 4c 24 08 mov 0x8(%esp),%ecx
784e960b: c1 e0 02 shl $0x2,%eax
784e960e: 89 44 24 0c mov %eax,0xc(%esp)
784e9612: 8b 49 24 mov 0x24(%ecx),%ecx
784e9615: 01 c8 add %ecx,%eax
784e9617: 89 44 24 0c mov %eax,0xc(%esp)
784e961b: c7 00 00 00 00 00 movl $0x0,(%eax)
784e9621: 8b 5c 24 10 mov 0x10(%esp),%ebx
784e9625: 8b 7c 24 10 mov 0x10(%esp),%edi
784e9629: 8b 53 04 mov 0x4(%ebx),%edx
784e962c: 03 53 0c add 0xc(%ebx),%edx
784e962f: 8b 07 mov (%edi),%eax
784e9631: 8b 1d f8 ec d7 78 mov 0x78d7ecf8,%ebx
784e9637: 2b 55 4c sub 0x4c(%ebp),%edx
784e963a: 29 d8 sub %ebx,%eax
784e963c: 89 d1 mov %edx,%ecx
784e963e: c1 f8 05 sar $0x5,%eax
784e9641: 81 e2 ff 0f 00 00 and $0xfff,%edx
784e9647: c1 e9 0c shr $0xc,%ecx
784e964a: 01 c8 add %ecx,%eax
784e964c: c1 e0 05 shl $0x5,%eax
784e964f: 01 c3 add %eax,%ebx
784e9651: f6 45 10 08 testb $0x8,0x10(%ebp)
784e9655: 89 5d 74 mov %ebx,0x74(%ebp)
784e9658: 89 55 78 mov %edx,0x78(%ebp)
784e965b: 0f 85 88 00 00 00 jne 784e96e9 <ata_qc_issue+0x2f9>
784e9661: 8b 54 24 08 mov 0x8(%esp),%edx
784e9665: 8b 45 38 mov 0x38(%ebp),%eax
784e9668: c1 e0 02 shl $0x2,%eax
784e966b: 03 42 28 add 0x28(%edx),%eax
784e966e: c7 85 80 00 00 00 04 movl $0x4,0x80(%ebp)
784e9675: 00 00 00
784e9678: 89 45 7c mov %eax,0x7c(%ebp)
784e967b: 8b 4c 24 10 mov 0x10(%esp),%ecx
784e967f: 8b 41 0c mov 0xc(%ecx),%eax
784e9682: 2b 45 4c sub 0x4c(%ebp),%eax
784e9685: 85 c0 test %eax,%eax
784e9687: 89 41 0c mov %eax,0xc(%ecx)
784e968a: 75 32 jne 784e96be <ata_qc_issue+0x2ce>
784e968c: 8b 4d 3c mov 0x3c(%ebp),%ecx
784e968f: 85 c9 test %ecx,%ecx
784e9691: 74 08 je 784e969b <ata_qc_issue+0x2ab>
784e9693: 83 e9 01 sub $0x1,%ecx
784e9696: e9 18 fe ff ff jmp 784e94b3 <ata_qc_issue+0xc3>
784e969b: 31 c0 xor %eax,%eax
784e969d: 8d 76 00 lea 0x0(%esi),%esi
784e96a0: e9 59 fe ff ff jmp 784e94fe <ata_qc_issue+0x10e>
784e96a5: 8b 85 84 00 00 00 mov 0x84(%ebp),%eax
784e96ab: f0 83 44 24 00 00 lock addl $0x0,0x0(%esp)
784e96b1: 2d 00 00 00 78 sub $0x78000000,%eax
784e96b6: 89 42 08 mov %eax,0x8(%edx)
784e96b9: e9 56 fe ff ff jmp 784e9514 <ata_qc_issue+0x124>
784e96be: 8b 75 3c mov 0x3c(%ebp),%esi
784e96c1: e9 eb fd ff ff jmp 784e94b1 <ata_qc_issue+0xc1>
784e96c6: 8b 45 4c mov 0x4c(%ebp),%eax
784e96c9: 8b 72 0c mov 0xc(%edx),%esi
784e96cc: 03 b5 84 00 00 00 add 0x84(%ebp),%esi
784e96d2: 89 c1 mov %eax,%ecx
784e96d4: c1 e9 02 shr $0x2,%ecx
784e96d7: 29 c6 sub %eax,%esi
784e96d9: f3 a5 rep movsl %ds:(%esi),%es:(%edi)
784e96db: 89 c1 mov %eax,%ecx
784e96dd: 83 e1 03 and $0x3,%ecx
784e96e0: 74 02 je 784e96e4 <ata_qc_issue+0x2f4>
784e96e2: f3 a4 rep movsb %ds:(%esi),%es:(%edi)
784e96e4: e9 a3 fe ff ff jmp 784e958c <ata_qc_issue+0x19c>
784e96e9: 89 e2 mov %esp,%edx
784e96eb: 81 e2 00 e0 ff ff and $0xffffe000,%edx
784e96f1: 83 42 14 01 addl $0x1,0x14(%edx)
784e96f5: 8b 45 4c mov 0x4c(%ebp),%eax
784e96f8: 2b 1d f8 ec d7 78 sub 0x78d7ecf8,%ebx
784e96fe: 8b 7c 24 0c mov 0xc(%esp),%edi
784e9702: c1 fb 05 sar $0x5,%ebx
784e9705: 89 c1 mov %eax,%ecx
784e9707: c1 e3 0c shl $0xc,%ebx
784e970a: 8d b3 00 00 00 78 lea 0x78000000(%ebx),%esi
784e9710: c1 e9 02 shr $0x2,%ecx
784e9713: 03 75 78 add 0x78(%ebp),%esi
784e9716: f3 a5 rep movsl %ds:(%esi),%es:(%edi)
784e9718: 89 c1 mov %eax,%ecx
784e971a: 83 e1 03 and $0x3,%ecx
784e971d: 74 02 je 784e9721 <ata_qc_issue+0x331>
784e971f: f3 a4 rep movsb %ds:(%esi),%es:(%edi)
784e9721: 83 6a 14 01 subl $0x1,0x14(%edx)
784e9725: e9 37 ff ff ff jmp 784e9661 <ata_qc_issue+0x271>
784e972a: 8b 54 24 10 mov 0x10(%esp),%edx
784e972e: 8b 45 4c mov 0x4c(%ebp),%eax
784e9731: 01 42 0c add %eax,0xc(%edx)
784e9734: 83 65 34 f9 andl $0xfffffff9,0x34(%ebp)
784e9738: 83 8d 8c 00 00 00 40 orl $0x40,0x8c(%ebp)
784e973f: 83 c4 14 add $0x14,%esp
784e9742: 89 e8 mov %ebp,%eax
784e9744: 5b pop %ebx
784e9745: 5e pop %esi
784e9746: 5f pop %edi
784e9747: 5d pop %ebp
784e9748: e9 23 fc ff ff jmp 784e9370 <ata_qc_complete>
784e974d: 89 d1 mov %edx,%ecx
784e974f: 83 e1 fe and $0xfffffffe,%ecx
784e9752: e9 38 fd ff ff jmp 784e948f <ata_qc_issue+0x9f>
784e9757: 83 e0 fe and $0xfffffffe,%eax
784e975a: 89 04 24 mov %eax,(%esp)
784e975d: e9 88 fd ff ff jmp 784e94ea <ata_qc_issue+0xfa>
784e9762: 8d b4 26 00 00 00 00 lea 0x0(%esi),%esi
784e9769: 8d bc 27 00 00 00 00 lea 0x0(%edi),%edi
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> OK, the below should actually be safe, I don't know why I talked myself
> into the next_sg stuff in the beginning. It's always safe to zero sg,
> since it's a valid entry - nothing to save in ->page. Ingo, does this
> work for you?
I really don't think this should work.
Doing "sg_next()" on a valid sg is *always* ok. So if the old code didn't
work, then "sg" wasn't valid to start with (and the code *after* the
sg_next() would have oopsed even if you try to avoid using sg_next.
So avoiding the "sg_next()" on the last entry is pointless.
Also, your patch makes the code almost totally unreadable, with that
subtle issue of the "if (bvprv && cluster)" case not triggering on the
first case, so the NULL initial sg is "safe".
So at a guess, I think the *real* problem is simply that the passed-in
sglist was just too small. What guarantees that the sg list allocation
(apparently done by scsi_alloc_sgtable()) is big enough?
If I read things right, scsi_alloc_sgtable() will allocate "cmd->use_sg"
SG enties, no? But I also notice that it does not seem to initialize the
SG allocation, so those SG entries contain random crap - including,
perhaps, a random - and bogus - chain pointer in sg->page..
Yes, we set sh->page *if* we create a chain, but if we don't chain, we
leave the old random contents around which in turn may include old and
stale chain pointers. Or am I missing something?
So when you added that "memset(sg, 0, sizeof(*sg))" into blk_rq_map_sg(),
you did it way too late - it needs to be done when the sg chain is
allocated, and for every entry (and then the "link" entry needs to be
linked in separately)
I think.
Linus
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> ok, here's a different but similar crash that triggers on the testbox:
>
> [ 233.438890] BUG: unable to handle kernel paging request at virtual address 7d93e000
> [ 233.446390] printing eip: 784e9480 *pde = 01000067 *pte = 0593e000
> [ 233.452630] Oops: 0000 [#1] DEBUG_PAGEALLOC
> [ 233.456790]
> [ 233.458264] Pid: 0, comm: swapper Not tainted (2.6.23 #5)
> [ 233.463637] EIP: 0060:[<784e9480>] EFLAGS: 00010087 CPU: 0
> [ 233.469101] EIP is at ata_qc_issue+0x90/0x380
> [ 233.473429] EAX: 7d93dff0 EBX: 0000001f ECX: 7d93dff0 EDX: 798daf80
> [ 233.479668] ESI: 00000020 EDI: 7d93de00 EBP: 7b54007c ESP: 78a13e14
> [ 233.485908] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> [ 233.491282] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
> [ 233.498473] Stack: 7d93de00 7b540000 7b540000 00000000 7d93dfe0 7b54007c 7d93db00 7b5417a4
> [ 233.506793] 784c2490 784ef69e 784f21f3 7b52de98 7d93db00 7b540000 7b5417a4 7d93db00
> [ 233.515112] 7b540000 7b524004 784f22e0 784ef380 784c2490 7d93db00 00000202 7b524004
> [ 233.523432] Call Trace:
> [ 233.526033] [<784c2490>] scsi_done+0x0/0x20
> [ 233.530279] [<784ef69e>] ata_scsi_translate+0xbe/0x140
> [ 233.535478] [<784f21f3>] ata_scsi_queuecmd+0x33/0x200
> [ 233.540591] [<784f22e0>] ata_scsi_queuecmd+0x120/0x200
> [ 233.545791] [<784ef380>] ata_scsi_rw_xlat+0x0/0x220
> [ 233.550730] [<784c2490>] scsi_done+0x0/0x20
> [ 233.554976] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
> [ 233.560177] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> [ 233.565202] [<784c8c7e>] scsi_request_fn+0x1be/0x370
> [ 233.570229] [<78408086>] blk_run_queue+0x36/0x80
> [ 233.574909] [<784c7520>] scsi_next_command+0x30/0x50
> [ 233.579935] [<784c76ab>] scsi_end_request+0xab/0xe0
> [ 233.584875] [<784c83f9>] scsi_io_completion+0xa9/0x3d0
> [ 233.590075] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> [ 233.595100] [<78405125>] blk_done_softirq+0x45/0x80
> [ 233.600040] [<78405153>] blk_done_softirq+0x73/0x80
> [ 233.604981] [<7811d4c3>] __do_softirq+0x53/0xb0
> [ 233.609573] [<7811d588>] do_softirq+0x68/0x70
> [ 233.613993] [<78105351>] do_IRQ+0x51/0x90
> [ 233.618066] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
> [ 233.623092] [<7810f2d0>] pgd_dtor+0x0/0x50
> [ 233.627252] [<7810388e>] common_interrupt+0x2e/0x40
> [ 233.632192] [<7810f2d0>] pgd_dtor+0x0/0x50
> [ 233.636352] [<7815f3be>] quicklist_trim+0x5e/0x90
> [ 233.641118] [<7810f2cb>] check_pgt_cache+0x1b/0x20
> [ 233.645971] [<78100c52>] cpu_idle+0x32/0x60
> [ 233.650217] [<78a14b35>] start_kernel+0x265/0x300
> [ 233.654983] [<78a14380>] unknown_bootoption+0x0/0x1e0
> [ 233.660097] =======================
> [ 233.663649] Code: 00 00 00 8b 45 34 a8 02 0f 84 ed 00 00 00 8b bd 88 00 00 00 31 db 89 3c 24 8b 75 3c 89 f8 c7 44 24 10 00 00 00 00 eb 1b 8d 76 00 <8b> 50 10 8d 48 10 f6 c2 01 0f 85 be 02 00 00 89 44 24 10 83 c3
> [ 233.682455] EIP: [<784e9480>] ata_qc_issue+0x90/0x380 SS:ESP 0068:78a13e14
> [ 233.689302] Kernel panic - not syncing: Fatal exception in interrupt
>
> (gdb) list *0x784e9480
> 0x784e9480 is in ata_qc_issue (include/linux/scatterlist.h:48).
> 43 */
> 44 static inline struct scatterlist *sg_next(struct scatterlist *sg)
> 45 {
> 46 sg++;
> 47
> 48 if (unlikely(sg_is_chain(sg)))
> 49 sg = sg_chain_ptr(sg);
> 50
> 51 return sg;
> 52 }
> (gdb)
>
> so there's sg_next() involvement too. Below is the disassembly.
You must have a magic test box :-)
Will investigate... libata doesn't actually enable chaining, but since
i386 supports it, it ends up using the chain helpers anyway.
There seems to be some automatic inlining involved here, it must be
dying inside ata_sg_setup().
--
Jens Axboe
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Ingo Molnar wrote:
> >
> > ok, here's a different but similar crash that triggers on the testbox:
> >
> > [ 233.438890] BUG: unable to handle kernel paging request at virtual address 7d93e000
> > [ 233.446390] printing eip: 784e9480 *pde = 01000067 *pte = 0593e000
> > [ 233.452630] Oops: 0000 [#1] DEBUG_PAGEALLOC
> > [ 233.456790]
> > [ 233.458264] Pid: 0, comm: swapper Not tainted (2.6.23 #5)
> > [ 233.463637] EIP: 0060:[<784e9480>] EFLAGS: 00010087 CPU: 0
> > [ 233.469101] EIP is at ata_qc_issue+0x90/0x380
> > [ 233.473429] EAX: 7d93dff0 EBX: 0000001f ECX: 7d93dff0 EDX: 798daf80
> > [ 233.479668] ESI: 00000020 EDI: 7d93de00 EBP: 7b54007c ESP: 78a13e14
> > [ 233.485908] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> > [ 233.491282] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
> > [ 233.498473] Stack: 7d93de00 7b540000 7b540000 00000000 7d93dfe0 7b54007c 7d93db00 7b5417a4
> > [ 233.506793] 784c2490 784ef69e 784f21f3 7b52de98 7d93db00 7b540000 7b5417a4 7d93db00
> > [ 233.515112] 7b540000 7b524004 784f22e0 784ef380 784c2490 7d93db00 00000202 7b524004
> > [ 233.523432] Call Trace:
> > [ 233.526033] [<784c2490>] scsi_done+0x0/0x20
> > [ 233.530279] [<784ef69e>] ata_scsi_translate+0xbe/0x140
> > [ 233.535478] [<784f21f3>] ata_scsi_queuecmd+0x33/0x200
> > [ 233.540591] [<784f22e0>] ata_scsi_queuecmd+0x120/0x200
> > [ 233.545791] [<784ef380>] ata_scsi_rw_xlat+0x0/0x220
> > [ 233.550730] [<784c2490>] scsi_done+0x0/0x20
> > [ 233.554976] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
> > [ 233.560177] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> > [ 233.565202] [<784c8c7e>] scsi_request_fn+0x1be/0x370
> > [ 233.570229] [<78408086>] blk_run_queue+0x36/0x80
> > [ 233.574909] [<784c7520>] scsi_next_command+0x30/0x50
> > [ 233.579935] [<784c76ab>] scsi_end_request+0xab/0xe0
> > [ 233.584875] [<784c83f9>] scsi_io_completion+0xa9/0x3d0
> > [ 233.590075] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> > [ 233.595100] [<78405125>] blk_done_softirq+0x45/0x80
> > [ 233.600040] [<78405153>] blk_done_softirq+0x73/0x80
> > [ 233.604981] [<7811d4c3>] __do_softirq+0x53/0xb0
> > [ 233.609573] [<7811d588>] do_softirq+0x68/0x70
> > [ 233.613993] [<78105351>] do_IRQ+0x51/0x90
> > [ 233.618066] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
> > [ 233.623092] [<7810f2d0>] pgd_dtor+0x0/0x50
> > [ 233.627252] [<7810388e>] common_interrupt+0x2e/0x40
> > [ 233.632192] [<7810f2d0>] pgd_dtor+0x0/0x50
> > [ 233.636352] [<7815f3be>] quicklist_trim+0x5e/0x90
> > [ 233.641118] [<7810f2cb>] check_pgt_cache+0x1b/0x20
> > [ 233.645971] [<78100c52>] cpu_idle+0x32/0x60
> > [ 233.650217] [<78a14b35>] start_kernel+0x265/0x300
> > [ 233.654983] [<78a14380>] unknown_bootoption+0x0/0x1e0
> > [ 233.660097] =======================
> > [ 233.663649] Code: 00 00 00 8b 45 34 a8 02 0f 84 ed 00 00 00 8b bd 88 00 00 00 31 db 89 3c 24 8b 75 3c 89 f8 c7 44 24 10 00 00 00 00 eb 1b 8d 76 00 <8b> 50 10 8d 48 10 f6 c2 01 0f 85 be 02 00 00 89 44 24 10 83 c3
> > [ 233.682455] EIP: [<784e9480>] ata_qc_issue+0x90/0x380 SS:ESP 0068:78a13e14
> > [ 233.689302] Kernel panic - not syncing: Fatal exception in interrupt
> >
> > (gdb) list *0x784e9480
> > 0x784e9480 is in ata_qc_issue (include/linux/scatterlist.h:48).
> > 43 */
> > 44 static inline struct scatterlist *sg_next(struct scatterlist *sg)
> > 45 {
> > 46 sg++;
> > 47
> > 48 if (unlikely(sg_is_chain(sg)))
> > 49 sg = sg_chain_ptr(sg);
> > 50
> > 51 return sg;
> > 52 }
> > (gdb)
> >
> > so there's sg_next() involvement too. Below is the disassembly.
>
> You must have a magic test box :-)
>
> Will investigate... libata doesn't actually enable chaining, but since
> i386 supports it, it ends up using the chain helpers anyway.
>
> There seems to be some automatic inlining involved here, it must be
> dying inside ata_sg_setup().
Also, can you send a dmesg from that system so I can see which libata
drivers are involved?
--
Jens Axboe
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> OK, it is fine, as long as the sglist is cleared initially. And I don't
> think there's anyway around that, clearly I didn't think long enough
> before including the memset() removal from Tomo.
Ok, I think that one-liner fixes the real bug.
But I think the rest of your changes are simply bad.
The fix to block/ll_rw_block.c should likely be something like the
appended instead:
- remove the "memset()" you had added earlier. It's bogus. It cannot be
the right thing. If the sg list wasn't initialized correctly much
earlier, trying to initialize it late is pointless - it contains crap.
- the old code was fine, but let's initialize "sg" to NULL to make it
clear that the initial value of sg is pointless, and only "next_sg"
matters (since sg had better be assigned from that).
Hmm?
Linus
---
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 9e3f3cc..54d974e 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1333,7 +1333,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
* for each bio in rq
*/
bvprv = NULL;
- sg = next_sg = &sglist[0];
+ sg = NULL;
+ next_sg = &sglist[0];
rq_for_each_segment(bvec, rq, iter) {
int nbytes = bvec->bv_len;
@@ -1352,7 +1353,6 @@ new_segment:
sg = next_sg;
next_sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
sg->page = bvec->bv_page;
sg->length = nbytes;
sg->offset = bvec->bv_offset;
* Jens Axboe <[email protected]> wrote:
> Also, can you send a dmesg from that system so I can see which libata
> drivers are involved?
same system for which i sent the earlier bootlog - but here's the second
crashlog too. Crash is reproducible - crashed twice in a row already.
(NOTE: the attached log is from this second crash.)
Ingo
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > OK, the below should actually be safe, I don't know why I talked myself
> > into the next_sg stuff in the beginning. It's always safe to zero sg,
> > since it's a valid entry - nothing to save in ->page. Ingo, does this
> > work for you?
>
> I really don't think this should work.
>
> Doing "sg_next()" on a valid sg is *always* ok. So if the old code didn't
> work, then "sg" wasn't valid to start with (and the code *after* the
> sg_next() would have oopsed even if you try to avoid using sg_next.
>
> So avoiding the "sg_next()" on the last entry is pointless.
Yeah, I didn't quite understand why if sg was valid, why dereferencing
*(sg + 1)->page would crap out :/
> Also, your patch makes the code almost totally unreadable, with that
> subtle issue of the "if (bvprv && cluster)" case not triggering on the
> first case, so the NULL initial sg is "safe".
Hmm I think it's quite readable, but perhaps that's just me :-). The
first is much cleaner, and the last part just reads 'if sg is not set
yet, set to list. otherwise, goto next entry'.
> So at a guess, I think the *real* problem is simply that the passed-in
> sglist was just too small. What guarantees that the sg list allocation
> (apparently done by scsi_alloc_sgtable()) is big enough?
>
> If I read things right, scsi_alloc_sgtable() will allocate "cmd->use_sg"
> SG enties, no? But I also notice that it does not seem to initialize the
> SG allocation, so those SG entries contain random crap - including,
> perhaps, a random - and bogus - chain pointer in sg->page..
Right, we allocate an sgtable that will hold ->use_sg entries, which
contains request->nr_phys_segments. And that should definitely fit.
Regarding the init of the sglist, that was the revert I was talking
about. We do need that memset() in there, so all those sg entries will
be properly zeroed.
> Yes, we set sh->page *if* we create a chain, but if we don't chain, we
> leave the old random contents around which in turn may include old and
> stale chain pointers. Or am I missing something?
>
> So when you added that "memset(sg, 0, sizeof(*sg))" into blk_rq_map_sg(),
> you did it way too late - it needs to be done when the sg chain is
> allocated, and for every entry (and then the "link" entry needs to be
> linked in separately)
>
> I think.
Yep, and that is what Ingo did test as well and it worked. For that
case, now libata is crapping out elsewhere in sg_next().
--
Jens Axboe
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > OK, it is fine, as long as the sglist is cleared initially. And I don't
> > think there's anyway around that, clearly I didn't think long enough
> > before including the memset() removal from Tomo.
>
> Ok, I think that one-liner fixes the real bug.
>
> But I think the rest of your changes are simply bad.
>
> The fix to block/ll_rw_block.c should likely be something like the
> appended instead:
>
> - remove the "memset()" you had added earlier. It's bogus. It cannot be
> the right thing. If the sg list wasn't initialized correctly much
> earlier, trying to initialize it late is pointless - it contains crap.
It's required to clear output members (like dma_len and so on), since
some of the IOMU code really wants that initialized.
> - the old code was fine, but let's initialize "sg" to NULL to make it
> clear that the initial value of sg is pointless, and only "next_sg"
> matters (since sg had better be assigned from that).
If you prefer the old next_sg approach to my alternative, that is fine
with me. But we do need the memset().
--
Jens Axboe
* Linus Torvalds <[email protected]> wrote:
> But I think the rest of your changes are simply bad.
>
> The fix to block/ll_rw_block.c should likely be something like the
> appended instead:
>
> - remove the "memset()" you had added earlier. It's bogus. It cannot be
> the right thing. If the sg list wasn't initialized correctly much
> earlier, trying to initialize it late is pointless - it contains crap.
>
> - the old code was fine, but let's initialize "sg" to NULL to make it
> clear that the initial value of sg is pointless, and only "next_sg"
> matters (since sg had better be assigned from that).
>
> Hmm?
built and booted your patch (removed Jens's) but it still crashes in
blk_rq_map_sg() - see below. Should i have left something from Jens's
patch?
Ingo
------------------->
[ 34.698199] VFS: Mounted root (ext3 filesystem) readonly.
[ 34.703917] Freeing unused kernel memory: 372k freed
[ 34.746609] BUG: unable to handle kernel paging request at virtual address 6e616872
[ 34.754106] printing eip: 7840503b *pde = 00000000
[ 34.758960] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 34.763120]
[ 34.764594] Pid: 1, comm: swapper Not tainted (2.6.23 #7)
[ 34.769965] EIP: 0060:[<7840503b>] EFLAGS: 00010006 CPU: 0
[ 34.775429] EIP is at blk_rq_map_sg+0xbb/0x170
[ 34.779845] EAX: 3ffcd000 EBX: 7c88a68c ECX: 3ffce000 EDX: 007ff9a0
[ 34.786085] ESI: 6e616862 EDI: 798ea9a0 EBP: 00001000 ESP: 7b421bf4
[ 34.792324] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 34.797697] Process swapper (pid: 1, ti=7b420000 task=7b416000 task.ti=7b420000)
[ 34.804890] Stack: 00000046 7b520000 00002000 3ffcf000 7c88ac80 00000001 00000001 00000001
[ 34.813209] 7c88a600 01000000 7c88ab00 7b521d1c 7c88a708 7b520000 784c7ac5 7b520000
[ 34.821529] 784c75ca 7c88a708 7b524ce4 7b521d1c 7c88a708 784e4570 7b416000 7813661b
[ 34.829847] Call Trace:
[ 34.832448] [<784c7ac5>] scsi_init_io+0x55/0xe0
[ 34.837042] [<784c75ca>] scsi_get_cmd_from_req+0x2a/0x40
[ 34.842414] [<784e4570>] sd_prep_fn+0x80/0x940
[ 34.846920] [<7813661b>] __lock_acquire+0x4ab/0xe20
[ 34.851860] [<781450a6>] add_to_page_cache+0x66/0xb0
[ 34.856887] [<78404633>] elv_dispatch_sort+0x23/0xe0
[ 34.861912] [<784041a0>] elv_next_request+0xa0/0x130
[ 34.866938] [<784c8c04>] scsi_request_fn+0x1e4/0x370
[ 34.871965] [<78120f02>] del_timer+0x62/0x70
[ 34.876298] [<78407445>] __generic_unplug_device+0x25/0x30
[ 34.881844] [<78407715>] generic_unplug_device+0x15/0x30
[ 34.887217] [<78404e00>] blk_backing_dev_unplug+0x0/0x10
[ 34.892590] [<78404e0c>] blk_backing_dev_unplug+0xc/0x10
[ 34.897963] [<78180a9d>] block_sync_page+0x2d/0x40
[ 34.902817] [<78144dc9>] sync_page+0x29/0x40
[ 34.907150] [<7876f5bc>] __wait_on_bit_lock+0x3c/0x70
[ 34.912263] [<78144da0>] sync_page+0x0/0x40
[ 34.916509] [<78144d82>] __lock_page+0x52/0x60
[ 34.921016] [<7812adb0>] wake_bit_function+0x0/0x60
[ 34.925955] [<7814554c>] do_generic_mapping_read+0x21c/0x450
[ 34.931675] [<78144b10>] file_read_actor+0x0/0x130
[ 34.936528] [<78147107>] generic_file_aio_read+0x137/0x180
[ 34.942074] [<78144b10>] file_read_actor+0x0/0x130
[ 34.946927] [<78161245>] do_sync_read+0xd5/0x120
[ 34.951607] [<78135646>] mark_lock+0x76/0x550
[ 34.956027] [<7812ad70>] autoremove_wake_function+0x0/0x40
[ 34.961573] [<78155026>] __vma_link+0x36/0x70
[ 34.965993] [<78161170>] do_sync_read+0x0/0x120
[ 34.970586] [<78161ae3>] vfs_read+0xb3/0x110
[ 34.974919] [<781652cd>] kernel_read+0x3d/0x60
[ 34.979425] [<7816539f>] prepare_binprm+0xaf/0xe0
[ 34.984192] [<78166973>] do_execve+0x123/0x1c0
[ 34.988698] [<78100f1f>] sys_execve+0x2f/0x90
[ 34.993118] [<781028a2>] syscall_call+0x7/0xb
[ 34.997538] [<78106f1c>] kernel_execve+0x1c/0x30
[ 35.002217] [<7810016e>] init_post+0x9e/0xe0
[ 35.006550] [<78a147b3>] kernel_init+0x163/0x280
[ 35.011230] [<78a14650>] kernel_init+0x0/0x280
[ 35.015736] [<78103a97>] kernel_thread_helper+0x7/0x10
[ 35.020936] =======================
[ 35.024489] Code: 29 c1 c1 f9 05 c1 e1 0c 03 4a 08 8b 52 04 01 ca 89 54 24 0c 8b 3b 89 fa 29 c2 89 d0 c1 f8 05 c1 e0 0c 03 43 08 39 44 24 0c 74 7e <8b> 46 10 8d 56 10 a8 01 75 4c 89 3e 89 6e 0c 8b 43 08 89 46 04
[ 35.043294] EIP: [<7840503b>] blk_rq_map_sg+0xbb/0x170 SS:ESP 0068:7b421bf4
[ 35.050228] Kernel panic - not syncing: Attempted to kill init!
* Linus Torvalds <[email protected]> wrote:
>
>
> On Wed, 17 Oct 2007, Ingo Molnar wrote:
> >
> > ok, here's a different but similar crash that triggers on the testbox:
>
> Just to confirm - is this with the added memset() in
> drivers/scsi/scsi_lib.c?
yes, Jens's last patch was applied - apparently this ATA crash was
masked by the other one. I got one successful bootup after applying
Jens's patch and it has crashed on bootup twice since then.
Ingo
On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > - remove the "memset()" you had added earlier. It's bogus. It cannot be
> > the right thing. If the sg list wasn't initialized correctly much
> > earlier, trying to initialize it late is pointless - it contains crap.
>
> It's required to clear output members (like dma_len and so on), since
> some of the IOMU code really wants that initialized.
No it's NOT!
The whole point here is that the sg had *already* better be cleared.
If it wasn't cleared before, that's a bug regardless of anything else. So
a memset() is guaranteed to be either a no-op or hiding another bug!
So yes, those members had better be zero. It's just that they had better
be zero long before that memset! The memset should have been done when
allocating the SG list.
> If you prefer the old next_sg approach to my alternative, that is fine
> with me. But we do need the memset().
Really? Explain why. Entering that code with a non-initialized SG list is
a bug to begin with.
Linus
On Wed, 17 Oct 2007, Ingo Molnar wrote:
>
> built and booted your patch (removed Jens's) but it still crashes in
> blk_rq_map_sg() - see below. Should i have left something from Jens's
> patch?
The *real* fix in Jens' patch (and the only thing that really matters) is
the one to drivers/scsi/scsi_lib.c.
The rest is all fluff.
Linus
On Wed, 17 Oct 2007, Ingo Molnar wrote:
>
> ok, here's a different but similar crash that triggers on the testbox:
Just to confirm - is this with the added memset() in
drivers/scsi/scsi_lib.c?
Or did you get this oops before that patch?
Linus
On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > So avoiding the "sg_next()" on the last entry is pointless.
>
> Yeah, I didn't quite understand why if sg was valid, why dereferencing
> *(sg + 1)->page would crap out :/
Actually, I take that back. If 'sg' is the last entry in a *non*linked
scatter-gather list (ie we don't use the last entry as a link, we actually
use it as a real SG entry), then "sg_next(sg)" will indeed access past the
end of the whole allocated array, and will access one past the end.
And with page-alloc debugging, that *will* blow up.
So I think your change to use "sg_next()" only when you actually need a
next pointer is the correct one after all.
Linus
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> > >
> > > - remove the "memset()" you had added earlier. It's bogus. It cannot be
> > > the right thing. If the sg list wasn't initialized correctly much
> > > earlier, trying to initialize it late is pointless - it contains crap.
> >
> > It's required to clear output members (like dma_len and so on), since
> > some of the IOMU code really wants that initialized.
>
> No it's NOT!
>
> The whole point here is that the sg had *already* better be cleared.
>
> If it wasn't cleared before, that's a bug regardless of anything else. So
> a memset() is guaranteed to be either a no-op or hiding another bug!
>
> So yes, those members had better be zero. It's just that they had better
> be zero long before that memset! The memset should have been done when
> allocating the SG list.
For the chain elements - yes, definitely! But we also want to clear dma
mapping output values, at least sparc64 wants that. You could argue that
the IOMMU code should be fixed up, but I don't think we should mix the
two.
So we need the memset() in blk_rq_map_sg() AS WELL AS the initial sg
table clear. I'm not arguing about the latter, we clearly do need that.
> > If you prefer the old next_sg approach to my alternative, that is fine
> > with me. But we do need the memset().
>
> Really? Explain why. Entering that code with a non-initialized SG list is
> a bug to begin with.
Not for the output members, they will be filled AFTER blk_rq_map_sg()
returns and someone use iommu_map_sg() to re-iterate the sglist and map
the pages appropriately.
--
Jens Axboe
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> > >
> > > So avoiding the "sg_next()" on the last entry is pointless.
> >
> > Yeah, I didn't quite understand why if sg was valid, why dereferencing
> > *(sg + 1)->page would crap out :/
>
> Actually, I take that back. If 'sg' is the last entry in a *non*linked
> scatter-gather list (ie we don't use the last entry as a link, we actually
> use it as a real SG entry), then "sg_next(sg)" will indeed access past the
> end of the whole allocated array, and will access one past the end.
>
> And with page-alloc debugging, that *will* blow up.
>
> So I think your change to use "sg_next()" only when you actually need a
> next pointer is the correct one after all.
Thanks, so I'm not totally crazy :-)
Can you just pull:
git://git.kernel.dk/data/git/linux-2.6-block.git for-linus
then so we get those two pieces correct? Then the remaining issue seems
to be a new one that is biting Ingo elsewhere, at least we'll all be on
the same page then.
--
Jens Axboe
On Wed, 17 Oct 2007, Linus Torvalds wrote:
>
> So I think your change to use "sg_next()" only when you actually need a
> next pointer is the correct one after all.
That still leaves the initialization issue. The link pointers need to all
be initialized at SG allocation time (and not just the last one for the
case where it's a linked entry).
And if you initialize them at allocation time, does an allocation ever get
re-used without being free'd in between (if it does, you do indeed need to
initialize the non-link pointers each time - but I don't see it being the
case, so ..)
Linus
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Ingo Molnar wrote:
> >
> > ok, here's a different but similar crash that triggers on the testbox:
> >
> > [ 233.438890] BUG: unable to handle kernel paging request at virtual address 7d93e000
> > [ 233.446390] printing eip: 784e9480 *pde = 01000067 *pte = 0593e000
> > [ 233.452630] Oops: 0000 [#1] DEBUG_PAGEALLOC
> > [ 233.456790]
> > [ 233.458264] Pid: 0, comm: swapper Not tainted (2.6.23 #5)
> > [ 233.463637] EIP: 0060:[<784e9480>] EFLAGS: 00010087 CPU: 0
> > [ 233.469101] EIP is at ata_qc_issue+0x90/0x380
> > [ 233.473429] EAX: 7d93dff0 EBX: 0000001f ECX: 7d93dff0 EDX: 798daf80
> > [ 233.479668] ESI: 00000020 EDI: 7d93de00 EBP: 7b54007c ESP: 78a13e14
> > [ 233.485908] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
> > [ 233.491282] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
> > [ 233.498473] Stack: 7d93de00 7b540000 7b540000 00000000 7d93dfe0 7b54007c 7d93db00 7b5417a4
> > [ 233.506793] 784c2490 784ef69e 784f21f3 7b52de98 7d93db00 7b540000 7b5417a4 7d93db00
> > [ 233.515112] 7b540000 7b524004 784f22e0 784ef380 784c2490 7d93db00 00000202 7b524004
> > [ 233.523432] Call Trace:
> > [ 233.526033] [<784c2490>] scsi_done+0x0/0x20
> > [ 233.530279] [<784ef69e>] ata_scsi_translate+0xbe/0x140
> > [ 233.535478] [<784f21f3>] ata_scsi_queuecmd+0x33/0x200
> > [ 233.540591] [<784f22e0>] ata_scsi_queuecmd+0x120/0x200
> > [ 233.545791] [<784ef380>] ata_scsi_rw_xlat+0x0/0x220
> > [ 233.550730] [<784c2490>] scsi_done+0x0/0x20
> > [ 233.554976] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
> > [ 233.560177] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> > [ 233.565202] [<784c8c7e>] scsi_request_fn+0x1be/0x370
> > [ 233.570229] [<78408086>] blk_run_queue+0x36/0x80
> > [ 233.574909] [<784c7520>] scsi_next_command+0x30/0x50
> > [ 233.579935] [<784c76ab>] scsi_end_request+0xab/0xe0
> > [ 233.584875] [<784c83f9>] scsi_io_completion+0xa9/0x3d0
> > [ 233.590075] [<78135c67>] trace_hardirqs_on+0x67/0xb0
> > [ 233.595100] [<78405125>] blk_done_softirq+0x45/0x80
> > [ 233.600040] [<78405153>] blk_done_softirq+0x73/0x80
> > [ 233.604981] [<7811d4c3>] __do_softirq+0x53/0xb0
> > [ 233.609573] [<7811d588>] do_softirq+0x68/0x70
> > [ 233.613993] [<78105351>] do_IRQ+0x51/0x90
> > [ 233.618066] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
> > [ 233.623092] [<7810f2d0>] pgd_dtor+0x0/0x50
> > [ 233.627252] [<7810388e>] common_interrupt+0x2e/0x40
> > [ 233.632192] [<7810f2d0>] pgd_dtor+0x0/0x50
> > [ 233.636352] [<7815f3be>] quicklist_trim+0x5e/0x90
> > [ 233.641118] [<7810f2cb>] check_pgt_cache+0x1b/0x20
> > [ 233.645971] [<78100c52>] cpu_idle+0x32/0x60
> > [ 233.650217] [<78a14b35>] start_kernel+0x265/0x300
> > [ 233.654983] [<78a14380>] unknown_bootoption+0x0/0x1e0
> > [ 233.660097] =======================
> > [ 233.663649] Code: 00 00 00 8b 45 34 a8 02 0f 84 ed 00 00 00 8b bd 88 00 00 00 31 db 89 3c 24 8b 75 3c 89 f8 c7 44 24 10 00 00 00 00 eb 1b 8d 76 00 <8b> 50 10 8d 48 10 f6 c2 01 0f 85 be 02 00 00 89 44 24 10 83 c3
> > [ 233.682455] EIP: [<784e9480>] ata_qc_issue+0x90/0x380 SS:ESP 0068:78a13e14
> > [ 233.689302] Kernel panic - not syncing: Fatal exception in interrupt
> >
> > (gdb) list *0x784e9480
> > 0x784e9480 is in ata_qc_issue (include/linux/scatterlist.h:48).
> > 43 */
> > 44 static inline struct scatterlist *sg_next(struct scatterlist *sg)
> > 45 {
> > 46 sg++;
> > 47
> > 48 if (unlikely(sg_is_chain(sg)))
> > 49 sg = sg_chain_ptr(sg);
> > 50
> > 51 return sg;
> > 52 }
> > (gdb)
> >
> > so there's sg_next() involvement too. Below is the disassembly.
>
> You must have a magic test box :-)
>
> Will investigate... libata doesn't actually enable chaining, but since
> i386 supports it, it ends up using the chain helpers anyway.
>
> There seems to be some automatic inlining involved here, it must be
> dying inside ata_sg_setup().
OK, something to try out - libata doesn't enable sg chaining, since the
support isn't complete yet. Here's a dumb check just to verify that
scsi_alloc_sgtable() will NEVER return a chain entry for a host that
doesn't have it enabled. If this triggers for you, then that could
explain your problem.
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..cc89c86 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -777,8 +777,12 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
* sglist must be the biggest one, or we would not have
* ended up doing another loop.
*/
- if (prev)
+ if (prev) {
+ struct Scsi_Host *shost = cmd->device->host;
+
+ BUG_ON(!shost->use_sg_chaining);
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
+ }
/*
* don't allow subsequent mempool allocs to sleep, it would
--
Jens Axboe
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Linus Torvalds wrote:
> >
> > So I think your change to use "sg_next()" only when you actually need a
> > next pointer is the correct one after all.
>
> That still leaves the initialization issue. The link pointers need to all
> be initialized at SG allocation time (and not just the last one for the
> case where it's a linked entry).
>
> And if you initialize them at allocation time, does an allocation ever get
> re-used without being free'd in between (if it does, you do indeed need to
> initialize the non-link pointers each time - but I don't see it being the
> case, so ..)
The links will not change, so reuse should actually be OK. The reuse
cannot be larger than the original would hold, but that would be a bug
without chaining as well of course.
WRT reuse, if we end up requeuing a request, than we will go through the
whole thing again. I don't think Ingo is hitting that either, though.
--
Jens Axboe
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> For the chain elements - yes, definitely! But we also want to clear dma
> mapping output values, at least sparc64 wants that. You could argue that
> the IOMMU code should be fixed up, but I don't think we should mix the
> two.
>
> So we need the memset() in blk_rq_map_sg() AS WELL AS the initial sg
> table clear. I'm not arguing about the latter, we clearly do need that.
I still don't see your argument.
The SG table is *already*zeroed*.
The fact that output values will be changed later is irrelevant. They
haven't been changed *before* - or at least you haven't given a reasonable
explanation for why they should have.
So explain to me why the memset() in blk_rq_map_sg() helps, considering
that the memory must have been zeroed at allocation time anyway? Tell me
what it is that fills any of the fields we don't already initialize, and
that can happen *before* that memset?
Linus
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > For the chain elements - yes, definitely! But we also want to clear dma
> > mapping output values, at least sparc64 wants that. You could argue that
> > the IOMMU code should be fixed up, but I don't think we should mix the
> > two.
> >
> > So we need the memset() in blk_rq_map_sg() AS WELL AS the initial sg
> > table clear. I'm not arguing about the latter, we clearly do need that.
>
> I still don't see your argument.
>
> The SG table is *already*zeroed*.
>
> The fact that output values will be changed later is irrelevant. They
> haven't been changed *before* - or at least you haven't given a reasonable
> explanation for why they should have.
>
> So explain to me why the memset() in blk_rq_map_sg() helps, considering
> that the memory must have been zeroed at allocation time anyway? Tell me
> what it is that fills any of the fields we don't already initialize, and
> that can happen *before* that memset?
Ah ok, I see why you are confused. The SCSI case is one, it allocs and
frees the sg table each time. The entries are thus always initialized
when they end up in blk_rq_map_sg(). However, other drivers allocate one
at driver init time and use that one all the time. It needs to be
initially cleared, but it wont be cleared before each request mapping.
And that is fine, as long as we do clear entries in blk_rq_map_sg().
--
Jens Axboe
btw., i get the following build warning:
drivers/scsi/scsi_lib.c: In function 'scsi_free_sgtable':
drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
drivers/scsi/scsi_lib.c: In function 'scsi_alloc_sgtable':
drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
not sure it matters.
Ingo
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> btw., i get the following build warning:
>
> drivers/scsi/scsi_lib.c: In function 'scsi_free_sgtable':
> drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> drivers/scsi/scsi_lib.c: In function 'scsi_alloc_sgtable':
> drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
>
> not sure it matters.
Hmm, I don't see them here (gcc 4.2.1). Must be the BUG(), does it go
away if you add a index = -1 in the default: case? Just to be absolutely
sure...
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> @@ -777,8 +777,12 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
> * sglist must be the biggest one, or we would not have
> * ended up doing another loop.
> */
> - if (prev)
> + if (prev) {
> + struct Scsi_Host *shost = cmd->device->host;
> +
> + BUG_ON(!shost->use_sg_chaining);
> sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
> + }
>
nope, this did not help. First bootup went fine, second bootup crashed
again (see below), without hitting the BUG_ON().
Ingo
[ 39.351198] BUG: unable to handle kernel paging request at virtual address 7ca58000
[ 39.358698] printing eip: 784e9480 *pde = 00dda027 *pte = 04a58000
[ 39.364939] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 39.369099]
[ 39.370573] Pid: 0, comm: swapper Not tainted (2.6.23 #9)
[ 39.375944] EIP: 0060:[<784e9480>] EFLAGS: 00010087 CPU: 0
[ 39.381408] EIP is at ata_qc_issue+0x90/0x380
[ 39.385737] EAX: 7ca57ff0 EBX: 0000001f ECX: 7ca57ff0 EDX: 79181c20
[ 39.391977] ESI: 00000020 EDI: 7ca57e00 EBP: 7b54007c ESP: 78a13e10
[ 39.398217] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 39.403590] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
[ 39.410781] Stack: 7ca57e00 7b540000 7b540000 00000000 7ca57fe0 7b54007c 7c991980 7b5417a4
[ 39.419101] 784c2490 784ef69e 784f21f3 7b52de98 7c991980 7b540000 7b5417a4 7c991980
[ 39.427421] 7b540000 7b524004 784f22e0 784ef380 784c2490 7c991980 00000206 7b524004
[ 39.435740] Call Trace:
[ 39.438340] [<784c2490>] scsi_done+0x0/0x20
[ 39.442586] [<784ef69e>] ata_scsi_translate+0xbe/0x140
[ 39.447785] [<784f21f3>] ata_scsi_queuecmd+0x33/0x200
[ 39.452899] [<784f22e0>] ata_scsi_queuecmd+0x120/0x200
[ 39.458099] [<784ef380>] ata_scsi_rw_xlat+0x0/0x220
[ 39.463038] [<784c2490>] scsi_done+0x0/0x20
[ 39.467285] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
[ 39.472484] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 39.477511] [<784c8c7e>] scsi_request_fn+0x1be/0x370
[ 39.482538] [<78408086>] blk_run_queue+0x36/0x80
[ 39.487217] [<784c7520>] scsi_next_command+0x30/0x50
[ 39.492243] [<784c76ab>] scsi_end_request+0xab/0xe0
[ 39.497183] [<784c83f9>] scsi_io_completion+0xa9/0x3d0
[ 39.502383] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 39.507409] [<78405125>] blk_done_softirq+0x45/0x80
[ 39.512348] [<78405153>] blk_done_softirq+0x73/0x80
[ 39.517288] [<7811d4c3>] __do_softirq+0x53/0xb0
[ 39.521882] [<7811d588>] do_softirq+0x68/0x70
[ 39.526302] [<78105351>] do_IRQ+0x51/0x90
[ 39.530374] [<78770a54>] _spin_unlock+0x14/0x20
[ 39.534968] [<7810290f>] restore_nocheck+0x12/0x15
[ 39.539820] [<7810f2d0>] pgd_dtor+0x0/0x50
[ 39.543980] [<7810388e>] common_interrupt+0x2e/0x40
[ 39.548920] [<7810f2d0>] pgd_dtor+0x0/0x50
[ 39.553079] [<7815f3bd>] quicklist_trim+0x5d/0x90
[ 39.557846] [<7810f2cb>] check_pgt_cache+0x1b/0x20
[ 39.562698] [<78100c52>] cpu_idle+0x32/0x60
[ 39.566945] [<78a14b35>] start_kernel+0x265/0x300
[ 39.571712] [<78a14380>] unknown_bootoption+0x0/0x1e0
[ 39.576825] =======================
[ 39.580377] Code: 00 00 00 8b 45 34 a8 02 0f 84 ed 00 00 00 8b bd 88 00 00 00 31 db 89 3c 24 8b 75 3c 89 f8 c7 44 24 10 00 00 00 00 eb 1b 8d 76 00 <8b> 50 10 8d 48 10 f6 c2 01 0f 85 be 02 00 00 89 44 24 10 83 c3
[ 39.599183] EIP: [<784e9480>] ata_qc_issue+0x90/0x380 SS:ESP 0068:78a13e10
[ 39.606030] Kernel panic - not syncing: Fatal exception in interrupt
* Jens Axboe <[email protected]> wrote:
> On Wed, Oct 17 2007, Ingo Molnar wrote:
> >
> > btw., i get the following build warning:
> >
> > drivers/scsi/scsi_lib.c: In function 'scsi_free_sgtable':
> > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > drivers/scsi/scsi_lib.c: In function 'scsi_alloc_sgtable':
> > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> >
> > not sure it matters.
>
> Hmm, I don't see them here (gcc 4.2.1). Must be the BUG(), does it go
> away if you add a index = -1 in the default: case? Just to be
> absolutely sure...
well if i initialize the variable then of course the warning goes away
:)
NOTE: i get the same warning even without the BUG_ON() patch, and
without your other fix patch as well. I'm using gcc 4.2.2. (Do you get
the warning if you pick up the config i sent you earlier today?)
Ingo
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> Ah ok, I see why you are confused. The SCSI case is one, it allocs and
> frees the sg table each time. The entries are thus always initialized
> when they end up in blk_rq_map_sg(). However, other drivers allocate one
> at driver init time and use that one all the time. It needs to be
> initially cleared, but it wont be cleared before each request mapping.
> And that is fine, as long as we do clear entries in blk_rq_map_sg().
Ahh, ok, thanks for clearing that up.
Linus
* Ingo Molnar <[email protected]> wrote:
> > > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > >
> > > not sure it matters.
> >
> > Hmm, I don't see them here (gcc 4.2.1). Must be the BUG(), does it go
> > away if you add a index = -1 in the default: case? Just to be
> > absolutely sure...
>
> well if i initialize the variable then of course the warning goes away
> :)
>
> NOTE: i get the same warning even without the BUG_ON() patch, and
> without your other fix patch as well. I'm using gcc 4.2.2. (Do you get
> the warning if you pick up the config i sent you earlier today?)
btw., i changed the initialization to -1 and the crash still occurs
(as expected).
Ingo
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > On Wed, Oct 17 2007, Ingo Molnar wrote:
> > >
> > > btw., i get the following build warning:
> > >
> > > drivers/scsi/scsi_lib.c: In function 'scsi_free_sgtable':
> > > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > > drivers/scsi/scsi_lib.c: In function 'scsi_alloc_sgtable':
> > > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > >
> > > not sure it matters.
> >
> > Hmm, I don't see them here (gcc 4.2.1). Must be the BUG(), does it go
> > away if you add a index = -1 in the default: case? Just to be
> > absolutely sure...
>
> well if i initialize the variable then of course the warning goes away
> :)
Just making 100% sure that was the missing place, I try not to take
anything for granted with crazy bugs like this :-)
>
> NOTE: i get the same warning even without the BUG_ON() patch, and
> without your other fix patch as well. I'm using gcc 4.2.2. (Do you get
> the warning if you pick up the config i sent you earlier today?)
Let me check... Yep, I do!
--
Jens Axboe
On Wed, Oct 17 2007, Ingo Molnar wrote:
>
> * Ingo Molnar <[email protected]> wrote:
>
> > > > drivers/scsi/scsi_lib.c:708: warning: 'index' may be used uninitialized in this function
> > > > drivers/scsi/scsi_lib.c:708: note: 'index' was declared here
> > > >
> > > > not sure it matters.
> > >
> > > Hmm, I don't see them here (gcc 4.2.1). Must be the BUG(), does it go
> > > away if you add a index = -1 in the default: case? Just to be
> > > absolutely sure...
> >
> > well if i initialize the variable then of course the warning goes away
> > :)
> >
> > NOTE: i get the same warning even without the BUG_ON() patch, and
> > without your other fix patch as well. I'm using gcc 4.2.2. (Do you get
> > the warning if you pick up the config i sent you earlier today?)
>
> btw., i changed the initialization to -1 and the crash still occurs
> (as expected).
Would think so, thanks for checking though.
--
Jens Axboe
On Wed, 17 Oct 2007, Ingo Molnar wrote:
>
> nope, this did not help. First bootup went fine, second bootup crashed
> again (see below), without hitting the BUG_ON().
I think you'll always hit it if you have a scatter-gather list that is
exactly filled up.
Why? Because those things do "sg_next()" on the last entry, and as
mentioned, that ends up actually accessing one past the end - even if the
end result is not actually ever *used* (because we just effectively
incremented it to past the last entry when the code was done with the SG
list).
So I think the sg_next() interface is fundamentally mis-designed. It
should do the scatter-gather link following on *starting* to use the SG
entry, not after finishing with it.
Put another way: I suspect pretty much every single sg_next() out there is
likely to hit this issue. The way that blk_rq_map_sg() fixed its problem
was exactly to move the "sg_next()" to *before* the use of the SG (and
even that one is somewhat bogus, in that it just blindly assumes that the
first entry is not a link entry).
I suspect the "the next entry is a link" bit should be in the *previous*
entry, and then sg_next() could look like
if (next_is_link(sg))
sg = sg_chain_ptr(sg+1);
else
sg++;
return sg;
and that would work.
The alternative is to always make sure to allocate one more SG entry than
required, so that the last entry is always either the link, or an unused
entry!
Linus
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Ingo Molnar wrote:
> >
> > nope, this did not help. First bootup went fine, second bootup crashed
> > again (see below), without hitting the BUG_ON().
>
> I think you'll always hit it if you have a scatter-gather list that is
> exactly filled up.
>
> Why? Because those things do "sg_next()" on the last entry, and as
> mentioned, that ends up actually accessing one past the end - even if the
> end result is not actually ever *used* (because we just effectively
> incremented it to past the last entry when the code was done with the SG
> list).
>
> So I think the sg_next() interface is fundamentally mis-designed. It
> should do the scatter-gather link following on *starting* to use the SG
> entry, not after finishing with it.
>
> Put another way: I suspect pretty much every single sg_next() out there is
> likely to hit this issue. The way that blk_rq_map_sg() fixed its problem
> was exactly to move the "sg_next()" to *before* the use of the SG (and
> even that one is somewhat bogus, in that it just blindly assumes that the
> first entry is not a link entry).
>
> I suspect the "the next entry is a link" bit should be in the *previous*
> entry, and then sg_next() could look like
>
> if (next_is_link(sg))
> sg = sg_chain_ptr(sg+1);
> else
> sg++;
> return sg;
>
> and that would work.
>
> The alternative is to always make sure to allocate one more SG entry than
> required, so that the last entry is always either the link, or an unused
> entry!
OK, I think you have a very good point here. Ingo, can you verify it
goes away if apply the below? I have to tend to Other Life stuff but
will take this up again tomorrow morning and fix the sg_next() usage.
Linus, please still pull the branch I asked you to earlier. Thanks!
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..58ede7e 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -39,7 +39,7 @@
* (unless chaining is used). Should ideally fit inside a single page, to
* avoid a higher order allocation.
*/
-#define SCSI_MAX_SG_SEGMENTS 128
+#define SCSI_MAX_SG_SEGMENTS 129
struct scsi_host_sg_pool {
size_t size;
--
Jens Axboe
On Wed, 17 Oct 2007, Linus Torvalds wrote:
>
> I think you'll always hit it if you have a scatter-gather list that is
> exactly filled up.
In fact, I think you'll hit it even if you use the "for_each_sg()"
helper function. Exactly because it does the sg = sg_next(sg) at the end
of the for-loop, so it will do it for the last entry too, even though that
will access one past the end.
So it really *is* the case that every *single* use of that SG chain needs
to be switched over to only do the sg_next() when required, or sg_next()
needs to be fixed to look at the current-in-use entry (which we *can*
access) when it decides what to do about the next one (which we can *not*
access).
Moving the sg_is_chain() bit to the previous entry would work, but it
requires that nobody who could have a chained scatterlist must *ever*
access "sg->page" directly - you'd always need to use some helper function
that masks off the bit, eg
- rename "sg->page" into "sh->page_and_flag", and make it "unsigned long"
instead of a pointer.
- change every single "sg->page" access to use "sg_page(sg)" instead:
#define sg_pointer(sg) ((void *)((sg)->page_and_flag & ~1ul))
static inline struct page *sg_page(struct scatterist *sg)
{
return sg_pointer(sg);
}
- change "sg_next()" to do
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
if (sg->page_and_flag & 1)
sg = sg_pointer(sg+1)-1;
return ++sg;
}
where the magic is exactly the fact that now "sg_next()" will *not*
derefence the next SG entry unless the current one was marked
appropriately.
And then *creating* the chain obviously needs to properly mark the
next-to-last entry with the "next entry is a pointer" flag.
Big diff, it sounds like. But I don't see many alternatives. Jens?
Linus
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> OK, I think you have a very good point here. Ingo, can you verify it
> goes away if apply the below? I have to tend to Other Life stuff but
> will take this up again tomorrow morning and fix the sg_next() usage.
This one will only fix the ones that use the SCSI-lib routines, so it will
leave everything else broken, no?
> Linus, please still pull the branch I asked you to earlier. Thanks!
Already did.
Linus
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Linus Torvalds wrote:
> >
> >
> > On Wed, 17 Oct 2007, Ingo Molnar wrote:
> > >
> > > nope, this did not help. First bootup went fine, second bootup crashed
> > > again (see below), without hitting the BUG_ON().
> >
> > I think you'll always hit it if you have a scatter-gather list that is
> > exactly filled up.
> >
> > Why? Because those things do "sg_next()" on the last entry, and as
> > mentioned, that ends up actually accessing one past the end - even if the
> > end result is not actually ever *used* (because we just effectively
> > incremented it to past the last entry when the code was done with the SG
> > list).
Well, hang on - where does it end up doing sg_next() on the LAST sg
entry? I'd argue that this is a bug, like it was in ll_rw_blk.c. I still
agree that I should make the interface more robust, I just don't see
where libata ends up doing the sg_next() on the last entry.
I'm assuming that Ingo is using the previous patches, so that
blk_rq_map_sg() is using this construct:
new_segment:
if (!sg)
sg = sglist;
else
sg = sg_next(sg);
and the memset() in scsi_alloc_sgtable(), which I'm pretty sure he is.
I'm assuming we're not hitting pio paths, so there are no manual
sg_next() calls. Ingo, if you care, can you test this one as well?
No rush, as mentioned I'll be back tomorrow morning...
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..0246b61 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4664,7 +4664,7 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
{
struct ata_port *ap = qc->ap;
struct scatterlist *sg = qc->__sg;
- struct scatterlist *lsg = sg_last(qc->__sg, qc->n_elem);
+ struct scatterlist *lsg = &qc->__sg[qc->n_elem - 1];
int n_elem, pre_n_elem, dir, trim_sg = 0;
VPRINTK("ENTER, ata%u\n", ap->print_id);
--
Jens Axboe
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Linus Torvalds wrote:
> >
> > I think you'll always hit it if you have a scatter-gather list that is
> > exactly filled up.
>
> In fact, I think you'll hit it even if you use the "for_each_sg()"
> helper function. Exactly because it does the sg = sg_next(sg) at the end
> of the for-loop, so it will do it for the last entry too, even though that
> will access one past the end.
>
> So it really *is* the case that every *single* use of that SG chain needs
> to be switched over to only do the sg_next() when required, or sg_next()
> needs to be fixed to look at the current-in-use entry (which we *can*
> access) when it decides what to do about the next one (which we can *not*
> access).
Auch yes, ok nevermind that last email. There really is no way around
allocating an extra 'pad' entry right now, so the SCSI_MAX_SG_SEGMENTS
at 129 should fix Ingo's oops and the other crap is void.
> Moving the sg_is_chain() bit to the previous entry would work, but it
> requires that nobody who could have a chained scatterlist must *ever*
> access "sg->page" directly - you'd always need to use some helper function
> that masks off the bit, eg
>
> - rename "sg->page" into "sh->page_and_flag", and make it "unsigned long"
> instead of a pointer.
>
> - change every single "sg->page" access to use "sg_page(sg)" instead:
>
> #define sg_pointer(sg) ((void *)((sg)->page_and_flag & ~1ul))
>
> static inline struct page *sg_page(struct scatterist *sg)
> {
> return sg_pointer(sg);
> }
>
> - change "sg_next()" to do
>
> static inline struct scatterlist *sg_next(struct scatterlist *sg)
> {
> if (sg->page_and_flag & 1)
> sg = sg_pointer(sg+1)-1;
> return ++sg;
> }
>
> where the magic is exactly the fact that now "sg_next()" will *not*
> derefence the next SG entry unless the current one was marked
> appropriately.
>
> And then *creating* the chain obviously needs to properly mark the
> next-to-last entry with the "next entry is a pointer" flag.
>
> Big diff, it sounds like. But I don't see many alternatives. Jens?
Big diff is not a problem for me, my primary concern would be making
sure that the interface is solid. The above also has the nice side
effect of making all sg elements usable, so we don't waste any. IIRC
this was even debated months ago when I first proposed sg chaining, I
shied away from it because I thought it would seem huge and too
invasive. Ah well. I'll get digging on this.
--
Jens Axboe
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > OK, I think you have a very good point here. Ingo, can you verify it
> > goes away if apply the below? I have to tend to Other Life stuff but
> > will take this up again tomorrow morning and fix the sg_next() usage.
>
> This one will only fix the ones that use the SCSI-lib routines, so it will
> leave everything else broken, no?
Yes, it'll still crap out if you use debug page alloc for anything else
:/
> > Linus, please still pull the branch I asked you to earlier. Thanks!
>
> Already did.
Thanks
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> Ingo, if you care, can you test this one as well?
hm, it still crashes - find the log below.
Ingo
[ 34.653682] EXT3-fs: mounted filesystem with ordered data mode.
[ 34.659487] VFS: Mounted root (ext3 filesystem) readonly.
[ 34.665208] Freeing unused kernel memory: 372k freed
[ 35.809988] BUG: unable to handle kernel paging request at virtual address 7c8d3000
[ 35.817483] printing eip: 784e9480 *pde = 00dda027 *pte = 048d3000
[ 35.823723] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 35.827883]
[ 35.829357] Pid: 47, comm: rc.sysinit Not tainted (2.6.23 #14)
[ 35.835162] EIP: 0060:[<784e9480>] EFLAGS: 00010006 CPU: 0
[ 35.840626] EIP is at ata_qc_issue+0xd0/0x340
[ 35.844954] EAX: 3fd79000 EBX: 7c8d3000 ECX: 00000020 EDX: 00000020
[ 35.851194] ESI: 7c8a1a80 EDI: 7c8d2e00 EBP: 7b54007c ESP: 7c8b3d5c
[ 35.857434] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068
[ 35.862807] Process rc.sysinit (pid: 47, ti=7c8b2000 task=7c86d000 task.ti=7c8b2000)
[ 35.870346] Stack: 7c8d3000 7b540000 7b540000 00000000 7c8d2ff0 7b54007c 7c8a1a80 7b5417a4
[ 35.878665] 784c2490 784ef61e 784f2173 7b52de98 7c8a1a80 7b540000 7b5417a4 7c8a1a80
[ 35.886985] 7b540000 7b524004 784f2260 784ef300 784c2490 7c8a1a80 00000216 7b524004
[ 35.895304] Call Trace:
[ 35.897905] [<784c2490>] scsi_done+0x0/0x20
[ 35.902151] [<784ef61e>] ata_scsi_translate+0xbe/0x140
[ 35.907350] [<784f2173>] ata_scsi_queuecmd+0x33/0x200
[ 35.912463] [<784f2260>] ata_scsi_queuecmd+0x120/0x200
[ 35.917663] [<784ef300>] ata_scsi_rw_xlat+0x0/0x220
[ 35.922602] [<784c2490>] scsi_done+0x0/0x20
[ 35.926849] [<784c2d12>] scsi_dispatch_cmd+0x152/0x290
[ 35.932049] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 35.937075] [<784c8c3e>] scsi_request_fn+0x1be/0x370
[ 35.942102] [<78120f02>] del_timer+0x62/0x70
[ 35.946434] [<784072d5>] __generic_unplug_device+0x25/0x30
[ 35.951981] [<784075a5>] generic_unplug_device+0x15/0x30
[ 35.957354] [<78404e00>] blk_backing_dev_unplug+0x0/0x10
[ 35.962726] [<78404e0c>] blk_backing_dev_unplug+0xc/0x10
[ 35.968100] [<78180a9d>] block_sync_page+0x2d/0x40
[ 35.972952] [<78144dc9>] sync_page+0x29/0x40
[ 35.977286] [<7876f5dc>] __wait_on_bit_lock+0x3c/0x70
[ 35.982399] [<78144da0>] sync_page+0x0/0x40
[ 35.986645] [<78144d82>] __lock_page+0x52/0x60
[ 35.991152] [<7812adb0>] wake_bit_function+0x0/0x60
[ 35.996092] [<78144f60>] find_lock_page+0x60/0xa0
[ 36.000858] [<781473b9>] filemap_fault+0x269/0x3b0
[ 36.005711] [<78150b7f>] __do_fault+0x4f/0x3e0
[ 36.010217] [<78147150>] filemap_fault+0x0/0x3b0
[ 36.014897] [<78152640>] handle_mm_fault+0xf0/0x6c0
[ 36.019837] [<7810fd5c>] do_page_fault+0x21c/0x670
[ 36.024689] [<7812e6fd>] down_read_trylock+0x4d/0x60
[ 36.029716] [<7810fdbb>] do_page_fault+0x27b/0x670
[ 36.034568] [<7810fb40>] do_page_fault+0x0/0x670
[ 36.039248] [<787710ea>] error_code+0x6a/0x70
[ 36.043669] =======================
[ 36.047221] Code: 84 d9 01 00 00 7e 32 31 d2 89 f6 8b 1c 24 83 c2 01 8b 03 2b 05 f8 ec d7 78 c1 f8 05 c1 e0 0c 03 43 04 89 43 08 83 c3 10 89 1c 24 <8b> 03 a8 01 0f 85 58 02 00 00 39 ca 75 d2 f0 83 44 24 00 00 85
[ 36.066027] EIP: [<784e9480>] ata_qc_issue+0xd0/0x340 SS:ESP 0068:7c8b3d5c
[ 100.077701] BUG: spinlock lockup on CPU#0, scsi_eh_0/32, 7b52de98
[ 100.083667] [<78414b9b>] _raw_spin_lock+0x13b/0x140
[ 100.088605] [<784c6610>] scsi_error_handler+0x0/0x4f0
On Wed, Oct 17 2007, Jens Axboe wrote:
> On Wed, Oct 17 2007, Linus Torvalds wrote:
> >
> >
> > On Wed, 17 Oct 2007, Jens Axboe wrote:
> > >
> > > OK, I think you have a very good point here. Ingo, can you verify it
> > > goes away if apply the below? I have to tend to Other Life stuff but
> > > will take this up again tomorrow morning and fix the sg_next() usage.
> >
> > This one will only fix the ones that use the SCSI-lib routines, so it will
> > leave everything else broken, no?
>
> Yes, it'll still crap out if you use debug page alloc for anything else
> :/
So until that is fixed up, how about this? Should cover all block
devices, and I don't think sg_next()/for_each_sg() has spread outside of
that yet.
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..8708ad0 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -631,7 +631,7 @@ void blk_queue_max_phys_segments(struct request_queue *q,
printk("%s: set to minimum %d\n", __FUNCTION__, max_segments);
}
- q->max_phys_segments = max_segments;
+ q->max_phys_segments = max_segments - 1;
}
EXPORT_SYMBOL(blk_queue_max_phys_segments);
--
Jens Axboe
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> Well, hang on - where does it end up doing sg_next() on the LAST sg
> entry?
They pretty much *all* do, as far as I can tell.
For example, to look at the very first one:
#define for_each_sg(sglist, sg, nr, __i) \
for (__i = 0, sg = (sglist); __i < (nr); __i++, sg = sg_next(sg))
let's say that "nr" is 1 (and that's also the allocation size), and look
at what that causes.
Right. It causes us to do "sg = sg_next(sg)" once. Which will do what? It
will increment sg (so that it now points past the single-entry array!) and
then it will dereference that invalid entry to see if it's a chain entry!
And no, "1" is not the special case. The special case is *any* time when
you iterate over as many sg entries as you allocated. You *always* have to
leave the last one unused in order to avoid this "access past the end"
problem.
The alternative is to rewrite the loop, but it's going to be ugly as hell,
and you need to do that for *every*single*user* of sg_next(). You can
re-write the above loop as something like
define for_each_sg(sglist, sg, nr, __i)
for (__i = 0, sg = NULL ;
__i < (nr) && sg = (sg ? sglist : sg_next(sg) ;
__i++))
but the important part here is that you must do that "sg_next()" *after*
you have broken out of the loop, and you must basically do it one less
time than you go through the loop.
IOW, any loop that leaves "sg" pointing to past the array is inevitably
buggy, because it will have accessed that last past-tne-end entry as part
of tryign to decide whether it should perhaps follow a link.
> I'd argue that this is a bug, like it was in ll_rw_blk.c. I still
> agree that I should make the interface more robust, I just don't see
> where libata ends up doing the sg_next() on the last entry.
See above. I think the exact sequence may be:
ata_qc_issue()
(implicitly inlined) ata_sg_setup()
(explicitly inlined) dma_map_sg()
(macro) for_each_sg()
but I didn't see if there are other possible chains that get you to one of
those invalid sg loops.
And no, it's *not* just for_each_sg(). Pretty much any "natural" loop over
the SG list will cause it, because that's how you write loops in C: you
almost always end up pointing to one past the last entry after the loop.
Linus
Il Wed, Oct 17, 2007 at 07:29:32PM +0200, Jens Axboe ha scritto:
> OK, it is fine, as long as the sglist is cleared initially. And I don't
> think there's anyway around that, clearly I didn't think long enough
> before including the memset() removal from Tomo.
>
> Ingo, please try this rolled up version.
>
> Linus, this should work. It would probably be best if you first did a
> git revert on f5c0dde4c66421a3a2d7d6fa604a712c9b0744e5 and then applied
> the ll_rw_blk.c bit alone. Do you want me to stuff that (revert + patch)
> into a branch for you to pull?
>
> diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
> index 9e3f3cc..3935469 100644
> --- a/block/ll_rw_blk.c
> +++ b/block/ll_rw_blk.c
> @@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> struct scatterlist *sglist)
> {
> struct bio_vec *bvec, *bvprv;
> - struct scatterlist *next_sg, *sg;
> struct req_iterator iter;
> + struct scatterlist *sg;
> int nsegs, cluster;
>
> nsegs = 0;
> @@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> * for each bio in rq
> */
> bvprv = NULL;
> - sg = next_sg = &sglist[0];
> + sg = NULL;
> rq_for_each_segment(bvec, rq, iter) {
> int nbytes = bvec->bv_len;
>
> @@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
> sg->length += nbytes;
> } else {
> new_segment:
> - sg = next_sg;
> - next_sg = sg_next(sg);
> + if (!sg)
> + sg = sglist;
> + else
> + sg = sg_next(sg);
>
> memset(sg, 0, sizeof(*sg));
> sg->page = bvec->bv_page;
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 0c86be7..aac8a02 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -764,6 +764,8 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
> if (unlikely(!sgl))
> goto enomem;
>
> + memset(sgl, 0, sizeof(*sgl) * sgp->size);
> +
> /*
> * first loop through, set initial index and return value
> */
>
Hi Jens,
I just hit what I believe is the same bug, though with a slightly
different OOPS (I see a NULL pointer dereference, see below). The
patches fixes the bug, and - as Linus suggested - the piece touching
scsi_lib.c is enough.
The OOPS:
scsi4 : ata_piix
scsi5 : ata_piix
ata5: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14
ata6: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15
sda1 sda2 sda3 < sda5 sda6 >
sd 0:0:0:0: [sda] Attached SCSI disk
ata5.00: ATAPI: MATSHITADVD-RAM UJ-850S, 1.22, max UDMA/33
Unable to handle kernel NULL pointer dereference at 0000000000000020 RIP:
[<ffffffff802d275e>] blk_rq_map_sg+0xf6/0x16a
PGD 4711067 PUD 508c067 PMD 0
Oops: 0000 [1] PREEMPT SMP
CPU 0
Modules linked in: sd_mod ohci1394 ehci_hcd ata_piix ieee1394 ahci uhci_hcd libata scsi_mod usbcore thermal processor fan
Pid: 0, comm: swapper Not tainted 2.6.23-ge6d5a11d #3
RIP: 0010:[<ffffffff802d275e>] [<ffffffff802d275e>] blk_rq_map_sg+0xf6/0x16a
RSP: 0018:ffffffff804eccd8 EFLAGS: 00010087
RAX: 0000000005331000 RBX: ffff810004740220 RCX: ffff810001000000
RDX: 0000000000000000 RSI: ffff8100052d4f50 RDI: 0000000005142c00
RBP: 0000000000001400 R08: 0000000000000000 R09: ffff8100052a3980
R10: 0000000005143000 R11: 0000000000000400 R12: ffff8100047a4000
R13: 0000000000000000 R14: 0000000000000002 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffffffff8049a000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000020 CR3: 0000000005118000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper (pid: 0, threadinfo ffffffff804ac000, task ffffffff8046c340)
Stack: ffffffff88041834 000000010314ba00 ffff81000477fa80 ffff81000503dd80
ffff81000503dd80 ffff81000517fc00 000000000000001e 0000000000001d4c
ffffffff8804193c 0000000015925849 ffff81000473f800 ffff81000503dd80
Call Trace:
<IRQ> [<ffffffff88041834>] :scsi_mod:scsi_alloc_sgtable+0xad/0x13c
[<ffffffff8804193c>] :scsi_mod:scsi_init_io+0x79/0xd9
[<ffffffff880d1328>] :sd_mod:sd_prep_fn+0x59/0x367
[<ffffffff802d07ff>] elv_next_request+0xc0/0x14a
[<ffffffff88042367>] :scsi_mod:scsi_request_fn+0x88/0x356
[<ffffffff802d46f7>] blk_run_queue+0x41/0x72
[<ffffffff88041135>] :scsi_mod:scsi_next_command+0x2d/0x39
[<ffffffff880412ae>] :scsi_mod:scsi_end_request+0xbb/0xc9
[<ffffffff880415ac>] :scsi_mod:scsi_io_completion+0xf8/0x2d3
[<ffffffff802d4a83>] blk_done_softirq+0x62/0x6f
[<ffffffff80238f28>] __do_softirq+0x59/0xc3
[<ffffffff8020d13c>] call_softirq+0x1c/0x28
[<ffffffff8020ec26>] do_softirq+0x2c/0x7d
[<ffffffff80238e30>] irq_exit+0x36/0x81
[<ffffffff8020edb4>] do_IRQ+0x13d/0x161
[<ffffffff8020c491>] ret_from_intr+0x0/0xa
<EOI> [<ffffffff88005249>] :processor:acpi_processor_idle+0x2cb/0x496
[<ffffffff88005245>] :processor:acpi_processor_idle+0x2c7/0x496
[<ffffffff88004f7e>] :processor:acpi_processor_idle+0x0/0x496
[<ffffffff8020ad00>] default_idle+0x0/0x3d
[<ffffffff8020adcd>] cpu_idle+0x90/0xcc
[<ffffffff804b4a60>] start_kernel+0x2ab/0x2b7
[<ffffffff804b411b>] _sinittext+0x11b/0x122
Code: 49 8b 40 20 4d 8d 50 20 4c 89 c7 fc b9 08 00 00 00 4c 89 c3
RIP [<ffffffff802d275e>] blk_rq_map_sg+0xf6/0x16a
RSP <ffffffff804eccd8>
CR2: 0000000000000020
Kernel panic - not syncing: Aiee, killing interrupt handler!
Luca
--
The trouble with computers is that they do what you tell them,
not what you want.
D. Cohen
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> So until that is fixed up, how about this? Should cover all block
> devices, and I don't think sg_next()/for_each_sg() has spread outside of
> that yet.
Heh. Not good. There are drivers that set max phys segmsnts to 1. Either
through some host-specific setting (MMC), or explicitly (eg a grep shows
viodasd uses just a constant 1 there).
Linus
On Wed, Oct 17 2007, Linus Torvalds wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > So until that is fixed up, how about this? Should cover all block
> > devices, and I don't think sg_next()/for_each_sg() has spread outside of
> > that yet.
>
> Heh. Not good. There are drivers that set max phys segmsnts to 1. Either
> through some host-specific setting (MMC), or explicitly (eg a grep shows
> viodasd uses just a constant 1 there).
That would hurt... Care to commit your for_each_sg() uglification fixup
for now then? Or disable the allocation debug config entry, so that the
sg+1 deref wont crash? Whichever your prefer, just don't want to cause
crash pains for the unfortunate souls that happen to run into this
before the real fix is done.
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> --- a/block/ll_rw_blk.c
> +++ b/block/ll_rw_blk.c
> @@ -631,7 +631,7 @@ void blk_queue_max_phys_segments(struct request_queue *q,
> printk("%s: set to minimum %d\n", __FUNCTION__, max_segments);
> }
>
> - q->max_phys_segments = max_segments;
> + q->max_phys_segments = max_segments - 1;
this one crashes too, if added ontop of all the other fixes so far.
Ingo
[ 34.711185] EXT3-fs: mounted filesystem with ordered data mode.
[ 34.716991] VFS: Mounted root (ext3 filesystem) readonly.
[ 34.722712] Freeing unused kernel memory: 372k freed
[ 36.112742] BUG: unable to handle kernel paging request at virtual address 7c8b1000
[ 36.120246] printing eip: 784e9490 *pde = 00dda027 *pte = 048b1000
[ 36.126486] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 36.130645]
[ 36.132119] Pid: 0, comm: swapper Not tainted (2.6.23 #15)
[ 36.137579] EIP: 0060:[<784e9490>] EFLAGS: 00010006 CPU: 0
[ 36.143043] EIP is at ata_qc_issue+0xd0/0x340
[ 36.147371] EAX: 3f896000 EBX: 7c8b1000 ECX: 00000008 EDX: 00000008
[ 36.153610] ESI: 7c86de80 EDI: 7c8b0f80 EBP: 7b54007c ESP: 78a13e28
[ 36.159851] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 36.165223] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
[ 36.172416] Stack: 7c8b1000 7b540000 7b540000 00000000 7c8b0ff0 7b54007c 7c86de80 7b5417a4
[ 36.180735] 784c24a0 784ef62e 784f2183 7b52de98 7c86de80 7b540000 7b5417a4 7c86de80
[ 36.189055] 7b540000 7b524004 784f2270 784ef310 784c24a0 7c86de80 00000206 7b524004
[ 36.197374] Call Trace:
[ 36.199975] [<784c24a0>] scsi_done+0x0/0x20
[ 36.204221] [<784ef62e>] ata_scsi_translate+0xbe/0x140
[ 36.209420] [<784f2183>] ata_scsi_queuecmd+0x33/0x200
[ 36.214533] [<784f2270>] ata_scsi_queuecmd+0x120/0x200
[ 36.219733] [<784ef310>] ata_scsi_rw_xlat+0x0/0x220
[ 36.224672] [<784c24a0>] scsi_done+0x0/0x20
[ 36.228918] [<784c2d22>] scsi_dispatch_cmd+0x152/0x290
[ 36.234118] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 36.239145] [<784c8c4e>] scsi_request_fn+0x1be/0x370
[ 36.244171] [<78408096>] blk_run_queue+0x36/0x80
[ 36.248850] [<784c7530>] scsi_next_command+0x30/0x50
[ 36.253877] [<784c76bb>] scsi_end_request+0xab/0xe0
[ 36.258817] [<784c83c9>] scsi_io_completion+0xa9/0x3d0
[ 36.264016] [<78135c67>] trace_hardirqs_on+0x67/0xb0
[ 36.269043] [<78405125>] blk_done_softirq+0x45/0x80
[ 36.273983] [<78405153>] blk_done_softirq+0x73/0x80
[ 36.278922] [<7811d4c3>] __do_softirq+0x53/0xb0
[ 36.283515] [<7811d588>] do_softirq+0x68/0x70
[ 36.287935] [<78105351>] do_IRQ+0x51/0x90
[ 36.292008] [<78135c9c>] trace_hardirqs_on+0x9c/0xb0
[ 36.297034] [<7810388e>] common_interrupt+0x2e/0x40
[ 36.301974] [<78100c55>] cpu_idle+0x35/0x60
[ 36.306220] [<78a14b35>] start_kernel+0x265/0x300
[ 36.310987] [<78a14380>] unknown_bootoption+0x0/0x1e0
[ 36.316100] =======================
[ 36.319653] Code: 84 d9 01 00 00 7e 32 31 d2 89 f6 8b 1c 24 83 c2 01 8b 03 2b 05 f8 ec d7 78 c1 f8 05 c1 e0 0c 03 43 04 89 43 08 83 c3 10 89 1c 24 <8b> 03 a8 01 0f 85 58 02 00 00 39 ca 75 d2 f0 83 44 24 00 00 85
[ 36.338458] EIP: [<784e9490>] ata_qc_issue+0xd0/0x340 SS:ESP 0068:78a13e28
[ 36.345305] Kernel panic - not syncing: Fatal exception in interrupt
On Wed, 17 Oct 2007, Jens Axboe wrote:
>
> That would hurt... Care to commit your for_each_sg() uglification fixup
> for now then? Or disable the allocation debug config entry, so that the
> sg+1 deref wont crash?
Well, in practice, it will only crash with DEBUG_PAGEALLOC, so few enough
are going to be hit by it. In that sense I don't think we're in any deep
trouble yet.
That said, maybe this is an acceptable, if hacky, replacement for the
current "for_each_sg()" loop.
It does:
- starts at one *before* the sglist
- does the sg_next() at the *top* of the loop rather than the bottom of it
- has a "--count" before that sg_next, so that we don't do it for the
case when we break out and have used up all segments.
Totally untested, but it *may* work, and it doesn't look horribly ugly.
Ingo, does this actually make any difference?
Linus
---
include/linux/scatterlist.h | 9 ++++++++-
1 files changed, 8 insertions(+), 1 deletions(-)
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..f5c8e11 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -51,11 +51,18 @@ static inline struct scatterlist *sg_next(struct scatterlist *sg)
return sg;
}
+static inline struct scatterlist *sg_safe_next(struct scatterlist *sg, int left)
+{
+ if (left < 0)
+ return NULL;
+ return sg_next(sg);
+}
+
/*
* Loop over each sg element, following the pointer to a new list if necessary
*/
#define for_each_sg(sglist, sg, nr, __i) \
- for (__i = 0, sg = (sglist); __i < (nr); __i++, sg = sg_next(sg))
+ for (__i = (nr), sg = (sglist)-1; (sg = sg_safe_next(sg, --__i)) != NULL ; )
/**
* sg_last - return the last scatterlist entry in a list
On Wed, 17 Oct 2007 14:11:34 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:
>
>
> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >
> > That would hurt... Care to commit your for_each_sg() uglification fixup
> > for now then? Or disable the allocation debug config entry, so that the
> > sg+1 deref wont crash?
>
> Well, in practice, it will only crash with DEBUG_PAGEALLOC, so few enough
> are going to be hit by it. In that sense I don't think we're in any deep
> trouble yet.
>
> That said, maybe this is an acceptable, if hacky, replacement for the
> current "for_each_sg()" loop.
>
> It does:
> - starts at one *before* the sglist
> - does the sg_next() at the *top* of the loop rather than the bottom of it
> - has a "--count" before that sg_next, so that we don't do it for the
> case when we break out and have used up all segments.
>
> Totally untested, but it *may* work, and it doesn't look horribly ugly.
>
> Ingo, does this actually make any difference?
>
> Linus
>
> ---
> include/linux/scatterlist.h | 9 ++++++++-
> 1 files changed, 8 insertions(+), 1 deletions(-)
>
> diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
> index 2dc7464..f5c8e11 100644
> --- a/include/linux/scatterlist.h
> +++ b/include/linux/scatterlist.h
> @@ -51,11 +51,18 @@ static inline struct scatterlist *sg_next(struct scatterlist *sg)
> return sg;
> }
>
> +static inline struct scatterlist *sg_safe_next(struct scatterlist *sg, int left)
> +{
> + if (left < 0)
> + return NULL;
> + return sg_next(sg);
> +}
> +
> /*
> * Loop over each sg element, following the pointer to a new list if necessary
> */
> #define for_each_sg(sglist, sg, nr, __i) \
> - for (__i = 0, sg = (sglist); __i < (nr); __i++, sg = sg_next(sg))
> + for (__i = (nr), sg = (sglist)-1; (sg = sg_safe_next(sg, --__i)) != NULL ; )
Looks that (sglist) - 1 isn't initialized and we use sg_next for it?
On Thu, 18 Oct 2007, FUJITA Tomonori wrote:
>
> Looks that (sglist) - 1 isn't initialized and we use sg_next for it?
sg_next() - as it stands now - never actually looks at the SG that its
argument points to: it explicitly *only* looks at the next one.
That's the bug. If sg_next() looked at the actual *current* sg entry, we
wouldn't have any issues to begin with, and that's what I'm arguing we
should do in the longer run (where "longer run" is defined as "when Jens
does it asap").
So the hacky solution as it stands right now is to actually use the
behaviour of "sg_next()" to our advantage in for_each_sg(), and starting
off by setting sg to (sglist)-1. That way we can do the "sg_next()" (which
will *not* look at the uninitialized space before the array) before
entering the loop, regardless of whether it's the first time through the
loop or not.
So no, it's not technically "strictly conforming ANSI C", because we use a
pointer (not do not dereference it!) outside of its allocation, which is
strictly speaking against the standard. However, the kernel does those
kinds of things all the time, since the kernel does its own memory
allocation, so that's not actually relevant.
The *standard* may say that you cannot decrement a pointer to before the
beginning of the object and then increment it back up again and be
strictly conforming, but the fact is, we very much depend on contiguous
and flat kernel pointer model, and always have (and probably always will)
Linus
On a related hardware note:
FWIW, most ATA controllers have scatter/gather tables that terminate
themselves by a bit in the final s/g entry. The 90% case needs to know
the last scatterlist entry, at the end of the s/g walk.
So however this all gets worked out, please make sure not to unduly
punish libata controllers with an expensive sg_last() inside a loop, or
something like that. :)
Jeff
From: Linus Torvalds <[email protected]>
Date: Wed, 17 Oct 2007 18:07:19 -0700 (PDT)
> sg_next() - as it stands now - never actually looks at the SG that its
> argument points to: it explicitly *only* looks at the next one.
>
> That's the bug. If sg_next() looked at the actual *current* sg entry, we
> wouldn't have any issues to begin with, and that's what I'm arguing we
> should do in the longer run (where "longer run" is defined as "when Jens
> does it asap").
What the thing really wants is some kind of indication of state,
without having to bloat up the scatterlist structure.
I believe that we have enough of a limited set of accessors to
sg->page that we can more aggressively encode things in the lower
bits.
I'm thinking of encoding the low two bits of sg->page as
follows:
1) bits == 0
then the SG list is linear and sg_next() is sg++
2) bits == 1
the nest SG is an indirect chunk, sg_next() is
therefore something like:
next = *((struct scatterlist **)(sg + 1));
3) bits == 2
this is the last entry in the scatterlist, sg_next() is NULL
So for the cases of ARCH_HAS_SG_CHAIN not being set (ie. back
compatible), we can do no bit encoding in page->flags and just do
sg_next() == sg++, as is done now.
When doing SG chaining, in each non-linear chunk we have to allocate
one more pointer past the end of the scatterlist array (instead of a
full extra scatterlist entry for the indirect pointer encode). Next,
all sg->page accesses have to be guarded to clear the state bits
out first.
I don't know, maybe it would work, and would make the loop termination
issues easier to handle properly.
On Wed, 17 Oct 2007, David Miller wrote:
>
> I believe that we have enough of a limited set of accessors to
> sg->page that we can more aggressively encode things in the lower
> bits.
>
> I'm thinking of encoding the low two bits of sg->page as
> follows:
>
> 1) bits == 0
>
> then the SG list is linear and sg_next() is sg++
>
> 2) bits == 1
>
> the nest SG is an indirect chunk, sg_next() is
> therefore something like:
>
> next = *((struct scatterlist **)(sg + 1));
>
> 3) bits == 2
>
> this is the last entry in the scatterlist, sg_next() is NULL
>
> So for the cases of ARCH_HAS_SG_CHAIN not being set (ie. back
> compatible), we can do no bit encoding in page->flags and just do
> sg_next() == sg++, as is done now.
Yes, that sounds sane.
Although I also wonder whether we want one global per-arch
ARCH_HAS_SG_CHAIN, or perhaps have something more dynamic, which allows a
per-SG-list choice (which in turn would require some kind of "head" entry
that is passed into sg_next(), and in general descibes the SG list)
Linus
From: Linus Torvalds <[email protected]>
Date: Wed, 17 Oct 2007 18:36:34 -0700 (PDT)
> Although I also wonder whether we want one global per-arch
> ARCH_HAS_SG_CHAIN
It's there because the DMA mapping support code for a platform has to
be converted to handle these chains and audited to make sure the
conversion is right. Some platforms, such as sparc64, took a lot of
work to get right :-)
Once that macro is set, the block device driver has to support it too,
and there are knobs all the way down to the scsi host driver to
indicate this.
The idea is that all of this mess gets deleted in the end, it was
meant to be a safe transition scheme.
Linus Torvalds wrote:
>
> On Wed, 17 Oct 2007, David Miller wrote:
>> I believe that we have enough of a limited set of accessors to
>> sg->page that we can more aggressively encode things in the lower
>> bits.
>>
>> I'm thinking of encoding the low two bits of sg->page as
>> follows:
...
> Yes, that sounds sane.
It would be good to have something soon-ish.
This "dead at boot time" issue is impacting the general ability to test
patches against latest -git in time for the current merge window.
:)
On Wed, 17 Oct 2007, Mark Lord wrote:
>
> It would be good to have something soon-ish.
> This "dead at boot time" issue is impacting the general ability to test
> patches against latest -git in time for the current merge window.
In the meantime, does the patch I sent out help people? I'd like to get
feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC myself, so I
was hoping that people who actually see this could comment on my untested
suggestion.
Linus
Linus Torvalds wrote:
>
> On Wed, 17 Oct 2007, Mark Lord wrote:
>> It would be good to have something soon-ish.
>> This "dead at boot time" issue is impacting the general ability to test
>> patches against latest -git in time for the current merge window.
>
> In the meantime, does the patch I sent out help people? I'd like to get
> feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC myself, so I
> was hoping that people who actually see this could comment on my untested
> suggestion.
Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
Then something else is broken, perhaps. I just saw a long traceback
scroll off the top of the screen, with lots of bio_* functions in the list
and assumed it was the same bug.
Mmm.. I'll get out the camera and try it again now..
Mark Lord wrote:
> Linus Torvalds wrote:
>>
>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>> It would be good to have something soon-ish.
>>> This "dead at boot time" issue is impacting the general ability to test
>>> patches against latest -git in time for the current merge window.
>>
>> In the meantime, does the patch I sent out help people? I'd like to
>> get feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC
>> myself, so I was hoping that people who actually see this could
>> comment on my untested suggestion.
>
> Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
>
> Then something else is broken, perhaps. I just saw a long traceback
> scroll off the top of the screen, with lots of bio_* functions in the list
> and assumed it was the same bug.
>
> Mmm.. I'll get out the camera and try it again now..
I'm currently bisecting on two machines (sata_mv and sata_nv). sata_mv
suddenly started spewing errors in recent kernels, but kernels from ~48
hours ago are just fine. AMD64 sata_nv machine will simply lock up if I
push it too hard. Again, reproducible, and kernels from ~48 hours ago
are fine.
AHCI machine so far seems fine (kernel ~24 hours ago), storage-wise, but
the hda_intel audio decided to stop working :)
I'm quite happy to test patches, but I never run with DEBUG_PAGEALLOC so
that shouldn't be causing the problems here.
Jeff
Mark Lord wrote:
> Linus Torvalds wrote:
>>
>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>> It would be good to have something soon-ish.
>>> This "dead at boot time" issue is impacting the general ability to test
>>> patches against latest -git in time for the current merge window.
>>
>> In the meantime, does the patch I sent out help people? I'd like to
>> get feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC
>> myself, so I was hoping that people who actually see this could
>> comment on my untested suggestion.
>
> Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
>
> Then something else is broken, perhaps. I just saw a long traceback
> scroll off the top of the screen, with lots of bio_* functions in the list
> and assumed it was the same bug.
>
> Mmm.. I'll get out the camera and try it again now..
Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
but the top was cut off (isn't there a new config option or patch
to do double-columns or scrollback or something ???.
Cheers
On Thu, 18 Oct 2007, Mark Lord wrote:
>
> Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
Yeah, this particular one really should only bite you with
DEBUG_PAGEALLOC.
The SG code potentially _derefences_ a field past the end of the SG array,
but it should be a read, and the result should never be used (it's used to
calculate the pointer to past the end of the queue, so if it's used,
there's another bug lurking).
> Then something else is broken, perhaps. I just saw a long traceback
> scroll off the top of the screen, with lots of bio_* functions in the
> list and assumed it was the same bug.
>
> Mmm.. I'll get out the camera and try it again now..
Please do. I wouldn't be surprised if it's also related to the SG changes,
but I don't think it's the exact particular bug that Ingo saw.
Linus
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..d8cf8b1 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -421,7 +421,6 @@ static void mv_error_handler(struct ata_port *ap);
static void mv_post_int_cmd(struct ata_queued_cmd *qc);
static void mv_eh_freeze(struct ata_port *ap);
static void mv_eh_thaw(struct ata_port *ap);
-static int mv_slave_config(struct scsi_device *sdev);
static int mv_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
static void mv5_phy_errata(struct mv_host_priv *hpriv, void __iomem *mmio,
@@ -459,7 +458,7 @@ static struct scsi_host_template mv5_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -477,7 +476,7 @@ static struct scsi_host_template mv6_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -756,17 +755,6 @@ static void mv_irq_clear(struct ata_port *ap)
{
}
-static int mv_slave_config(struct scsi_device *sdev)
-{
- int rc = ata_scsi_slave_config(sdev);
- if (rc)
- return rc;
-
- blk_queue_max_phys_segments(sdev->request_queue, MV_MAX_SG_CT / 2);
-
- return 0; /* scsi layer doesn't check return value, sigh */
-}
-
static void mv_set_edma_ptrs(void __iomem *port_mmio,
struct mv_host_priv *hpriv,
struct mv_port_priv *pp)
Jeff Garzik wrote:
> Mark Lord wrote:
>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>
>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>> but the top was cut off (isn't there a new config option or patch
>> to do double-columns or scrollback or something ???.
>
> Is this a sata_mv box? If so, could you try this patch?
No.. it's a pretty ordinary ata_piix machine, with no serial ports either.
Cheers
On Thu, 18 Oct 2007, Mark Lord wrote:
>
> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
Ok, I think your picture cut off the last hex digits on the right, but
what I can make out of the disassembly, I have to admit that it looks
very much like it might be exactly the same thing Ingo was seeing.
The disassembly is a bit scrambled, but the particular instruction that
oopses for you is
mov 0x10(%ebx),%eax
and the instructions around it do seem to bear some similarity to the
whole "sg_next()" thing, ie I see a
test $0x1,%al
and a
lea 0x10(%ebx),%eax
around there too.
If you cut down the number of entries printed for the call trace (which
involves editing the source code, no nice config options, I'm afraid), you
could get the rest of it..
Linus
On Thu, 18 Oct 2007, Jeff Garzik wrote:
>
> Is this a sata_mv box? If so, could you try this patch?
That could explain it: if the SG allocation is simply too small, the
scatter-gather code will run off the end of the SG list, and encounter
random uninitialized entries, and if any of those have the low bit set,
they will be considered to be "link" entries, and the SG list goes wild.
That SG code is really pretty fragile. I don't see it *ever* checking that
the SG allocation is large enough when it fills it in. And these things
take the sizes from different places (ie "cmd->use_sg" bs
"req->nr_phys_segments" vs God knows what else..)
Linus
Mark Lord wrote:
> Mark Lord wrote:
>> Linus Torvalds wrote:
>>>
>>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>>> It would be good to have something soon-ish.
>>>> This "dead at boot time" issue is impacting the general ability to test
>>>> patches against latest -git in time for the current merge window.
>>>
>>> In the meantime, does the patch I sent out help people? I'd like to
>>> get feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC
>>> myself, so I was hoping that people who actually see this could
>>> comment on my untested suggestion.
>>
>> Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
>>
>> Then something else is broken, perhaps. I just saw a long traceback
>> scroll off the top of the screen, with lots of bio_* functions in the
>> list
>> and assumed it was the same bug.
>>
>> Mmm.. I'll get out the camera and try it again now..
>
> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>
> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
> but the top was cut off (isn't there a new config option or patch
> to do double-columns or scrollback or something ???.
I tried the fancy "boot_delay=nnn" feature, but that doesn't slow down
the tracebacks at all. So I hardcoded some mdelay(1000) lines into
the traceback code, and here's the top part of the oops now:
http://rtr.ca/recent/2.6.23-git12-crash2.jpg
And here's the .config for the kernel, just in case it's of any use here:
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.23-git12
# Thu Oct 18 00:28:52 2007
#
CONFIG_X86_32=y
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_QUICKLIST=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set
# CONFIG_USER_NS is not set
# CONFIG_AUDIT is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=16
# CONFIG_CPUSETS is not set
# CONFIG_FAIR_GROUP_SCHED is not set
CONFIG_SYSFS_DEPRECATED=y
# CONFIG_RELAY is not set
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_EMBEDDED=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_ANON_INODES=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_VM_EVENT_COUNTERS=y
# CONFIG_SLUB_DEBUG is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_RT_MUTEXES=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
# CONFIG_LBD is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_LSF is not set
CONFIG_BLK_DEV_BSG=y
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
CONFIG_SCHED_NO_NO_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
CONFIG_MCORE2=y
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
# CONFIG_ARCH_HAS_ILOG2_U32 is not set
# CONFIG_ARCH_HAS_ILOG2_U64 is not set
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_MINIMUM_CPU_FAMILY=4
CONFIG_HPET_TIMER=y
CONFIG_NR_CPUS=2
# CONFIG_SCHED_SMT is not set
CONFIG_SCHED_MC=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_MCE is not set
CONFIG_VM86=y
# CONFIG_TOSHIBA is not set
CONFIG_I8K=m
CONFIG_X86_REBOOTFIXUPS=y
CONFIG_MICROCODE=m
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m
#
# Firmware Drivers
#
CONFIG_EDD=m
CONFIG_DELL_RBU=m
CONFIG_DCDBAS=m
CONFIG_DMIID=y
# CONFIG_NOHIGHMEM is not set
CONFIG_HIGHMEM4G=y
# CONFIG_HIGHMEM64G is not set
CONFIG_VMSPLIT_3G=y
# CONFIG_VMSPLIT_3G_OPT is not set
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_2G_OPT is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_HIGHMEM=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
# CONFIG_SPARSEMEM_VMEMMAP_ENABLE is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_RESOURCES_64BIT is not set
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_NR_QUICK=1
CONFIG_VIRT_TO_BUS=y
# CONFIG_HIGHPTE is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
# CONFIG_IRQBALANCE is not set
# CONFIG_SECCOMP is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
CONFIG_HZ_300=y
# CONFIG_HZ_1000 is not set
CONFIG_HZ=300
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x100000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x100000
CONFIG_HOTPLUG_CPU=y
# CONFIG_COMPAT_VDSO is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_SLEEP=y
CONFIG_SUSPEND_SMP_POSSIBLE=y
CONFIG_SUSPEND=y
CONFIG_HIBERNATION_SMP_POSSIBLE=y
CONFIG_HIBERNATION=y
CONFIG_PM_STD_PARTITION="/dev/sda3"
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_FAN=m
# CONFIG_ACPI_DOCK is not set
CONFIG_ACPI_PROCESSOR=m
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_THERMAL=m
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=m
# CONFIG_ACPI_SBS is not set
# CONFIG_APM is not set
#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=m
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=m
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m
#
# CPUFreq processor drivers
#
CONFIG_X86_ACPI_CPUFREQ=m
# CONFIG_X86_POWERNOW_K6 is not set
# CONFIG_X86_POWERNOW_K7 is not set
# CONFIG_X86_POWERNOW_K8 is not set
# CONFIG_X86_GX_SUSPMOD is not set
CONFIG_X86_SPEEDSTEP_CENTRINO=m
CONFIG_X86_SPEEDSTEP_CENTRINO_TABLE=y
CONFIG_X86_SPEEDSTEP_ICH=m
# CONFIG_X86_SPEEDSTEP_SMI is not set
# CONFIG_X86_P4_CLOCKMOD is not set
# CONFIG_X86_CPUFREQ_NFORCE2 is not set
# CONFIG_X86_LONGRUN is not set
# CONFIG_X86_LONGHAUL is not set
# CONFIG_X86_E_POWERSAVER is not set
#
# shared options
#
# CONFIG_X86_ACPI_CPUFREQ_PROC_INTF is not set
CONFIG_X86_SPEEDSTEP_LIB=m
# CONFIG_X86_SPEEDSTEP_RELAXED_CAP_CHECK is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
CONFIG_PCIEAER=y
CONFIG_ARCH_SUPPORTS_MSI=y
# CONFIG_PCI_MSI is not set
# CONFIG_PCI_DEBUG is not set
# CONFIG_HT_IRQ is not set
CONFIG_ISA_DMA_API=y
# CONFIG_ISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
# CONFIG_PCCARD is not set
CONFIG_HOTPLUG_PCI=m
# CONFIG_HOTPLUG_PCI_FAKE is not set
# CONFIG_HOTPLUG_PCI_COMPAQ is not set
# CONFIG_HOTPLUG_PCI_IBM is not set
# CONFIG_HOTPLUG_PCI_ACPI is not set
# CONFIG_HOTPLUG_PCI_CPCI is not set
# CONFIG_HOTPLUG_PCI_SHPC is not set
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m
#
# Networking
#
CONFIG_NET=y
#
# Networking options
#
CONFIG_PACKET=m
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=m
CONFIG_XFRM=y
CONFIG_XFRM_USER=m
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
CONFIG_IP_SCTP=m
# CONFIG_SCTP_DBG_MSG is not set
# CONFIG_SCTP_DBG_OBJCNT is not set
# CONFIG_SCTP_HMAC_NONE is not set
# CONFIG_SCTP_HMAC_SHA1 is not set
CONFIG_SCTP_HMAC_MD5=y
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=m
CONFIG_VLAN_8021Q=m
# CONFIG_DECNET is not set
CONFIG_LLC=m
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_FIFO=y
#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RR=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
CONFIG_NET_SCH_NETEM=m
CONFIG_NET_SCH_INGRESS=m
#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_EMATCH=y
CONFIG_NET_EMATCH_STACK=32
CONFIG_NET_EMATCH_CMP=m
CONFIG_NET_EMATCH_NBYTE=m
CONFIG_NET_EMATCH_U32=m
CONFIG_NET_EMATCH_META=m
CONFIG_NET_EMATCH_TEXT=m
CONFIG_NET_CLS_ACT=y
CONFIG_NET_ACT_POLICE=m
CONFIG_NET_ACT_GACT=m
CONFIG_GACT_PROB=y
CONFIG_NET_ACT_MIRRED=m
# CONFIG_NET_ACT_NAT is not set
CONFIG_NET_ACT_PEDIT=m
CONFIG_NET_ACT_SIMP=m
# CONFIG_NET_CLS_POLICE is not set
# CONFIG_NET_CLS_IND is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
CONFIG_BT=m
CONFIG_BT_L2CAP=m
CONFIG_BT_SCO=m
CONFIG_BT_RFCOMM=m
CONFIG_BT_RFCOMM_TTY=y
CONFIG_BT_BNEP=m
CONFIG_BT_BNEP_MC_FILTER=y
CONFIG_BT_BNEP_PROTO_FILTER=y
CONFIG_BT_HIDP=m
#
# Bluetooth device drivers
#
CONFIG_BT_HCIUSB=m
CONFIG_BT_HCIUSB_SCO=y
CONFIG_BT_HCIUART=m
CONFIG_BT_HCIUART_H4=y
CONFIG_BT_HCIUART_BCSP=y
CONFIG_BT_HCIBCM203X=m
CONFIG_BT_HCIBPA10X=m
CONFIG_BT_HCIBFUSB=m
CONFIG_BT_HCIVHCI=m
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y
#
# Wireless
#
CONFIG_CFG80211=m
CONFIG_NL80211=y
CONFIG_WIRELESS_EXT=y
CONFIG_MAC80211=m
# CONFIG_MAC80211_DEBUG is not set
CONFIG_IEEE80211=m
# CONFIG_IEEE80211_DEBUG is not set
CONFIG_IEEE80211_CRYPT_WEP=m
CONFIG_IEEE80211_CRYPT_CCMP=m
CONFIG_IEEE80211_CRYPT_TKIP=m
CONFIG_IEEE80211_SOFTMAC=m
# CONFIG_IEEE80211_SOFTMAC_DEBUG is not set
CONFIG_RFKILL=m
CONFIG_RFKILL_INPUT=m
# CONFIG_NET_9P is not set
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_STANDALONE=y
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set
#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=4
CONFIG_BLK_DEV_RAM_SIZE=8192
CONFIG_BLK_DEV_RAM_BLOCKSIZE=1024
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
CONFIG_ATA_OVER_ETH=m
CONFIG_MISC_DEVICES=y
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
CONFIG_EEPROM_93CX6=m
# CONFIG_SGI_IOC4 is not set
CONFIG_TIFM_CORE=m
# CONFIG_TIFM_7XX1 is not set
# CONFIG_MSI_LAPTOP is not set
# CONFIG_SONY_LAPTOP is not set
# CONFIG_THINKPAD_ACPI is not set
# CONFIG_IDE is not set
#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=m
# CONFIG_CHR_DEV_SCH is not set
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m
#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
# CONFIG_SATA_AHCI is not set
# CONFIG_SATA_SVW is not set
CONFIG_ATA_PIIX=y
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not set
CONFIG_SATA_SIL24=m
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CS5535 is not set
# CONFIG_PATA_CS5536 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_PLATFORM is not set
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
# CONFIG_MD_LINEAR is not set
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
# CONFIG_MD_RAID10 is not set
# CONFIG_MD_RAID456 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=m
# CONFIG_DM_DEBUG is not set
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
# CONFIG_DM_ZERO is not set
# CONFIG_DM_MULTIPATH is not set
# CONFIG_DM_DELAY is not set
# CONFIG_FUSION is not set
#
# IEEE 1394 (FireWire) support
#
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_SBP2=m
# CONFIG_IEEE1394 is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
# CONFIG_NETDEVICES_MULTIQUEUE is not set
# CONFIG_IFB is not set
CONFIG_DUMMY=m
CONFIG_BONDING=m
CONFIG_MACVLAN=m
CONFIG_EQUALIZER=m
CONFIG_TUN=m
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_IP1000 is not set
# CONFIG_ARCNET is not set
CONFIG_PHYLIB=m
#
# MII PHY device drivers
#
# CONFIG_MARVELL_PHY is not set
# CONFIG_DAVICOM_PHY is not set
# CONFIG_QSEMI_PHY is not set
# CONFIG_LXT_PHY is not set
# CONFIG_CICADA_PHY is not set
# CONFIG_VITESSE_PHY is not set
# CONFIG_SMSC_PHY is not set
# CONFIG_BROADCOM_PHY is not set
# CONFIG_ICPLUS_PHY is not set
CONFIG_FIXED_PHY=m
CONFIG_FIXED_MII_10_FDX=y
CONFIG_FIXED_MII_100_FDX=y
CONFIG_FIXED_MII_1000_FDX=y
CONFIG_FIXED_MII_AMNT=1
# CONFIG_MDIO_BITBANG is not set
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_NET_TULIP is not set
# CONFIG_HP100 is not set
# CONFIG_IBM_NEW_EMAC_ZMII is not set
# CONFIG_IBM_NEW_EMAC_RGMII is not set
# CONFIG_IBM_NEW_EMAC_TAH is not set
# CONFIG_IBM_NEW_EMAC_EMAC4 is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
CONFIG_B44=m
CONFIG_B44_PCI_AUTOSELECT=y
CONFIG_B44_PCICORE_AUTOSELECT=y
CONFIG_B44_PCI=y
# CONFIG_FORCEDETH is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_SC92031 is not set
# CONFIG_NETDEV_1000 is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set
#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
CONFIG_WLAN_80211=y
CONFIG_IPW2100=m
CONFIG_IPW2100_MONITOR=y
# CONFIG_IPW2100_DEBUG is not set
CONFIG_IPW2200=m
CONFIG_IPW2200_MONITOR=y
CONFIG_IPW2200_RADIOTAP=y
CONFIG_IPW2200_PROMISCUOUS=y
CONFIG_IPW2200_QOS=y
# CONFIG_IPW2200_DEBUG is not set
# CONFIG_LIBERTAS is not set
# CONFIG_AIRO is not set
# CONFIG_HERMES is not set
# CONFIG_ATMEL is not set
# CONFIG_PRISM54 is not set
CONFIG_USB_ZD1201=m
CONFIG_RTL8187=m
# CONFIG_ADM8211 is not set
# CONFIG_P54_COMMON is not set
# CONFIG_IWLWIFI is not set
# CONFIG_HOSTAP is not set
CONFIG_BCM43XX=m
# CONFIG_BCM43XX_DEBUG is not set
CONFIG_BCM43XX_DMA=y
CONFIG_BCM43XX_PIO=y
CONFIG_BCM43XX_DMA_AND_PIO_MODE=y
# CONFIG_BCM43XX_DMA_MODE is not set
# CONFIG_BCM43XX_PIO_MODE is not set
CONFIG_B43=m
CONFIG_B43_PCI_AUTOSELECT=y
CONFIG_B43_PCICORE_AUTOSELECT=y
CONFIG_B43_RFKILL=y
# CONFIG_B43_DEBUG is not set
CONFIG_B43_DMA=y
CONFIG_B43_PIO=y
CONFIG_B43_DMA_AND_PIO_MODE=y
# CONFIG_B43_DMA_MODE is not set
# CONFIG_B43_PIO_MODE is not set
CONFIG_B43LEGACY=m
CONFIG_B43LEGACY_PCI_AUTOSELECT=y
CONFIG_B43LEGACY_PCICORE_AUTOSELECT=y
# CONFIG_B43LEGACY_DEBUG is not set
CONFIG_B43LEGACY_DMA=y
CONFIG_B43LEGACY_PIO=y
CONFIG_B43LEGACY_DMA_AND_PIO_MODE=y
# CONFIG_B43LEGACY_DMA_MODE is not set
# CONFIG_B43LEGACY_PIO_MODE is not set
CONFIG_ZD1211RW=m
# CONFIG_ZD1211RW_DEBUG is not set
# CONFIG_RT2X00 is not set
#
# USB Network Adapters
#
CONFIG_USB_CATC=m
CONFIG_USB_KAWETH=m
CONFIG_USB_PEGASUS=m
CONFIG_USB_RTL8150=m
CONFIG_USB_USBNET_MII=m
CONFIG_USB_USBNET=m
CONFIG_USB_NET_AX8817X=m
CONFIG_USB_NET_CDCETHER=m
CONFIG_USB_NET_DM9601=m
CONFIG_USB_NET_GL620A=m
CONFIG_USB_NET_NET1080=m
CONFIG_USB_NET_PLUSB=m
# CONFIG_USB_NET_MCS7830 is not set
CONFIG_USB_NET_RNDIS_HOST=m
CONFIG_USB_NET_CDC_SUBSET=m
CONFIG_USB_ALI_M5632=y
CONFIG_USB_AN2720=y
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
CONFIG_USB_EPSON2888=y
CONFIG_USB_KC2190=y
CONFIG_USB_NET_ZAURUS=m
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPP_MPPE=m
CONFIG_PPPOE=m
CONFIG_PPPOL2TP=m
# CONFIG_SLIP is not set
CONFIG_SLHC=m
# CONFIG_NET_FC is not set
CONFIG_SHAPER=m
CONFIG_NETCONSOLE=m
# CONFIG_NETCONSOLE_DYNAMIC is not set
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
CONFIG_INPUT_POLLDEV=m
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=m
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
CONFIG_INPUT_EVBUG=m
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=m
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
# CONFIG_MOUSE_PS2_LIFEBOOK is not set
# CONFIG_MOUSE_PS2_TRACKPOINT is not set
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
CONFIG_MOUSE_SERIAL=m
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_WISTRON_BTNS is not set
# CONFIG_INPUT_ATLAS_BTNS is not set
CONFIG_INPUT_ATI_REMOTE=m
CONFIG_INPUT_ATI_REMOTE2=m
CONFIG_INPUT_KEYSPAN_REMOTE=m
# CONFIG_INPUT_POWERMATE is not set
CONFIG_INPUT_YEALINK=m
CONFIG_INPUT_UINPUT=m
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=m
CONFIG_SERIO_CT82C710=m
CONFIG_SERIO_PCIPS2=m
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
# CONFIG_GAMEPORT is not set
#
# Character devices
#
CONFIG_VT=y
# CONFIG_VT_UNICODE is not set
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#
CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_CONSOLE is not set
CONFIG_FIX_EARLYCON_MEM=y
# CONFIG_SERIAL_8250_PCI is not set
# CONFIG_SERIAL_8250_PNP is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=2
# CONFIG_SERIAL_8250_EXTENDED is not set
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_IPMI_HANDLER is not set
# CONFIG_WATCHDOG is not set
CONFIG_HW_RANDOM=y
CONFIG_HW_RANDOM_INTEL=m
# CONFIG_HW_RANDOM_AMD is not set
# CONFIG_HW_RANDOM_GEODE is not set
# CONFIG_HW_RANDOM_VIA is not set
CONFIG_NVRAM=m
# CONFIG_RTC is not set
CONFIG_GEN_RTC=y
CONFIG_GEN_RTC_X=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
# CONFIG_MWAVE is not set
# CONFIG_PC8736x_GPIO is not set
# CONFIG_NSC_GPIO is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
# CONFIG_HPET_RTC_IRQ is not set
CONFIG_HPET_MMAP=y
CONFIG_HANGCHECK_TIMER=m
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_CHARDEV=m
#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set
#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_I2C_SIMTEC is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_TINY_USB is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
# CONFIG_I2C_VOODOO3 is not set
#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
# CONFIG_DS1682 is not set
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_SENSORS_TSL2550 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set
# CONFIG_W1 is not set
CONFIG_POWER_SUPPLY=m
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_BATTERY_DS2760 is not set
# CONFIG_HWMON is not set
#
# Sonics Silicon Backplane
#
CONFIG_SSB_POSSIBLE=y
CONFIG_SSB=m
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_SILENT=y
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
CONFIG_SSB_DRIVER_PCICORE=y
#
# Multifunction device drivers
#
# CONFIG_MFD_SM501 is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
# CONFIG_DVB_CORE is not set
# CONFIG_DAB is not set
#
# Graphics support
#
CONFIG_AGP=m
# CONFIG_AGP_ALI is not set
CONFIG_AGP_ATI=m
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not set
CONFIG_AGP_INTEL=m
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_AGP_EFFICEON is not set
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=m
# CONFIG_DRM_I810 is not set
# CONFIG_DRM_I830 is not set
# CONFIG_DRM_I915 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
CONFIG_FB=y
CONFIG_FIRMWARE_EDID=y
CONFIG_FB_DDC=m
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_SYS_FOPS is not set
CONFIG_FB_DEFERRED_IO=y
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
CONFIG_FB_BACKLIGHT=y
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y
#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
CONFIG_FB_VESA=y
# CONFIG_FB_HECUBA is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_I810 is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not set
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
CONFIG_FB_RADEON_BACKLIGHT=y
# CONFIG_FB_RADEON_DEBUG is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_CYBLA is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set
CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_CORGI is not set
# CONFIG_BACKLIGHT_PROGEAR is not set
#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
CONFIG_VIDEO_SELECT=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=m
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
# CONFIG_LOGO is not set
#
# Sound
#
CONFIG_SOUND=m
#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
# CONFIG_SND_DYNAMIC_MINORS is not set
CONFIG_SND_SUPPORT_OLD_API=y
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
#
# Generic devices
#
CONFIG_SND_AC97_CODEC=m
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set
#
# PCI devices
#
CONFIG_SND_AD1889=m
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS5530 is not set
# CONFIG_SND_CS5535AUDIO is not set
# CONFIG_SND_DARLA20 is not set
# CONFIG_SND_GINA20 is not set
# CONFIG_SND_LAYLA20 is not set
# CONFIG_SND_DARLA24 is not set
# CONFIG_SND_GINA24 is not set
# CONFIG_SND_LAYLA24 is not set
# CONFIG_SND_MONA is not set
# CONFIG_SND_MIA is not set
# CONFIG_SND_ECHO3G is not set
# CONFIG_SND_INDIGO is not set
# CONFIG_SND_INDIGOIO is not set
# CONFIG_SND_INDIGODJ is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
CONFIG_SND_HDA_INTEL=m
# CONFIG_SND_HDA_HWDEP is not set
CONFIG_SND_HDA_CODEC_REALTEK=y
CONFIG_SND_HDA_CODEC_ANALOG=y
CONFIG_SND_HDA_CODEC_SIGMATEL=y
CONFIG_SND_HDA_CODEC_VIA=y
CONFIG_SND_HDA_CODEC_ATIHDMI=y
CONFIG_SND_HDA_CODEC_CONEXANT=y
CONFIG_SND_HDA_CODEC_CMEDIA=y
CONFIG_SND_HDA_CODEC_SI3054=y
CONFIG_SND_HDA_GENERIC=y
CONFIG_SND_HDA_POWER_SAVE=y
CONFIG_SND_HDA_POWER_SAVE_DEFAULT=10
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
CONFIG_SND_INTEL8X0=m
CONFIG_SND_INTEL8X0M=m
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RIPTIDE is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set
CONFIG_SND_AC97_POWER_SAVE=y
CONFIG_SND_AC97_POWER_SAVE_DEFAULT=10
#
# USB devices
#
CONFIG_SND_USB_AUDIO=m
CONFIG_SND_USB_USX2Y=m
CONFIG_SND_USB_CAIAQ=m
# CONFIG_SND_USB_CAIAQ_INPUT is not set
#
# System on Chip audio support
#
# CONFIG_SND_SOC is not set
#
# SoC Audio support for SuperH
#
#
# Open Sound System
#
CONFIG_SOUND_PRIME=m
# CONFIG_SOUND_TRIDENT is not set
# CONFIG_SOUND_MSNDCLAS is not set
# CONFIG_SOUND_MSNDPIN is not set
CONFIG_SOUND_OSS=m
# CONFIG_SOUND_TRACEINIT is not set
# CONFIG_SOUND_DMAP is not set
# CONFIG_SOUND_SSCAPE is not set
CONFIG_SOUND_VMIDI=m
# CONFIG_SOUND_TRIX is not set
CONFIG_SOUND_MSS=m
CONFIG_SOUND_MPU401=m
# CONFIG_SOUND_PAS is not set
# CONFIG_SOUND_PSS is not set
CONFIG_SOUND_SB=m
# CONFIG_SOUND_YM3812 is not set
CONFIG_SOUND_UART6850=m
# CONFIG_SOUND_AEDSP16 is not set
# CONFIG_SOUND_KAHLUA is not set
CONFIG_AC97_BUS=m
CONFIG_HID_SUPPORT=y
CONFIG_HID=m
# CONFIG_HID_DEBUG is not set
CONFIG_HIDRAW=y
#
# USB Input Devices
#
CONFIG_USB_HID=m
# CONFIG_USB_HIDINPUT_POWERBOOK is not set
# CONFIG_HID_FF is not set
CONFIG_USB_HIDDEV=y
#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DEVICE_CLASS=y
# CONFIG_USB_DYNAMIC_MINORS is not set
CONFIG_USB_SUSPEND=y
CONFIG_USB_PERSIST=y
# CONFIG_USB_OTG is not set
#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
CONFIG_USB_EHCI_SPLIT_ISO=y
CONFIG_USB_EHCI_ROOT_HUB_TT=y
CONFIG_USB_EHCI_TT_NEWSCHED=y
# CONFIG_USB_ISP116X_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_HCD_SSB is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_U132_HCD is not set
CONFIG_USB_SL811_HCD=m
# CONFIG_USB_R8A66597_HCD is not set
#
# USB Device Class drivers
#
CONFIG_USB_ACM=m
CONFIG_USB_PRINTER=m
#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#
#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
CONFIG_USB_STORAGE_DATAFAB=y
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_DPCM=y
CONFIG_USB_STORAGE_USBAT=y
CONFIG_USB_STORAGE_SDDR09=y
CONFIG_USB_STORAGE_SDDR55=y
CONFIG_USB_STORAGE_JUMPSHOT=y
CONFIG_USB_STORAGE_ALAUDA=y
# CONFIG_USB_STORAGE_KARMA is not set
CONFIG_USB_LIBUSUAL=y
#
# USB Imaging devices
#
CONFIG_USB_MDC800=m
CONFIG_USB_MICROTEK=m
CONFIG_USB_MON=y
#
# USB port drivers
#
#
# USB Serial Converter support
#
CONFIG_USB_SERIAL=m
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_AIRCABLE=m
CONFIG_USB_SERIAL_AIRPRIME=m
CONFIG_USB_SERIAL_ARK3116=m
CONFIG_USB_SERIAL_BELKIN=m
CONFIG_USB_SERIAL_CH341=m
CONFIG_USB_SERIAL_WHITEHEAT=m
CONFIG_USB_SERIAL_DIGI_ACCELEPORT=m
CONFIG_USB_SERIAL_CP2101=m
CONFIG_USB_SERIAL_CYPRESS_M8=m
CONFIG_USB_SERIAL_EMPEG=m
CONFIG_USB_SERIAL_FTDI_SIO=m
CONFIG_USB_SERIAL_FUNSOFT=m
CONFIG_USB_SERIAL_VISOR=m
CONFIG_USB_SERIAL_IPAQ=m
CONFIG_USB_SERIAL_IR=m
CONFIG_USB_SERIAL_EDGEPORT=m
CONFIG_USB_SERIAL_EDGEPORT_TI=m
CONFIG_USB_SERIAL_GARMIN=m
CONFIG_USB_SERIAL_IPW=m
CONFIG_USB_SERIAL_KEYSPAN_PDA=m
CONFIG_USB_SERIAL_KEYSPAN=m
CONFIG_USB_SERIAL_KEYSPAN_MPR=y
CONFIG_USB_SERIAL_KEYSPAN_USA28=y
CONFIG_USB_SERIAL_KEYSPAN_USA28X=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XA=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XB=y
CONFIG_USB_SERIAL_KEYSPAN_USA19=y
CONFIG_USB_SERIAL_KEYSPAN_USA18X=y
CONFIG_USB_SERIAL_KEYSPAN_USA19W=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QW=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QI=y
CONFIG_USB_SERIAL_KEYSPAN_USA49W=y
CONFIG_USB_SERIAL_KEYSPAN_USA49WLC=y
CONFIG_USB_SERIAL_KLSI=m
CONFIG_USB_SERIAL_KOBIL_SCT=m
CONFIG_USB_SERIAL_MCT_U232=m
CONFIG_USB_SERIAL_MOS7720=m
CONFIG_USB_SERIAL_MOS7840=m
CONFIG_USB_SERIAL_NAVMAN=m
CONFIG_USB_SERIAL_PL2303=m
CONFIG_USB_SERIAL_OTI6858=m
# CONFIG_USB_SERIAL_HP4X is not set
CONFIG_USB_SERIAL_SAFE=m
CONFIG_USB_SERIAL_SAFE_PADDED=y
CONFIG_USB_SERIAL_SIERRAWIRELESS=m
CONFIG_USB_SERIAL_TI=m
CONFIG_USB_SERIAL_CYBERJACK=m
CONFIG_USB_SERIAL_XIRCOM=m
CONFIG_USB_SERIAL_OPTION=m
CONFIG_USB_SERIAL_OMNINET=m
# CONFIG_USB_SERIAL_DEBUG is not set
CONFIG_USB_EZUSB=y
#
# USB Miscellaneous drivers
#
CONFIG_USB_EMI62=m
CONFIG_USB_EMI26=m
CONFIG_USB_ADUTUX=m
CONFIG_USB_AUERSWALD=m
CONFIG_USB_RIO500=m
CONFIG_USB_LEGOTOWER=m
CONFIG_USB_LCD=m
# CONFIG_USB_BERRY_CHARGE is not set
CONFIG_USB_LED=m
CONFIG_USB_CYPRESS_CY7C63=m
CONFIG_USB_CYTHERM=m
# CONFIG_USB_PHIDGET is not set
CONFIG_USB_IDMOUSE=m
CONFIG_USB_FTDI_ELAN=m
# CONFIG_USB_APPLEDISPLAY is not set
CONFIG_USB_SISUSBVGA=m
# CONFIG_USB_SISUSBVGA_CON is not set
CONFIG_USB_LD=m
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
CONFIG_USB_TEST=m
#
# USB DSL modem support
#
#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
CONFIG_MMC=m
# CONFIG_MMC_DEBUG is not set
# CONFIG_MMC_UNSAFE_RESUME is not set
#
# MMC/SD Card Drivers
#
CONFIG_MMC_BLOCK=m
CONFIG_MMC_BLOCK_BOUNCE=y
# CONFIG_SDIO_UART is not set
#
# MMC/SD Host Controller Drivers
#
CONFIG_MMC_SDHCI=m
CONFIG_MMC_RICOH_MMC=m
# CONFIG_MMC_WBSD is not set
CONFIG_MMC_TIFM_SD=m
# CONFIG_NEW_LEDS is not set
# CONFIG_INFINIBAND is not set
# CONFIG_EDAC is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set
#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
CONFIG_RTC_DRV_TEST=m
#
# I2C RTC drivers
#
CONFIG_RTC_DRV_DS1307=m
CONFIG_RTC_DRV_DS1374=m
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_ISL1208=m
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8563=m
CONFIG_RTC_DRV_PCF8583=m
# CONFIG_RTC_DRV_M41T80 is not set
#
# SPI RTC drivers
#
#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=y
CONFIG_RTC_DRV_DS1553=m
# CONFIG_RTC_DRV_STK17TA8 is not set
CONFIG_RTC_DRV_DS1742=m
CONFIG_RTC_DRV_M48T86=m
# CONFIG_RTC_DRV_M48T59 is not set
CONFIG_RTC_DRV_V3020=m
#
# on-CPU RTC drivers
#
CONFIG_DMADEVICES=y
#
# DMA Devices
#
CONFIG_INTEL_IOATDMA=m
CONFIG_DMA_ENGINE=y
#
# DMA Clients
#
CONFIG_NET_DMA=y
CONFIG_DCA=m
# CONFIG_VIRTUALIZATION is not set
#
# Userspace I/O
#
CONFIG_UIO=m
# CONFIG_UIO_CIF is not set
#
# File systems
#
CONFIG_EXT2_FS=m
# CONFIG_EXT2_FS_XATTR is not set
CONFIG_EXT2_FS_XIP=y
CONFIG_FS_XIP=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_FS_XATTR is not set
# CONFIG_EXT4DEV_FS is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
# CONFIG_QUOTA is not set
CONFIG_DNOTIFY=y
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
CONFIG_FUSE_FS=m
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_CONFIGFS_FS=m
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
CONFIG_HFS_FS=m
CONFIG_HFSPLUS_FS=m
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_NFS_ACL_SUPPORT=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_SUNRPC_BIND34=y
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
CONFIG_SMB_NLS_DEFAULT=y
CONFIG_SMB_NLS_REMOTE="cp437"
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
CONFIG_CIFS_WEAK_PW_HASH=y
# CONFIG_CIFS_XATTR is not set
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
# CONFIG_EFI_PARTITION is not set
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="cp437"
CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
CONFIG_NLS_CODEPAGE_850=m
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
CONFIG_NLS_CODEPAGE_863=m
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=m
CONFIG_NLS_ISO8859_4=m
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=m
CONFIG_NLS_ISO8859_7=m
CONFIG_NLS_ISO8859_9=m
CONFIG_NLS_ISO8859_13=m
CONFIG_NLS_ISO8859_14=m
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_KOI8_R=m
CONFIG_NLS_KOI8_U=m
CONFIG_NLS_UTF8=m
CONFIG_DLM=m
# CONFIG_DLM_DEBUG is not set
# CONFIG_INSTRUMENTATION is not set
#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHED_DEBUG is not set
# CONFIG_SCHEDSTATS is not set
CONFIG_TIMER_STATS=y
# CONFIG_DEBUG_PREEMPT is not set
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_HIGHMEM is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_LIST is not set
# CONFIG_FRAME_POINTER is not set
# CONFIG_FORCED_INLINING is not set
CONFIG_BOOT_PRINTK_DELAY=y
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_EARLY_PRINTK is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
#
# Page alloc debug is incompatible with Software Suspend on i386
#
# CONFIG_DEBUG_RODATA is not set
CONFIG_4KSTACKS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
CONFIG_DOUBLEFAULT=y
#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
CONFIG_CRYPTO=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ABLKCIPHER=m
CONFIG_CRYPTO_BLKCIPHER=m
CONFIG_CRYPTO_HASH=m
CONFIG_CRYPTO_MANAGER=m
CONFIG_CRYPTO_HMAC=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_GF128MUL=m
CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_LRW=m
# CONFIG_CRYPTO_XTS is not set
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
CONFIG_CRYPTO_TWOFISH_586=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_586=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_ANUBIS=m
# CONFIG_CRYPTO_SEED is not set
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_CAMELLIA=m
# CONFIG_CRYPTO_TEST is not set
# CONFIG_CRYPTO_AUTHENC is not set
# CONFIG_CRYPTO_HW is not set
#
# Library routines
#
CONFIG_BITREVERSE=m
CONFIG_CRC_CCITT=m
CONFIG_CRC16=m
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=m
CONFIG_CRC7=m
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_X86_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_KTIME_SCALAR=y
Mark Lord wrote:
> Mark Lord wrote:
>> Mark Lord wrote:
>>> Linus Torvalds wrote:
>>>>
>>>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>>>> It would be good to have something soon-ish.
>>>>> This "dead at boot time" issue is impacting the general ability to
>>>>> test
>>>>> patches against latest -git in time for the current merge window.
>>>>
>>>> In the meantime, does the patch I sent out help people? I'd like to
>>>> get feedback, but I'm a lazy bum, and don't use DEBUG_PAGEALLOC
>>>> myself, so I was hoping that people who actually see this could
>>>> comment on my untested suggestion.
>>>
>>> Oh.. so this bug is supposed to only bite with DEBUG_PAGEALLOC=y ??
>>>
>>> Then something else is broken, perhaps. I just saw a long traceback
>>> scroll off the top of the screen, with lots of bio_* functions in the
>>> list
>>> and assumed it was the same bug.
>>>
>>> Mmm.. I'll get out the camera and try it again now..
>>
>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>
>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>> but the top was cut off (isn't there a new config option or patch
>> to do double-columns or scrollback or something ???.
>
> I tried the fancy "boot_delay=nnn" feature, but that doesn't slow down
> the tracebacks at all. So I hardcoded some mdelay(1000) lines into
> the traceback code, and here's the top part of the oops now:
>
> http://rtr.ca/recent/2.6.23-git12-crash2.jpg
...
And, yes, I make that out as being this line from blk_rq_map_sg():
next_sg = sg_next(sg);
"objdump -d" output from my actual kernel:
00003ce0 <blk_rq_map_sg>:
3ce0: 55 push %ebp
3ce1: 57 push %edi
3ce2: 56 push %esi
3ce3: 53 push %ebx
3ce4: 83 ec 24 sub $0x24,%esp
3ce7: 89 44 24 04 mov %eax,0x4(%esp)
3ceb: 8b 98 44 01 00 00 mov 0x144(%eax),%ebx
3cf1: 83 e3 01 and $0x1,%ebx
3cf4: 89 5c 24 14 mov %ebx,0x14(%esp)
3cf8: 8b 52 34 mov 0x34(%edx),%edx
3cfb: c7 44 24 10 00 00 00 movl $0x0,0x10(%esp)
3d02: 00
3d03: 85 d2 test %edx,%edx
3d05: 89 54 24 1c mov %edx,0x1c(%esp)
3d09: 0f 84 f4 00 00 00 je 3e03 <blk_rq_map_sg+0x123>
3d0f: 89 cb mov %ecx,%ebx
3d11: 31 ff xor %edi,%edi
3d13: 89 4c 24 0c mov %ecx,0xc(%esp)
3d17: 8b 44 24 1c mov 0x1c(%esp),%eax
3d1b: 0f b7 48 16 movzwl 0x16(%eax),%ecx
3d1f: 8b 50 2c mov 0x2c(%eax),%edx
3d22: 89 4c 24 18 mov %ecx,0x18(%esp)
3d26: 0f b7 40 14 movzwl 0x14(%eax),%eax
3d2a: 39 c1 cmp %eax,%ecx
3d2c: 0f 8d be 00 00 00 jge 3df0 <blk_rq_map_sg+0x110>
3d32: 8d 04 49 lea (%ecx,%ecx,2),%eax
3d35: 8d 34 82 lea (%edx,%eax,4),%esi
3d38: 0f b6 44 24 14 movzbl 0x14(%esp),%eax
3d3d: 88 44 24 23 mov %al,0x23(%esp)
3d41: eb 05 jmp 3d48 <blk_rq_map_sg+0x68>
3d43: 89 f7 mov %esi,%edi
3d45: 83 c6 0c add $0xc,%esi
3d48: 80 7c 24 23 00 cmpb $0x0,0x23(%esp)
3d4d: 8b 6e 04 mov 0x4(%esi),%ebp
3d50: 74 4e je 3da0 <blk_rq_map_sg+0xc0>
3d52: 85 ff test %edi,%edi
3d54: 74 4a je 3da0 <blk_rq_map_sg+0xc0>
3d56: 8b 54 24 0c mov 0xc(%esp),%edx
3d5a: 8b 44 24 04 mov 0x4(%esp),%eax
3d5e: 8b 4a 0c mov 0xc(%edx),%ecx
3d61: 01 e9 add %ebp,%ecx
3d63: 89 4c 24 08 mov %ecx,0x8(%esp)
3d67: 3b 88 94 01 00 00 cmp 0x194(%eax),%ecx
3d6d: 77 31 ja 3da0 <blk_rq_map_sg+0xc0>
3d6f: 8b 15 00 00 00 00 mov 0x0,%edx
3d75: 8b 0f mov (%edi),%ecx
3d77: 8b 47 04 mov 0x4(%edi),%eax
3d7a: 29 d1 sub %edx,%ecx
3d7c: c1 f9 05 sar $0x5,%ecx
3d7f: c1 e1 0c shl $0xc,%ecx
3d82: 03 4f 08 add 0x8(%edi),%ecx
3d85: 8d 3c 01 lea (%ecx,%eax,1),%edi
3d88: 8b 06 mov (%esi),%eax
3d8a: 29 d0 sub %edx,%eax
3d8c: c1 f8 05 sar $0x5,%eax
3d8f: c1 e0 0c shl $0xc,%eax
3d92: 03 46 08 add 0x8(%esi),%eax
3d95: 39 c7 cmp %eax,%edi
3d97: 74 76 je 3e0f <blk_rq_map_sg+0x12f>
3d99: 8d b4 26 00 00 00 00 lea 0x0(%esi),%esi
3da0: 8d 43 10 lea 0x10(%ebx),%eax
3da3: b9 04 00 00 00 mov $0x4,%ecx
3da8: 89 04 24 mov %eax,(%esp)
3dab: 8b 43 10 mov 0x10(%ebx),%eax <<<<--------- dies here
3dae: 89 df mov %ebx,%edi
3db0: 89 c2 mov %eax,%edx
3db2: 83 e2 fe and $0xfffffffe,%edx
3db5: a8 01 test $0x1,%al
3db7: 0f 44 14 24 cmove (%esp),%edx
3dbb: 31 c0 xor %eax,%eax
3dbd: f3 ab rep stos %eax,%es:(%edi)
3dbf: 8b 06 mov (%esi),%eax
3dc1: 89 6b 0c mov %ebp,0xc(%ebx)
3dc4: 89 03 mov %eax,(%ebx)
3dc6: 8b 46 08 mov 0x8(%esi),%eax
3dc9: 89 43 04 mov %eax,0x4(%ebx)
3dcc: 83 44 24 10 01 addl $0x1,0x10(%esp)
3dd1: 89 5c 24 0c mov %ebx,0xc(%esp)
3dd5: 89 d3 mov %edx,%ebx
3dd7: 8b 54 24 1c mov 0x1c(%esp),%edx
Linus Torvalds wrote:
>
> On Wed, 17 Oct 2007, Mark Lord wrote:
>> It would be good to have something soon-ish.
>> This "dead at boot time" issue is impacting the general ability to test
>> patches against latest -git in time for the current merge window.
>
> In the meantime, does the patch I sent out help people?
Your patch from this posting http://lkml.org/lkml/2007/10/17/285
does not seem to make much difference here.
It still crashes at exactly the same place.
Mark Lord wrote:
> Linus Torvalds wrote:
>>
>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>> It would be good to have something soon-ish.
>>> This "dead at boot time" issue is impacting the general ability to test
>>> patches against latest -git in time for the current merge window.
>>
>> In the meantime, does the patch I sent out help people?
>
> Your patch from this posting http://lkml.org/lkml/2007/10/17/285
> does not seem to make much difference here.
>
> It still crashes at exactly the same place.
However, Jens's patch from that same thread:
http://lkml.org/lkml/2007/10/17/269
..allowed me to boot and post this followup message from -git12
Jeff: try that one.
Patch reproduced here for convenience:
Jens Axboe wrote:
> OK, it is fine, as long as the sglist is cleared initially. And I don't
> think there's anyway around that, clearly I didn't think long enough
> before including the memset() removal from Tomo.
>
> Ingo, please try this rolled up version.
>
> Linus, this should work. It would probably be best if you first did a
> git revert on f5c0dde4c66421a3a2d7d6fa604a712c9b0744e5 and then applied
> the ll_rw_blk.c bit alone. Do you want me to stuff that (revert + patch)
> into a branch for you to pull?
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 9e3f3cc..3935469 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1322,8 +1322,8 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
struct scatterlist *sglist)
{
struct bio_vec *bvec, *bvprv;
- struct scatterlist *next_sg, *sg;
struct req_iterator iter;
+ struct scatterlist *sg;
int nsegs, cluster;
nsegs = 0;
@@ -1333,7 +1333,7 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
* for each bio in rq
*/
bvprv = NULL;
- sg = next_sg = &sglist[0];
+ sg = NULL;
rq_for_each_segment(bvec, rq, iter) {
int nbytes = bvec->bv_len;
@@ -1349,8 +1349,10 @@ int blk_rq_map_sg(struct request_queue *q, struct request *rq,
sg->length += nbytes;
} else {
new_segment:
- sg = next_sg;
- next_sg = sg_next(sg);
+ if (!sg)
+ sg = sglist;
+ else
+ sg = sg_next(sg);
memset(sg, 0, sizeof(*sg));
sg->page = bvec->bv_page;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 0c86be7..aac8a02 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -764,6 +764,8 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
+ memset(sgl, 0, sizeof(*sgl) * sgp->size);
+
/*
* first loop through, set initial index and return value
*/
Mark Lord wrote:
> Mark Lord wrote:
>> Linus Torvalds wrote:
>>>
>>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>>> It would be good to have something soon-ish.
>>>> This "dead at boot time" issue is impacting the general ability to test
>>>> patches against latest -git in time for the current merge window.
>>>
>>> In the meantime, does the patch I sent out help people?
>>
>> Your patch from this posting http://lkml.org/lkml/2007/10/17/285
>> does not seem to make much difference here.
>>
>> It still crashes at exactly the same place.
>
>
> However, Jens's patch from that same thread:
>
> http://lkml.org/lkml/2007/10/17/269
>
> ..allowed me to boot and post this followup message from -git12
>
> Jeff: try that one.
That's already in my upstream kernel, here. commits
ba951841ceb7fa5b06ad48caa5270cc2ae17941e and
a3bec5c5aea0da263111c4d8f8eabc1f8560d7bf.
sata_mv and sata_nv still reliably poop themselves here, whereas its
rock solid with 2.6.23.1. Sounds like different issues from yours, as I
see a stream of SATA errors on the bad kernels, errors which are often a
symptom of something whacked in the DMA engine (misprogramming causes
the silicon to generate bogus FIS's, which the device then chokes on)
Jeff
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Mark Lord wrote:
>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>> but the top was cut off (isn't there a new config option or patch
>> to do double-columns or scrollback or something ???.
>
> Is this a sata_mv box? If so, could you try this patch?
If anything, that shrinks the size of the resulting request. Did this
patch make any difference to you?
Now sata_mv should not be doing this (already discussed), but as long as
it's only lowering the physical sg count then it should not cause any
bugs at least.
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -39,7 +39,7 @@
> * (unless chaining is used). Should ideally fit inside a single page, to
> * avoid a higher order allocation.
> */
> -#define SCSI_MAX_SG_SEGMENTS 128
> +#define SCSI_MAX_SG_SEGMENTS 129
this one finally made the trick and it's booting fine now, without any
crashes!
Tested-by: Ingo Molnar <[email protected]>
Ingo
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Mark Lord wrote:
>> Mark Lord wrote:
>>> Linus Torvalds wrote:
>>>>
>>>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>>>> It would be good to have something soon-ish.
>>>>> This "dead at boot time" issue is impacting the general ability to test
>>>>> patches against latest -git in time for the current merge window.
>>>>
>>>> In the meantime, does the patch I sent out help people?
>>>
>>> Your patch from this posting http://lkml.org/lkml/2007/10/17/285
>>> does not seem to make much difference here.
>>>
>>> It still crashes at exactly the same place.
>> However, Jens's patch from that same thread:
>> http://lkml.org/lkml/2007/10/17/269
>> ..allowed me to boot and post this followup message from -git12
>> Jeff: try that one.
>
> That's already in my upstream kernel, here. commits
> ba951841ceb7fa5b06ad48caa5270cc2ae17941e and
> a3bec5c5aea0da263111c4d8f8eabc1f8560d7bf.
>
> sata_mv and sata_nv still reliably poop themselves here, whereas its rock
> solid with 2.6.23.1. Sounds like different issues from yours, as I see a
> stream of SATA errors on the bad kernels, errors which are often a symptom
> of something whacked in the DMA engine (misprogramming causes the silicon
> to generate bogus FIS's, which the device then chokes on)
Do you know if this poop involves the segment padding that sometimes
goes on in libata?
--
Jens Axboe
On Thu, Oct 18 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > --- a/drivers/scsi/scsi_lib.c
> > +++ b/drivers/scsi/scsi_lib.c
> > @@ -39,7 +39,7 @@
> > * (unless chaining is used). Should ideally fit inside a single page, to
> > * avoid a higher order allocation.
> > */
> > -#define SCSI_MAX_SG_SEGMENTS 128
> > +#define SCSI_MAX_SG_SEGMENTS 129
>
> this one finally made the trick and it's booting fine now, without any
> crashes!
Super, that validates the theory the theory that Linus put forth. So the
bug is clear now, this morning I'll work on proper sg looping.
--
Jens Axboe
Jens Axboe wrote:
> On Thu, Oct 18 2007, Jeff Garzik wrote:
>> Mark Lord wrote:
>>> Mark Lord wrote:
>>>> Linus Torvalds wrote:
>>>>> On Wed, 17 Oct 2007, Mark Lord wrote:
>>>>>> It would be good to have something soon-ish.
>>>>>> This "dead at boot time" issue is impacting the general ability to test
>>>>>> patches against latest -git in time for the current merge window.
>>>>> In the meantime, does the patch I sent out help people?
>>>> Your patch from this posting http://lkml.org/lkml/2007/10/17/285
>>>> does not seem to make much difference here.
>>>>
>>>> It still crashes at exactly the same place.
>>> However, Jens's patch from that same thread:
>>> http://lkml.org/lkml/2007/10/17/269
>>> ..allowed me to boot and post this followup message from -git12
>>> Jeff: try that one.
>> That's already in my upstream kernel, here. commits
>> ba951841ceb7fa5b06ad48caa5270cc2ae17941e and
>> a3bec5c5aea0da263111c4d8f8eabc1f8560d7bf.
>>
>> sata_mv and sata_nv still reliably poop themselves here, whereas its rock
>> solid with 2.6.23.1. Sounds like different issues from yours, as I see a
>> stream of SATA errors on the bad kernels, errors which are often a symptom
>> of something whacked in the DMA engine (misprogramming causes the silicon
>> to generate bogus FIS's, which the device then chokes on)
>
> Do you know if this poop involves the segment padding that sometimes
> goes on in libata?
Definitely not, in this case -- it's all ATA, nothing ATAPI.
It throws SError { Handshk } which then triggers the EH to reset the
link, and so it goes, over and over :) The same thing happens when I
intentionally screw up the PRD tables. Not much more data points than
that, so far.
I'll try the SCSI_MAX_SG_SEGMENTS patch too, to see if that fixes things.
Jeff
On Wed, Oct 17 2007, David Miller wrote:
> From: Linus Torvalds <[email protected]>
> Date: Wed, 17 Oct 2007 18:07:19 -0700 (PDT)
>
> > sg_next() - as it stands now - never actually looks at the SG that its
> > argument points to: it explicitly *only* looks at the next one.
> >
> > That's the bug. If sg_next() looked at the actual *current* sg entry, we
> > wouldn't have any issues to begin with, and that's what I'm arguing we
> > should do in the longer run (where "longer run" is defined as "when Jens
> > does it asap").
>
> What the thing really wants is some kind of indication of state,
> without having to bloat up the scatterlist structure.
>
> I believe that we have enough of a limited set of accessors to
> sg->page that we can more aggressively encode things in the lower
> bits.
>
> I'm thinking of encoding the low two bits of sg->page as
> follows:
>
> 1) bits == 0
>
> then the SG list is linear and sg_next() is sg++
>
> 2) bits == 1
>
> the nest SG is an indirect chunk, sg_next() is
> therefore something like:
>
> next = *((struct scatterlist **)(sg + 1));
>
> 3) bits == 2
>
> this is the last entry in the scatterlist, sg_next() is NULL
>
> So for the cases of ARCH_HAS_SG_CHAIN not being set (ie. back
> compatible), we can do no bit encoding in page->flags and just do
> sg_next() == sg++, as is done now.
>
> When doing SG chaining, in each non-linear chunk we have to allocate
> one more pointer past the end of the scatterlist array (instead of a
> full extra scatterlist entry for the indirect pointer encode). Next,
> all sg->page accesses have to be guarded to clear the state bits
> out first.
>
> I don't know, maybe it would work, and would make the loop termination
> issues easier to handle properly.
I like it. Basically the only real change is using bit 2 as a
termination point, so we avoid going beyond the end of the sgtable.
Here's a starting point, it actually booted for me in the first go
(boggle). Only x86 so far, archs will need to be converted. And lots
more drivers I'm sure, I only fixed up the ones that botched my compile.
So just consider this a directional patch.
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..d02783c 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,10 +1354,11 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
nsegs++;
}
bvprv = bvec;
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..4ee32e5 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..98c455d 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -443,7 +443,7 @@ int usb_sg_init (
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..cbc2b57 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,10 +5,24 @@
#include <linux/mm.h>
#include <linux/string.h>
+#define SG_MAGIC 0x87654321
+
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
@@ -25,9 +39,9 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
-#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
+#define sg_chain_ptr(sg) ((struct scatterlist *) ((sg)->page_link & ~0x01))
/**
* sg_next - return the next scatterlist entry in a list
@@ -43,10 +57,15 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
-
- if (unlikely(sg_is_chain(sg)))
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ sg = NULL;
+ else if (sg_is_chain(sg))
sg = sg_chain_ptr(sg);
+ else
+ sg++;
return sg;
}
@@ -83,6 +102,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+#endif
return ret;
}
@@ -101,7 +123,41 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
+}
+
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+/**
+ * sg_init_table - Initialize an sg table
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
}
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..c17aace 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,17 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables. Runtime overhead is very small, but it does
+ increase an sg entry by sizeof(unsigned long).
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..1a77877 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,7 +170,8 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(pos));
sg.offset = offset_in_page(pos);
sg.length = len + 4;
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
@@ -212,7 +213,8 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(pos));
sg.offset = offset_in_page(pos);
sg.length = plen + 4;
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..9857961 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,7 +138,8 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(data));
sg.offset = offset_in_page(data);
sg.length = data_len + WEP_ICV_LEN;
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
@@ -204,7 +205,8 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(data));
sg.offset = offset_in_page(data);
sg.length = data_len + WEP_ICV_LEN;
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
Ingo Molnar wrote:
> * Jens Axboe <[email protected]> wrote:
>
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -39,7 +39,7 @@
>> * (unless chaining is used). Should ideally fit inside a single page, to
>> * avoid a higher order allocation.
>> */
>> -#define SCSI_MAX_SG_SEGMENTS 128
>> +#define SCSI_MAX_SG_SEGMENTS 129
>
> this one finally made the trick and it's booting fine now, without any
> crashes!
Alas, this didn't help me here. I did manage to capture the error
messages this time, and in the two machines I'm actively testing on,
sata_mv is the driver that's dying in both cases. Machine A
additionally has sata_nv, which is working. Machine B additionally has
ata_piix, which is working. So in both cases, its sata_mv that is
throwing SError complaints:
> ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x400000 action 0x6 frozen
> ata7.00: edma_err 0x04000080, EDMA self-disable
> ata7: SError: { Handshk }
> ata7.00: cmd ca/00:08:c7:40:00/00:00:00:00:00/e0 tag 0 cdb 0x0 data 4096 out
> res 50/00:00:ce:40:00/00:00:00:00:00/e0 Emask 0x100 (unknown error)
> ata7.00: status: { DRDY }
> ata7: hard resetting link
> ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
Still digging... this behavior showed up after libata changes went in,
FWIW.
Jeff
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Ingo Molnar wrote:
>> * Jens Axboe <[email protected]> wrote:
>>> --- a/drivers/scsi/scsi_lib.c
>>> +++ b/drivers/scsi/scsi_lib.c
>>> @@ -39,7 +39,7 @@
>>> * (unless chaining is used). Should ideally fit inside a single page,
>>> to
>>> * avoid a higher order allocation.
>>> */
>>> -#define SCSI_MAX_SG_SEGMENTS 128
>>> +#define SCSI_MAX_SG_SEGMENTS 129
>> this one finally made the trick and it's booting fine now, without any
>> crashes!
>
> Alas, this didn't help me here. I did manage to capture the error messages
> this time, and in the two machines I'm actively testing on, sata_mv is the
> driver that's dying in both cases. Machine A additionally has sata_nv,
> which is working. Machine B additionally has ata_piix, which is working.
> So in both cases, its sata_mv that is throwing SError complaints:
Theory - ata_sg_is_last() isn't returning true for the last entry. Can
you double check that it correcly marks the last entry in mv_fill_sg()?
Alternatively, just try this patch.
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..b858183 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -1139,6 +1139,7 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
struct mv_port_priv *pp = qc->ap->private_data;
struct scatterlist *sg;
struct mv_sg *mv_sg;
+ int end_marked = 0;
mv_sg = pp->sg_tbl;
ata_for_each_sg(sg, qc) {
@@ -1159,13 +1160,15 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
sg_len -= len;
addr += len;
- if (!sg_len && ata_sg_is_last(sg, qc))
+ if (!sg_len && ata_sg_is_last(sg, qc)) {
mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
+ end_marked++;
+ }
mv_sg++;
}
-
}
+ BUG_ON(end_marked != 1);
}
static inline void mv_crqb_pack_cmd(__le16 *cmdw, u8 data, u8 addr, unsigned last)
--
Jens Axboe
Jens Axboe wrote:
> On Thu, Oct 18 2007, Jeff Garzik wrote:
>> Ingo Molnar wrote:
>>> * Jens Axboe <[email protected]> wrote:
>>>> --- a/drivers/scsi/scsi_lib.c
>>>> +++ b/drivers/scsi/scsi_lib.c
>>>> @@ -39,7 +39,7 @@
>>>> * (unless chaining is used). Should ideally fit inside a single page,
>>>> to
>>>> * avoid a higher order allocation.
>>>> */
>>>> -#define SCSI_MAX_SG_SEGMENTS 128
>>>> +#define SCSI_MAX_SG_SEGMENTS 129
>>> this one finally made the trick and it's booting fine now, without any
>>> crashes!
>> Alas, this didn't help me here. I did manage to capture the error messages
>> this time, and in the two machines I'm actively testing on, sata_mv is the
>> driver that's dying in both cases. Machine A additionally has sata_nv,
>> which is working. Machine B additionally has ata_piix, which is working.
>> So in both cases, its sata_mv that is throwing SError complaints:
>
> Theory - ata_sg_is_last() isn't returning true for the last entry. Can
> you double check that it correcly marks the last entry in mv_fill_sg()?
> Alternatively, just try this patch.
Will check in a few minutes, after my current test: I noticed that
sg_tablesize in sata_mv is not LIBATA_MAX_PRD. This is expected
behavior, but I wonder if that difference -- most notably being smaller
than SCSI_MAX_SG_SEGMENTS -- would trigger any latent bugs.
Anyway, we will both have answers momentarily...
Jeff
Jeff Garzik wrote:
> I noticed that
> sg_tablesize in sata_mv is not LIBATA_MAX_PRD. This is expected
> behavior, but I wonder if that difference -- most notably being smaller
> than SCSI_MAX_SG_SEGMENTS -- would trigger any latent bugs.
Another sata_mv difference from the rest: the chip does not support
ATAPI, so we never care about qc->pad
Jeff
Jens Axboe wrote:
> index 4df8311..b858183 100644
> --- a/drivers/ata/sata_mv.c
> +++ b/drivers/ata/sata_mv.c
> @@ -1139,6 +1139,7 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
> struct mv_port_priv *pp = qc->ap->private_data;
> struct scatterlist *sg;
> struct mv_sg *mv_sg;
> + int end_marked = 0;
>
> mv_sg = pp->sg_tbl;
> ata_for_each_sg(sg, qc) {
> @@ -1159,13 +1160,15 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
> sg_len -= len;
> addr += len;
>
> - if (!sg_len && ata_sg_is_last(sg, qc))
> + if (!sg_len && ata_sg_is_last(sg, qc)) {
> mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
> + end_marked++;
> + }
>
> mv_sg++;
> }
> -
> }
> + BUG_ON(end_marked != 1);
Your BUG_ON() does indeed trip, here.
Its surprising that other folks don't explode, considering that
mv_fill_sg() intentionally mirrors the logic in ata_fill_sg().
Jeff
Benny Halevy wrote:
>
>
> On 10/18/07, *Jeff Garzik* <[email protected] <mailto:[email protected]>> wrote:
>
> Jens Axboe wrote:
> > index 4df8311..b858183 100644
> > --- a/drivers/ata/sata_mv.c
> > +++ b/drivers/ata/sata_mv.c
> > @@ -1139,6 +1139,7 @@ static void mv_fill_sg(struct
> ata_queued_cmd *qc)
> > struct mv_port_priv *pp = qc->ap->private_data;
> > struct scatterlist *sg;
> > struct mv_sg *mv_sg;
> > + int end_marked = 0;
> >
> > mv_sg = pp->sg_tbl;
> > ata_for_each_sg(sg, qc) {
> > @@ -1159,13 +1160,15 @@ static void mv_fill_sg(struct
> ata_queued_cmd *qc)
> > sg_len -= len;
> > addr += len;
> >
> > - if (!sg_len && ata_sg_is_last(sg, qc))
> > + if (!sg_len && ata_sg_is_last(sg, qc)) {
>
>
> I'm not sure, but shouldn't that be || rather than &&?
sg_len is zero at the end of each scatterlist entry, so we need to test
the additional ata_sg_is_last() condition to determine if we are really
at the end of the PRD table.
Jeff
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Jens Axboe wrote:
>> index 4df8311..b858183 100644
>> --- a/drivers/ata/sata_mv.c
>> +++ b/drivers/ata/sata_mv.c
>> @@ -1139,6 +1139,7 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
>> struct mv_port_priv *pp = qc->ap->private_data;
>> struct scatterlist *sg;
>> struct mv_sg *mv_sg;
>> + int end_marked = 0;
>> mv_sg = pp->sg_tbl;
>> ata_for_each_sg(sg, qc) {
>> @@ -1159,13 +1160,15 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
>> sg_len -= len;
>> addr += len;
>> - if (!sg_len && ata_sg_is_last(sg, qc))
>> + if (!sg_len && ata_sg_is_last(sg, qc)) {
>> mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
>> + end_marked++;
>> + }
>> mv_sg++;
>> }
>> -
>> }
>> + BUG_ON(end_marked != 1);
>
>
> Your BUG_ON() does indeed trip, here.
>
> Its surprising that other folks don't explode, considering that
> mv_fill_sg() intentionally mirrors the logic in ata_fill_sg().
The sata_mv construct looks a bit odd. Does this work? That last
end_mv_sg test should always be true, just being paranoid...
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..5397eea 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -1138,8 +1138,9 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
{
struct mv_port_priv *pp = qc->ap->private_data;
struct scatterlist *sg;
- struct mv_sg *mv_sg;
+ struct mv_sg *mv_sg, *end_mv_sg;
+ end_mv_sg = NULL;
mv_sg = pp->sg_tbl;
ata_for_each_sg(sg, qc) {
dma_addr_t addr = sg_dma_address(sg);
@@ -1158,14 +1159,12 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
sg_len -= len;
addr += len;
-
- if (!sg_len && ata_sg_is_last(sg, qc))
- mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
-
+ end_mv_sg = mv_sg;
mv_sg++;
}
-
}
+ if (end_mv_sg)
+ end_mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
}
static inline void mv_crqb_pack_cmd(__le16 *cmdw, u8 data, u8 addr, unsigned last)
--
Jens Axboe
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..42b5a9e 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -421,7 +421,6 @@ static void mv_error_handler(struct ata_port *ap);
static void mv_post_int_cmd(struct ata_queued_cmd *qc);
static void mv_eh_freeze(struct ata_port *ap);
static void mv_eh_thaw(struct ata_port *ap);
-static int mv_slave_config(struct scsi_device *sdev);
static int mv_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
static void mv5_phy_errata(struct mv_host_priv *hpriv, void __iomem *mmio,
@@ -459,7 +458,7 @@ static struct scsi_host_template mv5_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -477,7 +476,7 @@ static struct scsi_host_template mv6_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -756,17 +755,6 @@ static void mv_irq_clear(struct ata_port *ap)
{
}
-static int mv_slave_config(struct scsi_device *sdev)
-{
- int rc = ata_scsi_slave_config(sdev);
- if (rc)
- return rc;
-
- blk_queue_max_phys_segments(sdev->request_queue, MV_MAX_SG_CT / 2);
-
- return 0; /* scsi layer doesn't check return value, sigh */
-}
-
static void mv_set_edma_ptrs(void __iomem *port_mmio,
struct mv_host_priv *hpriv,
struct mv_port_priv *pp)
@@ -1138,34 +1126,35 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
{
struct mv_port_priv *pp = qc->ap->private_data;
struct scatterlist *sg;
- struct mv_sg *mv_sg;
+ struct mv_sg *mv_sg = pp->sg_tbl;
+ unsigned int idx = 0;
- mv_sg = pp->sg_tbl;
ata_for_each_sg(sg, qc) {
- dma_addr_t addr = sg_dma_address(sg);
- u32 sg_len = sg_dma_len(sg);
+ u64 addr;
+ u32 offset, sg_len, len;
+
+ addr = sg_dma_address(sg);
+ sg_len = sg_dma_len(sg);
while (sg_len) {
- u32 offset = addr & 0xffff;
- u32 len = sg_len;
+ offset = addr & 0xffff;
+ len = sg_len;
if ((offset + sg_len > 0x10000))
len = 0x10000 - offset;
- mv_sg->addr = cpu_to_le32(addr & 0xffffffff);
- mv_sg->addr_hi = cpu_to_le32((addr >> 16) >> 16);
- mv_sg->flags_size = cpu_to_le32(len & 0xffff);
+ mv_sg[idx].addr = cpu_to_le32(addr & 0xffffffff);
+ mv_sg[idx].addr_hi = cpu_to_le32(addr >> 32);
+ mv_sg[idx].flags_size = cpu_to_le32(len & 0xffff);
+ idx++;
sg_len -= len;
addr += len;
-
- if (!sg_len && ata_sg_is_last(sg, qc))
- mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
-
- mv_sg++;
}
-
}
+
+ if (idx)
+ mv_sg[idx - 1].flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
}
static inline void mv_crqb_pack_cmd(__le16 *cmdw, u8 data, u8 addr, unsigned last)
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Jens Axboe wrote:
>> The sata_mv construct looks a bit odd. Does this work? That last
>
> The sata_mv construct worked just fine before sg chaining :)
Yes I know, but I'm trying to works towards getting rid of sg_last() and
ata_sg_is_last() anyway :-)
>> end_mv_sg test should always be true, just being paranoid...
>> diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
>> index 4df8311..5397eea 100644
>> --- a/drivers/ata/sata_mv.c
>> +++ b/drivers/ata/sata_mv.c
>> @@ -1138,8 +1138,9 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
>> {
>> struct mv_port_priv *pp = qc->ap->private_data;
>> struct scatterlist *sg;
>> - struct mv_sg *mv_sg;
>> + struct mv_sg *mv_sg, *end_mv_sg;
>> + end_mv_sg = NULL;
>> mv_sg = pp->sg_tbl;
>> ata_for_each_sg(sg, qc) {
>> dma_addr_t addr = sg_dma_address(sg);
>> @@ -1158,14 +1159,12 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
>> sg_len -= len;
>> addr += len;
>> -
>> - if (!sg_len && ata_sg_is_last(sg, qc))
>> - mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
>> -
>> + end_mv_sg = mv_sg;
>> mv_sg++;
>> }
>> -
>> }
>> + if (end_mv_sg)
>> + end_mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
>> }
>>
>
> I'm testing a similar patch based on ata_fill_sg()'s method, which
> basically does something similar to what you've done here (see attached).
> I had noticed that ata_fill_sg() did not call ata_sg_is_last().
>
> If this fixes the problem, I think the best solution would be to delete
> ata_sg_is_last(). In the few users that exist, we should be able to
> eliminate the test programmatically as you and ata_fill_sg() have done --
> thereby eliminating a branch per loop in a hotpath.
>
> Off to test the attached... if that doesn't work I'll try your version,
> though there shouldn't be much difference.
That should work as well. WRT ata_sg_is_last(), if we go ahead with my
recent sg chaining updates, we can keep the test as it would be a single
conditional and not require any looping.
Let me know when you have tested this!
--
Jens Axboe
diff --git a/drivers/ata/pdc_adma.c b/drivers/ata/pdc_adma.c
index 8d1b03d..199f7e1 100644
--- a/drivers/ata/pdc_adma.c
+++ b/drivers/ata/pdc_adma.c
@@ -318,7 +318,7 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
struct scatterlist *sg;
struct ata_port *ap = qc->ap;
struct adma_port_priv *pp = ap->private_data;
- u8 *buf = pp->pkt;
+ u8 *buf = pp->pkt, *last_buf = NULL;
int i = (2 + buf[3]) * 8;
u8 pFLAGS = pORD | ((qc->tf.flags & ATA_TFLAG_WRITE) ? pDIRO : 0);
@@ -334,8 +334,7 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
*(__le32 *)(buf + i) = cpu_to_le32(len);
i += 4;
- if (ata_sg_is_last(sg, qc))
- pFLAGS |= pEND;
+ last_buf = &buf[i];
buf[i++] = pFLAGS;
buf[i++] = qc->dev->dma_mode & 0xf;
buf[i++] = 0; /* pPKLW */
@@ -348,6 +347,10 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
VPRINTK("PRD[%u] = (0x%lX, 0x%X)\n", i/4,
(unsigned long)addr, len);
}
+
+ if (likely(last_buf))
+ *last_buf |= pEND;
+
return i;
}
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..7f1b13e 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -421,7 +421,6 @@ static void mv_error_handler(struct ata_port *ap);
static void mv_post_int_cmd(struct ata_queued_cmd *qc);
static void mv_eh_freeze(struct ata_port *ap);
static void mv_eh_thaw(struct ata_port *ap);
-static int mv_slave_config(struct scsi_device *sdev);
static int mv_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
static void mv5_phy_errata(struct mv_host_priv *hpriv, void __iomem *mmio,
@@ -459,7 +458,7 @@ static struct scsi_host_template mv5_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -477,7 +476,7 @@ static struct scsi_host_template mv6_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -756,17 +755,6 @@ static void mv_irq_clear(struct ata_port *ap)
{
}
-static int mv_slave_config(struct scsi_device *sdev)
-{
- int rc = ata_scsi_slave_config(sdev);
- if (rc)
- return rc;
-
- blk_queue_max_phys_segments(sdev->request_queue, MV_MAX_SG_CT / 2);
-
- return 0; /* scsi layer doesn't check return value, sigh */
-}
-
static void mv_set_edma_ptrs(void __iomem *port_mmio,
struct mv_host_priv *hpriv,
struct mv_port_priv *pp)
@@ -1138,7 +1126,7 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
{
struct mv_port_priv *pp = qc->ap->private_data;
struct scatterlist *sg;
- struct mv_sg *mv_sg;
+ struct mv_sg *mv_sg, *last_sg = NULL;
mv_sg = pp->sg_tbl;
ata_for_each_sg(sg, qc) {
@@ -1159,13 +1147,13 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
sg_len -= len;
addr += len;
- if (!sg_len && ata_sg_is_last(sg, qc))
- mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
-
+ last_sg = mv_sg;
mv_sg++;
}
-
}
+
+ if (likely(last_sg))
+ last_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
}
static inline void mv_crqb_pack_cmd(__le16 *cmdw, u8 data, u8 addr, unsigned last)
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index b061927..26ebffc 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -796,16 +796,19 @@ static inline void sil24_fill_sg(struct ata_queued_cmd *qc,
struct sil24_sge *sge)
{
struct scatterlist *sg;
+ struct sil24_sge *last_sge = NULL;
ata_for_each_sg(sg, qc) {
sge->addr = cpu_to_le64(sg_dma_address(sg));
sge->cnt = cpu_to_le32(sg_dma_len(sg));
- if (ata_sg_is_last(sg, qc))
- sge->flags = cpu_to_le32(SGE_TRM);
- else
- sge->flags = 0;
+ sge->flags = 0;
+
+ last_sge = sge;
sge++;
}
+
+ if (likely(last_sge))
+ last_sge->flags = cpu_to_le32(SGE_TRM);
}
static int sil24_qc_defer(struct ata_queued_cmd *qc)
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..c316a0b 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -5134,6 +5134,7 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
u32 ioadl_flags = 0;
struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
+ struct ipr_ioadl_desc *last_ioadl = NULL;
int len = qc->nbytes + qc->pad_len;
struct scatterlist *sg;
@@ -5156,11 +5157,13 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
ata_for_each_sg(sg, qc) {
ioadl->flags_and_data_len = cpu_to_be32(ioadl_flags | sg_dma_len(sg));
ioadl->address = cpu_to_be32(sg_dma_address(sg));
- if (ata_sg_is_last(sg, qc))
- ioadl->flags_and_data_len |= cpu_to_be32(IPR_IOADL_FLAGS_LAST);
- else
- ioadl++;
+
+ last_ioadl = ioadl;
+ ioadl++;
}
+
+ if (likely(last_ioadl))
+ last_ioadl->flags_and_data_len |= cpu_to_be32(IPR_IOADL_FLAGS_LAST);
}
/**
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 377e6d4..bc3b6fc 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -1037,18 +1037,6 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
/*
* qc helpers
*/
-static inline int
-ata_sg_is_last(struct scatterlist *sg, struct ata_queued_cmd *qc)
-{
- if (sg == &qc->pad_sgent)
- return 1;
- if (qc->pad_len)
- return 0;
- if (qc->n_iter == qc->n_elem)
- return 1;
- return 0;
-}
-
static inline struct scatterlist *
ata_qc_first_sg(struct ata_queued_cmd *qc)
{
On Thu, Oct 18 2007, Jeff Garzik wrote:
> Jens Axboe wrote:
>> That should work as well. WRT ata_sg_is_last(), if we go ahead with my
>> recent sg chaining updates, we can keep the test as it would be a single
>> conditional and not require any looping.
>> Let me know when you have tested this!
>
> The patch I attached to the last email got both sata_mv test boxes working
> reliably (so far).
>
> I worked up a patch that kills ata_sg_is_last() (plus the max_phys_segments
> sata_mv fix), see attached. I'm thinking this is what I like to see in
> upstream.
Great!
> Of course, this doesn't explain why ata_sg_is_last() was broken, but since
> it's working _and_ slightly more efficient, I don't really care :)
Tomo and I agreed to kill sg_last() a few days ago anyways, so this is
perfectly fine with me.
--
Jens Axboe
* Jens Axboe <[email protected]> wrote:
> > Of course, this doesn't explain why ata_sg_is_last() was broken, but
> > since it's working _and_ slightly more efficient, I don't really
> > care :)
>
> Tomo and I agreed to kill sg_last() a few days ago anyways, so this is
> perfectly fine with me.
it would still be nice to figure out exactly why it was broken.
Ingo
On Thu, Oct 18 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > > Of course, this doesn't explain why ata_sg_is_last() was broken, but
> > > since it's working _and_ slightly more efficient, I don't really
> > > care :)
> >
> > Tomo and I agreed to kill sg_last() a few days ago anyways, so this is
> > perfectly fine with me.
>
> it would still be nice to figure out exactly why it was broken.
It would always be nice. For this case I don't think it's very
interesting, if we pursue the improved sg iteration setup.
--
Jens Axboe
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, Ingo Molnar wrote:
> >
> > * Jens Axboe <[email protected]> wrote:
> >
> > > > Of course, this doesn't explain why ata_sg_is_last() was broken, but
> > > > since it's working _and_ slightly more efficient, I don't really
> > > > care :)
> > >
> > > Tomo and I agreed to kill sg_last() a few days ago anyways, so this is
> > > perfectly fine with me.
> >
> > it would still be nice to figure out exactly why it was broken.
>
> It would always be nice. For this case I don't think it's very
> interesting, if we pursue the improved sg iteration setup.
BTW, I think it's pretty clear that ata_sg_is_last() is broken. It's
likely a one-off in the n_iter test.
--
Jens Axboe
diff --git a/drivers/ata/pdc_adma.c b/drivers/ata/pdc_adma.c
index 8d1b03d..199f7e1 100644
--- a/drivers/ata/pdc_adma.c
+++ b/drivers/ata/pdc_adma.c
@@ -318,7 +318,7 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
struct scatterlist *sg;
struct ata_port *ap = qc->ap;
struct adma_port_priv *pp = ap->private_data;
- u8 *buf = pp->pkt;
+ u8 *buf = pp->pkt, *last_buf = NULL;
int i = (2 + buf[3]) * 8;
u8 pFLAGS = pORD | ((qc->tf.flags & ATA_TFLAG_WRITE) ? pDIRO : 0);
@@ -334,8 +334,7 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
*(__le32 *)(buf + i) = cpu_to_le32(len);
i += 4;
- if (ata_sg_is_last(sg, qc))
- pFLAGS |= pEND;
+ last_buf = &buf[i];
buf[i++] = pFLAGS;
buf[i++] = qc->dev->dma_mode & 0xf;
buf[i++] = 0; /* pPKLW */
@@ -348,6 +347,10 @@ static int adma_fill_sg(struct ata_queued_cmd *qc)
VPRINTK("PRD[%u] = (0x%lX, 0x%X)\n", i/4,
(unsigned long)addr, len);
}
+
+ if (likely(last_buf))
+ *last_buf |= pEND;
+
return i;
}
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 4df8311..7f1b13e 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -421,7 +421,6 @@ static void mv_error_handler(struct ata_port *ap);
static void mv_post_int_cmd(struct ata_queued_cmd *qc);
static void mv_eh_freeze(struct ata_port *ap);
static void mv_eh_thaw(struct ata_port *ap);
-static int mv_slave_config(struct scsi_device *sdev);
static int mv_init_one(struct pci_dev *pdev, const struct pci_device_id *ent);
static void mv5_phy_errata(struct mv_host_priv *hpriv, void __iomem *mmio,
@@ -459,7 +458,7 @@ static struct scsi_host_template mv5_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -477,7 +476,7 @@ static struct scsi_host_template mv6_sht = {
.use_clustering = 1,
.proc_name = DRV_NAME,
.dma_boundary = MV_DMA_BOUNDARY,
- .slave_configure = mv_slave_config,
+ .slave_configure = ata_scsi_slave_config,
.slave_destroy = ata_scsi_slave_destroy,
.bios_param = ata_std_bios_param,
};
@@ -756,17 +755,6 @@ static void mv_irq_clear(struct ata_port *ap)
{
}
-static int mv_slave_config(struct scsi_device *sdev)
-{
- int rc = ata_scsi_slave_config(sdev);
- if (rc)
- return rc;
-
- blk_queue_max_phys_segments(sdev->request_queue, MV_MAX_SG_CT / 2);
-
- return 0; /* scsi layer doesn't check return value, sigh */
-}
-
static void mv_set_edma_ptrs(void __iomem *port_mmio,
struct mv_host_priv *hpriv,
struct mv_port_priv *pp)
@@ -1138,7 +1126,7 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
{
struct mv_port_priv *pp = qc->ap->private_data;
struct scatterlist *sg;
- struct mv_sg *mv_sg;
+ struct mv_sg *mv_sg, *last_sg = NULL;
mv_sg = pp->sg_tbl;
ata_for_each_sg(sg, qc) {
@@ -1159,13 +1147,13 @@ static void mv_fill_sg(struct ata_queued_cmd *qc)
sg_len -= len;
addr += len;
- if (!sg_len && ata_sg_is_last(sg, qc))
- mv_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
-
+ last_sg = mv_sg;
mv_sg++;
}
-
}
+
+ if (likely(last_sg))
+ last_sg->flags_size |= cpu_to_le32(EPRD_FLAG_END_OF_TBL);
}
static inline void mv_crqb_pack_cmd(__le16 *cmdw, u8 data, u8 addr, unsigned last)
diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c
index b061927..26ebffc 100644
--- a/drivers/ata/sata_sil24.c
+++ b/drivers/ata/sata_sil24.c
@@ -796,16 +796,19 @@ static inline void sil24_fill_sg(struct ata_queued_cmd *qc,
struct sil24_sge *sge)
{
struct scatterlist *sg;
+ struct sil24_sge *last_sge = NULL;
ata_for_each_sg(sg, qc) {
sge->addr = cpu_to_le64(sg_dma_address(sg));
sge->cnt = cpu_to_le32(sg_dma_len(sg));
- if (ata_sg_is_last(sg, qc))
- sge->flags = cpu_to_le32(SGE_TRM);
- else
- sge->flags = 0;
+ sge->flags = 0;
+
+ last_sge = sge;
sge++;
}
+
+ if (likely(last_sge))
+ last_sge->flags = cpu_to_le32(SGE_TRM);
}
static int sil24_qc_defer(struct ata_queued_cmd *qc)
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..c316a0b 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -5134,6 +5134,7 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
u32 ioadl_flags = 0;
struct ipr_ioarcb *ioarcb = &ipr_cmd->ioarcb;
struct ipr_ioadl_desc *ioadl = ipr_cmd->ioadl;
+ struct ipr_ioadl_desc *last_ioadl = NULL;
int len = qc->nbytes + qc->pad_len;
struct scatterlist *sg;
@@ -5156,11 +5157,13 @@ static void ipr_build_ata_ioadl(struct ipr_cmnd *ipr_cmd,
ata_for_each_sg(sg, qc) {
ioadl->flags_and_data_len = cpu_to_be32(ioadl_flags | sg_dma_len(sg));
ioadl->address = cpu_to_be32(sg_dma_address(sg));
- if (ata_sg_is_last(sg, qc))
- ioadl->flags_and_data_len |= cpu_to_be32(IPR_IOADL_FLAGS_LAST);
- else
- ioadl++;
+
+ last_ioadl = ioadl;
+ ioadl++;
}
+
+ if (likely(last_ioadl))
+ last_ioadl->flags_and_data_len |= cpu_to_be32(IPR_IOADL_FLAGS_LAST);
}
/**
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 377e6d4..bc3b6fc 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -1037,18 +1037,6 @@ extern void ata_port_pbar_desc(struct ata_port *ap, int bar, ssize_t offset,
/*
* qc helpers
*/
-static inline int
-ata_sg_is_last(struct scatterlist *sg, struct ata_queued_cmd *qc)
-{
- if (sg == &qc->pad_sgent)
- return 1;
- if (qc->pad_len)
- return 0;
- if (qc->n_iter == qc->n_elem)
- return 1;
- return 0;
-}
-
static inline struct scatterlist *
ata_qc_first_sg(struct ata_queued_cmd *qc)
{
* Jens Axboe <[email protected]> wrote:
> > It would always be nice. For this case I don't think it's very
> > interesting, if we pursue the improved sg iteration setup.
>
> BTW, I think it's pretty clear that ata_sg_is_last() is broken. It's
> likely a one-off in the n_iter test.
fixing that would be a -stable candidate, as a potential data corruptor
- or is it more benign?
Ingo
Ingo Molnar wrote:
> * Jens Axboe <[email protected]> wrote:
>
>>> It would always be nice. For this case I don't think it's very
>>> interesting, if we pursue the improved sg iteration setup.
>> BTW, I think it's pretty clear that ata_sg_is_last() is broken. It's
>> likely a one-off in the n_iter test.
>
> fixing that would be a -stable candidate, as a potential data corruptor
> - or is it more benign?
It is confirmed working prior to the sg-chaining stuff, so 2.6.23.1 is OK...
Jeff
On Oct. 17, 2007, 20:22 +0200, Jens Axboe <[email protected]> wrote:
> On Wed, Oct 17 2007, Linus Torvalds wrote:
>>
>> On Wed, 17 Oct 2007, Jens Axboe wrote:
>>>> So avoiding the "sg_next()" on the last entry is pointless.
>>> Yeah, I didn't quite understand why if sg was valid, why dereferencing
>>> *(sg + 1)->page would crap out :/
>> Actually, I take that back. If 'sg' is the last entry in a *non*linked
>> scatter-gather list (ie we don't use the last entry as a link, we actually
>> use it as a real SG entry), then "sg_next(sg)" will indeed access past the
>> end of the whole allocated array, and will access one past the end.
>>
>> And with page-alloc debugging, that *will* blow up.
>>
>> So I think your change to use "sg_next()" only when you actually need a
>> next pointer is the correct one after all.
>
> Thanks, so I'm not totally crazy :-)
>
> Can you just pull:
>
> git://git.kernel.dk/data/git/linux-2.6-block.git for-linus
>
> then so we get those two pieces correct? Then the remaining issue seems
> to be a new one that is biting Ingo elsewhere, at least we'll all be on
> the same page then.
>
Jens, for_each_sg still calls sg_next on the last entry which will
dereference a possibly bogus sg->page (for the sg_is_chain(sg)
condition in sg_next) if the last entry is the last one on the page
of unchained entry and sg+1 falls over into an uninitialized page.
How about the following?
(untested yet.
sg.c included here as an example for usage out of scatterlist.h)
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..3a27e03 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -30,7 +30,7 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
/**
- * sg_next - return the next scatterlist entry in a list
+ * sg_next_unsafe - return the next scatterlist entry in a list
* @sg: The current sg entry
*
* Usually the next entry will be @sg@ + 1, but if this sg element is part
@@ -41,7 +41,7 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* the current entry, this function will NOT return NULL for an end-of-list.
*
*/
-static inline struct scatterlist *sg_next(struct scatterlist *sg)
+static inline struct scatterlist *sg_next_unsafe(struct scatterlist *sg)
{
sg++;
@@ -51,11 +51,27 @@ static inline struct scatterlist *sg_next(struct scatterlist *sg)
return sg;
}
+/**
+ * sg_next - return the next scatterlist entry in a list
+ * @sg: The current sg entry
+ * @next: Index of next sg entry
+ * @nr: Number of sg entries in the list
+ *
+ * Note that the caller must ensure that there are further entries after
+ * the current entry, this function will NOT return NULL for an end-of-list.
+ *
+ */
+static inline struct scatterlist *sg_next(struct scatterlist *sg,
+ int next, int nr)
+{
+ return next < nr ? sg_next_unsafe(sg) : NULL;
+}
+
/*
* Loop over each sg element, following the pointer to a new list if necessary
*/
#define for_each_sg(sglist, sg, nr, __i) \
- for (__i = 0, sg = (sglist); __i < (nr); __i++, sg = sg_next(sg))
+ for (__i = 0, sg = (sglist); sg; sg = sg_next(sg, ++__i, nr))
/**
* sg_last - return the last scatterlist entry in a list
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..57cc1dd 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1165,7 +1165,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
sg = rsv_schp->buffer;
sa = vma->vm_start;
for (k = 0; (k < rsv_schp->k_use_sg) && (sa < vma->vm_end);
- ++k, sg = sg_next(sg)) {
+ sg = sg_next(sg, ++k, rsv_schp->k_use_sg)) {
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
@@ -1209,7 +1209,7 @@ sg_mmap(struct file *filp, struct vm_area_struct *vma)
sa = vma->vm_start;
sg = rsv_schp->buffer;
for (k = 0; (k < rsv_schp->k_use_sg) && (sa < vma->vm_end);
- ++k, sg = sg_next(sg)) {
+ sg = sg_next(sg, ++k, rsv_schp->k_use_sg)) {
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
sa += len;
@@ -1840,7 +1840,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
}
for (k = 0, sg = schp->buffer, rem_sz = blk_size;
(rem_sz > 0) && (k < mx_sc_elems);
- ++k, rem_sz -= ret_sz, sg = sg_next(sg)) {
+ rem_sz -= ret_sz, sg = sg_next(sg, ++k, mx_sc_elems)) {
num = (rem_sz > scatter_elem_sz_prev) ?
scatter_elem_sz_prev : rem_sz;
@@ -1913,7 +1913,7 @@ sg_write_xfer(Sg_request * srp)
if (res)
return res;
- for (; p; sg = sg_next(sg), ksglen = sg->length,
+ for (; p; sg = sg_next_unsafe(sg), ksglen = sg->length,
p = page_address(sg->page)) {
if (usglen <= 0)
break;
@@ -1991,8 +1991,8 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
- ++k, sg = sg_next(sg)) {
+ for (k = 0; sg && sg->page;
+ sg = sg_next(sg, ++k, schp->k_use_sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
k, sg->page, sg->length));
@@ -2045,7 +2045,7 @@ sg_read_xfer(Sg_request * srp)
if (res)
return res;
- for (; p; sg = sg_next(sg), ksglen = sg->length,
+ for (; p; sg = sg_next_unsafe(sg), ksglen = sg->length,
p = page_address(sg->page)) {
if (usglen <= 0)
break;
@@ -2092,7 +2092,7 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; sg && sg->page; sg = sg_next(sg, ++k, schp->k_use_sg)) {
num = sg->length;
if (num > num_read_xfer) {
if (__copy_to_user(outp, page_address(sg->page),
@@ -2142,7 +2142,7 @@ sg_link_reserve(Sg_fd * sfp, Sg_request * srp, int size)
SCSI_LOG_TIMEOUT(4, printk("sg_link_reserve: size=%d\n", size));
rem = size;
- for (k = 0; k < rsv_schp->k_use_sg; ++k, sg = sg_next(sg)) {
+ for (k = 0; sg; sg = sg_next(sg, ++k, rsv_schp->k_use_sg)) {
num = sg->length;
if (rem <= num) {
sfp->save_scat_len = num;
* Jeff Garzik <[email protected]> wrote:
> >Tomo and I agreed to kill sg_last() a few days ago anyways, so this
> >is perfectly fine with me.
>
> Yep, the [attached] patch that kills ata_sg_is_last() is working here
> on both machines that were previously croaking.
>
> It would be nice to get pdc_adma, sata_sil24 and ipr it-works test
> done, but IMO the patch is pretty straightforward and should be OK.
just a quick question: i have Jens's workarounds applied right now (see
patch below). Am i now crash/corruption-safe, or do i need your patch
too? And once your patch [and the other sg_*() patches] are upstream i
dont need the workaround anymore, correct?
Ingo
---
block/ll_rw_blk.c | 2 +-
drivers/ata/libata-core.c | 2 +-
drivers/scsi/scsi_lib.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
Index: linux/block/ll_rw_blk.c
===================================================================
--- linux.orig/block/ll_rw_blk.c
+++ linux/block/ll_rw_blk.c
@@ -631,7 +631,7 @@ void blk_queue_max_phys_segments(struct
printk("%s: set to minimum %d\n", __FUNCTION__, max_segments);
}
- q->max_phys_segments = max_segments;
+ q->max_phys_segments = max_segments - 1;
}
EXPORT_SYMBOL(blk_queue_max_phys_segments);
Index: linux/drivers/ata/libata-core.c
===================================================================
--- linux.orig/drivers/ata/libata-core.c
+++ linux/drivers/ata/libata-core.c
@@ -4664,7 +4664,7 @@ static int ata_sg_setup(struct ata_queue
{
struct ata_port *ap = qc->ap;
struct scatterlist *sg = qc->__sg;
- struct scatterlist *lsg = sg_last(qc->__sg, qc->n_elem);
+ struct scatterlist *lsg = &qc->__sg[qc->n_elem - 1];
int n_elem, pre_n_elem, dir, trim_sg = 0;
VPRINTK("ENTER, ata%u\n", ap->print_id);
Index: linux/drivers/scsi/scsi_lib.c
===================================================================
--- linux.orig/drivers/scsi/scsi_lib.c
+++ linux/drivers/scsi/scsi_lib.c
@@ -39,7 +39,7 @@
* (unless chaining is used). Should ideally fit inside a single page, to
* avoid a higher order allocation.
*/
-#define SCSI_MAX_SG_SEGMENTS 128
+#define SCSI_MAX_SG_SEGMENTS 129
struct scsi_host_sg_pool {
size_t size;
On Thu, Oct 18 2007, Benny Halevy wrote:
> On Oct. 17, 2007, 20:22 +0200, Jens Axboe <[email protected]> wrote:
> > On Wed, Oct 17 2007, Linus Torvalds wrote:
> >>
> >> On Wed, 17 Oct 2007, Jens Axboe wrote:
> >>>> So avoiding the "sg_next()" on the last entry is pointless.
> >>> Yeah, I didn't quite understand why if sg was valid, why dereferencing
> >>> *(sg + 1)->page would crap out :/
> >> Actually, I take that back. If 'sg' is the last entry in a *non*linked
> >> scatter-gather list (ie we don't use the last entry as a link, we actually
> >> use it as a real SG entry), then "sg_next(sg)" will indeed access past the
> >> end of the whole allocated array, and will access one past the end.
> >>
> >> And with page-alloc debugging, that *will* blow up.
> >>
> >> So I think your change to use "sg_next()" only when you actually need a
> >> next pointer is the correct one after all.
> >
> > Thanks, so I'm not totally crazy :-)
> >
> > Can you just pull:
> >
> > git://git.kernel.dk/data/git/linux-2.6-block.git for-linus
> >
> > then so we get those two pieces correct? Then the remaining issue seems
> > to be a new one that is biting Ingo elsewhere, at least we'll all be on
> > the same page then.
> >
>
> Jens, for_each_sg still calls sg_next on the last entry which will
> dereference a possibly bogus sg->page (for the sg_is_chain(sg)
> condition in sg_next) if the last entry is the last one on the page
> of unchained entry and sg+1 falls over into an uninitialized page.
Things have progressed a lot since, see my recent posting based on
Davem's proposal. Will post another patch soonish, that is also tested.
--
Jens Axboe
On Thu, Oct 18 2007, Ingo Molnar wrote:
>
> * Jens Axboe <[email protected]> wrote:
>
> > > It would always be nice. For this case I don't think it's very
> > > interesting, if we pursue the improved sg iteration setup.
> >
> > BTW, I think it's pretty clear that ata_sg_is_last() is broken. It's
> > likely a one-off in the n_iter test.
>
> fixing that would be a -stable candidate, as a potential data corruptor
> - or is it more benign?
I think it's safe to say that it was sg chaining introduced breakage, so
it should work fine in 2.6.23.x.
--
Jens Axboe
Ingo Molnar wrote:
> * Jeff Garzik <[email protected]> wrote:
>
>>> Tomo and I agreed to kill sg_last() a few days ago anyways, so this
>>> is perfectly fine with me.
>> Yep, the [attached] patch that kills ata_sg_is_last() is working here
>> on both machines that were previously croaking.
>>
>> It would be nice to get pdc_adma, sata_sil24 and ipr it-works test
>> done, but IMO the patch is pretty straightforward and should be OK.
>
> just a quick question: i have Jens's workarounds applied right now (see
> patch below). Am i now crash/corruption-safe, or do i need your patch
> too? And once your patch [and the other sg_*() patches] are upstream i
> dont need the workaround anymore, correct?
You need my patch if and only if you use one of the drivers touched by
the patch. ata_sg_is_last() was a driver helper function, so my fix
never really touched core code.
I never had to apply the changes you included, to fix problems here.
And looking at those changes...
> - q->max_phys_segments = max_segments;
> + q->max_phys_segments = max_segments - 1;
...
> -#define SCSI_MAX_SG_SEGMENTS 128
> +#define SCSI_MAX_SG_SEGMENTS 129
I wonder if libata should be doing
blk_queue_max_phys_segments(q, q->max_phys_segments - 1)
to account for the pad entry that libata owns.
Jeff
* Ingo Molnar <[email protected]> wrote:
> > -#define SCSI_MAX_SG_SEGMENTS 128
> > +#define SCSI_MAX_SG_SEGMENTS 129
>
> this one finally made the trick and it's booting fine now, without any
> crashes!
hm, spoke too soon - i just got the ata_qc_issue() crash below. I've
attached further below the current fixes/workarounds that i have applied
at the moment. Any ideas?
Ingo
------------->
[ 155.259466] kjournald starting. Commit interval 5 seconds
[ 155.265103] EXT3 FS on sda5, internal journal
[ 155.269319] EXT3-fs: mounted filesystem with ordered data mode.
[ 156.458225] BUG: unable to handle kernel paging request at virtual address 7d5ac000
[ 156.465723] printing eip: 784e9300 *pde = 00ddd027 *pte = 055ac000
[ 156.471964] Oops: 0000 [#1] DEBUG_PAGEALLOC
[ 156.476123]
[ 156.477597] Pid: 0, comm: swapper Not tainted (2.6.23 #40)
[ 156.483055] EIP: 0060:[<784e9300>] EFLAGS: 00010006 CPU: 0
[ 156.488520] EIP is at ata_qc_issue+0xd0/0x340
[ 156.492848] EAX: 3d328000 EBX: 7d5ac000 ECX: 00000020 EDX: 00000020
[ 156.499087] ESI: 7d5ab480 EDI: 7d5abe00 EBP: 7b54007c ESP: 78a13e1c
[ 156.505328] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
[ 156.510700] Process swapper (pid: 0, ti=78a12000 task=789753e0 task.ti=78a12000)
[ 156.517893] Stack: 7d5ac000 7b540000 7b540000 00000000 7d5abff0 7b54007c 7d5ab480 7b5417a4
[ 156.526213] 784c2330 784ef49e 784f1ff3 7b52de98 7d5ab480 7b540000 7b5417a4 7d5ab480
[ 156.534531] 7b540000 7b524004 784f20e0 784ef180 784c2330 7d5ab480 00000216 7b524004
[ 156.542851] Call Trace:
[ 156.545452] [<784c2330>] scsi_done+0x0/0x20
[ 156.549698] [<784ef49e>] ata_scsi_translate+0xbe/0x140
[ 156.554897] [<784f1ff3>] ata_scsi_queuecmd+0x33/0x200
[ 156.560010] [<784f20e0>] ata_scsi_queuecmd+0x120/0x200
[ 156.565210] [<784ef180>] ata_scsi_rw_xlat+0x0/0x220
[ 156.570150] [<784c2330>] scsi_done+0x0/0x20
[ 156.574395] [<784c2bb2>] scsi_dispatch_cmd+0x152/0x290
[ 156.579596] [<78135aa7>] trace_hardirqs_on+0x67/0xb0
[ 156.584622] [<784c8abe>] scsi_request_fn+0x1be/0x370
[ 156.589649] [<78407ef6>] blk_run_queue+0x36/0x80
[ 156.594328] [<784c73c0>] scsi_next_command+0x30/0x50
[ 156.599354] [<784c754b>] scsi_end_request+0xab/0xe0
[ 156.604294] [<784c8239>] scsi_io_completion+0xa9/0x3d0
[ 156.609493] [<78135aa7>] trace_hardirqs_on+0x67/0xb0
[ 156.614520] [<78404f85>] blk_done_softirq+0x45/0x80
[ 156.619460] [<78404fb3>] blk_done_softirq+0x73/0x80
[ 156.624400] [<7811d2f3>] __do_softirq+0x53/0xb0
[ 156.628992] [<7811d3b8>] do_softirq+0x68/0x70
[ 156.633412] [<78105351>] do_IRQ+0x51/0x90
[ 156.637486] [<7810290f>] restore_nocheck+0x12/0x15
[ 156.642339] [<7810388e>] common_interrupt+0x2e/0x40
[ 156.647277] [<7810f4c0>] pgd_dtor+0x0/0x50
[ 156.651437] [<7815f1d0>] quicklist_trim+0x0/0x90
[ 156.656117] [<7810f4bb>] check_pgt_cache+0x1b/0x20
[ 156.660970] [<78100c52>] cpu_idle+0x32/0x60
[ 156.665217] [<78a14b35>] start_kernel+0x265/0x300
[ 156.669983] [<78a14380>] unknown_bootoption+0x0/0x1e0
[ 156.675096] =======================
[ 156.678649] Code: 84 d9 01 00 00 7e 32 31 d2 89 f6 8b 1c 24 83 c2 01 8b 03 2b 05 18 ed d7 78 c1 f8 05 c1 e0 0c 03 43 04 89 43 08 83 c3 10 89 1c 24 <8b> 03 a8 01 0f 85 58 02 00 00 39 ca 75 d2 f0 83 44 24 00 00 85
[ 156.697455] EIP: [<784e9300>] ata_qc_issue+0xd0/0x340 SS:ESP 0068:78a13e1c
[ 156.704822] Kernel panic - not syncing: Fatal exception in interrupt
---
block/ll_rw_blk.c | 2 +-
drivers/ata/libata-core.c | 2 +-
drivers/scsi/scsi_lib.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
Index: linux/block/ll_rw_blk.c
===================================================================
--- linux.orig/block/ll_rw_blk.c
+++ linux/block/ll_rw_blk.c
@@ -631,7 +631,7 @@ void blk_queue_max_phys_segments(struct
printk("%s: set to minimum %d\n", __FUNCTION__, max_segments);
}
- q->max_phys_segments = max_segments;
+ q->max_phys_segments = max_segments - 1;
}
EXPORT_SYMBOL(blk_queue_max_phys_segments);
Index: linux/drivers/ata/libata-core.c
===================================================================
--- linux.orig/drivers/ata/libata-core.c
+++ linux/drivers/ata/libata-core.c
@@ -4664,7 +4664,7 @@ static int ata_sg_setup(struct ata_queue
{
struct ata_port *ap = qc->ap;
struct scatterlist *sg = qc->__sg;
- struct scatterlist *lsg = sg_last(qc->__sg, qc->n_elem);
+ struct scatterlist *lsg = &qc->__sg[qc->n_elem - 1];
int n_elem, pre_n_elem, dir, trim_sg = 0;
VPRINTK("ENTER, ata%u\n", ap->print_id);
Index: linux/drivers/scsi/scsi_lib.c
===================================================================
--- linux.orig/drivers/scsi/scsi_lib.c
+++ linux/drivers/scsi/scsi_lib.c
@@ -39,7 +39,7 @@
* (unless chaining is used). Should ideally fit inside a single page, to
* avoid a higher order allocation.
*/
-#define SCSI_MAX_SG_SEGMENTS 128
+#define SCSI_MAX_SG_SEGMENTS 129
struct scsi_host_sg_pool {
size_t size;
On Thu, Oct 18 2007, Ingo Molnar wrote:
>
> * Ingo Molnar <[email protected]> wrote:
>
> > > -#define SCSI_MAX_SG_SEGMENTS 128
> > > +#define SCSI_MAX_SG_SEGMENTS 129
> >
> > this one finally made the trick and it's booting fine now, without any
> > crashes!
>
> hm, spoke too soon - i just got the ata_qc_issue() crash below. I've
> attached further below the current fixes/workarounds that i have applied
> at the moment. Any ideas?
Hang on Ingo, will post an updated patch soonish!
--
Jens Axboe
* Jeff Garzik <[email protected]> wrote:
> I never had to apply the changes you included, to fix problems here.
perhaps because you are not running a CONFIG_DEBUG_PAGEALLOC=y kernel?
I recently fixed DEBUG_PAGEALLOC (it would crash upon bootup on x86 most
of the time on any real hardware - so i doubt people were able to use it
all that much). As long as you try Linus' latest -git tree (which has
the latest arch/x86 merge) and use the 32-bit x86 kernel it should work
fine for you too, and you will probably be able to trigger similar
crashes too.
Ingo
From: Jens Axboe <[email protected]>
Date: Thu, 18 Oct 2007 10:21:45 +0200
> I like it. Basically the only real change is using bit 2 as a
> termination point, so we avoid going beyond the end of the sgtable.
> Here's a starting point, it actually booted for me in the first go
> (boggle). Only x86 so far, archs will need to be converted. And lots
> more drivers I'm sure, I only fixed up the ones that botched my compile.
>
> So just consider this a directional patch.
Here are some sparc64 bits:
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
On Thu, Oct 18 2007, David Miller wrote:
> From: Jens Axboe <[email protected]>
> Date: Thu, 18 Oct 2007 10:21:45 +0200
>
> > I like it. Basically the only real change is using bit 2 as a
> > termination point, so we avoid going beyond the end of the sgtable.
> > Here's a starting point, it actually booted for me in the first go
> > (boggle). Only x86 so far, archs will need to be converted. And lots
> > more drivers I'm sure, I only fixed up the ones that botched my compile.
> >
> > So just consider this a directional patch.
>
> Here are some sparc64 bits:
Thanks a lot, Dave! The patch is a monster right now, I'll work on
splitting it into a 3-step process. Any arch help is greatly
appreciated.
--
Jens Axboe
From: Jens Axboe <[email protected]>
Date: Thu, 18 Oct 2007 12:55:17 +0200
> Things have progressed a lot since, see my recent posting based on
> Davem's proposal. Will post another patch soonish, that is also
> tested.
One core issue here is that we need to decide whether this thing to be
iterated like an array or like a linked list. It's trying to be both.
If we decide upon a looping construct for consumers and stick to it,
we'll be in much better shape than we are now and bugs will be eaiser
to spot. It would be so much simpler to audit if all we saw in the
consumers were things like:
while (sg) {
do_stuff(sg);
sg = sg_next(sg);
}
I would suggest that we just get it over with and convert the whole
tree now rather than trying to do this kind of thing in stages.
Because then we can say that ever scatterlist creator has to set
the "end" bit and therefore you use well established patterns
for scatterlist iteration such as "traverse sg_next() until NULL"
as shown above.
I also noticed that there is the issue of on-stack and embedded
scatterlist users. We'll need some sort of "DECLARE_SCATTERLIST"
and a "scatterlist_init()" thing so that we can keep DEBUG_SG
working even in those cases. But for all I know Jens could be
working on that already :-)
The only other real option if we don't convert the whole tree now to
the "end" marker stuff, is to enforce that every scatterlist iterator
only traverse the number of entries there were told are in the one
given to them.
From: Jens Axboe <[email protected]>
Date: Thu, 18 Oct 2007 13:57:02 +0200
> Thanks a lot, Dave! The patch is a monster right now, I'll work on
> splitting it into a 3-step process. Any arch help is greatly
> appreciated.
I have some other bits that my compile hit, such as some things in the
crypto layer.
But I hesitate to send them to you because I think the on-stack cases
need some helpers such that DEBUG_SG works for them.
BTW, you missed a case in drivers/usb/core/message.c because of
the config used in your build. This thing below is a good
argument for trying to avoid HIGHMEM et al. ifdefs in drivers :-)
--- drivers/usb/core/message.c~ 2007-10-18 01:46:44.000000000 -0700
+++ drivers/usb/core/message.c 2007-10-18 03:15:20.000000000 -0700
@@ -438,7 +438,7 @@
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
On Thu, Oct 18 2007, David Miller wrote:
> From: Jens Axboe <[email protected]>
> Date: Thu, 18 Oct 2007 13:57:02 +0200
>
> > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > splitting it into a 3-step process. Any arch help is greatly
> > appreciated.
>
> I have some other bits that my compile hit, such as some things in the
> crypto layer.
Yeah, I have tons of that so far. I hope to have an allyesconfig
compiling pretty soonish, will send that out then.
> But I hesitate to send them to you because I think the on-stack cases
> need some helpers such that DEBUG_SG works for them.
Indeed. I convert where appropriate, but I'm not anal about it. If they
don't use sg_next() and/or for_each_sg() on their list, it should be ok.
Don't want to make the changes more than necessary right now.
> BTW, you missed a case in drivers/usb/core/message.c because of
> the config used in your build. This thing below is a good
> argument for trying to avoid HIGHMEM et al. ifdefs in drivers :-)
Thanks :-)
--
Jens Axboe
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, David Miller wrote:
> > From: Jens Axboe <[email protected]>
> > Date: Thu, 18 Oct 2007 13:57:02 +0200
> >
> > > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > > splitting it into a 3-step process. Any arch help is greatly
> > > appreciated.
> >
> > I have some other bits that my compile hit, such as some things in the
> > crypto layer.
>
> Yeah, I have tons of that so far. I hope to have an allyesconfig
> compiling pretty soonish, will send that out then.
OK here goes, this compiles with allyesconfig on x86-64. Not too bad,
the scsi/ drivers were by far the worst.
arch/sparc64/kernel/iommu.c | 4 -
arch/sparc64/kernel/iommu_common.c | 12 ++--
arch/sparc64/kernel/pci_sun4v.c | 4 -
arch/x86/kernel/pci-calgary_64.c | 12 ++--
arch/x86/kernel/pci-gart_64.c | 4 -
arch/x86/kernel/pci-nommu_64.c | 4 -
block/ll_rw_blk.c | 5 +
crypto/digest.c | 2
crypto/hmac.c | 3 -
crypto/scatterwalk.c | 2
crypto/scatterwalk.h | 6 +-
crypto/tcrypt.c | 4 -
crypto/xcbc.c | 2
drivers/ata/libata-core.c | 10 +--
drivers/ata/libata-scsi.c | 2
drivers/block/cciss.c | 4 -
drivers/block/cpqarray.c | 3 -
drivers/block/cryptoloop.c | 12 ++--
drivers/block/ub.c | 8 +-
drivers/ide/ide-probe.c | 4 +
drivers/ide/ide-taskfile.c | 2
drivers/ieee1394/dma.c | 2
drivers/ieee1394/sbp2.c | 2
drivers/infiniband/core/umem.c | 11 ++-
drivers/infiniband/hw/ipath/ipath_dma.c | 4 -
drivers/infiniband/hw/ipath/ipath_mr.c | 2
drivers/infiniband/hw/mthca/mthca_memfree.c | 24 +++++---
drivers/infiniband/ulp/iser/iser_memory.c | 6 +-
drivers/md/dm-crypt.c | 21 +++----
drivers/media/common/saa7146_core.c | 3 -
drivers/media/video/ivtv/ivtv-udma.c | 4 -
drivers/media/video/videobuf-dma-sg.c | 8 +-
drivers/mmc/card/queue.c | 15 ++---
drivers/mmc/host/mmc_spi.c | 8 +-
drivers/mmc/host/sdhci.c | 2
drivers/mmc/host/tifm_sd.c | 8 +-
drivers/mmc/host/wbsd.c | 6 +-
drivers/net/mlx4/icm.c | 14 +++-
drivers/net/ppp_mppe.c | 6 --
drivers/scsi/3w-9xxx.c | 4 -
drivers/scsi/3w-xxxx.c | 2
drivers/scsi/NCR5380.c | 4 -
drivers/scsi/aacraid/aachba.c | 2
drivers/scsi/arcmsr/arcmsr_hba.c | 4 -
drivers/scsi/fdomain.c | 6 +-
drivers/scsi/gdth.c | 4 -
drivers/scsi/ide-scsi.c | 8 +-
drivers/scsi/imm.c | 4 -
drivers/scsi/ipr.c | 19 ++++--
drivers/scsi/ips.c | 6 +-
drivers/scsi/iscsi_tcp.c | 15 ++---
drivers/scsi/megaraid.c | 6 +-
drivers/scsi/megaraid/megaraid_mbox.c | 8 +-
drivers/scsi/osst.c | 32 +++++-----
drivers/scsi/pcmcia/sym53c500_cs.c | 6 +-
drivers/scsi/ppa.c | 4 -
drivers/scsi/qlogicfas408.c | 2
drivers/scsi/scsi_debug.c | 4 -
drivers/scsi/scsi_lib.c | 13 +++-
drivers/scsi/sg.c | 30 +++++-----
drivers/scsi/st.c | 8 +-
drivers/scsi/tmscsim.c | 5 -
drivers/usb/core/message.c | 4 -
drivers/usb/image/microtek.c | 4 -
drivers/usb/misc/usbtest.c | 5 +
drivers/usb/storage/protocol.c | 2
fs/ecryptfs/crypto.c | 16 +++--
fs/ecryptfs/keystore.c | 3 +
fs/nfsd/nfs4recover.c | 8 +-
include/asm-sparc64/scatterlist.h | 5 +
include/asm-x86/dma-mapping_32.h | 4 -
include/asm-x86/scatterlist_32.h | 5 +
include/asm-x86/scatterlist_64.h | 5 +
include/linux/scatterlist.h | 82 +++++++++++++++++++++++-----
lib/Kconfig.debug | 10 +++
lib/swiotlb.c | 2
net/core/skbuff.c | 4 -
net/ieee80211/ieee80211_crypt_tkip.c | 13 +---
net/ieee80211/ieee80211_crypt_wep.c | 8 --
net/mac80211/wep.c | 8 --
net/sctp/auth.c | 3 -
net/sctp/sm_make_chunk.c | 6 +-
net/sunrpc/auth_gss/gss_krb5_crypto.c | 10 +--
net/sunrpc/xdr.c | 2
net/xfrm/xfrm_algo.c | 4 -
85 files changed, 389 insertions(+), 275 deletions(-)
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5098f58..c9f0eb0 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -411,8 +411,10 @@ static int calgary_nontranslate_map_sg(struct device* dev,
int i;
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
+ s->dma_address = virt_to_bus(page_address(p) + s->offset);
s->dma_length = s->length;
}
return nelems;
@@ -432,9 +434,11 @@ static int calgary_map_sg(struct device *dev, struct scatterlist *sg,
return calgary_nontranslate_map_sg(dev, sg, nelems, direction);
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(p) + s->offset;
npages = num_dma_pages(vaddr, s->length);
entry = iommu_range_alloc(tbl, npages);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..ce9754f 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,8 +1354,9 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
nsegs++;
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/hmac.c b/crypto/hmac.c
index 8802fb6..e4eb6ac 100644
--- a/crypto/hmac.c
+++ b/crypto/hmac.c
@@ -159,7 +159,8 @@ static int hmac_digest(struct hash_desc *pdesc, struct scatterlist *sg,
desc.flags = pdesc->flags & CRYPTO_TFM_REQ_MAY_SLEEP;
sg_set_buf(sg1, ipad, bs);
- sg1[1].page = (void *)sg;
+
+ sg_set_page(&sg[1], (void *) sg);
sg1[1].length = 0;
sg_set_buf(sg2, opad, bs + ds);
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 18d489c..d741c63 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -317,7 +317,7 @@ static void test_cipher(char *algo, int enc,
goto out;
}
- q = kmap(sg[0].page) + sg[0].offset;
+ q = kmap(sg_page(&sg[0])) + sg[0].offset;
hexdump(q, cipher_tv[i].rlen);
printk("%s\n",
@@ -390,7 +390,7 @@ static void test_cipher(char *algo, int enc,
temp = 0;
for (k = 0; k < cipher_tv[i].np; k++) {
printk("page %u\n", k);
- q = kmap(sg[k].page) + sg[k].offset;
+ q = kmap(sg_page(&sg[k])) + sg[k].offset;
hexdump(q, cipher_tv[i].tap[k]);
printk("%s\n",
memcmp(q, cipher_tv[i].result + temp,
diff --git a/crypto/xcbc.c b/crypto/xcbc.c
index 9f502b8..ac68f3b 100644
--- a/crypto/xcbc.c
+++ b/crypto/xcbc.c
@@ -120,7 +120,7 @@ static int crypto_xcbc_digest_update2(struct hash_desc *pdesc,
do {
- struct page *pg = sg[i].page;
+ struct page *pg = sg_page(&sg[i]);
unsigned int offset = sg[i].offset;
unsigned int slen = sg[i].length;
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
index 7c2cfde..5a6fe17 100644
--- a/drivers/block/cciss.c
+++ b/drivers/block/cciss.c
@@ -2610,7 +2610,7 @@ static void do_cciss_request(struct request_queue *q)
(int)creq->nr_sectors);
#endif /* CCISS_DEBUG */
- memset(tmp_sg, 0, sizeof(tmp_sg));
+ sg_init_table(tmp_sg, MAXSGENTRIES);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* get the DMA records for the setup */
@@ -2621,7 +2621,7 @@ static void do_cciss_request(struct request_queue *q)
for (i = 0; i < seg; i++) {
c->SG[i].Len = tmp_sg[i].length;
- temp64.val = (__u64) pci_map_page(h->pdev, tmp_sg[i].page,
+ temp64.val = (__u64) pci_map_page(h->pdev, sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
c->SG[i].Addr.lower = temp64.val32.lower;
diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c
index 568603d..efab27f 100644
--- a/drivers/block/cpqarray.c
+++ b/drivers/block/cpqarray.c
@@ -918,6 +918,7 @@ queue_next:
DBGPX(
printk("sector=%d, nr_sectors=%d\n", creq->sector, creq->nr_sectors);
);
+ sg_init_table(tmp_sg, SG_MAX);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* Now do all the DMA Mappings */
@@ -929,7 +930,7 @@ DBGPX(
{
c->req.sg[i].size = tmp_sg[i].length;
c->req.sg[i].addr = (__u32) pci_map_page(h->pci_dev,
- tmp_sg[i].page,
+ sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
}
diff --git a/drivers/block/cryptoloop.c b/drivers/block/cryptoloop.c
index 4053503..1b58b01 100644
--- a/drivers/block/cryptoloop.c
+++ b/drivers/block/cryptoloop.c
@@ -26,6 +26,7 @@
#include <linux/crypto.h>
#include <linux/blkdev.h>
#include <linux/loop.h>
+#include <linux/scatterlist.h>
#include <asm/semaphore.h>
#include <asm/uaccess.h>
@@ -119,14 +120,17 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
.tfm = tfm,
.flags = CRYPTO_TFM_REQ_MAY_SLEEP,
};
- struct scatterlist sg_out = { NULL, };
- struct scatterlist sg_in = { NULL, };
+ struct scatterlist sg_out;
+ struct scatterlist sg_in;
encdec_cbc_t encdecfunc;
struct page *in_page, *out_page;
unsigned in_offs, out_offs;
int err;
+ sg_init_table(&sg_out, 1);
+ sg_init_table(&sg_in, 1);
+
if (cmd == READ) {
in_page = raw_page;
in_offs = raw_off;
@@ -146,11 +150,11 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
u32 iv[4] = { 0, };
iv[0] = cpu_to_le32(IV & 0xffffffff);
- sg_in.page = in_page;
+ sg_set_page(&sg_in, in_page);
sg_in.offset = in_offs;
sg_in.length = sz;
- sg_out.page = out_page;
+ sg_set_page(&sg_out, out_page);
sg_out.offset = out_offs;
sg_out.length = sz;
diff --git a/drivers/block/ub.c b/drivers/block/ub.c
index c57dd2b..d6f7b43 100644
--- a/drivers/block/ub.c
+++ b/drivers/block/ub.c
@@ -25,6 +25,7 @@
#include <linux/usb_usual.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
+#include <linux/scatterlist.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
@@ -656,6 +657,7 @@ static int ub_request_fn_1(struct ub_lun *lun, struct request *rq)
if ((cmd = ub_get_cmd(lun)) == NULL)
return -1;
memset(cmd, 0, sizeof(struct ub_scsi_cmd));
+ sg_init_table(cmd->sgv, UB_MAX_REQ_SG);
blkdev_dequeue_request(rq);
@@ -1310,7 +1312,7 @@ static void ub_data_start(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
pipe = sc->send_bulk_pipe;
sc->last_pipe = pipe;
usb_fill_bulk_urb(&sc->work_urb, sc->dev, pipe,
- page_address(sg->page) + sg->offset, sg->length,
+ page_address(sg_page(sg)) + sg->offset, sg->length,
ub_urb_complete, sc);
sc->work_urb.actual_length = 0;
sc->work_urb.error_count = 0;
@@ -1427,7 +1429,7 @@ static void ub_state_sense(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
scmd->state = UB_CMDST_INIT;
scmd->nsg = 1;
sg = &scmd->sgv[0];
- sg->page = virt_to_page(sc->top_sense);
+ sg_set_page(sg, virt_to_page(sc->top_sense));
sg->offset = (unsigned long)sc->top_sense & (PAGE_SIZE-1);
sg->length = UB_SENSE_SIZE;
scmd->len = UB_SENSE_SIZE;
@@ -1863,7 +1865,7 @@ static int ub_sync_read_cap(struct ub_dev *sc, struct ub_lun *lun,
cmd->state = UB_CMDST_INIT;
cmd->nsg = 1;
sg = &cmd->sgv[0];
- sg->page = virt_to_page(p);
+ sg_set_page(sg, virt_to_page(p));
sg->offset = (unsigned long)p & (PAGE_SIZE-1);
sg->length = 8;
cmd->len = 8;
diff --git a/drivers/ide/ide-probe.c b/drivers/ide/ide-probe.c
index 3c945d6..7c640cd 100644
--- a/drivers/ide/ide-probe.c
+++ b/drivers/ide/ide-probe.c
@@ -1337,12 +1337,14 @@ static int hwif_init(ide_hwif_t *hwif)
if (!hwif->sg_max_nents)
hwif->sg_max_nents = PRD_ENTRIES;
- hwif->sg_table = kzalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
+ hwif->sg_table = kmalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
GFP_KERNEL);
if (!hwif->sg_table) {
printk(KERN_ERR "%s: unable to allocate SG table.\n", hwif->name);
goto out;
}
+
+ sg_init_table(hwif->sg_table, hwif->sg_max_nents);
if (init_irq(hwif) == 0)
goto done;
diff --git a/drivers/ide/ide-taskfile.c b/drivers/ide/ide-taskfile.c
index 2a3c8d4..086157f 100644
--- a/drivers/ide/ide-taskfile.c
+++ b/drivers/ide/ide-taskfile.c
@@ -278,7 +278,7 @@ static void ide_pio_sector(ide_drive_t *drive, unsigned int write)
hwif->cursg = sg;
}
- page = cursg->page;
+ page = sg_page(cursg);
offset = cursg->offset + hwif->cursg_ofs * SECTOR_SIZE;
/* get the current page and offset */
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/ieee1394/sbp2.c b/drivers/ieee1394/sbp2.c
index 1b353b9..d5dfe11 100644
--- a/drivers/ieee1394/sbp2.c
+++ b/drivers/ieee1394/sbp2.c
@@ -1466,7 +1466,7 @@ static void sbp2_prep_command_orb_sg(struct sbp2_command_orb *orb,
cmd->dma_size = sgpnt[0].length;
cmd->dma_type = CMD_DMA_PAGE;
cmd->cmd_dma = dma_map_page(hi->host->device.parent,
- sgpnt[0].page, sgpnt[0].offset,
+ sg_page(&sgpnt[0]), sgpnt[0].offset,
cmd->dma_size, cmd->dma_dir);
orb->data_descriptor_lo = cmd->cmd_dma;
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 2f54e29..14159ff 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -55,9 +55,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
ib_dma_unmap_sg(dev, chunk->page_list,
chunk->nents, DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->nents; ++i) {
+ struct page *page = sg_page(&chunk->page_list[i]);
+
if (umem->writable && dirty)
- set_page_dirty_lock(chunk->page_list[i].page);
- put_page(chunk->page_list[i].page);
+ set_page_dirty_lock(page);
+ put_page(page);
}
kfree(chunk);
@@ -164,11 +166,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
}
chunk->nents = min_t(int, ret, IB_UMEM_MAX_PAGE_CHUNK);
+ sg_init_table(chunk->page_list, chunk->nents);
for (i = 0; i < chunk->nents; ++i) {
if (vma_list &&
!is_vm_hugetlb_page(vma_list[i + off]))
umem->hugetlb = 0;
- chunk->page_list[i].page = page_list[i + off];
+ sg_set_page(&chunk->page_list[i], page_list[i + off]);
chunk->page_list[i].offset = 0;
chunk->page_list[i].length = PAGE_SIZE;
}
@@ -179,7 +182,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
DMA_BIDIRECTIONAL);
if (chunk->nmap <= 0) {
for (i = 0; i < chunk->nents; ++i)
- put_page(chunk->page_list[i].page);
+ put_page(sg_page(&chunk->page_list[i]));
kfree(chunk);
ret = -ENOMEM;
diff --git a/drivers/infiniband/hw/ipath/ipath_dma.c b/drivers/infiniband/hw/ipath/ipath_dma.c
index 22709a4..e90a0ea 100644
--- a/drivers/infiniband/hw/ipath/ipath_dma.c
+++ b/drivers/infiniband/hw/ipath/ipath_dma.c
@@ -108,7 +108,7 @@ static int ipath_map_sg(struct ib_device *dev, struct scatterlist *sgl,
BUG_ON(!valid_dma_direction(direction));
for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg->page);
+ addr = (u64) page_address(sg_page(sg));
/* TODO: handle highmem pages */
if (!addr) {
ret = 0;
@@ -127,7 +127,7 @@ static void ipath_unmap_sg(struct ib_device *dev,
static u64 ipath_sg_dma_address(struct ib_device *dev, struct scatterlist *sg)
{
- u64 addr = (u64) page_address(sg->page);
+ u64 addr = (u64) page_address(sg_page(sg));
if (addr)
addr += sg->offset;
diff --git a/drivers/infiniband/hw/ipath/ipath_mr.c b/drivers/infiniband/hw/ipath/ipath_mr.c
index e442470..db4ba92 100644
--- a/drivers/infiniband/hw/ipath/ipath_mr.c
+++ b/drivers/infiniband/hw/ipath/ipath_mr.c
@@ -225,7 +225,7 @@ struct ib_mr *ipath_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
for (i = 0; i < chunk->nents; i++) {
void *vaddr;
- vaddr = page_address(chunk->page_list[i].page);
+ vaddr = page_address(sg_page(&chunk->page_list[i]));
if (!vaddr) {
ret = ERR_PTR(-EINVAL);
goto bail;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index e61f3e6..007b381 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -71,7 +71,7 @@ static void mthca_free_icm_pages(struct mthca_dev *dev, struct mthca_icm_chunk *
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -81,7 +81,7 @@ static void mthca_free_icm_coherent(struct mthca_dev *dev, struct mthca_icm_chun
for (i = 0; i < chunk->npages; ++i) {
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
}
@@ -107,10 +107,13 @@ void mthca_free_icm(struct mthca_dev *dev, struct mthca_icm *icm, int coherent)
static int mthca_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -157,6 +160,7 @@ struct mthca_icm *mthca_alloc_icm(struct mthca_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MTHCA_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -304,7 +308,7 @@ void *mthca_table_find(struct mthca_icm_table *table, int obj, dma_addr_t *dma_h
* so if we found the page, dma_handle has already
* been assigned to. */
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
@@ -445,6 +449,7 @@ static u64 mthca_uarc_virt(struct mthca_dev *dev, struct mthca_uar *uar, int pag
int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
struct mthca_user_db_table *db_tab, int index, u64 uaddr)
{
+ struct page *pages[1];
int ret = 0;
u8 status;
int i;
@@ -472,16 +477,17 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
}
ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- &db_tab->page[i].mem.page, NULL);
+ pages, NULL);
if (ret < 0)
goto out;
+ sg_set_page(&db_tab->page[i].mem, pages[0]);
db_tab->page[i].mem.length = MTHCA_ICM_PAGE_SIZE;
db_tab->page[i].mem.offset = uaddr & ~PAGE_MASK;
ret = pci_map_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
if (ret < 0) {
- put_page(db_tab->page[i].mem.page);
+ put_page(pages[0]);
goto out;
}
@@ -491,7 +497,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
ret = -EINVAL;
if (ret) {
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
goto out;
}
@@ -557,7 +563,7 @@ void mthca_cleanup_user_db_tab(struct mthca_dev *dev, struct mthca_uar *uar,
if (db_tab->page[i].uvirt) {
mthca_UNMAP_ICM(dev, mthca_uarc_virt(dev, uar, i), 1, &status);
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
}
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index f3529b6..29c842e 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -131,7 +131,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, data->size, i) {
- from = kmap_atomic(sg->page, KM_USER0);
+ from = kmap_atomic(sg_page(sg), KM_USER0);
memcpy(p,
from + sg->offset,
sg->length);
@@ -191,7 +191,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, sg_size, i) {
- to = kmap_atomic(sg->page, KM_SOFTIRQ0);
+ to = kmap_atomic(sg_page(sg), KM_SOFTIRQ0);
memcpy(to + sg->offset,
p,
sg->length);
@@ -336,7 +336,7 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
- sg->page, sg->offset,
+ sg_page(sg), sg->offset,
sg->length, ib_sg_dma_len(ibdev, sg));
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 64fee90..cde915f 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -346,16 +346,17 @@ static int crypt_convert(struct crypt_config *cc,
ctx->idx_out < ctx->bio_out->bi_vcnt) {
struct bio_vec *bv_in = bio_iovec_idx(ctx->bio_in, ctx->idx_in);
struct bio_vec *bv_out = bio_iovec_idx(ctx->bio_out, ctx->idx_out);
- struct scatterlist sg_in = {
- .page = bv_in->bv_page,
- .offset = bv_in->bv_offset + ctx->offset_in,
- .length = 1 << SECTOR_SHIFT
- };
- struct scatterlist sg_out = {
- .page = bv_out->bv_page,
- .offset = bv_out->bv_offset + ctx->offset_out,
- .length = 1 << SECTOR_SHIFT
- };
+ struct scatterlist sg_in, sg_out;
+
+ sg_init_table(&sg_in, 1);
+ sg_set_page(&sg_in, bv_in->bv_page);
+ sg_in.offset = bv_in->bv_offset + ctx->offset_in;
+ sg_in.length = 1 << SECTOR_SHIFT;
+
+ sg_init_table(&sg_out, 1);
+ sg_set_page(&sg_out, bv_out->bv_page);
+ sg_out.offset = bv_out->bv_offset + ctx->offset_out;
+ sg_out.length = 1 << SECTOR_SHIFT;
ctx->offset_in += sg_in.length;
if (ctx->offset_in >= bv_in->bv_len) {
diff --git a/drivers/media/common/saa7146_core.c b/drivers/media/common/saa7146_core.c
index 365a221..2b1f8b4 100644
--- a/drivers/media/common/saa7146_core.c
+++ b/drivers/media/common/saa7146_core.c
@@ -112,12 +112,13 @@ static struct scatterlist* vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/media/video/ivtv/ivtv-udma.c b/drivers/media/video/ivtv/ivtv-udma.c
index c4626d1..912b424 100644
--- a/drivers/media/video/ivtv/ivtv-udma.c
+++ b/drivers/media/video/ivtv/ivtv-udma.c
@@ -63,10 +63,10 @@ int ivtv_udma_fill_sg_list (struct ivtv_user_dma *dma, struct ivtv_dma_page_info
memcpy(page_address(dma->bouncemap[map_offset]) + offset, src, len);
kunmap_atomic(src, KM_BOUNCE_READ);
local_irq_restore(flags);
- dma->SGlist[map_offset].page = dma->bouncemap[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->bouncemap[map_offset]);
}
else {
- dma->SGlist[map_offset].page = dma->map[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->map[map_offset]);
}
offset = 0;
map_offset++;
diff --git a/drivers/media/video/videobuf-dma-sg.c b/drivers/media/video/videobuf-dma-sg.c
index 3eb6123..0a18286 100644
--- a/drivers/media/video/videobuf-dma-sg.c
+++ b/drivers/media/video/videobuf-dma-sg.c
@@ -60,12 +60,13 @@ videobuf_vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
@@ -86,13 +87,14 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
sglist = kcalloc(nr_pages, sizeof(*sglist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
if (NULL == pages[0])
goto nopage;
if (PageHighMem(pages[0]))
/* DMA to highmem pages might not work */
goto highmem;
- sglist[0].page = pages[0];
+ sg_set_page(&sglist[0], pages[0]);
sglist[0].offset = offset;
sglist[0].length = PAGE_SIZE - offset;
for (i = 1; i < nr_pages; i++) {
@@ -100,7 +102,7 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
goto nopage;
if (PageHighMem(pages[i]))
goto highmem;
- sglist[i].page = pages[i];
+ sg_set_page(&sglist[i], pages[i]);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/mmc/card/queue.c b/drivers/mmc/card/queue.c
index a5d0354..68c1fe9 100644
--- a/drivers/mmc/card/queue.c
+++ b/drivers/mmc/card/queue.c
@@ -13,6 +13,7 @@
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/scatterlist.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -153,19 +154,21 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card, spinlock_t *lock
blk_queue_max_hw_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
- mq->sg = kzalloc(sizeof(struct scatterlist),
+ mq->sg = kmalloc(sizeof(struct scatterlist),
GFP_KERNEL);
if (!mq->sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->sg, 1);
- mq->bounce_sg = kzalloc(sizeof(struct scatterlist) *
+ mq->bounce_sg = kmalloc(sizeof(struct scatterlist) *
bouncesz / 512, GFP_KERNEL);
if (!mq->bounce_sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->bounce_sg, bouncesz / 512);
}
}
#endif
@@ -302,12 +305,12 @@ static void copy_sg(struct scatterlist *dst, unsigned int dst_len,
BUG_ON(dst_len == 0);
if (dst_size == 0) {
- dst_buf = page_address(dst->page) + dst->offset;
+ dst_buf = page_address(sg_page(dst)) + dst->offset;
dst_size = dst->length;
}
if (src_size == 0) {
- src_buf = page_address(src->page) + src->offset;
+ src_buf = page_address(sg_page(src)) + src->offset;
src_size = src->length;
}
@@ -353,9 +356,7 @@ unsigned int mmc_queue_map_sg(struct mmc_queue *mq)
return 1;
}
- mq->sg[0].page = virt_to_page(mq->bounce_buf);
- mq->sg[0].offset = offset_in_page(mq->bounce_buf);
- mq->sg[0].length = 0;
+ sg_init_one(mq->sg, mq->bounce_buf, 0);
while (sg_len) {
mq->sg[0].length += mq->bounce_sg[sg_len - 1].length;
diff --git a/drivers/mmc/host/mmc_spi.c b/drivers/mmc/host/mmc_spi.c
index 7ae18ea..12c2d80 100644
--- a/drivers/mmc/host/mmc_spi.c
+++ b/drivers/mmc/host/mmc_spi.c
@@ -813,7 +813,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
&& dir == DMA_FROM_DEVICE)
dir = DMA_BIDIRECTIONAL;
- dma_addr = dma_map_page(dma_dev, sg->page, 0,
+ dma_addr = dma_map_page(dma_dev, sg_page(sg), 0,
PAGE_SIZE, dir);
if (direction == DMA_TO_DEVICE)
t->tx_dma = dma_addr + sg->offset;
@@ -822,7 +822,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
}
/* allow pio too; we don't allow highmem */
- kmap_addr = kmap(sg->page);
+ kmap_addr = kmap(sg_page(sg));
if (direction == DMA_TO_DEVICE)
t->tx_buf = kmap_addr + sg->offset;
else
@@ -855,8 +855,8 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
/* discard mappings */
if (direction == DMA_FROM_DEVICE)
- flush_kernel_dcache_page(sg->page);
- kunmap(sg->page);
+ flush_kernel_dcache_page(sg_page(sg));
+ kunmap(sg_page(sg));
if (dma_dev)
dma_unmap_page(dma_dev, dma_addr, PAGE_SIZE, dir);
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index b397121..b6500e9 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -231,7 +231,7 @@ static void sdhci_deactivate_led(struct sdhci_host *host)
static inline char* sdhci_sg_to_buffer(struct sdhci_host* host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline int sdhci_next_sg(struct sdhci_host* host)
diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c
index 9b90479..c11a3d2 100644
--- a/drivers/mmc/host/tifm_sd.c
+++ b/drivers/mmc/host/tifm_sd.c
@@ -192,7 +192,7 @@ static void tifm_sd_transfer_data(struct tifm_sd *host)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
@@ -241,18 +241,18 @@ static void tifm_sd_bounce_block(struct tifm_sd *host, struct mmc_data *r_data)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
p_cnt = min(p_cnt, t_size);
if (r_data->flags & MMC_DATA_WRITE)
- tifm_sd_copy_page(host->bounce_buf.page,
+ tifm_sd_copy_page(sg_page(&host->bounce_buf),
r_data->blksz - t_size,
pg, p_off, p_cnt);
else if (r_data->flags & MMC_DATA_READ)
- tifm_sd_copy_page(pg, p_off, host->bounce_buf.page,
+ tifm_sd_copy_page(pg, p_off, sg_page(&host->bounce_buf),
r_data->blksz - t_size, p_cnt);
t_size -= p_cnt;
diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c
index 80db11c..971ca0f 100644
--- a/drivers/mmc/host/wbsd.c
+++ b/drivers/mmc/host/wbsd.c
@@ -269,7 +269,7 @@ static inline int wbsd_next_sg(struct wbsd_host *host)
static inline char *wbsd_sg_to_buffer(struct wbsd_host *host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
@@ -283,7 +283,7 @@ static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(dmabuf, sgbuf, sg[i].length);
dmabuf += sg[i].length;
}
@@ -300,7 +300,7 @@ static inline void wbsd_dma_to_sg(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(sgbuf, dmabuf, sg[i].length);
dmabuf += sg[i].length;
}
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 4b3c109..887633b 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -60,7 +60,7 @@ static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chu
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -70,7 +70,7 @@ static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *
for (i = 0; i < chunk->npages; ++i)
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
@@ -95,10 +95,13 @@ void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent)
static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -145,6 +148,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MLX4_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -334,7 +338,7 @@ void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_han
* been assigned to.
*/
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
diff --git a/drivers/net/ppp_mppe.c b/drivers/net/ppp_mppe.c
index c0b6d19..bcb0885 100644
--- a/drivers/net/ppp_mppe.c
+++ b/drivers/net/ppp_mppe.c
@@ -55,7 +55,7 @@
#include <linux/mm.h>
#include <linux/ppp_defs.h>
#include <linux/ppp-comp.h>
-#include <asm/scatterlist.h>
+#include <linux/scatterlist.h>
#include "ppp_mppe.h"
@@ -68,9 +68,7 @@ MODULE_VERSION("1.0.2");
static unsigned int
setup_sg(struct scatterlist *sg, const void *address, unsigned int length)
{
- sg[0].page = virt_to_page(address);
- sg[0].offset = offset_in_page(address);
- sg[0].length = length;
+ sg_init_one(sg, address, length);
return length;
}
diff --git a/drivers/scsi/3w-9xxx.c b/drivers/scsi/3w-9xxx.c
index fb14014..afb262b 100644
--- a/drivers/scsi/3w-9xxx.c
+++ b/drivers/scsi/3w-9xxx.c
@@ -1840,7 +1840,7 @@ static int twa_scsiop_execute_scsi(TW_Device_Extension *tw_dev, int request_id,
(scsi_bufflen(srb) < TW_MIN_SGL_LENGTH)) {
if (srb->sc_data_direction == DMA_TO_DEVICE || srb->sc_data_direction == DMA_BIDIRECTIONAL) {
struct scatterlist *sg = scsi_sglist(srb);
- char *buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ char *buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(tw_dev->generic_buffer_virt[request_id], buf, sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
}
@@ -1919,7 +1919,7 @@ static void twa_scsiop_execute_scsi_complete(TW_Device_Extension *tw_dev, int re
char *buf;
unsigned long flags = 0;
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buf, tw_dev->generic_buffer_virt[request_id], sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/3w-xxxx.c b/drivers/scsi/3w-xxxx.c
index a64153b..59716eb 100644
--- a/drivers/scsi/3w-xxxx.c
+++ b/drivers/scsi/3w-xxxx.c
@@ -1469,7 +1469,7 @@ static void tw_transfer_internal(TW_Device_Extension *tw_dev, int request_id,
struct scatterlist *sg = scsi_sglist(cmd);
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len);
memcpy(buf, data, transfer_len);
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 988f0bc..7a76b4b 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -298,7 +298,7 @@ static __inline__ void initialize_SCp(Scsi_Cmnd * cmd)
if (cmd->use_sg) {
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.buffers_residual = cmd->use_sg - 1;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
} else {
@@ -2143,7 +2143,7 @@ static void NCR5380_information_transfer(struct Scsi_Host *instance) {
++cmd->SCp.buffer;
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
}
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 80e448d..a77ab8d 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -356,7 +356,7 @@ static void aac_internal_transfer(struct scsi_cmnd *scsicmd, void *data, unsigne
int transfer_len;
struct scatterlist *sg = scsi_sglist(scsicmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len + offset);
transfer_len -= offset;
diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c
index f817775..f7a2528 100644
--- a/drivers/scsi/arcmsr/arcmsr_hba.c
+++ b/drivers/scsi/arcmsr/arcmsr_hba.c
@@ -1343,7 +1343,7 @@ static int arcmsr_iop_message_xfer(struct AdapterControlBlock *acb, \
/* 4 bytes: Areca io control code */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (scsi_sg_count(cmd) > 1) {
retvalue = ARCMSR_MESSAGE_FAIL;
goto message_out;
@@ -1593,7 +1593,7 @@ static void arcmsr_handle_virtual_command(struct AdapterControlBlock *acb,
strncpy(&inqdata[32], "R001", 4); /* Product Revision */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buffer, inqdata, sizeof(inqdata));
sg = scsi_sglist(cmd);
diff --git a/drivers/scsi/fdomain.c b/drivers/scsi/fdomain.c
index 5d282e6..dc4e463 100644
--- a/drivers/scsi/fdomain.c
+++ b/drivers/scsi/fdomain.c
@@ -1321,7 +1321,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
if (current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
} else
break;
@@ -1354,7 +1354,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
&& current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
}
}
@@ -1439,7 +1439,7 @@ static int fdomain_16x0_queue(struct scsi_cmnd *SCpnt,
if (scsi_sg_count(current_SC)) {
current_SC->SCp.buffer = scsi_sglist(current_SC);
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page)
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer))
+ current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
current_SC->SCp.buffers_residual = scsi_sg_count(current_SC) - 1;
diff --git a/drivers/scsi/gdth.c b/drivers/scsi/gdth.c
index 3ac080e..ac6c57a 100644
--- a/drivers/scsi/gdth.c
+++ b/drivers/scsi/gdth.c
@@ -2374,13 +2374,13 @@ static void gdth_copy_internal_data(gdth_ha_str *ha, Scsi_Cmnd *scp,
if (cpsum+cpnow > cpcount)
cpnow = cpcount - cpsum;
cpsum += cpnow;
- if (!sl->page) {
+ if (!sg_page(sl)) {
printk("GDT-HA %d: invalid sc/gt element in gdth_copy_internal_data()\n",
ha->hanum);
return;
}
local_irq_save(flags);
- address = kmap_atomic(sl->page, KM_BIO_SRC_IRQ) + sl->offset;
+ address = kmap_atomic(sg_page(sl), KM_BIO_SRC_IRQ) + sl->offset;
if (to_buffer)
memcpy(buffer, address, cpnow);
else
diff --git a/drivers/scsi/ide-scsi.c b/drivers/scsi/ide-scsi.c
index fa7ba64..7eb415d 100644
--- a/drivers/scsi/ide-scsi.c
+++ b/drivers/scsi/ide-scsi.c
@@ -179,14 +179,14 @@ static void idescsi_input_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsigne
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
}
@@ -216,14 +216,14 @@ static void idescsi_output_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsign
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
}
diff --git a/drivers/scsi/imm.c b/drivers/scsi/imm.c
index 74cdc1f..da9b0ba 100644
--- a/drivers/scsi/imm.c
+++ b/drivers/scsi/imm.c
@@ -706,7 +706,7 @@ static int imm_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
/*
@@ -845,7 +845,7 @@ static int imm_engine(imm_struct *dev, struct scsi_cmnd *cmd)
(struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..dc98a85 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2872,6 +2872,7 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
}
scatterlist = sglist->scatterlist;
+ sg_init_table(scatterlist, num_elem);
sglist->order = order;
sglist->num_sg = num_elem;
@@ -2884,12 +2885,12 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
/* Free up what we already allocated */
for (j = i - 1; j >= 0; j--)
- __free_pages(scatterlist[j].page, order);
+ __free_pages(sg_page(&scatterlist[j]), order);
kfree(sglist);
return NULL;
}
- scatterlist[i].page = page;
+ sg_set_page(&scatterlist[i], page);
}
return sglist;
@@ -2910,7 +2911,7 @@ static void ipr_free_ucode_buffer(struct ipr_sglist *sglist)
int i;
for (i = 0; i < sglist->num_sg; i++)
- __free_pages(sglist->scatterlist[i].page, sglist->order);
+ __free_pages(sg_page(&sglist->scatterlist[i]), sglist->order);
kfree(sglist);
}
@@ -2940,9 +2941,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
scatterlist = sglist->scatterlist;
for (i = 0; i < (len / bsize_elem); i++, buffer += bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = bsize_elem;
@@ -2953,9 +2956,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
}
if (len % bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, len % bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = len % bsize_elem;
}
diff --git a/drivers/scsi/ips.c b/drivers/scsi/ips.c
index edaac27..5c5a9b2 100644
--- a/drivers/scsi/ips.c
+++ b/drivers/scsi/ips.c
@@ -1515,7 +1515,7 @@ static int ips_is_passthru(struct scsi_cmnd *SC)
/* kmap_atomic() ensures addressability of the user buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (buffer && buffer[0] == 'C' && buffer[1] == 'O' &&
buffer[2] == 'P' && buffer[3] == 'P') {
kunmap_atomic(buffer - sg->offset, KM_IRQ0);
@@ -3523,7 +3523,7 @@ ips_scmd_buf_write(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(buffer, &cdata[xfer_cnt], min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
@@ -3556,7 +3556,7 @@ ips_scmd_buf_read(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(&cdata[xfer_cnt], buffer, min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index a21455d..6ce4109 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -70,9 +70,7 @@ module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO);
static inline void
iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
{
- ibuf->sg.page = virt_to_page(vbuf);
- ibuf->sg.offset = offset_in_page(vbuf);
- ibuf->sg.length = size;
+ sg_init_one(&ibuf->sg, vbuf, size);
ibuf->sent = 0;
ibuf->use_sendmsg = 1;
}
@@ -80,13 +78,14 @@ iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
static inline void
iscsi_buf_init_sg(struct iscsi_buf *ibuf, struct scatterlist *sg)
{
- ibuf->sg.page = sg->page;
+ sg_init_table(&ibuf->sg, 1);
+ sg_set_page(&ibuf->sg, sg_page(sg));
ibuf->sg.offset = sg->offset;
ibuf->sg.length = sg->length;
/*
* Fastpath: sg element fits into single page
*/
- if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg->page))
+ if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg_page(sg)))
ibuf->use_sendmsg = 0;
else
ibuf->use_sendmsg = 1;
@@ -716,7 +715,7 @@ static int iscsi_scsi_data_in(struct iscsi_conn *conn)
for (i = tcp_ctask->sg_count; i < scsi_sg_count(sc); i++) {
char *dest;
- dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0);
+ dest = kmap_atomic(sg_page(&sg[i]), KM_SOFTIRQ0);
rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset,
sg[i].length, offset);
kunmap_atomic(dest, KM_SOFTIRQ0);
@@ -1103,9 +1102,9 @@ iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags)
* slab case.
*/
if (buf->use_sendmsg)
- res = sock_no_sendpage(sk, buf->sg.page, offset, size, flags);
+ res = sock_no_sendpage(sk, sg_page(&buf->sg), offset, size, flags);
else
- res = tcp_conn->sendpage(sk, buf->sg.page, offset, size, flags);
+ res = tcp_conn->sendpage(sk, sg_page(&buf->sg), offset, size, flags);
if (res >= 0) {
conn->txdata_octets += res;
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 10d1aff..0a848e8 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -658,7 +658,7 @@ mega_build_cmd(adapter_t *adapter, Scsi_Cmnd *cmd, int *busy)
struct scatterlist *sg;
sg = scsi_sglist(cmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memset(buf, 0, cmd->cmnd[4]);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
@@ -1542,9 +1542,9 @@ mega_cmd_done(adapter_t *adapter, u8 completed[], int nstatus, int status)
if( cmd->cmnd[0] == INQUIRY && !islogical ) {
sgl = scsi_sglist(cmd);
- if( sgl->page ) {
+ if( sg_page(sgl) ) {
c = *(unsigned char *)
- page_address((&sgl[0])->page) +
+ page_address((sg_page(&sgl[0]))) +
(&sgl[0])->offset;
} else {
printk(KERN_WARNING
diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c
index e4e4c6a..376263c 100644
--- a/drivers/scsi/megaraid/megaraid_mbox.c
+++ b/drivers/scsi/megaraid/megaraid_mbox.c
@@ -1584,9 +1584,9 @@ megaraid_mbox_build_cmd(adapter_t *adapter, struct scsi_cmnd *scp, int *busy)
caddr_t vaddr;
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
vaddr = (caddr_t)
- (page_address((&sgl[0])->page)
+ (page_address(sg_page(&sgl[0]))
+ (&sgl[0])->offset);
memset(vaddr, 0, scp->cmnd[4]);
@@ -2328,9 +2328,9 @@ megaraid_mbox_dpc(unsigned long devp)
&& IS_RAID_CH(raid_dev, scb->dev_channel)) {
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
c = *(unsigned char *)
- (page_address((&sgl[0])->page) +
+ (page_address(sg_page(&sgl[0])) +
(&sgl[0])->offset);
} else {
con_log(CL_ANN, (KERN_WARNING
diff --git a/drivers/scsi/osst.c b/drivers/scsi/osst.c
index 331b789..1c5c4b6 100644
--- a/drivers/scsi/osst.c
+++ b/drivers/scsi/osst.c
@@ -542,7 +542,7 @@ static int osst_verify_frame(struct osst_tape * STp, int frame_seq_number, int q
if (STp->raw) {
if (STp->buffer->syscall_result) {
for (i=0; i < STp->buffer->sg_segs; i++)
- memset(page_address(STp->buffer->sg[i].page),
+ memset(page_address(sg_page(&STp->buffer->sg[i])),
0, STp->buffer->sg[i].length);
strcpy(STp->buffer->b_data, "READ ERROR ON FRAME");
} else
@@ -4437,7 +4437,7 @@ static int os_scsi_tape_open(struct inode * inode, struct file * filp)
for (i = 0, b_size = 0;
(i < STp->buffer->sg_segs) && ((b_size + STp->buffer->sg[i].length) <= OS_DATA_SIZE);
b_size += STp->buffer->sg[i++].length);
- STp->buffer->aux = (os_aux_t *) (page_address(STp->buffer->sg[i].page) + OS_DATA_SIZE - b_size);
+ STp->buffer->aux = (os_aux_t *) (page_address(sg_page(&STp->buffer->sg[i])) + OS_DATA_SIZE - b_size);
#if DEBUG
printk(OSST_DEB_MSG "%s:D: b_data points to %p in segment 0 at %p\n", name,
STp->buffer->b_data, page_address(STp->buffer->sg[0].page));
@@ -5252,25 +5252,26 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
/* Try to allocate the first segment up to OS_DATA_SIZE and the others
big enough to reach the goal (code assumes no segments in place) */
for (b_size = OS_DATA_SIZE, order = OSST_FIRST_ORDER; b_size >= PAGE_SIZE; order--, b_size /= 2) {
- STbuffer->sg[0].page = alloc_pages(priority, order);
+ struct page *page = alloc_pages(priority, order);
+
STbuffer->sg[0].offset = 0;
- if (STbuffer->sg[0].page != NULL) {
+ if (page != NULL) {
+ sg_set_page(&STbuffer->sg[0], page);
STbuffer->sg[0].length = b_size;
- STbuffer->b_data = page_address(STbuffer->sg[0].page);
+ STbuffer->b_data = page_address(page);
break;
}
}
- if (STbuffer->sg[0].page == NULL) {
+ if (sg_page(&STbuffer->sg[0]) == NULL) {
printk(KERN_NOTICE "osst :I: Can't allocate tape buffer main segment.\n");
return 0;
}
/* Got initial segment of 'bsize,order', continue with same size if possible, except for AUX */
for (segs=STbuffer->sg_segs=1, got=b_size;
segs < max_segs && got < OS_FRAME_SIZE; ) {
- STbuffer->sg[segs].page =
- alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
+ struct page *page = alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
STbuffer->sg[segs].offset = 0;
- if (STbuffer->sg[segs].page == NULL) {
+ if (page == NULL) {
if (OS_FRAME_SIZE - got <= (max_segs - segs) * b_size / 2 && order) {
b_size /= 2; /* Large enough for the rest of the buffers */
order--;
@@ -5284,6 +5285,7 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
normalize_buffer(STbuffer);
return 0;
}
+ sg_set_page(&STbuffer->sg[segs], page);
STbuffer->sg[segs].length = (OS_FRAME_SIZE - got <= PAGE_SIZE / 2) ? (OS_FRAME_SIZE - got) : b_size;
got += STbuffer->sg[segs].length;
STbuffer->buffer_size = got;
@@ -5316,7 +5318,7 @@ static void normalize_buffer(struct osst_buffer *STbuffer)
b_size < STbuffer->sg[i].length;
b_size *= 2, order++);
- __free_pages(STbuffer->sg[i].page, order);
+ __free_pages(sg_page(&STbuffer->sg[i]), order);
STbuffer->buffer_size -= STbuffer->sg[i].length;
}
#if DEBUG
@@ -5344,7 +5346,7 @@ static int append_to_buffer(const char __user *ubp, struct osst_buffer *st_bp, i
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->sg[i].page) + offset, ubp, cnt);
+ res = copy_from_user(page_address(sg_page(&st_bp->sg[i])) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5377,7 +5379,7 @@ static int from_buffer(struct osst_buffer *st_bp, char __user *ubp, int do_count
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->sg[i].page) + offset, cnt);
+ res = copy_to_user(ubp, page_address(sg_page(&st_bp->sg[i])) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5410,7 +5412,7 @@ static int osst_zero_buffer_tail(struct osst_buffer *st_bp)
i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count ;
- memset(page_address(st_bp->sg[i].page) + offset, 0, cnt);
+ memset(page_address(sg_page(&st_bp->sg[i])) + offset, 0, cnt);
do_count -= cnt;
offset = 0;
}
@@ -5430,7 +5432,7 @@ static int osst_copy_to_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(page_address(st_bp->sg[i].page), ptr, cnt);
+ memcpy(page_address(sg_page(&st_bp->sg[i])), ptr, cnt);
do_count -= cnt;
ptr += cnt;
}
@@ -5451,7 +5453,7 @@ static int osst_copy_from_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(ptr, page_address(st_bp->sg[i].page), cnt);
+ memcpy(ptr, page_address(sg_page(&st_bp->sg[i])), cnt);
do_count -= cnt;
ptr += cnt;
}
diff --git a/drivers/scsi/pcmcia/sym53c500_cs.c b/drivers/scsi/pcmcia/sym53c500_cs.c
index 190e2a7..26c41b0 100644
--- a/drivers/scsi/pcmcia/sym53c500_cs.c
+++ b/drivers/scsi/pcmcia/sym53c500_cs.c
@@ -443,7 +443,7 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_write(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
+ page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0(port_base);
@@ -463,8 +463,8 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_read(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
- sg->length);
+ page_address(sg_page(sg)) + sg->offset,
+ sg->length);
}
REG0(port_base);
}
diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c
index 67b6d76..07f31e5 100644
--- a/drivers/scsi/ppa.c
+++ b/drivers/scsi/ppa.c
@@ -609,7 +609,7 @@ static int ppa_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
}
}
@@ -756,7 +756,7 @@ static int ppa_engine(ppa_struct *dev, struct scsi_cmnd *cmd)
/* if many buffers are available, start filling the first */
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page) +
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/qlogicfas408.c b/drivers/scsi/qlogicfas408.c
index 2bfbf26..feff3f7 100644
--- a/drivers/scsi/qlogicfas408.c
+++ b/drivers/scsi/qlogicfas408.c
@@ -317,7 +317,7 @@ static unsigned int ql_pcmd(struct scsi_cmnd *cmd)
return ((priv->qabort == 1 ?
DID_ABORT : DID_RESET) << 16);
}
- buf = page_address(sg->page) + sg->offset;
+ buf = page_address(sg_page(sg)) + sg->offset;
if (ql_pdma(priv, phase, buf, sg->length))
break;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..61fdaf0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,7 +764,7 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
- memset(sgl, 0, sizeof(*sgl) * sgp->size);
+ sg_init_table(sgl, sgp->size);
/*
* first loop through, set initial index and return value
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 73c44cb..ce69b9e 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -3797,7 +3797,7 @@ static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
sg = &(STbp->sg[0]);
frp = STbp->frp;
for (i=count=0; count < length; i++) {
- sg[i].page = frp[i].page;
+ sg_set_page(&sg[i], frp[i].page);
if (length - count > frp[i].length)
sg[i].length = frp[i].length;
else
@@ -4446,14 +4446,14 @@ static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pa
}
/* Populate the scatter/gather list */
- sgl[0].page = pages[0];
+ sg_set_page(&sgl[0], pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
+ sg_set_page(&sgl[i], pages[i]);;
sgl[i].offset = 0;
- sgl[i].page = pages[i];
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -4483,7 +4483,7 @@ static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_p
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
diff --git a/drivers/scsi/tmscsim.c b/drivers/scsi/tmscsim.c
index 5c72ca3..4419304 100644
--- a/drivers/scsi/tmscsim.c
+++ b/drivers/scsi/tmscsim.c
@@ -430,10 +430,7 @@ static __inline__ void dc390_Going_remove (struct dc390_dcb* pDCB, struct dc390_
static struct scatterlist* dc390_sg_build_single(struct scatterlist *sg, void *addr, unsigned int length)
{
- memset(sg, 0, sizeof(struct scatterlist));
- sg->page = virt_to_page(addr);
- sg->length = length;
- sg->offset = (unsigned long)addr & ~PAGE_MASK;
+ sg_init_one(sg, addr, length);
return sg;
}
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..3577453 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -438,12 +438,12 @@ int usb_sg_init (
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/image/microtek.c b/drivers/usb/image/microtek.c
index e7d982a..4866baf 100644
--- a/drivers/usb/image/microtek.c
+++ b/drivers/usb/image/microtek.c
@@ -519,7 +519,7 @@ static void mts_do_sg (struct urb* transfer)
context->fragment++;
mts_int_submit_urb(transfer,
context->data_pipe,
- page_address(sg[context->fragment].page) +
+ page_address(sg_page(&sg[context->fragment])) +
sg[context->fragment].offset,
sg[context->fragment].length,
context->fragment + 1 == scsi_sg_count(context->srb) ?
@@ -557,7 +557,7 @@ mts_build_transfer_context(struct scsi_cmnd *srb, struct mts_desc* desc)
return;
} else {
sg = scsi_sglist(srb);
- desc->context.data = page_address(sg[0].page) + sg[0].offset;
+ desc->context.data = page_address(sg_page(&sg[0])) + sg[0].offset;
desc->context.data_length = sg[0].length;
}
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index e901d31..ca8cb34 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -360,9 +360,10 @@ static void free_sglist (struct scatterlist *sg, int nents)
if (!sg)
return;
for (i = 0; i < nents; i++) {
- if (!sg [i].page)
+ struct page *page = sg_page(&sg[i]);
+ if (!page)
continue;
- kfree (page_address (sg [i].page) + sg [i].offset);
+ kfree (page_address (page) + sg [i].offset);
}
kfree (sg);
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index 1ae90ef..0a9882e 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -283,7 +283,7 @@ int virt_to_scatterlist(const void *addr, int size, struct scatterlist *sg,
pg = virt_to_page(addr);
offset = offset_in_page(addr);
if (sg) {
- sg[i].page = pg;
+ sg_set_page(&sg[i], pg);
sg[i].offset = offset;
}
remainder_of_page = PAGE_CACHE_SIZE - offset;
@@ -713,10 +713,13 @@ ecryptfs_encrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return encrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
@@ -742,10 +745,13 @@ ecryptfs_decrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return decrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index 89d9710..263fed8 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -1040,6 +1040,9 @@ decrypt_passphrase_encrypted_session_key(struct ecryptfs_auth_tok *auth_tok,
};
int rc = 0;
+ sg_init_table(&dst_sg, 1);
+ sg_init_table(&src_sg, 1);
+
if (unlikely(ecryptfs_verbosity > 0)) {
ecryptfs_printk(
KERN_DEBUG, "Session key encryption key (size [%d]):\n",
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index ebd03cc..6f03918 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -88,7 +88,7 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
{
struct xdr_netobj cksum;
struct hash_desc desc;
- struct scatterlist sg[1];
+ struct scatterlist sg;
__be32 status = nfserr_resource;
dprintk("NFSD: nfs4_make_rec_clidname for %.*s\n",
@@ -102,11 +102,9 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
if (cksum.data == NULL)
goto out;
- sg[0].page = virt_to_page(clname->data);
- sg[0].offset = offset_in_page(clname->data);
- sg[0].length = clname->len;
+ sg_init_one(&sg, clname->data, clname->len);
- if (crypto_hash_digest(&desc, sg, sg->length, cksum.data))
+ if (crypto_hash_digest(&desc, &sg, sg.length, cksum.data))
goto out;
md5_to_hex(dname, cksum.data);
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..013164b 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,29 +5,37 @@
#include <linux/mm.h>
#include <linux/string.h>
+#define SG_MAGIC 0x87654321
+
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
-static inline void sg_init_one(struct scatterlist *sg, const void *buf,
- unsigned int buflen)
-{
- memset(sg, 0, sizeof(*sg));
- sg_set_buf(sg, buf, buflen);
-}
-
/*
* We overload the LSB of the page pointer to indicate whether it's
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+ ((struct scatterlist *) ((sg)->page_link & ~0x03))
/**
* sg_next - return the next scatterlist entry in a list
@@ -43,10 +51,15 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
-
- if (unlikely(sg_is_chain(sg)))
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ sg = NULL;
+ else if (sg_is_chain(sg))
sg = sg_chain_ptr(sg);
+ else
+ sg++;
return sg;
}
@@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+#endif
return ret;
}
@@ -101,7 +117,45 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
}
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+static inline void sg_init_one(struct scatterlist *sg, const void *buf,
+ unsigned int buflen)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_set_buf(sg, buf, buflen);
+ sg_mark_end(sg, 1);
+}
+
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
+}
+
+
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..183f42c 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,16 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables.
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_tkip.c b/net/ieee80211/ieee80211_crypt_tkip.c
index 6cc54ee..34d4c77 100644
--- a/net/ieee80211/ieee80211_crypt_tkip.c
+++ b/net/ieee80211/ieee80211_crypt_tkip.c
@@ -390,9 +390,7 @@ static int ieee80211_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(tkey->tx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -485,9 +483,7 @@ static int ieee80211_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 12;
crypto_blkcipher_setkey(tkey->rx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4)) {
if (net_ratelimit()) {
printk(KERN_DEBUG ": TKIP: failed to decrypt "
@@ -539,11 +535,12 @@ static int michael_mic(struct crypto_hash *tfm_michael, u8 * key, u8 * hdr,
printk(KERN_WARNING "michael_mic: tfm_michael == NULL\n");
return -1;
}
- sg[0].page = virt_to_page(hdr);
+ sg_init_table(sg, 2);
+ sg_set_page(&sg[0], virt_to_page(hdr));
sg[0].offset = offset_in_page(hdr);
sg[0].length = 16;
- sg[1].page = virt_to_page(data);
+ sg_set_page(&sg[1], virt_to_page(data));
sg[1].offset = offset_in_page(data);
sg[1].length = data_len;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..0af6103 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,9 +170,7 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -212,9 +210,7 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
return -7;
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..cc806d6 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,9 +138,7 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
}
@@ -204,9 +202,7 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
crc = cpu_to_le32(~crc32_le(~0, data, data_len));
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 7818107..cbd64b2 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -726,7 +726,8 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
/* set up scatter list */
end = skb_tail_pointer(skb);
- sg.page = virt_to_page(auth);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(auth));
sg.offset = (unsigned long)(auth) % PAGE_SIZE;
sg.length = end - (unsigned char *)auth;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index f983a36..d5a9785 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1513,7 +1513,8 @@ static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep,
struct hash_desc desc;
/* Sign the message. */
- sg.page = virt_to_page(&cookie->c);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(&cookie->c));
sg.offset = (unsigned long)(&cookie->c) % PAGE_SIZE;
sg.length = bodysize;
keylen = SCTP_SECRET_SIZE;
@@ -1585,7 +1586,8 @@ struct sctp_association *sctp_unpack_cookie(
/* Check the signature. */
keylen = SCTP_SECRET_SIZE;
- sg.page = virt_to_page(bear_cookie);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(bear_cookie));
sg.offset = (unsigned long)(bear_cookie) % PAGE_SIZE;
sg.length = bodysize;
key = (char *)ep->secret_key[ep->current_key];
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
On Thu, Oct 18 2007, David Miller wrote:
> From: Jens Axboe <[email protected]>
> Date: Thu, 18 Oct 2007 12:55:17 +0200
>
> > Things have progressed a lot since, see my recent posting based on
> > Davem's proposal. Will post another patch soonish, that is also
> > tested.
>
> One core issue here is that we need to decide whether this thing to be
> iterated like an array or like a linked list. It's trying to be both.
>
> If we decide upon a looping construct for consumers and stick to it,
> we'll be in much better shape than we are now and bugs will be eaiser
> to spot. It would be so much simpler to audit if all we saw in the
> consumers were things like:
>
> while (sg) {
> do_stuff(sg);
> sg = sg_next(sg);
> }
>
> I would suggest that we just get it over with and convert the whole
> tree now rather than trying to do this kind of thing in stages.
> Because then we can say that ever scatterlist creator has to set
> the "end" bit and therefore you use well established patterns
> for scatterlist iteration such as "traverse sg_next() until NULL"
> as shown above.
The above should work now, PROVIDED that the end has been marked
properly. I have added some helpers to init an sglist (or sg entry).
You can still use
for_each_sg()
and pass in the number of entries, that'll work even with an end marker.
So, it does both.
> I also noticed that there is the issue of on-stack and embedded
> scatterlist users. We'll need some sort of "DECLARE_SCATTERLIST"
> and a "scatterlist_init()" thing so that we can keep DEBUG_SG
> working even in those cases. But for all I know Jens could be
> working on that already :-)
Heh, got some of it covered! The stack usage is generally just converted
to either use sg_init_one() if we can, or sg_init_table(). Adding some
soft of DECLARE_SCATTERLIST() sounds like a good idea, though. I'll take
patches :-)
> The only other real option if we don't convert the whole tree now to
> the "end" marker stuff, is to enforce that every scatterlist iterator
> only traverse the number of entries there were told are in the one
> given to them.
Driver that do all their own sg management typically still use a manual
loop over the number of entries they know are there. I didn't change
those, and the original sg chaining didn't convert those either. Some
manual labor is required in utilizing chained sg lists, unless you are
part of a subsystem (like SCSI) that allocates and inits themf or you.
So I still think that baby steps are a good idea - only convert things
that matter. Leave the rest to janitors or others willing to dive in.
--
Jens Axboe
From: Jens Axboe <[email protected]>
Date: Thu, 18 Oct 2007 14:15:47 +0200
> On Thu, Oct 18 2007, Jens Axboe wrote:
> > On Thu, Oct 18 2007, David Miller wrote:
> > > From: Jens Axboe <[email protected]>
> > > Date: Thu, 18 Oct 2007 13:57:02 +0200
> > >
> > > > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > > > splitting it into a 3-step process. Any arch help is greatly
> > > > appreciated.
> > >
> > > I have some other bits that my compile hit, such as some things in the
> > > crypto layer.
> >
> > Yeah, I have tons of that so far. I hope to have an allyesconfig
> > compiling pretty soonish, will send that out then.
>
> OK here goes, this compiles with allyesconfig on x86-64. Not too bad,
> the scsi/ drivers were by far the worst.
It build cleanly here on sparc64 too.
It's late and I'm about to hit bed so I'm too chicken to do a test
boot it right now :-)
On Thu, Oct 18 2007, David Miller wrote:
> From: Jens Axboe <[email protected]>
> Date: Thu, 18 Oct 2007 14:15:47 +0200
>
> > On Thu, Oct 18 2007, Jens Axboe wrote:
> > > On Thu, Oct 18 2007, David Miller wrote:
> > > > From: Jens Axboe <[email protected]>
> > > > Date: Thu, 18 Oct 2007 13:57:02 +0200
> > > >
> > > > > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > > > > splitting it into a 3-step process. Any arch help is greatly
> > > > > appreciated.
> > > >
> > > > I have some other bits that my compile hit, such as some things in the
> > > > crypto layer.
> > >
> > > Yeah, I have tons of that so far. I hope to have an allyesconfig
> > > compiling pretty soonish, will send that out then.
> >
> > OK here goes, this compiles with allyesconfig on x86-64. Not too bad,
> > the scsi/ drivers were by far the worst.
>
> It build cleanly here on sparc64 too.
Super
> It's late and I'm about to hit bed so I'm too chicken to do a test
> boot it right now :-)
Don't boot it Dave, odds are that something will break and you'll then
be stuck debugging that since you can't relax and sleep until it's
working :-)
--
Jens Axboe
On Oct. 18, 2007, 14:15 +0200, Jens Axboe <[email protected]> wrote:
<snip>
> /**
> * sg_next - return the next scatterlist entry in a list
> @@ -43,10 +51,15 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
> */
> static inline struct scatterlist *sg_next(struct scatterlist *sg)
> {
> - sg++;
> -
> - if (unlikely(sg_is_chain(sg)))
> +#ifdef CONFIG_DEBUG_SG
> + BUG_ON(sg->sg_magic != SG_MAGIC);
> +#endif
> + if (sg_is_last(sg))
> + sg = NULL;
> + else if (sg_is_chain(sg))
> sg = sg_chain_ptr(sg);
> + else
> + sg++;
Hmm, sg_next is not supposed to return a pointer to the chain entry
itself, but rather skip it. I think that the fix needs only
check the "last" flag before incrementing sg.
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ return NULL;
+
sg++
if (unlikely(sg_is_chain(sg)))
sg = sg_chain_ptr(sg);
>
> return sg;
> }
> @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> ret = sg;
>
> #endif
> +#ifdef CONFIG_DEBUG_SG
> + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
can it also do BUG_ON(!sg_is_last(sg))?
> +#endif
> return ret;
> }
>
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, Jens Axboe wrote:
> > On Thu, Oct 18 2007, David Miller wrote:
> > > From: Jens Axboe <[email protected]>
> > > Date: Thu, 18 Oct 2007 13:57:02 +0200
> > >
> > > > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > > > splitting it into a 3-step process. Any arch help is greatly
> > > > appreciated.
> > >
> > > I have some other bits that my compile hit, such as some things in the
> > > crypto layer.
> >
> > Yeah, I have tons of that so far. I hope to have an allyesconfig
> > compiling pretty soonish, will send that out then.
>
> OK here goes, this compiles with allyesconfig on x86-64. Not too bad,
> the scsi/ drivers were by far the worst.
Alright, small modifications and tested somewhat. It boots on my laptop
just fine (CONFIG_DEBUG_SG is set) without warnings. Also tested with
PAGEALLOC set, that also boots and works fine without issue.
So if this is what we want, the question is how to proceed. Right now
i386 and x86-64 works, sparc64 PROBABLY works but is not tested. Any
other arch will NOT compile, since it's not converted to use page_link.
I'll try and do ppc64 now.
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5098f58..c9f0eb0 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -411,8 +411,10 @@ static int calgary_nontranslate_map_sg(struct device* dev,
int i;
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
+ s->dma_address = virt_to_bus(page_address(p) + s->offset);
s->dma_length = s->length;
}
return nelems;
@@ -432,9 +434,11 @@ static int calgary_map_sg(struct device *dev, struct scatterlist *sg,
return calgary_nontranslate_map_sg(dev, sg, nelems, direction);
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(p) + s->offset;
npages = num_dma_pages(vaddr, s->length);
entry = iommu_range_alloc(tbl, npages);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..ce9754f 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,8 +1354,9 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
nsegs++;
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/hmac.c b/crypto/hmac.c
index 8802fb6..e4eb6ac 100644
--- a/crypto/hmac.c
+++ b/crypto/hmac.c
@@ -159,7 +159,8 @@ static int hmac_digest(struct hash_desc *pdesc, struct scatterlist *sg,
desc.flags = pdesc->flags & CRYPTO_TFM_REQ_MAY_SLEEP;
sg_set_buf(sg1, ipad, bs);
- sg1[1].page = (void *)sg;
+
+ sg_set_page(&sg[1], (void *) sg);
sg1[1].length = 0;
sg_set_buf(sg2, opad, bs + ds);
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 18d489c..d741c63 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -317,7 +317,7 @@ static void test_cipher(char *algo, int enc,
goto out;
}
- q = kmap(sg[0].page) + sg[0].offset;
+ q = kmap(sg_page(&sg[0])) + sg[0].offset;
hexdump(q, cipher_tv[i].rlen);
printk("%s\n",
@@ -390,7 +390,7 @@ static void test_cipher(char *algo, int enc,
temp = 0;
for (k = 0; k < cipher_tv[i].np; k++) {
printk("page %u\n", k);
- q = kmap(sg[k].page) + sg[k].offset;
+ q = kmap(sg_page(&sg[k])) + sg[k].offset;
hexdump(q, cipher_tv[i].tap[k]);
printk("%s\n",
memcmp(q, cipher_tv[i].result + temp,
diff --git a/crypto/xcbc.c b/crypto/xcbc.c
index 9f502b8..ac68f3b 100644
--- a/crypto/xcbc.c
+++ b/crypto/xcbc.c
@@ -120,7 +120,7 @@ static int crypto_xcbc_digest_update2(struct hash_desc *pdesc,
do {
- struct page *pg = sg[i].page;
+ struct page *pg = sg_page(&sg[i]);
unsigned int offset = sg[i].offset;
unsigned int slen = sg[i].length;
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
index 7c2cfde..5a6fe17 100644
--- a/drivers/block/cciss.c
+++ b/drivers/block/cciss.c
@@ -2610,7 +2610,7 @@ static void do_cciss_request(struct request_queue *q)
(int)creq->nr_sectors);
#endif /* CCISS_DEBUG */
- memset(tmp_sg, 0, sizeof(tmp_sg));
+ sg_init_table(tmp_sg, MAXSGENTRIES);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* get the DMA records for the setup */
@@ -2621,7 +2621,7 @@ static void do_cciss_request(struct request_queue *q)
for (i = 0; i < seg; i++) {
c->SG[i].Len = tmp_sg[i].length;
- temp64.val = (__u64) pci_map_page(h->pdev, tmp_sg[i].page,
+ temp64.val = (__u64) pci_map_page(h->pdev, sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
c->SG[i].Addr.lower = temp64.val32.lower;
diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c
index 568603d..efab27f 100644
--- a/drivers/block/cpqarray.c
+++ b/drivers/block/cpqarray.c
@@ -918,6 +918,7 @@ queue_next:
DBGPX(
printk("sector=%d, nr_sectors=%d\n", creq->sector, creq->nr_sectors);
);
+ sg_init_table(tmp_sg, SG_MAX);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* Now do all the DMA Mappings */
@@ -929,7 +930,7 @@ DBGPX(
{
c->req.sg[i].size = tmp_sg[i].length;
c->req.sg[i].addr = (__u32) pci_map_page(h->pci_dev,
- tmp_sg[i].page,
+ sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
}
diff --git a/drivers/block/cryptoloop.c b/drivers/block/cryptoloop.c
index 4053503..1b58b01 100644
--- a/drivers/block/cryptoloop.c
+++ b/drivers/block/cryptoloop.c
@@ -26,6 +26,7 @@
#include <linux/crypto.h>
#include <linux/blkdev.h>
#include <linux/loop.h>
+#include <linux/scatterlist.h>
#include <asm/semaphore.h>
#include <asm/uaccess.h>
@@ -119,14 +120,17 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
.tfm = tfm,
.flags = CRYPTO_TFM_REQ_MAY_SLEEP,
};
- struct scatterlist sg_out = { NULL, };
- struct scatterlist sg_in = { NULL, };
+ struct scatterlist sg_out;
+ struct scatterlist sg_in;
encdec_cbc_t encdecfunc;
struct page *in_page, *out_page;
unsigned in_offs, out_offs;
int err;
+ sg_init_table(&sg_out, 1);
+ sg_init_table(&sg_in, 1);
+
if (cmd == READ) {
in_page = raw_page;
in_offs = raw_off;
@@ -146,11 +150,11 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
u32 iv[4] = { 0, };
iv[0] = cpu_to_le32(IV & 0xffffffff);
- sg_in.page = in_page;
+ sg_set_page(&sg_in, in_page);
sg_in.offset = in_offs;
sg_in.length = sz;
- sg_out.page = out_page;
+ sg_set_page(&sg_out, out_page);
sg_out.offset = out_offs;
sg_out.length = sz;
diff --git a/drivers/block/ub.c b/drivers/block/ub.c
index c57dd2b..d6f7b43 100644
--- a/drivers/block/ub.c
+++ b/drivers/block/ub.c
@@ -25,6 +25,7 @@
#include <linux/usb_usual.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
+#include <linux/scatterlist.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
@@ -656,6 +657,7 @@ static int ub_request_fn_1(struct ub_lun *lun, struct request *rq)
if ((cmd = ub_get_cmd(lun)) == NULL)
return -1;
memset(cmd, 0, sizeof(struct ub_scsi_cmd));
+ sg_init_table(cmd->sgv, UB_MAX_REQ_SG);
blkdev_dequeue_request(rq);
@@ -1310,7 +1312,7 @@ static void ub_data_start(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
pipe = sc->send_bulk_pipe;
sc->last_pipe = pipe;
usb_fill_bulk_urb(&sc->work_urb, sc->dev, pipe,
- page_address(sg->page) + sg->offset, sg->length,
+ page_address(sg_page(sg)) + sg->offset, sg->length,
ub_urb_complete, sc);
sc->work_urb.actual_length = 0;
sc->work_urb.error_count = 0;
@@ -1427,7 +1429,7 @@ static void ub_state_sense(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
scmd->state = UB_CMDST_INIT;
scmd->nsg = 1;
sg = &scmd->sgv[0];
- sg->page = virt_to_page(sc->top_sense);
+ sg_set_page(sg, virt_to_page(sc->top_sense));
sg->offset = (unsigned long)sc->top_sense & (PAGE_SIZE-1);
sg->length = UB_SENSE_SIZE;
scmd->len = UB_SENSE_SIZE;
@@ -1863,7 +1865,7 @@ static int ub_sync_read_cap(struct ub_dev *sc, struct ub_lun *lun,
cmd->state = UB_CMDST_INIT;
cmd->nsg = 1;
sg = &cmd->sgv[0];
- sg->page = virt_to_page(p);
+ sg_set_page(sg, virt_to_page(p));
sg->offset = (unsigned long)p & (PAGE_SIZE-1);
sg->length = 8;
cmd->len = 8;
diff --git a/drivers/ide/ide-probe.c b/drivers/ide/ide-probe.c
index 3c945d6..7c640cd 100644
--- a/drivers/ide/ide-probe.c
+++ b/drivers/ide/ide-probe.c
@@ -1337,12 +1337,14 @@ static int hwif_init(ide_hwif_t *hwif)
if (!hwif->sg_max_nents)
hwif->sg_max_nents = PRD_ENTRIES;
- hwif->sg_table = kzalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
+ hwif->sg_table = kmalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
GFP_KERNEL);
if (!hwif->sg_table) {
printk(KERN_ERR "%s: unable to allocate SG table.\n", hwif->name);
goto out;
}
+
+ sg_init_table(hwif->sg_table, hwif->sg_max_nents);
if (init_irq(hwif) == 0)
goto done;
diff --git a/drivers/ide/ide-taskfile.c b/drivers/ide/ide-taskfile.c
index 2a3c8d4..086157f 100644
--- a/drivers/ide/ide-taskfile.c
+++ b/drivers/ide/ide-taskfile.c
@@ -278,7 +278,7 @@ static void ide_pio_sector(ide_drive_t *drive, unsigned int write)
hwif->cursg = sg;
}
- page = cursg->page;
+ page = sg_page(cursg);
offset = cursg->offset + hwif->cursg_ofs * SECTOR_SIZE;
/* get the current page and offset */
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/ieee1394/sbp2.c b/drivers/ieee1394/sbp2.c
index 1b353b9..d5dfe11 100644
--- a/drivers/ieee1394/sbp2.c
+++ b/drivers/ieee1394/sbp2.c
@@ -1466,7 +1466,7 @@ static void sbp2_prep_command_orb_sg(struct sbp2_command_orb *orb,
cmd->dma_size = sgpnt[0].length;
cmd->dma_type = CMD_DMA_PAGE;
cmd->cmd_dma = dma_map_page(hi->host->device.parent,
- sgpnt[0].page, sgpnt[0].offset,
+ sg_page(&sgpnt[0]), sgpnt[0].offset,
cmd->dma_size, cmd->dma_dir);
orb->data_descriptor_lo = cmd->cmd_dma;
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 2f54e29..14159ff 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -55,9 +55,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
ib_dma_unmap_sg(dev, chunk->page_list,
chunk->nents, DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->nents; ++i) {
+ struct page *page = sg_page(&chunk->page_list[i]);
+
if (umem->writable && dirty)
- set_page_dirty_lock(chunk->page_list[i].page);
- put_page(chunk->page_list[i].page);
+ set_page_dirty_lock(page);
+ put_page(page);
}
kfree(chunk);
@@ -164,11 +166,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
}
chunk->nents = min_t(int, ret, IB_UMEM_MAX_PAGE_CHUNK);
+ sg_init_table(chunk->page_list, chunk->nents);
for (i = 0; i < chunk->nents; ++i) {
if (vma_list &&
!is_vm_hugetlb_page(vma_list[i + off]))
umem->hugetlb = 0;
- chunk->page_list[i].page = page_list[i + off];
+ sg_set_page(&chunk->page_list[i], page_list[i + off]);
chunk->page_list[i].offset = 0;
chunk->page_list[i].length = PAGE_SIZE;
}
@@ -179,7 +182,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
DMA_BIDIRECTIONAL);
if (chunk->nmap <= 0) {
for (i = 0; i < chunk->nents; ++i)
- put_page(chunk->page_list[i].page);
+ put_page(sg_page(&chunk->page_list[i]));
kfree(chunk);
ret = -ENOMEM;
diff --git a/drivers/infiniband/hw/ipath/ipath_dma.c b/drivers/infiniband/hw/ipath/ipath_dma.c
index 22709a4..e90a0ea 100644
--- a/drivers/infiniband/hw/ipath/ipath_dma.c
+++ b/drivers/infiniband/hw/ipath/ipath_dma.c
@@ -108,7 +108,7 @@ static int ipath_map_sg(struct ib_device *dev, struct scatterlist *sgl,
BUG_ON(!valid_dma_direction(direction));
for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg->page);
+ addr = (u64) page_address(sg_page(sg));
/* TODO: handle highmem pages */
if (!addr) {
ret = 0;
@@ -127,7 +127,7 @@ static void ipath_unmap_sg(struct ib_device *dev,
static u64 ipath_sg_dma_address(struct ib_device *dev, struct scatterlist *sg)
{
- u64 addr = (u64) page_address(sg->page);
+ u64 addr = (u64) page_address(sg_page(sg));
if (addr)
addr += sg->offset;
diff --git a/drivers/infiniband/hw/ipath/ipath_mr.c b/drivers/infiniband/hw/ipath/ipath_mr.c
index e442470..db4ba92 100644
--- a/drivers/infiniband/hw/ipath/ipath_mr.c
+++ b/drivers/infiniband/hw/ipath/ipath_mr.c
@@ -225,7 +225,7 @@ struct ib_mr *ipath_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
for (i = 0; i < chunk->nents; i++) {
void *vaddr;
- vaddr = page_address(chunk->page_list[i].page);
+ vaddr = page_address(sg_page(&chunk->page_list[i]));
if (!vaddr) {
ret = ERR_PTR(-EINVAL);
goto bail;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index e61f3e6..007b381 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -71,7 +71,7 @@ static void mthca_free_icm_pages(struct mthca_dev *dev, struct mthca_icm_chunk *
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -81,7 +81,7 @@ static void mthca_free_icm_coherent(struct mthca_dev *dev, struct mthca_icm_chun
for (i = 0; i < chunk->npages; ++i) {
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
}
@@ -107,10 +107,13 @@ void mthca_free_icm(struct mthca_dev *dev, struct mthca_icm *icm, int coherent)
static int mthca_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -157,6 +160,7 @@ struct mthca_icm *mthca_alloc_icm(struct mthca_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MTHCA_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -304,7 +308,7 @@ void *mthca_table_find(struct mthca_icm_table *table, int obj, dma_addr_t *dma_h
* so if we found the page, dma_handle has already
* been assigned to. */
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
@@ -445,6 +449,7 @@ static u64 mthca_uarc_virt(struct mthca_dev *dev, struct mthca_uar *uar, int pag
int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
struct mthca_user_db_table *db_tab, int index, u64 uaddr)
{
+ struct page *pages[1];
int ret = 0;
u8 status;
int i;
@@ -472,16 +477,17 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
}
ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- &db_tab->page[i].mem.page, NULL);
+ pages, NULL);
if (ret < 0)
goto out;
+ sg_set_page(&db_tab->page[i].mem, pages[0]);
db_tab->page[i].mem.length = MTHCA_ICM_PAGE_SIZE;
db_tab->page[i].mem.offset = uaddr & ~PAGE_MASK;
ret = pci_map_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
if (ret < 0) {
- put_page(db_tab->page[i].mem.page);
+ put_page(pages[0]);
goto out;
}
@@ -491,7 +497,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
ret = -EINVAL;
if (ret) {
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
goto out;
}
@@ -557,7 +563,7 @@ void mthca_cleanup_user_db_tab(struct mthca_dev *dev, struct mthca_uar *uar,
if (db_tab->page[i].uvirt) {
mthca_UNMAP_ICM(dev, mthca_uarc_virt(dev, uar, i), 1, &status);
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
}
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index f3529b6..29c842e 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -131,7 +131,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, data->size, i) {
- from = kmap_atomic(sg->page, KM_USER0);
+ from = kmap_atomic(sg_page(sg), KM_USER0);
memcpy(p,
from + sg->offset,
sg->length);
@@ -191,7 +191,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, sg_size, i) {
- to = kmap_atomic(sg->page, KM_SOFTIRQ0);
+ to = kmap_atomic(sg_page(sg), KM_SOFTIRQ0);
memcpy(to + sg->offset,
p,
sg->length);
@@ -336,7 +336,7 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
- sg->page, sg->offset,
+ sg_page(sg), sg->offset,
sg->length, ib_sg_dma_len(ibdev, sg));
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 64fee90..cde915f 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -346,16 +346,17 @@ static int crypt_convert(struct crypt_config *cc,
ctx->idx_out < ctx->bio_out->bi_vcnt) {
struct bio_vec *bv_in = bio_iovec_idx(ctx->bio_in, ctx->idx_in);
struct bio_vec *bv_out = bio_iovec_idx(ctx->bio_out, ctx->idx_out);
- struct scatterlist sg_in = {
- .page = bv_in->bv_page,
- .offset = bv_in->bv_offset + ctx->offset_in,
- .length = 1 << SECTOR_SHIFT
- };
- struct scatterlist sg_out = {
- .page = bv_out->bv_page,
- .offset = bv_out->bv_offset + ctx->offset_out,
- .length = 1 << SECTOR_SHIFT
- };
+ struct scatterlist sg_in, sg_out;
+
+ sg_init_table(&sg_in, 1);
+ sg_set_page(&sg_in, bv_in->bv_page);
+ sg_in.offset = bv_in->bv_offset + ctx->offset_in;
+ sg_in.length = 1 << SECTOR_SHIFT;
+
+ sg_init_table(&sg_out, 1);
+ sg_set_page(&sg_out, bv_out->bv_page);
+ sg_out.offset = bv_out->bv_offset + ctx->offset_out;
+ sg_out.length = 1 << SECTOR_SHIFT;
ctx->offset_in += sg_in.length;
if (ctx->offset_in >= bv_in->bv_len) {
diff --git a/drivers/media/common/saa7146_core.c b/drivers/media/common/saa7146_core.c
index 365a221..2b1f8b4 100644
--- a/drivers/media/common/saa7146_core.c
+++ b/drivers/media/common/saa7146_core.c
@@ -112,12 +112,13 @@ static struct scatterlist* vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/media/video/ivtv/ivtv-udma.c b/drivers/media/video/ivtv/ivtv-udma.c
index c4626d1..912b424 100644
--- a/drivers/media/video/ivtv/ivtv-udma.c
+++ b/drivers/media/video/ivtv/ivtv-udma.c
@@ -63,10 +63,10 @@ int ivtv_udma_fill_sg_list (struct ivtv_user_dma *dma, struct ivtv_dma_page_info
memcpy(page_address(dma->bouncemap[map_offset]) + offset, src, len);
kunmap_atomic(src, KM_BOUNCE_READ);
local_irq_restore(flags);
- dma->SGlist[map_offset].page = dma->bouncemap[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->bouncemap[map_offset]);
}
else {
- dma->SGlist[map_offset].page = dma->map[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->map[map_offset]);
}
offset = 0;
map_offset++;
diff --git a/drivers/media/video/videobuf-dma-sg.c b/drivers/media/video/videobuf-dma-sg.c
index 3eb6123..0a18286 100644
--- a/drivers/media/video/videobuf-dma-sg.c
+++ b/drivers/media/video/videobuf-dma-sg.c
@@ -60,12 +60,13 @@ videobuf_vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
@@ -86,13 +87,14 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
sglist = kcalloc(nr_pages, sizeof(*sglist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
if (NULL == pages[0])
goto nopage;
if (PageHighMem(pages[0]))
/* DMA to highmem pages might not work */
goto highmem;
- sglist[0].page = pages[0];
+ sg_set_page(&sglist[0], pages[0]);
sglist[0].offset = offset;
sglist[0].length = PAGE_SIZE - offset;
for (i = 1; i < nr_pages; i++) {
@@ -100,7 +102,7 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
goto nopage;
if (PageHighMem(pages[i]))
goto highmem;
- sglist[i].page = pages[i];
+ sg_set_page(&sglist[i], pages[i]);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/mmc/card/queue.c b/drivers/mmc/card/queue.c
index a5d0354..68c1fe9 100644
--- a/drivers/mmc/card/queue.c
+++ b/drivers/mmc/card/queue.c
@@ -13,6 +13,7 @@
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/scatterlist.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -153,19 +154,21 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card, spinlock_t *lock
blk_queue_max_hw_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
- mq->sg = kzalloc(sizeof(struct scatterlist),
+ mq->sg = kmalloc(sizeof(struct scatterlist),
GFP_KERNEL);
if (!mq->sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->sg, 1);
- mq->bounce_sg = kzalloc(sizeof(struct scatterlist) *
+ mq->bounce_sg = kmalloc(sizeof(struct scatterlist) *
bouncesz / 512, GFP_KERNEL);
if (!mq->bounce_sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->bounce_sg, bouncesz / 512);
}
}
#endif
@@ -302,12 +305,12 @@ static void copy_sg(struct scatterlist *dst, unsigned int dst_len,
BUG_ON(dst_len == 0);
if (dst_size == 0) {
- dst_buf = page_address(dst->page) + dst->offset;
+ dst_buf = page_address(sg_page(dst)) + dst->offset;
dst_size = dst->length;
}
if (src_size == 0) {
- src_buf = page_address(src->page) + src->offset;
+ src_buf = page_address(sg_page(src)) + src->offset;
src_size = src->length;
}
@@ -353,9 +356,7 @@ unsigned int mmc_queue_map_sg(struct mmc_queue *mq)
return 1;
}
- mq->sg[0].page = virt_to_page(mq->bounce_buf);
- mq->sg[0].offset = offset_in_page(mq->bounce_buf);
- mq->sg[0].length = 0;
+ sg_init_one(mq->sg, mq->bounce_buf, 0);
while (sg_len) {
mq->sg[0].length += mq->bounce_sg[sg_len - 1].length;
diff --git a/drivers/mmc/host/mmc_spi.c b/drivers/mmc/host/mmc_spi.c
index 7ae18ea..12c2d80 100644
--- a/drivers/mmc/host/mmc_spi.c
+++ b/drivers/mmc/host/mmc_spi.c
@@ -813,7 +813,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
&& dir == DMA_FROM_DEVICE)
dir = DMA_BIDIRECTIONAL;
- dma_addr = dma_map_page(dma_dev, sg->page, 0,
+ dma_addr = dma_map_page(dma_dev, sg_page(sg), 0,
PAGE_SIZE, dir);
if (direction == DMA_TO_DEVICE)
t->tx_dma = dma_addr + sg->offset;
@@ -822,7 +822,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
}
/* allow pio too; we don't allow highmem */
- kmap_addr = kmap(sg->page);
+ kmap_addr = kmap(sg_page(sg));
if (direction == DMA_TO_DEVICE)
t->tx_buf = kmap_addr + sg->offset;
else
@@ -855,8 +855,8 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
/* discard mappings */
if (direction == DMA_FROM_DEVICE)
- flush_kernel_dcache_page(sg->page);
- kunmap(sg->page);
+ flush_kernel_dcache_page(sg_page(sg));
+ kunmap(sg_page(sg));
if (dma_dev)
dma_unmap_page(dma_dev, dma_addr, PAGE_SIZE, dir);
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index b397121..b6500e9 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -231,7 +231,7 @@ static void sdhci_deactivate_led(struct sdhci_host *host)
static inline char* sdhci_sg_to_buffer(struct sdhci_host* host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline int sdhci_next_sg(struct sdhci_host* host)
diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c
index 9b90479..c11a3d2 100644
--- a/drivers/mmc/host/tifm_sd.c
+++ b/drivers/mmc/host/tifm_sd.c
@@ -192,7 +192,7 @@ static void tifm_sd_transfer_data(struct tifm_sd *host)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
@@ -241,18 +241,18 @@ static void tifm_sd_bounce_block(struct tifm_sd *host, struct mmc_data *r_data)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
p_cnt = min(p_cnt, t_size);
if (r_data->flags & MMC_DATA_WRITE)
- tifm_sd_copy_page(host->bounce_buf.page,
+ tifm_sd_copy_page(sg_page(&host->bounce_buf),
r_data->blksz - t_size,
pg, p_off, p_cnt);
else if (r_data->flags & MMC_DATA_READ)
- tifm_sd_copy_page(pg, p_off, host->bounce_buf.page,
+ tifm_sd_copy_page(pg, p_off, sg_page(&host->bounce_buf),
r_data->blksz - t_size, p_cnt);
t_size -= p_cnt;
diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c
index 80db11c..971ca0f 100644
--- a/drivers/mmc/host/wbsd.c
+++ b/drivers/mmc/host/wbsd.c
@@ -269,7 +269,7 @@ static inline int wbsd_next_sg(struct wbsd_host *host)
static inline char *wbsd_sg_to_buffer(struct wbsd_host *host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
@@ -283,7 +283,7 @@ static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(dmabuf, sgbuf, sg[i].length);
dmabuf += sg[i].length;
}
@@ -300,7 +300,7 @@ static inline void wbsd_dma_to_sg(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(sgbuf, dmabuf, sg[i].length);
dmabuf += sg[i].length;
}
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 4b3c109..887633b 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -60,7 +60,7 @@ static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chu
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -70,7 +70,7 @@ static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *
for (i = 0; i < chunk->npages; ++i)
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
@@ -95,10 +95,13 @@ void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent)
static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -145,6 +148,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MLX4_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -334,7 +338,7 @@ void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_han
* been assigned to.
*/
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
diff --git a/drivers/net/ppp_mppe.c b/drivers/net/ppp_mppe.c
index c0b6d19..bcb0885 100644
--- a/drivers/net/ppp_mppe.c
+++ b/drivers/net/ppp_mppe.c
@@ -55,7 +55,7 @@
#include <linux/mm.h>
#include <linux/ppp_defs.h>
#include <linux/ppp-comp.h>
-#include <asm/scatterlist.h>
+#include <linux/scatterlist.h>
#include "ppp_mppe.h"
@@ -68,9 +68,7 @@ MODULE_VERSION("1.0.2");
static unsigned int
setup_sg(struct scatterlist *sg, const void *address, unsigned int length)
{
- sg[0].page = virt_to_page(address);
- sg[0].offset = offset_in_page(address);
- sg[0].length = length;
+ sg_init_one(sg, address, length);
return length;
}
diff --git a/drivers/scsi/3w-9xxx.c b/drivers/scsi/3w-9xxx.c
index fb14014..afb262b 100644
--- a/drivers/scsi/3w-9xxx.c
+++ b/drivers/scsi/3w-9xxx.c
@@ -1840,7 +1840,7 @@ static int twa_scsiop_execute_scsi(TW_Device_Extension *tw_dev, int request_id,
(scsi_bufflen(srb) < TW_MIN_SGL_LENGTH)) {
if (srb->sc_data_direction == DMA_TO_DEVICE || srb->sc_data_direction == DMA_BIDIRECTIONAL) {
struct scatterlist *sg = scsi_sglist(srb);
- char *buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ char *buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(tw_dev->generic_buffer_virt[request_id], buf, sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
}
@@ -1919,7 +1919,7 @@ static void twa_scsiop_execute_scsi_complete(TW_Device_Extension *tw_dev, int re
char *buf;
unsigned long flags = 0;
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buf, tw_dev->generic_buffer_virt[request_id], sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/3w-xxxx.c b/drivers/scsi/3w-xxxx.c
index a64153b..59716eb 100644
--- a/drivers/scsi/3w-xxxx.c
+++ b/drivers/scsi/3w-xxxx.c
@@ -1469,7 +1469,7 @@ static void tw_transfer_internal(TW_Device_Extension *tw_dev, int request_id,
struct scatterlist *sg = scsi_sglist(cmd);
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len);
memcpy(buf, data, transfer_len);
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 988f0bc..7a76b4b 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -298,7 +298,7 @@ static __inline__ void initialize_SCp(Scsi_Cmnd * cmd)
if (cmd->use_sg) {
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.buffers_residual = cmd->use_sg - 1;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
} else {
@@ -2143,7 +2143,7 @@ static void NCR5380_information_transfer(struct Scsi_Host *instance) {
++cmd->SCp.buffer;
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
}
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 80e448d..a77ab8d 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -356,7 +356,7 @@ static void aac_internal_transfer(struct scsi_cmnd *scsicmd, void *data, unsigne
int transfer_len;
struct scatterlist *sg = scsi_sglist(scsicmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len + offset);
transfer_len -= offset;
diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c
index f817775..f7a2528 100644
--- a/drivers/scsi/arcmsr/arcmsr_hba.c
+++ b/drivers/scsi/arcmsr/arcmsr_hba.c
@@ -1343,7 +1343,7 @@ static int arcmsr_iop_message_xfer(struct AdapterControlBlock *acb, \
/* 4 bytes: Areca io control code */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (scsi_sg_count(cmd) > 1) {
retvalue = ARCMSR_MESSAGE_FAIL;
goto message_out;
@@ -1593,7 +1593,7 @@ static void arcmsr_handle_virtual_command(struct AdapterControlBlock *acb,
strncpy(&inqdata[32], "R001", 4); /* Product Revision */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buffer, inqdata, sizeof(inqdata));
sg = scsi_sglist(cmd);
diff --git a/drivers/scsi/fdomain.c b/drivers/scsi/fdomain.c
index 5d282e6..dc4e463 100644
--- a/drivers/scsi/fdomain.c
+++ b/drivers/scsi/fdomain.c
@@ -1321,7 +1321,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
if (current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
} else
break;
@@ -1354,7 +1354,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
&& current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
}
}
@@ -1439,7 +1439,7 @@ static int fdomain_16x0_queue(struct scsi_cmnd *SCpnt,
if (scsi_sg_count(current_SC)) {
current_SC->SCp.buffer = scsi_sglist(current_SC);
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page)
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer))
+ current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
current_SC->SCp.buffers_residual = scsi_sg_count(current_SC) - 1;
diff --git a/drivers/scsi/gdth.c b/drivers/scsi/gdth.c
index 3ac080e..ac6c57a 100644
--- a/drivers/scsi/gdth.c
+++ b/drivers/scsi/gdth.c
@@ -2374,13 +2374,13 @@ static void gdth_copy_internal_data(gdth_ha_str *ha, Scsi_Cmnd *scp,
if (cpsum+cpnow > cpcount)
cpnow = cpcount - cpsum;
cpsum += cpnow;
- if (!sl->page) {
+ if (!sg_page(sl)) {
printk("GDT-HA %d: invalid sc/gt element in gdth_copy_internal_data()\n",
ha->hanum);
return;
}
local_irq_save(flags);
- address = kmap_atomic(sl->page, KM_BIO_SRC_IRQ) + sl->offset;
+ address = kmap_atomic(sg_page(sl), KM_BIO_SRC_IRQ) + sl->offset;
if (to_buffer)
memcpy(buffer, address, cpnow);
else
diff --git a/drivers/scsi/ide-scsi.c b/drivers/scsi/ide-scsi.c
index fa7ba64..7eb415d 100644
--- a/drivers/scsi/ide-scsi.c
+++ b/drivers/scsi/ide-scsi.c
@@ -179,14 +179,14 @@ static void idescsi_input_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsigne
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
}
@@ -216,14 +216,14 @@ static void idescsi_output_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsign
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
}
diff --git a/drivers/scsi/imm.c b/drivers/scsi/imm.c
index 74cdc1f..da9b0ba 100644
--- a/drivers/scsi/imm.c
+++ b/drivers/scsi/imm.c
@@ -706,7 +706,7 @@ static int imm_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
/*
@@ -845,7 +845,7 @@ static int imm_engine(imm_struct *dev, struct scsi_cmnd *cmd)
(struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..dc98a85 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2872,6 +2872,7 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
}
scatterlist = sglist->scatterlist;
+ sg_init_table(scatterlist, num_elem);
sglist->order = order;
sglist->num_sg = num_elem;
@@ -2884,12 +2885,12 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
/* Free up what we already allocated */
for (j = i - 1; j >= 0; j--)
- __free_pages(scatterlist[j].page, order);
+ __free_pages(sg_page(&scatterlist[j]), order);
kfree(sglist);
return NULL;
}
- scatterlist[i].page = page;
+ sg_set_page(&scatterlist[i], page);
}
return sglist;
@@ -2910,7 +2911,7 @@ static void ipr_free_ucode_buffer(struct ipr_sglist *sglist)
int i;
for (i = 0; i < sglist->num_sg; i++)
- __free_pages(sglist->scatterlist[i].page, sglist->order);
+ __free_pages(sg_page(&sglist->scatterlist[i]), sglist->order);
kfree(sglist);
}
@@ -2940,9 +2941,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
scatterlist = sglist->scatterlist;
for (i = 0; i < (len / bsize_elem); i++, buffer += bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = bsize_elem;
@@ -2953,9 +2956,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
}
if (len % bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, len % bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = len % bsize_elem;
}
diff --git a/drivers/scsi/ips.c b/drivers/scsi/ips.c
index edaac27..5c5a9b2 100644
--- a/drivers/scsi/ips.c
+++ b/drivers/scsi/ips.c
@@ -1515,7 +1515,7 @@ static int ips_is_passthru(struct scsi_cmnd *SC)
/* kmap_atomic() ensures addressability of the user buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (buffer && buffer[0] == 'C' && buffer[1] == 'O' &&
buffer[2] == 'P' && buffer[3] == 'P') {
kunmap_atomic(buffer - sg->offset, KM_IRQ0);
@@ -3523,7 +3523,7 @@ ips_scmd_buf_write(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(buffer, &cdata[xfer_cnt], min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
@@ -3556,7 +3556,7 @@ ips_scmd_buf_read(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(&cdata[xfer_cnt], buffer, min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index a21455d..6ce4109 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -70,9 +70,7 @@ module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO);
static inline void
iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
{
- ibuf->sg.page = virt_to_page(vbuf);
- ibuf->sg.offset = offset_in_page(vbuf);
- ibuf->sg.length = size;
+ sg_init_one(&ibuf->sg, vbuf, size);
ibuf->sent = 0;
ibuf->use_sendmsg = 1;
}
@@ -80,13 +78,14 @@ iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
static inline void
iscsi_buf_init_sg(struct iscsi_buf *ibuf, struct scatterlist *sg)
{
- ibuf->sg.page = sg->page;
+ sg_init_table(&ibuf->sg, 1);
+ sg_set_page(&ibuf->sg, sg_page(sg));
ibuf->sg.offset = sg->offset;
ibuf->sg.length = sg->length;
/*
* Fastpath: sg element fits into single page
*/
- if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg->page))
+ if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg_page(sg)))
ibuf->use_sendmsg = 0;
else
ibuf->use_sendmsg = 1;
@@ -716,7 +715,7 @@ static int iscsi_scsi_data_in(struct iscsi_conn *conn)
for (i = tcp_ctask->sg_count; i < scsi_sg_count(sc); i++) {
char *dest;
- dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0);
+ dest = kmap_atomic(sg_page(&sg[i]), KM_SOFTIRQ0);
rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset,
sg[i].length, offset);
kunmap_atomic(dest, KM_SOFTIRQ0);
@@ -1103,9 +1102,9 @@ iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags)
* slab case.
*/
if (buf->use_sendmsg)
- res = sock_no_sendpage(sk, buf->sg.page, offset, size, flags);
+ res = sock_no_sendpage(sk, sg_page(&buf->sg), offset, size, flags);
else
- res = tcp_conn->sendpage(sk, buf->sg.page, offset, size, flags);
+ res = tcp_conn->sendpage(sk, sg_page(&buf->sg), offset, size, flags);
if (res >= 0) {
conn->txdata_octets += res;
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 10d1aff..0a848e8 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -658,7 +658,7 @@ mega_build_cmd(adapter_t *adapter, Scsi_Cmnd *cmd, int *busy)
struct scatterlist *sg;
sg = scsi_sglist(cmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memset(buf, 0, cmd->cmnd[4]);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
@@ -1542,9 +1542,9 @@ mega_cmd_done(adapter_t *adapter, u8 completed[], int nstatus, int status)
if( cmd->cmnd[0] == INQUIRY && !islogical ) {
sgl = scsi_sglist(cmd);
- if( sgl->page ) {
+ if( sg_page(sgl) ) {
c = *(unsigned char *)
- page_address((&sgl[0])->page) +
+ page_address((sg_page(&sgl[0]))) +
(&sgl[0])->offset;
} else {
printk(KERN_WARNING
diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c
index e4e4c6a..376263c 100644
--- a/drivers/scsi/megaraid/megaraid_mbox.c
+++ b/drivers/scsi/megaraid/megaraid_mbox.c
@@ -1584,9 +1584,9 @@ megaraid_mbox_build_cmd(adapter_t *adapter, struct scsi_cmnd *scp, int *busy)
caddr_t vaddr;
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
vaddr = (caddr_t)
- (page_address((&sgl[0])->page)
+ (page_address(sg_page(&sgl[0]))
+ (&sgl[0])->offset);
memset(vaddr, 0, scp->cmnd[4]);
@@ -2328,9 +2328,9 @@ megaraid_mbox_dpc(unsigned long devp)
&& IS_RAID_CH(raid_dev, scb->dev_channel)) {
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
c = *(unsigned char *)
- (page_address((&sgl[0])->page) +
+ (page_address(sg_page(&sgl[0])) +
(&sgl[0])->offset);
} else {
con_log(CL_ANN, (KERN_WARNING
diff --git a/drivers/scsi/osst.c b/drivers/scsi/osst.c
index 331b789..1c5c4b6 100644
--- a/drivers/scsi/osst.c
+++ b/drivers/scsi/osst.c
@@ -542,7 +542,7 @@ static int osst_verify_frame(struct osst_tape * STp, int frame_seq_number, int q
if (STp->raw) {
if (STp->buffer->syscall_result) {
for (i=0; i < STp->buffer->sg_segs; i++)
- memset(page_address(STp->buffer->sg[i].page),
+ memset(page_address(sg_page(&STp->buffer->sg[i])),
0, STp->buffer->sg[i].length);
strcpy(STp->buffer->b_data, "READ ERROR ON FRAME");
} else
@@ -4437,7 +4437,7 @@ static int os_scsi_tape_open(struct inode * inode, struct file * filp)
for (i = 0, b_size = 0;
(i < STp->buffer->sg_segs) && ((b_size + STp->buffer->sg[i].length) <= OS_DATA_SIZE);
b_size += STp->buffer->sg[i++].length);
- STp->buffer->aux = (os_aux_t *) (page_address(STp->buffer->sg[i].page) + OS_DATA_SIZE - b_size);
+ STp->buffer->aux = (os_aux_t *) (page_address(sg_page(&STp->buffer->sg[i])) + OS_DATA_SIZE - b_size);
#if DEBUG
printk(OSST_DEB_MSG "%s:D: b_data points to %p in segment 0 at %p\n", name,
STp->buffer->b_data, page_address(STp->buffer->sg[0].page));
@@ -5252,25 +5252,26 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
/* Try to allocate the first segment up to OS_DATA_SIZE and the others
big enough to reach the goal (code assumes no segments in place) */
for (b_size = OS_DATA_SIZE, order = OSST_FIRST_ORDER; b_size >= PAGE_SIZE; order--, b_size /= 2) {
- STbuffer->sg[0].page = alloc_pages(priority, order);
+ struct page *page = alloc_pages(priority, order);
+
STbuffer->sg[0].offset = 0;
- if (STbuffer->sg[0].page != NULL) {
+ if (page != NULL) {
+ sg_set_page(&STbuffer->sg[0], page);
STbuffer->sg[0].length = b_size;
- STbuffer->b_data = page_address(STbuffer->sg[0].page);
+ STbuffer->b_data = page_address(page);
break;
}
}
- if (STbuffer->sg[0].page == NULL) {
+ if (sg_page(&STbuffer->sg[0]) == NULL) {
printk(KERN_NOTICE "osst :I: Can't allocate tape buffer main segment.\n");
return 0;
}
/* Got initial segment of 'bsize,order', continue with same size if possible, except for AUX */
for (segs=STbuffer->sg_segs=1, got=b_size;
segs < max_segs && got < OS_FRAME_SIZE; ) {
- STbuffer->sg[segs].page =
- alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
+ struct page *page = alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
STbuffer->sg[segs].offset = 0;
- if (STbuffer->sg[segs].page == NULL) {
+ if (page == NULL) {
if (OS_FRAME_SIZE - got <= (max_segs - segs) * b_size / 2 && order) {
b_size /= 2; /* Large enough for the rest of the buffers */
order--;
@@ -5284,6 +5285,7 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
normalize_buffer(STbuffer);
return 0;
}
+ sg_set_page(&STbuffer->sg[segs], page);
STbuffer->sg[segs].length = (OS_FRAME_SIZE - got <= PAGE_SIZE / 2) ? (OS_FRAME_SIZE - got) : b_size;
got += STbuffer->sg[segs].length;
STbuffer->buffer_size = got;
@@ -5316,7 +5318,7 @@ static void normalize_buffer(struct osst_buffer *STbuffer)
b_size < STbuffer->sg[i].length;
b_size *= 2, order++);
- __free_pages(STbuffer->sg[i].page, order);
+ __free_pages(sg_page(&STbuffer->sg[i]), order);
STbuffer->buffer_size -= STbuffer->sg[i].length;
}
#if DEBUG
@@ -5344,7 +5346,7 @@ static int append_to_buffer(const char __user *ubp, struct osst_buffer *st_bp, i
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->sg[i].page) + offset, ubp, cnt);
+ res = copy_from_user(page_address(sg_page(&st_bp->sg[i])) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5377,7 +5379,7 @@ static int from_buffer(struct osst_buffer *st_bp, char __user *ubp, int do_count
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->sg[i].page) + offset, cnt);
+ res = copy_to_user(ubp, page_address(sg_page(&st_bp->sg[i])) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5410,7 +5412,7 @@ static int osst_zero_buffer_tail(struct osst_buffer *st_bp)
i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count ;
- memset(page_address(st_bp->sg[i].page) + offset, 0, cnt);
+ memset(page_address(sg_page(&st_bp->sg[i])) + offset, 0, cnt);
do_count -= cnt;
offset = 0;
}
@@ -5430,7 +5432,7 @@ static int osst_copy_to_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(page_address(st_bp->sg[i].page), ptr, cnt);
+ memcpy(page_address(sg_page(&st_bp->sg[i])), ptr, cnt);
do_count -= cnt;
ptr += cnt;
}
@@ -5451,7 +5453,7 @@ static int osst_copy_from_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(ptr, page_address(st_bp->sg[i].page), cnt);
+ memcpy(ptr, page_address(sg_page(&st_bp->sg[i])), cnt);
do_count -= cnt;
ptr += cnt;
}
diff --git a/drivers/scsi/pcmcia/sym53c500_cs.c b/drivers/scsi/pcmcia/sym53c500_cs.c
index 190e2a7..26c41b0 100644
--- a/drivers/scsi/pcmcia/sym53c500_cs.c
+++ b/drivers/scsi/pcmcia/sym53c500_cs.c
@@ -443,7 +443,7 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_write(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
+ page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0(port_base);
@@ -463,8 +463,8 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_read(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
- sg->length);
+ page_address(sg_page(sg)) + sg->offset,
+ sg->length);
}
REG0(port_base);
}
diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c
index 67b6d76..07f31e5 100644
--- a/drivers/scsi/ppa.c
+++ b/drivers/scsi/ppa.c
@@ -609,7 +609,7 @@ static int ppa_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
}
}
@@ -756,7 +756,7 @@ static int ppa_engine(ppa_struct *dev, struct scsi_cmnd *cmd)
/* if many buffers are available, start filling the first */
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page) +
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/qlogicfas408.c b/drivers/scsi/qlogicfas408.c
index 2bfbf26..feff3f7 100644
--- a/drivers/scsi/qlogicfas408.c
+++ b/drivers/scsi/qlogicfas408.c
@@ -317,7 +317,7 @@ static unsigned int ql_pcmd(struct scsi_cmnd *cmd)
return ((priv->qabort == 1 ?
DID_ABORT : DID_RESET) << 16);
}
- buf = page_address(sg->page) + sg->offset;
+ buf = page_address(sg_page(sg)) + sg->offset;
if (ql_pdma(priv, phase, buf, sg->length))
break;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..61fdaf0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,7 +764,7 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
- memset(sgl, 0, sizeof(*sgl) * sgp->size);
+ sg_init_table(sgl, sgp->size);
/*
* first loop through, set initial index and return value
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 73c44cb..ce69b9e 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -3797,7 +3797,7 @@ static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
sg = &(STbp->sg[0]);
frp = STbp->frp;
for (i=count=0; count < length; i++) {
- sg[i].page = frp[i].page;
+ sg_set_page(&sg[i], frp[i].page);
if (length - count > frp[i].length)
sg[i].length = frp[i].length;
else
@@ -4446,14 +4446,14 @@ static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pa
}
/* Populate the scatter/gather list */
- sgl[0].page = pages[0];
+ sg_set_page(&sgl[0], pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
+ sg_set_page(&sgl[i], pages[i]);;
sgl[i].offset = 0;
- sgl[i].page = pages[i];
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -4483,7 +4483,7 @@ static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_p
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
diff --git a/drivers/scsi/tmscsim.c b/drivers/scsi/tmscsim.c
index 5c72ca3..4419304 100644
--- a/drivers/scsi/tmscsim.c
+++ b/drivers/scsi/tmscsim.c
@@ -430,10 +430,7 @@ static __inline__ void dc390_Going_remove (struct dc390_dcb* pDCB, struct dc390_
static struct scatterlist* dc390_sg_build_single(struct scatterlist *sg, void *addr, unsigned int length)
{
- memset(sg, 0, sizeof(struct scatterlist));
- sg->page = virt_to_page(addr);
- sg->length = length;
- sg->offset = (unsigned long)addr & ~PAGE_MASK;
+ sg_init_one(sg, addr, length);
return sg;
}
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..3577453 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -438,12 +438,12 @@ int usb_sg_init (
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/image/microtek.c b/drivers/usb/image/microtek.c
index e7d982a..4866baf 100644
--- a/drivers/usb/image/microtek.c
+++ b/drivers/usb/image/microtek.c
@@ -519,7 +519,7 @@ static void mts_do_sg (struct urb* transfer)
context->fragment++;
mts_int_submit_urb(transfer,
context->data_pipe,
- page_address(sg[context->fragment].page) +
+ page_address(sg_page(&sg[context->fragment])) +
sg[context->fragment].offset,
sg[context->fragment].length,
context->fragment + 1 == scsi_sg_count(context->srb) ?
@@ -557,7 +557,7 @@ mts_build_transfer_context(struct scsi_cmnd *srb, struct mts_desc* desc)
return;
} else {
sg = scsi_sglist(srb);
- desc->context.data = page_address(sg[0].page) + sg[0].offset;
+ desc->context.data = page_address(sg_page(&sg[0])) + sg[0].offset;
desc->context.data_length = sg[0].length;
}
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index e901d31..ca8cb34 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -360,9 +360,10 @@ static void free_sglist (struct scatterlist *sg, int nents)
if (!sg)
return;
for (i = 0; i < nents; i++) {
- if (!sg [i].page)
+ struct page *page = sg_page(&sg[i]);
+ if (!page)
continue;
- kfree (page_address (sg [i].page) + sg [i].offset);
+ kfree (page_address (page) + sg [i].offset);
}
kfree (sg);
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index 1ae90ef..0a9882e 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -283,7 +283,7 @@ int virt_to_scatterlist(const void *addr, int size, struct scatterlist *sg,
pg = virt_to_page(addr);
offset = offset_in_page(addr);
if (sg) {
- sg[i].page = pg;
+ sg_set_page(&sg[i], pg);
sg[i].offset = offset;
}
remainder_of_page = PAGE_CACHE_SIZE - offset;
@@ -713,10 +713,13 @@ ecryptfs_encrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return encrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
@@ -742,10 +745,13 @@ ecryptfs_decrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return decrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index 89d9710..263fed8 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -1040,6 +1040,9 @@ decrypt_passphrase_encrypted_session_key(struct ecryptfs_auth_tok *auth_tok,
};
int rc = 0;
+ sg_init_table(&dst_sg, 1);
+ sg_init_table(&src_sg, 1);
+
if (unlikely(ecryptfs_verbosity > 0)) {
ecryptfs_printk(
KERN_DEBUG, "Session key encryption key (size [%d]):\n",
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index ebd03cc..6f03918 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -88,7 +88,7 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
{
struct xdr_netobj cksum;
struct hash_desc desc;
- struct scatterlist sg[1];
+ struct scatterlist sg;
__be32 status = nfserr_resource;
dprintk("NFSD: nfs4_make_rec_clidname for %.*s\n",
@@ -102,11 +102,9 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
if (cksum.data == NULL)
goto out;
- sg[0].page = virt_to_page(clname->data);
- sg[0].offset = offset_in_page(clname->data);
- sg[0].length = clname->len;
+ sg_init_one(&sg, clname->data, clname->len);
- if (crypto_hash_digest(&desc, sg, sg->length, cksum.data))
+ if (crypto_hash_digest(&desc, &sg, sg.length, cksum.data))
goto out;
md5_to_hex(dname, cksum.data);
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..b4d6a64 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,29 +5,72 @@
#include <linux/mm.h>
#include <linux/string.h>
+/*
+ * Notes on SG table design.
+ *
+ * Architectures must provide an unsigned long page_link field in the
+ * scatterlist struct. We use that to place the page pointer AND encode
+ * information about the sg table as well. The two lower bits are reserved
+ * for this information.
+ *
+ * If bit 0 is set, then the page_link contains a pointer to the next sg
+ * table list. Otherwise the next entry is at sg + 1.
+ *
+ * If bit 1 is set, then this sg entry is the last element in a list.
+ *
+ * See sg_next().
+ *
+ */
+
+#define SG_MAGIC 0x87654321
+
+/**
+ * sg_set_page - Set sg entry to point at given page
+ * @sg: SG entry
+ * @page: The page
+ *
+ * Use this function to set an sg entry pointing at a page, never assign
+ * the page directly. We encode sg table information in the lower bits
+ * of the page pointer. See sg_page() for looking up the page belonging
+ * to an sg entry.
+ *
+ */
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
+/**
+ * sg_set_buf - Set sg entry to point at given data
+ * @sg: SG entry
+ * @buf: Data
+ * @buflen: Data length
+ *
+*/
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
-static inline void sg_init_one(struct scatterlist *sg, const void *buf,
- unsigned int buflen)
-{
- memset(sg, 0, sizeof(*sg));
- sg_set_buf(sg, buf, buflen);
-}
-
/*
* We overload the LSB of the page pointer to indicate whether it's
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+ ((struct scatterlist *) ((sg)->page_link & ~0x03))
/**
* sg_next - return the next scatterlist entry in a list
@@ -37,16 +80,18 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* of a chained scatterlist, it could jump to the start of a new
* scatterlist array.
*
- * Note that the caller must ensure that there are further entries after
- * the current entry, this function will NOT return NULL for an end-of-list.
- *
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
-
- if (unlikely(sg_is_chain(sg)))
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ sg = NULL;
+ else if (sg_is_chain(sg))
sg = sg_chain_ptr(sg);
+ else
+ sg++;
return sg;
}
@@ -83,6 +128,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+#endif
return ret;
}
@@ -101,7 +149,63 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
+}
+
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+/**
+ * sg_init_one - Initialize a single entry sg list
+ * @sg: SG entry
+ * @buf: Virtual address for IO
+ * @buflen: IO length
+ *
+ * Note: this should not be used on a single entry that is part of a larger
+ * table. Use sg_init_table() for that.
+ *
+ */
+static inline void sg_init_one(struct scatterlist *sg, const void *buf,
+ unsigned int buflen)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_mark_end(sg, 1);
+ sg_set_buf(sg, buf, buflen);
+}
+
+/**
+ * sg_init_table - Initialize SG table
+ * @sgl: The SG table
+ * @nents: Number of entries in table
+ *
+ * Note: If this is part of a chained sg table, sg_mark_end() should be
+ * used only on the last table part.
+ *
+ */
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
}
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..183f42c 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,16 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables.
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_tkip.c b/net/ieee80211/ieee80211_crypt_tkip.c
index 6cc54ee..34d4c77 100644
--- a/net/ieee80211/ieee80211_crypt_tkip.c
+++ b/net/ieee80211/ieee80211_crypt_tkip.c
@@ -390,9 +390,7 @@ static int ieee80211_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(tkey->tx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -485,9 +483,7 @@ static int ieee80211_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 12;
crypto_blkcipher_setkey(tkey->rx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4)) {
if (net_ratelimit()) {
printk(KERN_DEBUG ": TKIP: failed to decrypt "
@@ -539,11 +535,12 @@ static int michael_mic(struct crypto_hash *tfm_michael, u8 * key, u8 * hdr,
printk(KERN_WARNING "michael_mic: tfm_michael == NULL\n");
return -1;
}
- sg[0].page = virt_to_page(hdr);
+ sg_init_table(sg, 2);
+ sg_set_page(&sg[0], virt_to_page(hdr));
sg[0].offset = offset_in_page(hdr);
sg[0].length = 16;
- sg[1].page = virt_to_page(data);
+ sg_set_page(&sg[1], virt_to_page(data));
sg[1].offset = offset_in_page(data);
sg[1].length = data_len;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..0af6103 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,9 +170,7 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -212,9 +210,7 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
return -7;
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..cc806d6 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,9 +138,7 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
}
@@ -204,9 +202,7 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
crc = cpu_to_le32(~crc32_le(~0, data, data_len));
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 7818107..cbd64b2 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -726,7 +726,8 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
/* set up scatter list */
end = skb_tail_pointer(skb);
- sg.page = virt_to_page(auth);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(auth));
sg.offset = (unsigned long)(auth) % PAGE_SIZE;
sg.length = end - (unsigned char *)auth;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index f983a36..d5a9785 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1513,7 +1513,8 @@ static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep,
struct hash_desc desc;
/* Sign the message. */
- sg.page = virt_to_page(&cookie->c);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(&cookie->c));
sg.offset = (unsigned long)(&cookie->c) % PAGE_SIZE;
sg.length = bodysize;
keylen = SCTP_SECRET_SIZE;
@@ -1585,7 +1586,8 @@ struct sctp_association *sctp_unpack_cookie(
/* Check the signature. */
keylen = SCTP_SECRET_SIZE;
- sg.page = virt_to_page(bear_cookie);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(bear_cookie));
sg.offset = (unsigned long)(bear_cookie) % PAGE_SIZE;
sg.length = bodysize;
key = (char *)ep->secret_key[ep->current_key];
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
Jens Axboe wrote:
> On Thu, Oct 18 2007, Jeff Garzik wrote:
>> Mark Lord wrote:
>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>> but the top was cut off (isn't there a new config option or patch
>>> to do double-columns or scrollback or something ???.
>> Is this a sata_mv box? If so, could you try this patch?
>
> If anything, that shrinks the size of the resulting request. Did this
> patch make any difference to you?
Not a sata_mv box, so no point here.
ata_piix.
On Thu, Oct 18 2007, Mark Lord wrote:
> Jens Axboe wrote:
>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>> Mark Lord wrote:
>>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>>> but the top was cut off (isn't there a new config option or patch
>>>> to do double-columns or scrollback or something ???.
>>> Is this a sata_mv box? If so, could you try this patch?
>> If anything, that shrinks the size of the resulting request. Did this
>> patch make any difference to you?
>
> Not a sata_mv box, so no point here.
Can you try the big patch I just posted?
--
Jens Axboe
Jens Axboe wrote:
> On Thu, Oct 18 2007, Mark Lord wrote:
>> Jens Axboe wrote:
>>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>>> Mark Lord wrote:
>>>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>>>> but the top was cut off (isn't there a new config option or patch
>>>>> to do double-columns or scrollback or something ???.
>>>> Is this a sata_mv box? If so, could you try this patch?
>>> If anything, that shrinks the size of the resulting request. Did this
>>> patch make any difference to you?
>> Not a sata_mv box, so no point here.
>
> Can you try the big patch I just posted?
I'll hunt for it and try it, but your earlier little patch already works fine.
Cheers
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, Jens Axboe wrote:
> > On Thu, Oct 18 2007, Jens Axboe wrote:
> > > On Thu, Oct 18 2007, David Miller wrote:
> > > > From: Jens Axboe <[email protected]>
> > > > Date: Thu, 18 Oct 2007 13:57:02 +0200
> > > >
> > > > > Thanks a lot, Dave! The patch is a monster right now, I'll work on
> > > > > splitting it into a 3-step process. Any arch help is greatly
> > > > > appreciated.
> > > >
> > > > I have some other bits that my compile hit, such as some things in the
> > > > crypto layer.
> > >
> > > Yeah, I have tons of that so far. I hope to have an allyesconfig
> > > compiling pretty soonish, will send that out then.
> >
> > OK here goes, this compiles with allyesconfig on x86-64. Not too bad,
> > the scsi/ drivers were by far the worst.
>
> Alright, small modifications and tested somewhat. It boots on my laptop
> just fine (CONFIG_DEBUG_SG is set) without warnings. Also tested with
> PAGEALLOC set, that also boots and works fine without issue.
>
> So if this is what we want, the question is how to proceed. Right now
> i386 and x86-64 works, sparc64 PROBABLY works but is not tested. Any
> other arch will NOT compile, since it's not converted to use page_link.
> I'll try and do ppc64 now.
OK, I think that covers every arch out there. I haven't been able to
compile any of them, but it's mostly search'n replace operations. I hope
nothing is missing linux/scatterlist.h includes...
arch/alpha/kernel/pci_iommu.c | 2
arch/arm/common/dmabounce.c | 2
arch/blackfin/kernel/dma-mapping.c | 2
arch/ia64/hp/common/sba_iommu.c | 2
arch/ia64/hp/sim/simscsi.c | 4
arch/ia64/sn/pci/pci_dma.c | 2
arch/m68k/kernel/dma.c | 2
arch/mips/mm/dma-default.c | 8 -
arch/powerpc/kernel/dma_64.c | 2
arch/powerpc/kernel/ibmebus.c | 2
arch/powerpc/kernel/iommu.c | 2
arch/powerpc/platforms/ps3/system-bus.c | 2
arch/sparc/kernel/ioport.c | 16 +-
arch/sparc/mm/io-unit.c | 2
arch/sparc/mm/iommu.c | 8 -
arch/sparc/mm/sun4c.c | 2
arch/sparc64/kernel/iommu.c | 4
arch/sparc64/kernel/iommu_common.c | 12 -
arch/sparc64/kernel/ldc.c | 2
arch/sparc64/kernel/pci_sun4v.c | 4
arch/um/drivers/ubd_kern.c | 2
arch/x86/kernel/pci-calgary_64.c | 12 +
arch/x86/kernel/pci-gart_64.c | 4
arch/x86/kernel/pci-nommu_64.c | 4
block/ll_rw_blk.c | 5
crypto/digest.c | 2
crypto/hmac.c | 3
crypto/scatterwalk.c | 2
crypto/scatterwalk.h | 6
crypto/tcrypt.c | 4
crypto/xcbc.c | 2
drivers/ata/libata-core.c | 10 -
drivers/ata/libata-scsi.c | 2
drivers/block/cciss.c | 4
drivers/block/cpqarray.c | 3
drivers/block/cryptoloop.c | 12 +
drivers/block/ub.c | 8 -
drivers/ide/cris/ide-cris.c | 4
drivers/ide/ide-probe.c | 4
drivers/ide/ide-taskfile.c | 2
drivers/ide/mips/au1xxx-ide.c | 4
drivers/ieee1394/dma.c | 2
drivers/ieee1394/sbp2.c | 2
drivers/infiniband/core/umem.c | 11 +
drivers/infiniband/hw/ipath/ipath_dma.c | 4
drivers/infiniband/hw/ipath/ipath_mr.c | 2
drivers/infiniband/hw/mthca/mthca_memfree.c | 24 ++-
drivers/infiniband/ulp/iser/iser_memory.c | 8 -
drivers/md/dm-crypt.c | 21 +--
drivers/media/common/saa7146_core.c | 3
drivers/media/video/ivtv/ivtv-udma.c | 4
drivers/media/video/videobuf-dma-sg.c | 8 -
drivers/mmc/card/queue.c | 15 +-
drivers/mmc/host/at91_mci.c | 8 -
drivers/mmc/host/au1xmmc.c | 8 -
drivers/mmc/host/imxmmc.c | 2
drivers/mmc/host/mmc_spi.c | 8 -
drivers/mmc/host/omap.c | 4
drivers/mmc/host/sdhci.c | 2
drivers/mmc/host/tifm_sd.c | 8 -
drivers/mmc/host/wbsd.c | 6
drivers/net/mlx4/icm.c | 14 +-
drivers/net/ppp_mppe.c | 6
drivers/scsi/3w-9xxx.c | 4
drivers/scsi/3w-xxxx.c | 2
drivers/scsi/NCR5380.c | 4
drivers/scsi/NCR53c406a.c | 4
drivers/scsi/aacraid/aachba.c | 2
drivers/scsi/aha1542.c | 6
drivers/scsi/arcmsr/arcmsr_hba.c | 4
drivers/scsi/fdomain.c | 6
drivers/scsi/gdth.c | 4
drivers/scsi/ibmmca.c | 2
drivers/scsi/ide-scsi.c | 12 -
drivers/scsi/imm.c | 4
drivers/scsi/ipr.c | 19 +-
drivers/scsi/ips.c | 6
drivers/scsi/iscsi_tcp.c | 15 +-
drivers/scsi/megaraid.c | 6
drivers/scsi/megaraid/megaraid_mbox.c | 8 -
drivers/scsi/osst.c | 32 ++--
drivers/scsi/pcmcia/sym53c500_cs.c | 6
drivers/scsi/ppa.c | 4
drivers/scsi/qlogicfas408.c | 2
drivers/scsi/scsi_debug.c | 4
drivers/scsi/scsi_lib.c | 13 +
drivers/scsi/sg.c | 30 ++--
drivers/scsi/st.c | 8 -
drivers/scsi/tmscsim.c | 5
drivers/scsi/ultrastor.c | 2
drivers/scsi/wd7000.c | 2
drivers/usb/core/message.c | 4
drivers/usb/image/microtek.c | 4
drivers/usb/misc/usbtest.c | 5
drivers/usb/storage/protocol.c | 2
fs/ecryptfs/crypto.c | 16 +-
fs/ecryptfs/keystore.c | 3
fs/nfsd/nfs4recover.c | 8 -
include/asm-alpha/scatterlist.h | 2
include/asm-arm/scatterlist.h | 2
include/asm-avr32/scatterlist.h | 2
include/asm-blackfin/scatterlist.h | 2
include/asm-cris/scatterlist.h | 2
include/asm-frv/scatterlist.h | 2
include/asm-h8300/scatterlist.h | 2
include/asm-ia64/scatterlist.h | 2
include/asm-m32r/scatterlist.h | 2
include/asm-m68k/scatterlist.h | 2
include/asm-m68knommu/scatterlist.h | 2
include/asm-mips/scatterlist.h | 2
include/asm-parisc/scatterlist.h | 2
include/asm-powerpc/scatterlist.h | 2
include/asm-s390/scatterlist.h | 2
include/asm-sh/scatterlist.h | 2
include/asm-sh64/scatterlist.h | 2
include/asm-sparc/scatterlist.h | 2
include/asm-sparc64/scatterlist.h | 5
include/asm-v850/scatterlist.h | 2
include/asm-x86/dma-mapping_32.h | 4
include/asm-x86/scatterlist_32.h | 5
include/asm-x86/scatterlist_64.h | 5
include/asm-xtensa/scatterlist.h | 2
include/linux/scatterlist.h | 138 +++++++++++++++++---
lib/Kconfig.debug | 10 +
lib/swiotlb.c | 2
net/core/skbuff.c | 4
net/ieee80211/ieee80211_crypt_tkip.c | 13 -
net/ieee80211/ieee80211_crypt_wep.c | 8 -
net/mac80211/wep.c | 8 -
net/sctp/auth.c | 3
net/sctp/sm_make_chunk.c | 6
net/sunrpc/auth_gss/gss_krb5_crypto.c | 10 -
net/sunrpc/xdr.c | 2
net/xfrm/xfrm_algo.c | 4
134 files changed, 520 insertions(+), 356 deletions(-)
diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index e1c4707..094a95e 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -465,7 +465,7 @@ EXPORT_SYMBOL(pci_free_consistent);
Write dma_length of each leader with the combined lengths of
the mergable followers. */
-#define SG_ENT_VIRT_ADDRESS(SG) (page_address((SG)->page) + (SG)->offset)
+#define SG_ENT_VIRT_ADDRESS(SG) (page_address(sg_page((SG)) + (SG)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) __pa(SG_ENT_VIRT_ADDRESS(SG))
static void
diff --git a/arch/arm/common/dmabounce.c b/arch/arm/common/dmabounce.c
index 44ab0da..9d371e4 100644
--- a/arch/arm/common/dmabounce.c
+++ b/arch/arm/common/dmabounce.c
@@ -442,7 +442,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(dir == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- struct page *page = sg->page;
+ struct page *page = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int length = sg->length;
void *ptr = page_address(page) + offset;
diff --git a/arch/blackfin/kernel/dma-mapping.c b/arch/blackfin/kernel/dma-mapping.c
index 94d7b11..dbf7222 100644
--- a/arch/blackfin/kernel/dma-mapping.c
+++ b/arch/blackfin/kernel/dma-mapping.c
@@ -160,7 +160,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(direction == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- sg->dma_address = (dma_addr_t)(page_address(sg->page) +
+ sg->dma_address = (dma_addr_t)(page_address(sg_page(sg)) +
sg->offset);
invalidate_dcache_range(sg_dma_address(sg),
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index 3c95f41..0b74c63 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -246,7 +246,7 @@ static int reserve_sba_gart = 1;
static SBA_INLINE void sba_mark_invalid(struct ioc *, dma_addr_t, size_t);
static SBA_INLINE void sba_free_range(struct ioc *, dma_addr_t, size_t);
-#define sba_sg_address(sg) (page_address((sg)->page) + (sg)->offset)
+#define sba_sg_address(sg) (page_address(sg_page((sg)) + (sg)->offset))
#ifdef FULL_VALID_PDIR
static u64 prefetch_spill_page;
diff --git a/arch/ia64/hp/sim/simscsi.c b/arch/ia64/hp/sim/simscsi.c
index a3a558a..7f1794f 100644
--- a/arch/ia64/hp/sim/simscsi.c
+++ b/arch/ia64/hp/sim/simscsi.c
@@ -131,7 +131,7 @@ simscsi_sg_readwrite (struct scsi_cmnd *sc, int mode, unsigned long offset)
stat.fd = desc[sc->device->id];
scsi_for_each_sg(sc, sl, scsi_sg_count(sc), i) {
- req.addr = __pa(page_address(sl->page) + sl->offset);
+ req.addr = __pa(page_address(sg_page(sl)) + sl->offset);
req.len = sl->length;
if (DBG)
printk("simscsi_sg_%s @ %lx (off %lx) use_sg=%d len=%d\n",
@@ -212,7 +212,7 @@ static void simscsi_fillresult(struct scsi_cmnd *sc, char *buf, unsigned len)
if (!len)
break;
thislen = min(len, slp->length);
- memcpy(page_address(slp->page) + slp->offset, buf, thislen);
+ memcpy(page_address(sg_page(slp)) + slp->offset, buf, thislen);
len -= thislen;
}
}
diff --git a/arch/ia64/sn/pci/pci_dma.c b/arch/ia64/sn/pci/pci_dma.c
index ecd8a52..6048eaf 100644
--- a/arch/ia64/sn/pci/pci_dma.c
+++ b/arch/ia64/sn/pci/pci_dma.c
@@ -16,7 +16,7 @@
#include <asm/sn/pcidev.h>
#include <asm/sn/sn_sal.h>
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) virt_to_phys(SG_ENT_VIRT_ADDRESS(SG))
/**
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index 9d4e4b5..b2d620b 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -121,7 +121,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
int i;
for (i = 0; i < nents; sg++, i++) {
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
dma_sync_single_for_device(dev, sg->dma_address, sg->length, dir);
}
return nents;
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index 98b5e5b..4aa143b 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -165,7 +165,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
for (i = 0; i < nents; i++, sg++) {
unsigned long addr;
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (!plat_device_is_coherent(dev) && addr)
__dma_sync(addr + sg->offset, sg->length, direction);
sg->dma_address = plat_map_dma_mem(dev,
@@ -223,7 +223,7 @@ void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries,
for (i = 0; i < nhwentries; i++, sg++) {
if (!plat_device_is_coherent(dev) &&
direction != DMA_TO_DEVICE) {
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (addr)
__dma_sync(addr + sg->offset, sg->length,
direction);
@@ -304,7 +304,7 @@ void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg, int nelems,
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (cpu_is_noncoherent_r10000(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
@@ -322,7 +322,7 @@ void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg, int nele
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (!plat_device_is_coherent(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
index 9001104..5d0aa90 100644
--- a/arch/powerpc/kernel/dma_64.c
+++ b/arch/powerpc/kernel/dma_64.c
@@ -161,7 +161,7 @@ static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (page_to_phys(sg->page) + sg->offset) |
+ sg->dma_address = (page_to_phys(sg_page(sg)) + sg->offset) |
dma_direct_offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index 289d7e9..3d0e71b 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -102,7 +102,7 @@ static int ibmebus_map_sg(struct device *dev,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (dma_addr_t)page_address(sg->page)
+ sg->dma_address = (dma_addr_t)page_address(sg_page(sg))
+ sg->offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index 306a6f7..47d745a 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -307,7 +307,7 @@ int iommu_map_sg(struct iommu_table *tbl, struct scatterlist *sglist,
continue;
}
/* Allocate iommu entries for that segment */
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(sg_page(s)) + s->offset;
npages = iommu_num_pages(vaddr, slen);
entry = iommu_range_alloc(tbl, npages, &handle, mask >> IOMMU_PAGE_SHIFT, 0);
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 07e64b4..e8202b6 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -629,7 +629,7 @@ static int ps3_sb_map_sg(struct device *_dev, struct scatterlist *sgl,
for_each_sg(sgl, sg, nents, i) {
int result = ps3_dma_map(dev->d_region,
- page_to_phys(sg->page) + sg->offset, sg->length,
+ page_to_phys(sg_page(sg)) + sg->offset, sg->length,
&sg->dma_address, 0);
if (result) {
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index 9c3ed88..abc4470 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -727,9 +727,9 @@ int pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
/* IIep is write-through, not flushing. */
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
sg->dvma_address =
- virt_to_phys(page_address(sg->page)) + sg->offset;
+ virt_to_phys(page_address(sg_page(sg))) + sg->offset;
sg->dvma_length = sg->length;
}
return nents;
@@ -748,9 +748,9 @@ void pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -798,9 +798,9 @@ void pci_dma_sync_sg_for_cpu(struct pci_dev *hwdev, struct scatterlist *sgl, int
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -814,9 +814,9 @@ void pci_dma_sync_sg_for_device(struct pci_dev *hwdev, struct scatterlist *sgl,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
diff --git a/arch/sparc/mm/io-unit.c b/arch/sparc/mm/io-unit.c
index 375b4db..0f2ad57 100644
--- a/arch/sparc/mm/io-unit.c
+++ b/arch/sparc/mm/io-unit.c
@@ -144,7 +144,7 @@ static void iounit_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus
spin_lock_irqsave(&iounit->lock, flags);
while (sz != 0) {
--sz;
- sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/iommu.c b/arch/sparc/mm/iommu.c
index 283656d..4b93427 100644
--- a/arch/sparc/mm/iommu.c
+++ b/arch/sparc/mm/iommu.c
@@ -238,7 +238,7 @@ static void iommu_get_scsi_sgl_noflush(struct scatterlist *sg, int sz, struct sb
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -252,7 +252,7 @@ static void iommu_get_scsi_sgl_gflush(struct scatterlist *sg, int sz, struct sbu
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -273,7 +273,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
* XXX Is this a good assumption?
* XXX What if someone else unmaps it here and races us?
*/
- if ((page = (unsigned long) page_address(sg->page)) != 0) {
+ if ((page = (unsigned long) page_address(sg_page(sg))) != 0) {
for (i = 0; i < n; i++) {
if (page != oldpage) { /* Already flushed? */
flush_page_for_dma(page);
@@ -283,7 +283,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
}
}
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/sun4c.c b/arch/sparc/mm/sun4c.c
index ee6708f..f172dbe 100644
--- a/arch/sparc/mm/sun4c.c
+++ b/arch/sparc/mm/sun4c.c
@@ -1228,7 +1228,7 @@ static void sun4c_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus *
{
while (sz != 0) {
--sz;
- sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/ldc.c b/arch/sparc64/kernel/ldc.c
index 85a2be0..c8313cb 100644
--- a/arch/sparc64/kernel/ldc.c
+++ b/arch/sparc64/kernel/ldc.c
@@ -2057,7 +2057,7 @@ static void fill_cookies(struct cookie_state *sp, unsigned long pa,
static int sg_count_one(struct scatterlist *sg)
{
- unsigned long base = page_to_pfn(sg->page) << PAGE_SHIFT;
+ unsigned long base = page_to_pfn(sg_page(sg)) << PAGE_SHIFT;
long len = sg->length;
if ((sg->offset | len) & (8UL - 1))
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 25b248a..3a8cd3d 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -1115,7 +1115,7 @@ static void do_ubd_request(struct request_queue *q)
}
prepare_request(req, io_req,
(unsigned long long) req->sector << 9,
- sg->offset, sg->length, sg->page);
+ sg->offset, sg->length, sg_page(sg));
last_sectors = sg->length >> 9;
n = os_write_file(thread_fd, &io_req,
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5098f58..c9f0eb0 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -411,8 +411,10 @@ static int calgary_nontranslate_map_sg(struct device* dev,
int i;
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
+ s->dma_address = virt_to_bus(page_address(p) + s->offset);
s->dma_length = s->length;
}
return nelems;
@@ -432,9 +434,11 @@ static int calgary_map_sg(struct device *dev, struct scatterlist *sg,
return calgary_nontranslate_map_sg(dev, sg, nelems, direction);
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(p) + s->offset;
npages = num_dma_pages(vaddr, s->length);
entry = iommu_range_alloc(tbl, npages);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..ce9754f 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,8 +1354,9 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
nsegs++;
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/hmac.c b/crypto/hmac.c
index 8802fb6..e4eb6ac 100644
--- a/crypto/hmac.c
+++ b/crypto/hmac.c
@@ -159,7 +159,8 @@ static int hmac_digest(struct hash_desc *pdesc, struct scatterlist *sg,
desc.flags = pdesc->flags & CRYPTO_TFM_REQ_MAY_SLEEP;
sg_set_buf(sg1, ipad, bs);
- sg1[1].page = (void *)sg;
+
+ sg_set_page(&sg[1], (void *) sg);
sg1[1].length = 0;
sg_set_buf(sg2, opad, bs + ds);
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 18d489c..d741c63 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -317,7 +317,7 @@ static void test_cipher(char *algo, int enc,
goto out;
}
- q = kmap(sg[0].page) + sg[0].offset;
+ q = kmap(sg_page(&sg[0])) + sg[0].offset;
hexdump(q, cipher_tv[i].rlen);
printk("%s\n",
@@ -390,7 +390,7 @@ static void test_cipher(char *algo, int enc,
temp = 0;
for (k = 0; k < cipher_tv[i].np; k++) {
printk("page %u\n", k);
- q = kmap(sg[k].page) + sg[k].offset;
+ q = kmap(sg_page(&sg[k])) + sg[k].offset;
hexdump(q, cipher_tv[i].tap[k]);
printk("%s\n",
memcmp(q, cipher_tv[i].result + temp,
diff --git a/crypto/xcbc.c b/crypto/xcbc.c
index 9f502b8..ac68f3b 100644
--- a/crypto/xcbc.c
+++ b/crypto/xcbc.c
@@ -120,7 +120,7 @@ static int crypto_xcbc_digest_update2(struct hash_desc *pdesc,
do {
- struct page *pg = sg[i].page;
+ struct page *pg = sg_page(&sg[i]);
unsigned int offset = sg[i].offset;
unsigned int slen = sg[i].length;
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
index 7c2cfde..5a6fe17 100644
--- a/drivers/block/cciss.c
+++ b/drivers/block/cciss.c
@@ -2610,7 +2610,7 @@ static void do_cciss_request(struct request_queue *q)
(int)creq->nr_sectors);
#endif /* CCISS_DEBUG */
- memset(tmp_sg, 0, sizeof(tmp_sg));
+ sg_init_table(tmp_sg, MAXSGENTRIES);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* get the DMA records for the setup */
@@ -2621,7 +2621,7 @@ static void do_cciss_request(struct request_queue *q)
for (i = 0; i < seg; i++) {
c->SG[i].Len = tmp_sg[i].length;
- temp64.val = (__u64) pci_map_page(h->pdev, tmp_sg[i].page,
+ temp64.val = (__u64) pci_map_page(h->pdev, sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
c->SG[i].Addr.lower = temp64.val32.lower;
diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c
index 568603d..efab27f 100644
--- a/drivers/block/cpqarray.c
+++ b/drivers/block/cpqarray.c
@@ -918,6 +918,7 @@ queue_next:
DBGPX(
printk("sector=%d, nr_sectors=%d\n", creq->sector, creq->nr_sectors);
);
+ sg_init_table(tmp_sg, SG_MAX);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* Now do all the DMA Mappings */
@@ -929,7 +930,7 @@ DBGPX(
{
c->req.sg[i].size = tmp_sg[i].length;
c->req.sg[i].addr = (__u32) pci_map_page(h->pci_dev,
- tmp_sg[i].page,
+ sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
}
diff --git a/drivers/block/cryptoloop.c b/drivers/block/cryptoloop.c
index 4053503..1b58b01 100644
--- a/drivers/block/cryptoloop.c
+++ b/drivers/block/cryptoloop.c
@@ -26,6 +26,7 @@
#include <linux/crypto.h>
#include <linux/blkdev.h>
#include <linux/loop.h>
+#include <linux/scatterlist.h>
#include <asm/semaphore.h>
#include <asm/uaccess.h>
@@ -119,14 +120,17 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
.tfm = tfm,
.flags = CRYPTO_TFM_REQ_MAY_SLEEP,
};
- struct scatterlist sg_out = { NULL, };
- struct scatterlist sg_in = { NULL, };
+ struct scatterlist sg_out;
+ struct scatterlist sg_in;
encdec_cbc_t encdecfunc;
struct page *in_page, *out_page;
unsigned in_offs, out_offs;
int err;
+ sg_init_table(&sg_out, 1);
+ sg_init_table(&sg_in, 1);
+
if (cmd == READ) {
in_page = raw_page;
in_offs = raw_off;
@@ -146,11 +150,11 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
u32 iv[4] = { 0, };
iv[0] = cpu_to_le32(IV & 0xffffffff);
- sg_in.page = in_page;
+ sg_set_page(&sg_in, in_page);
sg_in.offset = in_offs;
sg_in.length = sz;
- sg_out.page = out_page;
+ sg_set_page(&sg_out, out_page);
sg_out.offset = out_offs;
sg_out.length = sz;
diff --git a/drivers/block/ub.c b/drivers/block/ub.c
index c57dd2b..d6f7b43 100644
--- a/drivers/block/ub.c
+++ b/drivers/block/ub.c
@@ -25,6 +25,7 @@
#include <linux/usb_usual.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
+#include <linux/scatterlist.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
@@ -656,6 +657,7 @@ static int ub_request_fn_1(struct ub_lun *lun, struct request *rq)
if ((cmd = ub_get_cmd(lun)) == NULL)
return -1;
memset(cmd, 0, sizeof(struct ub_scsi_cmd));
+ sg_init_table(cmd->sgv, UB_MAX_REQ_SG);
blkdev_dequeue_request(rq);
@@ -1310,7 +1312,7 @@ static void ub_data_start(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
pipe = sc->send_bulk_pipe;
sc->last_pipe = pipe;
usb_fill_bulk_urb(&sc->work_urb, sc->dev, pipe,
- page_address(sg->page) + sg->offset, sg->length,
+ page_address(sg_page(sg)) + sg->offset, sg->length,
ub_urb_complete, sc);
sc->work_urb.actual_length = 0;
sc->work_urb.error_count = 0;
@@ -1427,7 +1429,7 @@ static void ub_state_sense(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
scmd->state = UB_CMDST_INIT;
scmd->nsg = 1;
sg = &scmd->sgv[0];
- sg->page = virt_to_page(sc->top_sense);
+ sg_set_page(sg, virt_to_page(sc->top_sense));
sg->offset = (unsigned long)sc->top_sense & (PAGE_SIZE-1);
sg->length = UB_SENSE_SIZE;
scmd->len = UB_SENSE_SIZE;
@@ -1863,7 +1865,7 @@ static int ub_sync_read_cap(struct ub_dev *sc, struct ub_lun *lun,
cmd->state = UB_CMDST_INIT;
cmd->nsg = 1;
sg = &cmd->sgv[0];
- sg->page = virt_to_page(p);
+ sg_set_page(sg, virt_to_page(p));
sg->offset = (unsigned long)p & (PAGE_SIZE-1);
sg->length = 8;
cmd->len = 8;
diff --git a/drivers/ide/cris/ide-cris.c b/drivers/ide/cris/ide-cris.c
index 06c75f1..3979bf4 100644
--- a/drivers/ide/cris/ide-cris.c
+++ b/drivers/ide/cris/ide-cris.c
@@ -934,11 +934,11 @@ static int cris_ide_build_dmatable (ide_drive_t *drive)
* than two possibly non-adjacent physical 4kB pages.
*/
/* group sequential buffers into one large buffer */
- addr = page_to_phys(sg->page) + sg->offset;
+ addr = page_to_phys(sg_page(sg)) + sg->offset;
size = sg_dma_len(sg);
while (--i) {
sg = sg_next(sg);
- if ((addr + size) != page_to_phys(sg->page) + sg->offset)
+ if ((addr + size) != page_to_phys(sg_page(sg)) + sg->offset)
break;
size += sg_dma_len(sg);
}
diff --git a/drivers/ide/ide-probe.c b/drivers/ide/ide-probe.c
index 3c945d6..7c640cd 100644
--- a/drivers/ide/ide-probe.c
+++ b/drivers/ide/ide-probe.c
@@ -1337,12 +1337,14 @@ static int hwif_init(ide_hwif_t *hwif)
if (!hwif->sg_max_nents)
hwif->sg_max_nents = PRD_ENTRIES;
- hwif->sg_table = kzalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
+ hwif->sg_table = kmalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
GFP_KERNEL);
if (!hwif->sg_table) {
printk(KERN_ERR "%s: unable to allocate SG table.\n", hwif->name);
goto out;
}
+
+ sg_init_table(hwif->sg_table, hwif->sg_max_nents);
if (init_irq(hwif) == 0)
goto done;
diff --git a/drivers/ide/ide-taskfile.c b/drivers/ide/ide-taskfile.c
index 2a3c8d4..086157f 100644
--- a/drivers/ide/ide-taskfile.c
+++ b/drivers/ide/ide-taskfile.c
@@ -278,7 +278,7 @@ static void ide_pio_sector(ide_drive_t *drive, unsigned int write)
hwif->cursg = sg;
}
- page = cursg->page;
+ page = sg_page(cursg);
offset = cursg->offset + hwif->cursg_ofs * SECTOR_SIZE;
/* get the current page and offset */
diff --git a/drivers/ide/mips/au1xxx-ide.c b/drivers/ide/mips/au1xxx-ide.c
index 47c035a..1b22ab7 100644
--- a/drivers/ide/mips/au1xxx-ide.c
+++ b/drivers/ide/mips/au1xxx-ide.c
@@ -276,7 +276,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
if (iswrite) {
if(!put_source_flags(ahwif->tx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
@@ -285,7 +285,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
} else
{
if(!put_dest_flags(ahwif->rx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/ieee1394/sbp2.c b/drivers/ieee1394/sbp2.c
index 1b353b9..d5dfe11 100644
--- a/drivers/ieee1394/sbp2.c
+++ b/drivers/ieee1394/sbp2.c
@@ -1466,7 +1466,7 @@ static void sbp2_prep_command_orb_sg(struct sbp2_command_orb *orb,
cmd->dma_size = sgpnt[0].length;
cmd->dma_type = CMD_DMA_PAGE;
cmd->cmd_dma = dma_map_page(hi->host->device.parent,
- sgpnt[0].page, sgpnt[0].offset,
+ sg_page(&sgpnt[0]), sgpnt[0].offset,
cmd->dma_size, cmd->dma_dir);
orb->data_descriptor_lo = cmd->cmd_dma;
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 2f54e29..14159ff 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -55,9 +55,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
ib_dma_unmap_sg(dev, chunk->page_list,
chunk->nents, DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->nents; ++i) {
+ struct page *page = sg_page(&chunk->page_list[i]);
+
if (umem->writable && dirty)
- set_page_dirty_lock(chunk->page_list[i].page);
- put_page(chunk->page_list[i].page);
+ set_page_dirty_lock(page);
+ put_page(page);
}
kfree(chunk);
@@ -164,11 +166,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
}
chunk->nents = min_t(int, ret, IB_UMEM_MAX_PAGE_CHUNK);
+ sg_init_table(chunk->page_list, chunk->nents);
for (i = 0; i < chunk->nents; ++i) {
if (vma_list &&
!is_vm_hugetlb_page(vma_list[i + off]))
umem->hugetlb = 0;
- chunk->page_list[i].page = page_list[i + off];
+ sg_set_page(&chunk->page_list[i], page_list[i + off]);
chunk->page_list[i].offset = 0;
chunk->page_list[i].length = PAGE_SIZE;
}
@@ -179,7 +182,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
DMA_BIDIRECTIONAL);
if (chunk->nmap <= 0) {
for (i = 0; i < chunk->nents; ++i)
- put_page(chunk->page_list[i].page);
+ put_page(sg_page(&chunk->page_list[i]));
kfree(chunk);
ret = -ENOMEM;
diff --git a/drivers/infiniband/hw/ipath/ipath_dma.c b/drivers/infiniband/hw/ipath/ipath_dma.c
index 22709a4..e90a0ea 100644
--- a/drivers/infiniband/hw/ipath/ipath_dma.c
+++ b/drivers/infiniband/hw/ipath/ipath_dma.c
@@ -108,7 +108,7 @@ static int ipath_map_sg(struct ib_device *dev, struct scatterlist *sgl,
BUG_ON(!valid_dma_direction(direction));
for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg->page);
+ addr = (u64) page_address(sg_page(sg));
/* TODO: handle highmem pages */
if (!addr) {
ret = 0;
@@ -127,7 +127,7 @@ static void ipath_unmap_sg(struct ib_device *dev,
static u64 ipath_sg_dma_address(struct ib_device *dev, struct scatterlist *sg)
{
- u64 addr = (u64) page_address(sg->page);
+ u64 addr = (u64) page_address(sg_page(sg));
if (addr)
addr += sg->offset;
diff --git a/drivers/infiniband/hw/ipath/ipath_mr.c b/drivers/infiniband/hw/ipath/ipath_mr.c
index e442470..db4ba92 100644
--- a/drivers/infiniband/hw/ipath/ipath_mr.c
+++ b/drivers/infiniband/hw/ipath/ipath_mr.c
@@ -225,7 +225,7 @@ struct ib_mr *ipath_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
for (i = 0; i < chunk->nents; i++) {
void *vaddr;
- vaddr = page_address(chunk->page_list[i].page);
+ vaddr = page_address(sg_page(&chunk->page_list[i]));
if (!vaddr) {
ret = ERR_PTR(-EINVAL);
goto bail;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index e61f3e6..007b381 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -71,7 +71,7 @@ static void mthca_free_icm_pages(struct mthca_dev *dev, struct mthca_icm_chunk *
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -81,7 +81,7 @@ static void mthca_free_icm_coherent(struct mthca_dev *dev, struct mthca_icm_chun
for (i = 0; i < chunk->npages; ++i) {
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
}
@@ -107,10 +107,13 @@ void mthca_free_icm(struct mthca_dev *dev, struct mthca_icm *icm, int coherent)
static int mthca_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -157,6 +160,7 @@ struct mthca_icm *mthca_alloc_icm(struct mthca_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MTHCA_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -304,7 +308,7 @@ void *mthca_table_find(struct mthca_icm_table *table, int obj, dma_addr_t *dma_h
* so if we found the page, dma_handle has already
* been assigned to. */
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
@@ -445,6 +449,7 @@ static u64 mthca_uarc_virt(struct mthca_dev *dev, struct mthca_uar *uar, int pag
int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
struct mthca_user_db_table *db_tab, int index, u64 uaddr)
{
+ struct page *pages[1];
int ret = 0;
u8 status;
int i;
@@ -472,16 +477,17 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
}
ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- &db_tab->page[i].mem.page, NULL);
+ pages, NULL);
if (ret < 0)
goto out;
+ sg_set_page(&db_tab->page[i].mem, pages[0]);
db_tab->page[i].mem.length = MTHCA_ICM_PAGE_SIZE;
db_tab->page[i].mem.offset = uaddr & ~PAGE_MASK;
ret = pci_map_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
if (ret < 0) {
- put_page(db_tab->page[i].mem.page);
+ put_page(pages[0]);
goto out;
}
@@ -491,7 +497,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
ret = -EINVAL;
if (ret) {
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
goto out;
}
@@ -557,7 +563,7 @@ void mthca_cleanup_user_db_tab(struct mthca_dev *dev, struct mthca_uar *uar,
if (db_tab->page[i].uvirt) {
mthca_UNMAP_ICM(dev, mthca_uarc_virt(dev, uar, i), 1, &status);
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
}
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index f3529b6..813b96b 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -131,7 +131,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, data->size, i) {
- from = kmap_atomic(sg->page, KM_USER0);
+ from = kmap_atomic(sg_page(sg), KM_USER0);
memcpy(p,
from + sg->offset,
sg->length);
@@ -191,7 +191,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, sg_size, i) {
- to = kmap_atomic(sg->page, KM_SOFTIRQ0);
+ to = kmap_atomic(sg_page(sg), KM_SOFTIRQ0);
memcpy(to + sg->offset,
p,
sg->length);
@@ -300,7 +300,7 @@ static unsigned int iser_data_buf_aligned_len(struct iser_data_buf *data,
for_each_sg(sgl, sg, data->dma_nents, i) {
/* iser_dbg("Checking sg iobuf [%d]: phys=0x%08lX "
"offset: %ld sz: %ld\n", i,
- (unsigned long)page_to_phys(sg->page),
+ (unsigned long)page_to_phys(sg_page(sg)),
(unsigned long)sg->offset,
(unsigned long)sg->length); */
end_addr = ib_sg_dma_address(ibdev, sg) +
@@ -336,7 +336,7 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
- sg->page, sg->offset,
+ sg_page(sg), sg->offset,
sg->length, ib_sg_dma_len(ibdev, sg));
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 64fee90..cde915f 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -346,16 +346,17 @@ static int crypt_convert(struct crypt_config *cc,
ctx->idx_out < ctx->bio_out->bi_vcnt) {
struct bio_vec *bv_in = bio_iovec_idx(ctx->bio_in, ctx->idx_in);
struct bio_vec *bv_out = bio_iovec_idx(ctx->bio_out, ctx->idx_out);
- struct scatterlist sg_in = {
- .page = bv_in->bv_page,
- .offset = bv_in->bv_offset + ctx->offset_in,
- .length = 1 << SECTOR_SHIFT
- };
- struct scatterlist sg_out = {
- .page = bv_out->bv_page,
- .offset = bv_out->bv_offset + ctx->offset_out,
- .length = 1 << SECTOR_SHIFT
- };
+ struct scatterlist sg_in, sg_out;
+
+ sg_init_table(&sg_in, 1);
+ sg_set_page(&sg_in, bv_in->bv_page);
+ sg_in.offset = bv_in->bv_offset + ctx->offset_in;
+ sg_in.length = 1 << SECTOR_SHIFT;
+
+ sg_init_table(&sg_out, 1);
+ sg_set_page(&sg_out, bv_out->bv_page);
+ sg_out.offset = bv_out->bv_offset + ctx->offset_out;
+ sg_out.length = 1 << SECTOR_SHIFT;
ctx->offset_in += sg_in.length;
if (ctx->offset_in >= bv_in->bv_len) {
diff --git a/drivers/media/common/saa7146_core.c b/drivers/media/common/saa7146_core.c
index 365a221..2b1f8b4 100644
--- a/drivers/media/common/saa7146_core.c
+++ b/drivers/media/common/saa7146_core.c
@@ -112,12 +112,13 @@ static struct scatterlist* vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/media/video/ivtv/ivtv-udma.c b/drivers/media/video/ivtv/ivtv-udma.c
index c4626d1..912b424 100644
--- a/drivers/media/video/ivtv/ivtv-udma.c
+++ b/drivers/media/video/ivtv/ivtv-udma.c
@@ -63,10 +63,10 @@ int ivtv_udma_fill_sg_list (struct ivtv_user_dma *dma, struct ivtv_dma_page_info
memcpy(page_address(dma->bouncemap[map_offset]) + offset, src, len);
kunmap_atomic(src, KM_BOUNCE_READ);
local_irq_restore(flags);
- dma->SGlist[map_offset].page = dma->bouncemap[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->bouncemap[map_offset]);
}
else {
- dma->SGlist[map_offset].page = dma->map[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->map[map_offset]);
}
offset = 0;
map_offset++;
diff --git a/drivers/media/video/videobuf-dma-sg.c b/drivers/media/video/videobuf-dma-sg.c
index 3eb6123..0a18286 100644
--- a/drivers/media/video/videobuf-dma-sg.c
+++ b/drivers/media/video/videobuf-dma-sg.c
@@ -60,12 +60,13 @@ videobuf_vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
@@ -86,13 +87,14 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
sglist = kcalloc(nr_pages, sizeof(*sglist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
if (NULL == pages[0])
goto nopage;
if (PageHighMem(pages[0]))
/* DMA to highmem pages might not work */
goto highmem;
- sglist[0].page = pages[0];
+ sg_set_page(&sglist[0], pages[0]);
sglist[0].offset = offset;
sglist[0].length = PAGE_SIZE - offset;
for (i = 1; i < nr_pages; i++) {
@@ -100,7 +102,7 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
goto nopage;
if (PageHighMem(pages[i]))
goto highmem;
- sglist[i].page = pages[i];
+ sg_set_page(&sglist[i], pages[i]);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/mmc/card/queue.c b/drivers/mmc/card/queue.c
index a5d0354..68c1fe9 100644
--- a/drivers/mmc/card/queue.c
+++ b/drivers/mmc/card/queue.c
@@ -13,6 +13,7 @@
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/scatterlist.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -153,19 +154,21 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card, spinlock_t *lock
blk_queue_max_hw_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
- mq->sg = kzalloc(sizeof(struct scatterlist),
+ mq->sg = kmalloc(sizeof(struct scatterlist),
GFP_KERNEL);
if (!mq->sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->sg, 1);
- mq->bounce_sg = kzalloc(sizeof(struct scatterlist) *
+ mq->bounce_sg = kmalloc(sizeof(struct scatterlist) *
bouncesz / 512, GFP_KERNEL);
if (!mq->bounce_sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->bounce_sg, bouncesz / 512);
}
}
#endif
@@ -302,12 +305,12 @@ static void copy_sg(struct scatterlist *dst, unsigned int dst_len,
BUG_ON(dst_len == 0);
if (dst_size == 0) {
- dst_buf = page_address(dst->page) + dst->offset;
+ dst_buf = page_address(sg_page(dst)) + dst->offset;
dst_size = dst->length;
}
if (src_size == 0) {
- src_buf = page_address(src->page) + src->offset;
+ src_buf = page_address(sg_page(src)) + src->offset;
src_size = src->length;
}
@@ -353,9 +356,7 @@ unsigned int mmc_queue_map_sg(struct mmc_queue *mq)
return 1;
}
- mq->sg[0].page = virt_to_page(mq->bounce_buf);
- mq->sg[0].offset = offset_in_page(mq->bounce_buf);
- mq->sg[0].length = 0;
+ sg_init_one(mq->sg, mq->bounce_buf, 0);
while (sg_len) {
mq->sg[0].length += mq->bounce_sg[sg_len - 1].length;
diff --git a/drivers/mmc/host/at91_mci.c b/drivers/mmc/host/at91_mci.c
index 7a452c2..b1edcef 100644
--- a/drivers/mmc/host/at91_mci.c
+++ b/drivers/mmc/host/at91_mci.c
@@ -149,7 +149,7 @@ static inline void at91_mci_sg_to_dma(struct at91mci_host *host, struct mmc_data
sg = &data->sg[i];
- sgbuffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ sgbuffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
amount = min(size, sg->length);
size -= amount;
@@ -226,7 +226,7 @@ static void at91_mci_pre_dma_read(struct at91mci_host *host)
sg = &data->sg[host->transfer_index++];
pr_debug("sg = %p\n", sg);
- sg->dma_address = dma_map_page(NULL, sg->page, sg->offset, sg->length, DMA_FROM_DEVICE);
+ sg->dma_address = dma_map_page(NULL, sg_page(sg), sg->offset, sg->length, DMA_FROM_DEVICE);
pr_debug("dma address = %08X, length = %d\n", sg->dma_address, sg->length);
@@ -283,7 +283,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
int index;
/* Swap the contents of the buffer */
- buffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
pr_debug("buffer = %p, length = %d\n", buffer, sg->length);
for (index = 0; index < (sg->length / 4); index++)
@@ -292,7 +292,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
kunmap_atomic(buffer, KM_BIO_SRC_IRQ);
}
- flush_dcache_page(sg->page);
+ flush_dcache_page(sg_page(sg));
}
/* Is there another transfer to trigger? */
diff --git a/drivers/mmc/host/au1xmmc.c b/drivers/mmc/host/au1xmmc.c
index 92c4d0d..0632f4d 100644
--- a/drivers/mmc/host/au1xmmc.c
+++ b/drivers/mmc/host/au1xmmc.c
@@ -340,7 +340,7 @@ static void au1xmmc_send_pio(struct au1xmmc_host *host)
/* This is the pointer to the data buffer */
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = data->sg[host->pio.index].length - host->pio.offset;
@@ -400,7 +400,7 @@ static void au1xmmc_receive_pio(struct au1xmmc_host *host)
if (host->pio.index < host->dma.len) {
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = sg_dma_len(&data->sg[host->pio.index]) - host->pio.offset;
@@ -613,13 +613,13 @@ au1xmmc_prepare_data(struct au1xmmc_host *host, struct mmc_data *data)
if (host->flags & HOST_F_XMIT){
ret = au1xxx_dbdma_put_source_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
else {
ret = au1xxx_dbdma_put_dest_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
diff --git a/drivers/mmc/host/imxmmc.c b/drivers/mmc/host/imxmmc.c
index 6ebc41e..581bb8f 100644
--- a/drivers/mmc/host/imxmmc.c
+++ b/drivers/mmc/host/imxmmc.c
@@ -262,7 +262,7 @@ static void imxmci_setup_data(struct imxmci_host *host, struct mmc_data *data)
}
/* Convert back to virtual address */
- host->data_ptr = (u16*)(page_address(data->sg->page) + data->sg->offset);
+ host->data_ptr = (u16*)(page_address(sg_page(data->sg)) + data->sg->offset);
host->data_cnt = 0;
clear_bit(IMXMCI_PEND_DMA_DATA_b, &host->pending_events);
diff --git a/drivers/mmc/host/mmc_spi.c b/drivers/mmc/host/mmc_spi.c
index 7ae18ea..12c2d80 100644
--- a/drivers/mmc/host/mmc_spi.c
+++ b/drivers/mmc/host/mmc_spi.c
@@ -813,7 +813,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
&& dir == DMA_FROM_DEVICE)
dir = DMA_BIDIRECTIONAL;
- dma_addr = dma_map_page(dma_dev, sg->page, 0,
+ dma_addr = dma_map_page(dma_dev, sg_page(sg), 0,
PAGE_SIZE, dir);
if (direction == DMA_TO_DEVICE)
t->tx_dma = dma_addr + sg->offset;
@@ -822,7 +822,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
}
/* allow pio too; we don't allow highmem */
- kmap_addr = kmap(sg->page);
+ kmap_addr = kmap(sg_page(sg));
if (direction == DMA_TO_DEVICE)
t->tx_buf = kmap_addr + sg->offset;
else
@@ -855,8 +855,8 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
/* discard mappings */
if (direction == DMA_FROM_DEVICE)
- flush_kernel_dcache_page(sg->page);
- kunmap(sg->page);
+ flush_kernel_dcache_page(sg_page(sg));
+ kunmap(sg_page(sg));
if (dma_dev)
dma_unmap_page(dma_dev, dma_addr, PAGE_SIZE, dir);
diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c
index 60a67df..649f647 100644
--- a/drivers/mmc/host/omap.c
+++ b/drivers/mmc/host/omap.c
@@ -24,10 +24,10 @@
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
#include <linux/clk.h>
+#include <linux/scatterlist.h>
#include <asm/io.h>
#include <asm/irq.h>
-#include <asm/scatterlist.h>
#include <asm/mach-types.h>
#include <asm/arch/board.h>
@@ -383,7 +383,7 @@ mmc_omap_sg_to_buf(struct mmc_omap_host *host)
sg = host->data->sg + host->sg_idx;
host->buffer_bytes_left = sg->length;
- host->buffer = page_address(sg->page) + sg->offset;
+ host->buffer = page_address(sg_page(sg)) + sg->offset;
if (host->buffer_bytes_left > host->total_bytes_left)
host->buffer_bytes_left = host->total_bytes_left;
}
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index b397121..b6500e9 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -231,7 +231,7 @@ static void sdhci_deactivate_led(struct sdhci_host *host)
static inline char* sdhci_sg_to_buffer(struct sdhci_host* host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline int sdhci_next_sg(struct sdhci_host* host)
diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c
index 9b90479..c11a3d2 100644
--- a/drivers/mmc/host/tifm_sd.c
+++ b/drivers/mmc/host/tifm_sd.c
@@ -192,7 +192,7 @@ static void tifm_sd_transfer_data(struct tifm_sd *host)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
@@ -241,18 +241,18 @@ static void tifm_sd_bounce_block(struct tifm_sd *host, struct mmc_data *r_data)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
p_cnt = min(p_cnt, t_size);
if (r_data->flags & MMC_DATA_WRITE)
- tifm_sd_copy_page(host->bounce_buf.page,
+ tifm_sd_copy_page(sg_page(&host->bounce_buf),
r_data->blksz - t_size,
pg, p_off, p_cnt);
else if (r_data->flags & MMC_DATA_READ)
- tifm_sd_copy_page(pg, p_off, host->bounce_buf.page,
+ tifm_sd_copy_page(pg, p_off, sg_page(&host->bounce_buf),
r_data->blksz - t_size, p_cnt);
t_size -= p_cnt;
diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c
index 80db11c..971ca0f 100644
--- a/drivers/mmc/host/wbsd.c
+++ b/drivers/mmc/host/wbsd.c
@@ -269,7 +269,7 @@ static inline int wbsd_next_sg(struct wbsd_host *host)
static inline char *wbsd_sg_to_buffer(struct wbsd_host *host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
@@ -283,7 +283,7 @@ static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(dmabuf, sgbuf, sg[i].length);
dmabuf += sg[i].length;
}
@@ -300,7 +300,7 @@ static inline void wbsd_dma_to_sg(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(sgbuf, dmabuf, sg[i].length);
dmabuf += sg[i].length;
}
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 4b3c109..887633b 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -60,7 +60,7 @@ static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chu
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -70,7 +70,7 @@ static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *
for (i = 0; i < chunk->npages; ++i)
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
@@ -95,10 +95,13 @@ void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent)
static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -145,6 +148,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MLX4_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -334,7 +338,7 @@ void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_han
* been assigned to.
*/
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
diff --git a/drivers/net/ppp_mppe.c b/drivers/net/ppp_mppe.c
index c0b6d19..bcb0885 100644
--- a/drivers/net/ppp_mppe.c
+++ b/drivers/net/ppp_mppe.c
@@ -55,7 +55,7 @@
#include <linux/mm.h>
#include <linux/ppp_defs.h>
#include <linux/ppp-comp.h>
-#include <asm/scatterlist.h>
+#include <linux/scatterlist.h>
#include "ppp_mppe.h"
@@ -68,9 +68,7 @@ MODULE_VERSION("1.0.2");
static unsigned int
setup_sg(struct scatterlist *sg, const void *address, unsigned int length)
{
- sg[0].page = virt_to_page(address);
- sg[0].offset = offset_in_page(address);
- sg[0].length = length;
+ sg_init_one(sg, address, length);
return length;
}
diff --git a/drivers/scsi/3w-9xxx.c b/drivers/scsi/3w-9xxx.c
index fb14014..afb262b 100644
--- a/drivers/scsi/3w-9xxx.c
+++ b/drivers/scsi/3w-9xxx.c
@@ -1840,7 +1840,7 @@ static int twa_scsiop_execute_scsi(TW_Device_Extension *tw_dev, int request_id,
(scsi_bufflen(srb) < TW_MIN_SGL_LENGTH)) {
if (srb->sc_data_direction == DMA_TO_DEVICE || srb->sc_data_direction == DMA_BIDIRECTIONAL) {
struct scatterlist *sg = scsi_sglist(srb);
- char *buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ char *buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(tw_dev->generic_buffer_virt[request_id], buf, sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
}
@@ -1919,7 +1919,7 @@ static void twa_scsiop_execute_scsi_complete(TW_Device_Extension *tw_dev, int re
char *buf;
unsigned long flags = 0;
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buf, tw_dev->generic_buffer_virt[request_id], sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/3w-xxxx.c b/drivers/scsi/3w-xxxx.c
index a64153b..59716eb 100644
--- a/drivers/scsi/3w-xxxx.c
+++ b/drivers/scsi/3w-xxxx.c
@@ -1469,7 +1469,7 @@ static void tw_transfer_internal(TW_Device_Extension *tw_dev, int request_id,
struct scatterlist *sg = scsi_sglist(cmd);
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len);
memcpy(buf, data, transfer_len);
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 988f0bc..7a76b4b 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -298,7 +298,7 @@ static __inline__ void initialize_SCp(Scsi_Cmnd * cmd)
if (cmd->use_sg) {
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.buffers_residual = cmd->use_sg - 1;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
} else {
@@ -2143,7 +2143,7 @@ static void NCR5380_information_transfer(struct Scsi_Host *instance) {
++cmd->SCp.buffer;
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
}
diff --git a/drivers/scsi/NCR53c406a.c b/drivers/scsi/NCR53c406a.c
index 3168a17..db1a8bc 100644
--- a/drivers/scsi/NCR53c406a.c
+++ b/drivers/scsi/NCR53c406a.c
@@ -875,7 +875,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_write(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_write(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
@@ -897,7 +897,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_read(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_read(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 80e448d..a77ab8d 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -356,7 +356,7 @@ static void aac_internal_transfer(struct scsi_cmnd *scsicmd, void *data, unsigne
int transfer_len;
struct scatterlist *sg = scsi_sglist(scsicmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len + offset);
transfer_len -= offset;
diff --git a/drivers/scsi/aha1542.c b/drivers/scsi/aha1542.c
index 961a188..50da0a3 100644
--- a/drivers/scsi/aha1542.c
+++ b/drivers/scsi/aha1542.c
@@ -49,7 +49,7 @@
#include "aha1542.h"
#define SCSI_BUF_PA(address) isa_virt_to_bus(address)
-#define SCSI_SG_PA(sgent) (isa_page_to_bus((sgent)->page) + (sgent)->offset)
+#define SCSI_SG_PA(sgent) (isa_page_to_bus(sg_page((sgent))) + (sgent)->offset)
static void BAD_DMA(void *address, unsigned int length)
{
@@ -67,7 +67,7 @@ static void BAD_SG_DMA(Scsi_Cmnd * SCpnt,
{
printk(KERN_CRIT "sgpnt[%d:%d] page %p/0x%llx length %u\n",
badseg, nseg,
- page_address(sgp->page) + sgp->offset,
+ page_address(sg_page(sgp)) + sgp->offset,
(unsigned long long)SCSI_SG_PA(sgp),
sgp->length);
@@ -712,7 +712,7 @@ static int aha1542_queuecommand(Scsi_Cmnd * SCpnt, void (*done) (Scsi_Cmnd *))
printk(KERN_CRIT "Bad segment list supplied to aha1542.c (%d, %d)\n", SCpnt->use_sg, i);
scsi_for_each_sg(SCpnt, sg, SCpnt->use_sg, i) {
printk(KERN_CRIT "%d: %p %d\n", i,
- (page_address(sg->page) +
+ (page_address(sg_page(sg)) +
sg->offset), sg->length);
};
printk(KERN_CRIT "cptr %x: ", (unsigned int) cptr);
diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c
index f817775..f7a2528 100644
--- a/drivers/scsi/arcmsr/arcmsr_hba.c
+++ b/drivers/scsi/arcmsr/arcmsr_hba.c
@@ -1343,7 +1343,7 @@ static int arcmsr_iop_message_xfer(struct AdapterControlBlock *acb, \
/* 4 bytes: Areca io control code */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (scsi_sg_count(cmd) > 1) {
retvalue = ARCMSR_MESSAGE_FAIL;
goto message_out;
@@ -1593,7 +1593,7 @@ static void arcmsr_handle_virtual_command(struct AdapterControlBlock *acb,
strncpy(&inqdata[32], "R001", 4); /* Product Revision */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buffer, inqdata, sizeof(inqdata));
sg = scsi_sglist(cmd);
diff --git a/drivers/scsi/fdomain.c b/drivers/scsi/fdomain.c
index 5d282e6..dc4e463 100644
--- a/drivers/scsi/fdomain.c
+++ b/drivers/scsi/fdomain.c
@@ -1321,7 +1321,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
if (current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
} else
break;
@@ -1354,7 +1354,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
&& current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
}
}
@@ -1439,7 +1439,7 @@ static int fdomain_16x0_queue(struct scsi_cmnd *SCpnt,
if (scsi_sg_count(current_SC)) {
current_SC->SCp.buffer = scsi_sglist(current_SC);
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page)
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer))
+ current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
current_SC->SCp.buffers_residual = scsi_sg_count(current_SC) - 1;
diff --git a/drivers/scsi/gdth.c b/drivers/scsi/gdth.c
index 3ac080e..ac6c57a 100644
--- a/drivers/scsi/gdth.c
+++ b/drivers/scsi/gdth.c
@@ -2374,13 +2374,13 @@ static void gdth_copy_internal_data(gdth_ha_str *ha, Scsi_Cmnd *scp,
if (cpsum+cpnow > cpcount)
cpnow = cpcount - cpsum;
cpsum += cpnow;
- if (!sl->page) {
+ if (!sg_page(sl)) {
printk("GDT-HA %d: invalid sc/gt element in gdth_copy_internal_data()\n",
ha->hanum);
return;
}
local_irq_save(flags);
- address = kmap_atomic(sl->page, KM_BIO_SRC_IRQ) + sl->offset;
+ address = kmap_atomic(sg_page(sl), KM_BIO_SRC_IRQ) + sl->offset;
if (to_buffer)
memcpy(buffer, address, cpnow);
else
diff --git a/drivers/scsi/ibmmca.c b/drivers/scsi/ibmmca.c
index 714e627..db004a4 100644
--- a/drivers/scsi/ibmmca.c
+++ b/drivers/scsi/ibmmca.c
@@ -1828,7 +1828,7 @@ static int ibmmca_queuecommand(Scsi_Cmnd * cmd, void (*done) (Scsi_Cmnd *))
BUG_ON(scsi_sg_count(cmd) > 16);
scsi_for_each_sg(cmd, sg, scsi_sg_count(cmd), i) {
- ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg->page) + sg->offset);
+ ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg_page(sg)) + sg->offset);
ld(shpnt)[ldn].sge[i].byte_length = sg->length;
}
scb->enable |= IM_POINTER_TO_LIST;
diff --git a/drivers/scsi/ide-scsi.c b/drivers/scsi/ide-scsi.c
index fa7ba64..2c7dd8b 100644
--- a/drivers/scsi/ide-scsi.c
+++ b/drivers/scsi/ide-scsi.c
@@ -175,18 +175,18 @@ static void idescsi_input_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsigne
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
}
@@ -212,18 +212,18 @@ static void idescsi_output_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsign
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
}
diff --git a/drivers/scsi/imm.c b/drivers/scsi/imm.c
index 74cdc1f..da9b0ba 100644
--- a/drivers/scsi/imm.c
+++ b/drivers/scsi/imm.c
@@ -706,7 +706,7 @@ static int imm_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
/*
@@ -845,7 +845,7 @@ static int imm_engine(imm_struct *dev, struct scsi_cmnd *cmd)
(struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..dc98a85 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2872,6 +2872,7 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
}
scatterlist = sglist->scatterlist;
+ sg_init_table(scatterlist, num_elem);
sglist->order = order;
sglist->num_sg = num_elem;
@@ -2884,12 +2885,12 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
/* Free up what we already allocated */
for (j = i - 1; j >= 0; j--)
- __free_pages(scatterlist[j].page, order);
+ __free_pages(sg_page(&scatterlist[j]), order);
kfree(sglist);
return NULL;
}
- scatterlist[i].page = page;
+ sg_set_page(&scatterlist[i], page);
}
return sglist;
@@ -2910,7 +2911,7 @@ static void ipr_free_ucode_buffer(struct ipr_sglist *sglist)
int i;
for (i = 0; i < sglist->num_sg; i++)
- __free_pages(sglist->scatterlist[i].page, sglist->order);
+ __free_pages(sg_page(&sglist->scatterlist[i]), sglist->order);
kfree(sglist);
}
@@ -2940,9 +2941,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
scatterlist = sglist->scatterlist;
for (i = 0; i < (len / bsize_elem); i++, buffer += bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = bsize_elem;
@@ -2953,9 +2956,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
}
if (len % bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, len % bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = len % bsize_elem;
}
diff --git a/drivers/scsi/ips.c b/drivers/scsi/ips.c
index edaac27..5c5a9b2 100644
--- a/drivers/scsi/ips.c
+++ b/drivers/scsi/ips.c
@@ -1515,7 +1515,7 @@ static int ips_is_passthru(struct scsi_cmnd *SC)
/* kmap_atomic() ensures addressability of the user buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (buffer && buffer[0] == 'C' && buffer[1] == 'O' &&
buffer[2] == 'P' && buffer[3] == 'P') {
kunmap_atomic(buffer - sg->offset, KM_IRQ0);
@@ -3523,7 +3523,7 @@ ips_scmd_buf_write(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(buffer, &cdata[xfer_cnt], min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
@@ -3556,7 +3556,7 @@ ips_scmd_buf_read(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(&cdata[xfer_cnt], buffer, min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index a21455d..6ce4109 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -70,9 +70,7 @@ module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO);
static inline void
iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
{
- ibuf->sg.page = virt_to_page(vbuf);
- ibuf->sg.offset = offset_in_page(vbuf);
- ibuf->sg.length = size;
+ sg_init_one(&ibuf->sg, vbuf, size);
ibuf->sent = 0;
ibuf->use_sendmsg = 1;
}
@@ -80,13 +78,14 @@ iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
static inline void
iscsi_buf_init_sg(struct iscsi_buf *ibuf, struct scatterlist *sg)
{
- ibuf->sg.page = sg->page;
+ sg_init_table(&ibuf->sg, 1);
+ sg_set_page(&ibuf->sg, sg_page(sg));
ibuf->sg.offset = sg->offset;
ibuf->sg.length = sg->length;
/*
* Fastpath: sg element fits into single page
*/
- if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg->page))
+ if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg_page(sg)))
ibuf->use_sendmsg = 0;
else
ibuf->use_sendmsg = 1;
@@ -716,7 +715,7 @@ static int iscsi_scsi_data_in(struct iscsi_conn *conn)
for (i = tcp_ctask->sg_count; i < scsi_sg_count(sc); i++) {
char *dest;
- dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0);
+ dest = kmap_atomic(sg_page(&sg[i]), KM_SOFTIRQ0);
rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset,
sg[i].length, offset);
kunmap_atomic(dest, KM_SOFTIRQ0);
@@ -1103,9 +1102,9 @@ iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags)
* slab case.
*/
if (buf->use_sendmsg)
- res = sock_no_sendpage(sk, buf->sg.page, offset, size, flags);
+ res = sock_no_sendpage(sk, sg_page(&buf->sg), offset, size, flags);
else
- res = tcp_conn->sendpage(sk, buf->sg.page, offset, size, flags);
+ res = tcp_conn->sendpage(sk, sg_page(&buf->sg), offset, size, flags);
if (res >= 0) {
conn->txdata_octets += res;
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 10d1aff..0a848e8 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -658,7 +658,7 @@ mega_build_cmd(adapter_t *adapter, Scsi_Cmnd *cmd, int *busy)
struct scatterlist *sg;
sg = scsi_sglist(cmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memset(buf, 0, cmd->cmnd[4]);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
@@ -1542,9 +1542,9 @@ mega_cmd_done(adapter_t *adapter, u8 completed[], int nstatus, int status)
if( cmd->cmnd[0] == INQUIRY && !islogical ) {
sgl = scsi_sglist(cmd);
- if( sgl->page ) {
+ if( sg_page(sgl) ) {
c = *(unsigned char *)
- page_address((&sgl[0])->page) +
+ page_address((sg_page(&sgl[0]))) +
(&sgl[0])->offset;
} else {
printk(KERN_WARNING
diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c
index e4e4c6a..376263c 100644
--- a/drivers/scsi/megaraid/megaraid_mbox.c
+++ b/drivers/scsi/megaraid/megaraid_mbox.c
@@ -1584,9 +1584,9 @@ megaraid_mbox_build_cmd(adapter_t *adapter, struct scsi_cmnd *scp, int *busy)
caddr_t vaddr;
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
vaddr = (caddr_t)
- (page_address((&sgl[0])->page)
+ (page_address(sg_page(&sgl[0]))
+ (&sgl[0])->offset);
memset(vaddr, 0, scp->cmnd[4]);
@@ -2328,9 +2328,9 @@ megaraid_mbox_dpc(unsigned long devp)
&& IS_RAID_CH(raid_dev, scb->dev_channel)) {
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
c = *(unsigned char *)
- (page_address((&sgl[0])->page) +
+ (page_address(sg_page(&sgl[0])) +
(&sgl[0])->offset);
} else {
con_log(CL_ANN, (KERN_WARNING
diff --git a/drivers/scsi/osst.c b/drivers/scsi/osst.c
index 331b789..1c5c4b6 100644
--- a/drivers/scsi/osst.c
+++ b/drivers/scsi/osst.c
@@ -542,7 +542,7 @@ static int osst_verify_frame(struct osst_tape * STp, int frame_seq_number, int q
if (STp->raw) {
if (STp->buffer->syscall_result) {
for (i=0; i < STp->buffer->sg_segs; i++)
- memset(page_address(STp->buffer->sg[i].page),
+ memset(page_address(sg_page(&STp->buffer->sg[i])),
0, STp->buffer->sg[i].length);
strcpy(STp->buffer->b_data, "READ ERROR ON FRAME");
} else
@@ -4437,7 +4437,7 @@ static int os_scsi_tape_open(struct inode * inode, struct file * filp)
for (i = 0, b_size = 0;
(i < STp->buffer->sg_segs) && ((b_size + STp->buffer->sg[i].length) <= OS_DATA_SIZE);
b_size += STp->buffer->sg[i++].length);
- STp->buffer->aux = (os_aux_t *) (page_address(STp->buffer->sg[i].page) + OS_DATA_SIZE - b_size);
+ STp->buffer->aux = (os_aux_t *) (page_address(sg_page(&STp->buffer->sg[i])) + OS_DATA_SIZE - b_size);
#if DEBUG
printk(OSST_DEB_MSG "%s:D: b_data points to %p in segment 0 at %p\n", name,
STp->buffer->b_data, page_address(STp->buffer->sg[0].page));
@@ -5252,25 +5252,26 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
/* Try to allocate the first segment up to OS_DATA_SIZE and the others
big enough to reach the goal (code assumes no segments in place) */
for (b_size = OS_DATA_SIZE, order = OSST_FIRST_ORDER; b_size >= PAGE_SIZE; order--, b_size /= 2) {
- STbuffer->sg[0].page = alloc_pages(priority, order);
+ struct page *page = alloc_pages(priority, order);
+
STbuffer->sg[0].offset = 0;
- if (STbuffer->sg[0].page != NULL) {
+ if (page != NULL) {
+ sg_set_page(&STbuffer->sg[0], page);
STbuffer->sg[0].length = b_size;
- STbuffer->b_data = page_address(STbuffer->sg[0].page);
+ STbuffer->b_data = page_address(page);
break;
}
}
- if (STbuffer->sg[0].page == NULL) {
+ if (sg_page(&STbuffer->sg[0]) == NULL) {
printk(KERN_NOTICE "osst :I: Can't allocate tape buffer main segment.\n");
return 0;
}
/* Got initial segment of 'bsize,order', continue with same size if possible, except for AUX */
for (segs=STbuffer->sg_segs=1, got=b_size;
segs < max_segs && got < OS_FRAME_SIZE; ) {
- STbuffer->sg[segs].page =
- alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
+ struct page *page = alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
STbuffer->sg[segs].offset = 0;
- if (STbuffer->sg[segs].page == NULL) {
+ if (page == NULL) {
if (OS_FRAME_SIZE - got <= (max_segs - segs) * b_size / 2 && order) {
b_size /= 2; /* Large enough for the rest of the buffers */
order--;
@@ -5284,6 +5285,7 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
normalize_buffer(STbuffer);
return 0;
}
+ sg_set_page(&STbuffer->sg[segs], page);
STbuffer->sg[segs].length = (OS_FRAME_SIZE - got <= PAGE_SIZE / 2) ? (OS_FRAME_SIZE - got) : b_size;
got += STbuffer->sg[segs].length;
STbuffer->buffer_size = got;
@@ -5316,7 +5318,7 @@ static void normalize_buffer(struct osst_buffer *STbuffer)
b_size < STbuffer->sg[i].length;
b_size *= 2, order++);
- __free_pages(STbuffer->sg[i].page, order);
+ __free_pages(sg_page(&STbuffer->sg[i]), order);
STbuffer->buffer_size -= STbuffer->sg[i].length;
}
#if DEBUG
@@ -5344,7 +5346,7 @@ static int append_to_buffer(const char __user *ubp, struct osst_buffer *st_bp, i
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->sg[i].page) + offset, ubp, cnt);
+ res = copy_from_user(page_address(sg_page(&st_bp->sg[i])) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5377,7 +5379,7 @@ static int from_buffer(struct osst_buffer *st_bp, char __user *ubp, int do_count
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->sg[i].page) + offset, cnt);
+ res = copy_to_user(ubp, page_address(sg_page(&st_bp->sg[i])) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5410,7 +5412,7 @@ static int osst_zero_buffer_tail(struct osst_buffer *st_bp)
i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count ;
- memset(page_address(st_bp->sg[i].page) + offset, 0, cnt);
+ memset(page_address(sg_page(&st_bp->sg[i])) + offset, 0, cnt);
do_count -= cnt;
offset = 0;
}
@@ -5430,7 +5432,7 @@ static int osst_copy_to_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(page_address(st_bp->sg[i].page), ptr, cnt);
+ memcpy(page_address(sg_page(&st_bp->sg[i])), ptr, cnt);
do_count -= cnt;
ptr += cnt;
}
@@ -5451,7 +5453,7 @@ static int osst_copy_from_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(ptr, page_address(st_bp->sg[i].page), cnt);
+ memcpy(ptr, page_address(sg_page(&st_bp->sg[i])), cnt);
do_count -= cnt;
ptr += cnt;
}
diff --git a/drivers/scsi/pcmcia/sym53c500_cs.c b/drivers/scsi/pcmcia/sym53c500_cs.c
index 190e2a7..26c41b0 100644
--- a/drivers/scsi/pcmcia/sym53c500_cs.c
+++ b/drivers/scsi/pcmcia/sym53c500_cs.c
@@ -443,7 +443,7 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_write(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
+ page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0(port_base);
@@ -463,8 +463,8 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_read(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
- sg->length);
+ page_address(sg_page(sg)) + sg->offset,
+ sg->length);
}
REG0(port_base);
}
diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c
index 67b6d76..07f31e5 100644
--- a/drivers/scsi/ppa.c
+++ b/drivers/scsi/ppa.c
@@ -609,7 +609,7 @@ static int ppa_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
}
}
@@ -756,7 +756,7 @@ static int ppa_engine(ppa_struct *dev, struct scsi_cmnd *cmd)
/* if many buffers are available, start filling the first */
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page) +
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/qlogicfas408.c b/drivers/scsi/qlogicfas408.c
index 2bfbf26..feff3f7 100644
--- a/drivers/scsi/qlogicfas408.c
+++ b/drivers/scsi/qlogicfas408.c
@@ -317,7 +317,7 @@ static unsigned int ql_pcmd(struct scsi_cmnd *cmd)
return ((priv->qabort == 1 ?
DID_ABORT : DID_RESET) << 16);
}
- buf = page_address(sg->page) + sg->offset;
+ buf = page_address(sg_page(sg)) + sg->offset;
if (ql_pdma(priv, phase, buf, sg->length))
break;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..61fdaf0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,7 +764,7 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
- memset(sgl, 0, sizeof(*sgl) * sgp->size);
+ sg_init_table(sgl, sgp->size);
/*
* first loop through, set initial index and return value
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 73c44cb..ce69b9e 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -3797,7 +3797,7 @@ static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
sg = &(STbp->sg[0]);
frp = STbp->frp;
for (i=count=0; count < length; i++) {
- sg[i].page = frp[i].page;
+ sg_set_page(&sg[i], frp[i].page);
if (length - count > frp[i].length)
sg[i].length = frp[i].length;
else
@@ -4446,14 +4446,14 @@ static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pa
}
/* Populate the scatter/gather list */
- sgl[0].page = pages[0];
+ sg_set_page(&sgl[0], pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
+ sg_set_page(&sgl[i], pages[i]);;
sgl[i].offset = 0;
- sgl[i].page = pages[i];
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -4483,7 +4483,7 @@ static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_p
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
diff --git a/drivers/scsi/tmscsim.c b/drivers/scsi/tmscsim.c
index 5c72ca3..4419304 100644
--- a/drivers/scsi/tmscsim.c
+++ b/drivers/scsi/tmscsim.c
@@ -430,10 +430,7 @@ static __inline__ void dc390_Going_remove (struct dc390_dcb* pDCB, struct dc390_
static struct scatterlist* dc390_sg_build_single(struct scatterlist *sg, void *addr, unsigned int length)
{
- memset(sg, 0, sizeof(struct scatterlist));
- sg->page = virt_to_page(addr);
- sg->length = length;
- sg->offset = (unsigned long)addr & ~PAGE_MASK;
+ sg_init_one(sg, addr, length);
return sg;
}
diff --git a/drivers/scsi/ultrastor.c b/drivers/scsi/ultrastor.c
index ea72bbe..6d1f0ed 100644
--- a/drivers/scsi/ultrastor.c
+++ b/drivers/scsi/ultrastor.c
@@ -681,7 +681,7 @@ static inline void build_sg_list(struct mscp *mscp, struct scsi_cmnd *SCpnt)
max = scsi_sg_count(SCpnt);
scsi_for_each_sg(SCpnt, sg, max, i) {
- mscp->sglist[i].address = isa_page_to_bus(sg->page) + sg->offset;
+ mscp->sglist[i].address = isa_page_to_bus(sg_page(sg)) + sg->offset;
mscp->sglist[i].num_bytes = sg->length;
transfer_length += sg->length;
}
diff --git a/drivers/scsi/wd7000.c b/drivers/scsi/wd7000.c
index 255c611..03cd44f 100644
--- a/drivers/scsi/wd7000.c
+++ b/drivers/scsi/wd7000.c
@@ -1123,7 +1123,7 @@ static int wd7000_queuecommand(struct scsi_cmnd *SCpnt,
any2scsi(scb->maxlen, nseg * sizeof(Sgb));
scsi_for_each_sg(SCpnt, sg, nseg, i) {
- any2scsi(sgb[i].ptr, isa_page_to_bus(sg->page) + sg->offset);
+ any2scsi(sgb[i].ptr, isa_page_to_bus(sg_page(sg)) + sg->offset);
any2scsi(sgb[i].len, sg->length);
}
} else {
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..3577453 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -438,12 +438,12 @@ int usb_sg_init (
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/image/microtek.c b/drivers/usb/image/microtek.c
index e7d982a..4866baf 100644
--- a/drivers/usb/image/microtek.c
+++ b/drivers/usb/image/microtek.c
@@ -519,7 +519,7 @@ static void mts_do_sg (struct urb* transfer)
context->fragment++;
mts_int_submit_urb(transfer,
context->data_pipe,
- page_address(sg[context->fragment].page) +
+ page_address(sg_page(&sg[context->fragment])) +
sg[context->fragment].offset,
sg[context->fragment].length,
context->fragment + 1 == scsi_sg_count(context->srb) ?
@@ -557,7 +557,7 @@ mts_build_transfer_context(struct scsi_cmnd *srb, struct mts_desc* desc)
return;
} else {
sg = scsi_sglist(srb);
- desc->context.data = page_address(sg[0].page) + sg[0].offset;
+ desc->context.data = page_address(sg_page(&sg[0])) + sg[0].offset;
desc->context.data_length = sg[0].length;
}
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index e901d31..ca8cb34 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -360,9 +360,10 @@ static void free_sglist (struct scatterlist *sg, int nents)
if (!sg)
return;
for (i = 0; i < nents; i++) {
- if (!sg [i].page)
+ struct page *page = sg_page(&sg[i]);
+ if (!page)
continue;
- kfree (page_address (sg [i].page) + sg [i].offset);
+ kfree (page_address (page) + sg [i].offset);
}
kfree (sg);
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index 1ae90ef..0a9882e 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -283,7 +283,7 @@ int virt_to_scatterlist(const void *addr, int size, struct scatterlist *sg,
pg = virt_to_page(addr);
offset = offset_in_page(addr);
if (sg) {
- sg[i].page = pg;
+ sg_set_page(&sg[i], pg);
sg[i].offset = offset;
}
remainder_of_page = PAGE_CACHE_SIZE - offset;
@@ -713,10 +713,13 @@ ecryptfs_encrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return encrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
@@ -742,10 +745,13 @@ ecryptfs_decrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return decrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index 89d9710..263fed8 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -1040,6 +1040,9 @@ decrypt_passphrase_encrypted_session_key(struct ecryptfs_auth_tok *auth_tok,
};
int rc = 0;
+ sg_init_table(&dst_sg, 1);
+ sg_init_table(&src_sg, 1);
+
if (unlikely(ecryptfs_verbosity > 0)) {
ecryptfs_printk(
KERN_DEBUG, "Session key encryption key (size [%d]):\n",
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index ebd03cc..6f03918 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -88,7 +88,7 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
{
struct xdr_netobj cksum;
struct hash_desc desc;
- struct scatterlist sg[1];
+ struct scatterlist sg;
__be32 status = nfserr_resource;
dprintk("NFSD: nfs4_make_rec_clidname for %.*s\n",
@@ -102,11 +102,9 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
if (cksum.data == NULL)
goto out;
- sg[0].page = virt_to_page(clname->data);
- sg[0].offset = offset_in_page(clname->data);
- sg[0].length = clname->len;
+ sg_init_one(&sg, clname->data, clname->len);
- if (crypto_hash_digest(&desc, sg, sg->length, cksum.data))
+ if (crypto_hash_digest(&desc, &sg, sg.length, cksum.data))
goto out;
md5_to_hex(dname, cksum.data);
diff --git a/include/asm-alpha/scatterlist.h b/include/asm-alpha/scatterlist.h
index 9173654..b764706 100644
--- a/include/asm-alpha/scatterlist.h
+++ b/include/asm-alpha/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-arm/scatterlist.h b/include/asm-arm/scatterlist.h
index de2f65e..ab1d85d 100644
--- a/include/asm-arm/scatterlist.h
+++ b/include/asm-arm/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page; /* buffer page */
+ unsigned long page_link;
unsigned int offset; /* buffer offset */
dma_addr_t dma_address; /* dma address */
unsigned int length; /* length */
diff --git a/include/asm-avr32/scatterlist.h b/include/asm-avr32/scatterlist.h
index c6d5ce3..1356f29 100644
--- a/include/asm-avr32/scatterlist.h
+++ b/include/asm-avr32/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-blackfin/scatterlist.h b/include/asm-blackfin/scatterlist.h
index 60e07b9..384af54 100644
--- a/include/asm-blackfin/scatterlist.h
+++ b/include/asm-blackfin/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/mm.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-cris/scatterlist.h b/include/asm-cris/scatterlist.h
index 4bdc44c..5a8a834 100644
--- a/include/asm-cris/scatterlist.h
+++ b/include/asm-cris/scatterlist.h
@@ -6,7 +6,7 @@ struct scatterlist {
unsigned int length;
/* The following is i386 highmem junk - not used by us */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
};
diff --git a/include/asm-frv/scatterlist.h b/include/asm-frv/scatterlist.h
index 8e827fa..53dade7 100644
--- a/include/asm-frv/scatterlist.h
+++ b/include/asm-frv/scatterlist.h
@@ -22,7 +22,7 @@
* and that's it. There's no excuse for not highmem enabling YOUR driver. /jens
*/
struct scatterlist {
- struct page *page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset; /* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-h8300/scatterlist.h b/include/asm-h8300/scatterlist.h
index 985fdf5..7e41983 100644
--- a/include/asm-h8300/scatterlist.h
+++ b/include/asm-h8300/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-ia64/scatterlist.h b/include/asm-ia64/scatterlist.h
index 7d5234d..2f76ce3 100644
--- a/include/asm-ia64/scatterlist.h
+++ b/include/asm-ia64/scatterlist.h
@@ -9,7 +9,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length; /* buffer length */
diff --git a/include/asm-m32r/scatterlist.h b/include/asm-m32r/scatterlist.h
index 352415f..33b4b4d 100644
--- a/include/asm-m32r/scatterlist.h
+++ b/include/asm-m32r/scatterlist.h
@@ -6,7 +6,7 @@
struct scatterlist {
char * address; /* Location data is to be transferred to, NULL for
* highmem page */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-m68k/scatterlist.h b/include/asm-m68k/scatterlist.h
index 24887a2..e06bb89 100644
--- a/include/asm-m68k/scatterlist.h
+++ b/include/asm-m68k/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-m68knommu/scatterlist.h b/include/asm-m68knommu/scatterlist.h
index 4da79d3..28bed41 100644
--- a/include/asm-m68knommu/scatterlist.h
+++ b/include/asm-m68knommu/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-mips/scatterlist.h b/include/asm-mips/scatterlist.h
index 7af104c..787797c 100644
--- a/include/asm-mips/scatterlist.h
+++ b/include/asm-mips/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-parisc/scatterlist.h b/include/asm-parisc/scatterlist.h
index e7211c7..26da914 100644
--- a/include/asm-parisc/scatterlist.h
+++ b/include/asm-parisc/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-powerpc/scatterlist.h b/include/asm-powerpc/scatterlist.h
index b075f61..b9f1dbc 100644
--- a/include/asm-powerpc/scatterlist.h
+++ b/include/asm-powerpc/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/dma.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-s390/scatterlist.h b/include/asm-s390/scatterlist.h
index a43b3af..eb39486 100644
--- a/include/asm-s390/scatterlist.h
+++ b/include/asm-s390/scatterlist.h
@@ -2,7 +2,7 @@
#define _ASMS390_SCATTERLIST_H
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
};
diff --git a/include/asm-sh/scatterlist.h b/include/asm-sh/scatterlist.h
index b9ae53c..bc7c809 100644
--- a/include/asm-sh/scatterlist.h
+++ b/include/asm-sh/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sh64/scatterlist.h b/include/asm-sh64/scatterlist.h
index 1c723f2..0afd856 100644
--- a/include/asm-sh64/scatterlist.h
+++ b/include/asm-sh64/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sparc/scatterlist.h b/include/asm-sparc/scatterlist.h
index 4055af9..45b16f1 100644
--- a/include/asm-sparc/scatterlist.h
+++ b/include/asm-sparc/scatterlist.h
@@ -5,7 +5,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-v850/scatterlist.h b/include/asm-v850/scatterlist.h
index 56f4029..db91feb 100644
--- a/include/asm-v850/scatterlist.h
+++ b/include/asm-v850/scatterlist.h
@@ -17,7 +17,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned offset;
dma_addr_t dma_address;
unsigned length;
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/asm-xtensa/scatterlist.h b/include/asm-xtensa/scatterlist.h
index ca337a2..3b8aba5 100644
--- a/include/asm-xtensa/scatterlist.h
+++ b/include/asm-xtensa/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..b4d6a64 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,29 +5,72 @@
#include <linux/mm.h>
#include <linux/string.h>
+/*
+ * Notes on SG table design.
+ *
+ * Architectures must provide an unsigned long page_link field in the
+ * scatterlist struct. We use that to place the page pointer AND encode
+ * information about the sg table as well. The two lower bits are reserved
+ * for this information.
+ *
+ * If bit 0 is set, then the page_link contains a pointer to the next sg
+ * table list. Otherwise the next entry is at sg + 1.
+ *
+ * If bit 1 is set, then this sg entry is the last element in a list.
+ *
+ * See sg_next().
+ *
+ */
+
+#define SG_MAGIC 0x87654321
+
+/**
+ * sg_set_page - Set sg entry to point at given page
+ * @sg: SG entry
+ * @page: The page
+ *
+ * Use this function to set an sg entry pointing at a page, never assign
+ * the page directly. We encode sg table information in the lower bits
+ * of the page pointer. See sg_page() for looking up the page belonging
+ * to an sg entry.
+ *
+ */
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
+/**
+ * sg_set_buf - Set sg entry to point at given data
+ * @sg: SG entry
+ * @buf: Data
+ * @buflen: Data length
+ *
+*/
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
-static inline void sg_init_one(struct scatterlist *sg, const void *buf,
- unsigned int buflen)
-{
- memset(sg, 0, sizeof(*sg));
- sg_set_buf(sg, buf, buflen);
-}
-
/*
* We overload the LSB of the page pointer to indicate whether it's
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+ ((struct scatterlist *) ((sg)->page_link & ~0x03))
/**
* sg_next - return the next scatterlist entry in a list
@@ -37,16 +80,18 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* of a chained scatterlist, it could jump to the start of a new
* scatterlist array.
*
- * Note that the caller must ensure that there are further entries after
- * the current entry, this function will NOT return NULL for an end-of-list.
- *
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
-
- if (unlikely(sg_is_chain(sg)))
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ sg = NULL;
+ else if (sg_is_chain(sg))
sg = sg_chain_ptr(sg);
+ else
+ sg++;
return sg;
}
@@ -83,6 +128,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+#endif
return ret;
}
@@ -101,7 +149,63 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
+}
+
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+/**
+ * sg_init_one - Initialize a single entry sg list
+ * @sg: SG entry
+ * @buf: Virtual address for IO
+ * @buflen: IO length
+ *
+ * Note: this should not be used on a single entry that is part of a larger
+ * table. Use sg_init_table() for that.
+ *
+ */
+static inline void sg_init_one(struct scatterlist *sg, const void *buf,
+ unsigned int buflen)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_mark_end(sg, 1);
+ sg_set_buf(sg, buf, buflen);
+}
+
+/**
+ * sg_init_table - Initialize SG table
+ * @sgl: The SG table
+ * @nents: Number of entries in table
+ *
+ * Note: If this is part of a chained sg table, sg_mark_end() should be
+ * used only on the last table part.
+ *
+ */
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
}
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..183f42c 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,16 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables.
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_tkip.c b/net/ieee80211/ieee80211_crypt_tkip.c
index 6cc54ee..34d4c77 100644
--- a/net/ieee80211/ieee80211_crypt_tkip.c
+++ b/net/ieee80211/ieee80211_crypt_tkip.c
@@ -390,9 +390,7 @@ static int ieee80211_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(tkey->tx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -485,9 +483,7 @@ static int ieee80211_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 12;
crypto_blkcipher_setkey(tkey->rx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4)) {
if (net_ratelimit()) {
printk(KERN_DEBUG ": TKIP: failed to decrypt "
@@ -539,11 +535,12 @@ static int michael_mic(struct crypto_hash *tfm_michael, u8 * key, u8 * hdr,
printk(KERN_WARNING "michael_mic: tfm_michael == NULL\n");
return -1;
}
- sg[0].page = virt_to_page(hdr);
+ sg_init_table(sg, 2);
+ sg_set_page(&sg[0], virt_to_page(hdr));
sg[0].offset = offset_in_page(hdr);
sg[0].length = 16;
- sg[1].page = virt_to_page(data);
+ sg_set_page(&sg[1], virt_to_page(data));
sg[1].offset = offset_in_page(data);
sg[1].length = data_len;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..0af6103 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,9 +170,7 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -212,9 +210,7 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
return -7;
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..cc806d6 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,9 +138,7 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
}
@@ -204,9 +202,7 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
crc = cpu_to_le32(~crc32_le(~0, data, data_len));
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 7818107..cbd64b2 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -726,7 +726,8 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
/* set up scatter list */
end = skb_tail_pointer(skb);
- sg.page = virt_to_page(auth);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(auth));
sg.offset = (unsigned long)(auth) % PAGE_SIZE;
sg.length = end - (unsigned char *)auth;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index f983a36..d5a9785 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1513,7 +1513,8 @@ static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep,
struct hash_desc desc;
/* Sign the message. */
- sg.page = virt_to_page(&cookie->c);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(&cookie->c));
sg.offset = (unsigned long)(&cookie->c) % PAGE_SIZE;
sg.length = bodysize;
keylen = SCTP_SECRET_SIZE;
@@ -1585,7 +1586,8 @@ struct sctp_association *sctp_unpack_cookie(
/* Check the signature. */
keylen = SCTP_SECRET_SIZE;
- sg.page = virt_to_page(bear_cookie);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(bear_cookie));
sg.offset = (unsigned long)(bear_cookie) % PAGE_SIZE;
sg.length = bodysize;
key = (char *)ep->secret_key[ep->current_key];
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
On Thu, Oct 18 2007, Mark Lord wrote:
> Jens Axboe wrote:
>> On Thu, Oct 18 2007, Mark Lord wrote:
>>> Jens Axboe wrote:
>>>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>>>> Mark Lord wrote:
>>>>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>>>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>>>>> but the top was cut off (isn't there a new config option or patch
>>>>>> to do double-columns or scrollback or something ???.
>>>>> Is this a sata_mv box? If so, could you try this patch?
>>>> If anything, that shrinks the size of the resulting request. Did this
>>>> patch make any difference to you?
>>> Not a sata_mv box, so no point here.
>> Can you try the big patch I just posted?
>
> I'll hunt for it and try it, but your earlier little patch already works
> fine.
I'll send it privately.
--
Jens Axboe
On Oct. 18, 2007, 15:32 +0200, Jens Axboe <[email protected]> wrote:
> static inline struct scatterlist *sg_next(struct scatterlist *sg)
> {
> - sg++;
> -
> - if (unlikely(sg_is_chain(sg)))
> +#ifdef CONFIG_DEBUG_SG
> + BUG_ON(sg->sg_magic != SG_MAGIC);
> +#endif
> + if (sg_is_last(sg))
> + sg = NULL;
> + else if (sg_is_chain(sg))
> sg = sg_chain_ptr(sg);
> + else
> + sg++;
>
Jens, again, please correct me if I'm wrong, but when sg points at the
entry right before a chain entry this implementation of sg_next will
return a pointer to the chain entry here, which I believe it must not.
> return sg;
> }
>
here's how I think sg_next should be implemented:
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ return NULL;
+
sg++;
if (unlikely(sg_is_chain(sg)))
sg = sg_chain_ptr(sg);
return sg;
}
Jens wrote:
> OK, I think that covers every arch out there. I haven't been able to
> compile any of them, but it's mostly search'n replace operations. I hope
> nothing is missing linux/scatterlist.h includes...
Patch fails on drivers/scsi/scsi_lib.c.
I replaced that part of the patch with this updated portion instead:
--- a/drivers/scsi/scsi_lib.c 2007-10-18 09:35:28.000000000 -0400
+++ b/drivers/scsi/scsi_lib.c 2007-10-18 09:46:47.000000000 -0400
@@ -295,7 +295,7 @@
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,6 +764,8 @@
if (unlikely(!sgl))
goto enomem;
+ sg_init_table(sgl, sgp->size);
+
/*
* first loop through, set initial index and return value
*/
@@ -779,6 +781,13 @@
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2351,7 +2360,7 @@
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
On Thu, Oct 18 2007, Benny Halevy wrote:
> On Oct. 18, 2007, 15:32 +0200, Jens Axboe <[email protected]> wrote:
> > static inline struct scatterlist *sg_next(struct scatterlist *sg)
> > {
> > - sg++;
> > -
> > - if (unlikely(sg_is_chain(sg)))
> > +#ifdef CONFIG_DEBUG_SG
> > + BUG_ON(sg->sg_magic != SG_MAGIC);
> > +#endif
> > + if (sg_is_last(sg))
> > + sg = NULL;
> > + else if (sg_is_chain(sg))
> > sg = sg_chain_ptr(sg);
> > + else
> > + sg++;
> >
>
> Jens, again, please correct me if I'm wrong, but when sg points at the
> entry right before a chain entry this implementation of sg_next will
> return a pointer to the chain entry here, which I believe it must not.
>
> > return sg;
> > }
> >
>
> here's how I think sg_next should be implemented:
>
> */
> static inline struct scatterlist *sg_next(struct scatterlist *sg)
> {
> +#ifdef CONFIG_DEBUG_SG
> + BUG_ON(sg->sg_magic != SG_MAGIC);
> +#endif
> + if (sg_is_last(sg))
> + return NULL;
> +
> sg++;
>
> if (unlikely(sg_is_chain(sg)))
> sg = sg_chain_ptr(sg);
>
> return sg;
> }
Yep, thanks for catching that!
--
Jens Axboe
On Thu, Oct 18 2007, Benny Halevy wrote:
> > return sg;
> > }
> > @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> > ret = sg;
> >
> > #endif
> > +#ifdef CONFIG_DEBUG_SG
> > + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
>
> can it also do BUG_ON(!sg_is_last(sg))?
That would make sense, definitely. I'll add that.
--
Jens Axboe
On Thu, Oct 18 2007, Mark Lord wrote:
> Jens wrote:
>> OK, I think that covers every arch out there. I haven't been able to
>> compile any of them, but it's mostly search'n replace operations. I hope
>> nothing is missing linux/scatterlist.h includes...
>
> Patch fails on drivers/scsi/scsi_lib.c.
>
> I replaced that part of the patch with this updated portion instead:
Hmm, what are you applying against? Must be a clean tree, throw away any
patches that you already applied in this thread.
Updated below.
diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index e1c4707..094a95e 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -465,7 +465,7 @@ EXPORT_SYMBOL(pci_free_consistent);
Write dma_length of each leader with the combined lengths of
the mergable followers. */
-#define SG_ENT_VIRT_ADDRESS(SG) (page_address((SG)->page) + (SG)->offset)
+#define SG_ENT_VIRT_ADDRESS(SG) (page_address(sg_page((SG)) + (SG)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) __pa(SG_ENT_VIRT_ADDRESS(SG))
static void
diff --git a/arch/arm/common/dmabounce.c b/arch/arm/common/dmabounce.c
index 44ab0da..9d371e4 100644
--- a/arch/arm/common/dmabounce.c
+++ b/arch/arm/common/dmabounce.c
@@ -442,7 +442,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(dir == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- struct page *page = sg->page;
+ struct page *page = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int length = sg->length;
void *ptr = page_address(page) + offset;
diff --git a/arch/blackfin/kernel/dma-mapping.c b/arch/blackfin/kernel/dma-mapping.c
index 94d7b11..dbf7222 100644
--- a/arch/blackfin/kernel/dma-mapping.c
+++ b/arch/blackfin/kernel/dma-mapping.c
@@ -160,7 +160,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(direction == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- sg->dma_address = (dma_addr_t)(page_address(sg->page) +
+ sg->dma_address = (dma_addr_t)(page_address(sg_page(sg)) +
sg->offset);
invalidate_dcache_range(sg_dma_address(sg),
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index 3c95f41..0b74c63 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -246,7 +246,7 @@ static int reserve_sba_gart = 1;
static SBA_INLINE void sba_mark_invalid(struct ioc *, dma_addr_t, size_t);
static SBA_INLINE void sba_free_range(struct ioc *, dma_addr_t, size_t);
-#define sba_sg_address(sg) (page_address((sg)->page) + (sg)->offset)
+#define sba_sg_address(sg) (page_address(sg_page((sg)) + (sg)->offset))
#ifdef FULL_VALID_PDIR
static u64 prefetch_spill_page;
diff --git a/arch/ia64/hp/sim/simscsi.c b/arch/ia64/hp/sim/simscsi.c
index a3a558a..7f1794f 100644
--- a/arch/ia64/hp/sim/simscsi.c
+++ b/arch/ia64/hp/sim/simscsi.c
@@ -131,7 +131,7 @@ simscsi_sg_readwrite (struct scsi_cmnd *sc, int mode, unsigned long offset)
stat.fd = desc[sc->device->id];
scsi_for_each_sg(sc, sl, scsi_sg_count(sc), i) {
- req.addr = __pa(page_address(sl->page) + sl->offset);
+ req.addr = __pa(page_address(sg_page(sl)) + sl->offset);
req.len = sl->length;
if (DBG)
printk("simscsi_sg_%s @ %lx (off %lx) use_sg=%d len=%d\n",
@@ -212,7 +212,7 @@ static void simscsi_fillresult(struct scsi_cmnd *sc, char *buf, unsigned len)
if (!len)
break;
thislen = min(len, slp->length);
- memcpy(page_address(slp->page) + slp->offset, buf, thislen);
+ memcpy(page_address(sg_page(slp)) + slp->offset, buf, thislen);
len -= thislen;
}
}
diff --git a/arch/ia64/sn/pci/pci_dma.c b/arch/ia64/sn/pci/pci_dma.c
index ecd8a52..6048eaf 100644
--- a/arch/ia64/sn/pci/pci_dma.c
+++ b/arch/ia64/sn/pci/pci_dma.c
@@ -16,7 +16,7 @@
#include <asm/sn/pcidev.h>
#include <asm/sn/sn_sal.h>
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) virt_to_phys(SG_ENT_VIRT_ADDRESS(SG))
/**
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index 9d4e4b5..b2d620b 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -121,7 +121,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
int i;
for (i = 0; i < nents; sg++, i++) {
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
dma_sync_single_for_device(dev, sg->dma_address, sg->length, dir);
}
return nents;
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index 98b5e5b..4aa143b 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -165,7 +165,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
for (i = 0; i < nents; i++, sg++) {
unsigned long addr;
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (!plat_device_is_coherent(dev) && addr)
__dma_sync(addr + sg->offset, sg->length, direction);
sg->dma_address = plat_map_dma_mem(dev,
@@ -223,7 +223,7 @@ void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries,
for (i = 0; i < nhwentries; i++, sg++) {
if (!plat_device_is_coherent(dev) &&
direction != DMA_TO_DEVICE) {
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (addr)
__dma_sync(addr + sg->offset, sg->length,
direction);
@@ -304,7 +304,7 @@ void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg, int nelems,
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (cpu_is_noncoherent_r10000(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
@@ -322,7 +322,7 @@ void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg, int nele
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (!plat_device_is_coherent(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
index 9001104..5d0aa90 100644
--- a/arch/powerpc/kernel/dma_64.c
+++ b/arch/powerpc/kernel/dma_64.c
@@ -161,7 +161,7 @@ static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (page_to_phys(sg->page) + sg->offset) |
+ sg->dma_address = (page_to_phys(sg_page(sg)) + sg->offset) |
dma_direct_offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index 289d7e9..3d0e71b 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -102,7 +102,7 @@ static int ibmebus_map_sg(struct device *dev,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (dma_addr_t)page_address(sg->page)
+ sg->dma_address = (dma_addr_t)page_address(sg_page(sg))
+ sg->offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index 306a6f7..47d745a 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -307,7 +307,7 @@ int iommu_map_sg(struct iommu_table *tbl, struct scatterlist *sglist,
continue;
}
/* Allocate iommu entries for that segment */
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(sg_page(s)) + s->offset;
npages = iommu_num_pages(vaddr, slen);
entry = iommu_range_alloc(tbl, npages, &handle, mask >> IOMMU_PAGE_SHIFT, 0);
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 07e64b4..e8202b6 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -629,7 +629,7 @@ static int ps3_sb_map_sg(struct device *_dev, struct scatterlist *sgl,
for_each_sg(sgl, sg, nents, i) {
int result = ps3_dma_map(dev->d_region,
- page_to_phys(sg->page) + sg->offset, sg->length,
+ page_to_phys(sg_page(sg)) + sg->offset, sg->length,
&sg->dma_address, 0);
if (result) {
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index 9c3ed88..abc4470 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -727,9 +727,9 @@ int pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
/* IIep is write-through, not flushing. */
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
sg->dvma_address =
- virt_to_phys(page_address(sg->page)) + sg->offset;
+ virt_to_phys(page_address(sg_page(sg))) + sg->offset;
sg->dvma_length = sg->length;
}
return nents;
@@ -748,9 +748,9 @@ void pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -798,9 +798,9 @@ void pci_dma_sync_sg_for_cpu(struct pci_dev *hwdev, struct scatterlist *sgl, int
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -814,9 +814,9 @@ void pci_dma_sync_sg_for_device(struct pci_dev *hwdev, struct scatterlist *sgl,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
diff --git a/arch/sparc/mm/io-unit.c b/arch/sparc/mm/io-unit.c
index 375b4db..0f2ad57 100644
--- a/arch/sparc/mm/io-unit.c
+++ b/arch/sparc/mm/io-unit.c
@@ -144,7 +144,7 @@ static void iounit_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus
spin_lock_irqsave(&iounit->lock, flags);
while (sz != 0) {
--sz;
- sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/iommu.c b/arch/sparc/mm/iommu.c
index 283656d..4b93427 100644
--- a/arch/sparc/mm/iommu.c
+++ b/arch/sparc/mm/iommu.c
@@ -238,7 +238,7 @@ static void iommu_get_scsi_sgl_noflush(struct scatterlist *sg, int sz, struct sb
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -252,7 +252,7 @@ static void iommu_get_scsi_sgl_gflush(struct scatterlist *sg, int sz, struct sbu
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -273,7 +273,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
* XXX Is this a good assumption?
* XXX What if someone else unmaps it here and races us?
*/
- if ((page = (unsigned long) page_address(sg->page)) != 0) {
+ if ((page = (unsigned long) page_address(sg_page(sg))) != 0) {
for (i = 0; i < n; i++) {
if (page != oldpage) { /* Already flushed? */
flush_page_for_dma(page);
@@ -283,7 +283,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
}
}
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/sun4c.c b/arch/sparc/mm/sun4c.c
index ee6708f..f172dbe 100644
--- a/arch/sparc/mm/sun4c.c
+++ b/arch/sparc/mm/sun4c.c
@@ -1228,7 +1228,7 @@ static void sun4c_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus *
{
while (sz != 0) {
--sz;
- sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/ldc.c b/arch/sparc64/kernel/ldc.c
index 85a2be0..c8313cb 100644
--- a/arch/sparc64/kernel/ldc.c
+++ b/arch/sparc64/kernel/ldc.c
@@ -2057,7 +2057,7 @@ static void fill_cookies(struct cookie_state *sp, unsigned long pa,
static int sg_count_one(struct scatterlist *sg)
{
- unsigned long base = page_to_pfn(sg->page) << PAGE_SHIFT;
+ unsigned long base = page_to_pfn(sg_page(sg)) << PAGE_SHIFT;
long len = sg->length;
if ((sg->offset | len) & (8UL - 1))
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 25b248a..3a8cd3d 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -1115,7 +1115,7 @@ static void do_ubd_request(struct request_queue *q)
}
prepare_request(req, io_req,
(unsigned long long) req->sector << 9,
- sg->offset, sg->length, sg->page);
+ sg->offset, sg->length, sg_page(sg));
last_sectors = sg->length >> 9;
n = os_write_file(thread_fd, &io_req,
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5098f58..c9f0eb0 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -411,8 +411,10 @@ static int calgary_nontranslate_map_sg(struct device* dev,
int i;
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
+ s->dma_address = virt_to_bus(page_address(p) + s->offset);
s->dma_length = s->length;
}
return nelems;
@@ -432,9 +434,11 @@ static int calgary_map_sg(struct device *dev, struct scatterlist *sg,
return calgary_nontranslate_map_sg(dev, sg, nelems, direction);
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(p) + s->offset;
npages = num_dma_pages(vaddr, s->length);
entry = iommu_range_alloc(tbl, npages);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..ce9754f 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,8 +1354,9 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
nsegs++;
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/hmac.c b/crypto/hmac.c
index 8802fb6..e4eb6ac 100644
--- a/crypto/hmac.c
+++ b/crypto/hmac.c
@@ -159,7 +159,8 @@ static int hmac_digest(struct hash_desc *pdesc, struct scatterlist *sg,
desc.flags = pdesc->flags & CRYPTO_TFM_REQ_MAY_SLEEP;
sg_set_buf(sg1, ipad, bs);
- sg1[1].page = (void *)sg;
+
+ sg_set_page(&sg[1], (void *) sg);
sg1[1].length = 0;
sg_set_buf(sg2, opad, bs + ds);
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 18d489c..d741c63 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -317,7 +317,7 @@ static void test_cipher(char *algo, int enc,
goto out;
}
- q = kmap(sg[0].page) + sg[0].offset;
+ q = kmap(sg_page(&sg[0])) + sg[0].offset;
hexdump(q, cipher_tv[i].rlen);
printk("%s\n",
@@ -390,7 +390,7 @@ static void test_cipher(char *algo, int enc,
temp = 0;
for (k = 0; k < cipher_tv[i].np; k++) {
printk("page %u\n", k);
- q = kmap(sg[k].page) + sg[k].offset;
+ q = kmap(sg_page(&sg[k])) + sg[k].offset;
hexdump(q, cipher_tv[i].tap[k]);
printk("%s\n",
memcmp(q, cipher_tv[i].result + temp,
diff --git a/crypto/xcbc.c b/crypto/xcbc.c
index 9f502b8..ac68f3b 100644
--- a/crypto/xcbc.c
+++ b/crypto/xcbc.c
@@ -120,7 +120,7 @@ static int crypto_xcbc_digest_update2(struct hash_desc *pdesc,
do {
- struct page *pg = sg[i].page;
+ struct page *pg = sg_page(&sg[i]);
unsigned int offset = sg[i].offset;
unsigned int slen = sg[i].length;
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
index 7c2cfde..5a6fe17 100644
--- a/drivers/block/cciss.c
+++ b/drivers/block/cciss.c
@@ -2610,7 +2610,7 @@ static void do_cciss_request(struct request_queue *q)
(int)creq->nr_sectors);
#endif /* CCISS_DEBUG */
- memset(tmp_sg, 0, sizeof(tmp_sg));
+ sg_init_table(tmp_sg, MAXSGENTRIES);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* get the DMA records for the setup */
@@ -2621,7 +2621,7 @@ static void do_cciss_request(struct request_queue *q)
for (i = 0; i < seg; i++) {
c->SG[i].Len = tmp_sg[i].length;
- temp64.val = (__u64) pci_map_page(h->pdev, tmp_sg[i].page,
+ temp64.val = (__u64) pci_map_page(h->pdev, sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
c->SG[i].Addr.lower = temp64.val32.lower;
diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c
index 568603d..efab27f 100644
--- a/drivers/block/cpqarray.c
+++ b/drivers/block/cpqarray.c
@@ -918,6 +918,7 @@ queue_next:
DBGPX(
printk("sector=%d, nr_sectors=%d\n", creq->sector, creq->nr_sectors);
);
+ sg_init_table(tmp_sg, SG_MAX);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* Now do all the DMA Mappings */
@@ -929,7 +930,7 @@ DBGPX(
{
c->req.sg[i].size = tmp_sg[i].length;
c->req.sg[i].addr = (__u32) pci_map_page(h->pci_dev,
- tmp_sg[i].page,
+ sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
}
diff --git a/drivers/block/cryptoloop.c b/drivers/block/cryptoloop.c
index 4053503..1b58b01 100644
--- a/drivers/block/cryptoloop.c
+++ b/drivers/block/cryptoloop.c
@@ -26,6 +26,7 @@
#include <linux/crypto.h>
#include <linux/blkdev.h>
#include <linux/loop.h>
+#include <linux/scatterlist.h>
#include <asm/semaphore.h>
#include <asm/uaccess.h>
@@ -119,14 +120,17 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
.tfm = tfm,
.flags = CRYPTO_TFM_REQ_MAY_SLEEP,
};
- struct scatterlist sg_out = { NULL, };
- struct scatterlist sg_in = { NULL, };
+ struct scatterlist sg_out;
+ struct scatterlist sg_in;
encdec_cbc_t encdecfunc;
struct page *in_page, *out_page;
unsigned in_offs, out_offs;
int err;
+ sg_init_table(&sg_out, 1);
+ sg_init_table(&sg_in, 1);
+
if (cmd == READ) {
in_page = raw_page;
in_offs = raw_off;
@@ -146,11 +150,11 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
u32 iv[4] = { 0, };
iv[0] = cpu_to_le32(IV & 0xffffffff);
- sg_in.page = in_page;
+ sg_set_page(&sg_in, in_page);
sg_in.offset = in_offs;
sg_in.length = sz;
- sg_out.page = out_page;
+ sg_set_page(&sg_out, out_page);
sg_out.offset = out_offs;
sg_out.length = sz;
diff --git a/drivers/block/ub.c b/drivers/block/ub.c
index c57dd2b..d6f7b43 100644
--- a/drivers/block/ub.c
+++ b/drivers/block/ub.c
@@ -25,6 +25,7 @@
#include <linux/usb_usual.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
+#include <linux/scatterlist.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
@@ -656,6 +657,7 @@ static int ub_request_fn_1(struct ub_lun *lun, struct request *rq)
if ((cmd = ub_get_cmd(lun)) == NULL)
return -1;
memset(cmd, 0, sizeof(struct ub_scsi_cmd));
+ sg_init_table(cmd->sgv, UB_MAX_REQ_SG);
blkdev_dequeue_request(rq);
@@ -1310,7 +1312,7 @@ static void ub_data_start(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
pipe = sc->send_bulk_pipe;
sc->last_pipe = pipe;
usb_fill_bulk_urb(&sc->work_urb, sc->dev, pipe,
- page_address(sg->page) + sg->offset, sg->length,
+ page_address(sg_page(sg)) + sg->offset, sg->length,
ub_urb_complete, sc);
sc->work_urb.actual_length = 0;
sc->work_urb.error_count = 0;
@@ -1427,7 +1429,7 @@ static void ub_state_sense(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
scmd->state = UB_CMDST_INIT;
scmd->nsg = 1;
sg = &scmd->sgv[0];
- sg->page = virt_to_page(sc->top_sense);
+ sg_set_page(sg, virt_to_page(sc->top_sense));
sg->offset = (unsigned long)sc->top_sense & (PAGE_SIZE-1);
sg->length = UB_SENSE_SIZE;
scmd->len = UB_SENSE_SIZE;
@@ -1863,7 +1865,7 @@ static int ub_sync_read_cap(struct ub_dev *sc, struct ub_lun *lun,
cmd->state = UB_CMDST_INIT;
cmd->nsg = 1;
sg = &cmd->sgv[0];
- sg->page = virt_to_page(p);
+ sg_set_page(sg, virt_to_page(p));
sg->offset = (unsigned long)p & (PAGE_SIZE-1);
sg->length = 8;
cmd->len = 8;
diff --git a/drivers/ide/cris/ide-cris.c b/drivers/ide/cris/ide-cris.c
index 06c75f1..3979bf4 100644
--- a/drivers/ide/cris/ide-cris.c
+++ b/drivers/ide/cris/ide-cris.c
@@ -934,11 +934,11 @@ static int cris_ide_build_dmatable (ide_drive_t *drive)
* than two possibly non-adjacent physical 4kB pages.
*/
/* group sequential buffers into one large buffer */
- addr = page_to_phys(sg->page) + sg->offset;
+ addr = page_to_phys(sg_page(sg)) + sg->offset;
size = sg_dma_len(sg);
while (--i) {
sg = sg_next(sg);
- if ((addr + size) != page_to_phys(sg->page) + sg->offset)
+ if ((addr + size) != page_to_phys(sg_page(sg)) + sg->offset)
break;
size += sg_dma_len(sg);
}
diff --git a/drivers/ide/ide-probe.c b/drivers/ide/ide-probe.c
index 3c945d6..7c640cd 100644
--- a/drivers/ide/ide-probe.c
+++ b/drivers/ide/ide-probe.c
@@ -1337,12 +1337,14 @@ static int hwif_init(ide_hwif_t *hwif)
if (!hwif->sg_max_nents)
hwif->sg_max_nents = PRD_ENTRIES;
- hwif->sg_table = kzalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
+ hwif->sg_table = kmalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
GFP_KERNEL);
if (!hwif->sg_table) {
printk(KERN_ERR "%s: unable to allocate SG table.\n", hwif->name);
goto out;
}
+
+ sg_init_table(hwif->sg_table, hwif->sg_max_nents);
if (init_irq(hwif) == 0)
goto done;
diff --git a/drivers/ide/ide-taskfile.c b/drivers/ide/ide-taskfile.c
index 2a3c8d4..086157f 100644
--- a/drivers/ide/ide-taskfile.c
+++ b/drivers/ide/ide-taskfile.c
@@ -278,7 +278,7 @@ static void ide_pio_sector(ide_drive_t *drive, unsigned int write)
hwif->cursg = sg;
}
- page = cursg->page;
+ page = sg_page(cursg);
offset = cursg->offset + hwif->cursg_ofs * SECTOR_SIZE;
/* get the current page and offset */
diff --git a/drivers/ide/mips/au1xxx-ide.c b/drivers/ide/mips/au1xxx-ide.c
index 47c035a..1b22ab7 100644
--- a/drivers/ide/mips/au1xxx-ide.c
+++ b/drivers/ide/mips/au1xxx-ide.c
@@ -276,7 +276,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
if (iswrite) {
if(!put_source_flags(ahwif->tx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
@@ -285,7 +285,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
} else
{
if(!put_dest_flags(ahwif->rx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/ieee1394/sbp2.c b/drivers/ieee1394/sbp2.c
index 1b353b9..d5dfe11 100644
--- a/drivers/ieee1394/sbp2.c
+++ b/drivers/ieee1394/sbp2.c
@@ -1466,7 +1466,7 @@ static void sbp2_prep_command_orb_sg(struct sbp2_command_orb *orb,
cmd->dma_size = sgpnt[0].length;
cmd->dma_type = CMD_DMA_PAGE;
cmd->cmd_dma = dma_map_page(hi->host->device.parent,
- sgpnt[0].page, sgpnt[0].offset,
+ sg_page(&sgpnt[0]), sgpnt[0].offset,
cmd->dma_size, cmd->dma_dir);
orb->data_descriptor_lo = cmd->cmd_dma;
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 2f54e29..14159ff 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -55,9 +55,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
ib_dma_unmap_sg(dev, chunk->page_list,
chunk->nents, DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->nents; ++i) {
+ struct page *page = sg_page(&chunk->page_list[i]);
+
if (umem->writable && dirty)
- set_page_dirty_lock(chunk->page_list[i].page);
- put_page(chunk->page_list[i].page);
+ set_page_dirty_lock(page);
+ put_page(page);
}
kfree(chunk);
@@ -164,11 +166,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
}
chunk->nents = min_t(int, ret, IB_UMEM_MAX_PAGE_CHUNK);
+ sg_init_table(chunk->page_list, chunk->nents);
for (i = 0; i < chunk->nents; ++i) {
if (vma_list &&
!is_vm_hugetlb_page(vma_list[i + off]))
umem->hugetlb = 0;
- chunk->page_list[i].page = page_list[i + off];
+ sg_set_page(&chunk->page_list[i], page_list[i + off]);
chunk->page_list[i].offset = 0;
chunk->page_list[i].length = PAGE_SIZE;
}
@@ -179,7 +182,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
DMA_BIDIRECTIONAL);
if (chunk->nmap <= 0) {
for (i = 0; i < chunk->nents; ++i)
- put_page(chunk->page_list[i].page);
+ put_page(sg_page(&chunk->page_list[i]));
kfree(chunk);
ret = -ENOMEM;
diff --git a/drivers/infiniband/hw/ipath/ipath_dma.c b/drivers/infiniband/hw/ipath/ipath_dma.c
index 22709a4..e90a0ea 100644
--- a/drivers/infiniband/hw/ipath/ipath_dma.c
+++ b/drivers/infiniband/hw/ipath/ipath_dma.c
@@ -108,7 +108,7 @@ static int ipath_map_sg(struct ib_device *dev, struct scatterlist *sgl,
BUG_ON(!valid_dma_direction(direction));
for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg->page);
+ addr = (u64) page_address(sg_page(sg));
/* TODO: handle highmem pages */
if (!addr) {
ret = 0;
@@ -127,7 +127,7 @@ static void ipath_unmap_sg(struct ib_device *dev,
static u64 ipath_sg_dma_address(struct ib_device *dev, struct scatterlist *sg)
{
- u64 addr = (u64) page_address(sg->page);
+ u64 addr = (u64) page_address(sg_page(sg));
if (addr)
addr += sg->offset;
diff --git a/drivers/infiniband/hw/ipath/ipath_mr.c b/drivers/infiniband/hw/ipath/ipath_mr.c
index e442470..db4ba92 100644
--- a/drivers/infiniband/hw/ipath/ipath_mr.c
+++ b/drivers/infiniband/hw/ipath/ipath_mr.c
@@ -225,7 +225,7 @@ struct ib_mr *ipath_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
for (i = 0; i < chunk->nents; i++) {
void *vaddr;
- vaddr = page_address(chunk->page_list[i].page);
+ vaddr = page_address(sg_page(&chunk->page_list[i]));
if (!vaddr) {
ret = ERR_PTR(-EINVAL);
goto bail;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index e61f3e6..007b381 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -71,7 +71,7 @@ static void mthca_free_icm_pages(struct mthca_dev *dev, struct mthca_icm_chunk *
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -81,7 +81,7 @@ static void mthca_free_icm_coherent(struct mthca_dev *dev, struct mthca_icm_chun
for (i = 0; i < chunk->npages; ++i) {
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
}
@@ -107,10 +107,13 @@ void mthca_free_icm(struct mthca_dev *dev, struct mthca_icm *icm, int coherent)
static int mthca_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -157,6 +160,7 @@ struct mthca_icm *mthca_alloc_icm(struct mthca_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MTHCA_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -304,7 +308,7 @@ void *mthca_table_find(struct mthca_icm_table *table, int obj, dma_addr_t *dma_h
* so if we found the page, dma_handle has already
* been assigned to. */
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
@@ -445,6 +449,7 @@ static u64 mthca_uarc_virt(struct mthca_dev *dev, struct mthca_uar *uar, int pag
int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
struct mthca_user_db_table *db_tab, int index, u64 uaddr)
{
+ struct page *pages[1];
int ret = 0;
u8 status;
int i;
@@ -472,16 +477,17 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
}
ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- &db_tab->page[i].mem.page, NULL);
+ pages, NULL);
if (ret < 0)
goto out;
+ sg_set_page(&db_tab->page[i].mem, pages[0]);
db_tab->page[i].mem.length = MTHCA_ICM_PAGE_SIZE;
db_tab->page[i].mem.offset = uaddr & ~PAGE_MASK;
ret = pci_map_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
if (ret < 0) {
- put_page(db_tab->page[i].mem.page);
+ put_page(pages[0]);
goto out;
}
@@ -491,7 +497,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
ret = -EINVAL;
if (ret) {
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
goto out;
}
@@ -557,7 +563,7 @@ void mthca_cleanup_user_db_tab(struct mthca_dev *dev, struct mthca_uar *uar,
if (db_tab->page[i].uvirt) {
mthca_UNMAP_ICM(dev, mthca_uarc_virt(dev, uar, i), 1, &status);
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
}
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index f3529b6..813b96b 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -131,7 +131,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, data->size, i) {
- from = kmap_atomic(sg->page, KM_USER0);
+ from = kmap_atomic(sg_page(sg), KM_USER0);
memcpy(p,
from + sg->offset,
sg->length);
@@ -191,7 +191,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, sg_size, i) {
- to = kmap_atomic(sg->page, KM_SOFTIRQ0);
+ to = kmap_atomic(sg_page(sg), KM_SOFTIRQ0);
memcpy(to + sg->offset,
p,
sg->length);
@@ -300,7 +300,7 @@ static unsigned int iser_data_buf_aligned_len(struct iser_data_buf *data,
for_each_sg(sgl, sg, data->dma_nents, i) {
/* iser_dbg("Checking sg iobuf [%d]: phys=0x%08lX "
"offset: %ld sz: %ld\n", i,
- (unsigned long)page_to_phys(sg->page),
+ (unsigned long)page_to_phys(sg_page(sg)),
(unsigned long)sg->offset,
(unsigned long)sg->length); */
end_addr = ib_sg_dma_address(ibdev, sg) +
@@ -336,7 +336,7 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
- sg->page, sg->offset,
+ sg_page(sg), sg->offset,
sg->length, ib_sg_dma_len(ibdev, sg));
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 64fee90..cde915f 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -346,16 +346,17 @@ static int crypt_convert(struct crypt_config *cc,
ctx->idx_out < ctx->bio_out->bi_vcnt) {
struct bio_vec *bv_in = bio_iovec_idx(ctx->bio_in, ctx->idx_in);
struct bio_vec *bv_out = bio_iovec_idx(ctx->bio_out, ctx->idx_out);
- struct scatterlist sg_in = {
- .page = bv_in->bv_page,
- .offset = bv_in->bv_offset + ctx->offset_in,
- .length = 1 << SECTOR_SHIFT
- };
- struct scatterlist sg_out = {
- .page = bv_out->bv_page,
- .offset = bv_out->bv_offset + ctx->offset_out,
- .length = 1 << SECTOR_SHIFT
- };
+ struct scatterlist sg_in, sg_out;
+
+ sg_init_table(&sg_in, 1);
+ sg_set_page(&sg_in, bv_in->bv_page);
+ sg_in.offset = bv_in->bv_offset + ctx->offset_in;
+ sg_in.length = 1 << SECTOR_SHIFT;
+
+ sg_init_table(&sg_out, 1);
+ sg_set_page(&sg_out, bv_out->bv_page);
+ sg_out.offset = bv_out->bv_offset + ctx->offset_out;
+ sg_out.length = 1 << SECTOR_SHIFT;
ctx->offset_in += sg_in.length;
if (ctx->offset_in >= bv_in->bv_len) {
diff --git a/drivers/media/common/saa7146_core.c b/drivers/media/common/saa7146_core.c
index 365a221..2b1f8b4 100644
--- a/drivers/media/common/saa7146_core.c
+++ b/drivers/media/common/saa7146_core.c
@@ -112,12 +112,13 @@ static struct scatterlist* vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/media/video/ivtv/ivtv-udma.c b/drivers/media/video/ivtv/ivtv-udma.c
index c4626d1..912b424 100644
--- a/drivers/media/video/ivtv/ivtv-udma.c
+++ b/drivers/media/video/ivtv/ivtv-udma.c
@@ -63,10 +63,10 @@ int ivtv_udma_fill_sg_list (struct ivtv_user_dma *dma, struct ivtv_dma_page_info
memcpy(page_address(dma->bouncemap[map_offset]) + offset, src, len);
kunmap_atomic(src, KM_BOUNCE_READ);
local_irq_restore(flags);
- dma->SGlist[map_offset].page = dma->bouncemap[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->bouncemap[map_offset]);
}
else {
- dma->SGlist[map_offset].page = dma->map[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->map[map_offset]);
}
offset = 0;
map_offset++;
diff --git a/drivers/media/video/videobuf-dma-sg.c b/drivers/media/video/videobuf-dma-sg.c
index 3eb6123..0a18286 100644
--- a/drivers/media/video/videobuf-dma-sg.c
+++ b/drivers/media/video/videobuf-dma-sg.c
@@ -60,12 +60,13 @@ videobuf_vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
@@ -86,13 +87,14 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
sglist = kcalloc(nr_pages, sizeof(*sglist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
if (NULL == pages[0])
goto nopage;
if (PageHighMem(pages[0]))
/* DMA to highmem pages might not work */
goto highmem;
- sglist[0].page = pages[0];
+ sg_set_page(&sglist[0], pages[0]);
sglist[0].offset = offset;
sglist[0].length = PAGE_SIZE - offset;
for (i = 1; i < nr_pages; i++) {
@@ -100,7 +102,7 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
goto nopage;
if (PageHighMem(pages[i]))
goto highmem;
- sglist[i].page = pages[i];
+ sg_set_page(&sglist[i], pages[i]);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/mmc/card/queue.c b/drivers/mmc/card/queue.c
index a5d0354..68c1fe9 100644
--- a/drivers/mmc/card/queue.c
+++ b/drivers/mmc/card/queue.c
@@ -13,6 +13,7 @@
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/scatterlist.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -153,19 +154,21 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card, spinlock_t *lock
blk_queue_max_hw_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
- mq->sg = kzalloc(sizeof(struct scatterlist),
+ mq->sg = kmalloc(sizeof(struct scatterlist),
GFP_KERNEL);
if (!mq->sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->sg, 1);
- mq->bounce_sg = kzalloc(sizeof(struct scatterlist) *
+ mq->bounce_sg = kmalloc(sizeof(struct scatterlist) *
bouncesz / 512, GFP_KERNEL);
if (!mq->bounce_sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->bounce_sg, bouncesz / 512);
}
}
#endif
@@ -302,12 +305,12 @@ static void copy_sg(struct scatterlist *dst, unsigned int dst_len,
BUG_ON(dst_len == 0);
if (dst_size == 0) {
- dst_buf = page_address(dst->page) + dst->offset;
+ dst_buf = page_address(sg_page(dst)) + dst->offset;
dst_size = dst->length;
}
if (src_size == 0) {
- src_buf = page_address(src->page) + src->offset;
+ src_buf = page_address(sg_page(src)) + src->offset;
src_size = src->length;
}
@@ -353,9 +356,7 @@ unsigned int mmc_queue_map_sg(struct mmc_queue *mq)
return 1;
}
- mq->sg[0].page = virt_to_page(mq->bounce_buf);
- mq->sg[0].offset = offset_in_page(mq->bounce_buf);
- mq->sg[0].length = 0;
+ sg_init_one(mq->sg, mq->bounce_buf, 0);
while (sg_len) {
mq->sg[0].length += mq->bounce_sg[sg_len - 1].length;
diff --git a/drivers/mmc/host/at91_mci.c b/drivers/mmc/host/at91_mci.c
index 7a452c2..b1edcef 100644
--- a/drivers/mmc/host/at91_mci.c
+++ b/drivers/mmc/host/at91_mci.c
@@ -149,7 +149,7 @@ static inline void at91_mci_sg_to_dma(struct at91mci_host *host, struct mmc_data
sg = &data->sg[i];
- sgbuffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ sgbuffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
amount = min(size, sg->length);
size -= amount;
@@ -226,7 +226,7 @@ static void at91_mci_pre_dma_read(struct at91mci_host *host)
sg = &data->sg[host->transfer_index++];
pr_debug("sg = %p\n", sg);
- sg->dma_address = dma_map_page(NULL, sg->page, sg->offset, sg->length, DMA_FROM_DEVICE);
+ sg->dma_address = dma_map_page(NULL, sg_page(sg), sg->offset, sg->length, DMA_FROM_DEVICE);
pr_debug("dma address = %08X, length = %d\n", sg->dma_address, sg->length);
@@ -283,7 +283,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
int index;
/* Swap the contents of the buffer */
- buffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
pr_debug("buffer = %p, length = %d\n", buffer, sg->length);
for (index = 0; index < (sg->length / 4); index++)
@@ -292,7 +292,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
kunmap_atomic(buffer, KM_BIO_SRC_IRQ);
}
- flush_dcache_page(sg->page);
+ flush_dcache_page(sg_page(sg));
}
/* Is there another transfer to trigger? */
diff --git a/drivers/mmc/host/au1xmmc.c b/drivers/mmc/host/au1xmmc.c
index 92c4d0d..0632f4d 100644
--- a/drivers/mmc/host/au1xmmc.c
+++ b/drivers/mmc/host/au1xmmc.c
@@ -340,7 +340,7 @@ static void au1xmmc_send_pio(struct au1xmmc_host *host)
/* This is the pointer to the data buffer */
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = data->sg[host->pio.index].length - host->pio.offset;
@@ -400,7 +400,7 @@ static void au1xmmc_receive_pio(struct au1xmmc_host *host)
if (host->pio.index < host->dma.len) {
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = sg_dma_len(&data->sg[host->pio.index]) - host->pio.offset;
@@ -613,13 +613,13 @@ au1xmmc_prepare_data(struct au1xmmc_host *host, struct mmc_data *data)
if (host->flags & HOST_F_XMIT){
ret = au1xxx_dbdma_put_source_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
else {
ret = au1xxx_dbdma_put_dest_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
diff --git a/drivers/mmc/host/imxmmc.c b/drivers/mmc/host/imxmmc.c
index 6ebc41e..581bb8f 100644
--- a/drivers/mmc/host/imxmmc.c
+++ b/drivers/mmc/host/imxmmc.c
@@ -262,7 +262,7 @@ static void imxmci_setup_data(struct imxmci_host *host, struct mmc_data *data)
}
/* Convert back to virtual address */
- host->data_ptr = (u16*)(page_address(data->sg->page) + data->sg->offset);
+ host->data_ptr = (u16*)(page_address(sg_page(data->sg)) + data->sg->offset);
host->data_cnt = 0;
clear_bit(IMXMCI_PEND_DMA_DATA_b, &host->pending_events);
diff --git a/drivers/mmc/host/mmc_spi.c b/drivers/mmc/host/mmc_spi.c
index 7ae18ea..12c2d80 100644
--- a/drivers/mmc/host/mmc_spi.c
+++ b/drivers/mmc/host/mmc_spi.c
@@ -813,7 +813,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
&& dir == DMA_FROM_DEVICE)
dir = DMA_BIDIRECTIONAL;
- dma_addr = dma_map_page(dma_dev, sg->page, 0,
+ dma_addr = dma_map_page(dma_dev, sg_page(sg), 0,
PAGE_SIZE, dir);
if (direction == DMA_TO_DEVICE)
t->tx_dma = dma_addr + sg->offset;
@@ -822,7 +822,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
}
/* allow pio too; we don't allow highmem */
- kmap_addr = kmap(sg->page);
+ kmap_addr = kmap(sg_page(sg));
if (direction == DMA_TO_DEVICE)
t->tx_buf = kmap_addr + sg->offset;
else
@@ -855,8 +855,8 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
/* discard mappings */
if (direction == DMA_FROM_DEVICE)
- flush_kernel_dcache_page(sg->page);
- kunmap(sg->page);
+ flush_kernel_dcache_page(sg_page(sg));
+ kunmap(sg_page(sg));
if (dma_dev)
dma_unmap_page(dma_dev, dma_addr, PAGE_SIZE, dir);
diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c
index 60a67df..649f647 100644
--- a/drivers/mmc/host/omap.c
+++ b/drivers/mmc/host/omap.c
@@ -24,10 +24,10 @@
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
#include <linux/clk.h>
+#include <linux/scatterlist.h>
#include <asm/io.h>
#include <asm/irq.h>
-#include <asm/scatterlist.h>
#include <asm/mach-types.h>
#include <asm/arch/board.h>
@@ -383,7 +383,7 @@ mmc_omap_sg_to_buf(struct mmc_omap_host *host)
sg = host->data->sg + host->sg_idx;
host->buffer_bytes_left = sg->length;
- host->buffer = page_address(sg->page) + sg->offset;
+ host->buffer = page_address(sg_page(sg)) + sg->offset;
if (host->buffer_bytes_left > host->total_bytes_left)
host->buffer_bytes_left = host->total_bytes_left;
}
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index b397121..b6500e9 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -231,7 +231,7 @@ static void sdhci_deactivate_led(struct sdhci_host *host)
static inline char* sdhci_sg_to_buffer(struct sdhci_host* host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline int sdhci_next_sg(struct sdhci_host* host)
diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c
index 9b90479..c11a3d2 100644
--- a/drivers/mmc/host/tifm_sd.c
+++ b/drivers/mmc/host/tifm_sd.c
@@ -192,7 +192,7 @@ static void tifm_sd_transfer_data(struct tifm_sd *host)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
@@ -241,18 +241,18 @@ static void tifm_sd_bounce_block(struct tifm_sd *host, struct mmc_data *r_data)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
p_cnt = min(p_cnt, t_size);
if (r_data->flags & MMC_DATA_WRITE)
- tifm_sd_copy_page(host->bounce_buf.page,
+ tifm_sd_copy_page(sg_page(&host->bounce_buf),
r_data->blksz - t_size,
pg, p_off, p_cnt);
else if (r_data->flags & MMC_DATA_READ)
- tifm_sd_copy_page(pg, p_off, host->bounce_buf.page,
+ tifm_sd_copy_page(pg, p_off, sg_page(&host->bounce_buf),
r_data->blksz - t_size, p_cnt);
t_size -= p_cnt;
diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c
index 80db11c..971ca0f 100644
--- a/drivers/mmc/host/wbsd.c
+++ b/drivers/mmc/host/wbsd.c
@@ -269,7 +269,7 @@ static inline int wbsd_next_sg(struct wbsd_host *host)
static inline char *wbsd_sg_to_buffer(struct wbsd_host *host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
@@ -283,7 +283,7 @@ static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(dmabuf, sgbuf, sg[i].length);
dmabuf += sg[i].length;
}
@@ -300,7 +300,7 @@ static inline void wbsd_dma_to_sg(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(sgbuf, dmabuf, sg[i].length);
dmabuf += sg[i].length;
}
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 4b3c109..887633b 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -60,7 +60,7 @@ static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chu
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -70,7 +70,7 @@ static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *
for (i = 0; i < chunk->npages; ++i)
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
@@ -95,10 +95,13 @@ void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent)
static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -145,6 +148,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MLX4_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -334,7 +338,7 @@ void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_han
* been assigned to.
*/
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
diff --git a/drivers/net/ppp_mppe.c b/drivers/net/ppp_mppe.c
index c0b6d19..bcb0885 100644
--- a/drivers/net/ppp_mppe.c
+++ b/drivers/net/ppp_mppe.c
@@ -55,7 +55,7 @@
#include <linux/mm.h>
#include <linux/ppp_defs.h>
#include <linux/ppp-comp.h>
-#include <asm/scatterlist.h>
+#include <linux/scatterlist.h>
#include "ppp_mppe.h"
@@ -68,9 +68,7 @@ MODULE_VERSION("1.0.2");
static unsigned int
setup_sg(struct scatterlist *sg, const void *address, unsigned int length)
{
- sg[0].page = virt_to_page(address);
- sg[0].offset = offset_in_page(address);
- sg[0].length = length;
+ sg_init_one(sg, address, length);
return length;
}
diff --git a/drivers/scsi/3w-9xxx.c b/drivers/scsi/3w-9xxx.c
index fb14014..afb262b 100644
--- a/drivers/scsi/3w-9xxx.c
+++ b/drivers/scsi/3w-9xxx.c
@@ -1840,7 +1840,7 @@ static int twa_scsiop_execute_scsi(TW_Device_Extension *tw_dev, int request_id,
(scsi_bufflen(srb) < TW_MIN_SGL_LENGTH)) {
if (srb->sc_data_direction == DMA_TO_DEVICE || srb->sc_data_direction == DMA_BIDIRECTIONAL) {
struct scatterlist *sg = scsi_sglist(srb);
- char *buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ char *buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(tw_dev->generic_buffer_virt[request_id], buf, sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
}
@@ -1919,7 +1919,7 @@ static void twa_scsiop_execute_scsi_complete(TW_Device_Extension *tw_dev, int re
char *buf;
unsigned long flags = 0;
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buf, tw_dev->generic_buffer_virt[request_id], sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/3w-xxxx.c b/drivers/scsi/3w-xxxx.c
index a64153b..59716eb 100644
--- a/drivers/scsi/3w-xxxx.c
+++ b/drivers/scsi/3w-xxxx.c
@@ -1469,7 +1469,7 @@ static void tw_transfer_internal(TW_Device_Extension *tw_dev, int request_id,
struct scatterlist *sg = scsi_sglist(cmd);
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len);
memcpy(buf, data, transfer_len);
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 988f0bc..7a76b4b 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -298,7 +298,7 @@ static __inline__ void initialize_SCp(Scsi_Cmnd * cmd)
if (cmd->use_sg) {
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.buffers_residual = cmd->use_sg - 1;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
} else {
@@ -2143,7 +2143,7 @@ static void NCR5380_information_transfer(struct Scsi_Host *instance) {
++cmd->SCp.buffer;
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
}
diff --git a/drivers/scsi/NCR53c406a.c b/drivers/scsi/NCR53c406a.c
index 3168a17..db1a8bc 100644
--- a/drivers/scsi/NCR53c406a.c
+++ b/drivers/scsi/NCR53c406a.c
@@ -875,7 +875,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_write(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_write(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
@@ -897,7 +897,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_read(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_read(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 80e448d..a77ab8d 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -356,7 +356,7 @@ static void aac_internal_transfer(struct scsi_cmnd *scsicmd, void *data, unsigne
int transfer_len;
struct scatterlist *sg = scsi_sglist(scsicmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len + offset);
transfer_len -= offset;
diff --git a/drivers/scsi/aha1542.c b/drivers/scsi/aha1542.c
index 961a188..50da0a3 100644
--- a/drivers/scsi/aha1542.c
+++ b/drivers/scsi/aha1542.c
@@ -49,7 +49,7 @@
#include "aha1542.h"
#define SCSI_BUF_PA(address) isa_virt_to_bus(address)
-#define SCSI_SG_PA(sgent) (isa_page_to_bus((sgent)->page) + (sgent)->offset)
+#define SCSI_SG_PA(sgent) (isa_page_to_bus(sg_page((sgent))) + (sgent)->offset)
static void BAD_DMA(void *address, unsigned int length)
{
@@ -67,7 +67,7 @@ static void BAD_SG_DMA(Scsi_Cmnd * SCpnt,
{
printk(KERN_CRIT "sgpnt[%d:%d] page %p/0x%llx length %u\n",
badseg, nseg,
- page_address(sgp->page) + sgp->offset,
+ page_address(sg_page(sgp)) + sgp->offset,
(unsigned long long)SCSI_SG_PA(sgp),
sgp->length);
@@ -712,7 +712,7 @@ static int aha1542_queuecommand(Scsi_Cmnd * SCpnt, void (*done) (Scsi_Cmnd *))
printk(KERN_CRIT "Bad segment list supplied to aha1542.c (%d, %d)\n", SCpnt->use_sg, i);
scsi_for_each_sg(SCpnt, sg, SCpnt->use_sg, i) {
printk(KERN_CRIT "%d: %p %d\n", i,
- (page_address(sg->page) +
+ (page_address(sg_page(sg)) +
sg->offset), sg->length);
};
printk(KERN_CRIT "cptr %x: ", (unsigned int) cptr);
diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c
index f817775..f7a2528 100644
--- a/drivers/scsi/arcmsr/arcmsr_hba.c
+++ b/drivers/scsi/arcmsr/arcmsr_hba.c
@@ -1343,7 +1343,7 @@ static int arcmsr_iop_message_xfer(struct AdapterControlBlock *acb, \
/* 4 bytes: Areca io control code */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (scsi_sg_count(cmd) > 1) {
retvalue = ARCMSR_MESSAGE_FAIL;
goto message_out;
@@ -1593,7 +1593,7 @@ static void arcmsr_handle_virtual_command(struct AdapterControlBlock *acb,
strncpy(&inqdata[32], "R001", 4); /* Product Revision */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buffer, inqdata, sizeof(inqdata));
sg = scsi_sglist(cmd);
diff --git a/drivers/scsi/fdomain.c b/drivers/scsi/fdomain.c
index 5d282e6..dc4e463 100644
--- a/drivers/scsi/fdomain.c
+++ b/drivers/scsi/fdomain.c
@@ -1321,7 +1321,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
if (current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
} else
break;
@@ -1354,7 +1354,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
&& current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
}
}
@@ -1439,7 +1439,7 @@ static int fdomain_16x0_queue(struct scsi_cmnd *SCpnt,
if (scsi_sg_count(current_SC)) {
current_SC->SCp.buffer = scsi_sglist(current_SC);
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page)
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer))
+ current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
current_SC->SCp.buffers_residual = scsi_sg_count(current_SC) - 1;
diff --git a/drivers/scsi/gdth.c b/drivers/scsi/gdth.c
index 3ac080e..ac6c57a 100644
--- a/drivers/scsi/gdth.c
+++ b/drivers/scsi/gdth.c
@@ -2374,13 +2374,13 @@ static void gdth_copy_internal_data(gdth_ha_str *ha, Scsi_Cmnd *scp,
if (cpsum+cpnow > cpcount)
cpnow = cpcount - cpsum;
cpsum += cpnow;
- if (!sl->page) {
+ if (!sg_page(sl)) {
printk("GDT-HA %d: invalid sc/gt element in gdth_copy_internal_data()\n",
ha->hanum);
return;
}
local_irq_save(flags);
- address = kmap_atomic(sl->page, KM_BIO_SRC_IRQ) + sl->offset;
+ address = kmap_atomic(sg_page(sl), KM_BIO_SRC_IRQ) + sl->offset;
if (to_buffer)
memcpy(buffer, address, cpnow);
else
diff --git a/drivers/scsi/ibmmca.c b/drivers/scsi/ibmmca.c
index 714e627..db004a4 100644
--- a/drivers/scsi/ibmmca.c
+++ b/drivers/scsi/ibmmca.c
@@ -1828,7 +1828,7 @@ static int ibmmca_queuecommand(Scsi_Cmnd * cmd, void (*done) (Scsi_Cmnd *))
BUG_ON(scsi_sg_count(cmd) > 16);
scsi_for_each_sg(cmd, sg, scsi_sg_count(cmd), i) {
- ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg->page) + sg->offset);
+ ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg_page(sg)) + sg->offset);
ld(shpnt)[ldn].sge[i].byte_length = sg->length;
}
scb->enable |= IM_POINTER_TO_LIST;
diff --git a/drivers/scsi/ide-scsi.c b/drivers/scsi/ide-scsi.c
index fa7ba64..2c7dd8b 100644
--- a/drivers/scsi/ide-scsi.c
+++ b/drivers/scsi/ide-scsi.c
@@ -175,18 +175,18 @@ static void idescsi_input_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsigne
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
}
@@ -212,18 +212,18 @@ static void idescsi_output_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsign
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
}
diff --git a/drivers/scsi/imm.c b/drivers/scsi/imm.c
index 74cdc1f..da9b0ba 100644
--- a/drivers/scsi/imm.c
+++ b/drivers/scsi/imm.c
@@ -706,7 +706,7 @@ static int imm_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
/*
@@ -845,7 +845,7 @@ static int imm_engine(imm_struct *dev, struct scsi_cmnd *cmd)
(struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..dc98a85 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2872,6 +2872,7 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
}
scatterlist = sglist->scatterlist;
+ sg_init_table(scatterlist, num_elem);
sglist->order = order;
sglist->num_sg = num_elem;
@@ -2884,12 +2885,12 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
/* Free up what we already allocated */
for (j = i - 1; j >= 0; j--)
- __free_pages(scatterlist[j].page, order);
+ __free_pages(sg_page(&scatterlist[j]), order);
kfree(sglist);
return NULL;
}
- scatterlist[i].page = page;
+ sg_set_page(&scatterlist[i], page);
}
return sglist;
@@ -2910,7 +2911,7 @@ static void ipr_free_ucode_buffer(struct ipr_sglist *sglist)
int i;
for (i = 0; i < sglist->num_sg; i++)
- __free_pages(sglist->scatterlist[i].page, sglist->order);
+ __free_pages(sg_page(&sglist->scatterlist[i]), sglist->order);
kfree(sglist);
}
@@ -2940,9 +2941,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
scatterlist = sglist->scatterlist;
for (i = 0; i < (len / bsize_elem); i++, buffer += bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = bsize_elem;
@@ -2953,9 +2956,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
}
if (len % bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, len % bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = len % bsize_elem;
}
diff --git a/drivers/scsi/ips.c b/drivers/scsi/ips.c
index edaac27..5c5a9b2 100644
--- a/drivers/scsi/ips.c
+++ b/drivers/scsi/ips.c
@@ -1515,7 +1515,7 @@ static int ips_is_passthru(struct scsi_cmnd *SC)
/* kmap_atomic() ensures addressability of the user buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (buffer && buffer[0] == 'C' && buffer[1] == 'O' &&
buffer[2] == 'P' && buffer[3] == 'P') {
kunmap_atomic(buffer - sg->offset, KM_IRQ0);
@@ -3523,7 +3523,7 @@ ips_scmd_buf_write(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(buffer, &cdata[xfer_cnt], min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
@@ -3556,7 +3556,7 @@ ips_scmd_buf_read(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(&cdata[xfer_cnt], buffer, min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index a21455d..6ce4109 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -70,9 +70,7 @@ module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO);
static inline void
iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
{
- ibuf->sg.page = virt_to_page(vbuf);
- ibuf->sg.offset = offset_in_page(vbuf);
- ibuf->sg.length = size;
+ sg_init_one(&ibuf->sg, vbuf, size);
ibuf->sent = 0;
ibuf->use_sendmsg = 1;
}
@@ -80,13 +78,14 @@ iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
static inline void
iscsi_buf_init_sg(struct iscsi_buf *ibuf, struct scatterlist *sg)
{
- ibuf->sg.page = sg->page;
+ sg_init_table(&ibuf->sg, 1);
+ sg_set_page(&ibuf->sg, sg_page(sg));
ibuf->sg.offset = sg->offset;
ibuf->sg.length = sg->length;
/*
* Fastpath: sg element fits into single page
*/
- if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg->page))
+ if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg_page(sg)))
ibuf->use_sendmsg = 0;
else
ibuf->use_sendmsg = 1;
@@ -716,7 +715,7 @@ static int iscsi_scsi_data_in(struct iscsi_conn *conn)
for (i = tcp_ctask->sg_count; i < scsi_sg_count(sc); i++) {
char *dest;
- dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0);
+ dest = kmap_atomic(sg_page(&sg[i]), KM_SOFTIRQ0);
rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset,
sg[i].length, offset);
kunmap_atomic(dest, KM_SOFTIRQ0);
@@ -1103,9 +1102,9 @@ iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags)
* slab case.
*/
if (buf->use_sendmsg)
- res = sock_no_sendpage(sk, buf->sg.page, offset, size, flags);
+ res = sock_no_sendpage(sk, sg_page(&buf->sg), offset, size, flags);
else
- res = tcp_conn->sendpage(sk, buf->sg.page, offset, size, flags);
+ res = tcp_conn->sendpage(sk, sg_page(&buf->sg), offset, size, flags);
if (res >= 0) {
conn->txdata_octets += res;
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 10d1aff..0a848e8 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -658,7 +658,7 @@ mega_build_cmd(adapter_t *adapter, Scsi_Cmnd *cmd, int *busy)
struct scatterlist *sg;
sg = scsi_sglist(cmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memset(buf, 0, cmd->cmnd[4]);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
@@ -1542,9 +1542,9 @@ mega_cmd_done(adapter_t *adapter, u8 completed[], int nstatus, int status)
if( cmd->cmnd[0] == INQUIRY && !islogical ) {
sgl = scsi_sglist(cmd);
- if( sgl->page ) {
+ if( sg_page(sgl) ) {
c = *(unsigned char *)
- page_address((&sgl[0])->page) +
+ page_address((sg_page(&sgl[0]))) +
(&sgl[0])->offset;
} else {
printk(KERN_WARNING
diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c
index e4e4c6a..376263c 100644
--- a/drivers/scsi/megaraid/megaraid_mbox.c
+++ b/drivers/scsi/megaraid/megaraid_mbox.c
@@ -1584,9 +1584,9 @@ megaraid_mbox_build_cmd(adapter_t *adapter, struct scsi_cmnd *scp, int *busy)
caddr_t vaddr;
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
vaddr = (caddr_t)
- (page_address((&sgl[0])->page)
+ (page_address(sg_page(&sgl[0]))
+ (&sgl[0])->offset);
memset(vaddr, 0, scp->cmnd[4]);
@@ -2328,9 +2328,9 @@ megaraid_mbox_dpc(unsigned long devp)
&& IS_RAID_CH(raid_dev, scb->dev_channel)) {
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
c = *(unsigned char *)
- (page_address((&sgl[0])->page) +
+ (page_address(sg_page(&sgl[0])) +
(&sgl[0])->offset);
} else {
con_log(CL_ANN, (KERN_WARNING
diff --git a/drivers/scsi/osst.c b/drivers/scsi/osst.c
index 331b789..1c5c4b6 100644
--- a/drivers/scsi/osst.c
+++ b/drivers/scsi/osst.c
@@ -542,7 +542,7 @@ static int osst_verify_frame(struct osst_tape * STp, int frame_seq_number, int q
if (STp->raw) {
if (STp->buffer->syscall_result) {
for (i=0; i < STp->buffer->sg_segs; i++)
- memset(page_address(STp->buffer->sg[i].page),
+ memset(page_address(sg_page(&STp->buffer->sg[i])),
0, STp->buffer->sg[i].length);
strcpy(STp->buffer->b_data, "READ ERROR ON FRAME");
} else
@@ -4437,7 +4437,7 @@ static int os_scsi_tape_open(struct inode * inode, struct file * filp)
for (i = 0, b_size = 0;
(i < STp->buffer->sg_segs) && ((b_size + STp->buffer->sg[i].length) <= OS_DATA_SIZE);
b_size += STp->buffer->sg[i++].length);
- STp->buffer->aux = (os_aux_t *) (page_address(STp->buffer->sg[i].page) + OS_DATA_SIZE - b_size);
+ STp->buffer->aux = (os_aux_t *) (page_address(sg_page(&STp->buffer->sg[i])) + OS_DATA_SIZE - b_size);
#if DEBUG
printk(OSST_DEB_MSG "%s:D: b_data points to %p in segment 0 at %p\n", name,
STp->buffer->b_data, page_address(STp->buffer->sg[0].page));
@@ -5252,25 +5252,26 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
/* Try to allocate the first segment up to OS_DATA_SIZE and the others
big enough to reach the goal (code assumes no segments in place) */
for (b_size = OS_DATA_SIZE, order = OSST_FIRST_ORDER; b_size >= PAGE_SIZE; order--, b_size /= 2) {
- STbuffer->sg[0].page = alloc_pages(priority, order);
+ struct page *page = alloc_pages(priority, order);
+
STbuffer->sg[0].offset = 0;
- if (STbuffer->sg[0].page != NULL) {
+ if (page != NULL) {
+ sg_set_page(&STbuffer->sg[0], page);
STbuffer->sg[0].length = b_size;
- STbuffer->b_data = page_address(STbuffer->sg[0].page);
+ STbuffer->b_data = page_address(page);
break;
}
}
- if (STbuffer->sg[0].page == NULL) {
+ if (sg_page(&STbuffer->sg[0]) == NULL) {
printk(KERN_NOTICE "osst :I: Can't allocate tape buffer main segment.\n");
return 0;
}
/* Got initial segment of 'bsize,order', continue with same size if possible, except for AUX */
for (segs=STbuffer->sg_segs=1, got=b_size;
segs < max_segs && got < OS_FRAME_SIZE; ) {
- STbuffer->sg[segs].page =
- alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
+ struct page *page = alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
STbuffer->sg[segs].offset = 0;
- if (STbuffer->sg[segs].page == NULL) {
+ if (page == NULL) {
if (OS_FRAME_SIZE - got <= (max_segs - segs) * b_size / 2 && order) {
b_size /= 2; /* Large enough for the rest of the buffers */
order--;
@@ -5284,6 +5285,7 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
normalize_buffer(STbuffer);
return 0;
}
+ sg_set_page(&STbuffer->sg[segs], page);
STbuffer->sg[segs].length = (OS_FRAME_SIZE - got <= PAGE_SIZE / 2) ? (OS_FRAME_SIZE - got) : b_size;
got += STbuffer->sg[segs].length;
STbuffer->buffer_size = got;
@@ -5316,7 +5318,7 @@ static void normalize_buffer(struct osst_buffer *STbuffer)
b_size < STbuffer->sg[i].length;
b_size *= 2, order++);
- __free_pages(STbuffer->sg[i].page, order);
+ __free_pages(sg_page(&STbuffer->sg[i]), order);
STbuffer->buffer_size -= STbuffer->sg[i].length;
}
#if DEBUG
@@ -5344,7 +5346,7 @@ static int append_to_buffer(const char __user *ubp, struct osst_buffer *st_bp, i
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->sg[i].page) + offset, ubp, cnt);
+ res = copy_from_user(page_address(sg_page(&st_bp->sg[i])) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5377,7 +5379,7 @@ static int from_buffer(struct osst_buffer *st_bp, char __user *ubp, int do_count
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->sg[i].page) + offset, cnt);
+ res = copy_to_user(ubp, page_address(sg_page(&st_bp->sg[i])) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5410,7 +5412,7 @@ static int osst_zero_buffer_tail(struct osst_buffer *st_bp)
i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count ;
- memset(page_address(st_bp->sg[i].page) + offset, 0, cnt);
+ memset(page_address(sg_page(&st_bp->sg[i])) + offset, 0, cnt);
do_count -= cnt;
offset = 0;
}
@@ -5430,7 +5432,7 @@ static int osst_copy_to_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(page_address(st_bp->sg[i].page), ptr, cnt);
+ memcpy(page_address(sg_page(&st_bp->sg[i])), ptr, cnt);
do_count -= cnt;
ptr += cnt;
}
@@ -5451,7 +5453,7 @@ static int osst_copy_from_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(ptr, page_address(st_bp->sg[i].page), cnt);
+ memcpy(ptr, page_address(sg_page(&st_bp->sg[i])), cnt);
do_count -= cnt;
ptr += cnt;
}
diff --git a/drivers/scsi/pcmcia/sym53c500_cs.c b/drivers/scsi/pcmcia/sym53c500_cs.c
index 190e2a7..26c41b0 100644
--- a/drivers/scsi/pcmcia/sym53c500_cs.c
+++ b/drivers/scsi/pcmcia/sym53c500_cs.c
@@ -443,7 +443,7 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_write(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
+ page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0(port_base);
@@ -463,8 +463,8 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_read(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
- sg->length);
+ page_address(sg_page(sg)) + sg->offset,
+ sg->length);
}
REG0(port_base);
}
diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c
index 67b6d76..07f31e5 100644
--- a/drivers/scsi/ppa.c
+++ b/drivers/scsi/ppa.c
@@ -609,7 +609,7 @@ static int ppa_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
}
}
@@ -756,7 +756,7 @@ static int ppa_engine(ppa_struct *dev, struct scsi_cmnd *cmd)
/* if many buffers are available, start filling the first */
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page) +
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/qlogicfas408.c b/drivers/scsi/qlogicfas408.c
index 2bfbf26..feff3f7 100644
--- a/drivers/scsi/qlogicfas408.c
+++ b/drivers/scsi/qlogicfas408.c
@@ -317,7 +317,7 @@ static unsigned int ql_pcmd(struct scsi_cmnd *cmd)
return ((priv->qabort == 1 ?
DID_ABORT : DID_RESET) << 16);
}
- buf = page_address(sg->page) + sg->offset;
+ buf = page_address(sg_page(sg)) + sg->offset;
if (ql_pdma(priv, phase, buf, sg->length))
break;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..61fdaf0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,7 +764,7 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
- memset(sgl, 0, sizeof(*sgl) * sgp->size);
+ sg_init_table(sgl, sgp->size);
/*
* first loop through, set initial index and return value
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 73c44cb..ce69b9e 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -3797,7 +3797,7 @@ static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
sg = &(STbp->sg[0]);
frp = STbp->frp;
for (i=count=0; count < length; i++) {
- sg[i].page = frp[i].page;
+ sg_set_page(&sg[i], frp[i].page);
if (length - count > frp[i].length)
sg[i].length = frp[i].length;
else
@@ -4446,14 +4446,14 @@ static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pa
}
/* Populate the scatter/gather list */
- sgl[0].page = pages[0];
+ sg_set_page(&sgl[0], pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
+ sg_set_page(&sgl[i], pages[i]);;
sgl[i].offset = 0;
- sgl[i].page = pages[i];
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -4483,7 +4483,7 @@ static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_p
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
diff --git a/drivers/scsi/tmscsim.c b/drivers/scsi/tmscsim.c
index 5c72ca3..4419304 100644
--- a/drivers/scsi/tmscsim.c
+++ b/drivers/scsi/tmscsim.c
@@ -430,10 +430,7 @@ static __inline__ void dc390_Going_remove (struct dc390_dcb* pDCB, struct dc390_
static struct scatterlist* dc390_sg_build_single(struct scatterlist *sg, void *addr, unsigned int length)
{
- memset(sg, 0, sizeof(struct scatterlist));
- sg->page = virt_to_page(addr);
- sg->length = length;
- sg->offset = (unsigned long)addr & ~PAGE_MASK;
+ sg_init_one(sg, addr, length);
return sg;
}
diff --git a/drivers/scsi/ultrastor.c b/drivers/scsi/ultrastor.c
index ea72bbe..6d1f0ed 100644
--- a/drivers/scsi/ultrastor.c
+++ b/drivers/scsi/ultrastor.c
@@ -681,7 +681,7 @@ static inline void build_sg_list(struct mscp *mscp, struct scsi_cmnd *SCpnt)
max = scsi_sg_count(SCpnt);
scsi_for_each_sg(SCpnt, sg, max, i) {
- mscp->sglist[i].address = isa_page_to_bus(sg->page) + sg->offset;
+ mscp->sglist[i].address = isa_page_to_bus(sg_page(sg)) + sg->offset;
mscp->sglist[i].num_bytes = sg->length;
transfer_length += sg->length;
}
diff --git a/drivers/scsi/wd7000.c b/drivers/scsi/wd7000.c
index 255c611..03cd44f 100644
--- a/drivers/scsi/wd7000.c
+++ b/drivers/scsi/wd7000.c
@@ -1123,7 +1123,7 @@ static int wd7000_queuecommand(struct scsi_cmnd *SCpnt,
any2scsi(scb->maxlen, nseg * sizeof(Sgb));
scsi_for_each_sg(SCpnt, sg, nseg, i) {
- any2scsi(sgb[i].ptr, isa_page_to_bus(sg->page) + sg->offset);
+ any2scsi(sgb[i].ptr, isa_page_to_bus(sg_page(sg)) + sg->offset);
any2scsi(sgb[i].len, sg->length);
}
} else {
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..3577453 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -438,12 +438,12 @@ int usb_sg_init (
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/image/microtek.c b/drivers/usb/image/microtek.c
index e7d982a..4866baf 100644
--- a/drivers/usb/image/microtek.c
+++ b/drivers/usb/image/microtek.c
@@ -519,7 +519,7 @@ static void mts_do_sg (struct urb* transfer)
context->fragment++;
mts_int_submit_urb(transfer,
context->data_pipe,
- page_address(sg[context->fragment].page) +
+ page_address(sg_page(&sg[context->fragment])) +
sg[context->fragment].offset,
sg[context->fragment].length,
context->fragment + 1 == scsi_sg_count(context->srb) ?
@@ -557,7 +557,7 @@ mts_build_transfer_context(struct scsi_cmnd *srb, struct mts_desc* desc)
return;
} else {
sg = scsi_sglist(srb);
- desc->context.data = page_address(sg[0].page) + sg[0].offset;
+ desc->context.data = page_address(sg_page(&sg[0])) + sg[0].offset;
desc->context.data_length = sg[0].length;
}
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index e901d31..ca8cb34 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -360,9 +360,10 @@ static void free_sglist (struct scatterlist *sg, int nents)
if (!sg)
return;
for (i = 0; i < nents; i++) {
- if (!sg [i].page)
+ struct page *page = sg_page(&sg[i]);
+ if (!page)
continue;
- kfree (page_address (sg [i].page) + sg [i].offset);
+ kfree (page_address (page) + sg [i].offset);
}
kfree (sg);
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index 1ae90ef..0a9882e 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -283,7 +283,7 @@ int virt_to_scatterlist(const void *addr, int size, struct scatterlist *sg,
pg = virt_to_page(addr);
offset = offset_in_page(addr);
if (sg) {
- sg[i].page = pg;
+ sg_set_page(&sg[i], pg);
sg[i].offset = offset;
}
remainder_of_page = PAGE_CACHE_SIZE - offset;
@@ -713,10 +713,13 @@ ecryptfs_encrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return encrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
@@ -742,10 +745,13 @@ ecryptfs_decrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return decrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index 89d9710..263fed8 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -1040,6 +1040,9 @@ decrypt_passphrase_encrypted_session_key(struct ecryptfs_auth_tok *auth_tok,
};
int rc = 0;
+ sg_init_table(&dst_sg, 1);
+ sg_init_table(&src_sg, 1);
+
if (unlikely(ecryptfs_verbosity > 0)) {
ecryptfs_printk(
KERN_DEBUG, "Session key encryption key (size [%d]):\n",
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index ebd03cc..6f03918 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -88,7 +88,7 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
{
struct xdr_netobj cksum;
struct hash_desc desc;
- struct scatterlist sg[1];
+ struct scatterlist sg;
__be32 status = nfserr_resource;
dprintk("NFSD: nfs4_make_rec_clidname for %.*s\n",
@@ -102,11 +102,9 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
if (cksum.data == NULL)
goto out;
- sg[0].page = virt_to_page(clname->data);
- sg[0].offset = offset_in_page(clname->data);
- sg[0].length = clname->len;
+ sg_init_one(&sg, clname->data, clname->len);
- if (crypto_hash_digest(&desc, sg, sg->length, cksum.data))
+ if (crypto_hash_digest(&desc, &sg, sg.length, cksum.data))
goto out;
md5_to_hex(dname, cksum.data);
diff --git a/include/asm-alpha/scatterlist.h b/include/asm-alpha/scatterlist.h
index 9173654..b764706 100644
--- a/include/asm-alpha/scatterlist.h
+++ b/include/asm-alpha/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-arm/scatterlist.h b/include/asm-arm/scatterlist.h
index de2f65e..ab1d85d 100644
--- a/include/asm-arm/scatterlist.h
+++ b/include/asm-arm/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page; /* buffer page */
+ unsigned long page_link;
unsigned int offset; /* buffer offset */
dma_addr_t dma_address; /* dma address */
unsigned int length; /* length */
diff --git a/include/asm-avr32/scatterlist.h b/include/asm-avr32/scatterlist.h
index c6d5ce3..1356f29 100644
--- a/include/asm-avr32/scatterlist.h
+++ b/include/asm-avr32/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-blackfin/scatterlist.h b/include/asm-blackfin/scatterlist.h
index 60e07b9..384af54 100644
--- a/include/asm-blackfin/scatterlist.h
+++ b/include/asm-blackfin/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/mm.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-cris/scatterlist.h b/include/asm-cris/scatterlist.h
index 4bdc44c..5a8a834 100644
--- a/include/asm-cris/scatterlist.h
+++ b/include/asm-cris/scatterlist.h
@@ -6,7 +6,7 @@ struct scatterlist {
unsigned int length;
/* The following is i386 highmem junk - not used by us */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
};
diff --git a/include/asm-frv/scatterlist.h b/include/asm-frv/scatterlist.h
index 8e827fa..53dade7 100644
--- a/include/asm-frv/scatterlist.h
+++ b/include/asm-frv/scatterlist.h
@@ -22,7 +22,7 @@
* and that's it. There's no excuse for not highmem enabling YOUR driver. /jens
*/
struct scatterlist {
- struct page *page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset; /* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-h8300/scatterlist.h b/include/asm-h8300/scatterlist.h
index 985fdf5..7e41983 100644
--- a/include/asm-h8300/scatterlist.h
+++ b/include/asm-h8300/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-ia64/scatterlist.h b/include/asm-ia64/scatterlist.h
index 7d5234d..2f76ce3 100644
--- a/include/asm-ia64/scatterlist.h
+++ b/include/asm-ia64/scatterlist.h
@@ -9,7 +9,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length; /* buffer length */
diff --git a/include/asm-m32r/scatterlist.h b/include/asm-m32r/scatterlist.h
index 352415f..33b4b4d 100644
--- a/include/asm-m32r/scatterlist.h
+++ b/include/asm-m32r/scatterlist.h
@@ -6,7 +6,7 @@
struct scatterlist {
char * address; /* Location data is to be transferred to, NULL for
* highmem page */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-m68k/scatterlist.h b/include/asm-m68k/scatterlist.h
index 24887a2..e06bb89 100644
--- a/include/asm-m68k/scatterlist.h
+++ b/include/asm-m68k/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-m68knommu/scatterlist.h b/include/asm-m68knommu/scatterlist.h
index 4da79d3..28bed41 100644
--- a/include/asm-m68knommu/scatterlist.h
+++ b/include/asm-m68knommu/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-mips/scatterlist.h b/include/asm-mips/scatterlist.h
index 7af104c..787797c 100644
--- a/include/asm-mips/scatterlist.h
+++ b/include/asm-mips/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-parisc/scatterlist.h b/include/asm-parisc/scatterlist.h
index e7211c7..26da914 100644
--- a/include/asm-parisc/scatterlist.h
+++ b/include/asm-parisc/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-powerpc/scatterlist.h b/include/asm-powerpc/scatterlist.h
index b075f61..b9f1dbc 100644
--- a/include/asm-powerpc/scatterlist.h
+++ b/include/asm-powerpc/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/dma.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-s390/scatterlist.h b/include/asm-s390/scatterlist.h
index a43b3af..eb39486 100644
--- a/include/asm-s390/scatterlist.h
+++ b/include/asm-s390/scatterlist.h
@@ -2,7 +2,7 @@
#define _ASMS390_SCATTERLIST_H
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
};
diff --git a/include/asm-sh/scatterlist.h b/include/asm-sh/scatterlist.h
index b9ae53c..bc7c809 100644
--- a/include/asm-sh/scatterlist.h
+++ b/include/asm-sh/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sh64/scatterlist.h b/include/asm-sh64/scatterlist.h
index 1c723f2..0afd856 100644
--- a/include/asm-sh64/scatterlist.h
+++ b/include/asm-sh64/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sparc/scatterlist.h b/include/asm-sparc/scatterlist.h
index 4055af9..45b16f1 100644
--- a/include/asm-sparc/scatterlist.h
+++ b/include/asm-sparc/scatterlist.h
@@ -5,7 +5,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-v850/scatterlist.h b/include/asm-v850/scatterlist.h
index 56f4029..db91feb 100644
--- a/include/asm-v850/scatterlist.h
+++ b/include/asm-v850/scatterlist.h
@@ -17,7 +17,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned offset;
dma_addr_t dma_address;
unsigned length;
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/asm-xtensa/scatterlist.h b/include/asm-xtensa/scatterlist.h
index ca337a2..3b8aba5 100644
--- a/include/asm-xtensa/scatterlist.h
+++ b/include/asm-xtensa/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..3ab94c8 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,29 +5,72 @@
#include <linux/mm.h>
#include <linux/string.h>
+/*
+ * Notes on SG table design.
+ *
+ * Architectures must provide an unsigned long page_link field in the
+ * scatterlist struct. We use that to place the page pointer AND encode
+ * information about the sg table as well. The two lower bits are reserved
+ * for this information.
+ *
+ * If bit 0 is set, then the page_link contains a pointer to the next sg
+ * table list. Otherwise the next entry is at sg + 1.
+ *
+ * If bit 1 is set, then this sg entry is the last element in a list.
+ *
+ * See sg_next().
+ *
+ */
+
+#define SG_MAGIC 0x87654321
+
+/**
+ * sg_set_page - Set sg entry to point at given page
+ * @sg: SG entry
+ * @page: The page
+ *
+ * Use this function to set an sg entry pointing at a page, never assign
+ * the page directly. We encode sg table information in the lower bits
+ * of the page pointer. See sg_page() for looking up the page belonging
+ * to an sg entry.
+ *
+ */
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
+/**
+ * sg_set_buf - Set sg entry to point at given data
+ * @sg: SG entry
+ * @buf: Data
+ * @buflen: Data length
+ *
+*/
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
-static inline void sg_init_one(struct scatterlist *sg, const void *buf,
- unsigned int buflen)
-{
- memset(sg, 0, sizeof(*sg));
- sg_set_buf(sg, buf, buflen);
-}
-
/*
* We overload the LSB of the page pointer to indicate whether it's
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+ ((struct scatterlist *) ((sg)->page_link & ~0x03))
/**
* sg_next - return the next scatterlist entry in a list
@@ -37,14 +80,16 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* of a chained scatterlist, it could jump to the start of a new
* scatterlist array.
*
- * Note that the caller must ensure that there are further entries after
- * the current entry, this function will NOT return NULL for an end-of-list.
- *
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ return NULL;
+ sg++;
if (unlikely(sg_is_chain(sg)))
sg = sg_chain_ptr(sg);
@@ -83,6 +128,10 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+ BUG_ON(!sg_is_last(ret));
+#endif
return ret;
}
@@ -101,7 +150,63 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
+}
+
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+/**
+ * sg_init_one - Initialize a single entry sg list
+ * @sg: SG entry
+ * @buf: Virtual address for IO
+ * @buflen: IO length
+ *
+ * Note: this should not be used on a single entry that is part of a larger
+ * table. Use sg_init_table() for that.
+ *
+ */
+static inline void sg_init_one(struct scatterlist *sg, const void *buf,
+ unsigned int buflen)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_mark_end(sg, 1);
+ sg_set_buf(sg, buf, buflen);
+}
+
+/**
+ * sg_init_table - Initialize SG table
+ * @sgl: The SG table
+ * @nents: Number of entries in table
+ *
+ * Note: If this is part of a chained sg table, sg_mark_end() should be
+ * used only on the last table part.
+ *
+ */
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
}
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..183f42c 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,16 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables.
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_tkip.c b/net/ieee80211/ieee80211_crypt_tkip.c
index 6cc54ee..34d4c77 100644
--- a/net/ieee80211/ieee80211_crypt_tkip.c
+++ b/net/ieee80211/ieee80211_crypt_tkip.c
@@ -390,9 +390,7 @@ static int ieee80211_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(tkey->tx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -485,9 +483,7 @@ static int ieee80211_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 12;
crypto_blkcipher_setkey(tkey->rx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4)) {
if (net_ratelimit()) {
printk(KERN_DEBUG ": TKIP: failed to decrypt "
@@ -539,11 +535,12 @@ static int michael_mic(struct crypto_hash *tfm_michael, u8 * key, u8 * hdr,
printk(KERN_WARNING "michael_mic: tfm_michael == NULL\n");
return -1;
}
- sg[0].page = virt_to_page(hdr);
+ sg_init_table(sg, 2);
+ sg_set_page(&sg[0], virt_to_page(hdr));
sg[0].offset = offset_in_page(hdr);
sg[0].length = 16;
- sg[1].page = virt_to_page(data);
+ sg_set_page(&sg[1], virt_to_page(data));
sg[1].offset = offset_in_page(data);
sg[1].length = data_len;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..0af6103 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,9 +170,7 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -212,9 +210,7 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
return -7;
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..cc806d6 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,9 +138,7 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
}
@@ -204,9 +202,7 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
crc = cpu_to_le32(~crc32_le(~0, data, data_len));
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 7818107..cbd64b2 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -726,7 +726,8 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
/* set up scatter list */
end = skb_tail_pointer(skb);
- sg.page = virt_to_page(auth);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(auth));
sg.offset = (unsigned long)(auth) % PAGE_SIZE;
sg.length = end - (unsigned char *)auth;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index f983a36..d5a9785 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1513,7 +1513,8 @@ static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep,
struct hash_desc desc;
/* Sign the message. */
- sg.page = virt_to_page(&cookie->c);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(&cookie->c));
sg.offset = (unsigned long)(&cookie->c) % PAGE_SIZE;
sg.length = bodysize;
keylen = SCTP_SECRET_SIZE;
@@ -1585,7 +1586,8 @@ struct sctp_association *sctp_unpack_cookie(
/* Check the signature. */
keylen = SCTP_SECRET_SIZE;
- sg.page = virt_to_page(bear_cookie);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(bear_cookie));
sg.offset = (unsigned long)(bear_cookie) % PAGE_SIZE;
sg.length = bodysize;
key = (char *)ep->secret_key[ep->current_key];
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
Jens Axboe wrote:
> On Thu, Oct 18 2007, Mark Lord wrote:
>> Jens Axboe wrote:
>>> On Thu, Oct 18 2007, Mark Lord wrote:
>>>> Jens Axboe wrote:
>>>>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>>>>> Mark Lord wrote:
>>>>>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>>>>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>>>>>> but the top was cut off (isn't there a new config option or patch
>>>>>>> to do double-columns or scrollback or something ???.
>>>>>> Is this a sata_mv box? If so, could you try this patch?
>>>>> If anything, that shrinks the size of the resulting request. Did this
>>>>> patch make any difference to you?
>>>> Not a sata_mv box, so no point here.
>>> Can you try the big patch I just posted?
>> I'll hunt for it and try it, but your earlier little patch already works
>> fine.
I found the latest rev, and it failed to apply cleanly on -git12 or -git13
due to scsi_lib.c. After fixing that portion (replacement chunk below),
I'm now running with -git12, with the sg list debug option enabled (no messages).
Looks okay so far
--- a/drivers/scsi/scsi_lib.c 2007-10-18 09:35:28.000000000 -0400
+++ b/drivers/scsi/scsi_lib.c 2007-10-18 09:46:47.000000000 -0400
@@ -295,7 +295,7 @@
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,6 +764,8 @@
if (unlikely(!sgl))
goto enomem;
+ sg_init_table(sgl, sgp->size);
+
/*
* first loop through, set initial index and return value
*/
@@ -779,6 +781,13 @@
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2351,7 +2360,7 @@
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
Jens Axboe wrote:
> On Thu, Oct 18 2007, Mark Lord wrote:
>> Jens wrote:
>>> OK, I think that covers every arch out there. I haven't been able to
>>> compile any of them, but it's mostly search'n replace operations. I hope
>>> nothing is missing linux/scatterlist.h includes...
>> Patch fails on drivers/scsi/scsi_lib.c.
>>
>> I replaced that part of the patch with this updated portion instead:
>
> Hmm, what are you applying against? Must be a clean tree, throw away any
> patches that you already applied in this thread.
Squeaky-clean linux-2.6.23 + patch-2.6.23-git13 + your patch.
Fails on scsi_lib.c.
-ml
On Thu, Oct 18 2007, Mark Lord wrote:
> Jens Axboe wrote:
>> On Thu, Oct 18 2007, Mark Lord wrote:
>>> Jens Axboe wrote:
>>>> On Thu, Oct 18 2007, Mark Lord wrote:
>>>>> Jens Axboe wrote:
>>>>>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>>>>>> Mark Lord wrote:
>>>>>>>> Okay, mine is dying with EIP at blk_rq_map_sg+0xcb/0x160.
>>>>>>>> Screen photo is at http://rtr.ca/recent/2.6.23-git12-crash.jpg,
>>>>>>>> but the top was cut off (isn't there a new config option or patch
>>>>>>>> to do double-columns or scrollback or something ???.
>>>>>>> Is this a sata_mv box? If so, could you try this patch?
>>>>>> If anything, that shrinks the size of the resulting request. Did this
>>>>>> patch make any difference to you?
>>>>> Not a sata_mv box, so no point here.
>>>> Can you try the big patch I just posted?
>>> I'll hunt for it and try it, but your earlier little patch already works
>>> fine.
>
> I found the latest rev, and it failed to apply cleanly on -git12 or -git13
> due to scsi_lib.c. After fixing that portion (replacement chunk below),
> I'm now running with -git12, with the sg list debug option enabled (no
> messages).
>
> Looks okay so far
OK, thanks a lot for testing!
--
Jens Axboe
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, Benny Halevy wrote:
> > > return sg;
> > > }
> > > @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> > > ret = sg;
> > >
> > > #endif
> > > +#ifdef CONFIG_DEBUG_SG
> > > + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
> >
> > can it also do BUG_ON(!sg_is_last(sg))?
>
> That would make sense, definitely. I'll add that.
BUG_ON(!sg_is_last(ret));
it should be, not sg. That's what I merged.
--
Jens Axboe
> On Thu, Oct 18 2007, Mark Lord wrote:
>> > Jens wrote:
>>> >> OK, I think that covers every arch out there. I haven't been able to
>>> >> compile any of them, but it's mostly search'n replace operations. I hope
>>> >> nothing is missing linux/scatterlist.h includes...
>> >
>> > Patch fails on drivers/scsi/scsi_lib.c.
>> >
>> > I replaced that part of the patch with this updated portion instead:
>
> Hmm, what are you applying against? Must be a clean tree, throw away any
> patches that you already applied in this thread.
>
> Updated below.
I'll re-pull everything fresh again from kernel.org and rebuild
with the "updated below" patch you posted. Thanks.
Mark Lord wrote:
>> On Thu, Oct 18 2007, Mark Lord wrote:
>>> > Jens wrote:
>>>> >> OK, I think that covers every arch out there. I haven't been able to
>>>> >> compile any of them, but it's mostly search'n replace operations.
>>>> I hope
>>>> >> nothing is missing linux/scatterlist.h includes...
>>> >
>>> > Patch fails on drivers/scsi/scsi_lib.c.
>>> >
>>> > I replaced that part of the patch with this updated portion instead:
>>
>> Hmm, what are you applying against? Must be a clean tree, throw away any
>> patches that you already applied in this thread.
>>
>> Updated below.
>
>
> I'll re-pull everything fresh again from kernel.org and rebuild
> with the "updated below" patch you posted. Thanks.
Okay, fresh pull of everything from kernel.org,
and now your latest patch does apply cleanly to -git13.
Something weird (at this end).
Thanks.
On Thu, Oct 18 2007, Mark Lord wrote:
> Mark Lord wrote:
>>> On Thu, Oct 18 2007, Mark Lord wrote:
>>>> > Jens wrote:
>>>>> >> OK, I think that covers every arch out there. I haven't been able to
>>>>> >> compile any of them, but it's mostly search'n replace operations. I
>>>>> hope
>>>>> >> nothing is missing linux/scatterlist.h includes...
>>>> >
>>>> > Patch fails on drivers/scsi/scsi_lib.c.
>>>> >
>>>> > I replaced that part of the patch with this updated portion instead:
>>>
>>> Hmm, what are you applying against? Must be a clean tree, throw away any
>>> patches that you already applied in this thread.
>>>
>>> Updated below.
>> I'll re-pull everything fresh again from kernel.org and rebuild
>> with the "updated below" patch you posted. Thanks.
>
> Okay, fresh pull of everything from kernel.org,
> and now your latest patch does apply cleanly to -git13.
>
> Something weird (at this end).
Thanks for confirming, I did double check that my HEAD was uptodate -
and it is.
--
Jens Axboe
On Oct. 18, 2007, 16:05 +0200, Jens Axboe <[email protected]> wrote:
> On Thu, Oct 18 2007, Jens Axboe wrote:
>> On Thu, Oct 18 2007, Benny Halevy wrote:
>>>> return sg;
>>>> }
>>>> @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
>>>> ret = sg;
>>>>
>>>> #endif
>>>> +#ifdef CONFIG_DEBUG_SG
>>>> + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
>>> can it also do BUG_ON(!sg_is_last(sg))?
>> That would make sense, definitely. I'll add that.
>
> BUG_ON(!sg_is_last(ret));
>
> it should be, not sg. That's what I merged.
>
right. of course.
On Thu, Oct 18 2007, Benny Halevy wrote:
> On Oct. 18, 2007, 16:05 +0200, Jens Axboe <[email protected]> wrote:
> > On Thu, Oct 18 2007, Jens Axboe wrote:
> >> On Thu, Oct 18 2007, Benny Halevy wrote:
> >>>> return sg;
> >>>> }
> >>>> @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> >>>> ret = sg;
> >>>>
> >>>> #endif
> >>>> +#ifdef CONFIG_DEBUG_SG
> >>>> + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
> >>> can it also do BUG_ON(!sg_is_last(sg))?
> >> That would make sense, definitely. I'll add that.
> >
> > BUG_ON(!sg_is_last(ret));
> >
> > it should be, not sg. That's what I merged.
> >
> right. of course.
OK, that found something interesting - mapping a request may shrink it,
so we need to update the end marker to move it earlier in the list.
Basically just a
if (sg)
__sg_mark_end(sg);
at the bottom of blk_rq_map_sg(). Updated patch below, booted on other
machines now as well without incident.
diff --git a/arch/alpha/kernel/pci_iommu.c b/arch/alpha/kernel/pci_iommu.c
index e1c4707..094a95e 100644
--- a/arch/alpha/kernel/pci_iommu.c
+++ b/arch/alpha/kernel/pci_iommu.c
@@ -465,7 +465,7 @@ EXPORT_SYMBOL(pci_free_consistent);
Write dma_length of each leader with the combined lengths of
the mergable followers. */
-#define SG_ENT_VIRT_ADDRESS(SG) (page_address((SG)->page) + (SG)->offset)
+#define SG_ENT_VIRT_ADDRESS(SG) (page_address(sg_page((SG)) + (SG)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) __pa(SG_ENT_VIRT_ADDRESS(SG))
static void
diff --git a/arch/arm/common/dmabounce.c b/arch/arm/common/dmabounce.c
index 44ab0da..9d371e4 100644
--- a/arch/arm/common/dmabounce.c
+++ b/arch/arm/common/dmabounce.c
@@ -442,7 +442,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(dir == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- struct page *page = sg->page;
+ struct page *page = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int length = sg->length;
void *ptr = page_address(page) + offset;
diff --git a/arch/blackfin/kernel/dma-mapping.c b/arch/blackfin/kernel/dma-mapping.c
index 94d7b11..dbf7222 100644
--- a/arch/blackfin/kernel/dma-mapping.c
+++ b/arch/blackfin/kernel/dma-mapping.c
@@ -160,7 +160,7 @@ dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
BUG_ON(direction == DMA_NONE);
for (i = 0; i < nents; i++, sg++) {
- sg->dma_address = (dma_addr_t)(page_address(sg->page) +
+ sg->dma_address = (dma_addr_t)(page_address(sg_page(sg)) +
sg->offset);
invalidate_dcache_range(sg_dma_address(sg),
diff --git a/arch/ia64/hp/common/sba_iommu.c b/arch/ia64/hp/common/sba_iommu.c
index 3c95f41..0b74c63 100644
--- a/arch/ia64/hp/common/sba_iommu.c
+++ b/arch/ia64/hp/common/sba_iommu.c
@@ -246,7 +246,7 @@ static int reserve_sba_gart = 1;
static SBA_INLINE void sba_mark_invalid(struct ioc *, dma_addr_t, size_t);
static SBA_INLINE void sba_free_range(struct ioc *, dma_addr_t, size_t);
-#define sba_sg_address(sg) (page_address((sg)->page) + (sg)->offset)
+#define sba_sg_address(sg) (page_address(sg_page((sg)) + (sg)->offset))
#ifdef FULL_VALID_PDIR
static u64 prefetch_spill_page;
diff --git a/arch/ia64/hp/sim/simscsi.c b/arch/ia64/hp/sim/simscsi.c
index a3a558a..7f1794f 100644
--- a/arch/ia64/hp/sim/simscsi.c
+++ b/arch/ia64/hp/sim/simscsi.c
@@ -131,7 +131,7 @@ simscsi_sg_readwrite (struct scsi_cmnd *sc, int mode, unsigned long offset)
stat.fd = desc[sc->device->id];
scsi_for_each_sg(sc, sl, scsi_sg_count(sc), i) {
- req.addr = __pa(page_address(sl->page) + sl->offset);
+ req.addr = __pa(page_address(sg_page(sl)) + sl->offset);
req.len = sl->length;
if (DBG)
printk("simscsi_sg_%s @ %lx (off %lx) use_sg=%d len=%d\n",
@@ -212,7 +212,7 @@ static void simscsi_fillresult(struct scsi_cmnd *sc, char *buf, unsigned len)
if (!len)
break;
thislen = min(len, slp->length);
- memcpy(page_address(slp->page) + slp->offset, buf, thislen);
+ memcpy(page_address(sg_page(slp)) + slp->offset, buf, thislen);
len -= thislen;
}
}
diff --git a/arch/ia64/sn/pci/pci_dma.c b/arch/ia64/sn/pci/pci_dma.c
index ecd8a52..6048eaf 100644
--- a/arch/ia64/sn/pci/pci_dma.c
+++ b/arch/ia64/sn/pci/pci_dma.c
@@ -16,7 +16,7 @@
#include <asm/sn/pcidev.h>
#include <asm/sn/sn_sal.h>
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(SG) virt_to_phys(SG_ENT_VIRT_ADDRESS(SG))
/**
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index 9d4e4b5..b2d620b 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -121,7 +121,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
int i;
for (i = 0; i < nents; sg++, i++) {
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
dma_sync_single_for_device(dev, sg->dma_address, sg->length, dir);
}
return nents;
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index 98b5e5b..4aa143b 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -165,7 +165,7 @@ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents,
for (i = 0; i < nents; i++, sg++) {
unsigned long addr;
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (!plat_device_is_coherent(dev) && addr)
__dma_sync(addr + sg->offset, sg->length, direction);
sg->dma_address = plat_map_dma_mem(dev,
@@ -223,7 +223,7 @@ void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries,
for (i = 0; i < nhwentries; i++, sg++) {
if (!plat_device_is_coherent(dev) &&
direction != DMA_TO_DEVICE) {
- addr = (unsigned long) page_address(sg->page);
+ addr = (unsigned long) page_address(sg_page(sg));
if (addr)
__dma_sync(addr + sg->offset, sg->length,
direction);
@@ -304,7 +304,7 @@ void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg, int nelems,
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (cpu_is_noncoherent_r10000(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
@@ -322,7 +322,7 @@ void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg, int nele
/* Make sure that gcc doesn't leave the empty loop body. */
for (i = 0; i < nelems; i++, sg++) {
if (!plat_device_is_coherent(dev))
- __dma_sync((unsigned long)page_address(sg->page),
+ __dma_sync((unsigned long)page_address(sg_page(sg)),
sg->length, direction);
plat_unmap_dma_mem(sg->dma_address);
}
diff --git a/arch/powerpc/kernel/dma_64.c b/arch/powerpc/kernel/dma_64.c
index 9001104..5d0aa90 100644
--- a/arch/powerpc/kernel/dma_64.c
+++ b/arch/powerpc/kernel/dma_64.c
@@ -161,7 +161,7 @@ static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (page_to_phys(sg->page) + sg->offset) |
+ sg->dma_address = (page_to_phys(sg_page(sg)) + sg->offset) |
dma_direct_offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/ibmebus.c b/arch/powerpc/kernel/ibmebus.c
index 289d7e9..3d0e71b 100644
--- a/arch/powerpc/kernel/ibmebus.c
+++ b/arch/powerpc/kernel/ibmebus.c
@@ -102,7 +102,7 @@ static int ibmebus_map_sg(struct device *dev,
int i;
for_each_sg(sgl, sg, nents, i) {
- sg->dma_address = (dma_addr_t)page_address(sg->page)
+ sg->dma_address = (dma_addr_t)page_address(sg_page(sg))
+ sg->offset;
sg->dma_length = sg->length;
}
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index 306a6f7..47d745a 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -307,7 +307,7 @@ int iommu_map_sg(struct iommu_table *tbl, struct scatterlist *sglist,
continue;
}
/* Allocate iommu entries for that segment */
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(sg_page(s)) + s->offset;
npages = iommu_num_pages(vaddr, slen);
entry = iommu_range_alloc(tbl, npages, &handle, mask >> IOMMU_PAGE_SHIFT, 0);
diff --git a/arch/powerpc/platforms/ps3/system-bus.c b/arch/powerpc/platforms/ps3/system-bus.c
index 07e64b4..e8202b6 100644
--- a/arch/powerpc/platforms/ps3/system-bus.c
+++ b/arch/powerpc/platforms/ps3/system-bus.c
@@ -629,7 +629,7 @@ static int ps3_sb_map_sg(struct device *_dev, struct scatterlist *sgl,
for_each_sg(sgl, sg, nents, i) {
int result = ps3_dma_map(dev->d_region,
- page_to_phys(sg->page) + sg->offset, sg->length,
+ page_to_phys(sg_page(sg)) + sg->offset, sg->length,
&sg->dma_address, 0);
if (result) {
diff --git a/arch/sparc/kernel/ioport.c b/arch/sparc/kernel/ioport.c
index 9c3ed88..abc4470 100644
--- a/arch/sparc/kernel/ioport.c
+++ b/arch/sparc/kernel/ioport.c
@@ -727,9 +727,9 @@ int pci_map_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
/* IIep is write-through, not flushing. */
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
sg->dvma_address =
- virt_to_phys(page_address(sg->page)) + sg->offset;
+ virt_to_phys(page_address(sg_page(sg))) + sg->offset;
sg->dvma_length = sg->length;
}
return nents;
@@ -748,9 +748,9 @@ void pci_unmap_sg(struct pci_dev *hwdev, struct scatterlist *sgl, int nents,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -798,9 +798,9 @@ void pci_dma_sync_sg_for_cpu(struct pci_dev *hwdev, struct scatterlist *sgl, int
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
@@ -814,9 +814,9 @@ void pci_dma_sync_sg_for_device(struct pci_dev *hwdev, struct scatterlist *sgl,
BUG_ON(direction == PCI_DMA_NONE);
if (direction != PCI_DMA_TODEVICE) {
for_each_sg(sgl, sg, nents, n) {
- BUG_ON(page_address(sg->page) == NULL);
+ BUG_ON(page_address(sg_page(sg)) == NULL);
mmu_inval_dma_area(
- (unsigned long) page_address(sg->page),
+ (unsigned long) page_address(sg_page(sg)),
(sg->length + PAGE_SIZE-1) & PAGE_MASK);
}
}
diff --git a/arch/sparc/mm/io-unit.c b/arch/sparc/mm/io-unit.c
index 375b4db..0f2ad57 100644
--- a/arch/sparc/mm/io-unit.c
+++ b/arch/sparc/mm/io-unit.c
@@ -144,7 +144,7 @@ static void iounit_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus
spin_lock_irqsave(&iounit->lock, flags);
while (sz != 0) {
--sz;
- sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = iounit_get_area(iounit, (unsigned long)page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/iommu.c b/arch/sparc/mm/iommu.c
index 283656d..4b93427 100644
--- a/arch/sparc/mm/iommu.c
+++ b/arch/sparc/mm/iommu.c
@@ -238,7 +238,7 @@ static void iommu_get_scsi_sgl_noflush(struct scatterlist *sg, int sz, struct sb
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -252,7 +252,7 @@ static void iommu_get_scsi_sgl_gflush(struct scatterlist *sg, int sz, struct sbu
while (sz != 0) {
--sz;
n = (sg->length + sg->offset + PAGE_SIZE-1) >> PAGE_SHIFT;
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
@@ -273,7 +273,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
* XXX Is this a good assumption?
* XXX What if someone else unmaps it here and races us?
*/
- if ((page = (unsigned long) page_address(sg->page)) != 0) {
+ if ((page = (unsigned long) page_address(sg_page(sg))) != 0) {
for (i = 0; i < n; i++) {
if (page != oldpage) { /* Already flushed? */
flush_page_for_dma(page);
@@ -283,7 +283,7 @@ static void iommu_get_scsi_sgl_pflush(struct scatterlist *sg, int sz, struct sbu
}
}
- sg->dvma_address = iommu_get_one(sg->page, n, sbus) + sg->offset;
+ sg->dvma_address = iommu_get_one(sg_page(sg), n, sbus) + sg->offset;
sg->dvma_length = (__u32) sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc/mm/sun4c.c b/arch/sparc/mm/sun4c.c
index ee6708f..f172dbe 100644
--- a/arch/sparc/mm/sun4c.c
+++ b/arch/sparc/mm/sun4c.c
@@ -1228,7 +1228,7 @@ static void sun4c_get_scsi_sgl(struct scatterlist *sg, int sz, struct sbus_bus *
{
while (sz != 0) {
--sz;
- sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg->page) + sg->offset, sg->length);
+ sg->dvma_address = (__u32)sun4c_lockarea(page_address(sg_page(sg)) + sg->offset, sg->length);
sg->dvma_length = sg->length;
sg = sg_next(sg);
}
diff --git a/arch/sparc64/kernel/iommu.c b/arch/sparc64/kernel/iommu.c
index 29af777..73852a2 100644
--- a/arch/sparc64/kernel/iommu.c
+++ b/arch/sparc64/kernel/iommu.c
@@ -473,7 +473,7 @@ static void dma_4u_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static void fill_sg(iopte_t *iopte, struct scatterlist *sg,
int nused, int nelems,
@@ -566,7 +566,7 @@ static int dma_4u_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4u_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/sparc64/kernel/iommu_common.c b/arch/sparc64/kernel/iommu_common.c
index d7ca900..ec863e0 100644
--- a/arch/sparc64/kernel/iommu_common.c
+++ b/arch/sparc64/kernel/iommu_common.c
@@ -73,7 +73,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
daddr = dma_sg->dma_address;
sglen = sg->length;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
while (dlen > 0) {
unsigned long paddr;
@@ -123,7 +123,7 @@ static int verify_one_map(struct scatterlist *dma_sg, struct scatterlist **__sg,
sg = sg_next(sg);
if (--nents <= 0)
break;
- sgaddr = (unsigned long) (page_address(sg->page) + sg->offset);
+ sgaddr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
sglen = sg->length;
}
if (dlen < 0) {
@@ -191,7 +191,7 @@ void verify_sglist(struct scatterlist *sglist, int nents, iopte_t *iopte, int np
printk("sg(%d): page_addr(%p) off(%x) length(%x) "
"dma_address[%016x] dma_length[%016x]\n",
i,
- page_address(sg->page), sg->offset,
+ page_address(sg_page(sg)), sg->offset,
sg->length,
sg->dma_address, sg->dma_length);
}
@@ -207,15 +207,15 @@ unsigned long prepare_sg(struct scatterlist *sg, int nents)
unsigned long prev;
u32 dent_addr, dent_len;
- prev = (unsigned long) (page_address(sg->page) + sg->offset);
+ prev = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
prev += (unsigned long) (dent_len = sg->length);
- dent_addr = (u32) ((unsigned long)(page_address(sg->page) + sg->offset)
+ dent_addr = (u32) ((unsigned long)(page_address(sg_page(sg)) + sg->offset)
& (IO_PAGE_SIZE - 1UL));
while (--nents) {
unsigned long addr;
sg = sg_next(sg);
- addr = (unsigned long) (page_address(sg->page) + sg->offset);
+ addr = (unsigned long) (page_address(sg_page(sg)) + sg->offset);
if (! VCONTIG(prev, addr)) {
dma_sg->dma_address = dent_addr;
dma_sg->dma_length = dent_len;
diff --git a/arch/sparc64/kernel/ldc.c b/arch/sparc64/kernel/ldc.c
index 85a2be0..c8313cb 100644
--- a/arch/sparc64/kernel/ldc.c
+++ b/arch/sparc64/kernel/ldc.c
@@ -2057,7 +2057,7 @@ static void fill_cookies(struct cookie_state *sp, unsigned long pa,
static int sg_count_one(struct scatterlist *sg)
{
- unsigned long base = page_to_pfn(sg->page) << PAGE_SHIFT;
+ unsigned long base = page_to_pfn(sg_page(sg)) << PAGE_SHIFT;
long len = sg->length;
if ((sg->offset | len) & (8UL - 1))
diff --git a/arch/sparc64/kernel/pci_sun4v.c b/arch/sparc64/kernel/pci_sun4v.c
index fe46ace..5324a34 100644
--- a/arch/sparc64/kernel/pci_sun4v.c
+++ b/arch/sparc64/kernel/pci_sun4v.c
@@ -366,7 +366,7 @@ static void dma_4v_unmap_single(struct device *dev, dma_addr_t bus_addr,
}
#define SG_ENT_PHYS_ADDRESS(SG) \
- (__pa(page_address((SG)->page)) + (SG)->offset)
+ (__pa(page_address(sg_page(SG))) + (SG)->offset)
static long fill_sg(long entry, struct device *dev,
struct scatterlist *sg,
@@ -478,7 +478,7 @@ static int dma_4v_map_sg(struct device *dev, struct scatterlist *sglist,
if (nelems == 1) {
sglist->dma_address =
dma_4v_map_single(dev,
- (page_address(sglist->page) +
+ (page_address(sg_page(sglist)) +
sglist->offset),
sglist->length, direction);
if (unlikely(sglist->dma_address == DMA_ERROR_CODE))
diff --git a/arch/um/drivers/ubd_kern.c b/arch/um/drivers/ubd_kern.c
index 25b248a..3a8cd3d 100644
--- a/arch/um/drivers/ubd_kern.c
+++ b/arch/um/drivers/ubd_kern.c
@@ -1115,7 +1115,7 @@ static void do_ubd_request(struct request_queue *q)
}
prepare_request(req, io_req,
(unsigned long long) req->sector << 9,
- sg->offset, sg->length, sg->page);
+ sg->offset, sg->length, sg_page(sg));
last_sectors = sg->length >> 9;
n = os_write_file(thread_fd, &io_req,
diff --git a/arch/x86/kernel/pci-calgary_64.c b/arch/x86/kernel/pci-calgary_64.c
index 5098f58..c9f0eb0 100644
--- a/arch/x86/kernel/pci-calgary_64.c
+++ b/arch/x86/kernel/pci-calgary_64.c
@@ -411,8 +411,10 @@ static int calgary_nontranslate_map_sg(struct device* dev,
int i;
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
+ s->dma_address = virt_to_bus(page_address(p) + s->offset);
s->dma_length = s->length;
}
return nelems;
@@ -432,9 +434,11 @@ static int calgary_map_sg(struct device *dev, struct scatterlist *sg,
return calgary_nontranslate_map_sg(dev, sg, nelems, direction);
for_each_sg(sg, s, nelems, i) {
- BUG_ON(!s->page);
+ struct page *p = sg_page(s);
+
+ BUG_ON(!p);
- vaddr = (unsigned long)page_address(s->page) + s->offset;
+ vaddr = (unsigned long)page_address(p) + s->offset;
npages = num_dma_pages(vaddr, s->length);
entry = iommu_range_alloc(tbl, npages);
diff --git a/arch/x86/kernel/pci-gart_64.c b/arch/x86/kernel/pci-gart_64.c
index 5cdfab6..daaf636 100644
--- a/arch/x86/kernel/pci-gart_64.c
+++ b/arch/x86/kernel/pci-gart_64.c
@@ -302,7 +302,7 @@ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg,
#endif
for_each_sg(sg, s, nents, i) {
- unsigned long addr = page_to_phys(s->page) + s->offset;
+ unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
if (nonforced_iommu(dev, addr, s->length)) {
addr = dma_map_area(dev, addr, s->length, dir);
if (addr == bad_dma_address) {
@@ -397,7 +397,7 @@ static int gart_map_sg(struct device *dev, struct scatterlist *sg, int nents,
start_sg = sgmap = sg;
ps = NULL; /* shut up gcc */
for_each_sg(sg, s, nents, i) {
- dma_addr_t addr = page_to_phys(s->page) + s->offset;
+ dma_addr_t addr = page_to_phys(sg_page(s)) + s->offset;
s->dma_address = addr;
BUG_ON(s->length == 0);
diff --git a/arch/x86/kernel/pci-nommu_64.c b/arch/x86/kernel/pci-nommu_64.c
index e85d436..d64a4a5 100644
--- a/arch/x86/kernel/pci-nommu_64.c
+++ b/arch/x86/kernel/pci-nommu_64.c
@@ -62,8 +62,8 @@ static int nommu_map_sg(struct device *hwdev, struct scatterlist *sg,
int i;
for_each_sg(sg, s, nents, i) {
- BUG_ON(!s->page);
- s->dma_address = virt_to_bus(page_address(s->page) +s->offset);
+ BUG_ON(!sg_page(s));
+ s->dma_address = virt_to_bus(page_address(sg_page(s)) +s->offset);
if (!check_addr("map_sg", hwdev, s->dma_address, s->length))
return 0;
s->dma_length = s->length;
diff --git a/block/ll_rw_blk.c b/block/ll_rw_blk.c
index 3935469..37ed29b 100644
--- a/block/ll_rw_blk.c
+++ b/block/ll_rw_blk.c
@@ -1354,8 +1354,9 @@ new_segment:
else
sg = sg_next(sg);
- memset(sg, 0, sizeof(*sg));
- sg->page = bvec->bv_page;
+ sg_dma_len(sg) = 0;
+ sg_dma_address(sg) = 0;
+ sg_set_page(sg, bvec->bv_page);
sg->length = nbytes;
sg->offset = bvec->bv_offset;
nsegs++;
@@ -1363,6 +1364,9 @@ new_segment:
bvprv = bvec;
} /* segments in rq */
+ if (sg)
+ __sg_mark_end(sg);
+
return nsegs;
}
diff --git a/crypto/digest.c b/crypto/digest.c
index e56de67..8871dec 100644
--- a/crypto/digest.c
+++ b/crypto/digest.c
@@ -41,7 +41,7 @@ static int update2(struct hash_desc *desc,
return 0;
for (;;) {
- struct page *pg = sg->page;
+ struct page *pg = sg_page(sg);
unsigned int offset = sg->offset;
unsigned int l = sg->length;
diff --git a/crypto/hmac.c b/crypto/hmac.c
index 8802fb6..e4eb6ac 100644
--- a/crypto/hmac.c
+++ b/crypto/hmac.c
@@ -159,7 +159,8 @@ static int hmac_digest(struct hash_desc *pdesc, struct scatterlist *sg,
desc.flags = pdesc->flags & CRYPTO_TFM_REQ_MAY_SLEEP;
sg_set_buf(sg1, ipad, bs);
- sg1[1].page = (void *)sg;
+
+ sg_set_page(&sg[1], (void *) sg);
sg1[1].length = 0;
sg_set_buf(sg2, opad, bs + ds);
diff --git a/crypto/scatterwalk.c b/crypto/scatterwalk.c
index d6852c3..b9bbda0 100644
--- a/crypto/scatterwalk.c
+++ b/crypto/scatterwalk.c
@@ -54,7 +54,7 @@ static void scatterwalk_pagedone(struct scatter_walk *walk, int out,
if (out) {
struct page *page;
- page = walk->sg->page + ((walk->offset - 1) >> PAGE_SHIFT);
+ page = sg_page(walk->sg) + ((walk->offset - 1) >> PAGE_SHIFT);
flush_dcache_page(page);
}
diff --git a/crypto/scatterwalk.h b/crypto/scatterwalk.h
index 9c73e37..87ed681 100644
--- a/crypto/scatterwalk.h
+++ b/crypto/scatterwalk.h
@@ -22,13 +22,13 @@
static inline struct scatterlist *scatterwalk_sg_next(struct scatterlist *sg)
{
- return (++sg)->length ? sg : (void *)sg->page;
+ return (++sg)->length ? sg : (void *) sg_page(sg);
}
static inline unsigned long scatterwalk_samebuf(struct scatter_walk *walk_in,
struct scatter_walk *walk_out)
{
- return !(((walk_in->sg->page - walk_out->sg->page) << PAGE_SHIFT) +
+ return !(((sg_page(walk_in->sg) - sg_page(walk_out->sg)) << PAGE_SHIFT) +
(int)(walk_in->offset - walk_out->offset));
}
@@ -60,7 +60,7 @@ static inline unsigned int scatterwalk_aligned(struct scatter_walk *walk,
static inline struct page *scatterwalk_page(struct scatter_walk *walk)
{
- return walk->sg->page + (walk->offset >> PAGE_SHIFT);
+ return sg_page(walk->sg) + (walk->offset >> PAGE_SHIFT);
}
static inline void scatterwalk_unmap(void *vaddr, int out)
diff --git a/crypto/tcrypt.c b/crypto/tcrypt.c
index 18d489c..d741c63 100644
--- a/crypto/tcrypt.c
+++ b/crypto/tcrypt.c
@@ -317,7 +317,7 @@ static void test_cipher(char *algo, int enc,
goto out;
}
- q = kmap(sg[0].page) + sg[0].offset;
+ q = kmap(sg_page(&sg[0])) + sg[0].offset;
hexdump(q, cipher_tv[i].rlen);
printk("%s\n",
@@ -390,7 +390,7 @@ static void test_cipher(char *algo, int enc,
temp = 0;
for (k = 0; k < cipher_tv[i].np; k++) {
printk("page %u\n", k);
- q = kmap(sg[k].page) + sg[k].offset;
+ q = kmap(sg_page(&sg[k])) + sg[k].offset;
hexdump(q, cipher_tv[i].tap[k]);
printk("%s\n",
memcmp(q, cipher_tv[i].result + temp,
diff --git a/crypto/xcbc.c b/crypto/xcbc.c
index 9f502b8..ac68f3b 100644
--- a/crypto/xcbc.c
+++ b/crypto/xcbc.c
@@ -120,7 +120,7 @@ static int crypto_xcbc_digest_update2(struct hash_desc *pdesc,
do {
- struct page *pg = sg[i].page;
+ struct page *pg = sg_page(&sg[i]);
unsigned int offset = sg[i].offset;
unsigned int slen = sg[i].length;
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index bbaa545..b1fa70a 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -4296,7 +4296,7 @@ void ata_sg_clean(struct ata_queued_cmd *qc)
sg_last(sg, qc->orig_n_elem)->length += qc->pad_len;
if (pad_buf) {
struct scatterlist *psg = &qc->pad_sgent;
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(addr + psg->offset, pad_buf, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4686,11 +4686,11 @@ static int ata_sg_setup(struct ata_queued_cmd *qc)
* data in this function or read data in ata_sg_clean.
*/
offset = lsg->offset + lsg->length - qc->pad_len;
- psg->page = nth_page(lsg->page, offset >> PAGE_SHIFT);
+ sg_set_page(psg, nth_page(sg_page(lsg), offset >> PAGE_SHIFT));
psg->offset = offset_in_page(offset);
if (qc->tf.flags & ATA_TFLAG_WRITE) {
- void *addr = kmap_atomic(psg->page, KM_IRQ0);
+ void *addr = kmap_atomic(sg_page(psg), KM_IRQ0);
memcpy(pad_buf, addr + psg->offset, qc->pad_len);
kunmap_atomic(addr, KM_IRQ0);
}
@@ -4836,7 +4836,7 @@ static void ata_pio_sector(struct ata_queued_cmd *qc)
if (qc->curbytes == qc->nbytes - qc->sect_size)
ap->hsm_task_state = HSM_ST_LAST;
- page = qc->cursg->page;
+ page = sg_page(qc->cursg);
offset = qc->cursg->offset + qc->cursg_ofs;
/* get the current page and offset */
@@ -4988,7 +4988,7 @@ next_sg:
sg = qc->cursg;
- page = sg->page;
+ page = sg_page(sg);
offset = sg->offset + qc->cursg_ofs;
/* get the current page and offset */
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9fbb39c..5b758b9 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1544,7 +1544,7 @@ static unsigned int ata_scsi_rbuf_get(struct scsi_cmnd *cmd, u8 **buf_out)
struct scatterlist *sg = scsi_sglist(cmd);
if (sg) {
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
buflen = sg->length;
} else {
buf = NULL;
diff --git a/drivers/block/cciss.c b/drivers/block/cciss.c
index 7c2cfde..5a6fe17 100644
--- a/drivers/block/cciss.c
+++ b/drivers/block/cciss.c
@@ -2610,7 +2610,7 @@ static void do_cciss_request(struct request_queue *q)
(int)creq->nr_sectors);
#endif /* CCISS_DEBUG */
- memset(tmp_sg, 0, sizeof(tmp_sg));
+ sg_init_table(tmp_sg, MAXSGENTRIES);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* get the DMA records for the setup */
@@ -2621,7 +2621,7 @@ static void do_cciss_request(struct request_queue *q)
for (i = 0; i < seg; i++) {
c->SG[i].Len = tmp_sg[i].length;
- temp64.val = (__u64) pci_map_page(h->pdev, tmp_sg[i].page,
+ temp64.val = (__u64) pci_map_page(h->pdev, sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
c->SG[i].Addr.lower = temp64.val32.lower;
diff --git a/drivers/block/cpqarray.c b/drivers/block/cpqarray.c
index 568603d..efab27f 100644
--- a/drivers/block/cpqarray.c
+++ b/drivers/block/cpqarray.c
@@ -918,6 +918,7 @@ queue_next:
DBGPX(
printk("sector=%d, nr_sectors=%d\n", creq->sector, creq->nr_sectors);
);
+ sg_init_table(tmp_sg, SG_MAX);
seg = blk_rq_map_sg(q, creq, tmp_sg);
/* Now do all the DMA Mappings */
@@ -929,7 +930,7 @@ DBGPX(
{
c->req.sg[i].size = tmp_sg[i].length;
c->req.sg[i].addr = (__u32) pci_map_page(h->pci_dev,
- tmp_sg[i].page,
+ sg_page(&tmp_sg[i]),
tmp_sg[i].offset,
tmp_sg[i].length, dir);
}
diff --git a/drivers/block/cryptoloop.c b/drivers/block/cryptoloop.c
index 4053503..1b58b01 100644
--- a/drivers/block/cryptoloop.c
+++ b/drivers/block/cryptoloop.c
@@ -26,6 +26,7 @@
#include <linux/crypto.h>
#include <linux/blkdev.h>
#include <linux/loop.h>
+#include <linux/scatterlist.h>
#include <asm/semaphore.h>
#include <asm/uaccess.h>
@@ -119,14 +120,17 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
.tfm = tfm,
.flags = CRYPTO_TFM_REQ_MAY_SLEEP,
};
- struct scatterlist sg_out = { NULL, };
- struct scatterlist sg_in = { NULL, };
+ struct scatterlist sg_out;
+ struct scatterlist sg_in;
encdec_cbc_t encdecfunc;
struct page *in_page, *out_page;
unsigned in_offs, out_offs;
int err;
+ sg_init_table(&sg_out, 1);
+ sg_init_table(&sg_in, 1);
+
if (cmd == READ) {
in_page = raw_page;
in_offs = raw_off;
@@ -146,11 +150,11 @@ cryptoloop_transfer(struct loop_device *lo, int cmd,
u32 iv[4] = { 0, };
iv[0] = cpu_to_le32(IV & 0xffffffff);
- sg_in.page = in_page;
+ sg_set_page(&sg_in, in_page);
sg_in.offset = in_offs;
sg_in.length = sz;
- sg_out.page = out_page;
+ sg_set_page(&sg_out, out_page);
sg_out.offset = out_offs;
sg_out.length = sz;
diff --git a/drivers/block/ub.c b/drivers/block/ub.c
index c57dd2b..d6f7b43 100644
--- a/drivers/block/ub.c
+++ b/drivers/block/ub.c
@@ -25,6 +25,7 @@
#include <linux/usb_usual.h>
#include <linux/blkdev.h>
#include <linux/timer.h>
+#include <linux/scatterlist.h>
#include <scsi/scsi.h>
#define DRV_NAME "ub"
@@ -656,6 +657,7 @@ static int ub_request_fn_1(struct ub_lun *lun, struct request *rq)
if ((cmd = ub_get_cmd(lun)) == NULL)
return -1;
memset(cmd, 0, sizeof(struct ub_scsi_cmd));
+ sg_init_table(cmd->sgv, UB_MAX_REQ_SG);
blkdev_dequeue_request(rq);
@@ -1310,7 +1312,7 @@ static void ub_data_start(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
pipe = sc->send_bulk_pipe;
sc->last_pipe = pipe;
usb_fill_bulk_urb(&sc->work_urb, sc->dev, pipe,
- page_address(sg->page) + sg->offset, sg->length,
+ page_address(sg_page(sg)) + sg->offset, sg->length,
ub_urb_complete, sc);
sc->work_urb.actual_length = 0;
sc->work_urb.error_count = 0;
@@ -1427,7 +1429,7 @@ static void ub_state_sense(struct ub_dev *sc, struct ub_scsi_cmd *cmd)
scmd->state = UB_CMDST_INIT;
scmd->nsg = 1;
sg = &scmd->sgv[0];
- sg->page = virt_to_page(sc->top_sense);
+ sg_set_page(sg, virt_to_page(sc->top_sense));
sg->offset = (unsigned long)sc->top_sense & (PAGE_SIZE-1);
sg->length = UB_SENSE_SIZE;
scmd->len = UB_SENSE_SIZE;
@@ -1863,7 +1865,7 @@ static int ub_sync_read_cap(struct ub_dev *sc, struct ub_lun *lun,
cmd->state = UB_CMDST_INIT;
cmd->nsg = 1;
sg = &cmd->sgv[0];
- sg->page = virt_to_page(p);
+ sg_set_page(sg, virt_to_page(p));
sg->offset = (unsigned long)p & (PAGE_SIZE-1);
sg->length = 8;
cmd->len = 8;
diff --git a/drivers/ide/cris/ide-cris.c b/drivers/ide/cris/ide-cris.c
index 06c75f1..3979bf4 100644
--- a/drivers/ide/cris/ide-cris.c
+++ b/drivers/ide/cris/ide-cris.c
@@ -934,11 +934,11 @@ static int cris_ide_build_dmatable (ide_drive_t *drive)
* than two possibly non-adjacent physical 4kB pages.
*/
/* group sequential buffers into one large buffer */
- addr = page_to_phys(sg->page) + sg->offset;
+ addr = page_to_phys(sg_page(sg)) + sg->offset;
size = sg_dma_len(sg);
while (--i) {
sg = sg_next(sg);
- if ((addr + size) != page_to_phys(sg->page) + sg->offset)
+ if ((addr + size) != page_to_phys(sg_page(sg)) + sg->offset)
break;
size += sg_dma_len(sg);
}
diff --git a/drivers/ide/ide-probe.c b/drivers/ide/ide-probe.c
index 3c945d6..7c640cd 100644
--- a/drivers/ide/ide-probe.c
+++ b/drivers/ide/ide-probe.c
@@ -1337,12 +1337,14 @@ static int hwif_init(ide_hwif_t *hwif)
if (!hwif->sg_max_nents)
hwif->sg_max_nents = PRD_ENTRIES;
- hwif->sg_table = kzalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
+ hwif->sg_table = kmalloc(sizeof(struct scatterlist)*hwif->sg_max_nents,
GFP_KERNEL);
if (!hwif->sg_table) {
printk(KERN_ERR "%s: unable to allocate SG table.\n", hwif->name);
goto out;
}
+
+ sg_init_table(hwif->sg_table, hwif->sg_max_nents);
if (init_irq(hwif) == 0)
goto done;
diff --git a/drivers/ide/ide-taskfile.c b/drivers/ide/ide-taskfile.c
index 2a3c8d4..086157f 100644
--- a/drivers/ide/ide-taskfile.c
+++ b/drivers/ide/ide-taskfile.c
@@ -278,7 +278,7 @@ static void ide_pio_sector(ide_drive_t *drive, unsigned int write)
hwif->cursg = sg;
}
- page = cursg->page;
+ page = sg_page(cursg);
offset = cursg->offset + hwif->cursg_ofs * SECTOR_SIZE;
/* get the current page and offset */
diff --git a/drivers/ide/mips/au1xxx-ide.c b/drivers/ide/mips/au1xxx-ide.c
index 47c035a..1b22ab7 100644
--- a/drivers/ide/mips/au1xxx-ide.c
+++ b/drivers/ide/mips/au1xxx-ide.c
@@ -276,7 +276,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
if (iswrite) {
if(!put_source_flags(ahwif->tx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
@@ -285,7 +285,7 @@ static int auide_build_dmatable(ide_drive_t *drive)
} else
{
if(!put_dest_flags(ahwif->rx_chan,
- (void*)(page_address(sg->page)
+ (void*)(page_address(sg_page(sg))
+ sg->offset),
tc, flags)) {
printk(KERN_ERR "%s failed %d\n",
diff --git a/drivers/ieee1394/dma.c b/drivers/ieee1394/dma.c
index 45d6055..25e113b 100644
--- a/drivers/ieee1394/dma.c
+++ b/drivers/ieee1394/dma.c
@@ -111,7 +111,7 @@ int dma_region_alloc(struct dma_region *dma, unsigned long n_bytes,
unsigned long va =
(unsigned long)dma->kvirt + (i << PAGE_SHIFT);
- dma->sglist[i].page = vmalloc_to_page((void *)va);
+ sg_set_page(&dma->sglist[i], vmalloc_to_page((void *)va));
dma->sglist[i].length = PAGE_SIZE;
}
diff --git a/drivers/ieee1394/sbp2.c b/drivers/ieee1394/sbp2.c
index 1b353b9..d5dfe11 100644
--- a/drivers/ieee1394/sbp2.c
+++ b/drivers/ieee1394/sbp2.c
@@ -1466,7 +1466,7 @@ static void sbp2_prep_command_orb_sg(struct sbp2_command_orb *orb,
cmd->dma_size = sgpnt[0].length;
cmd->dma_type = CMD_DMA_PAGE;
cmd->cmd_dma = dma_map_page(hi->host->device.parent,
- sgpnt[0].page, sgpnt[0].offset,
+ sg_page(&sgpnt[0]), sgpnt[0].offset,
cmd->dma_size, cmd->dma_dir);
orb->data_descriptor_lo = cmd->cmd_dma;
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 2f54e29..14159ff 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -55,9 +55,11 @@ static void __ib_umem_release(struct ib_device *dev, struct ib_umem *umem, int d
ib_dma_unmap_sg(dev, chunk->page_list,
chunk->nents, DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->nents; ++i) {
+ struct page *page = sg_page(&chunk->page_list[i]);
+
if (umem->writable && dirty)
- set_page_dirty_lock(chunk->page_list[i].page);
- put_page(chunk->page_list[i].page);
+ set_page_dirty_lock(page);
+ put_page(page);
}
kfree(chunk);
@@ -164,11 +166,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
}
chunk->nents = min_t(int, ret, IB_UMEM_MAX_PAGE_CHUNK);
+ sg_init_table(chunk->page_list, chunk->nents);
for (i = 0; i < chunk->nents; ++i) {
if (vma_list &&
!is_vm_hugetlb_page(vma_list[i + off]))
umem->hugetlb = 0;
- chunk->page_list[i].page = page_list[i + off];
+ sg_set_page(&chunk->page_list[i], page_list[i + off]);
chunk->page_list[i].offset = 0;
chunk->page_list[i].length = PAGE_SIZE;
}
@@ -179,7 +182,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
DMA_BIDIRECTIONAL);
if (chunk->nmap <= 0) {
for (i = 0; i < chunk->nents; ++i)
- put_page(chunk->page_list[i].page);
+ put_page(sg_page(&chunk->page_list[i]));
kfree(chunk);
ret = -ENOMEM;
diff --git a/drivers/infiniband/hw/ipath/ipath_dma.c b/drivers/infiniband/hw/ipath/ipath_dma.c
index 22709a4..e90a0ea 100644
--- a/drivers/infiniband/hw/ipath/ipath_dma.c
+++ b/drivers/infiniband/hw/ipath/ipath_dma.c
@@ -108,7 +108,7 @@ static int ipath_map_sg(struct ib_device *dev, struct scatterlist *sgl,
BUG_ON(!valid_dma_direction(direction));
for_each_sg(sgl, sg, nents, i) {
- addr = (u64) page_address(sg->page);
+ addr = (u64) page_address(sg_page(sg));
/* TODO: handle highmem pages */
if (!addr) {
ret = 0;
@@ -127,7 +127,7 @@ static void ipath_unmap_sg(struct ib_device *dev,
static u64 ipath_sg_dma_address(struct ib_device *dev, struct scatterlist *sg)
{
- u64 addr = (u64) page_address(sg->page);
+ u64 addr = (u64) page_address(sg_page(sg));
if (addr)
addr += sg->offset;
diff --git a/drivers/infiniband/hw/ipath/ipath_mr.c b/drivers/infiniband/hw/ipath/ipath_mr.c
index e442470..db4ba92 100644
--- a/drivers/infiniband/hw/ipath/ipath_mr.c
+++ b/drivers/infiniband/hw/ipath/ipath_mr.c
@@ -225,7 +225,7 @@ struct ib_mr *ipath_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
for (i = 0; i < chunk->nents; i++) {
void *vaddr;
- vaddr = page_address(chunk->page_list[i].page);
+ vaddr = page_address(sg_page(&chunk->page_list[i]));
if (!vaddr) {
ret = ERR_PTR(-EINVAL);
goto bail;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index e61f3e6..007b381 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -71,7 +71,7 @@ static void mthca_free_icm_pages(struct mthca_dev *dev, struct mthca_icm_chunk *
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -81,7 +81,7 @@ static void mthca_free_icm_coherent(struct mthca_dev *dev, struct mthca_icm_chun
for (i = 0; i < chunk->npages; ++i) {
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
}
@@ -107,10 +107,13 @@ void mthca_free_icm(struct mthca_dev *dev, struct mthca_icm *icm, int coherent)
static int mthca_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -157,6 +160,7 @@ struct mthca_icm *mthca_alloc_icm(struct mthca_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MTHCA_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -304,7 +308,7 @@ void *mthca_table_find(struct mthca_icm_table *table, int obj, dma_addr_t *dma_h
* so if we found the page, dma_handle has already
* been assigned to. */
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
@@ -445,6 +449,7 @@ static u64 mthca_uarc_virt(struct mthca_dev *dev, struct mthca_uar *uar, int pag
int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
struct mthca_user_db_table *db_tab, int index, u64 uaddr)
{
+ struct page *pages[1];
int ret = 0;
u8 status;
int i;
@@ -472,16 +477,17 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
}
ret = get_user_pages(current, current->mm, uaddr & PAGE_MASK, 1, 1, 0,
- &db_tab->page[i].mem.page, NULL);
+ pages, NULL);
if (ret < 0)
goto out;
+ sg_set_page(&db_tab->page[i].mem, pages[0]);
db_tab->page[i].mem.length = MTHCA_ICM_PAGE_SIZE;
db_tab->page[i].mem.offset = uaddr & ~PAGE_MASK;
ret = pci_map_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
if (ret < 0) {
- put_page(db_tab->page[i].mem.page);
+ put_page(pages[0]);
goto out;
}
@@ -491,7 +497,7 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
ret = -EINVAL;
if (ret) {
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
goto out;
}
@@ -557,7 +563,7 @@ void mthca_cleanup_user_db_tab(struct mthca_dev *dev, struct mthca_uar *uar,
if (db_tab->page[i].uvirt) {
mthca_UNMAP_ICM(dev, mthca_uarc_virt(dev, uar, i), 1, &status);
pci_unmap_sg(dev->pdev, &db_tab->page[i].mem, 1, PCI_DMA_TODEVICE);
- put_page(db_tab->page[i].mem.page);
+ put_page(sg_page(&db_tab->page[i].mem));
}
}
diff --git a/drivers/infiniband/ulp/iser/iser_memory.c b/drivers/infiniband/ulp/iser/iser_memory.c
index f3529b6..813b96b 100644
--- a/drivers/infiniband/ulp/iser/iser_memory.c
+++ b/drivers/infiniband/ulp/iser/iser_memory.c
@@ -131,7 +131,7 @@ static int iser_start_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, data->size, i) {
- from = kmap_atomic(sg->page, KM_USER0);
+ from = kmap_atomic(sg_page(sg), KM_USER0);
memcpy(p,
from + sg->offset,
sg->length);
@@ -191,7 +191,7 @@ void iser_finalize_rdma_unaligned_sg(struct iscsi_iser_cmd_task *iser_ctask,
p = mem;
for_each_sg(sgl, sg, sg_size, i) {
- to = kmap_atomic(sg->page, KM_SOFTIRQ0);
+ to = kmap_atomic(sg_page(sg), KM_SOFTIRQ0);
memcpy(to + sg->offset,
p,
sg->length);
@@ -300,7 +300,7 @@ static unsigned int iser_data_buf_aligned_len(struct iser_data_buf *data,
for_each_sg(sgl, sg, data->dma_nents, i) {
/* iser_dbg("Checking sg iobuf [%d]: phys=0x%08lX "
"offset: %ld sz: %ld\n", i,
- (unsigned long)page_to_phys(sg->page),
+ (unsigned long)page_to_phys(sg_page(sg)),
(unsigned long)sg->offset,
(unsigned long)sg->length); */
end_addr = ib_sg_dma_address(ibdev, sg) +
@@ -336,7 +336,7 @@ static void iser_data_buf_dump(struct iser_data_buf *data,
iser_err("sg[%d] dma_addr:0x%lX page:0x%p "
"off:0x%x sz:0x%x dma_len:0x%x\n",
i, (unsigned long)ib_sg_dma_address(ibdev, sg),
- sg->page, sg->offset,
+ sg_page(sg), sg->offset,
sg->length, ib_sg_dma_len(ibdev, sg));
}
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 64fee90..cde915f 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -346,16 +346,17 @@ static int crypt_convert(struct crypt_config *cc,
ctx->idx_out < ctx->bio_out->bi_vcnt) {
struct bio_vec *bv_in = bio_iovec_idx(ctx->bio_in, ctx->idx_in);
struct bio_vec *bv_out = bio_iovec_idx(ctx->bio_out, ctx->idx_out);
- struct scatterlist sg_in = {
- .page = bv_in->bv_page,
- .offset = bv_in->bv_offset + ctx->offset_in,
- .length = 1 << SECTOR_SHIFT
- };
- struct scatterlist sg_out = {
- .page = bv_out->bv_page,
- .offset = bv_out->bv_offset + ctx->offset_out,
- .length = 1 << SECTOR_SHIFT
- };
+ struct scatterlist sg_in, sg_out;
+
+ sg_init_table(&sg_in, 1);
+ sg_set_page(&sg_in, bv_in->bv_page);
+ sg_in.offset = bv_in->bv_offset + ctx->offset_in;
+ sg_in.length = 1 << SECTOR_SHIFT;
+
+ sg_init_table(&sg_out, 1);
+ sg_set_page(&sg_out, bv_out->bv_page);
+ sg_out.offset = bv_out->bv_offset + ctx->offset_out;
+ sg_out.length = 1 << SECTOR_SHIFT;
ctx->offset_in += sg_in.length;
if (ctx->offset_in >= bv_in->bv_len) {
diff --git a/drivers/media/common/saa7146_core.c b/drivers/media/common/saa7146_core.c
index 365a221..2b1f8b4 100644
--- a/drivers/media/common/saa7146_core.c
+++ b/drivers/media/common/saa7146_core.c
@@ -112,12 +112,13 @@ static struct scatterlist* vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/media/video/ivtv/ivtv-udma.c b/drivers/media/video/ivtv/ivtv-udma.c
index c4626d1..912b424 100644
--- a/drivers/media/video/ivtv/ivtv-udma.c
+++ b/drivers/media/video/ivtv/ivtv-udma.c
@@ -63,10 +63,10 @@ int ivtv_udma_fill_sg_list (struct ivtv_user_dma *dma, struct ivtv_dma_page_info
memcpy(page_address(dma->bouncemap[map_offset]) + offset, src, len);
kunmap_atomic(src, KM_BOUNCE_READ);
local_irq_restore(flags);
- dma->SGlist[map_offset].page = dma->bouncemap[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->bouncemap[map_offset]);
}
else {
- dma->SGlist[map_offset].page = dma->map[map_offset];
+ sg_set_page(&dma->SGlist[map_offset], dma->map[map_offset]);
}
offset = 0;
map_offset++;
diff --git a/drivers/media/video/videobuf-dma-sg.c b/drivers/media/video/videobuf-dma-sg.c
index 3eb6123..0a18286 100644
--- a/drivers/media/video/videobuf-dma-sg.c
+++ b/drivers/media/video/videobuf-dma-sg.c
@@ -60,12 +60,13 @@ videobuf_vmalloc_to_sg(unsigned char *virt, int nr_pages)
sglist = kcalloc(nr_pages, sizeof(struct scatterlist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
pg = vmalloc_to_page(virt);
if (NULL == pg)
goto err;
BUG_ON(PageHighMem(pg));
- sglist[i].page = pg;
+ sg_set_page(&sglist[i], pg);
sglist[i].length = PAGE_SIZE;
}
return sglist;
@@ -86,13 +87,14 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
sglist = kcalloc(nr_pages, sizeof(*sglist), GFP_KERNEL);
if (NULL == sglist)
return NULL;
+ sg_init_table(sglist, nr_pages);
if (NULL == pages[0])
goto nopage;
if (PageHighMem(pages[0]))
/* DMA to highmem pages might not work */
goto highmem;
- sglist[0].page = pages[0];
+ sg_set_page(&sglist[0], pages[0]);
sglist[0].offset = offset;
sglist[0].length = PAGE_SIZE - offset;
for (i = 1; i < nr_pages; i++) {
@@ -100,7 +102,7 @@ videobuf_pages_to_sg(struct page **pages, int nr_pages, int offset)
goto nopage;
if (PageHighMem(pages[i]))
goto highmem;
- sglist[i].page = pages[i];
+ sg_set_page(&sglist[i], pages[i]);
sglist[i].length = PAGE_SIZE;
}
return sglist;
diff --git a/drivers/mmc/card/queue.c b/drivers/mmc/card/queue.c
index a5d0354..68c1fe9 100644
--- a/drivers/mmc/card/queue.c
+++ b/drivers/mmc/card/queue.c
@@ -13,6 +13,7 @@
#include <linux/blkdev.h>
#include <linux/freezer.h>
#include <linux/kthread.h>
+#include <linux/scatterlist.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
@@ -153,19 +154,21 @@ int mmc_init_queue(struct mmc_queue *mq, struct mmc_card *card, spinlock_t *lock
blk_queue_max_hw_segments(mq->queue, bouncesz / 512);
blk_queue_max_segment_size(mq->queue, bouncesz);
- mq->sg = kzalloc(sizeof(struct scatterlist),
+ mq->sg = kmalloc(sizeof(struct scatterlist),
GFP_KERNEL);
if (!mq->sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->sg, 1);
- mq->bounce_sg = kzalloc(sizeof(struct scatterlist) *
+ mq->bounce_sg = kmalloc(sizeof(struct scatterlist) *
bouncesz / 512, GFP_KERNEL);
if (!mq->bounce_sg) {
ret = -ENOMEM;
goto cleanup_queue;
}
+ sg_init_table(mq->bounce_sg, bouncesz / 512);
}
}
#endif
@@ -302,12 +305,12 @@ static void copy_sg(struct scatterlist *dst, unsigned int dst_len,
BUG_ON(dst_len == 0);
if (dst_size == 0) {
- dst_buf = page_address(dst->page) + dst->offset;
+ dst_buf = page_address(sg_page(dst)) + dst->offset;
dst_size = dst->length;
}
if (src_size == 0) {
- src_buf = page_address(src->page) + src->offset;
+ src_buf = page_address(sg_page(src)) + src->offset;
src_size = src->length;
}
@@ -353,9 +356,7 @@ unsigned int mmc_queue_map_sg(struct mmc_queue *mq)
return 1;
}
- mq->sg[0].page = virt_to_page(mq->bounce_buf);
- mq->sg[0].offset = offset_in_page(mq->bounce_buf);
- mq->sg[0].length = 0;
+ sg_init_one(mq->sg, mq->bounce_buf, 0);
while (sg_len) {
mq->sg[0].length += mq->bounce_sg[sg_len - 1].length;
diff --git a/drivers/mmc/host/at91_mci.c b/drivers/mmc/host/at91_mci.c
index 7a452c2..b1edcef 100644
--- a/drivers/mmc/host/at91_mci.c
+++ b/drivers/mmc/host/at91_mci.c
@@ -149,7 +149,7 @@ static inline void at91_mci_sg_to_dma(struct at91mci_host *host, struct mmc_data
sg = &data->sg[i];
- sgbuffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ sgbuffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
amount = min(size, sg->length);
size -= amount;
@@ -226,7 +226,7 @@ static void at91_mci_pre_dma_read(struct at91mci_host *host)
sg = &data->sg[host->transfer_index++];
pr_debug("sg = %p\n", sg);
- sg->dma_address = dma_map_page(NULL, sg->page, sg->offset, sg->length, DMA_FROM_DEVICE);
+ sg->dma_address = dma_map_page(NULL, sg_page(sg), sg->offset, sg->length, DMA_FROM_DEVICE);
pr_debug("dma address = %08X, length = %d\n", sg->dma_address, sg->length);
@@ -283,7 +283,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
int index;
/* Swap the contents of the buffer */
- buffer = kmap_atomic(sg->page, KM_BIO_SRC_IRQ) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_BIO_SRC_IRQ) + sg->offset;
pr_debug("buffer = %p, length = %d\n", buffer, sg->length);
for (index = 0; index < (sg->length / 4); index++)
@@ -292,7 +292,7 @@ static void at91_mci_post_dma_read(struct at91mci_host *host)
kunmap_atomic(buffer, KM_BIO_SRC_IRQ);
}
- flush_dcache_page(sg->page);
+ flush_dcache_page(sg_page(sg));
}
/* Is there another transfer to trigger? */
diff --git a/drivers/mmc/host/au1xmmc.c b/drivers/mmc/host/au1xmmc.c
index 92c4d0d..0632f4d 100644
--- a/drivers/mmc/host/au1xmmc.c
+++ b/drivers/mmc/host/au1xmmc.c
@@ -340,7 +340,7 @@ static void au1xmmc_send_pio(struct au1xmmc_host *host)
/* This is the pointer to the data buffer */
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = data->sg[host->pio.index].length - host->pio.offset;
@@ -400,7 +400,7 @@ static void au1xmmc_receive_pio(struct au1xmmc_host *host)
if (host->pio.index < host->dma.len) {
sg = &data->sg[host->pio.index];
- sg_ptr = page_address(sg->page) + sg->offset + host->pio.offset;
+ sg_ptr = page_address(sg_page(sg)) + sg->offset + host->pio.offset;
/* This is the space left inside the buffer */
sg_len = sg_dma_len(&data->sg[host->pio.index]) - host->pio.offset;
@@ -613,13 +613,13 @@ au1xmmc_prepare_data(struct au1xmmc_host *host, struct mmc_data *data)
if (host->flags & HOST_F_XMIT){
ret = au1xxx_dbdma_put_source_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
else {
ret = au1xxx_dbdma_put_dest_flags(channel,
- (void *) (page_address(sg->page) +
+ (void *) (page_address(sg_page(sg)) +
sg->offset),
len, flags);
}
diff --git a/drivers/mmc/host/imxmmc.c b/drivers/mmc/host/imxmmc.c
index 6ebc41e..581bb8f 100644
--- a/drivers/mmc/host/imxmmc.c
+++ b/drivers/mmc/host/imxmmc.c
@@ -262,7 +262,7 @@ static void imxmci_setup_data(struct imxmci_host *host, struct mmc_data *data)
}
/* Convert back to virtual address */
- host->data_ptr = (u16*)(page_address(data->sg->page) + data->sg->offset);
+ host->data_ptr = (u16*)(page_address(sg_page(data->sg)) + data->sg->offset);
host->data_cnt = 0;
clear_bit(IMXMCI_PEND_DMA_DATA_b, &host->pending_events);
diff --git a/drivers/mmc/host/mmc_spi.c b/drivers/mmc/host/mmc_spi.c
index 7ae18ea..12c2d80 100644
--- a/drivers/mmc/host/mmc_spi.c
+++ b/drivers/mmc/host/mmc_spi.c
@@ -813,7 +813,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
&& dir == DMA_FROM_DEVICE)
dir = DMA_BIDIRECTIONAL;
- dma_addr = dma_map_page(dma_dev, sg->page, 0,
+ dma_addr = dma_map_page(dma_dev, sg_page(sg), 0,
PAGE_SIZE, dir);
if (direction == DMA_TO_DEVICE)
t->tx_dma = dma_addr + sg->offset;
@@ -822,7 +822,7 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
}
/* allow pio too; we don't allow highmem */
- kmap_addr = kmap(sg->page);
+ kmap_addr = kmap(sg_page(sg));
if (direction == DMA_TO_DEVICE)
t->tx_buf = kmap_addr + sg->offset;
else
@@ -855,8 +855,8 @@ mmc_spi_data_do(struct mmc_spi_host *host, struct mmc_command *cmd,
/* discard mappings */
if (direction == DMA_FROM_DEVICE)
- flush_kernel_dcache_page(sg->page);
- kunmap(sg->page);
+ flush_kernel_dcache_page(sg_page(sg));
+ kunmap(sg_page(sg));
if (dma_dev)
dma_unmap_page(dma_dev, dma_addr, PAGE_SIZE, dir);
diff --git a/drivers/mmc/host/omap.c b/drivers/mmc/host/omap.c
index 60a67df..649f647 100644
--- a/drivers/mmc/host/omap.c
+++ b/drivers/mmc/host/omap.c
@@ -24,10 +24,10 @@
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
#include <linux/clk.h>
+#include <linux/scatterlist.h>
#include <asm/io.h>
#include <asm/irq.h>
-#include <asm/scatterlist.h>
#include <asm/mach-types.h>
#include <asm/arch/board.h>
@@ -383,7 +383,7 @@ mmc_omap_sg_to_buf(struct mmc_omap_host *host)
sg = host->data->sg + host->sg_idx;
host->buffer_bytes_left = sg->length;
- host->buffer = page_address(sg->page) + sg->offset;
+ host->buffer = page_address(sg_page(sg)) + sg->offset;
if (host->buffer_bytes_left > host->total_bytes_left)
host->buffer_bytes_left = host->total_bytes_left;
}
diff --git a/drivers/mmc/host/sdhci.c b/drivers/mmc/host/sdhci.c
index b397121..b6500e9 100644
--- a/drivers/mmc/host/sdhci.c
+++ b/drivers/mmc/host/sdhci.c
@@ -231,7 +231,7 @@ static void sdhci_deactivate_led(struct sdhci_host *host)
static inline char* sdhci_sg_to_buffer(struct sdhci_host* host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline int sdhci_next_sg(struct sdhci_host* host)
diff --git a/drivers/mmc/host/tifm_sd.c b/drivers/mmc/host/tifm_sd.c
index 9b90479..c11a3d2 100644
--- a/drivers/mmc/host/tifm_sd.c
+++ b/drivers/mmc/host/tifm_sd.c
@@ -192,7 +192,7 @@ static void tifm_sd_transfer_data(struct tifm_sd *host)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
@@ -241,18 +241,18 @@ static void tifm_sd_bounce_block(struct tifm_sd *host, struct mmc_data *r_data)
}
off = sg[host->sg_pos].offset + host->block_pos;
- pg = nth_page(sg[host->sg_pos].page, off >> PAGE_SHIFT);
+ pg = nth_page(sg_page(&sg[host->sg_pos]), off >> PAGE_SHIFT);
p_off = offset_in_page(off);
p_cnt = PAGE_SIZE - p_off;
p_cnt = min(p_cnt, cnt);
p_cnt = min(p_cnt, t_size);
if (r_data->flags & MMC_DATA_WRITE)
- tifm_sd_copy_page(host->bounce_buf.page,
+ tifm_sd_copy_page(sg_page(&host->bounce_buf),
r_data->blksz - t_size,
pg, p_off, p_cnt);
else if (r_data->flags & MMC_DATA_READ)
- tifm_sd_copy_page(pg, p_off, host->bounce_buf.page,
+ tifm_sd_copy_page(pg, p_off, sg_page(&host->bounce_buf),
r_data->blksz - t_size, p_cnt);
t_size -= p_cnt;
diff --git a/drivers/mmc/host/wbsd.c b/drivers/mmc/host/wbsd.c
index 80db11c..971ca0f 100644
--- a/drivers/mmc/host/wbsd.c
+++ b/drivers/mmc/host/wbsd.c
@@ -269,7 +269,7 @@ static inline int wbsd_next_sg(struct wbsd_host *host)
static inline char *wbsd_sg_to_buffer(struct wbsd_host *host)
{
- return page_address(host->cur_sg->page) + host->cur_sg->offset;
+ return page_address(sg_page(host->cur_sg)) + host->cur_sg->offset;
}
static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
@@ -283,7 +283,7 @@ static inline void wbsd_sg_to_dma(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(dmabuf, sgbuf, sg[i].length);
dmabuf += sg[i].length;
}
@@ -300,7 +300,7 @@ static inline void wbsd_dma_to_sg(struct wbsd_host *host, struct mmc_data *data)
len = data->sg_len;
for (i = 0; i < len; i++) {
- sgbuf = page_address(sg[i].page) + sg[i].offset;
+ sgbuf = page_address(sg_page(&sg[i])) + sg[i].offset;
memcpy(sgbuf, dmabuf, sg[i].length);
dmabuf += sg[i].length;
}
diff --git a/drivers/net/mlx4/icm.c b/drivers/net/mlx4/icm.c
index 4b3c109..887633b 100644
--- a/drivers/net/mlx4/icm.c
+++ b/drivers/net/mlx4/icm.c
@@ -60,7 +60,7 @@ static void mlx4_free_icm_pages(struct mlx4_dev *dev, struct mlx4_icm_chunk *chu
PCI_DMA_BIDIRECTIONAL);
for (i = 0; i < chunk->npages; ++i)
- __free_pages(chunk->mem[i].page,
+ __free_pages(sg_page(&chunk->mem[i]),
get_order(chunk->mem[i].length));
}
@@ -70,7 +70,7 @@ static void mlx4_free_icm_coherent(struct mlx4_dev *dev, struct mlx4_icm_chunk *
for (i = 0; i < chunk->npages; ++i)
dma_free_coherent(&dev->pdev->dev, chunk->mem[i].length,
- lowmem_page_address(chunk->mem[i].page),
+ lowmem_page_address(sg_page(&chunk->mem[i])),
sg_dma_address(&chunk->mem[i]));
}
@@ -95,10 +95,13 @@ void mlx4_free_icm(struct mlx4_dev *dev, struct mlx4_icm *icm, int coherent)
static int mlx4_alloc_icm_pages(struct scatterlist *mem, int order, gfp_t gfp_mask)
{
- mem->page = alloc_pages(gfp_mask, order);
- if (!mem->page)
+ struct page *page;
+
+ page = alloc_pages(gfp_mask, order);
+ if (!page)
return -ENOMEM;
+ sg_set_page(mem, page);
mem->length = PAGE_SIZE << order;
mem->offset = 0;
return 0;
@@ -145,6 +148,7 @@ struct mlx4_icm *mlx4_alloc_icm(struct mlx4_dev *dev, int npages,
if (!chunk)
goto fail;
+ sg_init_table(chunk->mem, MLX4_ICM_CHUNK_LEN);
chunk->npages = 0;
chunk->nsg = 0;
list_add_tail(&chunk->list, &icm->chunk_list);
@@ -334,7 +338,7 @@ void *mlx4_table_find(struct mlx4_icm_table *table, int obj, dma_addr_t *dma_han
* been assigned to.
*/
if (chunk->mem[i].length > offset) {
- page = chunk->mem[i].page;
+ page = sg_page(&chunk->mem[i]);
goto out;
}
offset -= chunk->mem[i].length;
diff --git a/drivers/net/ppp_mppe.c b/drivers/net/ppp_mppe.c
index c0b6d19..bcb0885 100644
--- a/drivers/net/ppp_mppe.c
+++ b/drivers/net/ppp_mppe.c
@@ -55,7 +55,7 @@
#include <linux/mm.h>
#include <linux/ppp_defs.h>
#include <linux/ppp-comp.h>
-#include <asm/scatterlist.h>
+#include <linux/scatterlist.h>
#include "ppp_mppe.h"
@@ -68,9 +68,7 @@ MODULE_VERSION("1.0.2");
static unsigned int
setup_sg(struct scatterlist *sg, const void *address, unsigned int length)
{
- sg[0].page = virt_to_page(address);
- sg[0].offset = offset_in_page(address);
- sg[0].length = length;
+ sg_init_one(sg, address, length);
return length;
}
diff --git a/drivers/scsi/3w-9xxx.c b/drivers/scsi/3w-9xxx.c
index fb14014..afb262b 100644
--- a/drivers/scsi/3w-9xxx.c
+++ b/drivers/scsi/3w-9xxx.c
@@ -1840,7 +1840,7 @@ static int twa_scsiop_execute_scsi(TW_Device_Extension *tw_dev, int request_id,
(scsi_bufflen(srb) < TW_MIN_SGL_LENGTH)) {
if (srb->sc_data_direction == DMA_TO_DEVICE || srb->sc_data_direction == DMA_BIDIRECTIONAL) {
struct scatterlist *sg = scsi_sglist(srb);
- char *buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ char *buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(tw_dev->generic_buffer_virt[request_id], buf, sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
}
@@ -1919,7 +1919,7 @@ static void twa_scsiop_execute_scsi_complete(TW_Device_Extension *tw_dev, int re
char *buf;
unsigned long flags = 0;
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buf, tw_dev->generic_buffer_virt[request_id], sg->length);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/3w-xxxx.c b/drivers/scsi/3w-xxxx.c
index a64153b..59716eb 100644
--- a/drivers/scsi/3w-xxxx.c
+++ b/drivers/scsi/3w-xxxx.c
@@ -1469,7 +1469,7 @@ static void tw_transfer_internal(TW_Device_Extension *tw_dev, int request_id,
struct scatterlist *sg = scsi_sglist(cmd);
local_irq_save(flags);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len);
memcpy(buf, data, transfer_len);
diff --git a/drivers/scsi/NCR5380.c b/drivers/scsi/NCR5380.c
index 988f0bc..7a76b4b 100644
--- a/drivers/scsi/NCR5380.c
+++ b/drivers/scsi/NCR5380.c
@@ -298,7 +298,7 @@ static __inline__ void initialize_SCp(Scsi_Cmnd * cmd)
if (cmd->use_sg) {
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.buffers_residual = cmd->use_sg - 1;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
} else {
@@ -2143,7 +2143,7 @@ static void NCR5380_information_transfer(struct Scsi_Host *instance) {
++cmd->SCp.buffer;
--cmd->SCp.buffers_residual;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page)+
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer))+
cmd->SCp.buffer->offset;
dprintk(NDEBUG_INFORMATION, ("scsi%d : %d bytes and %d buffers left\n", instance->host_no, cmd->SCp.this_residual, cmd->SCp.buffers_residual));
}
diff --git a/drivers/scsi/NCR53c406a.c b/drivers/scsi/NCR53c406a.c
index 3168a17..db1a8bc 100644
--- a/drivers/scsi/NCR53c406a.c
+++ b/drivers/scsi/NCR53c406a.c
@@ -875,7 +875,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_write(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_write(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
@@ -897,7 +897,7 @@ static void NCR53c406a_intr(void *dev_id)
outb(TRANSFER_INFO | DMA_OP, CMD_REG);
#if USE_PIO
scsi_for_each_sg(current_SC, sg, scsi_sg_count(current_SC), i) {
- NCR53c406a_pio_read(page_address(sg->page) + sg->offset,
+ NCR53c406a_pio_read(page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0;
diff --git a/drivers/scsi/aacraid/aachba.c b/drivers/scsi/aacraid/aachba.c
index 80e448d..a77ab8d 100644
--- a/drivers/scsi/aacraid/aachba.c
+++ b/drivers/scsi/aacraid/aachba.c
@@ -356,7 +356,7 @@ static void aac_internal_transfer(struct scsi_cmnd *scsicmd, void *data, unsigne
int transfer_len;
struct scatterlist *sg = scsi_sglist(scsicmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
transfer_len = min(sg->length, len + offset);
transfer_len -= offset;
diff --git a/drivers/scsi/aha1542.c b/drivers/scsi/aha1542.c
index 961a188..50da0a3 100644
--- a/drivers/scsi/aha1542.c
+++ b/drivers/scsi/aha1542.c
@@ -49,7 +49,7 @@
#include "aha1542.h"
#define SCSI_BUF_PA(address) isa_virt_to_bus(address)
-#define SCSI_SG_PA(sgent) (isa_page_to_bus((sgent)->page) + (sgent)->offset)
+#define SCSI_SG_PA(sgent) (isa_page_to_bus(sg_page((sgent))) + (sgent)->offset)
static void BAD_DMA(void *address, unsigned int length)
{
@@ -67,7 +67,7 @@ static void BAD_SG_DMA(Scsi_Cmnd * SCpnt,
{
printk(KERN_CRIT "sgpnt[%d:%d] page %p/0x%llx length %u\n",
badseg, nseg,
- page_address(sgp->page) + sgp->offset,
+ page_address(sg_page(sgp)) + sgp->offset,
(unsigned long long)SCSI_SG_PA(sgp),
sgp->length);
@@ -712,7 +712,7 @@ static int aha1542_queuecommand(Scsi_Cmnd * SCpnt, void (*done) (Scsi_Cmnd *))
printk(KERN_CRIT "Bad segment list supplied to aha1542.c (%d, %d)\n", SCpnt->use_sg, i);
scsi_for_each_sg(SCpnt, sg, SCpnt->use_sg, i) {
printk(KERN_CRIT "%d: %p %d\n", i,
- (page_address(sg->page) +
+ (page_address(sg_page(sg)) +
sg->offset), sg->length);
};
printk(KERN_CRIT "cptr %x: ", (unsigned int) cptr);
diff --git a/drivers/scsi/arcmsr/arcmsr_hba.c b/drivers/scsi/arcmsr/arcmsr_hba.c
index f817775..f7a2528 100644
--- a/drivers/scsi/arcmsr/arcmsr_hba.c
+++ b/drivers/scsi/arcmsr/arcmsr_hba.c
@@ -1343,7 +1343,7 @@ static int arcmsr_iop_message_xfer(struct AdapterControlBlock *acb, \
/* 4 bytes: Areca io control code */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (scsi_sg_count(cmd) > 1) {
retvalue = ARCMSR_MESSAGE_FAIL;
goto message_out;
@@ -1593,7 +1593,7 @@ static void arcmsr_handle_virtual_command(struct AdapterControlBlock *acb,
strncpy(&inqdata[32], "R001", 4); /* Product Revision */
sg = scsi_sglist(cmd);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memcpy(buffer, inqdata, sizeof(inqdata));
sg = scsi_sglist(cmd);
diff --git a/drivers/scsi/fdomain.c b/drivers/scsi/fdomain.c
index 5d282e6..dc4e463 100644
--- a/drivers/scsi/fdomain.c
+++ b/drivers/scsi/fdomain.c
@@ -1321,7 +1321,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
if (current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
} else
break;
@@ -1354,7 +1354,7 @@ static irqreturn_t do_fdomain_16x0_intr(int irq, void *dev_id)
&& current_SC->SCp.buffers_residual) {
--current_SC->SCp.buffers_residual;
++current_SC->SCp.buffer;
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page) + current_SC->SCp.buffer->offset;
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer)) + current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
}
}
@@ -1439,7 +1439,7 @@ static int fdomain_16x0_queue(struct scsi_cmnd *SCpnt,
if (scsi_sg_count(current_SC)) {
current_SC->SCp.buffer = scsi_sglist(current_SC);
- current_SC->SCp.ptr = page_address(current_SC->SCp.buffer->page)
+ current_SC->SCp.ptr = page_address(sg_page(current_SC->SCp.buffer))
+ current_SC->SCp.buffer->offset;
current_SC->SCp.this_residual = current_SC->SCp.buffer->length;
current_SC->SCp.buffers_residual = scsi_sg_count(current_SC) - 1;
diff --git a/drivers/scsi/gdth.c b/drivers/scsi/gdth.c
index 3ac080e..ac6c57a 100644
--- a/drivers/scsi/gdth.c
+++ b/drivers/scsi/gdth.c
@@ -2374,13 +2374,13 @@ static void gdth_copy_internal_data(gdth_ha_str *ha, Scsi_Cmnd *scp,
if (cpsum+cpnow > cpcount)
cpnow = cpcount - cpsum;
cpsum += cpnow;
- if (!sl->page) {
+ if (!sg_page(sl)) {
printk("GDT-HA %d: invalid sc/gt element in gdth_copy_internal_data()\n",
ha->hanum);
return;
}
local_irq_save(flags);
- address = kmap_atomic(sl->page, KM_BIO_SRC_IRQ) + sl->offset;
+ address = kmap_atomic(sg_page(sl), KM_BIO_SRC_IRQ) + sl->offset;
if (to_buffer)
memcpy(buffer, address, cpnow);
else
diff --git a/drivers/scsi/ibmmca.c b/drivers/scsi/ibmmca.c
index 714e627..db004a4 100644
--- a/drivers/scsi/ibmmca.c
+++ b/drivers/scsi/ibmmca.c
@@ -1828,7 +1828,7 @@ static int ibmmca_queuecommand(Scsi_Cmnd * cmd, void (*done) (Scsi_Cmnd *))
BUG_ON(scsi_sg_count(cmd) > 16);
scsi_for_each_sg(cmd, sg, scsi_sg_count(cmd), i) {
- ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg->page) + sg->offset);
+ ld(shpnt)[ldn].sge[i].address = (void *) (isa_page_to_bus(sg_page(sg)) + sg->offset);
ld(shpnt)[ldn].sge[i].byte_length = sg->length;
}
scb->enable |= IM_POINTER_TO_LIST;
diff --git a/drivers/scsi/ide-scsi.c b/drivers/scsi/ide-scsi.c
index fa7ba64..2c7dd8b 100644
--- a/drivers/scsi/ide-scsi.c
+++ b/drivers/scsi/ide-scsi.c
@@ -175,18 +175,18 @@ static void idescsi_input_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsigne
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_input_bytes(drive,
buf + pc->b_count, count);
}
@@ -212,18 +212,18 @@ static void idescsi_output_buffers (ide_drive_t *drive, idescsi_pc_t *pc, unsign
while (bcount) {
count = min(pc->sg->length - pc->b_count, bcount);
- if (PageHighMem(pc->sg->page)) {
+ if (PageHighMem(sg_page(pc->sg))) {
unsigned long flags;
local_irq_save(flags);
- buf = kmap_atomic(pc->sg->page, KM_IRQ0) +
+ buf = kmap_atomic(sg_page(pc->sg), KM_IRQ0) +
pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
kunmap_atomic(buf - pc->sg->offset, KM_IRQ0);
local_irq_restore(flags);
} else {
- buf = page_address(pc->sg->page) + pc->sg->offset;
+ buf = page_address(sg_page(pc->sg)) + pc->sg->offset;
drive->hwif->atapi_output_bytes(drive,
buf + pc->b_count, count);
}
diff --git a/drivers/scsi/imm.c b/drivers/scsi/imm.c
index 74cdc1f..da9b0ba 100644
--- a/drivers/scsi/imm.c
+++ b/drivers/scsi/imm.c
@@ -706,7 +706,7 @@ static int imm_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
/*
@@ -845,7 +845,7 @@ static int imm_engine(imm_struct *dev, struct scsi_cmnd *cmd)
(struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c
index b41dfb5..dc98a85 100644
--- a/drivers/scsi/ipr.c
+++ b/drivers/scsi/ipr.c
@@ -2872,6 +2872,7 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
}
scatterlist = sglist->scatterlist;
+ sg_init_table(scatterlist, num_elem);
sglist->order = order;
sglist->num_sg = num_elem;
@@ -2884,12 +2885,12 @@ static struct ipr_sglist *ipr_alloc_ucode_buffer(int buf_len)
/* Free up what we already allocated */
for (j = i - 1; j >= 0; j--)
- __free_pages(scatterlist[j].page, order);
+ __free_pages(sg_page(&scatterlist[j]), order);
kfree(sglist);
return NULL;
}
- scatterlist[i].page = page;
+ sg_set_page(&scatterlist[i], page);
}
return sglist;
@@ -2910,7 +2911,7 @@ static void ipr_free_ucode_buffer(struct ipr_sglist *sglist)
int i;
for (i = 0; i < sglist->num_sg; i++)
- __free_pages(sglist->scatterlist[i].page, sglist->order);
+ __free_pages(sg_page(&sglist->scatterlist[i]), sglist->order);
kfree(sglist);
}
@@ -2940,9 +2941,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
scatterlist = sglist->scatterlist;
for (i = 0; i < (len / bsize_elem); i++, buffer += bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = bsize_elem;
@@ -2953,9 +2956,11 @@ static int ipr_copy_ucode_buffer(struct ipr_sglist *sglist,
}
if (len % bsize_elem) {
- kaddr = kmap(scatterlist[i].page);
+ struct page *page = sg_page(&scatterlist[i]);
+
+ kaddr = kmap(page);
memcpy(kaddr, buffer, len % bsize_elem);
- kunmap(scatterlist[i].page);
+ kunmap(page);
scatterlist[i].length = len % bsize_elem;
}
diff --git a/drivers/scsi/ips.c b/drivers/scsi/ips.c
index edaac27..5c5a9b2 100644
--- a/drivers/scsi/ips.c
+++ b/drivers/scsi/ips.c
@@ -1515,7 +1515,7 @@ static int ips_is_passthru(struct scsi_cmnd *SC)
/* kmap_atomic() ensures addressability of the user buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buffer = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
if (buffer && buffer[0] == 'C' && buffer[1] == 'O' &&
buffer[2] == 'P' && buffer[3] == 'P') {
kunmap_atomic(buffer - sg->offset, KM_IRQ0);
@@ -3523,7 +3523,7 @@ ips_scmd_buf_write(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(buffer, &cdata[xfer_cnt], min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
@@ -3556,7 +3556,7 @@ ips_scmd_buf_read(struct scsi_cmnd *scmd, void *data, unsigned int count)
/* kmap_atomic() ensures addressability of the data buffer.*/
/* local_irq_save() protects the KM_IRQ0 address slot. */
local_irq_save(flags);
- buffer = kmap_atomic(sg[i].page, KM_IRQ0) + sg[i].offset;
+ buffer = kmap_atomic(sg_page(&sg[i]), KM_IRQ0) + sg[i].offset;
memcpy(&cdata[xfer_cnt], buffer, min_cnt);
kunmap_atomic(buffer - sg[i].offset, KM_IRQ0);
local_irq_restore(flags);
diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index a21455d..6ce4109 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -70,9 +70,7 @@ module_param_named(max_lun, iscsi_max_lun, uint, S_IRUGO);
static inline void
iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
{
- ibuf->sg.page = virt_to_page(vbuf);
- ibuf->sg.offset = offset_in_page(vbuf);
- ibuf->sg.length = size;
+ sg_init_one(&ibuf->sg, vbuf, size);
ibuf->sent = 0;
ibuf->use_sendmsg = 1;
}
@@ -80,13 +78,14 @@ iscsi_buf_init_iov(struct iscsi_buf *ibuf, char *vbuf, int size)
static inline void
iscsi_buf_init_sg(struct iscsi_buf *ibuf, struct scatterlist *sg)
{
- ibuf->sg.page = sg->page;
+ sg_init_table(&ibuf->sg, 1);
+ sg_set_page(&ibuf->sg, sg_page(sg));
ibuf->sg.offset = sg->offset;
ibuf->sg.length = sg->length;
/*
* Fastpath: sg element fits into single page
*/
- if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg->page))
+ if (sg->length + sg->offset <= PAGE_SIZE && !PageSlab(sg_page(sg)))
ibuf->use_sendmsg = 0;
else
ibuf->use_sendmsg = 1;
@@ -716,7 +715,7 @@ static int iscsi_scsi_data_in(struct iscsi_conn *conn)
for (i = tcp_ctask->sg_count; i < scsi_sg_count(sc); i++) {
char *dest;
- dest = kmap_atomic(sg[i].page, KM_SOFTIRQ0);
+ dest = kmap_atomic(sg_page(&sg[i]), KM_SOFTIRQ0);
rc = iscsi_ctask_copy(tcp_conn, ctask, dest + sg[i].offset,
sg[i].length, offset);
kunmap_atomic(dest, KM_SOFTIRQ0);
@@ -1103,9 +1102,9 @@ iscsi_send(struct iscsi_conn *conn, struct iscsi_buf *buf, int size, int flags)
* slab case.
*/
if (buf->use_sendmsg)
- res = sock_no_sendpage(sk, buf->sg.page, offset, size, flags);
+ res = sock_no_sendpage(sk, sg_page(&buf->sg), offset, size, flags);
else
- res = tcp_conn->sendpage(sk, buf->sg.page, offset, size, flags);
+ res = tcp_conn->sendpage(sk, sg_page(&buf->sg), offset, size, flags);
if (res >= 0) {
conn->txdata_octets += res;
diff --git a/drivers/scsi/megaraid.c b/drivers/scsi/megaraid.c
index 10d1aff..0a848e8 100644
--- a/drivers/scsi/megaraid.c
+++ b/drivers/scsi/megaraid.c
@@ -658,7 +658,7 @@ mega_build_cmd(adapter_t *adapter, Scsi_Cmnd *cmd, int *busy)
struct scatterlist *sg;
sg = scsi_sglist(cmd);
- buf = kmap_atomic(sg->page, KM_IRQ0) + sg->offset;
+ buf = kmap_atomic(sg_page(sg), KM_IRQ0) + sg->offset;
memset(buf, 0, cmd->cmnd[4]);
kunmap_atomic(buf - sg->offset, KM_IRQ0);
@@ -1542,9 +1542,9 @@ mega_cmd_done(adapter_t *adapter, u8 completed[], int nstatus, int status)
if( cmd->cmnd[0] == INQUIRY && !islogical ) {
sgl = scsi_sglist(cmd);
- if( sgl->page ) {
+ if( sg_page(sgl) ) {
c = *(unsigned char *)
- page_address((&sgl[0])->page) +
+ page_address((sg_page(&sgl[0]))) +
(&sgl[0])->offset;
} else {
printk(KERN_WARNING
diff --git a/drivers/scsi/megaraid/megaraid_mbox.c b/drivers/scsi/megaraid/megaraid_mbox.c
index e4e4c6a..376263c 100644
--- a/drivers/scsi/megaraid/megaraid_mbox.c
+++ b/drivers/scsi/megaraid/megaraid_mbox.c
@@ -1584,9 +1584,9 @@ megaraid_mbox_build_cmd(adapter_t *adapter, struct scsi_cmnd *scp, int *busy)
caddr_t vaddr;
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
vaddr = (caddr_t)
- (page_address((&sgl[0])->page)
+ (page_address(sg_page(&sgl[0]))
+ (&sgl[0])->offset);
memset(vaddr, 0, scp->cmnd[4]);
@@ -2328,9 +2328,9 @@ megaraid_mbox_dpc(unsigned long devp)
&& IS_RAID_CH(raid_dev, scb->dev_channel)) {
sgl = scsi_sglist(scp);
- if (sgl->page) {
+ if (sg_page(sgl)) {
c = *(unsigned char *)
- (page_address((&sgl[0])->page) +
+ (page_address(sg_page(&sgl[0])) +
(&sgl[0])->offset);
} else {
con_log(CL_ANN, (KERN_WARNING
diff --git a/drivers/scsi/osst.c b/drivers/scsi/osst.c
index 331b789..1c5c4b6 100644
--- a/drivers/scsi/osst.c
+++ b/drivers/scsi/osst.c
@@ -542,7 +542,7 @@ static int osst_verify_frame(struct osst_tape * STp, int frame_seq_number, int q
if (STp->raw) {
if (STp->buffer->syscall_result) {
for (i=0; i < STp->buffer->sg_segs; i++)
- memset(page_address(STp->buffer->sg[i].page),
+ memset(page_address(sg_page(&STp->buffer->sg[i])),
0, STp->buffer->sg[i].length);
strcpy(STp->buffer->b_data, "READ ERROR ON FRAME");
} else
@@ -4437,7 +4437,7 @@ static int os_scsi_tape_open(struct inode * inode, struct file * filp)
for (i = 0, b_size = 0;
(i < STp->buffer->sg_segs) && ((b_size + STp->buffer->sg[i].length) <= OS_DATA_SIZE);
b_size += STp->buffer->sg[i++].length);
- STp->buffer->aux = (os_aux_t *) (page_address(STp->buffer->sg[i].page) + OS_DATA_SIZE - b_size);
+ STp->buffer->aux = (os_aux_t *) (page_address(sg_page(&STp->buffer->sg[i])) + OS_DATA_SIZE - b_size);
#if DEBUG
printk(OSST_DEB_MSG "%s:D: b_data points to %p in segment 0 at %p\n", name,
STp->buffer->b_data, page_address(STp->buffer->sg[0].page));
@@ -5252,25 +5252,26 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
/* Try to allocate the first segment up to OS_DATA_SIZE and the others
big enough to reach the goal (code assumes no segments in place) */
for (b_size = OS_DATA_SIZE, order = OSST_FIRST_ORDER; b_size >= PAGE_SIZE; order--, b_size /= 2) {
- STbuffer->sg[0].page = alloc_pages(priority, order);
+ struct page *page = alloc_pages(priority, order);
+
STbuffer->sg[0].offset = 0;
- if (STbuffer->sg[0].page != NULL) {
+ if (page != NULL) {
+ sg_set_page(&STbuffer->sg[0], page);
STbuffer->sg[0].length = b_size;
- STbuffer->b_data = page_address(STbuffer->sg[0].page);
+ STbuffer->b_data = page_address(page);
break;
}
}
- if (STbuffer->sg[0].page == NULL) {
+ if (sg_page(&STbuffer->sg[0]) == NULL) {
printk(KERN_NOTICE "osst :I: Can't allocate tape buffer main segment.\n");
return 0;
}
/* Got initial segment of 'bsize,order', continue with same size if possible, except for AUX */
for (segs=STbuffer->sg_segs=1, got=b_size;
segs < max_segs && got < OS_FRAME_SIZE; ) {
- STbuffer->sg[segs].page =
- alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
+ struct page *page = alloc_pages(priority, (OS_FRAME_SIZE - got <= PAGE_SIZE) ? 0 : order);
STbuffer->sg[segs].offset = 0;
- if (STbuffer->sg[segs].page == NULL) {
+ if (page == NULL) {
if (OS_FRAME_SIZE - got <= (max_segs - segs) * b_size / 2 && order) {
b_size /= 2; /* Large enough for the rest of the buffers */
order--;
@@ -5284,6 +5285,7 @@ static int enlarge_buffer(struct osst_buffer *STbuffer, int need_dma)
normalize_buffer(STbuffer);
return 0;
}
+ sg_set_page(&STbuffer->sg[segs], page);
STbuffer->sg[segs].length = (OS_FRAME_SIZE - got <= PAGE_SIZE / 2) ? (OS_FRAME_SIZE - got) : b_size;
got += STbuffer->sg[segs].length;
STbuffer->buffer_size = got;
@@ -5316,7 +5318,7 @@ static void normalize_buffer(struct osst_buffer *STbuffer)
b_size < STbuffer->sg[i].length;
b_size *= 2, order++);
- __free_pages(STbuffer->sg[i].page, order);
+ __free_pages(sg_page(&STbuffer->sg[i]), order);
STbuffer->buffer_size -= STbuffer->sg[i].length;
}
#if DEBUG
@@ -5344,7 +5346,7 @@ static int append_to_buffer(const char __user *ubp, struct osst_buffer *st_bp, i
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_from_user(page_address(st_bp->sg[i].page) + offset, ubp, cnt);
+ res = copy_from_user(page_address(sg_page(&st_bp->sg[i])) + offset, ubp, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5377,7 +5379,7 @@ static int from_buffer(struct osst_buffer *st_bp, char __user *ubp, int do_count
for ( ; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count;
- res = copy_to_user(ubp, page_address(st_bp->sg[i].page) + offset, cnt);
+ res = copy_to_user(ubp, page_address(sg_page(&st_bp->sg[i])) + offset, cnt);
if (res)
return (-EFAULT);
do_count -= cnt;
@@ -5410,7 +5412,7 @@ static int osst_zero_buffer_tail(struct osst_buffer *st_bp)
i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length - offset < do_count ?
st_bp->sg[i].length - offset : do_count ;
- memset(page_address(st_bp->sg[i].page) + offset, 0, cnt);
+ memset(page_address(sg_page(&st_bp->sg[i])) + offset, 0, cnt);
do_count -= cnt;
offset = 0;
}
@@ -5430,7 +5432,7 @@ static int osst_copy_to_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(page_address(st_bp->sg[i].page), ptr, cnt);
+ memcpy(page_address(sg_page(&st_bp->sg[i])), ptr, cnt);
do_count -= cnt;
ptr += cnt;
}
@@ -5451,7 +5453,7 @@ static int osst_copy_from_buffer(struct osst_buffer *st_bp, unsigned char *ptr)
for (i = 0; i < st_bp->sg_segs && do_count > 0; i++) {
cnt = st_bp->sg[i].length < do_count ?
st_bp->sg[i].length : do_count ;
- memcpy(ptr, page_address(st_bp->sg[i].page), cnt);
+ memcpy(ptr, page_address(sg_page(&st_bp->sg[i])), cnt);
do_count -= cnt;
ptr += cnt;
}
diff --git a/drivers/scsi/pcmcia/sym53c500_cs.c b/drivers/scsi/pcmcia/sym53c500_cs.c
index 190e2a7..26c41b0 100644
--- a/drivers/scsi/pcmcia/sym53c500_cs.c
+++ b/drivers/scsi/pcmcia/sym53c500_cs.c
@@ -443,7 +443,7 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_write(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
+ page_address(sg_page(sg)) + sg->offset,
sg->length);
}
REG0(port_base);
@@ -463,8 +463,8 @@ SYM53C500_intr(int irq, void *dev_id)
scsi_for_each_sg(curSC, sg, scsi_sg_count(curSC), i) {
SYM53C500_pio_read(fast_pio, port_base,
- page_address(sg->page) + sg->offset,
- sg->length);
+ page_address(sg_page(sg)) + sg->offset,
+ sg->length);
}
REG0(port_base);
}
diff --git a/drivers/scsi/ppa.c b/drivers/scsi/ppa.c
index 67b6d76..07f31e5 100644
--- a/drivers/scsi/ppa.c
+++ b/drivers/scsi/ppa.c
@@ -609,7 +609,7 @@ static int ppa_completion(struct scsi_cmnd *cmd)
cmd->SCp.this_residual =
cmd->SCp.buffer->length;
cmd->SCp.ptr =
- page_address(cmd->SCp.buffer->page) +
+ page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
}
}
@@ -756,7 +756,7 @@ static int ppa_engine(ppa_struct *dev, struct scsi_cmnd *cmd)
/* if many buffers are available, start filling the first */
cmd->SCp.buffer = (struct scatterlist *) cmd->request_buffer;
cmd->SCp.this_residual = cmd->SCp.buffer->length;
- cmd->SCp.ptr = page_address(cmd->SCp.buffer->page) +
+ cmd->SCp.ptr = page_address(sg_page(cmd->SCp.buffer)) +
cmd->SCp.buffer->offset;
} else {
/* else fill the only available buffer */
diff --git a/drivers/scsi/qlogicfas408.c b/drivers/scsi/qlogicfas408.c
index 2bfbf26..feff3f7 100644
--- a/drivers/scsi/qlogicfas408.c
+++ b/drivers/scsi/qlogicfas408.c
@@ -317,7 +317,7 @@ static unsigned int ql_pcmd(struct scsi_cmnd *cmd)
return ((priv->qabort == 1 ?
DID_ABORT : DID_RESET) << 16);
}
- buf = page_address(sg->page) + sg->offset;
+ buf = page_address(sg_page(sg)) + sg->offset;
if (ql_pdma(priv, phase, buf, sg->length))
break;
}
diff --git a/drivers/scsi/scsi_debug.c b/drivers/scsi/scsi_debug.c
index 72ee4c9..46cae5a 100644
--- a/drivers/scsi/scsi_debug.c
+++ b/drivers/scsi/scsi_debug.c
@@ -625,7 +625,7 @@ static int fill_from_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
scsi_for_each_sg(scp, sg, scp->use_sg, k) {
if (active) {
kaddr = (unsigned char *)
- kmap_atomic(sg->page, KM_USER0);
+ kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return (DID_ERROR << 16);
kaddr_off = (unsigned char *)kaddr + sg->offset;
@@ -672,7 +672,7 @@ static int fetch_to_dev_buffer(struct scsi_cmnd * scp, unsigned char * arr,
sg = scsi_sglist(scp);
req_len = fin = 0;
for (k = 0; k < scp->use_sg; ++k, sg = sg_next(sg)) {
- kaddr = (unsigned char *)kmap_atomic(sg->page, KM_USER0);
+ kaddr = (unsigned char *)kmap_atomic(sg_page(sg), KM_USER0);
if (NULL == kaddr)
return -1;
kaddr_off = (unsigned char *)kaddr + sg->offset;
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index aac8a02..61fdaf0 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -295,7 +295,7 @@ static int scsi_req_map_sg(struct request *rq, struct scatterlist *sgl,
int i, err, nr_vecs = 0;
for_each_sg(sgl, sg, nsegs, i) {
- page = sg->page;
+ page = sg_page(sg);
off = sg->offset;
len = sg->length;
data_len += len;
@@ -764,7 +764,7 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
if (unlikely(!sgl))
goto enomem;
- memset(sgl, 0, sizeof(*sgl) * sgp->size);
+ sg_init_table(sgl, sgp->size);
/*
* first loop through, set initial index and return value
@@ -781,6 +781,13 @@ struct scatterlist *scsi_alloc_sgtable(struct scsi_cmnd *cmd, gfp_t gfp_mask)
sg_chain(prev, SCSI_MAX_SG_SEGMENTS, sgl);
/*
+ * if we have nothing left, mark the last segment as
+ * end-of-list
+ */
+ if (!left)
+ sg_mark_end(sgl, this);
+
+ /*
* don't allow subsequent mempool allocs to sleep, it would
* violate the mempool principle.
*/
@@ -2353,7 +2360,7 @@ void *scsi_kmap_atomic_sg(struct scatterlist *sgl, int sg_count,
*offset = *offset - len_complete + sg->offset;
/* Assumption: contiguous pages can be accessed as "page + i" */
- page = nth_page(sg->page, (*offset >> PAGE_SHIFT));
+ page = nth_page(sg_page(sg), (*offset >> PAGE_SHIFT));
*offset &= ~PAGE_MASK;
/* Bytes in this sg-entry from *offset to the end of the page */
diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
index 7238b2d..cc19710 100644
--- a/drivers/scsi/sg.c
+++ b/drivers/scsi/sg.c
@@ -1169,7 +1169,7 @@ sg_vma_nopage(struct vm_area_struct *vma, unsigned long addr, int *type)
len = vma->vm_end - sa;
len = (len < sg->length) ? len : sg->length;
if (offset < len) {
- page = virt_to_page(page_address(sg->page) + offset);
+ page = virt_to_page(page_address(sg_page(sg)) + offset);
get_page(page); /* increment page count */
break;
}
@@ -1717,13 +1717,13 @@ st_map_user_pages(struct scatterlist *sgl, const unsigned int max_pages,
goto out_unlock; */
}
- sgl[0].page = pages[0];
+ sg_set_page(sgl, pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
- sgl[i].page = pages[i];
+ sg_set_page(&sgl[i], pages[i]);
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -1754,7 +1754,7 @@ st_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_pages,
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
@@ -1854,7 +1854,7 @@ sg_build_indirect(Sg_scatter_hold * schp, Sg_fd * sfp, int buff_size)
scatter_elem_sz_prev = ret_sz;
}
}
- sg->page = p;
+ sg_set_page(sg, p);
sg->length = (ret_sz > num) ? num : ret_sz;
SCSI_LOG_TIMEOUT(5, printk("sg_build_indirect: k=%d, num=%d, "
@@ -1907,14 +1907,14 @@ sg_write_xfer(Sg_request * srp)
onum = 1;
ksglen = sg->length;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 1, &usglen, &up);
if (res)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -1991,12 +1991,12 @@ sg_remove_scat(Sg_scatter_hold * schp)
} else {
int k;
- for (k = 0; (k < schp->k_use_sg) && sg->page;
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg);
++k, sg = sg_next(sg)) {
SCSI_LOG_TIMEOUT(5, printk(
"sg_remove_scat: k=%d, pg=0x%p, len=%d\n",
- k, sg->page, sg->length));
- sg_page_free(sg->page, sg->length);
+ k, sg_page(sg), sg->length));
+ sg_page_free(sg_page(sg), sg->length);
}
}
kfree(schp->buffer);
@@ -2038,7 +2038,7 @@ sg_read_xfer(Sg_request * srp)
} else
onum = 1;
- p = page_address(sg->page);
+ p = page_address(sg_page(sg));
ksglen = sg->length;
for (j = 0, k = 0; j < onum; ++j) {
res = sg_u_iovec(hp, iovec_count, j, 0, &usglen, &up);
@@ -2046,7 +2046,7 @@ sg_read_xfer(Sg_request * srp)
return res;
for (; p; sg = sg_next(sg), ksglen = sg->length,
- p = page_address(sg->page)) {
+ p = page_address(sg_page(sg))) {
if (usglen <= 0)
break;
if (ksglen > usglen) {
@@ -2092,15 +2092,15 @@ sg_read_oxfer(Sg_request * srp, char __user *outp, int num_read_xfer)
if ((!outp) || (num_read_xfer <= 0))
return 0;
- for (k = 0; (k < schp->k_use_sg) && sg->page; ++k, sg = sg_next(sg)) {
+ for (k = 0; (k < schp->k_use_sg) && sg_page(sg); ++k, sg = sg_next(sg)) {
num = sg->length;
if (num > num_read_xfer) {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num_read_xfer))
return -EFAULT;
break;
} else {
- if (__copy_to_user(outp, page_address(sg->page),
+ if (__copy_to_user(outp, page_address(sg_page(sg)),
num))
return -EFAULT;
num_read_xfer -= num;
diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
index 73c44cb..ce69b9e 100644
--- a/drivers/scsi/st.c
+++ b/drivers/scsi/st.c
@@ -3797,7 +3797,7 @@ static void buf_to_sg(struct st_buffer *STbp, unsigned int length)
sg = &(STbp->sg[0]);
frp = STbp->frp;
for (i=count=0; count < length; i++) {
- sg[i].page = frp[i].page;
+ sg_set_page(&sg[i], frp[i].page);
if (length - count > frp[i].length)
sg[i].length = frp[i].length;
else
@@ -4446,14 +4446,14 @@ static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int max_pa
}
/* Populate the scatter/gather list */
- sgl[0].page = pages[0];
+ sg_set_page(&sgl[0], pages[0]);
sgl[0].offset = uaddr & ~PAGE_MASK;
if (nr_pages > 1) {
sgl[0].length = PAGE_SIZE - sgl[0].offset;
count -= sgl[0].length;
for (i=1; i < nr_pages ; i++) {
+ sg_set_page(&sgl[i], pages[i]);;
sgl[i].offset = 0;
- sgl[i].page = pages[i];
sgl[i].length = count < PAGE_SIZE ? count : PAGE_SIZE;
count -= PAGE_SIZE;
}
@@ -4483,7 +4483,7 @@ static int sgl_unmap_user_pages(struct scatterlist *sgl, const unsigned int nr_p
int i;
for (i=0; i < nr_pages; i++) {
- struct page *page = sgl[i].page;
+ struct page *page = sg_page(&sgl[i]);
if (dirtied)
SetPageDirty(page);
diff --git a/drivers/scsi/tmscsim.c b/drivers/scsi/tmscsim.c
index 5c72ca3..4419304 100644
--- a/drivers/scsi/tmscsim.c
+++ b/drivers/scsi/tmscsim.c
@@ -430,10 +430,7 @@ static __inline__ void dc390_Going_remove (struct dc390_dcb* pDCB, struct dc390_
static struct scatterlist* dc390_sg_build_single(struct scatterlist *sg, void *addr, unsigned int length)
{
- memset(sg, 0, sizeof(struct scatterlist));
- sg->page = virt_to_page(addr);
- sg->length = length;
- sg->offset = (unsigned long)addr & ~PAGE_MASK;
+ sg_init_one(sg, addr, length);
return sg;
}
diff --git a/drivers/scsi/ultrastor.c b/drivers/scsi/ultrastor.c
index ea72bbe..6d1f0ed 100644
--- a/drivers/scsi/ultrastor.c
+++ b/drivers/scsi/ultrastor.c
@@ -681,7 +681,7 @@ static inline void build_sg_list(struct mscp *mscp, struct scsi_cmnd *SCpnt)
max = scsi_sg_count(SCpnt);
scsi_for_each_sg(SCpnt, sg, max, i) {
- mscp->sglist[i].address = isa_page_to_bus(sg->page) + sg->offset;
+ mscp->sglist[i].address = isa_page_to_bus(sg_page(sg)) + sg->offset;
mscp->sglist[i].num_bytes = sg->length;
transfer_length += sg->length;
}
diff --git a/drivers/scsi/wd7000.c b/drivers/scsi/wd7000.c
index 255c611..03cd44f 100644
--- a/drivers/scsi/wd7000.c
+++ b/drivers/scsi/wd7000.c
@@ -1123,7 +1123,7 @@ static int wd7000_queuecommand(struct scsi_cmnd *SCpnt,
any2scsi(scb->maxlen, nseg * sizeof(Sgb));
scsi_for_each_sg(SCpnt, sg, nseg, i) {
- any2scsi(sgb[i].ptr, isa_page_to_bus(sg->page) + sg->offset);
+ any2scsi(sgb[i].ptr, isa_page_to_bus(sg_page(sg)) + sg->offset);
any2scsi(sgb[i].len, sg->length);
}
} else {
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index c021af3..3577453 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -438,12 +438,12 @@ int usb_sg_init (
io->urbs[i]->transfer_buffer = NULL;
#else
io->urbs[i]->transfer_buffer =
- page_address(sg[i].page) + sg[i].offset;
+ page_address(sg_page(&sg[i])) + sg[i].offset;
#endif
} else {
/* hc may use _only_ transfer_buffer */
io->urbs [i]->transfer_buffer =
- page_address (sg [i].page) + sg [i].offset;
+ page_address(sg_page(&sg[i])) + sg [i].offset;
len = sg [i].length;
}
diff --git a/drivers/usb/image/microtek.c b/drivers/usb/image/microtek.c
index e7d982a..4866baf 100644
--- a/drivers/usb/image/microtek.c
+++ b/drivers/usb/image/microtek.c
@@ -519,7 +519,7 @@ static void mts_do_sg (struct urb* transfer)
context->fragment++;
mts_int_submit_urb(transfer,
context->data_pipe,
- page_address(sg[context->fragment].page) +
+ page_address(sg_page(&sg[context->fragment])) +
sg[context->fragment].offset,
sg[context->fragment].length,
context->fragment + 1 == scsi_sg_count(context->srb) ?
@@ -557,7 +557,7 @@ mts_build_transfer_context(struct scsi_cmnd *srb, struct mts_desc* desc)
return;
} else {
sg = scsi_sglist(srb);
- desc->context.data = page_address(sg[0].page) + sg[0].offset;
+ desc->context.data = page_address(sg_page(&sg[0])) + sg[0].offset;
desc->context.data_length = sg[0].length;
}
diff --git a/drivers/usb/misc/usbtest.c b/drivers/usb/misc/usbtest.c
index e901d31..ca8cb34 100644
--- a/drivers/usb/misc/usbtest.c
+++ b/drivers/usb/misc/usbtest.c
@@ -360,9 +360,10 @@ static void free_sglist (struct scatterlist *sg, int nents)
if (!sg)
return;
for (i = 0; i < nents; i++) {
- if (!sg [i].page)
+ struct page *page = sg_page(&sg[i]);
+ if (!page)
continue;
- kfree (page_address (sg [i].page) + sg [i].offset);
+ kfree (page_address (page) + sg [i].offset);
}
kfree (sg);
}
diff --git a/drivers/usb/storage/protocol.c b/drivers/usb/storage/protocol.c
index cc8f7c5..889622b 100644
--- a/drivers/usb/storage/protocol.c
+++ b/drivers/usb/storage/protocol.c
@@ -195,7 +195,7 @@ unsigned int usb_stor_access_xfer_buf(unsigned char *buffer,
* the *offset and *index values for the next loop. */
cnt = 0;
while (cnt < buflen) {
- struct page *page = sg->page +
+ struct page *page = sg_page(sg) +
((sg->offset + *offset) >> PAGE_SHIFT);
unsigned int poff =
(sg->offset + *offset) & (PAGE_SIZE-1);
diff --git a/fs/ecryptfs/crypto.c b/fs/ecryptfs/crypto.c
index 1ae90ef..0a9882e 100644
--- a/fs/ecryptfs/crypto.c
+++ b/fs/ecryptfs/crypto.c
@@ -283,7 +283,7 @@ int virt_to_scatterlist(const void *addr, int size, struct scatterlist *sg,
pg = virt_to_page(addr);
offset = offset_in_page(addr);
if (sg) {
- sg[i].page = pg;
+ sg_set_page(&sg[i], pg);
sg[i].offset = offset;
}
remainder_of_page = PAGE_CACHE_SIZE - offset;
@@ -713,10 +713,13 @@ ecryptfs_encrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return encrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
@@ -742,10 +745,13 @@ ecryptfs_decrypt_page_offset(struct ecryptfs_crypt_stat *crypt_stat,
{
struct scatterlist src_sg, dst_sg;
- src_sg.page = src_page;
+ sg_init_table(&src_sg, 1);
+ sg_init_table(&dst_sg, 1);
+
+ sg_set_page(&src_sg, src_page);
src_sg.offset = src_offset;
src_sg.length = size;
- dst_sg.page = dst_page;
+ sg_set_page(&dst_sg, dst_page);
dst_sg.offset = dst_offset;
dst_sg.length = size;
return decrypt_scatterlist(crypt_stat, &dst_sg, &src_sg, size, iv);
diff --git a/fs/ecryptfs/keystore.c b/fs/ecryptfs/keystore.c
index 89d9710..263fed8 100644
--- a/fs/ecryptfs/keystore.c
+++ b/fs/ecryptfs/keystore.c
@@ -1040,6 +1040,9 @@ decrypt_passphrase_encrypted_session_key(struct ecryptfs_auth_tok *auth_tok,
};
int rc = 0;
+ sg_init_table(&dst_sg, 1);
+ sg_init_table(&src_sg, 1);
+
if (unlikely(ecryptfs_verbosity > 0)) {
ecryptfs_printk(
KERN_DEBUG, "Session key encryption key (size [%d]):\n",
diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c
index ebd03cc..6f03918 100644
--- a/fs/nfsd/nfs4recover.c
+++ b/fs/nfsd/nfs4recover.c
@@ -88,7 +88,7 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
{
struct xdr_netobj cksum;
struct hash_desc desc;
- struct scatterlist sg[1];
+ struct scatterlist sg;
__be32 status = nfserr_resource;
dprintk("NFSD: nfs4_make_rec_clidname for %.*s\n",
@@ -102,11 +102,9 @@ nfs4_make_rec_clidname(char *dname, struct xdr_netobj *clname)
if (cksum.data == NULL)
goto out;
- sg[0].page = virt_to_page(clname->data);
- sg[0].offset = offset_in_page(clname->data);
- sg[0].length = clname->len;
+ sg_init_one(&sg, clname->data, clname->len);
- if (crypto_hash_digest(&desc, sg, sg->length, cksum.data))
+ if (crypto_hash_digest(&desc, &sg, sg.length, cksum.data))
goto out;
md5_to_hex(dname, cksum.data);
diff --git a/include/asm-alpha/scatterlist.h b/include/asm-alpha/scatterlist.h
index 9173654..b764706 100644
--- a/include/asm-alpha/scatterlist.h
+++ b/include/asm-alpha/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-arm/scatterlist.h b/include/asm-arm/scatterlist.h
index de2f65e..ab1d85d 100644
--- a/include/asm-arm/scatterlist.h
+++ b/include/asm-arm/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page; /* buffer page */
+ unsigned long page_link;
unsigned int offset; /* buffer offset */
dma_addr_t dma_address; /* dma address */
unsigned int length; /* length */
diff --git a/include/asm-avr32/scatterlist.h b/include/asm-avr32/scatterlist.h
index c6d5ce3..1356f29 100644
--- a/include/asm-avr32/scatterlist.h
+++ b/include/asm-avr32/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-blackfin/scatterlist.h b/include/asm-blackfin/scatterlist.h
index 60e07b9..384af54 100644
--- a/include/asm-blackfin/scatterlist.h
+++ b/include/asm-blackfin/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/mm.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-cris/scatterlist.h b/include/asm-cris/scatterlist.h
index 4bdc44c..5a8a834 100644
--- a/include/asm-cris/scatterlist.h
+++ b/include/asm-cris/scatterlist.h
@@ -6,7 +6,7 @@ struct scatterlist {
unsigned int length;
/* The following is i386 highmem junk - not used by us */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
};
diff --git a/include/asm-frv/scatterlist.h b/include/asm-frv/scatterlist.h
index 8e827fa..53dade7 100644
--- a/include/asm-frv/scatterlist.h
+++ b/include/asm-frv/scatterlist.h
@@ -22,7 +22,7 @@
* and that's it. There's no excuse for not highmem enabling YOUR driver. /jens
*/
struct scatterlist {
- struct page *page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset; /* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-h8300/scatterlist.h b/include/asm-h8300/scatterlist.h
index 985fdf5..7e41983 100644
--- a/include/asm-h8300/scatterlist.h
+++ b/include/asm-h8300/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-ia64/scatterlist.h b/include/asm-ia64/scatterlist.h
index 7d5234d..2f76ce3 100644
--- a/include/asm-ia64/scatterlist.h
+++ b/include/asm-ia64/scatterlist.h
@@ -9,7 +9,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length; /* buffer length */
diff --git a/include/asm-m32r/scatterlist.h b/include/asm-m32r/scatterlist.h
index 352415f..33b4b4d 100644
--- a/include/asm-m32r/scatterlist.h
+++ b/include/asm-m32r/scatterlist.h
@@ -6,7 +6,7 @@
struct scatterlist {
char * address; /* Location data is to be transferred to, NULL for
* highmem page */
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
diff --git a/include/asm-m68k/scatterlist.h b/include/asm-m68k/scatterlist.h
index 24887a2..e06bb89 100644
--- a/include/asm-m68k/scatterlist.h
+++ b/include/asm-m68k/scatterlist.h
@@ -4,7 +4,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-m68knommu/scatterlist.h b/include/asm-m68knommu/scatterlist.h
index 4da79d3..28bed41 100644
--- a/include/asm-m68knommu/scatterlist.h
+++ b/include/asm-m68knommu/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-mips/scatterlist.h b/include/asm-mips/scatterlist.h
index 7af104c..787797c 100644
--- a/include/asm-mips/scatterlist.h
+++ b/include/asm-mips/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-parisc/scatterlist.h b/include/asm-parisc/scatterlist.h
index e7211c7..26da914 100644
--- a/include/asm-parisc/scatterlist.h
+++ b/include/asm-parisc/scatterlist.h
@@ -5,7 +5,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-powerpc/scatterlist.h b/include/asm-powerpc/scatterlist.h
index b075f61..b9f1dbc 100644
--- a/include/asm-powerpc/scatterlist.h
+++ b/include/asm-powerpc/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/dma.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-s390/scatterlist.h b/include/asm-s390/scatterlist.h
index a43b3af..eb39486 100644
--- a/include/asm-s390/scatterlist.h
+++ b/include/asm-s390/scatterlist.h
@@ -2,7 +2,7 @@
#define _ASMS390_SCATTERLIST_H
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
};
diff --git a/include/asm-sh/scatterlist.h b/include/asm-sh/scatterlist.h
index b9ae53c..bc7c809 100644
--- a/include/asm-sh/scatterlist.h
+++ b/include/asm-sh/scatterlist.h
@@ -4,7 +4,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sh64/scatterlist.h b/include/asm-sh64/scatterlist.h
index 1c723f2..0afd856 100644
--- a/include/asm-sh64/scatterlist.h
+++ b/include/asm-sh64/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page * page; /* Location for highmem page, if any */
+ unsigned long page_link;
unsigned int offset;/* for highmem, page offset */
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-sparc/scatterlist.h b/include/asm-sparc/scatterlist.h
index 4055af9..45b16f1 100644
--- a/include/asm-sparc/scatterlist.h
+++ b/include/asm-sparc/scatterlist.h
@@ -5,7 +5,7 @@
#include <linux/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-sparc64/scatterlist.h b/include/asm-sparc64/scatterlist.h
index 703c5bb..6df23f0 100644
--- a/include/asm-sparc64/scatterlist.h
+++ b/include/asm-sparc64/scatterlist.h
@@ -6,7 +6,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
diff --git a/include/asm-v850/scatterlist.h b/include/asm-v850/scatterlist.h
index 56f4029..db91feb 100644
--- a/include/asm-v850/scatterlist.h
+++ b/include/asm-v850/scatterlist.h
@@ -17,7 +17,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned offset;
dma_addr_t dma_address;
unsigned length;
diff --git a/include/asm-x86/dma-mapping_32.h b/include/asm-x86/dma-mapping_32.h
index 6a2d26c..e0d38d8 100644
--- a/include/asm-x86/dma-mapping_32.h
+++ b/include/asm-x86/dma-mapping_32.h
@@ -45,9 +45,9 @@ dma_map_sg(struct device *dev, struct scatterlist *sglist, int nents,
WARN_ON(nents == 0 || sglist[0].length == 0);
for_each_sg(sglist, sg, nents, i) {
- BUG_ON(!sg->page);
+ BUG_ON(!sg_page(sg));
- sg->dma_address = page_to_phys(sg->page) + sg->offset;
+ sg->dma_address = page_to_phys(sg_page(sg)) + sg->offset;
}
flush_write_buffers();
diff --git a/include/asm-x86/scatterlist_32.h b/include/asm-x86/scatterlist_32.h
index bd5164a..0e7d997 100644
--- a/include/asm-x86/scatterlist_32.h
+++ b/include/asm-x86/scatterlist_32.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/asm-x86/scatterlist_64.h b/include/asm-x86/scatterlist_64.h
index ef3986b..1847c72 100644
--- a/include/asm-x86/scatterlist_64.h
+++ b/include/asm-x86/scatterlist_64.h
@@ -4,7 +4,10 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
+ unsigned long page_link;
unsigned int offset;
unsigned int length;
dma_addr_t dma_address;
diff --git a/include/asm-xtensa/scatterlist.h b/include/asm-xtensa/scatterlist.h
index ca337a2..3b8aba5 100644
--- a/include/asm-xtensa/scatterlist.h
+++ b/include/asm-xtensa/scatterlist.h
@@ -14,7 +14,7 @@
#include <asm/types.h>
struct scatterlist {
- struct page *page;
+ unsigned long page_link;
unsigned int offset;
dma_addr_t dma_address;
unsigned int length;
diff --git a/include/linux/scatterlist.h b/include/linux/scatterlist.h
index 2dc7464..eb70970 100644
--- a/include/linux/scatterlist.h
+++ b/include/linux/scatterlist.h
@@ -5,29 +5,72 @@
#include <linux/mm.h>
#include <linux/string.h>
+/*
+ * Notes on SG table design.
+ *
+ * Architectures must provide an unsigned long page_link field in the
+ * scatterlist struct. We use that to place the page pointer AND encode
+ * information about the sg table as well. The two lower bits are reserved
+ * for this information.
+ *
+ * If bit 0 is set, then the page_link contains a pointer to the next sg
+ * table list. Otherwise the next entry is at sg + 1.
+ *
+ * If bit 1 is set, then this sg entry is the last element in a list.
+ *
+ * See sg_next().
+ *
+ */
+
+#define SG_MAGIC 0x87654321
+
+/**
+ * sg_set_page - Set sg entry to point at given page
+ * @sg: SG entry
+ * @page: The page
+ *
+ * Use this function to set an sg entry pointing at a page, never assign
+ * the page directly. We encode sg table information in the lower bits
+ * of the page pointer. See sg_page() for looking up the page belonging
+ * to an sg entry.
+ *
+ */
+static inline void sg_set_page(struct scatterlist *sg, struct page *page)
+{
+ unsigned long page_link = sg->page_link & 0x3;
+
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ sg->page_link = page_link | (unsigned long) page;
+}
+
+#define sg_page(sg) ((struct page *) ((sg)->page_link & ~0x3))
+
+/**
+ * sg_set_buf - Set sg entry to point at given data
+ * @sg: SG entry
+ * @buf: Data
+ * @buflen: Data length
+ *
+*/
static inline void sg_set_buf(struct scatterlist *sg, const void *buf,
unsigned int buflen)
{
- sg->page = virt_to_page(buf);
+ sg_set_page(sg, virt_to_page(buf));
sg->offset = offset_in_page(buf);
sg->length = buflen;
}
-static inline void sg_init_one(struct scatterlist *sg, const void *buf,
- unsigned int buflen)
-{
- memset(sg, 0, sizeof(*sg));
- sg_set_buf(sg, buf, buflen);
-}
-
/*
* We overload the LSB of the page pointer to indicate whether it's
* a valid sg entry, or whether it points to the start of a new scatterlist.
* Those low bits are there for everyone! (thanks mason :-)
*/
-#define sg_is_chain(sg) ((unsigned long) (sg)->page & 0x01)
+#define sg_is_chain(sg) ((sg)->page_link & 0x01)
+#define sg_is_last(sg) ((sg)->page_link & 0x02)
#define sg_chain_ptr(sg) \
- ((struct scatterlist *) ((unsigned long) (sg)->page & ~0x01))
+ ((struct scatterlist *) ((sg)->page_link & ~0x03))
/**
* sg_next - return the next scatterlist entry in a list
@@ -37,14 +80,16 @@ static inline void sg_init_one(struct scatterlist *sg, const void *buf,
* of a chained scatterlist, it could jump to the start of a new
* scatterlist array.
*
- * Note that the caller must ensure that there are further entries after
- * the current entry, this function will NOT return NULL for an end-of-list.
- *
*/
static inline struct scatterlist *sg_next(struct scatterlist *sg)
{
- sg++;
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sg->sg_magic != SG_MAGIC);
+#endif
+ if (sg_is_last(sg))
+ return NULL;
+ sg++;
if (unlikely(sg_is_chain(sg)))
sg = sg_chain_ptr(sg);
@@ -83,6 +128,10 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
ret = sg;
#endif
+#ifdef CONFIG_DEBUG_SG
+ BUG_ON(sgl[0].sg_magic != SG_MAGIC);
+ BUG_ON(!sg_is_last(ret));
+#endif
return ret;
}
@@ -101,7 +150,68 @@ static inline void sg_chain(struct scatterlist *prv, unsigned int prv_nents,
#ifndef ARCH_HAS_SG_CHAIN
BUG();
#endif
- prv[prv_nents - 1].page = (struct page *) ((unsigned long) sgl | 0x01);
+ prv[prv_nents - 1].page_link = (unsigned long) sgl | 0x01;
+}
+
+/**
+ * sg_mark_end - Mark the end of the scatterlist
+ * @sgl: Scatterlist
+ * @nents: Number of entries in sgl
+ *
+ * Marks the last entry as the termination point for sg_next()
+ *
+ */
+static inline void sg_mark_end(struct scatterlist *sgl, unsigned int nents)
+{
+ sgl[nents - 1].page_link = 0x02;
+}
+
+static inline void __sg_mark_end(struct scatterlist *sg)
+{
+ sg->page_link |= 0x02;
+}
+
+/**
+ * sg_init_one - Initialize a single entry sg list
+ * @sg: SG entry
+ * @buf: Virtual address for IO
+ * @buflen: IO length
+ *
+ * Note: this should not be used on a single entry that is part of a larger
+ * table. Use sg_init_table() for that.
+ *
+ */
+static inline void sg_init_one(struct scatterlist *sg, const void *buf,
+ unsigned int buflen)
+{
+ memset(sg, 0, sizeof(*sg));
+#ifdef CONFIG_DEBUG_SG
+ sg->sg_magic = SG_MAGIC;
+#endif
+ sg_mark_end(sg, 1);
+ sg_set_buf(sg, buf, buflen);
+}
+
+/**
+ * sg_init_table - Initialize SG table
+ * @sgl: The SG table
+ * @nents: Number of entries in table
+ *
+ * Note: If this is part of a chained sg table, sg_mark_end() should be
+ * used only on the last table part.
+ *
+ */
+static inline void sg_init_table(struct scatterlist *sgl, unsigned int nents)
+{
+ memset(sgl, 0, sizeof(*sgl) * nents);
+ sg_mark_end(sgl, nents);
+#ifdef CONFIG_DEBUG_SG
+ {
+ int i;
+ for (i = 0; i < nents; i++)
+ sgl[i].sg_magic = SG_MAGIC;
+ }
+#endif
}
#endif /* _LINUX_SCATTERLIST_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 7d16e64..183f42c 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -389,6 +389,16 @@ config DEBUG_LIST
If unsure, say N.
+config DEBUG_SG
+ bool "Debug SG table operations"
+ depends on DEBUG_KERNEL
+ help
+ Enable this to turn on checks on scatter-gather tables. This can
+ help find problems with drivers that do not properly initialize
+ their sg tables.
+
+ If unsure, say N.
+
config FRAME_POINTER
bool "Compile the kernel with frame pointers"
depends on DEBUG_KERNEL && (X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || AVR32 || SUPERH || BFIN)
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 752fd95..e58909e 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -35,7 +35,7 @@
#define OFFSET(val,align) ((unsigned long) \
( (val) & ( (align) - 1)))
-#define SG_ENT_VIRT_ADDRESS(sg) (page_address((sg)->page) + (sg)->offset)
+#define SG_ENT_VIRT_ADDRESS(sg) (page_address(sg_page((sg)) + (sg)->offset))
#define SG_ENT_PHYS_ADDRESS(sg) virt_to_bus(SG_ENT_VIRT_ADDRESS(sg))
/*
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 70d9b5d..4e2c84f 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2045,7 +2045,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > 0) {
if (copy > len)
copy = len;
- sg[elt].page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg[elt], virt_to_page(skb->data + offset));
sg[elt].offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg[elt].length = copy;
elt++;
@@ -2065,7 +2065,7 @@ skb_to_sgvec(struct sk_buff *skb, struct scatterlist *sg, int offset, int len)
if (copy > len)
copy = len;
- sg[elt].page = frag->page;
+ sg_set_page(&sg[elt], frag->page);
sg[elt].offset = frag->page_offset+offset-start;
sg[elt].length = copy;
elt++;
diff --git a/net/ieee80211/ieee80211_crypt_tkip.c b/net/ieee80211/ieee80211_crypt_tkip.c
index 6cc54ee..34d4c77 100644
--- a/net/ieee80211/ieee80211_crypt_tkip.c
+++ b/net/ieee80211/ieee80211_crypt_tkip.c
@@ -390,9 +390,7 @@ static int ieee80211_tkip_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(tkey->tx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -485,9 +483,7 @@ static int ieee80211_tkip_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 12;
crypto_blkcipher_setkey(tkey->rx_tfm_arc4, rc4key, 16);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4)) {
if (net_ratelimit()) {
printk(KERN_DEBUG ": TKIP: failed to decrypt "
@@ -539,11 +535,12 @@ static int michael_mic(struct crypto_hash *tfm_michael, u8 * key, u8 * hdr,
printk(KERN_WARNING "michael_mic: tfm_michael == NULL\n");
return -1;
}
- sg[0].page = virt_to_page(hdr);
+ sg_init_table(sg, 2);
+ sg_set_page(&sg[0], virt_to_page(hdr));
sg[0].offset = offset_in_page(hdr);
sg[0].length = 16;
- sg[1].page = virt_to_page(data);
+ sg_set_page(&sg[1], virt_to_page(data));
sg[1].offset = offset_in_page(data);
sg[1].length = data_len;
diff --git a/net/ieee80211/ieee80211_crypt_wep.c b/net/ieee80211/ieee80211_crypt_wep.c
index 8d18245..0af6103 100644
--- a/net/ieee80211/ieee80211_crypt_wep.c
+++ b/net/ieee80211/ieee80211_crypt_wep.c
@@ -170,9 +170,7 @@ static int prism2_wep_encrypt(struct sk_buff *skb, int hdr_len, void *priv)
icv[3] = crc >> 24;
crypto_blkcipher_setkey(wep->tx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = len + 4;
+ sg_init_one(&sg, pos, len + 4);
return crypto_blkcipher_encrypt(&desc, &sg, &sg, len + 4);
}
@@ -212,9 +210,7 @@ static int prism2_wep_decrypt(struct sk_buff *skb, int hdr_len, void *priv)
plen = skb->len - hdr_len - 8;
crypto_blkcipher_setkey(wep->rx_tfm, key, klen);
- sg.page = virt_to_page(pos);
- sg.offset = offset_in_page(pos);
- sg.length = plen + 4;
+ sg_init_one(&sg, pos, plen + 4);
if (crypto_blkcipher_decrypt(&desc, &sg, &sg, plen + 4))
return -7;
diff --git a/net/mac80211/wep.c b/net/mac80211/wep.c
index 6675261..cc806d6 100644
--- a/net/mac80211/wep.c
+++ b/net/mac80211/wep.c
@@ -138,9 +138,7 @@ void ieee80211_wep_encrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
*icv = cpu_to_le32(~crc32_le(~0, data, data_len));
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_encrypt(&desc, &sg, &sg, sg.length);
}
@@ -204,9 +202,7 @@ int ieee80211_wep_decrypt_data(struct crypto_blkcipher *tfm, u8 *rc4key,
__le32 crc;
crypto_blkcipher_setkey(tfm, rc4key, klen);
- sg.page = virt_to_page(data);
- sg.offset = offset_in_page(data);
- sg.length = data_len + WEP_ICV_LEN;
+ sg_init_one(&sg, data, data_len + WEP_ICV_LEN);
crypto_blkcipher_decrypt(&desc, &sg, &sg, sg.length);
crc = cpu_to_le32(~crc32_le(~0, data, data_len));
diff --git a/net/sctp/auth.c b/net/sctp/auth.c
index 7818107..cbd64b2 100644
--- a/net/sctp/auth.c
+++ b/net/sctp/auth.c
@@ -726,7 +726,8 @@ void sctp_auth_calculate_hmac(const struct sctp_association *asoc,
/* set up scatter list */
end = skb_tail_pointer(skb);
- sg.page = virt_to_page(auth);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(auth));
sg.offset = (unsigned long)(auth) % PAGE_SIZE;
sg.length = end - (unsigned char *)auth;
diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c
index f983a36..d5a9785 100644
--- a/net/sctp/sm_make_chunk.c
+++ b/net/sctp/sm_make_chunk.c
@@ -1513,7 +1513,8 @@ static sctp_cookie_param_t *sctp_pack_cookie(const struct sctp_endpoint *ep,
struct hash_desc desc;
/* Sign the message. */
- sg.page = virt_to_page(&cookie->c);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(&cookie->c));
sg.offset = (unsigned long)(&cookie->c) % PAGE_SIZE;
sg.length = bodysize;
keylen = SCTP_SECRET_SIZE;
@@ -1585,7 +1586,8 @@ struct sctp_association *sctp_unpack_cookie(
/* Check the signature. */
keylen = SCTP_SECRET_SIZE;
- sg.page = virt_to_page(bear_cookie);
+ sg_init_table(&sg, 1);
+ sg_set_page(&sg, virt_to_page(bear_cookie));
sg.offset = (unsigned long)(bear_cookie) % PAGE_SIZE;
sg.length = bodysize;
key = (char *)ep->secret_key[ep->current_key];
diff --git a/net/sunrpc/auth_gss/gss_krb5_crypto.c b/net/sunrpc/auth_gss/gss_krb5_crypto.c
index bfb6a29..32be431 100644
--- a/net/sunrpc/auth_gss/gss_krb5_crypto.c
+++ b/net/sunrpc/auth_gss/gss_krb5_crypto.c
@@ -197,9 +197,9 @@ encryptor(struct scatterlist *sg, void *data)
int i = (page_pos + outbuf->page_base) >> PAGE_CACHE_SHIFT;
in_page = desc->pages[i];
} else {
- in_page = sg->page;
+ in_page = sg_page(sg);
}
- desc->infrags[desc->fragno].page = in_page;
+ sg_set_page(&desc->infrags[desc->fragno], in_page);
desc->fragno++;
desc->fraglen += sg->length;
desc->pos += sg->length;
@@ -215,11 +215,11 @@ encryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->outfrags[0].page = sg->page;
+ sg_set_page(&desc->outfrags[0], sg_page(sg));
desc->outfrags[0].offset = sg->offset + sg->length - fraglen;
desc->outfrags[0].length = fraglen;
desc->infrags[0] = desc->outfrags[0];
- desc->infrags[0].page = in_page;
+ sg_set_page(&desc->infrags[0], in_page);
desc->fragno = 1;
desc->fraglen = fraglen;
} else {
@@ -287,7 +287,7 @@ decryptor(struct scatterlist *sg, void *data)
if (ret)
return ret;
if (fraglen) {
- desc->frags[0].page = sg->page;
+ sg_set_page(&desc->frags[0], sg_page(sg));
desc->frags[0].offset = sg->offset + sg->length - fraglen;
desc->frags[0].length = fraglen;
desc->fragno = 1;
diff --git a/net/sunrpc/xdr.c b/net/sunrpc/xdr.c
index 6a59180..3d1f7cd 100644
--- a/net/sunrpc/xdr.c
+++ b/net/sunrpc/xdr.c
@@ -1059,7 +1059,7 @@ xdr_process_buf(struct xdr_buf *buf, unsigned int offset, unsigned int len,
do {
if (thislen > page_len)
thislen = page_len;
- sg->page = buf->pages[i];
+ sg_set_page(sg, buf->pages[i]);
sg->offset = page_offset;
sg->length = thislen;
ret = actor(sg, data);
diff --git a/net/xfrm/xfrm_algo.c b/net/xfrm/xfrm_algo.c
index 5ced62c..fb2220a 100644
--- a/net/xfrm/xfrm_algo.c
+++ b/net/xfrm/xfrm_algo.c
@@ -552,7 +552,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = virt_to_page(skb->data + offset);
+ sg_set_page(&sg, virt_to_page(skb->data + offset));
sg.offset = (unsigned long)(skb->data + offset) % PAGE_SIZE;
sg.length = copy;
@@ -577,7 +577,7 @@ int skb_icv_walk(const struct sk_buff *skb, struct hash_desc *desc,
if (copy > len)
copy = len;
- sg.page = frag->page;
+ sg_set_page(&sg, frag->page);
sg.offset = frag->page_offset + offset-start;
sg.length = copy;
--
Jens Axboe
On Thu, Oct 18, 2007 at 06:42:46AM -0400, Jeff Garzik wrote:
> Jens Axboe wrote:
>> On Thu, Oct 18 2007, Jeff Garzik wrote:
>>> Jens Axboe wrote:
>>>> That should work as well. WRT ata_sg_is_last(), if we go ahead with my
>>>> recent sg chaining updates, we can keep the test as it would be a single
>>>> conditional and not require any looping.
>>>> Let me know when you have tested this!
>>> The patch I attached to the last email got both sata_mv test boxes
>>> working reliably (so far).
>>>
>>> I worked up a patch that kills ata_sg_is_last() (plus the
>>> max_phys_segments sata_mv fix), see attached. I'm thinking this is what
>>> I like to see in upstream.
>> Great!
>>> Of course, this doesn't explain why ata_sg_is_last() was broken, but
>>> since it's working _and_ slightly more efficient, I don't really care :)
>> Tomo and I agreed to kill sg_last() a few days ago anyways, so this is
>> perfectly fine with me.
>
> Yep, the [attached] patch that kills ata_sg_is_last() is working here on
> both machines that were previously croaking.
>
> It would be nice to get pdc_adma, sata_sil24 and ipr it-works test done,
> but IMO the patch is pretty straightforward and should be OK.
Tested-by: Olof Johansson <[email protected]>
Looks ok on SATA_SIL24 and SATA_MV here on PPC (together with Jens' latest
patch). Both barfed before.
Thanks!
-Olof
On Thu, Oct 18, 2007 at 04:38:36PM +0200, Jens Axboe wrote:
> On Thu, Oct 18 2007, Benny Halevy wrote:
> > On Oct. 18, 2007, 16:05 +0200, Jens Axboe <[email protected]> wrote:
> > > On Thu, Oct 18 2007, Jens Axboe wrote:
> > >> On Thu, Oct 18 2007, Benny Halevy wrote:
> > >>>> return sg;
> > >>>> }
> > >>>> @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> > >>>> ret = sg;
> > >>>>
> > >>>> #endif
> > >>>> +#ifdef CONFIG_DEBUG_SG
> > >>>> + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
> > >>> can it also do BUG_ON(!sg_is_last(sg))?
> > >> That would make sense, definitely. I'll add that.
> > >
> > > BUG_ON(!sg_is_last(ret));
> > >
> > > it should be, not sg. That's what I merged.
> > >
> > right. of course.
>
> OK, that found something interesting - mapping a request may shrink it,
> so we need to update the end marker to move it earlier in the list.
> Basically just a
>
> if (sg)
> __sg_mark_end(sg);
>
> at the bottom of blk_rq_map_sg(). Updated patch below, booted on other
> machines now as well without incident.
PPC needs this. I'm guessing other arches might too. Otherwise seems to boot
and work well (with Jeff's sata patch).
Thanks,
-Olof
diff --git a/include/asm-powerpc/scatterlist.h b/include/asm-powerpc/scatterlist.h
index b9f1dbc..fcf7d55 100644
--- a/include/asm-powerpc/scatterlist.h
+++ b/include/asm-powerpc/scatterlist.h
@@ -14,6 +14,9 @@
#include <asm/dma.h>
struct scatterlist {
+#ifdef CONFIG_DEBUG_SG
+ unsigned long sg_magic;
+#endif
unsigned long page_link;
unsigned int offset;
unsigned int length;
On Thu, Oct 18 2007, Olof Johansson wrote:
> On Thu, Oct 18, 2007 at 04:38:36PM +0200, Jens Axboe wrote:
> > On Thu, Oct 18 2007, Benny Halevy wrote:
> > > On Oct. 18, 2007, 16:05 +0200, Jens Axboe <[email protected]> wrote:
> > > > On Thu, Oct 18 2007, Jens Axboe wrote:
> > > >> On Thu, Oct 18 2007, Benny Halevy wrote:
> > > >>>> return sg;
> > > >>>> }
> > > >>>> @@ -83,6 +96,9 @@ static inline struct scatterlist *sg_last(struct scatterlist *sgl,
> > > >>>> ret = sg;
> > > >>>>
> > > >>>> #endif
> > > >>>> +#ifdef CONFIG_DEBUG_SG
> > > >>>> + BUG_ON(sgl[0].sg_magic != SG_MAGIC);
> > > >>> can it also do BUG_ON(!sg_is_last(sg))?
> > > >> That would make sense, definitely. I'll add that.
> > > >
> > > > BUG_ON(!sg_is_last(ret));
> > > >
> > > > it should be, not sg. That's what I merged.
> > > >
> > > right. of course.
> >
> > OK, that found something interesting - mapping a request may shrink it,
> > so we need to update the end marker to move it earlier in the list.
> > Basically just a
> >
> > if (sg)
> > __sg_mark_end(sg);
> >
> > at the bottom of blk_rq_map_sg(). Updated patch below, booted on other
> > machines now as well without incident.
>
> PPC needs this. I'm guessing other arches might too. Otherwise seems to boot
> and work well (with Jeff's sata patch).
Oh duh, indeed I didn't add that to the archs when converting. Thanks
for the patch, I'll update the rest as well.
And thanks a lot for boot testing it!
--
Jens Axboe
On Thu, 18 Oct 2007, Jens Axboe wrote:
> - unsigned long addr = page_to_phys(s->page) + s->offset;
> + unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
Umm. May I suggest (I haven't read the whole thread yet, maybe somebody
else already did) that
static inline unsigned long sg_phys(struct scatterlist *sg)
{
return page_to_phys(sg_page(sg)) + sg->offset;
}
would be a good thing to have?
Very few drivers should care so much about the *page* itself (or the
offset). That's something that the generic allocation code etc cares
about, but the driver is almost bound to care mostly about the actual DMA
address, so adding that helper function that abstracts the sg access would
be helpful in hiding some of the cruft?
Linus
On Thu, Oct 18 2007, Linus Torvalds wrote:
>
>
> On Thu, 18 Oct 2007, Jens Axboe wrote:
> > - unsigned long addr = page_to_phys(s->page) + s->offset;
> > + unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
>
> Umm. May I suggest (I haven't read the whole thread yet, maybe somebody
> else already did) that
>
> static inline unsigned long sg_phys(struct scatterlist *sg)
> {
> return page_to_phys(sg_page(sg)) + sg->offset;
> }
>
> would be a good thing to have?
Sure thing, it's used quite a lot.
> Very few drivers should care so much about the *page* itself (or the
> offset). That's something that the generic allocation code etc cares
> about, but the driver is almost bound to care mostly about the actual DMA
> address, so adding that helper function that abstracts the sg access would
> be helpful in hiding some of the cruft?
I'll add it to the mix.
--
Jens Axboe
On Thu, Oct 18 2007, Jens Axboe wrote:
> On Thu, Oct 18 2007, Linus Torvalds wrote:
> >
> >
> > On Thu, 18 Oct 2007, Jens Axboe wrote:
> > > - unsigned long addr = page_to_phys(s->page) + s->offset;
> > > + unsigned long addr = page_to_phys(sg_page(s)) + s->offset;
> >
> > Umm. May I suggest (I haven't read the whole thread yet, maybe somebody
> > else already did) that
> >
> > static inline unsigned long sg_phys(struct scatterlist *sg)
> > {
> > return page_to_phys(sg_page(sg)) + sg->offset;
> > }
> >
> > would be a good thing to have?
>
> Sure thing, it's used quite a lot.
Actually, only 11 occurrences in the patch. But still a nice little
cleanup.
--
Jens Axboe
On Thu, 18 Oct 2007 09:55:04 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:
>
>
> On Thu, 18 Oct 2007, Jens Axboe wrote:
> > - unsigned long addr = page_to_phys(s->page) +
> > s->offset;
> > + unsigned long addr = page_to_phys(sg_page(s)) +
> > s->offset;
>
> Umm. May I suggest (I haven't read the whole thread yet, maybe
> somebody else already did) that
>
> static inline unsigned long sg_phys(struct scatterlist *sg)
> {
> return page_to_phys(sg_page(sg)) + sg->offset;
> }
>
> would be a good thing to have?
>
> Very few drivers should care so much about the *page* itself (or the
> offset). That's something that the generic allocation code etc cares
> about, but the driver is almost bound to care mostly about the actual
> DMA address
.... but will that work for systems with IOMMU ? or is it fundamentally
the wrong interface
On Thu, Oct 18 2007, Arjan van de Ven wrote:
> On Thu, 18 Oct 2007 09:55:04 -0700 (PDT)
> Linus Torvalds <[email protected]> wrote:
>
> >
> >
> > On Thu, 18 Oct 2007, Jens Axboe wrote:
> > > - unsigned long addr = page_to_phys(s->page) +
> > > s->offset;
> > > + unsigned long addr = page_to_phys(sg_page(s)) +
> > > s->offset;
> >
> > Umm. May I suggest (I haven't read the whole thread yet, maybe
> > somebody else already did) that
> >
> > static inline unsigned long sg_phys(struct scatterlist *sg)
> > {
> > return page_to_phys(sg_page(sg)) + sg->offset;
> > }
> >
> > would be a good thing to have?
> >
> > Very few drivers should care so much about the *page* itself (or the
> > offset). That's something that the generic allocation code etc cares
> > about, but the driver is almost bound to care mostly about the actual
> > DMA address
>
> .... but will that work for systems with IOMMU ? or is it fundamentally
> the wrong interface
They use foo_to_bus() on the address. sg_phys() should of course only be
used where the user previously did page_to_phys() on the sg page.
--
Jens Axboe
Linus Torvalds wrote:
> Very few drivers should care so much about the *page* itself (or the
> offset). That's something that the generic allocation code etc cares
> about, but the driver is almost bound to care mostly about the actual DMA
> address, so adding that helper function that abstracts the sg access would
> be helpful in hiding some of the cruft?
FWIW libata cares about both. When doing DMA, it cares about the DMA
address. When doing PIO, it cares about the actual page, because it
does a kmap before sending the data through a 16-bit/32-bit data FIFO.
Jeff
On Thu, 18 Oct 2007 19:14:29 +0200
Jens Axboe <[email protected]> wrote:
> On Thu, Oct 18 2007, Arjan van de Ven wrote:
> > On Thu, 18 Oct 2007 09:55:04 -0700 (PDT)
> > Linus Torvalds <[email protected]> wrote:
> >
> > >
> > >
> > > On Thu, 18 Oct 2007, Jens Axboe wrote:
> > > > - unsigned long addr = page_to_phys(s->page) +
> > > > s->offset;
> > > > + unsigned long addr = page_to_phys(sg_page(s)) +
> > > > s->offset;
> > >
> > > Umm. May I suggest (I haven't read the whole thread yet, maybe
> > > somebody else already did) that
> > >
> > > static inline unsigned long sg_phys(struct scatterlist *sg)
> > > {
> > > return page_to_phys(sg_page(sg)) + sg->offset;
> > > }
> > >
> > > would be a good thing to have?
> > >
> > > Very few drivers should care so much about the *page* itself (or the
> > > offset). That's something that the generic allocation code etc cares
> > > about, but the driver is almost bound to care mostly about the actual
> > > DMA address
> >
> > .... but will that work for systems with IOMMU ? or is it fundamentally
> > the wrong interface
>
> They use foo_to_bus() on the address. sg_phys() should of course only be
> used where the user previously did page_to_phys() on the sg page.
I can take care of IOMMU stuff when I'll send IOMMU merging fix
patchset:
http://marc.info/?l=linux-scsi&m=119079718126157&w=2
[Just catching with reading lkml to this post]
On 10/18/07, Jens Axboe <[email protected]> wrote:
>
> Theory - ata_sg_is_last() isn't returning true for the last entry. Can
> you double check that it correcly marks the last entry in mv_fill_sg()?
> Alternatively, just try this patch.
I "hate" to point this out, but I already reported that sata_sil24
fails on 1. Sep.:
http://lkml.org/lkml/2007/9/1/95
In the thread "sata_sil24 broken since 2.6.23-rc4-mm1" I spent over a
week to trace this back to ata_sg_is_last (finally found on 7. Oct):
http://lkml.org/lkml/2007/10/7/43
Thanks for finally patching this now...
Torsten
