2002-11-03 14:11:24

by vasya vasyaev

[permalink] [raw]
Subject: Machine's high load when HIGHMEM is enabled

Hello,

I have some strange kind of problem:
When HIGHMEM-enabled kernel is used, there is too high
CPU load on any task - computer get loaded high while
it is doing some minor, usual jobs (load average grows
significantly).
I don't know how to explain - everything is fine,
computer boots ok, there is no errors (warnings, etc.)
in logs, but as to compare, computer is working slower
that i386...
e.g.: 'gzip some_filename' takes about 5-7 times more
than on HIGHMEM-disabled kernel, ssh login to this
machine takes about 2-3 minutes (computing, disk
operations, network throughput)
I have tried different kernels from RH distro's and
updates:
kernel-2.4.18-17.7
kernel-2.4.18-10
kernel-2.4.18-5

machine has 2 processors, but using SMP/non-SMP
kernels does not matter (I've used both variants)

I have also compiled these kernels from source, and my
results are:
HIGHMEM-disabled kernels works fine, but they can see
only ~900 Mb of memory
the same kernels, but only with HIGHMEM=4G enabled has
such strange effect of high load.

I've also tried following kernels from RH:
kernel-enterprise-2.4.9-34
kernel-enterprise-2.4.9-33
kernel-enterprise-2.4.9-31
and effect is the same...


Any ideas ? need more info?


Here follows this computer configuration:

# lspci
00:00.0 Host bridge: Intel Corp. 440GX - 82443GX Host
bridge
00:01.0 PCI bridge: Intel Corp. 440GX - 82443GX AGP
bridge
00:07.0 ISA bridge: Intel Corp. 82371AB/EB/MB PIIX4
ISA (rev 02)
00:07.1 IDE interface: Intel Corp. 82371AB/EB/MB PIIX4
IDE (rev 01)
00:07.2 USB Controller: Intel Corp. 82371AB/EB/MB
PIIX4 USB (rev 01)
00:07.3 Bridge: Intel Corp. 82371AB/EB/MB PIIX4 ACPI
(rev 02)
00:0f.0 PCI bridge: Intel Corp. 80960RP [i960 RP
Microprocessor/Bridge] (rev 05)
00:0f.1 I2O: Intel Corp. 80960RP [i960RP
Microprocessor] (rev 05)
00:10.0 Ethernet controller: 3Com Corporation 3c905B
100BaseTX [Cyclone] (rev 64)
01:00.0 VGA compatible controller: ATI Technologies
Inc 3D Rage IIC AGP (rev 3a)


# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 7
model name : Pentium III (Katmai)
stepping : 3
cpu MHz : 551.261
cache size : 32 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips : 1101.00

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 7
model name : Pentium III (Katmai)
stepping : 3
cpu MHz : 551.261
cache size : 32 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips : 1101.00


# cat /proc/meminfo
total: used: free: shared: buffers:
cached:
Mem: 2100666368 80003072 2020663296 0 5554176
43016192
Swap: 1069277184 0 1069277184
MemTotal: 2051432 kB
MemFree: 1973304 kB
MemShared: 0 kB
Buffers: 5424 kB
Cached: 42008 kB
SwapCached: 0 kB
Active: 34376 kB
Inact_dirty: 13056 kB
Inact_clean: 0 kB
Inact_target: 522240 kB
HighTotal: 1171456 kB
HighFree: 1124452 kB
LowTotal: 879976 kB
LowFree: 848852 kB
SwapTotal: 1044216 kB
SwapFree: 1044216 kB


# lsmod
Module Size Used by Not tainted
3c59x 31976 1
reiserfs 181312 1
megaraid 27808 2
sd_mod 13756 2
scsi_mod 123524 2 [megaraid sd_mod]


binutils-2.11.93.0.2-11
gcc-2.96-112
glibc-2.2.5-40


Thanks in advance.


__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/


2002-11-03 14:26:03

by Brian Gerst

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

vasya vasyaev wrote:
> Hello,
>
> I have some strange kind of problem:
> When HIGHMEM-enabled kernel is used, there is too high
> CPU load on any task - computer get loaded high while
> it is doing some minor, usual jobs (load average grows
> significantly).

2.4 can only do I/O to and from lowmem. This means highmem pages have
to use bounce buffers in lowmem, and th edata is copied to/from highmem
which is causing the cpu load. This has been corrected in 2.5, which
can do I/O to any page the device can DMA from.

--
Brian Gerst

2002-11-03 14:54:32

by Jens Axboe

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

On Sun, Nov 03 2002, Brian Gerst wrote:
> vasya vasyaev wrote:
> >Hello,
> >
> >I have some strange kind of problem:
> >When HIGHMEM-enabled kernel is used, there is too high
> >CPU load on any task - computer get loaded high while
> >it is doing some minor, usual jobs (load average grows
> >significantly).
>
> 2.4 can only do I/O to and from lowmem. This means highmem pages have

2.4.19 and below, 2.4.20-pre/rc can dma to/from highmem pages just fine.

> to use bounce buffers in lowmem, and th edata is copied to/from highmem
> which is causing the cpu load. This has been corrected in 2.5, which
> can do I/O to any page the device can DMA from.

I seriously doubt this is his problem, sounds like something else. For
gzip to be disk bound (and thus bounce bound) you would need a seriously
fast cpu. And the ssh problem cannot be explained by bouncing either.

--
Jens Axboe

2002-11-03 15:25:37

by Martin J. Bligh

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

> Any ideas ? need more info?

Can you profile it? Either get the oprofile patch, or put
"profile=2" on the kernel command line, and use readprofile
(built in). You'll have to search around for more extensive
instructions, but hopefully that'll tell you what's burning
the CPU time.

M.



2002-11-03 19:50:16

by Denis Vlasenko

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

On 3 November 2002 13:00, Jens Axboe wrote:
> On Sun, Nov 03 2002, Brian Gerst wrote:
> > vasya vasyaev wrote:
> > >Hello,
> > >
> > >I have some strange kind of problem:
> > >When HIGHMEM-enabled kernel is used, there is too high
> > >CPU load on any task - computer get loaded high while
> > >it is doing some minor, usual jobs (load average grows
> > >significantly).
> >
> > 2.4 can only do I/O to and from lowmem. This means highmem pages
> > have
>
> 2.4.19 and below, 2.4.20-pre/rc can dma to/from highmem pages just
> fine.
>
> > to use bounce buffers in lowmem, and th edata is copied to/from
> > highmem which is causing the cpu load. This has been corrected in
> > 2.5, which can do I/O to any page the device can DMA from.
>
> I seriously doubt this is his problem, sounds like something else.
> For gzip to be disk bound (and thus bounce bound) you would need a
> seriously fast cpu. And the ssh problem cannot be explained by
> bouncing either.

bad mtrr?
--
vda

2002-11-04 18:48:09

by vasya vasyaev

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

Hello,

First of all - thanks to these people, who responded
to my question.

I have some news...

I've tried kernels:
2.4.19 - the same result
2.5.44 - the same result
2.5.45 - the same result

If I take 1 Gb of memory away, then computer works
much better, faster (something like without enabled
HIGHMEM at all).
The same effect takes place if I say mem=1024M while
physically box has 2Gb of RAM - everything is fine!
But if I start HIGHMEM enabled kernel on this box (2Gb
RAM), then it works too slowly...


The questions are:
1. is there anyone on the list, who has the same or
near configuration ? (main is that the box has >=2Gb
of RAM)
2. do you experience the same problems ?
3. maybe it's not a problem at all?!? I mean maybe
linux kernel works this way with such amount of memory
and there is no problem with that?
4. the case I've described =
http://www.cs.helsinki.fi/linux/linux-kernel/2001-02/0292.html
?

and finally, if this could be solved - please tell,
how should I set up the kernel/kernel settings to get
appropriate results in performance ?


Thank you.


--- Brian Gerst <[email protected]> wrote:
> 2.4 can only do I/O to and from lowmem. This means
> highmem pages have
> to use bounce buffers in lowmem, and th edata is
> copied to/from highmem
> which is causing the cpu load. This has been
> corrected in 2.5, which
> can do I/O to any page the device can DMA from.


__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/

2002-11-04 19:21:30

by Andrew Morton

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

vasya vasyaev wrote:
>
> Hello,
>
> First of all - thanks to these people, who responded
> to my question.
>
> I have some news...
>
> I've tried kernels:
> 2.4.19 - the same result
> 2.5.44 - the same result
> 2.5.45 - the same result
>
> If I take 1 Gb of memory away, then computer works
> much better, faster (something like without enabled
> HIGHMEM at all).
> The same effect takes place if I say mem=1024M while
> physically box has 2Gb of RAM - everything is fine!
> But if I start HIGHMEM enabled kernel on this box (2Gb
> RAM), then it works too slowly...
>

Please ensure that the mtrr driver is enabled in kernel config,
boot with mem=2G and send the output of `cat /proc/mtrr'.

Also, `dmesg | head -120' would be interesting.

2002-11-04 19:42:55

by Nakajima, Jun

[permalink] [raw]
Subject: RE: Machine's high load when HIGHMEM is enabled

Also try to disable BIOS remapping in the BIOS setup menu, if any. I know
some BIOS that forgot to reset the proper memory attribute in the MTRR(s)
after the remapping.

Jun

> -----Original Message-----
> From: Andrew Morton [mailto:[email protected]]
> Sent: Monday, November 04, 2002 11:28 AM
> To: vasya vasyaev
> Cc: [email protected]
> Subject: Re: Machine's high load when HIGHMEM is enabled
>
> vasya vasyaev wrote:
> >
> > Hello,
> >
> > First of all - thanks to these people, who responded
> > to my question.
> >
> > I have some news...
> >
> > I've tried kernels:
> > 2.4.19 - the same result
> > 2.5.44 - the same result
> > 2.5.45 - the same result
> >
> > If I take 1 Gb of memory away, then computer works
> > much better, faster (something like without enabled
> > HIGHMEM at all).
> > The same effect takes place if I say mem=1024M while
> > physically box has 2Gb of RAM - everything is fine!
> > But if I start HIGHMEM enabled kernel on this box (2Gb
> > RAM), then it works too slowly...
> >
>
> Please ensure that the mtrr driver is enabled in kernel config,
> boot with mem=2G and send the output of `cat /proc/mtrr'.
>
> Also, `dmesg | head -120' would be interesting.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2002-11-05 09:01:26

by vasya vasyaev

[permalink] [raw]
Subject: RE: Machine's high load when HIGHMEM is enabled

Could you please explain what is this remapping ?
or how it can be named in other words - I cannot find
something like this in BIOS setup menu...


--- "Nakajima, Jun" <[email protected]> wrote:
> Also try to disable BIOS remapping in the BIOS setup
> menu, if any. I know
> some BIOS that forgot to reset the proper memory
> attribute in the MTRR(s)
> after the remapping.


__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/

2002-11-05 10:30:14

by vasya vasyaev

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

Hello,

Yes, mtrr driver is enabled in kernel config.
I have not tried to pass mem=2G at kernel boot,
because it can see the whole memory without this
option (or should I ? will it change something ?)


And here is the requested information from that box:
# cat /proc/mtrr
reg00: base=0x00000000 ( 0MB), size=1024MB:
write-back, count=1
reg01: base=0x40000000 (1024MB), size= 512MB:
write-back, count=1
reg02: base=0x60000000 (1536MB), size= 256MB:
write-back, count=1
reg03: base=0x70000000 (1792MB), size= 128MB:
write-back, count=1
reg04: base=0x78000000 (1920MB), size= 64MB:
write-back, count=1
reg05: base=0x7c000000 (1984MB), size= 32MB:
write-back, count=1



# dmesg
Linux version 2.4.18-17.7asp ([email protected]*)
(gcc version 2.96 20000731 (ASPLinux 7.3 2.96-112)) #2
SMP 2
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00
(usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000
(reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000
(reserved)
BIOS-e820: 0000000000100000 - 000000007f800000
(usable)
BIOS-e820: 00000000fec00000 - 00000000fec01000
(reserved)
BIOS-e820: 00000000fee00000 - 00000000fee01000
(reserved)
BIOS-e820: 00000000fffc0000 - 0000000100000000
(reserved)
1144MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000fb4f0
hm, page 000fb000 reserved twice.
hm, page 000fc000 reserved twice.
hm, page 000f2000 reserved twice.
hm, page 000f3000 reserved twice.
On node 0 totalpages: 522240
zone(0): 4096 pages.
zone(1): 225280 pages.
zone(2): 292864 pages.
Intel MultiProcessor Specification v1.4
Virtual Wire compatibility mode.
OEM ID: INTEL Product ID: 440BX APIC at:
0xFEE00000
Processor #0 Pentium(tm) Pro APIC version 17
Processor #1 Pentium(tm) Pro APIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Processors: 2
Kernel command line: BOOT_IMAGE=17.7aspSMP3 ro
root=801
Initializing CPU#0
Detected 551.260 MHz processor.
Console: colour VGA+ 80x60
Calibrating delay loop... 1095.84 BogoMIPS
Memory: 2051820k/2088960k available (1660k kernel
code, 32528k reserved, 1072k data, 144k init, 1171456k
highme)
Dentry cache hash table entries: 262144 (order: 9,
2097152 bytes)
Inode cache hash table entries: 131072 (order: 8,
1048576 bytes)
Mount cache hash table entries: 32768 (order: 6,
262144 bytes)
ramfs: mounted with options: <defaults>
ramfs: max_pages=257005 max_file_pages=0 max_inodes=0
max_dentries=257005
Buffer cache hash table entries: 131072 (order: 7,
524288 bytes)
Page-cache hash table entries: 524288 (order: 9,
2097152 bytes)
CPU: Before vendor init, caps: 0383fbff 00000000
00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: After vendor init, caps: 0383fbff 00000000
00000000 00000000
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU: After generic, caps: 0383fbff 00000000
00000000 00000000
CPU: Common caps: 0383fbff 00000000
00000000 00000000
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.40 (20010327) Richard Gooch
([email protected])
mtrr: detected mtrr type: Intel
CPU: Before vendor init, caps: 0383fbff 00000000
00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: After vendor init, caps: 0383fbff 00000000
00000000 00000000
Intel machine check reporting enabled on CPU#0.
CPU: After generic, caps: 0383fbff 00000000
00000000 00000000
CPU: Common caps: 0383fbff 00000000
00000000 00000000
CPU0: Intel Pentium III (Katmai) stepping 03
per-CPU timeslice cutoff: 91.41 usecs.
task migration cache decay timeout: 1 msecs.
enabled ExtINT on CPU#0
ESR value before enabling vector: 00000004
ESR value after enabling vector: 00000000
Booting processor 1/1 eip 2000
Initializing CPU#1
masked ExtINT on CPU#1
ESR value before enabling vector: 00000000
ESR value after enabling vector: 00000000
Calibrating delay loop... 1099.39 BogoMIPS
CPU: Before vendor init, caps: 0383fbff 00000000
00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: After vendor init, caps: 0383fbff 00000000
00000000 00000000
Intel machine check reporting enabled on CPU#1.
CPU: After generic, caps: 0383fbff 00000000
00000000 00000000
CPU: Common caps: 0383fbff 00000000
00000000 00000000
CPU1: Intel Pentium III (Katmai) stepping 03
Total of 2 processors activated (2194.23 BogoMIPS).
ENABLING IO-APIC IRQs
Setting 2 in the phys_id_present_map
...changing IO-APIC physical APIC ID to 2 ... ok.
init IO_APIC IRQs
IO-APIC (apicid-pin) 2-0, 2-10, 2-11, 2-18, 2-19,
2-20, 2-21, 2-22, 2-23 not connected.
..TIMER: vector=0x31 pin1=2 pin2=0
number of MP IRQ sources: 17.
number of IO-APIC #2 registers: 24.
testing the IO APIC.......................

IO APIC #2......
.... register #00: 02000000
....... : physical APIC id: 02
.... register #01: 00170011
....... : max redirection entries: 0017
....... : PRQ implemented: 0
....... : IO APIC version: 0011
.... register #02: 00000000
....... : arbitration: 00
.... IRQ redirection table:
NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
00 000 00 1 0 0 0 0 0 0 00
01 003 03 0 0 0 0 0 1 1 39
02 002 02 0 0 0 0 0 1 1 31
03 003 03 0 0 0 0 0 1 1 41
04 003 03 0 0 0 0 0 1 1 49
05 003 03 0 0 0 0 0 1 1 51
06 003 03 0 0 0 0 0 1 1 59
07 003 03 0 0 0 0 0 1 1 61
08 003 03 0 0 0 0 0 1 1 69
09 003 03 0 0 0 0 0 1 1 71
0a 000 00 1 0 0 0 0 0 0 00
0b 000 00 1 0 0 0 0 0 0 00
0c 003 03 0 0 0 0 0 1 1 79
0d 003 03 0 0 0 0 0 1 1 81
0e 003 03 0 0 0 0 0 1 1 89
0f 003 03 0 0 0 0 0 1 1 91
10 003 03 1 1 0 1 0 1 1 99
11 003 03 1 1 0 1 0 1 1 A1
12 000 00 1 0 0 0 0 0 0 00
13 000 00 1 0 0 0 0 0 0 00
14 000 00 1 0 0 0 0 0 0 00
15 000 00 1 0 0 0 0 0 0 00
16 000 00 1 0 0 0 0 0 0 00
17 000 00 1 0 0 0 0 0 0 00
IRQ to pin mappings:
IRQ0 -> 0:2
IRQ1 -> 0:1
IRQ3 -> 0:3
IRQ4 -> 0:4
IRQ5 -> 0:5
IRQ6 -> 0:6
IRQ7 -> 0:7
IRQ8 -> 0:8
IRQ9 -> 0:9
IRQ10 -> 0:16
IRQ11 -> 0:17
IRQ12 -> 0:12
IRQ13 -> 0:13
IRQ14 -> 0:14
IRQ15 -> 0:15
.................................... done.
Using local APIC timer interrupts.
calibrating APIC timer ...
..... CPU clock speed is 551.0245 MHz.
..... host bus clock speed is 100.0399 MHz.
cpu: 0, clocks: 195699, slice: 65233
CPU0<T0:195696,T1:130448,D:15,S:65233,C:195699>
cpu: 1, clocks: 195699, slice: 65233
CPU1<T0:195696,T1:65216,D:14,S:65233,C:195699>
checking TSC synchronization across CPUs: passed.
migration_task 0 on cpu=0
migration_task 1 on cpu=1
PCI: PCI BIOS revision 2.10 entry at 0xfdb81, last
bus=2
PCI: Using configuration type 1
PCI: Probing PCI hardware
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Linux NET4.0 for Linux 2.4
Based upon Swansea University Computer Society
NET3.039
Initializing RT netlink socket
Starting kswapd
allocated 64 pages and 64 bhs reserved for the highmem
bounces
VFS: Disk quotas vdquot_6.5.1
SGI XFS CVS-09/19/02:05 with ACLs, quota, no debug
enabled
pty: 2048 Unix98 ptys configured
Serial driver version 5.05c (2001-07-08) with
MANY_PORTS MULTIPORT SHARE_IRQ SERIAL_PCI ISAPNP
enabled
ttyS0 at 0x03f8 (irq = 4) is a 16550A
ttyS1 at 0x02f8 (irq = 3) is a 16550A
Real Time Clock Driver v1.10e
oprofile: APIC was already enabled
oprofile 0.2 loaded, major 254
block: 1024 slots per queue, batch=256
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes;
override with idebus=xx
PIIX4: IDE controller on PCI bus 00 dev 39
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings:
hda:pio, hdb:pio
ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings:
hdc:DMA, hdd:pio
hdc: ASUS CD-S400/A, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
ide-floppy driver 0.99.newide
Floppy drive(s): fd0 is 1.44M
FDC 0 is a post-1991 82077
NET4: Frame Diverter 0.46
RAMDISK driver initialized: 16 RAM disks of 4096K size
1024 blocksize
ide-floppy driver 0.99.newide
md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
pci_hotplug: PCI Hot Plug PCI Core version: 0.4
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 16384 buckets,
128Kbytes
TCP: Hash tables configured (established 262144 bind
65536)
Linux IP multicast router 0.06 plus PIM-SM
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
RAMDISK: Compressed image found at block 0
Freeing initrd memory: 231k freed
VFS: Mounted root (ext2 filesystem).
SCSI subsystem driver Revision: 1.00
kmod: failed to exec /sbin/modprobe -s -k
scsi_hostadapter, errno = 2
megaraid: v1.15 (Release Date: Thu Apr 19 09:39:35 EDT
2001)
megaraid: found 0x8086:0x1960:idx 0:bus 0:slot 15:func
1
scsi0 : Found a MegaRAID controller at 0xf884e000,
IRQ: 10
megaraid: [GH8E:1.48] detected 2 logical drives
scsi0 : AMI MegaRAID GH8E 254 commands 16 targs 3
chans 8 luns
scsi0: scanning channel 1 for devices.
scsi0: scanning channel 2 for devices.
scsi0: scanning channel 3 for devices.
scsi0: scanning virtual channel for logical drives.
Vendor: MegaRAID Model: LD0 RAID0 17494R Rev: GH8E
Type: Direct-Access ANSI SCSI
revision: 02
Vendor: MegaRAID Model: LD1 RAID0 52482R Rev: GH8E
Type: Direct-Access ANSI SCSI
revision: 02
Attached scsi disk sda at scsi0, channel 3, id 0, lun
0
Attached scsi disk sdb at scsi0, channel 3, id 0, lun
1
SCSI device sda: 35827712 512-byte hdwr sectors (18344
MB)
Partition check:
sda: sda1 sda2
SCSI device sdb: 107483136 512-byte hdwr sectors
(55031 MB)
sdb: sdb1
reiserfs: checking transaction log (device 08:01) ...
Using r5 hash to sort names
ReiserFS version 3.6.25
Freeing unused kernel memory: 144k freed
Adding Swap: 1044216k swap-space (priority -1)
3c59x: Donald Becker and others.
http://www.scyld.com/network/vortex.html
00:10.0: 3Com PCI 3c905B Cyclone 100baseTx at 0xec00.
Vers LK1.1.16
divert: allocating divert_blk for eth0





--- Andrew Morton <[email protected]> wrote:
>
> Please ensure that the mtrr driver is enabled in
> kernel config,
> boot with mem=2G and send the output of `cat
> /proc/mtrr'.
>
> Also, `dmesg | head -120' would be interesting.


__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/

2002-11-05 21:04:05

by vasya vasyaev

[permalink] [raw]
Subject: RE: Machine's high load when HIGHMEM is enabled

By the way, please CC all of your answers, because I'm
not soubscribed to the list.
Thank you.


--- "Nakajima, Jun" <[email protected]> wrote:
> Also try to disable BIOS remapping in the BIOS setup
> menu, if any. I know
> some BIOS that forgot to reset the proper memory
> attribute in the MTRR(s)
> after the remapping.
>
> Jun
>
> > -----Original Message-----
> > From: Andrew Morton [mailto:[email protected]]
> > Sent: Monday, November 04, 2002 11:28 AM
> > To: vasya vasyaev
> > Cc: [email protected]
> > Subject: Re: Machine's high load when HIGHMEM is
> enabled
> >
> > vasya vasyaev wrote:
> > >
> > > Hello,
> > >
> > > First of all - thanks to these people, who
> responded
> > > to my question.
> > >
> > > I have some news...
> > >
> > > I've tried kernels:
> > > 2.4.19 - the same result
> > > 2.5.44 - the same result
> > > 2.5.45 - the same result
> > >
> > > If I take 1 Gb of memory away, then computer
> works
> > > much better, faster (something like without
> enabled
> > > HIGHMEM at all).
> > > The same effect takes place if I say mem=1024M
> while
> > > physically box has 2Gb of RAM - everything is
> fine!
> > > But if I start HIGHMEM enabled kernel on this
> box (2Gb
> > > RAM), then it works too slowly...
> > >
> >
> > Please ensure that the mtrr driver is enabled in
> kernel config,
> > boot with mem=2G and send the output of `cat
> /proc/mtrr'.
> >
> > Also, `dmesg | head -120' would be interesting.



__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/

2002-11-05 21:32:27

by Nakajima, Jun

[permalink] [raw]
Subject: RE: Machine's high load when HIGHMEM is enabled

To me it looks this MTRR does not cover the memory range reported by E820.

> reg05: base=0x7c000000 (1984MB), size= 32MB:
> write-back, count=1

This covers [0x7c000000 - 0x7e000000).

> BIOS-e820: 0000000000100000 - 000000007f800000
> (usable)
But it says memory is available up to 0x7f800000.
So try mem=2016M ?
(2048 - 32 = 2016)

I guess you are mixing various memory modules.

Jun


> -----Original Message-----
> From: vasya vasyaev [mailto:[email protected]]
> Sent: Tuesday, November 05, 2002 2:37 AM
> To: Andrew Morton
> Cc: [email protected]
> Subject: Re: Machine's high load when HIGHMEM is enabled
>
> Hello,
>
> Yes, mtrr driver is enabled in kernel config.
> I have not tried to pass mem=2G at kernel boot,
> because it can see the whole memory without this
> option (or should I ? will it change something ?)
>
>
> And here is the requested information from that box:
> # cat /proc/mtrr
> reg00: base=0x00000000 ( 0MB), size=1024MB:
> write-back, count=1
> reg01: base=0x40000000 (1024MB), size= 512MB:
> write-back, count=1
> reg02: base=0x60000000 (1536MB), size= 256MB:
> write-back, count=1
> reg03: base=0x70000000 (1792MB), size= 128MB:
> write-back, count=1
> reg04: base=0x78000000 (1920MB), size= 64MB:
> write-back, count=1
> reg05: base=0x7c000000 (1984MB), size= 32MB:
> write-back, count=1
>
>
>
> # dmesg
> Linux version 2.4.18-17.7asp ([email protected]*)
> (gcc version 2.96 20000731 (ASPLinux 7.3 2.96-112)) #2
> SMP 2
> BIOS-provided physical RAM map:
> BIOS-e820: 0000000000000000 - 000000000009fc00
> (usable)
> BIOS-e820: 000000000009fc00 - 00000000000a0000
> (reserved)
> BIOS-e820: 00000000000e0000 - 0000000000100000
> (reserved)
> BIOS-e820: 0000000000100000 - 000000007f800000
> (usable)
> BIOS-e820: 00000000fec00000 - 00000000fec01000
> (reserved)
> BIOS-e820: 00000000fee00000 - 00000000fee01000
> (reserved)
> BIOS-e820: 00000000fffc0000 - 0000000100000000
> (reserved)
> 1144MB HIGHMEM available.
> 896MB LOWMEM available.
> found SMP MP-table at 000fb4f0
> hm, page 000fb000 reserved twice.
> hm, page 000fc000 reserved twice.
> hm, page 000f2000 reserved twice.
> hm, page 000f3000 reserved twice.
> On node 0 totalpages: 522240
> zone(0): 4096 pages.
> zone(1): 225280 pages.
> zone(2): 292864 pages.
> Intel MultiProcessor Specification v1.4
> Virtual Wire compatibility mode.
> OEM ID: INTEL Product ID: 440BX APIC at:
> 0xFEE00000
> Processor #0 Pentium(tm) Pro APIC version 17
> Processor #1 Pentium(tm) Pro APIC version 17
> I/O APIC #2 Version 17 at 0xFEC00000.
> Processors: 2
> Kernel command line: BOOT_IMAGE=17.7aspSMP3 ro
> root=801
> Initializing CPU#0
> Detected 551.260 MHz processor.
> Console: colour VGA+ 80x60
> Calibrating delay loop... 1095.84 BogoMIPS
> Memory: 2051820k/2088960k available (1660k kernel
> code, 32528k reserved, 1072k data, 144k init, 1171456k
> highme)
> Dentry cache hash table entries: 262144 (order: 9,
> 2097152 bytes)
> Inode cache hash table entries: 131072 (order: 8,
> 1048576 bytes)
> Mount cache hash table entries: 32768 (order: 6,
> 262144 bytes)
> ramfs: mounted with options: <defaults>
> ramfs: max_pages=257005 max_file_pages=0 max_inodes=0
> max_dentries=257005
> Buffer cache hash table entries: 131072 (order: 7,
> 524288 bytes)
> Page-cache hash table entries: 524288 (order: 9,
> 2097152 bytes)
> CPU: Before vendor init, caps: 0383fbff 00000000
> 00000000, vendor = 0
> CPU: L1 I cache: 16K, L1 D cache: 16K
> CPU: After vendor init, caps: 0383fbff 00000000
> 00000000 00000000
> Intel machine check architecture supported.
> Intel machine check reporting enabled on CPU#0.
> CPU: After generic, caps: 0383fbff 00000000
> 00000000 00000000
> CPU: Common caps: 0383fbff 00000000
> 00000000 00000000
> Enabling fast FPU save and restore... done.
> Enabling unmasked SIMD FPU exception support... done.
> Checking 'hlt' instruction... OK.
> POSIX conformance testing by UNIFIX
> mtrr: v1.40 (20010327) Richard Gooch
> ([email protected])
> mtrr: detected mtrr type: Intel
> CPU: Before vendor init, caps: 0383fbff 00000000
> 00000000, vendor = 0
> CPU: L1 I cache: 16K, L1 D cache: 16K
> CPU: After vendor init, caps: 0383fbff 00000000
> 00000000 00000000
> Intel machine check reporting enabled on CPU#0.
> CPU: After generic, caps: 0383fbff 00000000
> 00000000 00000000
> CPU: Common caps: 0383fbff 00000000
> 00000000 00000000
> CPU0: Intel Pentium III (Katmai) stepping 03
> per-CPU timeslice cutoff: 91.41 usecs.
> task migration cache decay timeout: 1 msecs.
> enabled ExtINT on CPU#0
> ESR value before enabling vector: 00000004
> ESR value after enabling vector: 00000000
> Booting processor 1/1 eip 2000
> Initializing CPU#1
> masked ExtINT on CPU#1
> ESR value before enabling vector: 00000000
> ESR value after enabling vector: 00000000
> Calibrating delay loop... 1099.39 BogoMIPS
> CPU: Before vendor init, caps: 0383fbff 00000000
> 00000000, vendor = 0
> CPU: L1 I cache: 16K, L1 D cache: 16K
> CPU: After vendor init, caps: 0383fbff 00000000
> 00000000 00000000
> Intel machine check reporting enabled on CPU#1.
> CPU: After generic, caps: 0383fbff 00000000
> 00000000 00000000
> CPU: Common caps: 0383fbff 00000000
> 00000000 00000000
> CPU1: Intel Pentium III (Katmai) stepping 03
> Total of 2 processors activated (2194.23 BogoMIPS).
> ENABLING IO-APIC IRQs
> Setting 2 in the phys_id_present_map
> ...changing IO-APIC physical APIC ID to 2 ... ok.
> init IO_APIC IRQs
> IO-APIC (apicid-pin) 2-0, 2-10, 2-11, 2-18, 2-19,
> 2-20, 2-21, 2-22, 2-23 not connected.
> ..TIMER: vector=0x31 pin1=2 pin2=0
> number of MP IRQ sources: 17.
> number of IO-APIC #2 registers: 24.
> testing the IO APIC.......................
>
> IO APIC #2......
> .... register #00: 02000000
> ....... : physical APIC id: 02
> .... register #01: 00170011
> ....... : max redirection entries: 0017
> ....... : PRQ implemented: 0
> ....... : IO APIC version: 0011
> .... register #02: 00000000
> ....... : arbitration: 00
> .... IRQ redirection table:
> NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> 00 000 00 1 0 0 0 0 0 0 00
> 01 003 03 0 0 0 0 0 1 1 39
> 02 002 02 0 0 0 0 0 1 1 31
> 03 003 03 0 0 0 0 0 1 1 41
> 04 003 03 0 0 0 0 0 1 1 49
> 05 003 03 0 0 0 0 0 1 1 51
> 06 003 03 0 0 0 0 0 1 1 59
> 07 003 03 0 0 0 0 0 1 1 61
> 08 003 03 0 0 0 0 0 1 1 69
> 09 003 03 0 0 0 0 0 1 1 71
> 0a 000 00 1 0 0 0 0 0 0 00
> 0b 000 00 1 0 0 0 0 0 0 00
> 0c 003 03 0 0 0 0 0 1 1 79
> 0d 003 03 0 0 0 0 0 1 1 81
> 0e 003 03 0 0 0 0 0 1 1 89
> 0f 003 03 0 0 0 0 0 1 1 91
> 10 003 03 1 1 0 1 0 1 1 99
> 11 003 03 1 1 0 1 0 1 1 A1
> 12 000 00 1 0 0 0 0 0 0 00
> 13 000 00 1 0 0 0 0 0 0 00
> 14 000 00 1 0 0 0 0 0 0 00
> 15 000 00 1 0 0 0 0 0 0 00
> 16 000 00 1 0 0 0 0 0 0 00
> 17 000 00 1 0 0 0 0 0 0 00
> IRQ to pin mappings:
> IRQ0 -> 0:2
> IRQ1 -> 0:1
> IRQ3 -> 0:3
> IRQ4 -> 0:4
> IRQ5 -> 0:5
> IRQ6 -> 0:6
> IRQ7 -> 0:7
> IRQ8 -> 0:8
> IRQ9 -> 0:9
> IRQ10 -> 0:16
> IRQ11 -> 0:17
> IRQ12 -> 0:12
> IRQ13 -> 0:13
> IRQ14 -> 0:14
> IRQ15 -> 0:15
> .................................... done.
> Using local APIC timer interrupts.
> calibrating APIC timer ...
> ..... CPU clock speed is 551.0245 MHz.
> ..... host bus clock speed is 100.0399 MHz.
> cpu: 0, clocks: 195699, slice: 65233
> CPU0<T0:195696,T1:130448,D:15,S:65233,C:195699>
> cpu: 1, clocks: 195699, slice: 65233
> CPU1<T0:195696,T1:65216,D:14,S:65233,C:195699>
> checking TSC synchronization across CPUs: passed.
> migration_task 0 on cpu=0
> migration_task 1 on cpu=1
> PCI: PCI BIOS revision 2.10 entry at 0xfdb81, last
> bus=2
> PCI: Using configuration type 1
> PCI: Probing PCI hardware
> isapnp: Scanning for PnP cards...
> isapnp: No Plug & Play device found
> Linux NET4.0 for Linux 2.4
> Based upon Swansea University Computer Society
> NET3.039
> Initializing RT netlink socket
> Starting kswapd
> allocated 64 pages and 64 bhs reserved for the highmem
> bounces
> VFS: Disk quotas vdquot_6.5.1
> SGI XFS CVS-09/19/02:05 with ACLs, quota, no debug
> enabled
> pty: 2048 Unix98 ptys configured
> Serial driver version 5.05c (2001-07-08) with
> MANY_PORTS MULTIPORT SHARE_IRQ SERIAL_PCI ISAPNP
> enabled
> ttyS0 at 0x03f8 (irq = 4) is a 16550A
> ttyS1 at 0x02f8 (irq = 3) is a 16550A
> Real Time Clock Driver v1.10e
> oprofile: APIC was already enabled
> oprofile 0.2 loaded, major 254
> block: 1024 slots per queue, batch=256
> Uniform Multi-Platform E-IDE driver Revision: 6.31
> ide: Assuming 33MHz system bus speed for PIO modes;
> override with idebus=xx
> PIIX4: IDE controller on PCI bus 00 dev 39
> PIIX4: chipset revision 1
> PIIX4: not 100% native mode: will probe irqs later
> ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings:
> hda:pio, hdb:pio
> ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings:
> hdc:DMA, hdd:pio
> hdc: ASUS CD-S400/A, ATAPI CD/DVD-ROM drive
> ide1 at 0x170-0x177,0x376 on irq 15
> ide-floppy driver 0.99.newide
> Floppy drive(s): fd0 is 1.44M
> FDC 0 is a post-1991 82077
> NET4: Frame Diverter 0.46
> RAMDISK driver initialized: 16 RAM disks of 4096K size
> 1024 blocksize
> ide-floppy driver 0.99.newide
> md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
> md: Autodetecting RAID arrays.
> md: autorun ...
> md: ... autorun DONE.
> pci_hotplug: PCI Hot Plug PCI Core version: 0.4
> NET4: Linux TCP/IP 1.0 for NET4.0
> IP Protocols: ICMP, UDP, TCP, IGMP
> IP: routing cache hash table of 16384 buckets,
> 128Kbytes
> TCP: Hash tables configured (established 262144 bind
> 65536)
> Linux IP multicast router 0.06 plus PIM-SM
> NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
> RAMDISK: Compressed image found at block 0
> Freeing initrd memory: 231k freed
> VFS: Mounted root (ext2 filesystem).
> SCSI subsystem driver Revision: 1.00
> kmod: failed to exec /sbin/modprobe -s -k
> scsi_hostadapter, errno = 2
> megaraid: v1.15 (Release Date: Thu Apr 19 09:39:35 EDT
> 2001)
> megaraid: found 0x8086:0x1960:idx 0:bus 0:slot 15:func
> 1
> scsi0 : Found a MegaRAID controller at 0xf884e000,
> IRQ: 10
> megaraid: [GH8E:1.48] detected 2 logical drives
> scsi0 : AMI MegaRAID GH8E 254 commands 16 targs 3
> chans 8 luns
> scsi0: scanning channel 1 for devices.
> scsi0: scanning channel 2 for devices.
> scsi0: scanning channel 3 for devices.
> scsi0: scanning virtual channel for logical drives.
> Vendor: MegaRAID Model: LD0 RAID0 17494R Rev: GH8E
> Type: Direct-Access ANSI SCSI
> revision: 02
> Vendor: MegaRAID Model: LD1 RAID0 52482R Rev: GH8E
> Type: Direct-Access ANSI SCSI
> revision: 02
> Attached scsi disk sda at scsi0, channel 3, id 0, lun
> 0
> Attached scsi disk sdb at scsi0, channel 3, id 0, lun
> 1
> SCSI device sda: 35827712 512-byte hdwr sectors (18344
> MB)
> Partition check:
> sda: sda1 sda2
> SCSI device sdb: 107483136 512-byte hdwr sectors
> (55031 MB)
> sdb: sdb1
> reiserfs: checking transaction log (device 08:01) ...
> Using r5 hash to sort names
> ReiserFS version 3.6.25
> Freeing unused kernel memory: 144k freed
> Adding Swap: 1044216k swap-space (priority -1)
> 3c59x: Donald Becker and others.
> http://www.scyld.com/network/vortex.html
> 00:10.0: 3Com PCI 3c905B Cyclone 100baseTx at 0xec00.
> Vers LK1.1.16
> divert: allocating divert_blk for eth0
>
>
>
>
>
> --- Andrew Morton <[email protected]> wrote:
> >
> > Please ensure that the mtrr driver is enabled in
> > kernel config,
> > boot with mem=2G and send the output of `cat
> > /proc/mtrr'.
> >
> > Also, `dmesg | head -120' would be interesting.
>
>
> __________________________________________________
> Do you Yahoo!?
> HotJobs - Search new jobs daily now
> http://hotjobs.yahoo.com/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2002-11-06 10:08:08

by vasya vasyaev

[permalink] [raw]
Subject: RE: Machine's high load when HIGHMEM is enabled

Hello,
It seems "mem=2016M" is what we need, box works
approximately as fast as without enabled HIGHMEM.
Thank you!

BTW, we are using 4x512 Mb ECC Registered memory
modules, so they seems not to be mixed...



As this problem has gone, there is last question (I
hope ;-):
How can I control amount of memory used for disk cache
in recent kernels (2.4.18, 19)?
("Cached:" field in `cat /proc/meminfo`)
I have to be sure that free memory is not used for
caching of disk operations (or how many of it is used
for caching)

Thanks and please CC.



--- "Nakajima, Jun" <[email protected]> wrote:
> To me it looks this MTRR does not cover the memory
> range reported by E820.
>
> > reg05: base=0x7c000000 (1984MB), size= 32MB:
> > write-back, count=1
>
> This covers [0x7c000000 - 0x7e000000).
>
> > BIOS-e820: 0000000000100000 - 000000007f800000
> > (usable)
> But it says memory is available up to 0x7f800000.
> So try mem=2016M ?
> (2048 - 32 = 2016)
>
> I guess you are mixing various memory modules.
>
> Jun


__________________________________________________
Do you Yahoo!?
HotJobs - Search new jobs daily now
http://hotjobs.yahoo.com/

2002-11-06 16:44:47

by Andrew Morton

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

vasya vasyaev wrote:
>
> Hello,
> It seems "mem=2016M" is what we need, box works
> approximately as fast as without enabled HIGHMEM.
> Thank you!
>
> BTW, we are using 4x512 Mb ECC Registered memory
> modules, so they seems not to be mixed...

For some reason your mtrr table was not covering the last 32 megabytes
of memory. Probably you could also have fixed this by altering the
mtrr settings. See Documentation/mtrr.txt in the kernel source tree.

> As this problem has gone, there is last question (I
> hope ;-):
> How can I control amount of memory used for disk cache
> in recent kernels (2.4.18, 19)?
> ("Cached:" field in `cat /proc/meminfo`)
> I have to be sure that free memory is not used for
> caching of disk operations (or how many of it is used
> for caching)

You have to use it for something else :)

Sorry, Linux will only leave a few megabytes of memory unused,
for emergency and interrupt-time allocations.

Why is this a problem?

2002-11-06 17:05:05

by Dave Jones

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

On Wed, Nov 06, 2002 at 08:51:17AM -0800, Andrew Morton wrote:
> For some reason your mtrr table was not covering the last 32 megabytes
> of memory. Probably you could also have fixed this by altering the
> mtrr settings. See Documentation/mtrr.txt in the kernel source tree.

I've seen this happen on systems with onboard graphics cards
that share system RAM as video RAM. The end result is that Linux
sees an amount of mem that isn't a power of two.

In extreme cases, the BIOS has done really mad things like
instead of covering the 1GB with 1 MTRR, it splits it into
7 MTRRs covering 512MB,256MB,128MB,64MB,32MB,16MB,8MB.
Icky.

Dave

--
| Dave Jones. http://www.codemonkey.org.uk

2002-11-19 09:22:11

by vasya vasyaev

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

Hi again,

Let me try to explain what is this all about...

Box has 1 GB of RAM, it's running oracle database.
After some disk activity disk cache has 400 Mb, so 600
Mb is free
Oracle is tuned for using of 800 Mb of RAM for SGA (as
shared memory segment), so Oracle needs 800 Mb of RAM
to be free before it's start, right ?
So when oracle starts it can't allocate this 800 Mb
for SGA and fails to start...
Where is a problem - in kernel which can't reduce disk
cache to allow allocating of shared memory segment or
in oracle ?

BTW, free doesn't show that shared memory is in use
when oracle is started and requested shared memory
segment is allocated (and ipcs shows it).

We need to control disk cache to reduce it as much as
possible because it's not needed for oracle, much
better is to allow oracle to control the RAM
for it's use.

As to compare, on solaris we mount ufs with
"forcedirectio" mount option, which tells not to use
disk cache.



--- Andrew Morton <[email protected]> wrote:
> >
> > How can I control amount of memory used for disk
> cache
> > in recent kernels (2.4.18, 19)?
> > ("Cached:" field in `cat /proc/meminfo`)
> > I have to be sure that free memory is not used for
> > caching of disk operations (or how many of it is
> used
> > for caching)
>
> You have to use it for something else :)
>
> Sorry, Linux will only leave a few megabytes of
> memory unused,
> for emergency and interrupt-time allocations.
>
> Why is this a problem?


__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - Let the expert host your site
http://webhosting.yahoo.com

2002-11-19 09:33:36

by Andrew Morton

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

vasya vasyaev wrote:
>
> Hi again,
>
> Let me try to explain what is this all about...
>
> Box has 1 GB of RAM, it's running oracle database.
> After some disk activity disk cache has 400 Mb, so 600
> Mb is free

And the other 400 megabytes will be freed up on demand.

> Oracle is tuned for using of 800 Mb of RAM for SGA (as
> shared memory segment), so Oracle needs 800 Mb of RAM
> to be free before it's start, right ?

No... If that were so, you'd never be able to start any
applications.

> So when oracle starts it can't allocate this 800 Mb
> for SGA and fails to start...

Well, maybe Oracle is failing to start. But maybe that's
not for the reasons you are assuming.

> Where is a problem - in kernel which can't reduce disk
> cache to allow allocating of shared memory segment or
> in oracle ?

If Oracle requests 800 megabytes from the kernel, it will get it.
It won't be able to mlock it (I'm guessing here).

> BTW, free doesn't show that shared memory is in use
> when oracle is started and requested shared memory
> segment is allocated (and ipcs shows it).

Yup, the "shared" accounting is always zero. Maybe we should
fix that, or remove it.

> We need to control disk cache to reduce it as much as
> possible because it's not needed for oracle, much
> better is to allow oracle to control the RAM
> for it's use.
>
> As to compare, on solaris we mount ufs with
> "forcedirectio" mount option, which tells not to use
> disk cache.

Please ensure that the failure is not some Oracle-specific setup
thing, and then provide specific details on the problem which you
are observing. The assumptions which you are making may not be
correct.

Thanks.

2002-11-19 15:31:26

by Martin J. Bligh

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

> Box has 1 GB of RAM, it's running oracle database.
> After some disk activity disk cache has 400 Mb, so 600
> Mb is free

What makes you assume that the only thing taking up RAM
in the whole box is disk cache? 1GB of ram with 400Mb
of disk cache != 600Mb free. Posting /proc/meminfo is
likely to be more revealing.

M.

2002-11-20 00:57:15

by Gerrit Huizenga

[permalink] [raw]
Subject: Re: Machine's high load when HIGHMEM is enabled

Oracle usually likes its SGA locked in memory, so if it is
set to request 800 MG of RAM on a 1 GB machine, I think you
have tuned Oracle to be requesting too much SGA for your
hardware.

gerrit

In message <[email protected]>, > : Andrew Morton writes:
> vasya vasyaev wrote:
> >
> > Hi again,
> >
> > Let me try to explain what is this all about...
> >
> > Box has 1 GB of RAM, it's running oracle database.
> > After some disk activity disk cache has 400 Mb, so 600
> > Mb is free
>
> And the other 400 megabytes will be freed up on demand.
>
> > Oracle is tuned for using of 800 Mb of RAM for SGA (as
> > shared memory segment), so Oracle needs 800 Mb of RAM
> > to be free before it's start, right ?
>
> No... If that were so, you'd never be able to start any
> applications.
>
> > So when oracle starts it can't allocate this 800 Mb
> > for SGA and fails to start...
>
> Well, maybe Oracle is failing to start. But maybe that's
> not for the reasons you are assuming.
>
> > Where is a problem - in kernel which can't reduce disk
> > cache to allow allocating of shared memory segment or
> > in oracle ?
>
> If Oracle requests 800 megabytes from the kernel, it will get it.
> It won't be able to mlock it (I'm guessing here).
>
> > BTW, free doesn't show that shared memory is in use
> > when oracle is started and requested shared memory
> > segment is allocated (and ipcs shows it).
>
> Yup, the "shared" accounting is always zero. Maybe we should
> fix that, or remove it.
>
> > We need to control disk cache to reduce it as much as
> > possible because it's not needed for oracle, much
> > better is to allow oracle to control the RAM
> > for it's use.
> >
> > As to compare, on solaris we mount ufs with
> > "forcedirectio" mount option, which tells not to use
> > disk cache.
>
> Please ensure that the failure is not some Oracle-specific setup
> thing, and then provide specific details on the problem which you
> are observing. The assumptions which you are making may not be
> correct.
>
> Thanks.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/