2005-03-25 15:47:26

by Alexander Y. Fomichev

[permalink] [raw]
Subject: 2.6.11: spend too many time in miragtion thread

Yesterday i faced strange behaviour of 2.6.11 on
one of my Dual PIII Gentoo servers.
( 2xPIII 1400, 2G RAM, Mylex 352, 2x Ethernet Pro 100, DL2k Gigabit
Ethernet
CONFIG_NFS_FS=y
CONFIG_NFSD=y
but actually only used as nfs server )

Description:
At some moment i've found out that kernel spend ~2/3
(60-70% actually, accoring to vmstat&top) time in miragtion/1,
and most of other time in idle ( although it have tasks to run
[ps out attached] )
kerneltop shows get_stats and poll_idle on the top permanently:

Linux s3.ir.ru 2.6.11 #1 SMP Sat Mar 5 16:22:27 MSK 2005 i686 GNU/Linux
Sampling_step: 4 | Address range: 0xc0100294 - 0xc02db797
address function ...... 2005-03-24/20:28:44 ...... ticks ( 1916)
c0266530 get_stats 953
c01006c0 poll_idle 674
c0265ba0 rio_interrupt 68
c0202b00 __copy_user_intel 5
c013a010 __mod_page_state 2
c0114a40 sub_preempt_count 1
c0139620 buffered_rmqueue 1
c013da40 kmem_cache_free 1
c013db00 kfree 1
c01446f0 do_anonymous_page 1
c016d2e0 lookup_mnt 1
c0202c60 __copy_to_user_ll 1
c0202cd0 __copy_from_user_ll 1
c028d350 ip_route_output_slow 1
c02921d0 ip_queue_xmit 1
c02a2250 tcp_transmit_skb 1

s3:/var/tmp/kerneltop-0.8#

Sorry, but at that time i've had no idea what get_stats related to
and i have no interfaces status now, only two pieces of "SysRq-t"
( dmesg too long to 32k buffer and i couldn't get more
then ~10k by netconsole, [both attached] )
Another thing, for a long time i observe sun/rpc messages
in the kernel log:
svc: unknown version (0)
svc: unknown version (0)
svc: unknown version (0)
svc: unknown version (0)
svc: unknown version (0)
.......................
but i've no problem with nfsd up to now.


--
Best regards.
Alexander Y. Fomichev <[email protected]>
Public PGP key: http://sysadminday.org.ru/gluk.asc


Attachments:
(No filename) (2.28 kB)
s3-netconsole-20050324.trc (12.27 kB)
s3-dmesg-20050324.trc (15.04 kB)
s3-ps-aux-20050324.out (6.95 kB)
Download all attachments

2005-03-25 16:21:04

by Alexander Y. Fomichev

[permalink] [raw]
Subject: Re: 2.6.11: spend too many time in miragtion thread

On Friday 25 March 2005 18:46, Alexander Y. Fomichev wrote:
> Yesterday i faced strange behaviour of 2.6.11 on
> one of my Dual PIII Gentoo servers.
> ( 2xPIII 1400, 2G RAM, Mylex 352, 2x Ethernet Pro 100, DL2k Gigabit

Here .config.
(sorry, i forgot to send it in previous message.)

CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y

CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_LOCK_KERNEL=y

CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_SYSCTL=y
CONFIG_LOG_BUF_SHIFT=15
CONFIG_HOTPLUG=y
CONFIG_KOBJECT_UEVENT=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_KALLSYMS=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_CC_ALIGN_FUNCTIONS=0
CONFIG_CC_ALIGN_LABELS=0
CONFIG_CC_ALIGN_LOOPS=0
CONFIG_CC_ALIGN_JUMPS=0

CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_OBSOLETE_MODPARM=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y

CONFIG_X86_PC=y
CONFIG_MPENTIUMIII=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_SMP=y
CONFIG_NR_CPUS=32
CONFIG_PREEMPT=y
CONFIG_PREEMPT_BKL=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_TSC=y
CONFIG_X86_MCE=y
CONFIG_MICROCODE=m
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m

CONFIG_EDD=m
CONFIG_HIGHMEM4G=y
CONFIG_HIGHMEM=y
CONFIG_MTRR=y
CONFIG_IRQBALANCE=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_REGPARM=y

CONFIG_PM=y

CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_BLACKLIST_YEAR=0
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y

CONFIG_APM=m

CONFIG_PCI=y
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_NAMES=y

CONFIG_HOTPLUG_PCI=m
CONFIG_HOTPLUG_PCI_ACPI=m

CONFIG_BINFMT_ELF=y

CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y

CONFIG_BLK_DEV_DAC960=y
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_INITRAMFS_SOURCE=""

CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y

CONFIG_NET=y

CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_SYN_COOKIES=y
CONFIG_IP_TCPDIAG=y

CONFIG_NETFILTER=y

CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_CT_ACCT=y
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m

CONFIG_ARCNET=m
CONFIG_ARCNET_1201=m
CONFIG_ARCNET_1051=m
CONFIG_ARCNET_RAW=m
CONFIG_ARCNET_COM90xx=m
CONFIG_ARCNET_COM90xxIO=m
CONFIG_ARCNET_RIM_I=m
CONFIG_ARCNET_COM20020=m
CONFIG_ARCNET_COM20020_PCI=m

CONFIG_NET_ETHERNET=y
CONFIG_MII=y

CONFIG_NET_PCI=y
CONFIG_E100=y
CONFIG_E100_NAPI=y

CONFIG_DL2K=y

CONFIG_NETCONSOLE=y

CONFIG_INPUT=y

CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768

CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_LIBPS2=y

CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y

CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_NONSTANDARD=y

CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=4

CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256

CONFIG_NVRAM=m

CONFIG_I2C=m

CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m

CONFIG_SCx200_ACB=m

CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y

CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y

CONFIG_EXT2_FS=m
CONFIG_REISERFS_FS=y
CONFIG_FS_POSIX_ACL=y

CONFIG_DNOTIFY=y

CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_RAMFS=y

CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_RPCSEC_GSS_KRB5=y

CONFIG_MSDOS_PARTITION=y

CONFIG_NLS=y
CONFIG_NLS_DEFAULT="cp437"
CONFIG_NLS_CODEPAGE_437=y

CONFIG_PROFILING=y
CONFIG_OPROFILE=y

CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_SCHEDSTATS=y
CONFIG_DEBUG_PREEMPT=y
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_EARLY_PRINTK=y
CONFIG_4KSTACKS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y

CONFIG_CRYPTO=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_DES=y

CONFIG_CRC_CCITT=m
CONFIG_CRC32=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_PC=y


--
Best regards.
Alexander Y. Fomichev <[email protected]>
Public PGP key: http://sysadminday.org.ru/gluk.asc