Linux 2.4.23 on x86, 128M memory available
Free outputs this:
total used free shared buffers cached
Mem: 119204 109592 9612 0 60 92136
-/+ buffers/cache: 17396 101808
Swap: 0 0 0
The root filesystem is NFS, no hard drives involved.
When I run a simple program that just mallocs memory and fills it with
random data, the kernel kills the process after it has only allocated 8 or
9 megs. The 92136K of cached memory can't get released for some reason.
What is happening?
Thanks very much!
-Dave
There is some new information that might be useful. The cache memory
lower limit seems to be going up by 1 or 2 megabytes whenever the kernel
kills the XFree86 process:
Jun 23 11:20:16 __alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
Jun 23 11:20:16 VM: killing process XFree86
Could it be when the kernel kills a process for trying to use up too much
memory, the pages used by the process get left in some locked state so can
never be reused?
This is the sort of behaviour we're seeing, it is very reproduceable.
Note this is kernel 2.4.23.
-Dave
David Ashley <[email protected]> writes:
> There is some new information that might be useful. The cache memory
> lower limit seems to be going up by 1 or 2 megabytes whenever the kernel
> kills the XFree86 process:
> Jun 23 11:20:16 __alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> Jun 23 11:20:16 VM: killing process XFree86
>
> Could it be when the kernel kills a process for trying to use up too much
> memory, the pages used by the process get left in some locked state so can
> never be reused?
>
> This is the sort of behaviour we're seeing, it is very reproduceable.
>
>
> Note this is kernel 2.4.23.
Have you tried a kernel that's less than 8 months old? 2.4.26 is current.
-Doug
Doug McNaught wrote:
>Have you tried a kernel that's less than 8 months old? 2.4.26 is current.
Not so easy, we've got some custom modules and the kernel is modified a
little to suit our needs (not related to buffer caches though).
It seems the problem is *not* brought on by the kernel killing XFree86
like I had posted before. Just normal use of the system seems to cause the
Cached value in /proc/meminfo to go up, and it seems it can't go back down
as needed when memory runs low.
What would be helpful is any advice as to where to look in the kernel source
to try and track this down. I followed the path:
fs/proc/proc_misc.c handles /proc/meminfo, the cached value is based on
page_cache_size
mm/swap.c is what changes page_cache_size in delta_nr_cache_pages() function
linux/swap.h has macros dec_nr_cache_pages inc_nr_cache_pages which call this
mm/filemap.c is the only place that calls dec_nr_cache_pages in function
remove_page_from_hash_queue
mm/filemap.c function __remove_inode_page and remove_inode_page call that
mm/filemap.c function invalidate_inode_pages and truncate_complete_page
call those
mm/filemap.c invalide_this_page2 and truncate_list_pages call
truncate_complete_page
invalidate_inode_pages is called all over the place...
So we're snowballing but I don't know what mechanism is supposed to actually
free the cached pages when the system is low on memory. Any advice would
be welcome.
Thanks--
Dave
On Thu, 24 Jun 2004 07:48:20 -0700, David Ashley <[email protected]> wrote:
> So we're snowballing but I don't know what mechanism is supposed to actually
> free the cached pages when the system is low on memory. Any advice would
> be welcome.
>
You may want to examine /proc/meminfo, /proc/slabinfo, and the output
of sysrq-m.
mm/vmscan.c (kswapd) is responsible for freeing most memory. The
routine you are probably most interested in is shrink_cache.
I would check to make sure that the pages in the icache are backed by
a mapping and if so, that they are clean. If either of those two
conditions are not met, then the page cannot be thrown away.
Ross Biro wrote:
>You may want to examine /proc/meminfo, /proc/slabinfo, and the output
>of sysrq-m.
>
>mm/vmscan.c (kswapd) is responsible for freeing most memory. The
>routine you are probably most interested in is shrink_cache.
>
>I would check to make sure that the pages in the icache are backed by
>a mapping and if so, that they are clean. If either of those two
>conditions are not met, then the page cannot be thrown away.
We're locating sysrq-m, I haven't used that before.
/proc/meminfo doesn't give any new insight. /proc/slabinfo the only
thing that jumped out at me was this:
BOX with 16M cached
buffer_head 448 600 96 15 15 1
BOX with 61M cached
buffer_head 1377 4160 96 104 104 1
The inode_cache lines didn't seem to differ much:
inode_cache 838 856 480 107 107 1
vs
inode_cache 313 784 480 97 98 1
The first was the 16M cached and the second was the 61M cached.
In both cases the root filesystem is mounted read-only. So I would think
it can't be a question of dirty pages. In one case the root filesystem
is nfs, and in the other it is a block device with an ext2 filesystem
on it.
Thanks--
Dave
I've done more experiments and have gotten rid of *every* kernel change
we've made, and the problem still exists. To summarize:
1) Stock linux 2.4.26 kernel.
2) VIA epia-m motherboard with linuxbios
3) 128M DDR ram
4) 16M devoted to video
5) XFree86 4.3.0
6) Mozilla + x86 flash 6.0 plugin
7) Flash calls for large unicode Japanese font/
8) No swap. System is diskless and its root filesystem is a 2M ramdisk with
symlinks to a read-only NFS mountpoint.
When I repeatedly load a flash page there are a number of problems
A) Due to flash bug 3M of memory gets lost due to font memory getting lost
This isn't the kernel's problem though.
B) The output of "free" cache number steadily grows. Eventually all of
system memory is locked up in the cache and the kernel starts killing processes
to free up resources. A simple malloc/fill program that keeps allocating
memory and filling it until it fails or the kernel kills it will reclaim some
of the cache, but in the end there is a fixed amount that never can get
released--almost all of the original free memory.
The memory leak in A) is associated with the growth rate in B). If I remove
the Japanese font loading mozilla/flash don't leak memory, and the cache
fillup problem is greatly reduced, but it still exists. It just takes a lot
longer to exhaust all of system memory with the cache.
I don't really understand how the kernel manages its cache memory. If
someone could post a function to say dump information about everything that
is in the cache I could integrate that in with the kernel and report results.
Some people have suggested if the cache doesn't have a mapping or if the
buffers are dirty they can't get reclaimed. The filesystem is read-only so
dirty can't apply. I don't know what the mapping means.
Any suggestions welcome. I don't know what else I can do since I'm down to
a stock kernel and so in my mind there is no way we've introduced this problem
somehow.
Thanks--
Dave
PS Here is my kernel config
#
# Automatically generated by make menuconfig: don't edit
#
CONFIG_X86=y
# CONFIG_SBUS is not set
CONFIG_UID16=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
#
# Loadable module support
#
CONFIG_MODULES=y
# CONFIG_MODVERSIONS is not set
CONFIG_KMOD=y
#
# Processor type and features
#
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MELAN is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
CONFIG_MCYRIXIII=y
# CONFIG_MVIAC3_2 is not set
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_X86_HAS_TSC=y
CONFIG_X86_ALIGNMENT_16=y
CONFIG_X86_USE_3DNOW=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_F00F_WORKS_OK=y
CONFIG_X86_MCE=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
CONFIG_X86_CPUID=y
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_HIGHMEM is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_SMP is not set
# CONFIG_X86_UP_APIC is not set
# CONFIG_X86_UP_IOAPIC is not set
# CONFIG_X86_TSC_DISABLE is not set
CONFIG_X86_TSC=y
#
# General setup
#
CONFIG_NET=y
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
CONFIG_PCI_GODIRECT=y
# CONFIG_PCI_GOANY is not set
CONFIG_PCI_DIRECT=y
CONFIG_ISA=y
CONFIG_PCI_NAMES=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_HOTPLUG is not set
# CONFIG_PCMCIA is not set
# CONFIG_HOTPLUG_PCI is not set
CONFIG_SYSVIPC=y
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
CONFIG_KCORE_ELF=y
# CONFIG_KCORE_AOUT is not set
# CONFIG_BINFMT_AOUT is not set
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=y
CONFIG_OOM_KILLER=y
# CONFIG_PM is not set
# CONFIG_APM is not set
#
# ACPI Support
#
# CONFIG_ACPI is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
# CONFIG_PARPORT is not set
#
# Plug and Play configuration
#
# CONFIG_PNP is not set
# CONFIG_ISAPNP is not set
#
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_DEV_XD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_CISS_SCSI_TAPE is not set
# CONFIG_CISS_MONITOR_THREAD is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_NBD is not set
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=32768
CONFIG_BLK_DEV_INITRD=y
# CONFIG_BLK_STATS is not set
#
# Multi-device support (RAID and LVM)
#
# CONFIG_MD is not set
# CONFIG_BLK_DEV_MD is not set
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not set
# CONFIG_MD_RAID1 is not set
# CONFIG_MD_RAID5 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_BLK_DEV_LVM is not set
#
# Networking options
#
CONFIG_PACKET=y
# CONFIG_PACKET_MMAP is not set
# CONFIG_NETLINK_DEV is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_FILTER=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
# CONFIG_INET_ECN is not set
# CONFIG_SYN_COOKIES is not set
#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=y
CONFIG_IP_NF_FTP=y
# CONFIG_IP_NF_AMANDA is not set
# CONFIG_IP_NF_TFTP is not set
CONFIG_IP_NF_IRC=y
CONFIG_IP_NF_QUEUE=y
CONFIG_IP_NF_IPTABLES=y
# CONFIG_IP_NF_MATCH_LIMIT is not set
CONFIG_IP_NF_MATCH_MAC=y
# CONFIG_IP_NF_MATCH_PKTTYPE is not set
# CONFIG_IP_NF_MATCH_MARK is not set
# CONFIG_IP_NF_MATCH_MULTIPORT is not set
# CONFIG_IP_NF_MATCH_TOS is not set
# CONFIG_IP_NF_MATCH_RECENT is not set
# CONFIG_IP_NF_MATCH_ECN is not set
# CONFIG_IP_NF_MATCH_DSCP is not set
# CONFIG_IP_NF_MATCH_AH_ESP is not set
# CONFIG_IP_NF_MATCH_LENGTH is not set
# CONFIG_IP_NF_MATCH_TTL is not set
# CONFIG_IP_NF_MATCH_TCPMSS is not set
# CONFIG_IP_NF_MATCH_HELPER is not set
CONFIG_IP_NF_MATCH_STATE=y
# CONFIG_IP_NF_MATCH_CONNTRACK is not set
# CONFIG_IP_NF_MATCH_UNCLEAN is not set
# CONFIG_IP_NF_MATCH_OWNER is not set
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
# CONFIG_IP_NF_TARGET_MIRROR is not set
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_IP_NF_TARGET_REDIRECT=y
# CONFIG_IP_NF_NAT_LOCAL is not set
# CONFIG_IP_NF_NAT_SNMP_BASIC is not set
CONFIG_IP_NF_NAT_IRC=y
CONFIG_IP_NF_NAT_FTP=y
# CONFIG_IP_NF_MANGLE is not set
CONFIG_IP_NF_TARGET_LOG=y
# CONFIG_IP_NF_TARGET_ULOG is not set
# CONFIG_IP_NF_TARGET_TCPMSS is not set
# CONFIG_IP_NF_ARPTABLES is not set
#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
# CONFIG_KHTTPD is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
CONFIG_VLAN_8021Q=y
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
#
# Appletalk devices
#
# CONFIG_DEV_APPLETALK is not set
# CONFIG_DECNET is not set
CONFIG_BRIDGE=y
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_LLC is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_FASTROUTE is not set
# CONFIG_NET_HW_FLOWCONTROL is not set
#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
# CONFIG_PHONE_IXJ is not set
# CONFIG_PHONE_IXJ_PCMCIA is not set
#
# ATA/IDE/MFM/RLL support
#
CONFIG_IDE=y
#
# IDE, ATA and ATAPI Block devices
#
CONFIG_BLK_DEV_IDE=y
# CONFIG_BLK_DEV_HD_IDE is not set
# CONFIG_BLK_DEV_HD is not set
CONFIG_BLK_DEV_IDEDISK=y
# CONFIG_IDEDISK_MULTI_MODE is not set
# CONFIG_IDEDISK_STROKE is not set
# CONFIG_BLK_DEV_IDECS is not set
# CONFIG_BLK_DEV_IDECD is not set
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set
# CONFIG_BLK_DEV_CMD640 is not set
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
# CONFIG_BLK_DEV_ISAPNP is not set
CONFIG_BLK_DEV_IDEPCI=y
# CONFIG_BLK_DEV_GENERIC is not set
# CONFIG_IDEPCI_SHARE_IRQ is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_OFFBOARD is not set
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_IDEDMA_PCI_WIP=y
# CONFIG_BLK_DEV_ADMA100 is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_WDC_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_AMD74XX_OVERRIDE is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_HPT34X_AUTODMA is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_PDC202XX_BURST is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_RZ1000 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
CONFIG_BLK_DEV_VIA82CXXX=y
# CONFIG_IDE_CHIPSETS is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_IDEDMA_IVB is not set
# CONFIG_DMA_NONPCI is not set
# CONFIG_BLK_DEV_ATARAID is not set
# CONFIG_BLK_DEV_ATARAID_PDC is not set
# CONFIG_BLK_DEV_ATARAID_HPT is not set
# CONFIG_BLK_DEV_ATARAID_MEDLEY is not set
# CONFIG_BLK_DEV_ATARAID_SII is not set
#
# SCSI support
#
# CONFIG_SCSI is not set
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_BOOT is not set
# CONFIG_FUSION_ISENSE is not set
# CONFIG_FUSION_CTL is not set
# CONFIG_FUSION_LAN is not set
#
# IEEE 1394 (FireWire) support (EXPERIMENTAL)
#
# CONFIG_IEEE1394 is not set
#
# I2O device support
#
# CONFIG_I2O is not set
# CONFIG_I2O_PCI is not set
# CONFIG_I2O_BLOCK is not set
# CONFIG_I2O_LAN is not set
# CONFIG_I2O_SCSI is not set
# CONFIG_I2O_PROC is not set
#
# Network device support
#
CONFIG_NETDEVICES=y
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
CONFIG_DUMMY=y
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_ETHERTAP is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
# CONFIG_SUNLANCE is not set
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNBMAC is not set
# CONFIG_SUNQE is not set
# CONFIG_SUNGEM is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
CONFIG_PCNET32=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_B44 is not set
# CONFIG_CS89x0 is not set
# CONFIG_TULIP is not set
# CONFIG_DE4X5 is not set
# CONFIG_DGRS is not set
# CONFIG_DM9102 is not set
# CONFIG_EEPRO100 is not set
# CONFIG_EEPRO100_PIO is not set
# CONFIG_E100 is not set
# CONFIG_LNE390 is not set
# CONFIG_FEALNX is not set
CONFIG_NATSEMI=y
# CONFIG_NE2K_PCI is not set
# CONFIG_FORCEDETH is not set
# CONFIG_NE3210 is not set
# CONFIG_ES3210 is not set
# CONFIG_8139CP is not set
CONFIG_8139TOO=y
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_SUNDANCE_MMIO is not set
# CONFIG_TLAN is not set
CONFIG_VIA_RHINE=y
# CONFIG_VIA_RHINE_MMIO is not set
# CONFIG_WINBOND_840 is not set
# CONFIG_NET_POCKET is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_MYRI_SBUS is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SK98LIN is not set
# CONFIG_TIGON3 is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
# CONFIG_NET_FC is not set
# CONFIG_RCPCI is not set
# CONFIG_SHAPER is not set
#
# Wan interfaces
#
# CONFIG_WAN is not set
#
# Amateur Radio support
#
# CONFIG_HAMRADIO is not set
#
# IrDA (infrared) support
#
# CONFIG_IRDA is not set
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Input core support
#
CONFIG_INPUT=y
CONFIG_INPUT_KEYBDEV=y
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=y
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_UINPUT is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_SERIAL_CONSOLE=y
# CONFIG_SERIAL_EXTENDED is not set
# CONFIG_SERIAL_NONSTANDARD is not set
CONFIG_UNIX98_PTYS=y
CONFIG_UNIX98_PTY_COUNT=256
#
# I2C support
#
# CONFIG_I2C is not set
#
# Mice
#
# CONFIG_BUSMOUSE is not set
CONFIG_MOUSE=y
CONFIG_PSMOUSE=y
# CONFIG_82C710_MOUSE is not set
# CONFIG_PC110_PAD is not set
# CONFIG_MK712_MOUSE is not set
#
# Joysticks
#
# CONFIG_INPUT_GAMEPORT is not set
# CONFIG_INPUT_NS558 is not set
# CONFIG_INPUT_LIGHTNING is not set
# CONFIG_INPUT_PCIGAME is not set
# CONFIG_INPUT_CS461X is not set
# CONFIG_INPUT_EMU10K1 is not set
# CONFIG_INPUT_SERIO is not set
# CONFIG_INPUT_SERPORT is not set
# CONFIG_INPUT_ANALOG is not set
# CONFIG_INPUT_A3D is not set
# CONFIG_INPUT_ADI is not set
# CONFIG_INPUT_COBRA is not set
# CONFIG_INPUT_GF2K is not set
# CONFIG_INPUT_GRIP is not set
# CONFIG_INPUT_INTERACT is not set
# CONFIG_INPUT_TMDC is not set
# CONFIG_INPUT_SIDEWINDER is not set
# CONFIG_INPUT_IFORCE_USB is not set
# CONFIG_INPUT_IFORCE_232 is not set
# CONFIG_INPUT_WARRIOR is not set
# CONFIG_INPUT_MAGELLAN is not set
# CONFIG_INPUT_SPACEORB is not set
# CONFIG_INPUT_SPACEBALL is not set
# CONFIG_INPUT_STINGER is not set
# CONFIG_INPUT_DB9 is not set
# CONFIG_INPUT_GAMECON is not set
# CONFIG_INPUT_TURBOGRAFX is not set
# CONFIG_QIC02_TAPE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_IPMI_PANIC_EVENT is not set
# CONFIG_IPMI_DEVICE_INTERFACE is not set
# CONFIG_IPMI_KCS is not set
# CONFIG_IPMI_WATCHDOG is not set
#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_SCx200 is not set
# CONFIG_SCx200_GPIO is not set
# CONFIG_AMD_RNG is not set
# CONFIG_INTEL_RNG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_AMD_PM768 is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=m
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_I810 is not set
CONFIG_AGP_VIA=y
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD_K8 is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_ATI is not set
#
# Direct Rendering Manager (XFree86 DRI support)
#
CONFIG_DRM=y
# CONFIG_DRM_OLD is not set
CONFIG_DRM_NEW=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_GAMMA is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_I810 is not set
# CONFIG_DRM_I810_XFREE_41 is not set
# CONFIG_DRM_I830 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_MWAVE is not set
# CONFIG_OBMOUSE is not set
#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
#
# File systems
#
# CONFIG_QUOTA is not set
# CONFIG_QFMT_V2 is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_ADFS_FS is not set
# CONFIG_ADFS_FS_RW is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BEFS_DEBUG is not set
# CONFIG_BFS_FS is not set
# CONFIG_EXT3_FS is not set
# CONFIG_JBD is not set
# CONFIG_JBD_DEBUG is not set
# CONFIG_FAT_FS is not set
# CONFIG_MSDOS_FS is not set
# CONFIG_UMSDOS_FS is not set
# CONFIG_VFAT_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_JFFS_FS is not set
# CONFIG_JFFS2_FS is not set
# CONFIG_CRAMFS is not set
CONFIG_TMPFS=y
CONFIG_RAMFS=y
# CONFIG_ISO9660_FS is not set
# CONFIG_JOLIET is not set
# CONFIG_ZISOFS is not set
# CONFIG_JFS_FS is not set
# CONFIG_JFS_DEBUG is not set
# CONFIG_JFS_STATISTICS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_NTFS_FS is not set
# CONFIG_NTFS_RW is not set
# CONFIG_HPFS_FS is not set
CONFIG_PROC_FS=y
# CONFIG_DEVFS_FS is not set
# CONFIG_DEVFS_MOUNT is not set
# CONFIG_DEVFS_DEBUG is not set
CONFIG_DEVPTS_FS=y
# CONFIG_QNX4FS_FS is not set
# CONFIG_QNX4FS_RW is not set
# CONFIG_ROMFS_FS is not set
CONFIG_EXT2_FS=y
# CONFIG_SYSV_FS is not set
# CONFIG_UDF_FS is not set
# CONFIG_UDF_RW is not set
# CONFIG_UFS_FS is not set
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_XFS_FS is not set
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_RT is not set
# CONFIG_XFS_TRACE is not set
# CONFIG_XFS_DEBUG is not set
#
# Network File Systems
#
# CONFIG_CODA_FS is not set
# CONFIG_INTERMEZZO_FS is not set
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
# CONFIG_NFS_DIRECTIO is not set
# CONFIG_ROOT_NFS is not set
# CONFIG_NFSD is not set
# CONFIG_NFSD_V3 is not set
# CONFIG_NFSD_TCP is not set
CONFIG_SUNRPC=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_SMB_FS=y
# CONFIG_SMB_NLS_DEFAULT is not set
# CONFIG_SMB_UNIX is not set
# CONFIG_NCP_FS is not set
# CONFIG_NCPFS_PACKET_SIGNING is not set
# CONFIG_NCPFS_IOCTL_LOCKING is not set
# CONFIG_NCPFS_STRONG is not set
# CONFIG_NCPFS_NFS_NS is not set
# CONFIG_NCPFS_OS2_NS is not set
# CONFIG_NCPFS_SMALLDOS is not set
# CONFIG_NCPFS_NLS is not set
# CONFIG_NCPFS_EXTRAS is not set
# CONFIG_ZISOFS_FS is not set
#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_SMB_NLS=y
CONFIG_NLS=y
#
# Native Language Support
#
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set
#
# Console drivers
#
# CONFIG_VGA_CONSOLE is not set
CONFIG_VIDEO_SELECT=y
# CONFIG_MDA_CONSOLE is not set
#
# Frame-buffer support
#
CONFIG_FB=y
CONFIG_DUMMY_CONSOLE=y
# CONFIG_FB_RIVA is not set
# CONFIG_FB_CLGEN is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_VESA is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_HGA is not set
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_MATROX is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_IT8181 is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FBCON_ADVANCED is not set
# CONFIG_FBCON_FONTWIDTH8_ONLY is not set
# CONFIG_FBCON_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
#
# Sound
#
CONFIG_SOUND=y
# CONFIG_SOUND_ALI5455 is not set
# CONFIG_SOUND_BT878 is not set
# CONFIG_SOUND_CMPCI is not set
# CONFIG_SOUND_EMU10K1 is not set
# CONFIG_MIDI_EMU10K1 is not set
# CONFIG_SOUND_FUSION is not set
# CONFIG_SOUND_CS4281 is not set
# CONFIG_SOUND_ES1370 is not set
# CONFIG_SOUND_ES1371 is not set
# CONFIG_SOUND_ESSSOLO1 is not set
# CONFIG_SOUND_MAESTRO is not set
# CONFIG_SOUND_MAESTRO3 is not set
# CONFIG_SOUND_FORTE is not set
# CONFIG_SOUND_ICH is not set
# CONFIG_SOUND_RME96XX is not set
# CONFIG_SOUND_SONICVIBES is not set
# CONFIG_SOUND_TRIDENT is not set
# CONFIG_SOUND_MSNDCLAS is not set
# CONFIG_SOUND_MSNDPIN is not set
CONFIG_SOUND_VIA82CXXX=m
# CONFIG_MIDI_VIA82CXXX is not set
# CONFIG_SOUND_OSS is not set
# CONFIG_SOUND_TVMIXER is not set
# CONFIG_SOUND_AD1980 is not set
# CONFIG_SOUND_WM97XX is not set
#
# USB support
#
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_BANDWIDTH is not set
# CONFIG_USB_EHCI_HCD is not set
CONFIG_USB_UHCI=y
# CONFIG_USB_UHCI_ALT is not set
# CONFIG_USB_OHCI is not set
# CONFIG_USB_SL811HS_ALT is not set
# CONFIG_USB_SL811HS is not set
# CONFIG_USB_AUDIO is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_BLUETOOTH is not set
# CONFIG_USB_MIDI is not set
# CONFIG_USB_STORAGE is not set
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_DPCM is not set
# CONFIG_USB_STORAGE_HP8200e is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
CONFIG_USB_HID=y
CONFIG_USB_HIDINPUT=y
# CONFIG_USB_HIDDEV is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_DC2XX is not set
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_SCANNER is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USB_HPUSBSCSI is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_CATC is not set
# CONFIG_USB_CDCETHER is not set
# CONFIG_USB_USBNET is not set
# CONFIG_USB_USS720 is not set
#
# USB Serial Converter support
#
# CONFIG_USB_SERIAL is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_TIGL is not set
# CONFIG_USB_BRLVGER is not set
# CONFIG_USB_LCD is not set
#
# Support for USB gadgets
#
# CONFIG_USB_GADGET is not set
#
# Bluetooth support
#
# CONFIG_BLUEZ is not set
#
# Kernel hacking
#
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_HIGHMEM is not set
# CONFIG_DEBUG_SLAB is not set
# CONFIG_DEBUG_IOVIRT is not set
# CONFIG_MAGIC_SYSRQ is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_LOG_BUF_SHIFT=0
#
# Cryptographic options
#
# CONFIG_CRYPTO is not set
#
# Library routines
#
# CONFIG_CRC32 is not set
# CONFIG_ZLIB_INFLATE is not set
# CONFIG_ZLIB_DEFLATE is not set
More developments:
I've removed the ramdisk + nfs elements for testing. Now the system
has the root filesystem on /dev/hda1 (normal ide hard drive). This makes
no difference, the cache memory still builds up.
I got the SysRq stuff working, here are 3 successive snapshots with
progressively more memory lost to the cache:
SysRq : Show Memory
Mem-info:
Free pages: 73804kB ( 0kB HighMem)
Zone:DMA freepages: 14008kB
Zone:Normal freepages: 59796kB
Zone:HighMem freepages: 0kB
( Active: 3146, inactive: 5217, free: 18451 )
6*4kB 6*8kB 5*16kB 5*32kB 4*64kB 9*128kB 6*256kB 3*512kB 3*1024kB 3*2048kB = 14008kB)
1231*4kB 881*8kB 565*16kB 262*32kB 103*64kB 28*128kB 5*256kB 1*512kB 0*1024kB 9*2048kB = 59796kB)
= 0kB)
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap: 0kB
28672 pages of RAM
0 pages of HIGHMEM
901 reserved pages
4863 pages shared
0 pages swap cached
18 pages in page table cache
Buffer memory: 80kB
Cache memory: 11680kB
SysRq : Show Memory
Mem-info:
Free pages: 67912kB ( 0kB HighMem)
Zone:DMA freepages: 13768kB
Zone:Normal freepages: 54144kB
Zone:HighMem freepages: 0kB
( Active: 4005, inactive: 5837, free: 16978 )
34*4kB 30*8kB 25*16kB 28*32kB 27*64kB 23*128kB 11*256kB 7*512kB 1*1024kB 0*2048kB = 13768kB)
422*4kB 807*8kB 315*16kB 272*32kB 190*64kB 73*128kB 20*256kB 7*512kB 2*1024kB 0*2048kB = 54144kB)
= 0kB)
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap: 0kB
28672 pages of RAM
0 pages of HIGHMEM
901 reserved pages
1324 pages shared
0 pages swap cached
34 pages in page table cache
Buffer memory: 76kB
Cache memory: 17440kB
SysRq : Show Memory
Mem-info:
Free pages: 67700kB ( 0kB HighMem)
Zone:DMA freepages: 14012kB
Zone:Normal freepages: 53688kB
Zone:HighMem freepages: 0kB
( Active: 9181, inactive: 874, free: 16925 )
5*4kB 5*8kB 4*16kB 4*32kB 3*64kB 8*128kB 5*256kB 2*512kB 2*1024kB 4*2048kB = 14012kB)
584*4kB 819*8kB 644*16kB 334*32kB 182*64kB 45*128kB 15*256kB 5*512kB 0*1024kB 0*2048kB = 53688kB)
= 0kB)
Swap cache: add 0, delete 0, find 0/0, race 0+0
Free swap: 0kB
28672 pages of RAM
0 pages of HIGHMEM
901 reserved pages
256 pages shared
0 pages swap cached
48 pages in page table cache
Buffer memory: 44kB
Cache memory: 38396kB
This problem is almost certainly within the linux kernel. It is very
difficult to reproduce since you've got to have
1) XFree86
2) Mozilla browser
3) Macromedia flash plugin
4) A certain flash page, not just any will do
Constantly reloading the flash page results in the cache buildup.
This represents a great denial of service opportunity if someone could
reduce this to the essence of how to make the problem happen.
Recently there was the announcement of the problem where if you get a
floating point exception within a signal handler repeatedly you could cause
the kernel to break. Perhaps this is a similiar situation, maybe if you
free a page within a signal handler, or unmap it, or something? I'm really
grasping here.
Note this is 2.4.x series kernel. I've tried 2.4.23 and 2.4.26. We've got a
modified kernel but going back to stock 2.4.26 with no custom modules has
the same problem. Usually the root filesystem is a 2M ramdisk with symlinks
to a readonly NFS mountpoint, but as stated above I tried replacing this with
a typical /dev/hda1 root filesystem with no change in behaviour.
Someone throw me a bone. I think I've been following up on everyone's
suggestions pretty well. I need ideas.
Thanks--
Dave
First it's absolutely normal for cache to fill up all available
memory. What's not normal is to not free unused cache when memory is
needed.
My best guess is that this isn't a kernel problem, but a bug in flash
and all that memory really is used. The next step I would do is go
through all of the processes and see how much memory they are all
using. For example has flashed been mapping files into memory and not
closing them or freeing the memory?
The first thing I would do is run top on a system that you think is
near dying and sort the processes by memory usage. If you find a
process using lots of memory, that is your culprit. In any case,
attach the output of ps auxw from a system that is out of memory.
Ross Biro wrote:
>First it's absolutely normal for cache to fill up all available
>memory. What's not normal is to not free unused cache when memory is
>needed.
>My best guess is that this isn't a kernel problem, but a bug in flash
>and all that memory really is used. The next step I would do is go
In the thread this is made clear somewhat, but when I post new
emails I don't summarize all that is known about the problem.
I can kill all user processes, including mozilla,
XFree86, and everything else except an sshd to
log into the system, and the cache is still not
released.
The cached memory is just *gone*, no recovery, bye-bye, history.
I am experimenting with strace to see what mozilla is
doing. It does repeatedly
Open file X with O_EXCL and result is -1 EEXIST
Open file Y with O_EXCL and it is created
Close that file
Open file Y without O_EXCL
Allocate some memory
write to Y
close file Y
later unlink file Y
I tried duplicating this sequence of events thinking it
could be something related to the
create/write/close/delete sequence of a file,
perhaps the file's inode isn't getting freed. However my
test program didn't cause the same trouble.
Specifically the discussed strace output includes this:
mkdir("/root/mynfsarea/mozilla/plugtmp", 0777) = -1 EEXIST (File exists)
open("/root/mynfsarea/mozilla/plugtmp/ipg_template.swf", O_WRONLY|O_CREAT|O_TRUNC|O_EXCL|0x8000, 0600) = -1 EEXIST (File exists)
open("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf", O_WRONLY|O_CREAT|O_TRUNC|O_EXCL|0x8000, 0600) = 30
close(30) = 0
open("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf", O_WRONLY|O_CREAT|O_TRUNC|0x8000, 0600) = 30
brk(0x82d8000) = 0x82d8000
gettimeofday({1088521806, 310960}, NULL) = 0
brk(0x82d9000) = 0x82d9000
brk(0x82db000) = 0x82db000
brk(0x82dc000) = 0x82dc000
brk(0x82dd000) = 0x82dd000
brk(0x82de000) = 0x82de000
brk(0x82df000) = 0x82df000
brk(0x82e0000) = 0x82e0000
brk(0x82e1000) = 0x82e1000
brk(0x82e2000) = 0x82e2000
write(30, "FWS\6o\211\0\0x\0\7\320\0\0\27p\0\0\36\n\0C\2\377\377\377"..., 35183) = 35183
brk(0x82ea000) = 0x82ea000
close(30) = 0
... very much later in log
unlink("/root/mynfsarea/mozilla/plugtmp/ipg_template-1.swf") = 0
My thinking here is that every reload of the page steals about 200K
of cache memory that can never be recovered, and that
the total size of these files in the plugtmp directory (mozilla's
cache) is on the order of 200k.
-Dave
On Tue, 29 Jun 2004 09:18:26 -0700, David Ashley <[email protected]> wrote:
>
>
> In the thread this is made clear somewhat, but when I post new
> emails I don't summarize all that is known about the problem.
Sorry, I missed that part of the thread.
It does sounds like you may have a real problem, so the next step I
would do is instrument kswapd to explain why it's not freeing cache
when it's under demand.
The first step is to add something to kernel/sysctrl.c to create a
variable to turn the debugging code on or off.
Then add a bunch of printk's to mm/vmscan.c explaining why every page
is not being freed, but only when the sysctrl variable is set.
Then get the machine into the bad state and turn on the printks.
Run your program that can't allocate memory
Turn off the printks and analyze the logs.
I'll be happy to help you figure out where to put printks and go over
the logs, but you have to make sure you are really getting an OOM kill
with that much cache for the output to be useful.