Received: by 2002:ac0:a5a6:0:0:0:0:0 with SMTP id m35-v6csp362219imm; Fri, 21 Sep 2018 01:17:00 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZaqJXF/U27DXOvjnDlK96vGaOJ3TuNREvzDuf66ktWIJpaN0mAtd7Qb19M1/dhz/ACVIqZ X-Received: by 2002:a17:902:2ac3:: with SMTP id j61-v6mr42857881plb.172.1537517820130; Fri, 21 Sep 2018 01:17:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537517820; cv=none; d=google.com; s=arc-20160816; b=cQKrcKZNnron1INMwpwHtuFIEgwrhf2PGiK423cxo0XTpLMQOJD1QtOSZOjVTEkchs rvQ6ku12wmtNMRhMXPW5W2n7XsRn1HtordGyr28PK+EUARZmmJ3CaWl6f4NufXK6xW7/ 6m69yuwBdVdvUef3ojKZZ0ElnQDUiRAIM/qq9/WQ2mZiZc8sZ+Yi9EMGPfpYjVeu772s l1fRoiaSEbfOKXKNsr5ufPcHf7Cg6hmVY8CrqnE0d8xtEDao3J4D9PR8PDYDLEcqiKlh kQDgNVTJgfYJiWOrqwmjQy46WUJny8d8hSrx+GzD8cs50l5J+3lhCiAS+2VtFhlgBn6Z b+GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:user-agent:in-reply-to :content-disposition:mime-version:references:subject:cc:to:from:date; bh=J2d9Tyb1y3YM8XpAVRQ6H/yQobLgqaARmS9PRltELwQ=; b=xaGcW8xrKonY19/nmlVACPg2EpaNIJYRM8syVHc9oNnpaE7GyevsoM2g1VJgLNo3Oc aUOCiXcLjEX5d0BNAw2iJwe238x30B4u4R/rXsYWdIvix0q1DZZ1FJyBJgBMRcoo5+Gw K7hdNzq6lTKCGmHQBrh/aKiYdAQEhMBQVqQD/XwbQZQDv3PkNOMXlQiHNeWtLEzP0W/f tfPK0kye4OL/DAE5wrVIeQgJOvObpAf8KZMG8LrR9sLUEj85A4a/4rVHyjkKmJWFITNa RH34HnT6+lnSiF59u3eaix4AJsxwkrQpC3Hdr9R7F4mNEEqwwUAldGqPHfdCskqY9p65 k/cw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 1-v6si25817034plz.220.2018.09.21.01.16.43; Fri, 21 Sep 2018 01:17:00 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389283AbeIUOEW (ORCPT + 99 others); Fri, 21 Sep 2018 10:04:22 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:39136 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388909AbeIUOEV (ORCPT ); Fri, 21 Sep 2018 10:04:21 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w8L8DlJ4060729 for ; Fri, 21 Sep 2018 04:16:36 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0a-001b2d01.pphosted.com with ESMTP id 2mmvrt8rxa-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 21 Sep 2018 04:16:35 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 21 Sep 2018 09:16:33 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 21 Sep 2018 09:16:30 +0100 Received: from d06av24.portsmouth.uk.ibm.com (d06av24.portsmouth.uk.ibm.com [9.149.105.60]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w8L8GT9v57081934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 21 Sep 2018 08:16:29 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3F8FD42041; Fri, 21 Sep 2018 11:16:16 +0100 (BST) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9D4A242042; Fri, 21 Sep 2018 11:16:15 +0100 (BST) Received: from rapoport-lnx (unknown [9.148.207.248]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Fri, 21 Sep 2018 11:16:15 +0100 (BST) Date: Fri, 21 Sep 2018 11:16:27 +0300 From: Mike Rapoport To: Pintu Kumar Cc: open list , Russell King - ARM Linux , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org Subject: Re: KSM not working in 4.9 Kernel References: <20180916153237.GC15699@rapoport-lnx> <20180917043724.GA12866@rapoport-lnx> <20180917145936.GA20945@rapoport-lnx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-TM-AS-GCONF: 00 x-cbid: 18092108-4275-0000-0000-000002BDA45F X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18092108-4276-0000-0000-000037C7A916 Message-Id: <20180921081626.GA12492@rapoport-lnx> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-09-21_03:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809210087 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 20, 2018 at 05:21:20PM +0530, Pintu Kumar wrote: > Hi, > > Thank you so much for all your reply so far. > I have few more doubts to understand the output from ksm sysfs. > Device: Hikey620 - ARM64 - Linux 4.9.20 > With HUGE page enabled: > CONFIG_TRANSPARENT_HUGEPAGE=y > CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y > # CONFIG_TRANSPARENT_HUGEPAGE_MADVISE is not set > > Currently, I get this output, when I run below program with ksm: > > ~ # grep -H '' /sys/kernel/mm/ksm/* > /sys/kernel/mm/ksm/full_scans:29 > /sys/kernel/mm/ksm/page_comparisons:39584 > /sys/kernel/mm/ksm/pages_hashed:11672 > /sys/kernel/mm/ksm/pages_scanned:21766 > /sys/kernel/mm/ksm/pages_shared:3 > /sys/kernel/mm/ksm/pages_sharing:10097 > /sys/kernel/mm/ksm/pages_to_scan:200 > /sys/kernel/mm/ksm/pages_unshared:53 > /sys/kernel/mm/ksm/pages_volatile:1 > /sys/kernel/mm/ksm/run:0 > /sys/kernel/mm/ksm/sleep_millisecs:1000 > --------------------------- > > int main(int argc, char *argv[]) > { > int i, n, size, ret; > char *buffer; > void *addr; > > n = 100; > size = 100 * getpagesize(); > for (i = 0; i < n; i++) { > buffer = (char *)malloc(size); > memset(buffer, 0xff, size); > madvise(buffer, size, MADV_MERGEABLE); > if (ret < 0) { > fprintf(stderr, "malloc madvise failed: ret: > %d, reason: %s\n", ret, strerror(errno)); > } > usleep(500); > } > printf("Done....press ^C\n"); > pause(); > return 0; > } > Note: madvise() system call is not failing here, as mentioned earlier. > I guess the page is aligned with getpagesize(). > Then I do this to invoke ksm: > # echo 1 > /sys/kernel/mm/ksm/run > # ./malloc-test.out & > # sleep 5 > # echo 0 > /sys/kernel/mm/ksm/run > # > > Also, the anon pages in the system shows like this: > BEFORE: > ------------- > ~ # cat /proc/meminfo | grep -i anon > Active(anon): 40740 kB > Inactive(anon): 0 kB > AnonPages: 40760 kB > AnonHugePages: 0 kB > > AFTER MERGING: > -------------------------- > ~ # cat /proc/meminfo | grep -i anon > Active(anon): 440 kB > Inactive(anon): 0 kB > AnonPages: 188 kB > AnonHugePages: 0 kB > > I want to understand the KSM output w.r.t to the above program, and > cross-check if the output is correct. > Can someone help me to understand it? > > As of now, what I understood is that: > - I am allocating around 400KB of memory 100 times. That is: 100 * 100 > * 4K = 10000 pages (which are all with similar content). > - Output says: 10097 page_sharing happened. > - Pages currently shared is: 3 > - So total pages are: 10097 + 3 = 10100 > > I could not understand from where the additional 100 pages came from? > Also, why some pages are shown as: pages_unshared ? > What can I interpret from this? > And, what does it mean by: pages_volatile:1 ? > > Basically, I wanted to understand, is there any problem with the above > output, or it is fine. > If it is fine, how to prove it? The ksm sysfs attributes are described at Documentation/admin-guide/mm/ksm.rst or online at [1]. The numbers look sane in general. The additional pages may come from malloc metadata that is created by libc when you allocate memory. I'd recommend to use mmap() or posix_memalign() with page size alignment to get exact amount of page aligned memory. [1] https://www.kernel.org/doc/html/latest/admin-guide/mm/ksm.html > Thanks, > Pintu > > On Mon, Sep 17, 2018 at 8:29 PM Mike Rapoport wrote: > > > > On Mon, Sep 17, 2018 at 05:25:27PM +0530, Pintu Kumar wrote: > > > On Mon, Sep 17, 2018 at 11:46 AM Pintu Kumar wrote: > > > > > > But still no effect. > > > > > > And I checked LTP test cases. It almost doing the same thing. > > > > > > > > > > > > I observed that [ksmd] thread is not waking up at all. > > > > > > I gave some print inside it, but I could never saw that prints coming. > > > > > > I could not find it running either in top command during the operation. > > > > > > Is there anything needs to be done, to wakw up ksmd? > > > > > > I already set: echo 1 > /sys/kernel/mm/ksm. > > > > > > > > > > It should be echo 1 > /sys/kernel/mm/ksm/run > > > > > > > > > > > > > Oh yes, sorry for the typo. > > > > I tried the same, but still ksm is not getting invoked. > > > > Could someone confirm if KSM was working in 4.9 kernel? > > > > > > > > > > Ok, it's working now. I have to explicitly stop the ksm thread to see > > > the statistics. > > > Also there was some internal patch that was setting vm_flags to > > > VM_MERGABLE thus causing ksm_advise call to return. > > > > > > # echo 1 > /sys/kernel/mm/ksm/run > > > # ./malloc-test.out & > > > # echo 0 > /sys/kernel/mm/ksm/run > > > > > > ~ # grep -H '' /sys/kernel/mm/ksm/* > > > /sys/kernel/mm/ksm/full_scans:105 > > > /sys/kernel/mm/ksm/pages_shared:1 > > > /sys/kernel/mm/ksm/pages_sharing:999 > > > /sys/kernel/mm/ksm/pages_to_scan:100 > > > /sys/kernel/mm/ksm/pages_unshared:0 > > > /sys/kernel/mm/ksm/pages_volatile:0 > > > /sys/kernel/mm/ksm/run:0 > > > /sys/kernel/mm/ksm/sleep_millisecs:20 > > > > > > > > > However, I have one doubt. > > > Is the above data correct, for the below program? > > > > You have 1 shared page and 999 additional references to that page > > > > > int main(int argc, char *argv[]) > > > { > > > int i, n, size, ret; > > > char *buffer; > > > void *addr; > > > > > > n = 10; > > > size = 100 * getpagesize(); > > > for (i = 0; i < n; i++) { > > > buffer = (char *)malloc(size); > > > memset(buffer, 0xff, size); > > > madvise(buffer, size, MADV_MERGEABLE);o > > > > This madvise() call should fail because buffer won't be page aligned > > > > > addr = mmap(NULL, size, > > > PROT_READ | PROT_EXEC | PROT_WRITE, > > > MAP_PRIVATE | MAP_ANONYMOUS, > > > -1, 0); > > > memset(addr, 0xff, size); > > > ret = madvise(addr, size, MADV_MERGEABLE); > > > if (ret < 0) { > > > fprintf(stderr, "madvise failed: ret: %d, > > > reason: %s\n", ret, strerror(errno)); > > > } > > > usleep(500); > > > } > > > printf("Done....press ^C\n"); > > > > > > pause(); > > > > > > return 0; > > > } > > > > > > > > > Thanks, > > > Pintu > > > > > > > -- > > Sincerely yours, > > Mike. > > > -- Sincerely yours, Mike.