Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <be281873-7c28-12b4-7eb5-50d08042549f@intel.com>
Date:   Wed, 31 Aug 2022 08:13:20 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: OpenWrt / MIPS benchmark with MGLRU
Content-Language: en-US
To:     Yu Zhao <yuzhao@google.com>, linux-mm@kvack.org
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Andi Kleen <ak@linux.intel.com>,
        Aneesh Kumar <aneesh.kumar@linux.ibm.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Hillf Danton <hdanton@sina.com>, Jens Axboe <axboe@kernel.dk>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Jonathan Corbet <corbet@lwn.net>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Matthew Wilcox <willy@infradead.org>,
        Mel Gorman <mgorman@suse.de>,
        Michael Larabel <Michael@michaellarabel.com>,
        Michal Hocko <mhocko@kernel.org>,
        Mike Rapoport <rppt@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Tejun Heo <tj@kernel.org>, Vlastimil Babka <vbabka@suse.cz>,
        Will Deacon <will@kernel.org>,
        linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org,
        linux-kernel@vger.kernel.org, x86@kernel.org,
        page-reclaim@google.com
References: <20220815071332.627393-1-yuzhao@google.com>
 <20220831041731.3836322-1-yuzhao@google.com>
From:   Dave Hansen <dave.hansen@intel.com>
In-Reply-To: <20220831041731.3836322-1-yuzhao@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 8/30/22 21:17, Yu Zhao wrote:
> TLDR
> ====
> RAM utilization  Throughput (95% CI)  P99 Latency (95% CI)
> ----------------------------------------------------------
> ~90%             NS                   NS
> ~110%            +[12, 16]%           -[20, 22]%

I'll give you points for thinking out of the box on this one.  This is a
piece of hardware where both latency and bandwidth theoretically matter.
 I've got a slightly older but similar piece of Ubiquiti hardware with
512MB of RAM.  It doesn't run OpenWRT, fwiw.  Maybe my firmware is a bit
outdated.

*But*, most of the heavy lifting for packet flow on these systems is
done in hardware.  They have some hardware acceleration to be able to
_route_ at gigabit speeds, so they're probably not quite as sensitive to
software hiccups as lower-end routers.

That said, my system at least does not typically have *any* memory
pressure.  Right now, it hasn't even filled free memory with page cache
and it's been up for over a month:

# cat /proc/meminfo
MemTotal:         491552 kB
MemFree:          160188 kB
MemAvailable:     373088 kB
Cached:           151004 kB

I think a better tl;dr would be:

	MGLRU doesn't help much or cause any regressions on this
	hardware.  Under (atypical) synthetic memory pressure, MGLRU did
	show some modest but measurable throughput and latency benefits.

In other words, this provides more of a data point that MGLRU doesn't
hurt medium-ish sized embedded systems.  I think you could make an even
stronger case with even smaller hardware or something that actually sees
memory pressure on a regular basis in the wild.