Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751843AbeAQApk (ORCPT + 1 other); Tue, 16 Jan 2018 19:45:40 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:57466 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750750AbeAQApj (ORCPT ); Tue, 16 Jan 2018 19:45:39 -0500 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: [PATCH] mm: numa: Do not trap faults on shared data section pages. From: Henry Willard In-Reply-To: <20180116212614.gudglzw7kwzd3get@suse.de> Date: Tue, 16 Jan 2018 16:45:22 -0800 Cc: akpm@linux-foundation.org, kstewart@linuxfoundation.org, zi.yan@cs.rutgers.edu, pombredanne@nexb.com, aarcange@redhat.com, gregkh@linuxfoundation.org, aneesh.kumar@linux.vnet.ibm.com, kirill.shutemov@linux.intel.com, jglisse@redhat.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8BIT Message-Id: References: <1516130924-3545-1-git-send-email-henry.willard@oracle.com> <1516130924-3545-2-git-send-email-henry.willard@oracle.com> <20180116212614.gudglzw7kwzd3get@suse.de> To: Mel Gorman X-Mailer: Apple Mail (2.3445.5.20) X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8776 signatures=668653 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=733 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1801170008 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Return-Path: > On Jan 16, 2018, at 1:26 PM, Mel Gorman wrote: > > On Tue, Jan 16, 2018 at 11:28:44AM -0800, Henry Willard wrote: >> Workloads consisting of a large number processes running the same program >> with a large shared data section may suffer from excessive numa balancing >> page migration of the pages in the shared data section. This shows up as >> high I/O wait time and degraded performance on machines with higher socket >> or node counts. >> >> This patch skips shared copy-on-write pages in change_pte_range() for the >> numa balancing case. >> >> Signed-off-by: Henry Willard >> Reviewed-by: Håkon Bugge >> Reviewed-by: Steve Sistare steven.sistare@oracle.com > > Merge the leader and this mail together. It would have been nice to see > data on other realistic workloads as well. > > My main source of discomfort is the fact that this is permanent as two > processes perfectly isolated but with a suitably shared COW mapping > will never migrate the data. A potential improvement to get the reported > bandwidth up in the test program would be to skip the rest of the VMA if > page_mapcount != 1 in a COW mapping as it would be reasonable to assume > the remaining pages in the VMA are also affected and the scan is wasteful. > There are counter-examples to this but I suspect that the full VMA being > shared is the common case. Whether you do that or not; > > Acked-by: Mel Gorman Thanks. The real customer cases where this was observed involved large, 1TB or more, eight socket machines running very active RDBMS workloads. These customers saw high iowait times and a loss in performance when numa balancing was enabled. Previously there was no reported iowait time. The extent of the loss of performance was variable depending on the activity and never quantified. The little test program is a distillation of what was observed. In the real workload, a large part of the VMA is shared, but not all of it, so this seemed the simplest and most reliable patch. Henry > > -- > Mel Gorman > SUSE Labs