Received: by 2002:a17:90a:1609:0:0:0:0 with SMTP id n9csp2283974pja; Thu, 26 Mar 2020 12:37:23 -0700 (PDT) X-Google-Smtp-Source: ADFU+vud8VNogF6BcGxem445UemRkoQ+jpcYJqaawd5fKnRRmarq/GUiPYKzR3MP3rHegmNyBzw8 X-Received: by 2002:aca:efd4:: with SMTP id n203mr1457805oih.159.1585251442828; Thu, 26 Mar 2020 12:37:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585251442; cv=none; d=google.com; s=arc-20160816; b=M3+fRVwjcV4pUZEDCrUFbXlNlbxbtS1U+/W1RVnQjrkNA08BJp72mH+F12SgQOxorL dJigcAGrHYUYLszyF90D/WCZN/bjIrwG2d25mQukN7+4UtwX6KlyMFNN6UyXQTena5K3 QPgotidQCvtkY25w2QDkfWLEoq813rQjgyemK691nE6i0OpLnB1DuDZ162mp/PIqSI69 SldKF3rZrJRLmygILrQGYfKGvd3osb3ZEx1+amknicjKrz3MaFL8NS1Up7FM2I0rBM+q sdZb9yNwb1vrvFLJsKjD94hlMc4CiFveAapLBo0MbbXoYaas/ohvYagBG1l1QB6XHc+8 HtXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=WugekyqNe14NZjUCrzuU8f/PMfy4ymjEA2uErv+hnjQ=; b=DnBCsCrwyeDxJt6PcYEXMRE6FHXpdD2zLVBuyMjAdwK6l4upZk9vTqQfv5JyoJCyNN +IQtATyfHT4+KqVXTre8QOZowoW/pldfhBxnf3xnkqWZabGcvF8BWQwo/ksvrmvUh4aS 8EbPib1X/PO7mqiI+wjPEBUvcxjnAQViDnUoVoD8iSca+f2qfzvkJuCdH07ad0NoaUoP eLlTm+3sXyZ3yqOnQ5wDb8bE7K91fkvUOtGkj78vYIFLAqCILgz6T96E4BOrsYXZO9bi G9o1kRP138TQRXeaXCMX9B9SqYs5tLA13CVmd348ID9lO+UOH3dP9ZMdQNPS3HEpzghe mUYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=dh6db2Px; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a13si1708780otk.158.2020.03.26.12.37.08; Thu, 26 Mar 2020 12:37:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@soleen.com header.s=google header.b=dh6db2Px; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727719AbgCZTgo (ORCPT + 99 others); Thu, 26 Mar 2020 15:36:44 -0400 Received: from mail-ed1-f66.google.com ([209.85.208.66]:43406 "EHLO mail-ed1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726067AbgCZTgo (ORCPT ); Thu, 26 Mar 2020 15:36:44 -0400 Received: by mail-ed1-f66.google.com with SMTP id bd14so8282572edb.10 for ; Thu, 26 Mar 2020 12:36:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WugekyqNe14NZjUCrzuU8f/PMfy4ymjEA2uErv+hnjQ=; b=dh6db2Pxk89VoPBRilHdEG0cV3Zu1adly6w6kDu3jnNIDxVlP7JBOlI2vZtqoSxrWM znN99ZukPEtiO3LavWJFP5reZ248Ntxhou87Nm71MmHDaa2BDM4cW28SEX14TyIBGS6e DKCHkENogCptCU+mAJAks/5NR+OcetEihnM2U2RVrBEhj5RopDG+xZo8aeuWz+Ahkr2+ x5/O0UWynQXC3rlf2kBgIc6qCclcRAFST0BXeiAPfzZZ9x+u2FLsMZSJvtsyC75tMMwW Xl9XSzIHgN23oCwJyEAdpmJvU9qv7p1Bxr4k3BRD931qfvinqUhRE+q/lOWoJf6xbCyI 7ErQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WugekyqNe14NZjUCrzuU8f/PMfy4ymjEA2uErv+hnjQ=; b=Bm1k5Psk++AZI3xo5GRE2dNTl3n5rmw2UOSvQhbp+Oj5yTvFcZq/ZBa9g34PYN1f79 v9G56H+K2ut2uwVvHEQSYCgaXO/doRWgz3zlPN1GAyjQ2KX36+qqCpgoXZVQDpCusrLD tuQm+RdB3Mt4aZiYpK2+zLT+AN8DJXFcQmgfG3y0/AySt+/eH20tToF2XxboYHh/zvjV R+WDeKQMh/DQob1pZZtUW+FfOxVlsbYOhiEzt6i+1cCE3sLIdyHtbJ9d4uZSacz6ivTe rWcoP81H4sRoTASXix2qJ5FLbnLLc4s2ylGrh6RcVIoU812OXg6nb+inSHK5ulu+2QPj Ra4A== X-Gm-Message-State: ANhLgQ03i3mWpdmKkOgGgeJrZZAHQ63z1nvNb9q1drJnEebjoMruuU9q SCQQn9TcmiLbNoGfgDLypnuNsGAcW0qq8lOsu8bzz29KMWw= X-Received: by 2002:a17:906:65ca:: with SMTP id z10mr9030915ejn.368.1585251400312; Thu, 26 Mar 2020 12:36:40 -0700 (PDT) MIME-Version: 1.0 References: <20200311123848.118638-1-shile.zhang@linux.alibaba.com> <20200319190512.cwnvgvv3upzcchkm@ca-dmjordan1.us.oracle.com> <20200326185822.6n56yl2llvdranur@ca-dmjordan1.us.oracle.com> In-Reply-To: <20200326185822.6n56yl2llvdranur@ca-dmjordan1.us.oracle.com> From: Pavel Tatashin Date: Thu, 26 Mar 2020 15:36:29 -0400 Message-ID: Subject: Re: [PATCH v3] mm: fix tick timer stall during deferred page init To: Daniel Jordan Cc: Shile Zhang , Andrew Morton , Kirill Tkhai , linux-mm , LKML Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I agree with Daniel, we should look into approach where pgdat_resize_lock is taken only for the duration of updating tracking values such as pgdat->first_deferred_pfn (perhaps we would need to add another tracker that would show chunks that are currently being worked on). The vast duration of struct page initialization process should happen outside of this lock, and only be taken when we update globally seen data structures: lists, tracking variables. This way we can solve several problems: 1. allow interrupt threads to grow zones if required. 2. keep jiffies happy. 3. allow future scaling when we will add inner node threads to initialize struct pages (i.e. ktasks from Daniel). Pasha On Thu, Mar 26, 2020 at 2:58 PM Daniel Jordan wrote: > > On Thu, Mar 19, 2020 at 03:05:12PM -0400, Daniel Jordan wrote: > > Regardless, > > Reviewed-by: Daniel Jordan > > Darn, I spoke too soon. > > On a two-socket Xeon, smaller values of TICK_PAGE_COUNT caused the deferred > init timestamp to grow by over 25%. This was with pgdatinit0 bound to the > timer interrupt CPU to make sure the issue always reproduces. > > TICK_PAGE_COUNT node 0 deferred > init time (ms) > --------------- --------------- > 4096 610 > 8192 587 > 16384 487 > 32768 480 // used in the patch > > Instead of trying to find a constant that lets the timer interrupt run often > enough, I think a better way forward is to reconsider how we handle the resize > lock. I plan to prototype something and reply back with what I get.