From: Michal Nazarewicz <mina86@mina86.com>
To: "Robin H. Johnson" <robbat2@gentoo.org>,
        Michal Hocko <mhocko@kernel.org>
Cc: "Robin H. Johnson" <robbat2@orbis-terrarum.net>,
        linux-kernel@vger.kernel.org, robbat2@gentoo.org, linux-mm@kvack.org
Subject: Re: PROBLEM-PERSISTS: dmesg spam: alloc_contig_range: [XX, YY) PFNs busy
In-Reply-To: <robbat2-20161130T195244-998539995Z@orbis-terrarum.net>
Organization: http://mina86.com/
References: <robbat2-20161129T223723-754929513Z@orbis-terrarum.net> <20161130092239.GD18437@dhcp22.suse.cz> <xa1ty4012k0f.fsf@mina86.com> <20161130132848.GG18432@dhcp22.suse.cz> <robbat2-20161130T195244-998539995Z@orbis-terrarum.net>
User-Agent: Notmuch/0.19+53~g2e63a09 (http://notmuchmail.org) Emacs/26.0.50.2 (x86_64-unknown-linux-gnu)
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWbfGlUPDDHgE57V0jUupKjgIObY0PLrom9mH4dFRK4gmjPs41MxjOgAAACP0lEQVQ4T23Sv2vbQBQHcBk1xE6WyALX107VUEgmn6+ouUwpEQQ6uRjttkWP4CkBg2M0BQLBdPFZYPsyFYo7qEtKDQ7on+t7+nF2Ux8ahD587717OmNYrOvycHsZ+o2r051wHTHysAvGb8ygvgu4QWT0sCmkgZCIEnlV2X8BtyraazFGDuxhmKSQJMlwHQ7v5MHSNxmz78rfElwAa3ieVD9e+hBhjaPDDG6NgFo2f4wBMNIo5YmRtF0RyDgFjJjlMIWbnuM4x9MMfABGTlN4qgIQB4A1DEyA1BHWtfeWNUMwiVJKoqh97KrkOO+qzgluVYLvFCUKAX73nONeBr7BGMdM6Sg0kuep03VywLaIzRiVr+GAzKlpQIsAFnWAG2e6DT5WmWDiudZMIc6hYrMOmeMQK9WX0B+/RfjzL9DI7Y9/Iayn29Ci0r2i4f9gMimMSZLCDMalgQGU5hnUtqAN0OGvEmO1Wnl0C0wWSCEHnuHBqmygxdxA8oWXwbipoc1EoNR9DqOpBpOJrnr0criQab9ZT4LL+wI+K7GBQH30CrhUruilgP9DRTrhVWZCiAyILP+wiuLeCKGTD6r/nc8LOJcAwR6IBTUs+7CASw3QFZ0MdA2PI3zNziH4ZKVhXCRMBjeZ1DWMekKwDCASwExy+NQ86TaykaDAFHO4aP48y4fIcDM5yOG8GcTLbOyp8A8azjJI93JFd1EA6yN8sSxMQJWoABqniRZVykYgRXErzrdqExAoUrRb0xfRp8p2A/4XmfilTtkDZ4cAAAAASUVORK5CYII=
Date: Thu, 01 Dec 2016 02:39:35 +0100
Message-ID: <xa1td1hcwhpk.fsf@mina86.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Sender: linux-kernel-owner@vger.kernel.org
Content-Transfer-Encoding: 8bit
Content-Length: 2581
Lines: 73

On Wed, Nov 30 2016, Robin H. Johnson wrote:
> (I'm going to respond directly to this email with the stack trace.)
>
> On Wed, Nov 30, 2016 at 02:28:49PM +0100, Michal Hocko wrote:
>> > On the other hand, if this didn’t happen and now happens all the time,
>> > this indicates a regression in CMA’s capability to allocate pages so
>> > just rate limiting the output would hide the potential actual issue.
>> 
>> Or there might be just a much larger demand on those large blocks, no?
>> But seriously, dumping those message again and again into the low (see
>> the 2.5_GB_/h to the log is just insane. So there really should be some
>> throttling.
>> 
>> Does the following help you Robin. At least to not get swamped by those
>> message.
> Here's what I whipped up based on that, to ensure that dump_stack got
> rate-limited at the same pass as PFNs-busy. It dropped the dmesg spew to
> ~25MB/hour (and is suppressing ~43 entries/second right now).
>
> commit 6ad4037e18ec2199f8755274d8a745a9904241a1
> Author: Robin H. Johnson <robbat2@gentoo.org>
> Date:   Wed Nov 30 10:32:57 2016 -0800
>
>     mm: ratelimit & trace PFNs busy.
>     
>     Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>

Acked-by: Michal Nazarewicz <mina86@mina86.com>

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 6de9440e3ae2..3c28ec3d18f8 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -7289,8 +7289,15 @@ int alloc_contig_range(unsigned long start, unsigned long end,
>  
>  	/* Make sure the range is really isolated. */
>  	if (test_pages_isolated(outer_start, end, false)) {
> -		pr_info("%s: [%lx, %lx) PFNs busy\n",
> -			__func__, outer_start, end);
> +		static DEFINE_RATELIMIT_STATE(ratelimit_pfn_busy,
> +					DEFAULT_RATELIMIT_INTERVAL,
> +					DEFAULT_RATELIMIT_BURST);
> +		if (__ratelimit(&ratelimit_pfn_busy)) {
> +			pr_info("%s: [%lx, %lx) PFNs busy\n",
> +				__func__, outer_start, end);

I’m thinking out loud here, but maybe it would be useful to include
a count of how many times this message has been suppressed?

> +			dump_stack();

Perhaps do it only if CMA_DEBUG?

+			if (IS_ENABLED(CONFIG_CMA_DEBUG))
+				dump_stack();

> +		}
> +
>  		ret = -EBUSY;
>  		goto done;
>  	}
>
> -- 
> Robin Hugh Johnson
> Gentoo Linux: Dev, Infra Lead, Foundation Trustee & Treasurer
> E-Mail   : robbat2@gentoo.org
> GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
> GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136

-- 
Best regards
ミハウ “????????????????86” ナザレヴイツ
«If at first you don’t succeed, give up skydiving»