Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e62b0abb-d728-cd22-f58f-a1bd415289c7@gmail.com>
Date:   Wed, 4 Jan 2023 11:10:42 -0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.2
Subject: Re: [PATCH v3 0/9] mm: introduce Designated Movable Blocks
Content-Language: en-US
To:     Mel Gorman <mgorman@suse.de>, Doug Berger <opendmb@gmail.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Jonathan Corbet <corbet@lwn.net>,
        Mike Rapoport <rppt@kernel.org>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Neeraj Upadhyay <quic_neeraju@quicinc.com>,
        Randy Dunlap <rdunlap@infradead.org>,
        Damien Le Moal <damien.lemoal@opensource.wdc.com>,
        Muchun Song <songmuchun@bytedance.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Michal Hocko <mhocko@suse.com>,
        KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        David Hildenbrand <david@redhat.com>,
        Oscar Salvador <osalvador@suse.de>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-mm@kvack.org
References: <20221020215318.4193269-1-opendmb@gmail.com>
 <20221026105500.n6ddzqqf5ozjswsp@suse.de>
 <9842ee9c-5fcc-5458-2779-ad9b88468b48@gmail.com>
 <20221118170510.kexdiqsfaqwledpm@suse.de>
 <342da4ea-d04a-996c-85c4-3065dd4dc01f@gmail.com>
 <20230104153724.mormtuefwaiojvqt@suse.de>
From:   Florian Fainelli <f.fainelli@gmail.com>
In-Reply-To: <20230104153724.mormtuefwaiojvqt@suse.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 1/4/23 07:43, Mel Gorman wrote:

[snip]

>> What is of interest to Broadcom customers is to better distribute user space
>> accesses across each memory controller to improve the bandwidth available to
>> user space dominated work flows. With no ZONE_MOVABLE, the BCM7278 SoC with
>> 1GB of memory on each memory controller will place the 1GB on the low
>> address memory controller in ZONE_DMA and the 1GB on the high address memory
>> controller in ZONE_NORMAL. With this layout movable allocation requests will
>> only fallback to the ZONE_DMA (low memory controller) once the ZONE_NORMAL
>> (high memory controller) is sufficiently depleted of free memory.
>>
>> Adding ZONE_MOVABLE memory above ZONE_NORMAL with the current movablecore
>> behavior does not improve this situation other than forcing more kernel
>> allocations off of the high memory controller. User space allocations are
>> even more likely to be on the high memory controller.
>>
> 
> But it's a weak promise that interleaving will happen. If only a portion
> of ZONE_MOVABLE is used, it might still be all on the same channel. This
> might improve over time if enough memory was used and the system was up
> for long enough.
It is indeed a weak promise for user-space allocations out of 
ZONE_MOVABLE, however the other consumer of the DMB region is a kernel 
driver (typically a video decoder engine) which is directly tied to a 
specific memory controller/DMB region. For the kernel driver using the 
DMB region there is a hard guarantee from the kernel that it gets memory 
from a specific PFN range mapping directly to the desired memory 
controller and thus it is meeting the desired bandwidth 
allocation/deadlines/bursts etc.

We care about both sides of the coin, though we acknowledge that 
"controlling" where user-space allocations are coming from such that 
they be steered towards a specific memory controller is a much harder 
task and so having some amount of non-determinism is acceptable here.
-- 
Florian