Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 1/2] mm/pageblock: mitigation cmpxchg false sharing in
 pageblock flags
To:     Wei Yang <richard.weiyang@linux.alibaba.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>,
        Alexander Duyck <alexander.h.duyck@linux.intel.com>,
        linux-mm@kvack.org, linux-kernel@vger.kernel.org
References: <1597549677-7480-1-git-send-email-alex.shi@linux.alibaba.com>
 <20200817095821.GA29756@L-31X9LVDL-1304.local>
From:   Alex Shi <alex.shi@linux.alibaba.com>
Message-ID: <301b82e3-4ee1-8b90-a6a6-256fd8885e59@linux.alibaba.com>
Date:   Mon, 17 Aug 2020 19:02:51 +0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0)
 Gecko/20100101 Thunderbird/68.7.0
MIME-Version: 1.0
In-Reply-To: <20200817095821.GA29756@L-31X9LVDL-1304.local>
Content-Type: text/plain; charset=gbk
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


?? 2020/8/17 ????5:58, Wei Yang д??:
> On Sun, Aug 16, 2020 at 11:47:56AM +0800, Alex Shi wrote:
>> pageblock_flags is used as long, since every pageblock_flags is just 4
>> bits, 'long' size will include 8(32bit machine) or 16 pageblocks' flags,
>> that flag setting has to sync in cmpxchg with 7 or 15 other pageblock
>> flags. It would cause long waiting for sync.
>>
>> If we could change the pageblock_flags variable as char, we could use
>> char size cmpxchg, which just sync up with 2 pageblock flags. it could
>> relief much false sharing in cmpxchg.
>>
> 
> If my understanding is correct, CPU reads data in the size of cacheline.
> Define a variable a char or other, doesn't help on false sharing. 
> 
> Correct me, if not.

Right and not,

Cacheline false sharing is right. but after that, cmpxchg still need to compare
the pointed data, if the data is long, it need to compare long word and make sure
nothing changes in the long word. If we narrow the comparsion data to byte, cmpxchg
will just sync up on a byte. So it looks like there are 2 level false sharing here.

Thanks
Alex

        for (;;) {
-               old_word = cmpxchg(&bitmap[word_bitidx], word, (word & ~mask) | flags);
-               if (word == old_word)
+               old_byte = cmpxchg(&bitmap[byte_bitidx], byte, (byte & ~mask) | flags);
+               if (byte == old_byte)
                        break;
-               word = old_word;
+               byte = old_byte;
        }