Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755329AbbKDMos (ORCPT ); Wed, 4 Nov 2015 07:44:48 -0500 Received: from mail-wm0-f44.google.com ([74.125.82.44]:33109 "EHLO mail-wm0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750806AbbKDMor (ORCPT ); Wed, 4 Nov 2015 07:44:47 -0500 MIME-Version: 1.0 In-Reply-To: References: Date: Wed, 4 Nov 2015 20:44:46 +0800 Message-ID: Subject: Re: Improve spinlock performance by moving work to one core From: Ling Ma To: linux-kernel@vger.kernel.org, Ling Content-Type: multipart/mixed; boundary=001a11468aa81087480523b66098 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 8199 Lines: 126 --001a11468aa81087480523b66098 Content-Type: text/plain; charset=UTF-8 Hi All, (send again for linux-kernel@vger.kernel.org) Spinlock caused cache line ping-pong between cores, we have to spend lots of time to get serialized execution. However if we present the serialized work to one core, it will help us save much time. In the attachment we changed code based on queued spinlock The data tell us the spinlock performance can be improved by over 2X on 68cores over 3X on 2 cores (Intel 2699v3 2 sockets, COD off, HT on) In the following time we will try to send out the formal patch Appreciate your comments. Thanks --001a11468aa81087480523b66098 Content-Type: application/x-bzip2; name="adv_spinlock.tar.bz2" Content-Disposition: attachment; filename="adv_spinlock.tar.bz2" Content-Transfer-Encoding: base64 X-Attachment-Id: f_igksonnv1 QlpoOTFBWSZTWf9eRL0ACxr/hP6yAFR/////f+//7/////8AAIAIYBn/eAOdg7lzc7jqAFGarGQ5 2aZ1tDd2AUCsrlqHR0OEGqnZToEpIumDsU6Kq6toUAMEkmhTAhqaaYgk/RT3qJk1NqbU2QaE009E 9QaNAAGj1PU0ASmiBGggFNkTIaptTE2jSNA2kBtT0jQADJoADQOBoGgGgaGgA0MhpoAaaAaAAMhi AA0EmkkgU09JR+mTVN6TSaNPTRNHqbUANANADQGgAAAOBoGgGgaGgA0MhpoAaaAaAAMhiAA0ESRA mgE0CGTTRGoeppPUYTTIaep6RppoaZNA0xDI9Q9T9j9P5fzV5ez45i8MFAmYoPAjMPdNx8ZpQX99 yueM4KHAZ4pUlAhSiTKkrSlZAntHe9UYhS/6EJKw9w3+5h9qIU928Os/O97QRiiLJpCqirIsUIKQ YDJN0rT0cdGDBhBZJFJHZnHiVdEneLTLqIYRDgcbJjBioiqCIrEYRYkYCCgMRioKEfbOoye+aJRY RijopQWVJUiCxUUiggwNFKIiDAYIoM7bKiwRIsgkZ6hCQCfv6Pe5+eeHWRZa1Kqxffa+bXn9HRu6 K1NsjiTl0UkLxoxDQwiakzwmUThyHK4fPEhTHqQyI9yCnqeSt1RZkWmBisXx0b9/9u3tl4hRwD5k its7dfryvjY07cKRpDKpkQ8NmKGN8DEycDKK7LlcnW3qs2uZ6ud2bzI7nf402NlNFbMxX0rg5dAw u4upa2kTLxS5dild12wKeM2IXBHal3UWaAcHLsbXUKhN1vkZDchoVRxkCpMEl12XSau7RHntvHw0 35vfr5287pUYLOKKcvNrWujObpNzznDmdm06Z6OubTrI1difsIneXKTAMnOu06bsK38rHj8dXNa7 bUxey/ro7cQTlWuxhDlkI07h85v1PRCWYutjt7F83UTQs0XvjEJl1mEcZutzumzhqhaTQKFOHGqX lBenwmEb4AsBtE4RcY7CXnre88BfxVXlW1OaOpMgJikUQIchCNRpN84s0+jbeNBv+9re4xXg0eTh OJmvPzOrQUlJUoQM7J2f0u9IqUWDIddyGyhHxcrhXhN6iiY8yVZEmdYJheVx5BvIHQJOw12+z6ch QiIwQGeCkosRkRYkREYIwYeCB5vh+wsGHZuIbd1leflPYOQXiXx2KbyLIO8gw0Sjxr9xPk+3Du/b 3H0db63dpUe4Ghk2cKDIx3eybTwNJ852jSs4VaWYZnArBhyJ8R07zE5jqiCv2lrLUN2iFibl1DqP xZg2D2DERZTX25DHPhFpOwQhd27VeTsgsFFvseYwyYg2Rz+ILaDseZBB7FLMnwmcGvETSqmnBIRy A8hQSoH9rhk6dqR57Kj4uRG2pE5hNMUG53SZOgUq6CPeOeU2S+MmPdXUJ2WRiBN4MsSUfOrKhVPC ktPokiKryCkSHuKCC/cHinw4sAu1AmBa3nr7ktlGiH76HlYyZC9GHmVr5Y4OxxoV7yZN0+f/GghI 3JCSfbnGUma5GWTNH2zaHcV3BUS1RqEnfPDrxm0Ng5FSBidjdWhelsT7eLFhEiWhTTsv4cOqe85P LVsz7kSgPW27+Ns6PuSkcDdiitSERsN+bjyJiCQ457RXFRjkjGGRujBsOQ00KBycdChk99HwpptH LU5nmk3eJjXpkZpiDpvobpMGYb5dfYTO4Lc0UG4YnWh01V4IJkDOz8UhGzY1rVzVg4+DFs63EZip GHR2JhiY5dvPnVq+J9FkUblA3IExdjBQDioW+Ki5KVmpod8kliUhyUiuGpm1VmJqU+XBZMdvrW34 F8fBVPsIoxWpNsrjjCboxoDnNM8HMpXQ5YGp4zJFiykT/2e9Ym0nmxBu3zlK0x0dRmalj6Fbadrt VDct5nOd4Ht/rykmqwjoPQ5uvesb78nupI39FgydiHCJ2GvDE1OmLskOvDIOnW+HZmJnyTJJyMTb 5VMQj3SROUbwOpM6D7xpXl2k8wZeXbKms8aGQy72vr35tnB4GyzgUjM3YONiAQSQH0w3nzHlh92H IiJoDGQip6zaSwyf69tS43AxiYU06cUIhw+eoskeqmoKVlkEONjgLfATjjAyTGDruKhA3XT2n5iF z90x+QI+3S7tZW89uPNq5n7u306T3oFnVEDhYyOppBm02pnXk0k0iRX2G2e0YmvMp1vkQQb6CQXN ZGsqEnPALAqbvVbdLbGZN5pCUTJ2DXqkQ3FNmHEduDIa93bb1uPSTGNinDGI27a/a8byJMJuejzS 1ev0fDzSqtWFcfkjX6G8zNGQ0Oa2OjI3IxntGId5gcixnHc7OazPcqad6VxcIiteBHV29B0E8fR9 eOzAeciCHjHoE/f7zyNYqmV+XQWM9jUz51rIxbnxv6Sfgwv45G6ptJnWQedfe1xjBzAqaxc3xQlX Tw2lSfpPbwJ5eXbphhgisZxv9FSUhHVyIyjAwIbFBqPEvBkxqX5d9Y3nPSMraFfH5+AyPWPreKCs EYjD51hZsWUY2UDAZhZYSsiKMgqJILCdTAs0FsPoBcIGAFKp1FkKODBEkQ956jcUWDx73iY47WIb T1yRTg0x1M8NelES4JeG+WJ2HVnjPM+1W/WPQhKspA6XinHfJngTFA8jjZ1eaTLE01Xh6gMvMDM6 mKmjKWKkxi0FfFfiUh6/Puwk5NuRojOHSbUEOjoJ00DvNhijV3mPf88HsrdUwbGZ9W3PKmqVTF0l L7Ho+XCR3BuYr79530z+khJIV/DPXHlkZExMSTJ7xT55CCT9lGJ/Gr9YdhUCQlAQy+H4n2nk5E6j GsZc/SeaPyGyL/p5hitgZDAMSEL9PX/PJ6QVZ+vipEv7FWGHNAUTCyMVl4at6PlH0NgyT7K7d3Dd SJhDHh+8GvH9OO7OCFkUtndGQpZFTIip/D5d9HpPPGUgoCEF2dlOTKVdytRaVp79fe6H4+CN/2Ut QQFNMtxE2VLUcZxVRugbsj7Nfgmz/CPpKUYj1jJdiBJiRLmOZejPyljdJSwlYecfAfCShZHK8Gy7 paJJJ3c8y1j4YMNfPGBhOBzbJoa/ovBXi5VE6j7Co4iLQlUdf3ht6Dsk6TV3uNzSTmLsEh3gXXvc J1SuSKk/v7MDoVsDn9srwy/CH1Qv+ggRSP1or+j9FK/lYJ5Sz8VAbU/TYTSFKZikmqtjGT8vjKAd JiF5xFQmCKTIEmAv4EvUq4hyujfFsoA+hU/rPv0H/ZxAYP0FBgr7yLwP6gpYjn+BfoQSkYhw8IYo GmPB/wONjEzDYqSw7BZkYkWggL+XohwAzPzC2fzspA95gYb+HUKc887GZ1rpRqg4BEiGr+NeBkm4 580Gg4hwH7v0IPVhEy6HjdWiO4oob/N0XlbvOCnh052R4i0cA3HBAMHALmKPcxDtR9J3J9COuRvO DxCEGKEVdYKZBudkHsgPqTPwtqTUe/tOfVvxJR2ia2FaIE1riOaAPaxEBMxbCuS+X47ImaQQQOFK SNgySJAyIcNoYxPl22hhOoQeId+SD4WpTtDyCziOWBceulTrR40ZoFzt2OYHVHFETqLhYO4rVOoO XI4hRk9yUb/Wdd2RkJdceCDvS28BoQ0so8bCSIn/i7UpMRxDCH0sbuuIuYh3GA/yp1XfyPiHlxMs QCOqpcN5DedHExfXYfYljHgQcwi6r+AfuT78+4w8/mPWdZYnDgp9/wjXnlHDp6bqtg8tWxA3TgwN D74JmeiMDSn2L8PaMEt8BoX6oe2+7gK/GGGOzdt31asUq15oy7D5zQWGN22rlenNTYTYomx3psCU 5SYKMDDkcIeXzUqAMVIKe2h5YMG0aqgck3jowjJClXMPq8nDawJFJH4vDkgR94pVVUD5TL/dO/jb X0pOz80+fJRvXQweoyHb0hUqN8VBmlVj9SU7S+Qj8YfIFUI/DlZzJsA01WROtXwDAYLuFrGTNfH3 cvlj4xvyqJLs24SZIfIOqHE8MmvCiqDz8DFV9mHOG8czrC3Wry7Jx59hNpIYYYvW6KZBpv0DQhaw dAgms6Zb0lVoTBzFRRFBV8FseNhucQ2pk5kKmmxx1KYEYUFi8Q3AVORDh886K4vACmB9pzw3QD8N A2gS2fTZgc9jxOHuMV0D9V0X+wgxYp3mBhCH4YbvU1W+rSpJgsT1sC2dG/iKJO7sQMHeYM+euxA2 8htDoAiBhFip5jCHR/0Jt+RS4g0FA8gNMiPp5XsbmZJMifwFhLdEDjOzMjGQkOTVPUuRsevUsDtQ xyiFFwqFzybmNjtNRdAEo4FQ5HLabd8S34qLjLdpwzC5sU3Egh7jcC6jadtQyQxkIkw440ijUijV J9ZP5EjaUqHxTrSy0hjFNCFy5czOhxRNc+m8xNd55CM02Com0gu4IqkBCOw8IBthqWp3juwwW4lz MhCgqwVBInB9k/GWCkzKb4VaBKAzU91Em9HBJu6InNVNu4bDlTdqPUShvOzQmpTAmhJdsC7QYqTR DETSZZhliaZLS4JEz1v2enN03ENxlhuGTSCI6LvxKaQzTKOhYhrJcDMxuRy5N2AaGZIjJNUnR3yy eGBuGAAWG8+LNA82BofrcYIHp44bw2CUoG2PcdZocnPXMbmMzFDQAnHN2el8HwvbaYMSrPkZC9LH vO8xhYS+YL54Voe3GWCxtPyZ0WuGk9hcDee2hxmWtsC+UNzG81Y3TrKZlXG5iZX0QxLw3XMVCF+O vBCFFnJhCii42KZvKmLY1gGKEEVTqTO6nEm1oZwaEBMCbVYwkZAjCLEgY4KupYwTkYlBcRggmahP AcutHZqZsUcbKgpsQEMzkOBmbKDdxoNNkV3muRt013lbkNkFCyYy+kWJIG5hgE+JDWC2u8MlLM2m N+wLpWO46oIGDMGwZmIDYWycocYkZXNQ3bbJIXUIRG+EXRgpYXcgKJq62CdCpJ2GWYaIEBDmYsoM ymy0kIRGCUQxYrFfvMXogkQ2sXIR5BBTG6oczdSUkKo/posVVHj4Z2dmSoM94TNKcmDigEke3pIO GEQn1kBAAxqDSGBmri5ZrRvnQesIh8Q0iW4sKYXGRX7bFIxC57C+eQSU3JSRjyrNe47uIYQm+YAZ hqWQ2tCU6eRW2e7bifeNj0mXSwCeSkiEPSERDmKG8j4hsDmQDU0E8VPPDsiJbeXYTILlLvAup6EA 3REfUeHUl4hIXo0fKCRUNlE6VIaj7DQF1gRgwU3HjA8dTdhhwMbxJw7CynYHxQM5vkZIMiHIFfsW DmHNgURklHcnZmPFOJB7l4vz9vNG5qODAgHhv9Pw17zLYN9azMXI4Ciq7+ZhzOZ4jeeQkJwZJpIb 2CgG0QFgVkCb1GGIQWBkEabDQATzCUCHYke3d9ft3GZnCEH3DGgwSKCGRIN68phSiSdoWWRVg0qW UVIIIWeeWGtQliKgd3Pyc+h6TxBiOZAPhAN1hCSaMMlDEHtfG71UdDAN693NloqS2P940GZC5A7X Ai/a6fZuLp4gB/2ZC4TVJ3mF1URqjU7SGiULLm1rDTDUNUvaVkDDabRl3nrk0aBEEgyJ5jYbAQO4 wpTR5NAcYnT3zM9SPG7ITsGKHWbC4dn7/fmf3beBMtqdoZlyCGPPx2zebM7BqUMoDHHsYYlMOu5V xUDgasuWZC/UhMsn2hOEm7zdMqX2UftvRHXCQIBVoXlYJwgaJjnm3i5XVDJyOJZC+VDgG0Zkn1XR lYjAOAIQLM3RkUhFFkIvlknL1oJyO/SVqSiV63xmDL6hS7bGxmw8DiDARWLGMgdx0lIKEAQQEBkT Cmu2LOJwZSZOAOkU1DiCYKOGh2IIb4cAOYcSpAowslQbDEEORQyiRhAeaGFHN28jXDmLnggHvDIa Uf1+PK2pAF+aIimqPgJBEgEE9Q7Mg55/XQT+RId73jA2RySxE8uYJgdic4bZiGc2zgpC6ZAljcPe bD5Mk8RNxuQtY5SYwwPiaQEgvsqFkVAF8M4Dls0BjS3bM9wc0skRwdoWDegbPZ5BDsJhU+OS/n0R v9Q2TvIdhcDzITuIBTiHoBB4Ju8zIxdxcpaLHCAytjNivIZgo6iydOnTp06owdjS2E5jYGQ0GA9U nbS3U0gUbEd7ltGJVVKpKKShOvQLWqgpreUuBrniJlBDvJiE2MRMhPse5KIYf/WwvbMvhS0Bpm+o H0+LxKh3PrVmLbBOC3oy7YjUruhnb+P1HYDsANzcsWY23lwNiDVnJ/9aC/Yu8hAjzNAR43dgQBe8 moR/WAtIEYv3sWctd6u3cmjEQUHSGBIJe4/auewQzBK5d45G4JVBnA6F4epdBCy8eEuHJM8ODruN rPu3gXJ/00YPcqGCAwsNh1TkZOI7ggMT7AG9UiiUBuMPDQhFXGJac0UNsKE2MzwszBd99hVMCQg2 JuyqXfjBmNW5QmYCTQIdB0lTUb0aGj/CFjtQkTEwaAz89xisMAz0C4FtGeZgwInRSIkDPIVevGtX fGZSwULQXZN0l7Ss4XuSqYQbhlrLFY1liQcQ3e44XUuBDZAN1auBxNX7LPluSE+GBtBuYC0pgSoQ wL2tKsUXgnUBsFTfsTjbcX8MChS/xFAUe+5ygOxBNkPz1SbwhwPE86COpmAF0BiTD/qDDBJxjduZ MmiDAsR0xJcwU2adXXk8SJGJHAEyA0dpgmW81HpZyBHBMmyv1tDDcl0HE0IbfUai7VuwuU0QhCik soeJ9TYBwPSCPuHbIqGKKcvcWsj6cua0ah0TUuGvBxN5w6A3LRlqrYgVvTRIg9V124dCd59L8rid XIepPkIHUakIBEWAQOLxJtppaD0LIECM9BTTeYdTx69j6CuCWXXD0Gh8LiUTWgC8LMMskTKJCy7A IFHoEKAwIZoRk/VWtrr3FCZs2c8qxR4DqhaiQ9/mNDczOnykAodYSQhxEfRpOXEO2mtfSWTDazmQ 0hktkuIff7TaK9kUAVSIj8Fk/aw65I6gStiVEEtGywoe0BD/4u5IpwoSH+vIl6A= --001a11468aa81087480523b66098-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/