Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 12 May 2023 15:09:26 +0800
From:   Philip Li <philip.li@intel.com>
To:     Masahiro Yamada <masahiroy@kernel.org>
CC:     kernel test robot <lkp@intel.com>,
        Linux Kbuild mailing list <linux-kbuild@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [RFC] [kbuild test robot] random-order parallel building
Message-ID: <ZF3mJlMuH2HbNxWp@rli9-mobl>
References: <CAK7LNATrFbr7kT9HEFgYO6ZstaSx1FB+Q4SQrAX+D8VwG5K0dQ@mail.gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CAK7LNATrFbr7kT9HEFgYO6ZstaSx1FB+Q4SQrAX+D8VwG5K0dQ@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?M+qZddtSCHxDTg03bz/zdUImsc6MIibk1II+Z/T+dJJIRMdj3Ra+tseh4gS5?=
 =?us-ascii?Q?ir7Ulthz2Qt0SpGLPCdfgi2sQwrthWyKtz1JegDvYC68TRqwNQhPIt6rJnih?=
 =?us-ascii?Q?AIkehbko23LZIOpoVtoUkkf4REuBE1q7QPsPh+Ix7MOLLmUv5y3GCbOnl2rc?=
 =?us-ascii?Q?59R1VkYr8KJNdXK1Ul4uPD8OqDOjuHxifdxJgW1gpZBHhFD6ea4iiriOs58d?=
 =?us-ascii?Q?3ufR4bjzmDmZeMo0aSux4qCu4cObGqUlmRUOrWXfg+euhqmmdb5Xce4aNqg/?=
 =?us-ascii?Q?qijw0i9SVb+uL1gGNjwh7petwm0IluHks9690ptDReJ1quYp4wx3oE+I6lHH?=
 =?us-ascii?Q?dSp1TQlnsMdqvKMKXhGWR4z/qTi25K7dSHVIaoB4gB6QsujpWtX9MoA5L0ba?=
 =?us-ascii?Q?EZrbi48b3pbg+ZVd3hYWXFJ9E9wm9dO1aLYTw8rfBoPBm9zACIgU7ZIxxpnM?=
 =?us-ascii?Q?3xNVOe7/KSSQ1wgtwN2uJGhCQEI2owVHLD99VJBHAdXu+kypIgB9rCHFC1ii?=
 =?us-ascii?Q?G9ftf1FLlmsaC4UMufP9OgNSEz8EMHywAhYBqTEBIaQcZkgSRYg2KzFcMJuC?=
 =?us-ascii?Q?3Ly61aSzbH6V5e7yDgY4MLM4+yopPTrZfTnMV2Xtc/zpBfZW4d/Z6KEbrki3?=
 =?us-ascii?Q?27KGBW88XpwcO0oF34sjTrfaHq0Egjak+jNu0kOQIiaXE+IckGll4ghpicac?=
 =?us-ascii?Q?hgUzQyTzwAYDX1ddKHE42K62Thsi5MxHZBfThrCeHWpIr2iKl7paQpe0rrjC?=
 =?us-ascii?Q?AxKHAcYw6gNuRdDDWqQfDYQos04tevMkkq/4V+oET6VJanF7gRTRQZ7Hiwrt?=
 =?us-ascii?Q?w2ajZWkzI9W2No6wnB4Ykm3gynb3i+6GMKyc/EGGB2eRKy/yZf5tzyUDwiwH?=
 =?us-ascii?Q?6uULX08JlbEJvAV3ehWss7y3Z8c4FHFVAQRPA0mU9fVRJRD5BxN4eLdXICpP?=
 =?us-ascii?Q?HIn33v6/BReClRgDueIQlx/acriDoYk6ja3E4fpQF5whxuqByt07Xh3UNNAW?=
 =?us-ascii?Q?wjJ/+C92+SLawIT7S1wrlsUnzawAFV9KGrOm6Y6cLa2jSXV9tkbqB/2mFggq?=
 =?us-ascii?Q?1R0F3S705AnhzJp4Cof20gZNL80Om1aqNFAMXUYupfU1xHisPd5sPGkCZNpD?=
 =?us-ascii?Q?YF/eaiMw0pNXRnfVATmEshGWfZb+4kQNvUqTUErx32nZamIGnG0mY0w7KJzh?=
 =?us-ascii?Q?NEWZeQY4zORty2RLg7SQYOrnIKMtpKK0x3aDKllVg595p6txIaoAe8KedD27?=
 =?us-ascii?Q?SAiqTmsSyJmoyyNOxY82NGvW2LtULCir8GHdUUYilvGTRdHVjEXzl81aQoQq?=
 =?us-ascii?Q?hlUhApijtPaZmZ4gKXkh0XYG/OqNpi/7mOetTlMMrqvRXo1wO25MPl7Uugmc?=
 =?us-ascii?Q?LmP91WI0b+9Hg8Kcf0b7JQSvGSNMAug31kBH+96EiRnlcAnzTTc1nvOJN6d0?=
 =?us-ascii?Q?Yz3UjhhB9YWY9euutOKoSQgda7IRn1fmfJ+FwdScHmwoqLakUBct4FsUqiDy?=
 =?us-ascii?Q?rIHufoBT3w79VlnyEqPw6oqkLJTxvwwdfBKgryOyuD+e/csuKYyZj5BMH/FY?=
 =?us-ascii?Q?tlI9ZEucHdD0vjfXLM2SZVDxFewuWb/L0ih37c0I?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9df30297-8bf5-4a27-765c-08db52b7d766
X-MS-Exchange-CrossTenant-AuthSource: BL0PR11MB2995.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2023 07:09:35.3336
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: oxihi6PmcpH2+TW3jlu/UNgY4LbdealS7C4CYz6WVh9sQLfeoF3hjHya09rWOQPBDqSCv/H04x4d3rXI+fLGWQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB7701
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, May 12, 2023 at 12:25:13PM +0900, Masahiro Yamada wrote:
> Hello, maintainers of the kbuild test robot.
> 
> I have a proposal for the 0day tests.

Thanks a lot for the proposal for the shuffle make, we will do some
investigation to try this random order parallel build. The gnu make
we currently use is 4.3, we will try the 4.4 to see any problem.

For the timeline, we may provide update later this month.

> 
> 
> GNU Make traditionally processes the dependency from left to right.
> 
> For example, if you have dependency like this:
> 
>      all: foo bar baz
> 
> GNU Make builds foo, bar, baz, in this order.
> 
> 
> Some projects that are not capable of parallel builds
> rely on that behavior implicitly.
> 
> Kbuild, however, is intended to work well in parallel.
> (As the maintainer, I really care about it.)
> 
> 
> From time to time, people add "just worked for me" code,
> but apparently that lacks proper dependency.
> Sometimes it requires an expensive CPU to reproduce
> parallel build issues.
> 
> 
> For example, see this report,
>   https://lkml.org/lkml/2016/11/30/587
> 
> The report says 'make -j112' reproduces the broken parallel build.
> Most people do not have such a build machine that comes with 112 cores.
> It is difficult to reproduce it (or even notice it).
> 
> (Some time later, it was root-caused by 07a422bb213a)
> 
> 
> 
> GNU Make 4.4 got this option.
> 
>   --shuffle[={SEED|random|reverse|none}]
>        Perform shuffle of prerequisites and goals.
> 
> 
> 
> 'make --shuffle=reverse' will build in reverse order.
> In the example above, baz, bar, foo.
> 
> 'make --shuffle' will randomize the build order.
> 
> 
> If there exists a missing dependency among foo, bar, baz,
> it will fail to build.
> 
> 
> 
> We already perform the randconfig daily basis.
> So, random-order parallel building is a similar idea.
> 
> Perhaps, it makes sense to add the "--shuffle=SEED" option
> but it requires GNU Make 4.4.  (or GNU Make 4.4.1)
> Is this too new?

Our production environment is 4.3 right now. It will take extra
time for us to upgrade the environment but it's doable for us.

> 
> 
> 
> --
> Best Regards
> Masahiro Yamada