From: Edward Shishkin Subject: Re: [Jfs-discussion] benchmark results Date: Fri, 15 Jan 2010 22:42:38 +0100 Message-ID: <4B50E14E.5050204__28237.8983371474$1263591917$gmane$org@gmail.com> References: <20091224212756.GM21594@thunk.org> <20091225161453.GD32757@thunk.org> <20091225162238.GB19303@bitmover.com> <4B36333B.3030600@hp.com> <4B365EBE.5050804@nerdbynature.de> <4B37BA76.7050403@hp.com> <20091227223307.GA4429@thunk.org> <20091228140855.GD10982@bitmover.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit To: Larry McVoy , tytso@mit.edu, Christian Kujau , jim owens , jfs-discussion@lists.sourceforge.net, linux-nilfs@vger.kernel.org, x Return-path: Received: from mx1.redhat.com ([209.132.183.28]:16826 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758452Ab0AOVpB (ORCPT ); Fri, 15 Jan 2010 16:45:01 -0500 In-Reply-To: <20091228140855.GD10982@bitmover.com> Sender: linux-ext4-owner@vger.kernel.org List-ID: > When things didn't match up that was a clue that either > > - the benchmark was broken > - the code was broken > [...] I would carry out an object-oriented dualism here. [1] methods (kernel module) ---- [2] objects (formatted partition) | | | | [3] benchmarks ----------------- [4] user-space utilities (fsck) User-space utilities investigate "object corruptions", whereas benchmarks investigate "software corruptions" (including bugs in source code, broken design, etc, etc..) It is clear that "software" can be "corrupted" by a larger number of ways than "objects". Indeed, it is known that dual space V* (of all linear functions over V) is a much more complex object than V. So benchmark is a process which takes a set of methods (we consider only "software" benchmarks) and puts numerical values populated with a special (the worst) value CRASH. Three main categories of benchmarks using: 1) Internal testing An engineer makes optimizations in a file system (e.g. for a customer) via choosing functions or plugins as winners in a set of internal (local) "nominations". 2) Business plans A system administrator chooses a "winner" in some (global) "nomination" of file systems in accordance with internal business-plans. 3) Flame and politics Someone presents a "nomination" (usually with the "winner" among restricted number of nominated members) to the public while nobody asked him to do it.