Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760014Ab2JYSD0 (ORCPT ); Thu, 25 Oct 2012 14:03:26 -0400 Received: from mail.lang.hm ([64.81.33.126]:34476 "EHLO bifrost.lang.hm" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759899Ab2JYSDY (ORCPT ); Thu, 25 Oct 2012 14:03:24 -0400 Date: Thu, 25 Oct 2012 11:03:13 -0700 (PDT) From: david@lang.hm X-X-Sender: dlang@asgard.lang.hm To: "Theodore Ts'o" cc: Nico Williams , General Discussion of SQLite Database , =?GB2312?Q?=D1=EE=CB=D5=C1=A2_Yang_Su_Li?= , linux-fsdevel@vger.kernel.org, linux-kernel , drh@hwaci.com Subject: Re: [sqlite] light weight write barriers In-Reply-To: <20121025140327.GB13562@thunk.org> Message-ID: References: <5086F5A7.9090406@vlnb.net> <20121025060231.GC9860@thunk.org> <20121025140327.GB13562@thunk.org> User-Agent: Alpine 2.02 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2109 Lines: 45 On Thu, 25 Oct 2012, Theodore Ts'o wrote: > Or does rsyslog *really* need to issue an fsync after each log > message? Or could it batch updates so that every N seconds, it > flushes writes to the disk? In part this depends on how paranoid the admin is. By default rsyslog doesn't do fsyncs, but admins can configure it to do so and can configure the batch size. However, what I'm talking about here is not normal message traffic, it's the case where the admin has decided that they don't want to use the normal inmemory queues, they want to have the queues be on disk so that if the system crashes the queued data will still be there to be processed after the crash (In addition, this can get used to cover cases where you want queue sizes larger than your available RAM) In this case, the extreme, and only at the explicit direction of the admin, is to fsync after every message. The norm is that it's acceptable to loose the last few messages, but loosing a chunk out of the middle of the queue file can cause a whole lot more to be lost, passing the threshold of acceptable. > Sometimes, the answer is not to try to create exotic database like > functionality in the file system --- the answer is to be more > intelligent at the application leyer. Not only will the application > be more portable, it will also in the end be more efficient, since > even with the most exotic database technologies, the most efficient > transactional commit is the unneeded commit that you optimize away at > the application layer. I agree, this is why I'm trying to figure out the recommended way to do this without needing to do full commits. Since in most cases it's acceptable to loose the last few chunks written, if we had some way of specifying ordering, without having to specify "write this NOW", the solution would be pretty obvious. David Lang -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/