2010-11-29 12:37:21

by Olaf van der Spek

[permalink] [raw]
Subject: Atomic non-durable file write API

Hi,

Since the introduction of ext4, some apps/users have had issues with
file corruption after a system crash. It's not a bug in the FS AFAIK
and it's not exclusive to ext4.
Writing a temp file, fsync, rename is often proposed. However, the
durable aspect of fsync isn't always required and this way has other
issues.
What is the recommended way for atomic non-durable (complete) file writes?

I'm also wondering why FSs commit after open/truncate but before
write/close. AFAIK this isn't necessary and thus suboptimal.

Greetings,

Olaf


2010-12-01 10:27:18

by Olaf van der Spek

[permalink] [raw]
Subject: Re: Atomic non-durable file write API

On Mon, Nov 29, 2010 at 1:37 PM, Olaf van der Spek <[email protected]> wrote:
> Hi,
>
> Since the introduction of ext4, some apps/users have had issues with
> file corruption after a system crash. It's not a bug in the FS AFAIK
> and it's not exclusive to ext4.
> Writing a temp file, fsync, rename is often proposed. However, the
> durable aspect of fsync isn't always required and this way has other
> issues.
> What is the recommended way for atomic non-durable (complete) file writes?
>
> I'm also wondering why FSs commit after open/truncate but before
> write/close. AFAIK this isn't necessary and thus suboptimal.

Is this the wrong list for this kind of question?

Greetings,

Olaf

2010-12-06 16:45:27

by Olaf van der Spek

[permalink] [raw]
Subject: Re: Atomic non-durable file write API

On Mon, Nov 29, 2010 at 1:37 PM, Olaf van der Spek <[email protected]> wrote:
> Hi,
>
> Since the introduction of ext4, some apps/users have had issues with
> file corruption after a system crash. It's not a bug in the FS AFAIK
> and it's not exclusive to ext4.
> Writing a temp file, fsync, rename is often proposed. However, the
> durable aspect of fsync isn't always required and this way has other
> issues.
> What is the recommended way for atomic non-durable (complete) file writes?
>
> I'm also wondering why FSs commit after open/truncate but before
> write/close. AFAIK this isn't necessary and thus suboptimal.

Somebody?

2010-12-06 17:03:34

by Randy Dunlap

[permalink] [raw]
Subject: Re: Atomic non-durable file write API

On Mon, 6 Dec 2010 17:45:23 +0100 Olaf van der Spek wrote:

> On Mon, Nov 29, 2010 at 1:37 PM, Olaf van der Spek <[email protected]> wrote:
> > Hi,
> >
> > Since the introduction of ext4, some apps/users have had issues with
> > file corruption after a system crash. It's not a bug in the FS AFAIK
> > and it's not exclusive to ext4.
> > Writing a temp file, fsync, rename is often proposed. However, the
> > durable aspect of fsync isn't always required and this way has other
> > issues.
> > What is the recommended way for atomic non-durable (complete) file writes?
> >
> > I'm also wondering why FSs commit after open/truncate but before
> > write/close. AFAIK this isn't necessary and thus suboptimal.
>
> Somebody?
> --

maybe try [email protected] or [email protected]
mailing lists?

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***