2001-11-07 19:09:57

by spamtrap

[permalink] [raw]
Subject: Yet another design for /proc. Or actually /kernel.


Here's my go at a new design for /proc. I designed it from a userland
point of view and tried not to drown myself into details.

Design goals
============

- Backward compatibility
The current /proc interface is coded into many applications.
A new design must leave room for backward compatibility.
- Forward compatibility
The new interface must be clearly defined and must not change
in the future, unless that change is clearly indicated and it
would cause horrible pain maintaining the current interface.
(Note: I mean the interface to userland, kernel interfaces may
change as much as they like)
- Parsable by applications
Information in /proc isn't really meant for humans to read.
Maybe programmers, but they aren't human anyway. The new /proc
interface must be parsable by applications as easily as
possible.
- Readable by humans
When there's no compelling need to obscure information, don't.
- As simple as possible
Don't bloat the interface with all kinds of fancy decorations.
Let's leave that to userspace.


>From these goals I come to the following design:

Design
======

- Multiple values per file when needed
A file is a two dimensional array: it has lines and every line
can consist of multiple fields.
A good example of this is the current /proc/mounts.
This can be parsed very easily in all languages.
No need for single-value files, that's oversimplification.
- No headers, no pretty formatting
Headers are nice to humans, but applications tend to dislike
them. Same goes for formatting.
An example of this is in /proc/partitions or -- horrors --
/proc/meminfo.
- ASCII values
ASCII is the only format which can be read by humans, shell
scripts, perl, C and the rest of the world.
- Maybe we should move out of /proc
Since this design must leave room for backwards compatibility
we must assume the current /proc stays around for a while. I
propose we create a new kernelfs which is to be mounted at
/kernel.
- Mandatory documentation
Each and every value must be documented. This documentation
may be very, very basic, but it must clearly define the entry
and its intention. This is forward compatibility.


As an example, this is how I would organize the info currently in
/proc/meminfo (any boy, that file is UGLY!):

Every value is in bytes. Rounding to kbytes or mbytes should be done
in userspace.

New Current entry in /proc/meminfo
--------------- -----------------------------------
mem/total MemTotal
mem/free MemFree
mem/shared MemShared
mem/buffers Buffers
mem/cached Cached
mem/active Active
mem/inactive Inactive

mem/high/total HighTotal
mem/high/free HighFree

mem/low/total LowTotal
mem/low/free LowFree

mem/swap/total SwapTotal
mem/swap/free SwapFree
mem/swap/cached SwapCached
mem/swap/files (current /proc/swaps)
--
Erik Hensema ([email protected])
I'm on the list, no need to Cc: me, though I appreciate one if your
mailer doesn't support the References header.


2001-11-07 19:20:27

by Alan

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

> Here's my go at a new design for /proc. I designed it from a userland
> point of view and tried not to drown myself into details.

Did you have to change the subject line. It makes it harder to kill file
when people keep doing that

2001-11-07 19:41:59

by William Knop

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

>>Here's my go at a new design for /proc. I designed it from a userland
>>point of view and tried not to drown myself into details.
>
>Did you have to change the subject line. It makes it harder to kill >file
>when people keep doing that

You really consider this discussion to be unimportant? Granted, the lack of
organization in /proc is not a bug, but keeping the kernel organized is part
of maintenence. Like cleaning my room-- it's not a showstopper if it is
messy, but it is nicer and easier to work in when neat.

Will Knop
[email protected]

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp

2001-11-07 20:59:21

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Followup to: <[email protected]>
By author: [email protected] (Erik Hensema)
In newsgroup: linux.dev.kernel
>
> - Multiple values per file when needed
> A file is a two dimensional array: it has lines and every line
> can consist of multiple fields.
> A good example of this is the current /proc/mounts.
> This can be parsed very easily in all languages.
> No need for single-value files, that's oversimplification.
>

Actually, /proc/mounts is currently broken, and is an excellent
example of why the above statement simply isn't true unless you apply
another level of indirection: try mounting something on a directory
the name of which contains whitespace in any form (remember, depending
on your setup this may be doable by an unprivileged user...)

-hpa
--
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt <[email protected]>

2001-11-07 21:14:57

by Daniel R. Warner

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Erik Hensema wrote:

> Here's my go at a new design for /proc. I designed it from a userland
> point of view and tried not to drown myself into details.

<snip>

> - Multiple values per file when needed
> A file is a two dimensional array: it has lines and every line
> can consist of multiple fields.
> A good example of this is the current /proc/mounts.
> This can be parsed very easily in all languages.


> No need for single-value files, that's oversimplification.
<snip>


This is fine for reading, but it makes it harder for humans to change
values in /proc - eg, echo 0 > /proc/sys/net/ipv4/tcp_ecn

-D


Subject: RE: Yet another design for /proc. Or actually /kernel.

>-----Original Message-----
>From: H. Peter Anvin [mailto:[email protected]]
>Sent: Wednesday, November 07, 2001 2:58 PM
>To: [email protected]
>Subject: Re: Yet another design for /proc. Or actually /kernel.


>Followup to: <[email protected]>
>By author: [email protected] (Erik Hensema)
>In newsgroup: linux.dev.kernel
>>
>> - Multiple values per file when needed
>> A file is a two dimensional array: it has lines and every line
>> can consist of multiple fields.
>> A good example of this is the current /proc/mounts.
>> This can be parsed very easily in all languages.
>> No need for single-value files, that's oversimplification.
>>

>Actually, /proc/mounts is currently broken, and is an excellent
>example of why the above statement simply isn't true unless you apply
>another level of indirection: try mounting something on a directory
>the name of which contains whitespace in any form (remember, depending
>on your setup this may be doable by an unprivileged user...)

> -hpa


mbrennek@spaceheater:/home/mbrennek# mkdir stuff\ and
mbrennek@spaceheater:/home/mbrennek# mount -t vfat /dev/hda1
/home/mbrennek/stuff\ and/
mbrennek@spaceheater:/home/mbrennek# cat /proc/mounts
/dev/ide/host0/bus0/target1/lun0/part1 / reiserfs rw 0 0
/dev/hdb2 /home reisferfs rw 0 0
none /dev/pts devpts rw 0 0
non /proc proc rw 0 0
/dev/hda5 /mnt/stuff vfat rw,nosuid,nodev,noexec 0 0
/dev/hda1 /home/mbrennek/stuff\040and vfat rw 0 0
mbrennek@spaceheater:/home/mbrennek#

Are you refering to the 040?

-Matt

2001-11-07 21:21:36

by Justin A

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Ahh yes I've seen that happen. It's not too common in standard unix
practices, but comes up a lot when mounting windows shares. Mounting
"//bobs computer/my folder with stuff in it" does not have good results.
Only way I found to fix that was to umount -a -t smbfs.

-Justin

On Wed, Nov 07, 2001 at 12:58:24PM -0800, H. Peter Anvin wrote:
> Followup to: <[email protected]>
> By author: [email protected] (Erik Hensema)
> In newsgroup: linux.dev.kernel
> >
> > - Multiple values per file when needed
> > A file is a two dimensional array: it has lines and every line
> > can consist of multiple fields.
> > A good example of this is the current /proc/mounts.
> > This can be parsed very easily in all languages.
> > No need for single-value files, that's oversimplification.
> >
>
> Actually, /proc/mounts is currently broken, and is an excellent
> example of why the above statement simply isn't true unless you apply
> another level of indirection: try mounting something on a directory
> the name of which contains whitespace in any form (remember, depending
> on your setup this may be doable by an unprivileged user...)
>
> -hpa
> --
> <[email protected]> at work, <[email protected]> in private!
> "Unix gives you enough rope to shoot yourself in the foot."
> http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2001-11-07 22:35:57

by Allen Campbell

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 02:42:48PM -0500, Daniel R. Warner wrote:
> Erik Hensema wrote:
>
> > - Multiple values per file when needed
> > A file is a two dimensional array: it has lines and every line
> > can consist of multiple fields.
> > A good example of this is the current /proc/mounts.
> > This can be parsed very easily in all languages.
>
>
> > No need for single-value files, that's oversimplification.
> <snip>
>
>
> This is fine for reading, but it makes it harder for humans to change
> values in /proc - eg, echo 0 > /proc/sys/net/ipv4/tcp_ecn
>

'Multiple value' files can be made easy to 'write'. The only
requirement is each 'field' in the file have a unique label. Then
it's a common associative array, requiring some generic filesystem
write magic to handle the input:

echo "label:1" > /proc/...

The 'generic write magic' would require (at least, without even
more magic) that all /proc files conform to the schema. This is
probably a _good_ thing.

--
Allen Campbell | Lurking at the bottom of the
[email protected] | gravity well, getting old.

2001-11-08 00:01:13

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Brenneke, Matthew Jeffrey (UMR-Student) wrote:

>
>>Actually, /proc/mounts is currently broken, and is an excellent
>>example of why the above statement simply isn't true unless you apply
>>another level of indirection: try mounting something on a directory
>>the name of which contains whitespace in any form (remember, depending
>>on your setup this may be doable by an unprivileged user...)
>>
>
>> -hpa
>>
>
>
> mbrennek@spaceheater:/home/mbrennek# mkdir stuff\ and
> mbrennek@spaceheater:/home/mbrennek# mount -t vfat /dev/hda1
> /home/mbrennek/stuff\ and/
> mbrennek@spaceheater:/home/mbrennek# cat /proc/mounts
> /dev/ide/host0/bus0/target1/lun0/part1 / reiserfs rw 0 0
> /dev/hdb2 /home reisferfs rw 0 0
> none /dev/pts devpts rw 0 0
> non /proc proc rw 0 0
> /dev/hda5 /mnt/stuff vfat rw,nosuid,nodev,noexec 0 0
> /dev/hda1 /home/mbrennek/stuff\040and vfat rw 0 0
> mbrennek@spaceheater:/home/mbrennek#
>
> Are you refering to the 040?
>


Right, a good example of "additional encapsulation".

-hpa


2001-11-08 00:27:34

by John Levon

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 02:41:26PM -0500, William Knop wrote:

> You really consider this discussion to be unimportant? Granted, the lack of
> organization in /proc is not a bug, but keeping the kernel organized is part
> of maintenence. Like cleaning my room-- it's not a showstopper if it is
> messy, but it is nicer and easier to work in when neat.

Like cleaning your room, talking endlessly about how doesn't get you anywhere.
Show some code ...

john

--
"This bulletin discusses three security vulnerabilities that are unrelated
except in the sense that both affect ISA Server 2000"
- Microsoft Product Security

2001-11-08 00:29:14

by antirez

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 03:13:25PM -0600, Brenneke, Matthew Jeffrey (UMR-Student) wrote:
> /dev/hda1 /home/mbrennek/stuff\040and vfat rw 0 0
[snip]
>
> Are you refering to the 040?

This works but, if /proc will really be replaced, another
idea can be to organize the stuff to get something of more
coherent than:

value1a value1b value1c
value2a value2b value2c

that's more human readable than machine parsable.
Something that can fix both the problems (quoting and format) is
the following:

put every string inside () and only quotes ( and ) with \
and quotes \ itself with \\, than use brackets to delimit
string _and_ provide information about the format:

((dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
((/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))

and so on. With a simple parser you get a safe delimiter
for a single element and you know that there are two "objects"
of 6 elements.

An extension may be to use the first row to describe the
elements inside every object. After this the output of
/proc/mounts will be:

((device)(mountpoint)(type)(option)(dump)(fsck))
((dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
((/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))

Good userspace software will never fail if you add more information.
If you need to change the format of something it's just possible
to add a new field and leave the old one for some time for backward
compatibility.

With some simple extension you can add comments or the ability
to redefine the fileds description for the next elements.
With the last improvments it looks like:

((type)(field0)(field1)...(fieldN))

/proc/mounts will be:

((rem)(mounted filesystems))
((format)(device)(mountpoint)(type)(option)(dump)(fsck))
((data)(dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
((data)(/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))
((rem)(number of filesystems mounted))
((format)(mount_count))
((data)(2))

Last fix can be to surround all this with (), to detect
truncated outputs, since it seems that often the /proc outputs
can only use one page.

It looks a bit too complex, but actually it's very simple to
generate and parse. Expecially if someone release a "standard"
userspace library to parse it.

I'm sure all here already thought at something like it, and
probably there is a good reason to do it different, sorry if I
missed the point.

regards,
Salvatore

2001-11-08 00:32:44

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

antirez wrote:

> On Wed, Nov 07, 2001 at 03:13:25PM -0600, Brenneke, Matthew Jeffrey (UMR-Student) wrote:
>
>>/dev/hda1 /home/mbrennek/stuff\040and vfat rw 0 0
>>
> [snip]
>
>>Are you refering to the 040?
>>
>
> This works but, if /proc will really be replaced, another
> idea can be to organize the stuff to get something of more
> coherent than:
>
> value1a value1b value1c
> value2a value2b value2c
>
> that's more human readable than machine parsable.
> Something that can fix both the problems (quoting and format) is
> the following:
>
> put every string inside () and only quotes ( and ) with \
> and quotes \ itself with \\, than use brackets to delimit
> string _and_ provide information about the format:
>
> ((dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
> ((/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))
>
> and so on. With a simple parser you get a safe delimiter
> for a single element and you know that there are two "objects"
> of 6 elements.
>


You still need quoting, so why on earth does this make it easier to parse?
It doesn't; it just makes it harder to read for humans.

-hpa


2001-11-08 00:36:45

by Stephen Satchell

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

At 12:58 PM 11/7/01 -0800, H. Peter Anvin wrote:
>Actually, /proc/mounts is currently broken, and is an excellent
>example of why the above statement simply isn't true unless you apply
>another level of indirection: try mounting something on a directory
>the name of which contains whitespace in any form (remember, depending
>on your setup this may be doable by an unprivileged user...)

Good man.

Now you know why I insisted that ALL STRINGS have a standard escape system
in my less-than-strawman proposal.

You'll find other places in /proc where white space is not necessarily a
delimiter. (Too lazy to go find them right now, however.)

Satch

2001-11-08 00:50:34

by Stephen Satchell

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

At 01:20 AM 11/8/01 +0100, antirez wrote:
>/proc/mounts will be:
>
>((rem)(mounted filesystems))
>((format)(device)(mountpoint)(type)(option)(dump)(fsck))
>((data)(dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
>((data)(/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))
>((rem)(number of filesystems mounted))
>((format)(mount_count))
>((data)(2))

Looks a little like LISPTH. Flex and Bison to the rescue!

(signed) anonymous

2001-11-08 00:51:54

by Tim Jansen

[permalink] [raw]
Subject: /proc discussions

On Wednesday 07 November 2001 20:27, Alan Cox wrote:
> > Here's my go at a new design for /proc. I designed it from a userland
> > point of view and tried not to drown myself into details.
> Did you have to change the subject line. It makes it harder to kill file
> when people keep doing that

There is an easy way for you, or even better, Linus to stop these discussions:

Just say, in unambigous words, what kind of patch you would accept, if any.
The open questions are: in which format should data (from driver and kernel)
be exported? How can the kernel, drivers and devices be configured from user
space?
Even better, you could also say a few words about the possible disadvantages
that the decision has and why we shouldn't care. Something like "performance
is irrelevant", "it doesn't matter if the format cannot be changed without
breaking user applications", "for every format there must be a strict
definition of how to parse it so that the new fields can be added later", "we
don't need to define the format as long at it is readable for human readers",
"if you need a pretty format write some user space tool that formats it", "it
doesnt matter if each file uses a completely different format", "it is not
acceptable to break any user space applications before 2.x", "strict typing
is irrelevant"...


I REALLY do have problems to define any proc interface without having at
least one of the disadvantages above. I have spent some time to find LT
statements about proc/sysctl in order to find out which way may have a chance
to get into the kernel (this time is possibly better spent than writing
another proc replacement :). Maybe I have missed some important ones, in
essence they are:

4 Nov 2001:
"In short: /proc is ASCII, and will so remain while I maintain a kernel.
Anything else is stupid.
Handling spaces and newlines is easy enough - see the patches from Al
Viro, for example."

4 Nov 2000:
"[context added]
> create_proc_entries(NULL,
> "test:{bar:{x:%d,y:%d,z:%d},foo:%f}",
> &x, &y, &z, foo_fun);
>
> creates a "/proc/test" directory, which further contains a
> subdirectory "bar" and a file "foo". The "bar" subdirectory contains
> three files "x", "y" and "z".
>
> The formatting argument "%d" takes a pointer to an integer. When
> reading such a file (in this case "x", "y", or "z"), the value is
> shown as ascii. Writing to the file (again in ascii) updates the
> value. The "%f" formatting argument allows you to pass an arbitrary
> user function for generating output. Clearly, there are potentially
> quite a number of standard/useful formatting arguments.
>
> I've done a quick, dirty, unfinished implementation of this idea, so
> people can get the picture. Attached.
>
> Many people will hate this because (1) it's doing parsing within the
> kernel, (2) it tends to favour ascii I/O, (3) it tends to favour deep
> directory hierarchies,(4) it uses recursion :-)

I like the idea a lot: it's simple, it's clean, and it does make it truly
trivial to build up a simple /proc architecture.
"

7 Jan 2000:
"In contrast, sysctl isn't all that performance-sensitive, AND they are
extremely hierarchical, AND they depend on configuration and timing.

In short, sysctl NEEDS:
- "naming": you cannot name the sysctl space with a number: it is much
too dynamic for that. How do you enumerate drivers? Give them random
numbers?
- "listing": showing which sysctl's are there, in a hierarchical manner.
Again, a listing is useless with a number.
- "hierarchy". You have different devices, but they have the same
controls. Do they get the same name? Yes. But in different places in
the hierarchy.

In short, you NEED a filesystem. You need to be able to "ls" the thing.
You need to be able to search the thing. You need to be doing all the
things you can do with a real filesystem."

6 Jan 2000:
"The thing to do is to create a

/proc/drivers/<drivername>/

directory. The /proc/drivers/ directory is already there, so you'd
basically do something like

create_proc_info_entry("driver/mydriver/status", 0, NULL,
mydriver_status_read);
"

1 Jan 1998:
"- I think it should be a separate filesystem type. It was a mistake in
the first place to overload /proc as much as we did, we shouldn't
continue that (I should really have split out the "per-process" things
and the "global" things to two separate filesystems). "


Is the 4-Nov-2000 mail combined with 1-Jan-1998 the answer?

bye...



2001-11-08 00:56:45

by David Ford

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

That doesn't solve anything if the data value includes ( or ). It just
avoids ' ' in the data value and adds complexity.

-d

antirez wrote:

>((dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
>((/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))
>

2001-11-08 01:01:54

by Jonathan Lundell

[permalink] [raw]
Subject: RE: Yet another design for /proc. Or actually /kernel.

At 3:13 PM -0600 11/7/01, Brenneke, Matthew Jeffrey (UMR-Student) wrote:
> >> - Multiple values per file when needed
>>> A file is a two dimensional array: it has lines and every line
>>> can consist of multiple fields.
>>> A good example of this is the current /proc/mounts.
>>> This can be parsed very easily in all languages.
>>> No need for single-value files, that's oversimplification.
>>>
>
>>Actually, /proc/mounts is currently broken, and is an excellent
>>example of why the above statement simply isn't true unless you apply
>>another level of indirection: try mounting something on a directory
>>the name of which contains whitespace in any form (remember, depending
>>on your setup this may be doable by an unprivileged user...)

If [tag plus] multiple values are allowed on a line, the field
separation needs to be unambiguous. So quoting/escaping is required
in some cases. Spaces are common enough in a value that white space
maybe doesn't make a very good field separator.

Or just quote all strings (and escape quotes). Interpret values as C
would, 0x for hex, "... for strings, and so on for floating point,
octal, &c. Heck, you could even have typing (3UL) or casts, though
that's probably going too far.
--
/Jonathan Lundell.

2001-11-08 01:13:24

by antirez

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 04:44:50PM -0800, Stephen Satchell wrote:
> At 01:20 AM 11/8/01 +0100, antirez wrote:
> >/proc/mounts will be:
> >
> >((rem)(mounted filesystems))
> >((format)(device)(mountpoint)(type)(option)(dump)(fsck))
> >((data)(dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
> >((data)(/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))
> >((rem)(number of filesystems mounted))
> >((format)(mount_count))
> >((data)(2))
>
> Looks a little like LISPTH. Flex and Bison to the rescue!
>
> (signed) anonymous

I'm really a newbie. Maybe I missed the whole point but
stay sure you don't need flex and bison to parse this.
It's _really_ simple, you need little code of little
complexity, to get something of flexible that can be
used for long time.

If you don't trust me I can write a little C library to parse
this just for provide a proof.
This isn't only simple to parse, but even simple to generate,
that's an important point.

regards,
Salvatore

2001-11-08 01:19:14

by antirez

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 07:54:21PM -0500, David Ford wrote:
> That doesn't solve anything if the data value includes ( or ). It just
> avoids ' ' in the data value and adds complexity.

Wrong, exampel of () in data:

((data)(\(\)))

About the complexity. It only "looks" complex. But from the
machine point of view it's very simple to parse.
Note that the strong advantage of this isn't the quoting,
you can quote anyway in 1000 different ways. The advantage
is that data is structured and parsing does not rely on
spaces or newlines, but just on ().
With this syntax you can express data as complex and structured
as you want but the parsing is still simple.

Regards,
Salvatore

2001-11-08 01:27:14

by H. Peter Anvin

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

antirez wrote:

> On Wed, Nov 07, 2001 at 07:54:21PM -0500, David Ford wrote:
>
>>That doesn't solve anything if the data value includes ( or ). It just
>>avoids ' ' in the data value and adds complexity.
>>
>
> Wrong, exampel of () in data:
>
> ((data)(\(\)))
>
> About the complexity. It only "looks" complex. But from the
> machine point of view it's very simple to parse.
> Note that the strong advantage of this isn't the quoting,
> you can quote anyway in 1000 different ways. The advantage
> is that data is structured and parsing does not rely on
> spaces or newlines, but just on ().
> With this syntax you can express data as complex and structured
> as you want but the parsing is still simple.
>


You just changed spaces and newlines to ( and ) -- it doesn't really solve
anything unless you want three levels of nesting or more; in which case
you have *WAY* too much data in a single proc item.

-hpa


2001-11-08 01:44:06

by Rusty Russell

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On 7 Nov 2001 19:09:35 GMT
[email protected] (Erik Hensema) wrote:

>
> Here's my go at a new design for /proc. I designed it from a userland
> point of view and tried not to drown myself into details.

> - Multiple values per file when needed
> A file is a two dimensional array: it has lines and every line
> can consist of multiple fields.
> A good example of this is the current /proc/mounts.
> This can be parsed very easily in all languages.
> No need for single-value files, that's oversimplification.

No, it deals nicely with any possible values in the file. And without headers,
how do I know what's what? And how do I update one value.

Meanwhile, there's far too much talk, far too little code. Will post new patch
next week.

Rusty.

2001-11-08 01:59:40

by antirez

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

On Wed, Nov 07, 2001 at 05:26:32PM -0800, H. Peter Anvin wrote:
> > About the complexity. It only "looks" complex. But from the
> > machine point of view it's very simple to parse.
> > Note that the strong advantage of this isn't the quoting,
> > you can quote anyway in 1000 different ways. The advantage
> > is that data is structured and parsing does not rely on
> > spaces or newlines, but just on ().
> > With this syntax you can express data as complex and structured
> > as you want but the parsing is still simple.
> >
>
>
> You just changed spaces and newlines to ( and ) -- it doesn't really solve
> anything unless you want three levels of nesting or more; in which case
> you have *WAY* too much data in a single proc item.
>
> -hpa

There are anyway different ways to output the same data, and yes,
probably spaces/tabs/newlines are more human readable, but I think
the right solution isn't something that limits a-priori the
complexity of the output. This will make developers more prone
to invent their own formats for special stuff. the lisp-like way
allows you to automagically put a description of the format with
little efforts, simple parsing, unlimited complexity.
Maybe you want limited complexity, but the format isn't your limit
anyway.

About the two level of nesting, take a look at /proc/net/netstat.
it's not very clear, but in lisp-like it can be translated to:

((TcpExt)((SyncookiesSent)(0)))

and so on. For every kind of proc output you can find today, there
is a good way to convert it in that format, that is at the same
time used by all the entries. I think you will hardly get the same
with space/tabs/newlines without to indirectly use it like (), that
will probably result in something of more complex to generate/parse.

I can't see any strong reason to adopt a format that will for sure
fail at some time in the future.

BTW I see that the idea isn't well accepted, so I'll be quiet ;)

Regards,
Salvatore

2001-11-08 02:24:02

by Alexander Viro

[permalink] [raw]
Subject: Re: /proc discussions



On Thu, 8 Nov 2001, Tim Jansen wrote:

> There is an easy way for you, or even better, Linus to stop these discussions:
>
> Just say, in unambigous words, what kind of patch you would accept, if any.

.procmailrc one would do nicely.

2001-11-08 03:07:40

by Stuart Young

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

At 01:20 AM 8/11/01 +0100, antirez wrote:
>put every string inside () and only quotes ( and ) with \
>and quotes \ itself with \\, than use brackets to delimit
>string _and_ provide information about the format:
>
>((dev/hda1)(/home/mbrennek/stuff and)(vfat)(rw)(0)(0))
>((/dev/hda2)(/var/tmp)(ext2)(rw)(0)(0))

Now that is just really annoying, and hardly useful.

Why not just allow the user to specify what the field separation character
is. If the entry/file is opened for writing, then the data passed to it is
used as the separator. If it's opened in read, just output as normal.

So if say /proc/kernel/foo normally exports...

bar-a1 bar-a2 bar-a3
bar-b1 bar-b2 bar-b3
[===> ] 40%

...and you pass a colon ":", it spits out...

bar-a1:bar-a2:bar-a3
bar-b1:bar-b2:bar-b3
40

This allows the user to make the choice. If they use the wrong character,
they can always change it. If they don't specify one, they get the default
output (which is what existing scripts will expect). From a code
perspective, you should only need to make all your separators (formatting)
as variables in the code, and substitute the separator character(s) if
there is one that's been passed, and just drop the remaining 'pretty'
garbage totally.

Also means you can easily use command line tools like cut (-d option),
readline (the read function in bash, and the $IFS variable), and so on, to
parse the data, quickly and effectively.


AMC Enterprises P/L - Stuart Young
First Floor - Network and Systems Admin
3 Chesterville Rd - [email protected]
Cheltenham Vic 3192 - Ph: (03) 9584-2700
http://www.amc.com.au/ - Fax: (03) 9584-2755

2001-11-08 08:56:47

by spamtrap

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

John Levon ([email protected]) wrote:
>On Wed, Nov 07, 2001 at 02:41:26PM -0500, William Knop wrote:
>
>> You really consider this discussion to be unimportant? Granted, the lack of
>> organization in /proc is not a bug, but keeping the kernel organized is part
>> of maintenence. Like cleaning my room-- it's not a showstopper if it is
>> messy, but it is nicer and easier to work in when neat.
>
>Like cleaning your room, talking endlessly about how doesn't get you anywhere.
>Show some code ...

Just starting to code without giving much thought into it give us... the
current /proc. It may be a good idea to actually design the thing this
time.

--
Erik Hensema ([email protected])
I'm on the list, no need to Cc: me, though I appreciate one if your
mailer doesn't support the References header.

2001-11-08 10:01:04

by Remco Post

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

> >>Here's my go at a new design for /proc. I designed it from a userland
> >>point of view and tried not to drown myself into details.
> >
> >Did you have to change the subject line. It makes it harder to kill >file
> >when people keep doing that
>
> You really consider this discussion to be unimportant? Granted, the lack of
> organization in /proc is not a bug, but keeping the kernel organized is part
> of maintenence. Like cleaning my room-- it's not a showstopper if it is
> messy, but it is nicer and easier to work in when neat.
>
> Will Knop
> [email protected]

Hi,

the discussion is irrelevant. Despite what everybody thinks, Linus thinks
/proc must be not binary, so it will stay that way for those of us who run
Linus kernels...

I can inmagine people like Alan ignoring this discussion after such a
statement, the outcome of the discussion is irrelevant for the kernel
development.


--
Met vriendelijke groeten,

Remco Post

SARA - Stichting Academisch Rekencentrum Amsterdam
High Performance Computing Tel. +31 20 592 8008 Fax. +31 20 668 3167

"I really didn't foresee the Internet. But then, neither did the computer
industry. Not that that tells us very much of course - the computer industry
didn't even foresee that the century was going to end." -- Douglas Adams


2001-11-09 16:44:49

by Ricky Beam

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

>the discussion is irrelevant. Despite what everybody thinks, Linus thinks
>/proc must be not binary, so it will stay that way for those of us who run
>Linus kernels...

Linus has been "wrong" before. It will require good code and numbers
backing that codes "goodness" before Linus will begin to listen. Yes,
a new procfs format will break a great deal of userland toys, so the
changes had better be worth it and sufficient to never, EVER require
a complete overhaul in the future.

>I can inmagine people like Alan ignoring this discussion after such a
>statement, the outcome of the discussion is irrelevant for the kernel
>development.

I think Alan is already ignoring the entire discussion.

--Ricky


2001-11-12 13:32:26

by Horst H. von Brand

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Ricky Beam <[email protected]> said:
> >the discussion is irrelevant. Despite what everybody thinks, Linus thinks
> >/proc must be not binary, so it will stay that way for those of us who run
> >Linus kernels...
>
> Linus has been "wrong" before. It will require good code and numbers
> backing that codes "goodness" before Linus will begin to listen. Yes,
> a new procfs format will break a great deal of userland toys, so the
> changes had better be worth it and sufficient to never, EVER require
> a complete overhaul in the future.

/proc for process info is a given (many Unices have it, it is nice at least
for compatibility).

/proc for random other garbage should go away. To get at some value you can
get via specialized calls by read(2) also is just kernel bloat.
--
Dr. Horst H. von Brand Usuario #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513

2001-11-12 13:39:36

by Martin Dalecki

[permalink] [raw]
Subject: Re: Yet another design for /proc. Or actually /kernel.

Horst von Brand wrote:
>
> Ricky Beam <[email protected]> said:
> > >the discussion is irrelevant. Despite what everybody thinks, Linus thinks
> > >/proc must be not binary, so it will stay that way for those of us who run
> > >Linus kernels...
> >
> > Linus has been "wrong" before. It will require good code and numbers
> > backing that codes "goodness" before Linus will begin to listen. Yes,
> > a new procfs format will break a great deal of userland toys, so the
> > changes had better be worth it and sufficient to never, EVER require
> > a complete overhaul in the future.
>
> /proc for process info is a given (many Unices have it, it is nice at least
> for compatibility).
>
> /proc for random other garbage should go away. To get at some value you can
> get via specialized calls by read(2) also is just kernel bloat.

My absolute favourite is the following in arch/i386/kernel/irq.c!!!!

Disease symptoms like this are distributed all over the kernel code.

========================================================================
static struct proc_dir_entry * root_irq_dir;
static struct proc_dir_entry * irq_dir [NR_IRQS];

#define HEX_DIGITS 8

static unsigned int parse_hex_value (const char *buffer,
unsigned long count, unsigned long *ret)
{
unsigned char hexnum [HEX_DIGITS];
unsigned long value;
int i;