On 4/28/06, linux-os (Dick Johnson) <[email protected]> wrote:
>
> On Fri, 28 Apr 2006, Avi Kivity wrote:
> >
> > If your time is bounded, your Python code might be running while you're
> > still typing in your C code, you're be profiling and making changes to
> > the alghorithm in Python while hunting for that mysterious segmentation
> > fault in C (thank goodness for valgrind), and adding multithreading to
> > the third and final version of your Python code while debating whether
> > to buy more memory or sit down and chase that memory leak.
> >
> > Developer performance equates to runtime performance.
> >
>
> Read what you wrote! It's absolutely, incredibly stupid!
>
> The cost in developer time is borne once. The cost of performance
> is borne every time you run the application.
The cost in developer time is borne every time someone needs to modify the code.
Dave
>>
>> The cost in developer time is borne once. The cost of performance
>> is borne every time you run the application.
>
> The cost in developer time is borne every time someone needs to modify the
> code.
>
The clever developer can cope with both situations.
Jan Engelhardt
--
> > The cost in developer time is borne once. The cost of performance
> > is borne every time you run the application.
> The cost in developer time is borne every time someone needs to
> modify the code.
Or understand the code. Or debug the code. Or verify that the code operates
correctly. Or reuse the code for another purpose.
In the bad old days, performance was the number one priority because
computers were slow and resources were scarce -- if you didn't your code
wasn't usable. There is still a small amount of code where performance is
truly the most important priority. Certainly, very low-level kernel code
falls in this category.
We aren't in the bad old days anymore. And there are quite a few things
that are important other than performance. Clear, simple code is easier to
understand and maintain and more likely to be correct. Modifications are
less likely to break hidden dependencies. Code that isn't heavily optimized
is more likely to be secure.
And the supreme irony is that the code often performs better anyway! There
are a lot of reasons why this is often the case. For example, clearer more
modular code is easier to optimize algorithmically. Hand optimizations may
remain in code long past the point where they made sense and to the point
where they become pessimizations because of new CPU architectures or smarter
compilers. Poor code organization mixes performance-critical code with code
that's not performance-critical so that the critical code is harder to
identify and optimize.
I am not saying that the use of C++ over C is likely to improve
performance. I'm saying that there's a lot of code where performance is not
the most important priority, and that this type of code accounts for the
majority of code in a monolithic kernel.
DS
On Mon, May 01, 2006 at 04:53:47PM -0700, David Schwartz wrote:
>
> > > The cost in developer time is borne once. The cost of performance
> > > is borne every time you run the application.
>
> > The cost in developer time is borne every time someone needs to
> > modify the code.
>
> Or understand the code. Or debug the code. Or verify that the code operates
> correctly. Or reuse the code for another purpose.
>
> In the bad old days, performance was the number one priority because
> computers were slow and resources were scarce -- if you didn't your code
> wasn't usable. There is still a small amount of code where performance is
> truly the most important priority. Certainly, very low-level kernel code
> falls in this category.
>
> We aren't in the bad old days anymore. And there are quite a few things
> that are important other than performance. Clear, simple code is easier to
> understand and maintain and more likely to be correct.
Sorry , but all the examples that have been given in C++ are clearly
unreadable and impossible to understand. I'd also like to note that
people were arguing about what the code was really doing, this means
that this language is absolutely not suited to such usages where you
want to know the exact behaviour. At least in C, this sort of thing
has never happened. People argue about what must be locked and important
things like this you'd never want the compiler to decide for you.
> Modifications are
> less likely to break hidden dependencies. Code that isn't heavily optimized
> is more likely to be secure.
To be secure, you first have to understand what the code precisely does,
not what it should do depending on how the compiler might optimise it.
> And the supreme irony is that the code often performs better anyway! There
> are a lot of reasons why this is often the case. For example, clearer more
> modular code is easier to optimize algorithmically. Hand optimizations may
> remain in code long past the point where they made sense and to the point
> where they become pessimizations because of new CPU architectures or smarter
> compilers. Poor code organization mixes performance-critical code with code
> that's not performance-critical so that the critical code is harder to
> identify and optimize.
>
> I am not saying that the use of C++ over C is likely to improve
> performance. I'm saying that there's a lot of code where performance is not
> the most important priority, and that this type of code accounts for the
> majority of code in a monolithic kernel.
I'm still thinking that people who have problems understanding what the
code does want a level of abstraction between them and the CPU so that
the compiler thinks for them. I still don't see the *current* problem
you are trying to fix. Linux is written in C, as many other kernels and
it works. Nobody knows what it would become if rewritten in C++. Maybe
it will be better, maybe it would not run anymore on embedded systems,
maybe it would become fully buggy because nobody except a little bunch
of C++ coders would understand it... At least, I'm sure it will not be
the smart people who currently work on it.
Best of all, I'm even sure that people who are trying to push C++ in
the kernel would never ever write a line of code once it would be
accepted, because they don't seem to know what they're talking about
when it applies to kernel code.
> DS
Regards,
Willy
Willy Tarreau wrote:
> Sorry , but all the examples that have been given in C++ are clearly
> unreadable and impossible to understand.
If you don't know C++, sure.
Maybe this piece of code
f [] = []
f (x:xs) = f [y | y <- xs, y < x] ++ [x] ++ f [y | y <- xs, y > x]
Isn't very readable to a C or C++ coder, but it's meaning is instantly
clear to one who knows the language it's written in. The equivalent C or
C++ code is ~30 lines long and you might need a pen and paper to work
out what it does.
> I'd also like to note that
> people were arguing about what the code was really doing, this means
> that this language is absolutely not suited to such usages where you
> want to know the exact behaviour.
Were they C coders or C++ coders?
> At least in C, this sort of thing
> has never happened.
Like the recent prevent_tail_call() thing? Granted, C++ is a lot tricker
than C. Much self-restraint is needed, and even then you can wind up
where you didn't want to go.
> People argue about what must be locked and important
> things like this you'd never want the compiler to decide for you.
>
>
No one suggests that C++ can decide what needs to be locked or not. To
be sure, if there was a language where you could specify the locking
rules in a central place (the class/struct declaration, for example) and
let the compiler apply them, races and deadlocks would be much scarcer,
and people could argue about what needs to be locked when the data
structure is written, not every time it is used.
> To be secure, you first have to understand what the code precisely does,
> not what it should do depending on how the compiler might optimise it.
>
If optimization changes your code's behaivor, your code is broken.
People rely on the C++ optimizer for much the same things as C: to
inline zero- or one- line functions and remove unused code. (There is
just one exception in C++ where the optimizer _is_ allowed to modify
behavior, but it is for an obviously correct scenario).
> I'm still thinking that people who have problems understanding what the
> code does want a level of abstraction between them and the CPU so that
> the compiler thinks for them.
No, they want not to repeat code and code patterns. It's the same
motivation that lead to the invention of functions:
- functions allow you to reuse code instead of open-coding common sequences
- constructors/destructors allow you to reuse the do/undo (lock/unlock,
etc.) pattern without writing it in full every time
- templates allow you to reuse code even when the data types change
(like the preprocessor but not limited to linked lists)
- virtual functions allow you to dispatch a function based on the
object's type, without writing the boilerplate casting
- exceptions allow you to do the detect error/undo partial
modifications/propagate error thing without blowing up the code by a
factor of five
It's just shorthand: but shorthand allows you to see what the code is
doing instead of how it handles all the standard problems that occur
again and again in programming.
The C people are content to stop at functions, but resist _all_ of the
rest (it's okay to do some template-like magic with typeof, because it's
still C, right?).
> I still don't see the *current* problem
> you are trying to fix. Linux is written in C, as many other kernels and
> it works. Nobody knows what it would become if rewritten in C++. Maybe
> it will be better, maybe it would not run anymore on embedded systems,
> maybe it would become fully buggy because nobody except a little bunch
> of C++ coders would understand it... At least, I'm sure it will not be
> the smart people who currently work on it.
>
Maybe it would be smaller, faster, more robust, and have even more
flexible and fast-paced development.
Perhaps people who developed kernel-level code in _both_ C and C++ would
be qualified to speculate on that (I have, but apparently I don't have a
clue).
> Best of all, I'm even sure that people who are trying to push C++ in
> the kernel would never ever write a line of code once it would be
> accepted, because they don't seem to know what they're talking about
> when it applies to kernel code.
>
Thanks.
--
error compiling committee.c: too many arguments to function
Hello!
> Perhaps people who developed kernel-level code in _both_ C and C++ would
> be qualified to speculate on that (I have, but apparently I don't have a
> clue).
Well, what about just showing an example of kernel code in C++, which
you consider nice?
Have a nice fortnight
--
Martin `MJ' Mares <[email protected]> http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
f u cn rd ths, u cn gt a gd jb n cmptr prgrmmng.
Martin Mares wrote:
> Hello!
>
>
>> Perhaps people who developed kernel-level code in _both_ C and C++ would
>> be qualified to speculate on that (I have, but apparently I don't have a
>> clue).
>>
>
> Well, what about just showing an example of kernel code in C++, which
> you consider nice?
>
I don't have access to that code (which was closed source anyway).
But it executed C++ code within a few cycles of entering the reset
vector (no standard BIOS), including but not limited to: programming the
memory controller (430MX chipset), servicing interrupts, hardware
accelerated 2D graphics (C&T 65550), IDE driver, simple filesystem,
simple windowing GUI, network driver (Tulip) etc.
More recently I participated in writing a filesystem in C++. That's in
userspace, but many of the techniques used in writing kernel code are
necessary there (extreme robustness, can't assume infinite memory,
locking, etc.)
There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/ and
http://ecos.sourceware.org/, but I haven't looked at them, so I can't
say whether I consider them nice or not.
--
error compiling committee.c: too many arguments to function
On Tue, 2 May 2006, Avi Kivity wrote:
> Martin Mares wrote:
>> Hello!
>>
>>
>>> Perhaps people who developed kernel-level code in _both_ C and C++ would
>>> be qualified to speculate on that (I have, but apparently I don't have a
>>> clue).
>>>
>>
>> Well, what about just showing an example of kernel code in C++, which
>> you consider nice?
>>
>
> I don't have access to that code (which was closed source anyway).
>
> But it executed C++ code within a few cycles of entering the reset
> vector (no standard BIOS), including but not limited to: programming the
> memory controller (430MX chipset), servicing interrupts, hardware
> accelerated 2D graphics (C&T 65550), IDE driver, simple filesystem,
> simple windowing GUI, network driver (Tulip) etc.
>
Oh wow! I'm impressed. You, know back in the '50s we did something
like that. It was a dynamic abacus for fast math. Since it's
proprietary, I can't disclose its exact nature. It's even used
by the CIA.
> More recently I participated in writing a filesystem in C++. That's in
> userspace, but many of the techniques used in writing kernel code are
> necessary there (extreme robustness, can't assume infinite memory,
> locking, etc.)
Sure, kernel code is probably written using symbols, too. It's a
lot like user-space as well.
>
> There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/ and
> http://ecos.sourceware.org/, but I haven't looked at them, so I can't
> say whether I consider them nice or not.
>
While you are at it, look at:
http://www.google.com/search?hl=en&q=bullshit
> --
> error compiling committee.c: too many arguments to function
>
Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.4 on an i686 machine (5592.89 BogoMips).
New book: http://www.lymanschool.com
_
****************************************************************
The information transmitted in this message is confidential and may be privileged. Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to [email protected] - and destroy all copies of this information, including any attachments, without reading or disclosing them.
Thank you.
On Tue, 2006-05-02 at 14:26 +0300, Avi Kivity wrote:
> There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/
> and http://ecos.sourceware.org/, but I haven't looked at them, so I
> can't say whether I consider them nice or not.
eCos is nice enough -- because it's mostly C :)
--
dwmw2
Hello!
> But it executed C++ code within a few cycles of entering the reset
> vector (no standard BIOS), including but not limited to: programming the
> memory controller (430MX chipset), servicing interrupts, hardware
> accelerated 2D graphics (C&T 65550), IDE driver, simple filesystem,
> simple windowing GUI, network driver (Tulip) etc.
I really don't claim it's impossible to write kernels in C++ -- it's
clearly possible given that everything you can do in C, you can do in
C++ as well. But what I argued about is whether kernel programming in C++
can be easier and more efficient, which is why I wanted you to show
some examples. Real code speaks better than thousand theories.
Have a nice fortnight
--
Martin `MJ' Mares <[email protected]> http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
Only dead fish swim with the stream.
On Tue, May 02, 2006 at 01:32:05PM +0300, Avi Kivity wrote:
> Willy Tarreau wrote:
> >Sorry , but all the examples that have been given in C++ are clearly
> >unreadable and impossible to understand.
>
> If you don't know C++, sure.
>
> Maybe this piece of code
>
> f [] = []
> f (x:xs) = f [y | y <- xs, y < x] ++ [x] ++ f [y | y <- xs, y > x]
>
> Isn't very readable to a C or C++ coder, but it's meaning is instantly
> clear to one who knows the language it's written in. The equivalent C or
> C++ code is ~30 lines long and you might need a pen and paper to work
> out what it does.
It would be the exact same problem I discribed above. If nobody understand
what it does, or if people need some deep thoughts to guess its behaviour,
the result will be negative. Currently, everyone can read the kernel code.
I was a terrible looser in C 11 years ago when I first looked at it, but I
did not need specific knowledge of implicit things that the kernel does to
understand what the code did, and to propose patches.
Right now, lots of newbies review the code and find bugs. If you write it
in a language such as above, it will quickly end up like the BSD kernels,
with a small very competent core team and lots of newbies around them.
> > I'd also like to note that
> >people were arguing about what the code was really doing, this means
> >that this language is absolutely not suited to such usages where you
> >want to know the exact behaviour.
>
> Were they C coders or C++ coders?
I don't know, what I can say is that they were people who regularly discuss
the kernel semantics on other threads. That's what matters. Those people are
good at finding kernel bugs and proposing interesting algorithms, it's a shame
that they had to discuss langage semantics on 5 lines of code !
> >At least in C, this sort of thing
> >has never happened.
>
> Like the recent prevent_tail_call() thing? Granted, C++ is a lot tricker
> than C. Much self-restraint is needed, and even then you can wind up
> where you didn't want to go.
I've not followed the issue, sorry.
> >People argue about what must be locked and important
> >things like this you'd never want the compiler to decide for you.
> >
> >
>
> No one suggests that C++ can decide what needs to be locked or not. To
> be sure, if there was a language where you could specify the locking
> rules in a central place (the class/struct declaration, for example) and
> let the compiler apply them, races and deadlocks would be much scarcer,
> and people could argue about what needs to be locked when the data
> structure is written, not every time it is used.
I seems like pro-C++ people constantly tell the other ones "look, C++ can
code for you so that you won't have to worry about boring parts. Of coure,
if you need to know, you're still able to write is the old way". Since we
always want to know, I guess we'd use the C++ compiler (the slow one), with
pure-C code borrowing nothing from the C++ language. That would only be a
pure loss.
> >To be secure, you first have to understand what the code precisely does,
> >not what it should do depending on how the compiler might optimise it.
> >
>
> If optimization changes your code's behaivor, your code is broken.
> People rely on the C++ optimizer for much the same things as C: to
> inline zero- or one- line functions and remove unused code. (There is
> just one exception in C++ where the optimizer _is_ allowed to modify
> behavior, but it is for an obviously correct scenario).
>
> >I'm still thinking that people who have problems understanding what the
> >code does want a level of abstraction between them and the CPU so that
> >the compiler thinks for them.
>
> No, they want not to repeat code and code patterns. It's the same
> motivation that lead to the invention of functions:
>
> - functions allow you to reuse code instead of open-coding common
> sequences
> - constructors/destructors allow you to reuse the do/undo (lock/unlock,
> etc.) pattern without writing it in full every time
> - templates allow you to reuse code even when the data types change
> (like the preprocessor but not limited to linked lists)
> - virtual functions allow you to dispatch a function based on the
> object's type, without writing the boilerplate casting
> - exceptions allow you to do the detect error/undo partial
> modifications/propagate error thing without blowing up the code by a
> factor of five
>
> It's just shorthand: but shorthand allows you to see what the code is
> doing instead of how it handles all the standard problems that occur
> again and again in programming.
I'm still not convinced. Every time I came across C++ code, it was an immense
unreadable crap (indentation, cAmEl case, "funny" operators which you have to
stop at because you need a few seconds of thought before confusing them with
others, etc...). Code that cannot be read at 3am. If you want a good example
of this, download and read p7zip. It does lots of magic^Wimplicit things which
quickly got me lost. I agree, I'm not a C++ developper. But many language are
slightly understandable to non-fellows, and this one looks really nasty.
You told us examples of programs you have written. If this is true, I have
a lot of respect for this, because it still seems impossible to me. I
understand they were closed source, but if you come across open source
kernel code that is readable by newbies, I think many people would be
interested to get a clue.
> The C people are content to stop at functions, but resist _all_ of the
> rest (it's okay to do some template-like magic with typeof, because it's
> still C, right?).
>
> > I still don't see the *current* problem
> >you are trying to fix. Linux is written in C, as many other kernels and
> >it works. Nobody knows what it would become if rewritten in C++. Maybe
> >it will be better, maybe it would not run anymore on embedded systems,
> >maybe it would become fully buggy because nobody except a little bunch
> >of C++ coders would understand it... At least, I'm sure it will not be
> >the smart people who currently work on it.
> >
>
> Maybe it would be smaller, faster, more robust, and have even more
> flexible and fast-paced development.
>
> Perhaps people who developed kernel-level code in _both_ C and C++ would
> be qualified to speculate on that (I have, but apparently I don't have a
> clue).
>
> >Best of all, I'm even sure that people who are trying to push C++ in
> >the kernel would never ever write a line of code once it would be
> >accepted, because they don't seem to know what they're talking about
> >when it applies to kernel code.
> >
>
> Thanks.
Regards,
Willy
On Tue, May 02, 2006 at 01:32:05PM +0300, Avi Kivity wrote:
> Like the recent prevent_tail_call() thing? Granted, C++ is a lot tricker
> than C. Much self-restraint is needed, and even then you can wind up
> where you didn't want to go.
Sigh... You know, once upon a time there was a language called
Algol 68. It had a _lot_ of expressive power and was fairly flexible -
both in type system and in syntax. And it had been a fscking nightmare
for large projects. Not because of lack of modularity - that part had
been all right. The thing that kept killing large projects was different;
in effect, each group ended up with a language subset and developed a
discipline for it. And as soon as they mixed, _especially_ if they were
close, but not quite the same, you had an ever-growing disaster.
It's not easy to quantify; each language has dark corners and
there are more or less odd constructs specific to individual programmers
and to groups. And yes, you certainly can write unreadable crap in any
language. The question is, how many variants of "needed self-restraint"
are widespread, how well do they mix and how easy it is to recognize the
mismatches? It's not a function of language per se, so it doesn't make
sense to compare C and C++ as languages in that respect. However, C++
and C _styles_ can be compared and that's where C++ requires more force
to keep a large project away from becoming a clusterfsck.
Sure, you need make sure that different groups of people use
more or less compatible styles anyway; it's just that with C++ you need
tighter control to get the same result. And for kernel it's a killer.
Martin Mares wrote:
> Hello!
>
>
>> But it executed C++ code within a few cycles of entering the reset
>> vector (no standard BIOS), including but not limited to: programming the
>> memory controller (430MX chipset), servicing interrupts, hardware
>> accelerated 2D graphics (C&T 65550), IDE driver, simple filesystem,
>> simple windowing GUI, network driver (Tulip) etc.
>>
>
> I really don't claim it's impossible to write kernels in C++ -- it's
> clearly possible given that everything you can do in C, you can do in
> C++ as well. But what I argued about is whether kernel programming in C++
> can be easier and more efficient, which is why I wanted you to show
> some examples. Real code speaks better than thousand theories.
>
>
Here are three versions of do_sendfile(). When you read it, please
assume you know light_file_ptr as well as you know f{get,put}_light, etc.
The (IMO) improvement is not dramatic since this is fairly much straight line code.
static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
size_t count, loff_t max)
{
struct file * in_file, * out_file;
struct inode * in_inode, * out_inode;
loff_t pos;
ssize_t retval;
int fput_needed_in, fput_needed_out;
/*
* Get input file, and verify that it is ok..
*/
retval = -EBADF;
in_file = fget_light(in_fd, &fput_needed_in);
if (!in_file)
goto out;
if (!(in_file->f_mode & FMODE_READ))
goto fput_in;
retval = -EINVAL;
in_inode = in_file->f_dentry->d_inode;
if (!in_inode)
goto fput_in;
if (!in_file->f_op || !in_file->f_op->sendfile)
goto fput_in;
retval = -ESPIPE;
if (!ppos)
ppos = &in_file->f_pos;
else
if (!(in_file->f_mode & FMODE_PREAD))
goto fput_in;
retval = rw_verify_area(READ, in_file, ppos, count);
if (retval < 0)
goto fput_in;
count = retval;
retval = security_file_permission (in_file, MAY_READ);
if (retval)
goto fput_in;
/*
* Get output file, and verify that it is ok..
*/
retval = -EBADF;
out_file = fget_light(out_fd, &fput_needed_out);
if (!out_file)
goto fput_in;
if (!(out_file->f_mode & FMODE_WRITE))
goto fput_out;
retval = -EINVAL;
if (!out_file->f_op || !out_file->f_op->sendpage)
goto fput_out;
out_inode = out_file->f_dentry->d_inode;
retval = rw_verify_area(WRITE, out_file, &out_file->f_pos, count);
if (retval < 0)
goto fput_out;
count = retval;
retval = security_file_permission (out_file, MAY_WRITE);
if (retval)
goto fput_out;
if (!max)
max = min(in_inode->i_sb->s_maxbytes, out_inode->i_sb->s_maxbytes);
pos = *ppos;
retval = -EINVAL;
if (unlikely(pos < 0))
goto fput_out;
if (unlikely(pos + count > max)) {
retval = -EOVERFLOW;
if (pos >= max)
goto fput_out;
count = max - pos;
}
retval = in_file->f_op->sendfile(in_file, ppos, count, file_send_actor, out_file);
if (retval > 0) {
current->rchar += retval;
current->wchar += retval;
}
current->syscr++;
current->syscw++;
if (*ppos > max)
retval = -EOVERFLOW;
fput_out:
fput_light(out_file, fput_needed_out);
fput_in:
fput_light(in_file, fput_needed_in);
out:
return retval;
}
static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
size_t count, loff_t max)
{
loff_t pos;
ssize_t retval;
/*
* Get input file, and verify that it is ok..
*/
light_file_ptr in_file(in_fd);
if (!in_file.valid())
return -EBADF;
if (!in_file->readable())
return -EBADF;
retval = -EINVAL;
struct inode *in_inode = in_file->dentry()->inode();
if (!in_inode)
return -EINVAL;
// I'm assuming here that the default sendfile() returns -EINVAL
if (!ppos)
ppos = &in_file->f_pos;
else
if (!(in_file->mode() & FMODE_PREAD))
return -ESPIPE;
retval = rw_verify_area(READ, in_file, ppos, count);
if (retval < 0)
return retval;
count = retval;
retval = security_file_permission (in_file, MAY_READ);
if (retval)
return retval;
/*
* Get output file, and verify that it is ok..
*/
light_file_ptr out_file(out_fd);
if (!out_file)
return -EBADF;
if (!(out_file->writable())
return -EBADF;
// Again, assuming default sendpage returns -EINVAL
struct inode *out_inode = out_file->dentry()->inode();
retval = rw_verify_area(WRITE, out_file, &out_file->f_pos, count);
if (retval < 0)
return retval;
count = retval;
retval = security_file_permission (out_file, MAY_WRITE);
if (retval)
return retval;
if (!max)
max = min(in_inode->i_sb->s_maxbytes, out_inode->i_sb->s_maxbytes);
pos = *ppos;
if (unlikely(pos < 0))
return -EINVAL;
if (unlikely(pos + count > max)) {
if (pos >= max)
return -EOVERFLOW;
count = max - pos;
}
retval = in_file->sendfile(ppos, count, file_send_actor, out_file);
if (retval > 0) {
current->rchar += retval;
current->wchar += retval;
}
current->syscr++;
current->syscw++;
if (*ppos > max)
return -EOVERFLOW;
return retval;
}
// now, with exceptions
static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
size_t count, loff_t max)
{
loff_t pos;
/*
* Get input file, and verify that it is ok..
*/
light_file_ptr in_file(in_fd);
in_file->verify_readable();
struct inode *in_inode = in_file->dentry()->inode();
if (!in_inode)
throw EINVAL;
// I'm assuming here that the default sendfile() returns -EINVAL
if (!ppos)
ppos = &in_file->f_pos;
else
in_file->verify_preadable();
count = in_file->verify_area(READ, ppos, count);
in_file->security_verify_permission(MAY_READ);
/*
* Get output file, and verify that it is ok..
*/
light_file_ptr out_file(out_fd);
out_file->verify_writable();
// Again, assuming default sendpage returns -EINVAL
struct inode *out_inode = out_file->dentry()->inode();
count = out_file->verify_area(WRITE, &out_file->f_pos, count);
out_file->security_verify_permission(MAY_WRITE);
if (!max)
max = min(in_inode->i_sb->s_maxbytes, out_inode->i_sb->s_maxbytes);
pos = *ppos;
if (unlikely(pos < 0))
throw EINVAL;
if (unlikely(pos + count > max)) {
if (pos >= max)
throw EOVERFLOW;
count = max - pos;
}
count = in_file->sendfile(ppos, count, file_send_actor, out_file);
current->rchar += count;
current->wchar += count;
current->syscr++;
current->syscw++;
if (*ppos > max)
throw EOVERFLOW;
return count;
}
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Al Viro wrote:
> On Tue, May 02, 2006 at 01:32:05PM +0300, Avi Kivity wrote:
>
>> Like the recent prevent_tail_call() thing? Granted, C++ is a lot tricker
>> than C. Much self-restraint is needed, and even then you can wind up
>> where you didn't want to go.
>>
>
> Sigh... You know, once upon a time there was a language called
> Algol 68. It had a _lot_ of expressive power and was fairly flexible -
> both in type system and in syntax. And it had been a fscking nightmare
> for large projects. Not because of lack of modularity - that part had
> been all right. The thing that kept killing large projects was different;
> in effect, each group ended up with a language subset and developed a
> discipline for it. And as soon as they mixed, _especially_ if they were
> close, but not quite the same, you had an ever-growing disaster.
>
> It's not easy to quantify; each language has dark corners and
> there are more or less odd constructs specific to individual programmers
> and to groups. And yes, you certainly can write unreadable crap in any
> language. The question is, how many variants of "needed self-restraint"
> are widespread, how well do they mix and how easy it is to recognize the
> mismatches? It's not a function of language per se, so it doesn't make
> sense to compare C and C++ as languages in that respect. However, C++
> and C _styles_ can be compared and that's where C++ requires more force
> to keep a large project away from becoming a clusterfsck.
>
> Sure, you need make sure that different groups of people use
> more or less compatible styles anyway; it's just that with C++ you need
> tighter control to get the same result. And for kernel it's a killer.
>
Hey, I agree 100%, except for the last 6 words :) C++ is the very worst
language I know in terms of badly thought out features, internal
inconsistencies, ways to shoot one's feet off, and general ugliness. It
will require _very_ tight control to avoid parts of the kernel going off
in mutually incompatible directions.
But I think that the control is there; and if C++ is introduced slowly,
one feature at a time, it can be kept sane. And I think there is
definitely a payoff to be won out of a switch.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
On Tue, May 02, 2006 at 04:52:53PM +0300, Avi Kivity wrote:
> static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
> size_t count, loff_t max)
> {
> loff_t pos;
> ssize_t retval;
>
> /*
> * Get input file, and verify that it is ok..
> */
> light_file_ptr in_file(in_fd);
*snerk*
Good luck defining copying and conversion to file * for that puppy.
> if (!in_file.valid())
> return -EBADF;
> if (!in_file->readable())
> return -EBADF;
> retval = -EINVAL;
> struct inode *in_inode = in_file->dentry()->inode();
Lovely. Let's expose all fields as methods?
> if (!in_inode)
> return -EINVAL;
BTW, that can't happen. Applies to the original as well.
> // I'm assuming here that the default sendfile() returns -EINVAL
> if (!ppos)
> ppos = &in_file->f_pos;
> else
> if (!(in_file->mode() & FMODE_PREAD))
> return -ESPIPE;
As opposed to ->readable() for checking FMODE_READ?
> light_file_ptr out_file(out_fd);
> if (!out_file)
> return -EBADF;
?
> if (!max)
> max = min(in_inode->i_sb->s_maxbytes,
> out_inode->i_sb->s_maxbytes);
While we are at it, that's the only place where in_inode and out_inode
are used. Also... how does one remember which of ->dentry, ->inode
and ->i_sb are methods and which are public fields?
> // now, with exceptions
> static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
> size_t count, loff_t max)
> {
> loff_t pos;
>
> /*
> * Get input file, and verify that it is ok..
> */
> light_file_ptr in_file(in_fd);
> in_file->verify_readable();
That assumes that error value returned in that case is the same everywhere.
It isn't.
On Tue, May 02, 2006 at 05:02:55PM +0300, Avi Kivity wrote:
> Hey, I agree 100%, except for the last 6 words :) C++ is the very worst
> language I know in terms of badly thought out features, internal
> inconsistencies, ways to shoot one's feet off, and general ugliness. It
> will require _very_ tight control to avoid parts of the kernel going off
> in mutually incompatible directions.
>
> But I think that the control is there; and if C++ is introduced slowly,
> one feature at a time, it can be kept sane. And I think there is
> definitely a payoff to be won out of a switch.
You are far too optimistic. In the best case it'll end up with higher
artificial entry barrier for contributors. In the worst (and much more
realistic) the crap will leak all over the tree in addition to the
already present classes of bugs.
"Everyone has his pet subset/extension" is a killer for anything that isn't
done by 5-6 people, or, at least reviewed by 5-6 people who really can
read through _all_ incoming code. For something like Linux kernel...
forget it. It's far outside of the area where that would work.
Willy Tarreau wrote:
> It would be the exact same problem I discribed above. If nobody understand
> what it does, or if people need some deep thoughts to guess its behaviour,
> the result will be negative. Currently, everyone can read the kernel code.
> I was a terrible looser in C 11 years ago when I first looked at it, but I
> did not need specific knowledge of implicit things that the kernel does to
> understand what the code did, and to propose patches.
>
>
11 years ago the kernel was _way_ simpler.
These days, you need to know that you can't call schedule() (or any
function which may sleep -- why is that not implicit) within spin locks.
You need to understand per-cpu variables, initcalls, rcu, memory
barriers, reference counting, and lots of other "infrastructure" details
that are engraved into your brain but not accessible to a newbie. Most
of these (except perhaps reference counting) cannot be understood from
looking at the code.
C++ allows you to add even more infrastructure, by providing support for
patterns that can't be done in C. So yes, that adds more to the
vocabulary you have to learn by heart but once you're over that the code
is _more_ readable, more maintainable.
> Right now, lots of newbies review the code and find bugs. If you write it
> in a language such as above, it will quickly end up like the BSD kernels,
> with a small very competent core team and lots of newbies around them.
>
>
C++ knowledge is now fairly widespread, I believe.
> I don't know, what I can say is that they were people who regularly discuss
> the kernel semantics on other threads. That's what matters. Those people are
> good at finding kernel bugs and proposing interesting algorithms, it's a shame
> that they had to discuss langage semantics on 5 lines of code !
>
I'm sorry to have wasted their time. But surely you understand these
things once and then move on? Like when someone proposes a new function
to replace repeated code, there's argument about it's name, coding
style, and hopefully about the content. The people replace the
open-coded sequence with the new function.
It's the same here at a slightly higher level. You argue what s spinlock
guard should look like, whether it's optimal or not, and then you can
use it where applicable and avoid those gotos or code duplications for
error cases.
>
> I seems like pro-C++ people constantly tell the other ones "look, C++ can
> code for you so that you won't have to worry about boring parts. Of coure,
> if you need to know, you're still able to write is the old way". Since we
> always want to know, I guess we'd use the C++ compiler (the slow one), with
> pure-C code borrowing nothing from the C++ language. That would only be a
> pure loss.
>
It's not "if you need to know", it's "if you want to break the rules".
For example, the spinlock guard thingie allows you to say, "please
spin_unlock this spinlock for me, whichever way I return from this
function". There is no knowledge taken away, you know exactly what it
will do and when; only the code is made clearer.
But if you want the function to lock but not unlock, or do the
atomic_dec_and_lock() thing, or do lock reordering to avoid ABBA
deadlocks, or whatever, the primitives are still there.
These constructs allow you to make simple things simple (and automatic)
but don't take away the complex things.
(example, for context:
{
spinlock_guard guard(some_lock);
if (x)
return -ESOMETHING;
blah();
return 0;
}
there is nothing implicit once you know spinlock_guard)
>> No, they want not to repeat code and code patterns. It's the same
>> motivation that lead to the invention of functions:
>>
>> - functions allow you to reuse code instead of open-coding common
>> sequences
>> - constructors/destructors allow you to reuse the do/undo (lock/unlock,
>> etc.) pattern without writing it in full every time
>> - templates allow you to reuse code even when the data types change
>> (like the preprocessor but not limited to linked lists)
>> - virtual functions allow you to dispatch a function based on the
>> object's type, without writing the boilerplate casting
>> - exceptions allow you to do the detect error/undo partial
>> modifications/propagate error thing without blowing up the code by a
>> factor of five
>>
>> It's just shorthand: but shorthand allows you to see what the code is
>> doing instead of how it handles all the standard problems that occur
>> again and again in programming.
>>
>
> I'm still not convinced. Every time I came across C++ code, it was an immense
> unreadable crap (indentation, cAmEl case, "funny" operators which you have to
> stop at because you need a few seconds of thought before confusing them with
> others, etc...). Code that cannot be read at 3am. If you want a good example
> of this, download and read p7zip. It does lots of magic^Wimplicit things which
> quickly got me lost. I agree, I'm not a C++ developper. But many language are
> slightly understandable to non-fellows, and this one looks really nasty.
>
I agree - I've seen it done many times. C++ has the potential for a lot
of nastiness, and what is more, it seems to bring out the worst in
people. We would definitely need an Al Viro for C++ if the kernel were
to be ported.
I've yet to find a language that combines low level access, efficiency,
cleanliness, and expressive power.
> You told us examples of programs you have written. If this is true, I have
> a lot of respect for this, because it still seems impossible to me. I
> understand they were closed source, but if you come across open source
> kernel code that is readable by newbies, I think many people would be
> interested to get a clue.
>
I posted an example of do_sendfile() as it might be incrementally converted.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Al Viro wrote:
> On Tue, May 02, 2006 at 04:52:53PM +0300, Avi Kivity wrote:
>
>> static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
>> size_t count, loff_t max)
>> {
>> loff_t pos;
>> ssize_t retval;
>>
>> /*
>> * Get input file, and verify that it is ok..
>> */
>> light_file_ptr in_file(in_fd);
>>
>
> *snerk*
> Good luck defining copying and conversion to file * for that puppy.
>
>
class light_file_ptr {
public:
explicit light_file_ptr(int fd)
{
_file = fget_light(fd, &_fput_needed);
}
~light_file_ptr()
{
fput_light(_file, _fput_needed);
}
bool valid() const
{
return _file != 0;
}
struct file *operator->() // allowed for libs :)
{
return _file;
}
private:
struct file *_file;
int _fput_needed;
};
>> struct inode *in_inode = in_file->dentry()->inode();
>>
>
> Lovely. Let's expose all fields as methods?
>
>
If you like. I won't insist.
>> if (!in_inode)
>> return -EINVAL;
>>
>
> BTW, that can't happen. Applies to the original as well.
>
>
I believe this bug is more visible when there is less code in the function.
>> // I'm assuming here that the default sendfile() returns -EINVAL
>> if (!ppos)
>> ppos = &in_file->f_pos;
>> else
>> if (!(in_file->mode() & FMODE_PREAD))
>> return -ESPIPE;
>>
>
> As opposed to ->readable() for checking FMODE_READ?
>
>
Forgot, sorry. I'll redo the patch.
>> light_file_ptr out_file(out_fd);
>> if (!out_file)
>> return -EBADF;
>>
>
> ?
>
Sorry, !out_file.valid().
>> if (!max)
>> max = min(in_inode->i_sb->s_maxbytes,
>> out_inode->i_sb->s_maxbytes);
>>
>
> While we are at it, that's the only place where in_inode and out_inode
> are used. Also... how does one remember which of ->dentry, ->inode
> and ->i_sb are methods and which are public fields?
>
I usually make all publics either methods (a class) or fields (a struct).
>> // now, with exceptions
>> static ssize_t do_sendfile(int out_fd, int in_fd, loff_t *ppos,
>> size_t count, loff_t max)
>> {
>> loff_t pos;
>>
>> /*
>> * Get input file, and verify that it is ok..
>> */
>> light_file_ptr in_file(in_fd);
>> in_file->verify_readable();
>>
>
> That assumes that error value returned in that case is the same everywhere.
> It isn't.
>
Okay, in_file->verify_readable(EBADF); Yuck.
Thanks for the review.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Al Viro wrote:
> On Tue, May 02, 2006 at 05:02:55PM +0300, Avi Kivity wrote:
>
>> Hey, I agree 100%, except for the last 6 words :) C++ is the very worst
>> language I know in terms of badly thought out features, internal
>> inconsistencies, ways to shoot one's feet off, and general ugliness. It
>> will require _very_ tight control to avoid parts of the kernel going off
>> in mutually incompatible directions.
>>
>> But I think that the control is there; and if C++ is introduced slowly,
>> one feature at a time, it can be kept sane. And I think there is
>> definitely a payoff to be won out of a switch.
>>
>
> You are far too optimistic. In the best case it'll end up with higher
> artificial entry barrier for contributors. In the worst (and much more
> realistic) the crap will leak all over the tree in addition to the
> already present classes of bugs.
>
> "Everyone has his pet subset/extension" is a killer for anything that isn't
> done by 5-6 people, or, at least reviewed by 5-6 people who really can
> read through _all_ incoming code. For something like Linux kernel...
> forget it. It's far outside of the area where that would work.
>
If it has 'template' or 'operator' in it, make sure some sane people
look at it.
IIRC, when the gcc people discussed this, they raised the possibility of
adding compiler switches to allow subsetting the language. But there's
nothing real out there that I know of.
BTW, C++ could take over some of sparse's function:
int foo(user_ptr<struct fugly_ioctl_arg> arg)
{
return bar(arg->field); // won't compile
struct fugly_ioctl_arg s;
if (arg->copy_from_user(&s))
return -EFAULT;
return bar(&s); // yes!
}
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
On Tue, May 02, 2006 at 06:04:23PM +0300, Avi Kivity wrote:
> BTW, C++ could take over some of sparse's function:
And the point of that would be? sparse is _fast_ and easy to modify;
g++ is neither.
Al Viro wrote:
> On Tue, May 02, 2006 at 06:04:23PM +0300, Avi Kivity wrote:
>
>> BTW, C++ could take over some of sparse's function:
>>
>
> And the point of that would be? sparse is _fast_ and easy to modify;
> g++ is neither.
>
I wasn't talking about modifying gcc to do the checking, rather using
language features so that the checks would happen during a regular
compile. That would mean we weren't dependent on somebody running sparse
with a configuration that triggers the bug, but those few who compile
the code before submitting the patch would get it automatically checked.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
On May 2, 2006, at 11:15:25, Al Viro wrote:
> On Tue, May 02, 2006 at 06:04:23PM +0300, Avi Kivity wrote:
>> BTW, C++ could take over some of sparse's function:
>
> And the point of that would be? sparse is _fast_ and easy to
> modify; g++ is neither.
For example; you can run _both_ a full sparse with all checks _and_
"gcc -Wall -Wextra -Werror -g -O2" in less time than it takes to
_just_ run "g++ -O0" on a nearly identical file. If you have some
new typechecking to do, figure out how to add it to sparse without
cluttering up the syntax, don't convert the thing to C++, slow down
the compile, etc.
Cheers,
Kyle Moffett
On May 2, 2006, at 11:19:35, Avi Kivity wrote:
> I wasn't talking about modifying gcc to do the checking, rather
> using language features so that the checks would happen during a
> regular compile. That would mean we weren't dependent on somebody
> running sparse with a configuration that triggers the bug, but
> those few who compile the code before submitting the patch would
> get it automatically checked.
There's a reason that we tell all patch submitters to run "make C=1"
on several configs before submitting patches. Besides, you seem to
have a vast misunderstanding of LK development processes; we frown
heavily on people who don't "compile their code before submitting the
patch", it's not a rare thing at all.
Cheers,
Kyle Moffett
On Tue, May 02, 2006 at 06:19:35PM +0300, Avi Kivity wrote:
> Al Viro wrote:
> >On Tue, May 02, 2006 at 06:04:23PM +0300, Avi Kivity wrote:
> >
> >>BTW, C++ could take over some of sparse's function:
> >>
> >
> >And the point of that would be? sparse is _fast_ and easy to modify;
> >g++ is neither.
> >
>
> I wasn't talking about modifying gcc to do the checking, rather using
> language features so that the checks would happen during a regular
> compile. That would mean we weren't dependent on somebody running sparse
> with a configuration that triggers the bug, but those few who compile
> the code before submitting the patch would get it automatically checked.
g++ won't cover all checks sparse is capable of, so you still want to
run the latter over new code anyway (== pass C=1 to make). IOW, type
safety from C++ isn't particulary good argument.
Kyle Moffett wrote:
> On May 2, 2006, at 11:19:35, Avi Kivity wrote:
>> I wasn't talking about modifying gcc to do the checking, rather using
>> language features so that the checks would happen during a regular
>> compile. That would mean we weren't dependent on somebody running
>> sparse with a configuration that triggers the bug, but those few who
>> compile the code before submitting the patch would get it
>> automatically checked.
>
> There's a reason that we tell all patch submitters to run "make C=1"
> on several configs before submitting patches. Besides, you seem to
> have a vast misunderstanding of LK development processes; we frown
> heavily on people who don't "compile their code before submitting the
> patch", it's not a rare thing at all.
>
That was tongue in cheek. Perhaps I should have added a smiley.
I follow lkml quite closely, even though I lack the time to contribute
in the areas that interest me.
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Al Viro wrote:
> g++ won't cover all checks sparse is capable of, so you still want to
> run the latter over new code anyway (== pass C=1 to make). IOW, type
> safety from C++ isn't particulary good argument.
>
I'm pretty sure that sparse can't validate all the casting from void
pointers and from "base classes". Nor can it find bugs in data
structures which are open-coded instead of template libraries. Do
correct me if I'm wrong.
I'm not familiar with sparse's capabilities beyond __user, locking
depth, and the like (pulling it now). Can you point me to any information?
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
On Tue, 2006-05-02 at 14:26 +0300, Avi Kivity wrote:
>
> There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/ and
> http://ecos.sourceware.org/, but I haven't looked at them, so I can't
> say whether I consider them nice or not.
You keep saying the eCos is written in C++, I see no evidence of that.
>
--
Brian Beattie
Firmware Engineer
APCON, Inc.
[email protected]
Brian Beattie wrote:
> On Tue, 2006-05-02 at 14:26 +0300, Avi Kivity wrote:
>
>
>> There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/ and
>> http://ecos.sourceware.org/, but I haven't looked at them, so I can't
>> say whether I consider them nice or not.
>>
>
> You keep saying the eCos is written in C++, I see no evidence of that.
>
Someone posted in on this thread; I have no prior experience with it.
However, this may satisfy you:
http://ecos.sourceware.org/cgi-bin/cvsweb.cgi/ecos/packages/kernel/current/src/sched/sched.cxx?rev=1.18&content-type=text/x-cvsweb-markup&cvsroot=ecos
--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
David Woodhouse <[email protected]> writes:
> On Tue, 2006-05-02 at 14:26 +0300, Avi Kivity wrote:
> > There are C++ embedded kernels in http://www.zipworld.com.au/~akpm/
> > and http://ecos.sourceware.org/, but I haven't looked at them, so I
> > can't say whether I consider them nice or not.
>
> eCos is nice enough -- because it's mostly C :)
And those parts that are C++ (from a 2 year old eCos dist) won't
compile with a modern g++. I tried to compile RedBoot on a Fedora
Core 5 system and it was a very painful experience, old deprecated C++
code was breaking all over the place. It may just have been the the
eCos configuration tool (which I belive is written in C++) that failed
to compile, but anyway, I had to use an older version of g++ and some
flag to make the old broken C++ code generate warnings, not errors.
/Christer
--
"Just how much can I get away with and still go to heaven?"
Freelance consultant specializing in device driver programming for Linux
Christer Weinigel <[email protected]> http://www.weinigel.se
El Tue, 02 May 2006 17:41:15 +0300,
Avi Kivity <[email protected]> escribi?:
> I've yet to find a language that combines low level access, efficiency,
> cleanliness, and expressive power.
In case someone is interested, the freebsd people is trying to design
something that tries to achieve such things for the particular case
of the kernel without falling in the dangers of c++
"...a dialect of the C language that simplifies the task of writing kernel
code. It should include language extensions that make it possible to write
kernel code more cleanly and with less bugs. An example of this would have
language support for linked lists, to obviate the need for messy MACROs"
http://wikitest.freebsd.org/moin.cgi/K
>In the bad old days, performance was the number one priority because
> computers were slow and resources were scarce
And that is still *very much* the case on huge numbers of systems
running Linux --> Embedded systems <---
Cheers
> >In the bad old days, performance was the number one priority because
> >computers were slow and resources were scarce
> And that is still *very much* the case on huge numbers of systems
> running Linux --> Embedded systems <---
Yes and no. Any embedded system likely to run a full Linux kernel today has
at least ten times the performance and resources of the computers from the
bad old days.
The majority of embedded systems that run very low-end hardware perform
tasks where performance is not particularly important anyway (microwave
oven, elevator). Those that perform more sophisticated tasks generally have
correspondingly more impressive hardware.
In my experience, reliability is much more important than performance in
most embedded systems. If C++ results in cleaner and better organized code,
for some particular piece of code, it would be madness to choose C out of
fear that C++ will perform worse. (Of course, that's a big 'if'.)
In fact, in all the years I developed embedded systems (from industrial
knitting machine controllers to medical devices) I can't remember a single
project where performance was the number one priority, or even a
particularly high one. Feature set, reliability, development cost and time,
ease of use, and the like were typically way ahead of performance. Perhaps
for a router or something like that it would be different. But I do not
believe that embedded systems in general are a class where performance is
more important than elsewhere.
There are small embedded operating systems that are written primarily in
C++.
DS