1999-07-31 22:35:15

by Joe Pranevich

[permalink] [raw]
Subject: [offtopic] Wonderful World of Linux 2.4


I have recently written an updated version of my Linux 2.2 article which
lists the features and whatnot of Linux 2.4. (Or rather, Linux 2.3. But
just try and explain that to many journalists...) I would very much
appreciate your input, as deveolopers, to the writing process. I'm not
as in the loop as I used to be, unfortunately. If anyone can provide any
input, I would appreciate it muchly.

Now, the last time I did this, I got flames when I sent the file to the
list and flames when I didn't. It's a lose-lose situation for me, but I
did nicely preface this whole thing with "offtopic" so I hope that too
many people whp pay by the byte don't try and download it.

I'm particularly lacking in input from the sound, video4linux, scsi, and
network side of things. But if you know of any major feature that I left
out, don't hesitate to let me know. Just be nice about it. :) Also, if
you would respond just to me or trim off the real article in any
replaies, it would save a lot of people bandwidth.

Thanks so much for any input that you might have,



Wonderful World of Linux 2.4 (7/31/99) "Making it right II"

A long time ago, in a galaxy not to far from this one, I wrote an
It wasn't much, just a laundry list of the new and improved features of
Linux 2.2. That was a long time ago and the newness of Linux 2.2 is
beginning to wear off; it's even lost that new kernel smell. And so with
anxious eyes, we set our eyes towards the future of Linux: Linux 2.4.
is it due? When it is ready. (But between you and me, we're shooting on
sometime in late fall.)

Before we move into the meat and bones of the situation, I always like
step back for a moment and say a few things. Linux 2.2 marked a major
milestone in Linux's evolution: it was the first kernel release which
caught the eye of the all-seeing media. No longer would just obscure
computer publications trumpet the coming of the open source revolution,
mainstream publications as well. With the attention has come the
FUD (Fear, Uncertainty, and Doubt) that we prayed would never come, but
of this media molasses we have emerged as a stronger community. While
of Linux's weak points were trumpeted across the sky and many members of
the Linux community despaired, there were still the legions of kernel
hackers that worked day and night to take those weaknesses, acknowledge
them, and work to fix them. I think they've done wonderful job.

As a side effect of the increased media attention, many members of the
media chose to use my original article as a guide, a reference to the
changes and features in Linux 2.2 in the absence of any official press
release. To be honest, I was flattered. I do ask however that you cite
source whenever possible (both myself and the community who made this
document possible) and if you require any clarifications, please do not
hesitate to email me at the address below with your questions. If you
like to reprint or translate this article in full or in part in any
please email me at the address below so that I can be aware of your work
before you do so. I will make a list of translations that I am aware of

Now, without further ado, Linux 2.4. This draft of the WWoL2.4 is
the "making it right edition" and will be definitely followed by a
of later drafts, at least 3 of which I will term "final". We all have
different notions of what changes are important, if I don't mention a
particular update that you feel is important, by all means please let me

Joe - [email protected]

< Many thanks to linuxtoday for providing me with the spiffy email addy.

The Heart - Linux Internals

Linux 2.2 was a great improvement over Linux 2.0. It supported many new
file systems, it supported a completely rethought caching system for
names, and it was much more scalable than Linux 2.0. Linux 2.4 will
on the great advancements provided under Linux 2.2 to become an even
platform for desktop, server, and embedded tasks. However, it is the
of the Linux kernel developers to get Linux 2.4 into the hands of the
users much more quickly than Linux 2.2 did. To meet this goal, Linux 2.4
will understandably not be as different from Linux 2.2 as Linux 2.2 was
from Linux 2.0. I think you will agree however that the advancements in
Linux 2.4 will be just as noteworthy as previous versions. (Or else I
wouldn't need to be writing this!)

What, at the core, is Linux? Linux is much more than just a collection
assorted device drivers, as any operating system must certainly be. It's
what binds these drivers together into a cohesive unit that matters.
the scheduler, the resource allocator, the virtual filesystem layer,
management, and so many other unsung features that are the real heroes
the Linux world. These are the portions of the Linux operating system
really define what is Linux because on every platform that Linux has
ported to from i386 (Intel-compatible PC), to ARM (embedded devices), to
Sparc64 (high-end servers) this code is the same. In many ways, the
of Linux 2.4 is different than Linux 2.2's and most of the subsystems
I just listed have been changed in one way or another.

Linux 2.2 and earlier Linux's included a base resource management system
which was used rather bluntly to allocate and keep track of IO ports and
IRQ lines and the other limited niceties of computer architectures.
Unfortunately it was deficient in a number of important ways which are
crucial to modern desktop operating systems: it lacked the ability to
cleanly handle nested resources (where we handle one device's memory as
subset of another, for example a PCI bus), it included a hardcoded list
resource types, and other deficiencies. In the interests of better
management, the old code was scrapped in favor of new code which
the issues that I just described. While it has not been put to this use
yet, it has been suggested that the new resource allocator code may put
one step closer to real PnP support in the kernel. (For a more detailed
discussion of PnP, please check the section on buses, below.)

The Linux model of network sockets is one which is standard across most
UNIX variants. Unfortunately however, the standard does have some
deficiencies but these deficiencies can be corrected without breaking
standard altogether. Under Linux 2.2 and previous versions, if you have
number of processes all waiting on an event from a network socket (a web
server, for instance), they will all be woken up when activity if
So, for every web page request received, Linux would wake up a number of
processes which would each try and get at the request. As it does not
sense for multiple processes to serve the same request, only one will
to the data; the remainder will notice that it doesn't have anything to
process and fall back asleep. Linux is quite efficient at making this
happen as quickly as possible, however it is still very inefficient...
there is a better way. Linux 2.4 includes changes which implement "wake
one" under Linux which will allow us to completely remove the "stampede
effect". In short, "wake one" does exactly as its name indicates: wakes
only one process in the case of activity. This will allow applications
as Apache to be even more efficient and make Linux an even better choice
a web server.

Linux includes other changes that will make it a better choice for
higher-end systems in particular. Linux 2.2 scaled to multiple
much better than did Linux 2.0 (the first version of Linux to support
multiple-processors at all.) This was primarily achieved by removing a
global kernel lock and replacing it with finer granularity locks which
would allow more work to be done on multiple processors at once. Linux
takes this advancement even further as many subsystems' locks have been
replaced or rethought to allow finer control. In fact, several key
subsystems including TCP/IP (IPv4) and the virtual filesystem layer
have been completely rewritten with the intent to provide more
8-way scalability under Linux is now a reality! (Linux can still scale
as many as 64 processors on some high level machines, however certain
kernel activities would have to have complete control of the machine.
these changes allow us to do is spend considerably less time in routines
that require complete control and let the rest of the system go faster.)

The virtual filesystem layer, as previously mentioned, has been
for multi-processing, but that is not the only change in that layer.
2.2 featured a number of wonderful changes to the filesystem layer to
support better caching, however it still used a split buffer system that
kept the reading and writing buffer separate. Linux 2.4 brings this wall
down with a complete rewrite of the page caching layer leading to a
subsequent reduction in overall memory needs and removing a number of
where Linux had to jump through hoops to keep the two caches in synch.
Overall, this is a more efficient system allowing for many operations,
including synching, to happen much more rapidly than could be managed
previously. There were other changes to made to this layer in the
including the removal of race conditions (when multiple processes can
manage to interfere with each other because they "race" for access to
variables), the speeding up of large writes to multiple disks at once,
the dynamic allocation of file descriptors (which allow more files to be
open at once, in some cases.) These changes led to some necessary evils
(described in the filesystem section) but overall I feel that this is a
total win for Linux.

One common problem with Linux 2.2 that interfered with high-end (Intel?)
machines was its process limitations. Linux 2.2 only allowed you to have
1024 processes or threads running at once. With high-end systems with
thousands of users, this could become a problem very quickly. Linux 2.4
gotten rid of this relic and implemented a scalable limit which can be
configured at run time. The number of simultaneous processes is limited
course to the amount of RAM you have in the system. On high-end servers
with as little as half a gigabyte of RAM installed, it is easily
to support as many as 16 thousand processes at once. Other users have
reported being able to run many more than that on their specific
This was one of the major bottlenecks that kept Linux out of the

In terms of memory consumption, Linux 2.4 will require approximately the
same amount of memory as does Linux 2.2. Some subsystems have been added
expanded and some have been streamlined. Some obsolete code has been
removed. There are even certain cases where some systems will require
memory than Linux 2.2!

Processors and Ports

While infrastructure is the heart of the Linux operating system, it is
parts of the operating system that are specific to the individual
of Linux that are most obvious to the end users. These "arms and legs"
the Linux operating system include all of the architecture dependent and
independent driver code which controls the processors, disk drives,
and everything else that provides real access to the computer. For the
purposes of this article, I will be concentrating on i386 Linux which is
the Linux that I use most often at home. That is not to say that there
not been great advancements in other areas, however they are beyond my
direct experience. If there is enough demand, and someone is interested
helping, I may be willing to write appendixes which describe some of the
big improvements in other areas. There have been no new ports entered
mainstream Linux since Linux 2.2 although this may change.

On Intel-compatible hardware, Linux 2.4 includes the same excellent
for processors as did Linux 2.2. This includes optimizations for 386,
586 (Pentium), and 686 (Pentium Pro / Pentium II / Pentium III)
processors, as well as "compatible" counterparts such as those made by
and Cyrix. Additionally, Linux 2.4 will include additional support for
hardware present with modern chips. While Linux 2.2 includes support for
Intel's Memory Type Range Registers (MTRRs) to raise performance to some
kinds of high-bandwidth devices, Linux 2.4 has taken this support even
further by supporting variants common to the compatible chips. This
includes the double MTRRs present with AMD K7 processors and MCR variant
preferred by Cyrix. Linux 2.2 also included support for the IO-APIC
allowed interrupts to be spread across multiple processors in a
multi-processing system. Linux 2.4 will, as expected, take this to the
level and support some high-end systems which actually contain multiple
IO-APIC controllers; this will increase performance on these machines.

To my knowledge, the only multi-processor systems which we still do not
completely support are some very old 486 ones that mix 486DX and 486SX
chips in the same system. This is mainly because the SX chips did not
contain math coprocessors and there is some difficulty in making sure
applications that need floating point math get to work on the right one.
you can imagine, there isn't much call for this feature.

In terms of what types of programs it can run, Linux 2.4 is very much
same as Linux 2.2. The only notable exception to this rule is that
has been removed to run Java applications directly, it has been replaced
a more modular system which allows for arbitrary associations, not
Windows's own associations. This change is not new to Linux 2.4, however
you will have to configure this before your Java applications will work.
The removal of this feature will save memory on some configurations,
not much.

Buses - ISA, PCI, USB, MCA, etc.

Processes however are just a small part of the guts of a computer.
important to its operation is its bus architecture because it is this
component of the system that is responsible for (or irresponsible
as the case may be) internal devices. Linux 2.4 has not yet touched much
the internal workings of many of the supported busses, including (E)ISA,
VLB, PCI, and MCA except to begin to work them into the new resource
management subsystem. Plug-and-Play, the somewhat misguided attempt to
support device configuration and detection on the ISA bus, is still not
supported at the kernel level. However, PnP devices can still be used
somewhat easily by configuring them from userspace with the isapnp
utilities package available with nearly every distribution. Once
configured, PnP devices can be loaded using modules just like any other

There is exciting news from this front however. Universal Serial Bus, a
bus type just now coming into prominence for devices such as keyboards,
mice, sound systems, and scanners is now supported in the Linux kernel.
the time of this writing, the support is not 100% and many individual
common USB devices are not supported or not completely supported. I
be confident however that the number of devices which are supported will
only rise over time, just as we observed a similar rise in the number of
framebuffer devices that are now supported. (The framebuffer was a new
feature to Linux 2.2, see below.) Currently, keyboards and mice are
mostly as you would expect. Support for sound systems is coming along
rapidly. Other devices, such as modems and network cards, already have
preliminary support however their drivers are not complete.

In addition to USB, I2O device (Intelligent Input/Output) support, an
extension of PCI, has been added in Linux 2.4. In theory, this will
for more operating system independent devices and drivers to exist. Many
I2O devices are already functioning and more will be added before Linux

PCMCIA support is still not provided in the kernel as shipped by Linus.
However the code is generally available and provided built-in to most
distributions. This is done by mutual agreement.

Block Devices - Disk Drives, RAID Controllers, etc.

Now, just having a bus and a processor does not' make a computer any
than a skeleton and heart would make any other organism. To spread this
analogy even further into the realm of not actually being applicable,
need a brain. No, not to think with, that's handled in our heart.
for a moment that we still subscribe to the classical theory of
Instead, we need a brain to hold our data and actually do meaningful

Linux 2.2 already supported the most common types of storage media
including RAID controllers, IDE and SCSI disks, and many other
advancements. Linux 2.4 has built on this by providing more of the same:
more drivers for more cards and more stability for the ones that already
exist. Linux 2.4 has also made it easier for larger systems at this
by doubling the number of IDE controllers supported internally in the
kernel to 8. If you have one of these beasts, you will now be able to
all of its interfaces. There have also been some kernel changes to allow
for better support of IDE CD changers although support for having
CDs in the changer mounted at once is not provided. (For obvious
the latency on a multi-user machine would be terrible!) And finally,
2.4 is much better at identifying buggy or special IDE hardware and
enabling workarounds or speed enhancements.

SCSI has received less advancements in Linux 2.4, except for the
down of a number of new SCSI controller drivers.

It's long been traditional for Linux developers to take ideas and
that "make sense" from one operating system and apply it to Linux.
however do these ideas emerge onto the Linux scene without an incredible
amount of thought and digestion, usually resulting in a better final
One particular idea was adapted from mainstream UNIX: raw device i/o,
ability to access a low-level block device directly and without any
intervening cache layer. Few doubt that this idea had its merits,
the traditional implementation of this idea (the creation of "raw"
corresponding to each and every block device) was unacceptable to Linux
developers. Eventually, a solution was developed that even Linus could
agree on: the ability to associate a raw character device with any
arbitrary block device without having to preallocate many unnecessary
device nodes. While applications to this system are not obvious to the
casual desktop user, this system allows greater control to applications
that want it, including and especially database applications which are
smart enough to do their own disk management..


Having a block device is only really useful if you can have some data on
it. But having data is useless unless it is in some prescribed format
allows for convenient storage and retrieval. By this point, we have
completely broken my analogy. Oh well. It should also be mentioned that
these filesystems (as well as nearly everything else) will work with all
versions of Linux and are not only applicable to i386 Linux.

Linux 2.4 includes all of the new filesystems present in Linux 2.2.
filesystems include FAT (for MSDOS), NTFS (for Windows NT/2000), VFAT
FAT32 (for Windows 9x), HFS (for Macintoshes), and many, many others.
of these filesystems have been rewritten to some extent to support the
page caching system and will be more efficient because of it. On the
side however, binary-only filesystem modules designed for Linux 2.2 will
not work with Linux 2.4. (Unlike some software firms, Linux does not
generally provide for back-compatibility at the module level. Generally,
open source modules can adapt quickly enough and binary module providers
are expected to do the same or release the code.)

Some users will however notice major improvements to allow for better
compatibility with other systems. OS/2 users will finally be able to
read and write to their disks under Linux. (This change is a long time
coming.) Linux 2.4 will also include a couple of improvements designed
make it interoperate better with other UNIX-like operating systems. Key
this is Linux 2.4's upcoming support for the IRIX efs filesystem and the
IRIX disklabel (partition table) format. Also, support for NextStep has
also improved as the UFS driver now supports its CDROMs.

Users who mount Windows shared drives via SMB (Server Message Block
protocol) will be pleased that there will no longer be a compile time
option for enabling workarounds for (released broken) Win9x systems.
Instead, Linux will be able to detect what kind of system it is
to and enable bug fixes as needed. This will make Linux a considerably
better option for heterogeneous networks.

Of special importance to many Linux users is Linux's ability to mount
shared drives of UNIX operating systems. Linux 2.4 includes for the
time the ability to access NFS shares which conform to version 3 of the
protocol. NFS version 3 includes many advantages over previous versions
it has been one of Linux's most often requested features.

There are still some pieces of support that is currently lacking in
2.4. There is no support for journalizing filesystems, for instance. Due
the relatively low fsck times and the ease of data recovery journalizing
filesystems support, this is considered by many to be an entrance
requirement to the enterprise. It is hoped that this and other "missing"
features will be completed before 2.4 is ready for release.

Input Devices - Keyboards and Mice

Now, storage is important and the bus is important, but if you can't do
anything with the machine yourself then it really is not any use to you.
desktop users, the keyboard and mouse are the input devices of choice.
is fitting then that Linux 2.4 includes some improvements to this arena.

The biggest news on this front is that Linux 2.4 will support for the
time keyboards and mice attached to the Universal Serial Bus. When
in, these input device will behave just as if they were "normal"
and mice. Additionally, Linux will now work on more systems, including
broken (or specially embedded) ones where the keyboard is not
pre-initialized by the BIOS. Also, better support is provided for
without keyboards in some cases. (Mostly for buggy machines that don't
handle the lack of a keyboard as well as one would like.)

There are some places where some people feel that Linux 2.4 could
of course. With the addition of USB we have the chance to have multiple
keyboards and mice attached to the same bus. Linux 2.4 however does not
have internal multi-heading of these devices; you cannot assign one
keyboard and one mouse to one terminal and another set to a different
terminal. Support for this is provided in the GGI project, however this
project's code has not yet (and may never be) synched into the
kernel. (It is however a good place to check if you need this

Video: Console, Frame-buffers

Now, you have ways of getting data into your Linux box, but that's
of no use unless you can get data out. That's pretty pointless! Linux
included some of the most revolutionary features to hit mainstream
console support since the introduction of virtual consoles. (which was
before v1.0) Primarily, these changes were in the modularity of the
subsystem and the inclusion of frame-buffer drivers into the kernel,
with a
virtual frame-buffer console. (This support had existed previously in
Linuxes, but was never made portable enough to be of any use to i386
users.) In theory, this may someday allow for every Linux user to use a
single, smaller, X server and keep video support in the kernel. In the
however the option to use a frame-buffer is just that, an option, and
many people may not agree in the extent of its usage, no one can doubt
important role that it plays in systems which do not have a built-in

Linux 2.4 expands on frame-buffer support in many ways. Many new drivers
have been added for video cards including better support for VGA
and even really old (16 color) VGA. As promised, we are seeing Linux
gradually support a wider and wider range of hardware in this capacity.

Linux 2.4 also includes support, useful primarily for servers, to
console output (including kernel and debug messages) to a device on a
parallel port (a printer, for example.) Like many features, this is just
option however it has already proven useful in a number of server and
debugging configurations.

Ports - Serial, Parallel, Infra-Red, etc.

Linux has always had excellent support for ports, such as serial ports,
that allows one to get around many of the drawbacks currently in the
keyboard and video layers. Linux really does have excellent support for
consoles, even system consoles with boot messages, on the serial ports.
is fitting that Linux 2.4 has improved support for many of these port
that are in common use today.

Serial support for Linux 2.4 has not changed much and many of the same
limitations still apply. (In particular, setting module options is
generally done with an external utility rather than the standard
usually passed to loadable modules.) Later versions of Linux 2.2 and all
versions of Linux 2.2 will allow one to share IRQs on PCI serial boards;
previously this was only allowed on ISA cards and on-board serial ports.
Some other pieces of multiport hardware will be better supported under
Linux 2.2. More updates and new drivers are flowing in regularly.

In contrast, the parallel port subsystem has undergone some major
since 2.2. There is now a generic parallel port driver for abstracted
communication with "unknown" types of parallel devices. This could be
for example, by programs that want to poll the parallel port for
Plug-and-Play information. (This is completely unrelated to the
Plug-and-Play capabilities of ISA cards.) Additionally, this has the
effect of allowing the root console to be spit out to a parallel port:
can configure this to cause console messages to be written out to a
printer. (This is especially useful for debugging in cases when copying
from the screen just won't do.) Also, Linux 2.4 supports writing to the
parallel ports using DMA, if supported in the hardware. This will speed
access to printers and other parallel devices.

Infra-red support has progressed since Linux 2.2 and there have been
changes in this area, including better network support.

In a separate department, there has been some work since 2.2 on
so-called "WinModems". These are modems which exist largely in software
whose drivers are only provided by the manufacturer for Windows. (Hence
name.) While no code has been submitted to Linus for the support of
beasts, it is possible that we may see some support for them before 3.0.
One major obstacle here is that each and every WinModem is different; it
unlikely that a driver for one would be applicable to another and the
number of different types of WinModems would make this difficult or
impossible to ever get a decent selection of hardware supported.
odds have never phased open source developers however and I for one will
not be surprised when the first driver makes it into the kernel,

Networking and Protocols

Even further distant from the simplicity of keyboards and mice,
and network hardware is one of the major areas where Linux has always
excelled. Linux 2.2 included many improvements to this layer including
drivers. Linux 2.4 has expanded on the already great amount of supported
hardware by providing bug fixes and new drivers for a variety of cards
including and especially ISDN devices.

Linux 2.4 also includes a completely rewritten networking layer. In
it has been made an unserialized as possible so that it will scale far
better than any previous version of Linux. In addition, it contains many
optimizations to allow it to work with the particular quirks of the
networking stacks in use in many common operating systems, including
Windows. It should also be mentioned at this point that Linux is still
only operating system completely compatible with the IPv4 specification.
can be sure to see many more improvements in this area before the Linux
release date is reached.

Next to the new network layer, the next most important improvement in
Linux 2.4 network layer is the addition of code to handle the DECNet
protocols. This allows for better interoperation with specialized
Digital/Compaq systems.

For the low-end desktop users, PPP is an important part of day to day
Linux 2.4 includes some major rewrites and modularization of much of the
code, including a long awaited combination of the PPP layers from the
layer and the PPP layer used on serial devices, such as modems.

Multimedia: Sound, TV, Radio, etc.

And gradually, we have worked our way from the essential components of
system to the fluffy components, the ones that may or may not be useful
your individual needs. Linux, in its emerging role as a desktop
tries very hard to support these devices, including sound cards, TV and
radio tuners, and other sound and video output devices. To be honest,
2.4 does not include as many ground-breaking changes as Linux 2.2 did in
this respect. (Yet...) Linux 2.4 does however include updates and new
drivers for a variety of sound and video cards, including full duplex
support. Linux 2.4 and some later versions of Linux 2.2 also include
which will allow some sound devices to more easily allocate memory in
ranges needed by some sound cards; this should make the configuration
use of these cards much easier.

Linux 2.4 (and later versions of Linux 2.2) now supports the DoubleTalk
speech synthesizer card. This is the first piece of hardware of this
which is supported under Linux, maybe we'll see more of these in the
future. (Really, all the "cool" hardware types are already supported.
take what we can get. :) )


I appreciate any comments that you may have, especially changes that you
feel are important that I missed. For the purpose of keeping my email
sorted into nice, neat piles, please respond to
[email protected].
(Yes, .com. Watch the .com.)