Date: Mon, 8 Jan 2007 22:20:45 +0100
From: Jean Delvare <khali@linux-fr.org>
To: "J.H." <warthog9@kernel.org>
Cc: Randy Dunlap <randy.dunlap@oracle.com>, Andrew Morton <akpm@osdl.org>,
       Pavel Machek <pavel@ucw.cz>, kernel list <linux-kernel@vger.kernel.org>,
       hpa@zytor.com, webmaster@kernel.org
Subject: Re: [KORG] Re: kernel.org lies about latest -mm kernel
Message-Id: <20070108222045.644ec0be.khali@linux-fr.org>
In-Reply-To: <1166297434.26330.34.camel@localhost.localdomain>
References: <20061214223718.GA3816@elf.ucw.cz>
	<20061216094421.416a271e.randy.dunlap@oracle.com>
	<20061216095702.3e6f1d1f.akpm@osdl.org>
	<458434B0.4090506@oracle.com>
	<1166297434.26330.34.camel@localhost.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3068
Lines: 60

Hi JH,

On Sat, 16 Dec 2006 11:30:34 -0800, J.H. wrote:
> The root cause boils down to with git, gitweb and the normal mirroring
> on the frontend machines our basic working set no longer stays resident
> in memory, which is forcing more and more to actively go to disk causing
> a much higher I/O load.  You have the added problem that one of the
> frontend machines is getting hit harder than the other due to several
> factors: various DNS servers not round robining, people explicitly
> hitting [git|mirrors|www|etc]1 instead of 2 for whatever reason and

I am trying to be a good citizen by explicitely asking for
www2.kernel.org, unfortunately I notice that many links on the main
page point to www.kernel.org rather than www2.kernel.org. Check the
location, patchtype, full source, patch, view patch, and changeset
links for example. Fixing these links would let people really use www2
if they want to, that might help.

BTW, I'm no DNS expert, but isn't it possible to favor one host in the
round robin mechanism? E.g. by listing the server 2 twice, so that it
gets 2/3 of the load? This could also help if server 1 otherwise gets
more load.

> So we know the problem is there, and we are working on it - we are
> getting e-mails about it if not daily than every other day or so.  If
> there are suggestions we are willing to hear them - but the general
> feeling with the admins is that we are probably hitting the biggest
> problems already.

I have a few suggestions although I realize that the other things
you're working on are likely to be much more helpful:

* Shorten the www.kernel.org main page. I guess that 99% of the hits on
this page are by people who just want to know the latest versions, and
possibly download a patch or access Linus' git tree through gitweb. All
the rest could be moved to a separate page, or if you think it's
better to keep all the general info on the main page, move the array
with the versions to a separate page, which developers can bookmark.
Splitting the dynamic content (top) from the essentially static content
(bottom) of this page should help with caching, BTW.

* Drop the bandwidth graphs. Most visitors certainly do not care, and
their presence generates traffic on all web servers regardless of the
one the visitor is using, as each graph is generated by the respective
server. If you really like these graphs, just move them to a separate
page for people who want to watch them. As far as I am concerned, I
find them rather confusing and uninformative - from a quick look you
just can't tell if the servers are loaded or not, you have to look at
the numbers, so what's the point of drawing a graph...

Of course the interest of these proposals directly depends on how much
the www.kernel.org/index page accounts in the total load of the servers.

-- 
Jean Delvare
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/