Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752524AbcD3AGD (ORCPT ); Fri, 29 Apr 2016 20:06:03 -0400 Received: from mail-ig0-f181.google.com ([209.85.213.181]:35399 "EHLO mail-ig0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750973AbcD3AGB (ORCPT ); Fri, 29 Apr 2016 20:06:01 -0400 MIME-Version: 1.0 In-Reply-To: <20160429233115.8864.qmail@ns.horizon.com> References: <20160429233115.8864.qmail@ns.horizon.com> Date: Fri, 29 Apr 2016 17:05:59 -0700 X-Google-Sender-Auth: _Rx3Uy_YWzvlhTzNuW47qCfkg18 Message-ID: Subject: Re: [patch 2/7] lib/hashmod: Add modulo based hash mechanism From: Linus Torvalds To: George Spelvin Cc: Linux Kernel Mailing List , Thomas Gleixner Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2338 Lines: 53 On Fri, Apr 29, 2016 at 4:31 PM, George Spelvin wrote: > > After researching it, I think that the "high bits of a multiply" is > in fact a decent way to do such a hash. Our emails crossed. Yes. My test harness actually likes the multiplication more than most of the specialized "spread out bits" versions I've found, but only if the constants are better-chosen than the ones we have now. > One thing I note is that the advice in the comments to choose a prime > number is misquoting Knuth! Knuth says (vol. 3 section 6.4) the number > should be *relatively* prime to the word size, which for binary computers > simply means odd. At least for my tests, even that seems to actually be a total non-issue. Yes, odd values *might* be better, but as mentioned in my crossing email, it doesn't actually seem to matter for any case the kernel cares about, since we tend to want to hash down to 10-20 bits of data, so the least significant bit (particularly for the 64-bit case) just doesn't matter all that much. For the 32-bit case I suspect it's more noticeable, since we might be using even half or more of the result. But as mentioned: that least-order bit seems to be a *lot* less important than the mix of the bits in the middle. Because even if your input ends up being all zeroes in the low bits (it that worst-case "page aligned pointers" case that Thomas had), and the hash multiplies by an even number, you'll still end up just dropping all those zero bits anyway. > When we have a hardware multiplier, keeping the Hamming weight low is > a waste of time. When we don't, clever organization can do > better than the very naive addition/subtraction chain in the > current hash_64(). Yeah. gcc will actually do the clever stuff for the 32-bit case, afaik. Nothing I know of does it for the 64-bit case, which is why our current hand-written one isn't even the smark one. > To multiply by the 32-bit constant 1640531527 = 0x61c88647 (which is > the negative of the golden ratio, so has identical distribution > properties) can be done in 6 shifts + adds, with a critical path > length of 7 operations (3 shifts + 4 adds). So the reason we don't do this for the 32-bit case is exactly that gcc can already do this. If you can do the same for the 64-bit case, that might be worth it. Linus