Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1764556AbXIMEd4 (ORCPT ); Thu, 13 Sep 2007 00:33:56 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752383AbXIMEds (ORCPT ); Thu, 13 Sep 2007 00:33:48 -0400 Received: from sj-iport-2-in.cisco.com ([171.71.176.71]:61619 "EHLO sj-iport-2.cisco.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752367AbXIMEdr (ORCPT ); Thu, 13 Sep 2007 00:33:47 -0400 X-IronPort-AV: E=Sophos;i="4.20,248,1186383600"; d="scan'208";a="398709018" To: Joachim Fenkes Cc: "LinuxPPC-Dev" , LKML , "OF-General" , Roland Dreier , "OF-EWG" , "Hoang-Nam Nguyen" , Christoph Raisch , Stefan Roscher Subject: Re: [PATCH] IB/ehca: Make sure user pages are from hugetlb before using MR large pages X-Message-Flag: Warning: May contain useful information References: <200709121439.32641.fenkes@de.ibm.com> From: Roland Dreier Date: Wed, 12 Sep 2007 21:33:45 -0700 In-Reply-To: <200709121439.32641.fenkes@de.ibm.com> (Joachim Fenkes's message of "Wed, 12 Sep 2007 14:39:31 +0200") Message-ID: User-Agent: Gnus/5.1008 (Gnus v5.10.8) XEmacs/21.4.20 (linux) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-OriginalArrivalTime: 13 Sep 2007 04:33:45.0987 (UTC) FILETIME=[45ACF530:01C7F5BF] Authentication-Results: sj-dkim-3; header.From=rdreier@cisco.com; dkim=pass ( sig from cisco.com/sjdkim3002 verified; ); Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2017 Lines: 55 > -#define HCA_CAP_MR_PGSIZE_4K 1 > -#define HCA_CAP_MR_PGSIZE_64K 2 > -#define HCA_CAP_MR_PGSIZE_1M 4 > -#define HCA_CAP_MR_PGSIZE_16M 8 > +#define HCA_CAP_MR_PGSIZE_4K 0x80000000 > +#define HCA_CAP_MR_PGSIZE_64K 0x40000000 > +#define HCA_CAP_MR_PGSIZE_1M 0x20000000 > +#define HCA_CAP_MR_PGSIZE_16M 0x10000000 Not sure I understand what this has to do with things... is this an unrelated fix? > +static int ehca_is_mem_hugetlb(unsigned long addr, unsigned long size) This is rather awful -- another call to get_user_pages() to iterate over all the vmas... I would suggest extending ib_umem_get() to check the vmas and adding a member to struct ib_umem to say whether the memory is entirely covered by hugetlb pages or not. > + ret = ehca_is_mem_hugetlb(virt, length); > + switch (ret) { > + case 0: /* mem is not from hugetlb */ > + hwpage_size = PAGE_SIZE; > + break; > + case 1: > + if (length <= EHCA_MR_PGSIZE4K > + && PAGE_SIZE == EHCA_MR_PGSIZE4K) > + hwpage_size = EHCA_MR_PGSIZE4K; > + else if (length <= EHCA_MR_PGSIZE64K) > + hwpage_size = EHCA_MR_PGSIZE64K; > + else if (length <= EHCA_MR_PGSIZE1M) > + hwpage_size = EHCA_MR_PGSIZE1M; > + else > + hwpage_size = EHCA_MR_PGSIZE16M; > + break; > + default: /* out of mem */ > + ib_mr = ERR_PTR(-ENOMEM); > + goto reg_user_mr_exit1; It seems like it would be better to just assume the memory is not from a hugetlb is ehca_is_mem_hugetlb() fails its memory allocation and fall back to the PAGE_SIZE case rather than failing entirely. Also if someone runs a kernel with 64K pages on a machine where they end up being simulated from 4K pages, do you have the same issue with the hypervisor ganging together non-contiguous pages? - R. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/