Received: by 2002:a05:6a10:22f:0:0:0:0 with SMTP id 15csp3028323pxk; Mon, 7 Sep 2020 00:31:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzyNIQoVIGu62ckw7LdhHwIoG++zofYWr6ghJJS5LbawZnqrEyAKrlt12CSo4583YoWsMMQ X-Received: by 2002:a17:906:7cb:: with SMTP id m11mr20439199ejc.41.1599463886426; Mon, 07 Sep 2020 00:31:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1599463886; cv=none; d=google.com; s=arc-20160816; b=HUdd9UVXg2BhYm4sOO0KbVy4Th97W6OFOBtc3IlsjvJp+2SxtblQSpu4VscNIueIis 8HuIXlJNG8LTTFjQ8Fbp9eh+H+wiUKEmrvr4urw37MKHPVaeKh7e4RNibW6I0z8hfeBx 9AsyxEMQEPfYtawIogLVwmPe3s4xehgQaPzUGvV+ZhpxvSxEbEc5iEyOE1zrKw1fM4Xz ZskOnWhSju3gzKHTgj8JvkayYnw1EG1K0P8dZ8pZEcKfFTIfosec+KJYIqDQDmH9T+pR 5itsC3yK5+wEmukiZC686hHhHsP4YglTo24rBgRqlQzlEEJkik5TXKVas9JGj+ARYakL EsrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date; bh=i/QRijOd7nCjAUlKub1OFwZYtHKnobABic+VZiZMOfM=; b=bXOJrmZh8NLiF8Ch4zPBqh0JfmVfIFbU1iIgGp32wkGACWvf79aieBdvOL0SbQTGq0 PIVqtvBSKa6WjYxOkDij+75oPdt3WHTcEQXiotVOf42426yAX5ql1XU+dgui9ToKPGN0 co3UDadpKIVHrzJ9Aj6oLMVSiKMVavTiO7fXW/F24scNbfn63U3vJBp6YMTpKVod4TH7 /4szlXkXW93PwhBMWWVIRjRlJYgeJ7Lin4E/ANNKJUftyeGFxGQu1+E3aaeyRCGyOm3u jp65u45HvfqzetTTf956GXAJgm8359clKWIAfDiobc7GJVe+UIWgrKMnraAwS8nOTBom llIw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id kt18si2761678ejb.338.2020.09.07.00.31.04; Mon, 07 Sep 2020 00:31:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727797AbgIGH3l (ORCPT + 99 others); Mon, 7 Sep 2020 03:29:41 -0400 Received: from verein.lst.de ([213.95.11.211]:47999 "EHLO verein.lst.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727780AbgIGH33 (ORCPT ); Mon, 7 Sep 2020 03:29:29 -0400 Received: by verein.lst.de (Postfix, from userid 2407) id 7252A68BFE; Mon, 7 Sep 2020 09:29:26 +0200 (CEST) Date: Mon, 7 Sep 2020 09:29:26 +0200 From: Christoph Hellwig To: Leon Romanovsky Cc: Christoph Hellwig , Doug Ledford , Jason Gunthorpe , Maor Gottlieb , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org Subject: Re: [PATCH rdma-next 4/4] RDMA/umem: Move to allocate SG table from pages Message-ID: <20200907072926.GD19875@lst.de> References: <20200903121853.1145976-1-leon@kernel.org> <20200903121853.1145976-5-leon@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200903121853.1145976-5-leon@kernel.org> User-Agent: Mutt/1.5.17 (2007-11-01) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Sep 03, 2020 at 03:18:53PM +0300, Leon Romanovsky wrote: > From: Maor Gottlieb > > Remove the implementation of ib_umem_add_sg_table and instead > call to sg_alloc_table_append which already has the logic to > merge contiguous pages. > > Besides that it removes duplicated functionality, it reduces the > memory consumption of the SG table significantly. Prior to this > patch, the SG table was allocated in advance regardless consideration > of contiguous pages. > > In huge pages system of 2MB page size, without this change, the SG table > would contain x512 SG entries. > E.g. for 100GB memory registration: > > Number of entries Size > Before 26214400 600.0MB > After 51200 1.2MB > > Signed-off-by: Maor Gottlieb > Signed-off-by: Leon Romanovsky Looks sensible for now, but the real fix is of course to avoid the scatterlist here entirely, and provide a bvec based pin_user_pages_fast. I'll need to finally get that done..