Received: by 2002:a25:86ce:0:0:0:0:0 with SMTP id y14csp1035536ybm; Wed, 22 May 2019 16:13:53 -0700 (PDT) X-Google-Smtp-Source: APXvYqxKdIyoPWpaDwQf46PNp3OiTVXurJDTcHcmSnACbzKSXTeGO5G1j638z53nWY+WEAcP8ptQ X-Received: by 2002:aa7:980e:: with SMTP id e14mr99410757pfl.142.1558566831956; Wed, 22 May 2019 16:13:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558566831; cv=none; d=google.com; s=arc-20160816; b=hBu8ww/TZFtDM9M0d4DYIzvW0wvqMHP2l5apfC7XD5pnS0XVDHLLtOfKTm5TCF+X7+ SOQ5/BA6DUdkX0WLyHrVNBz7scgLcSdu3SHTu56ZkieGQ98uVWByGT3nx0dvRXas5ZgR N/sT6PBc/adzprqBiufG1G/Gb4mnyDy+0IXXQqcCwrdmOiIkcKteioyB0BBC/MAfNBWF eKt/bH4PwvwPtwo8eacukKqqhaBqNwjYb8onrZ3oyBq4nMJ5elHJ+pPJDHqWdfaDhnq2 N7dDOD29BEk1Mtf+PnEOSZy16psQprMuujiYJp26ixJBYkR5uST5sqM4XkSumi6ylDpX lklg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=iSR8c9EVVMV5FED+J4MEd80CoCczKyDqKguuCikzCt4=; b=GJqZDrylqagUfrSWG2ZRo75nCMSMNdkzEj9pyswK90ymuq+nThWKEVKn7fC1GUQ3sa jt4KZqS6DhqEw1EXHkee6425FFMec1M8iDgNj3O5L7GDU17bnuxdfoys3Dp4gOAuHI+I S251ciodLUz3rwqREI++Wx8t2lZw8UGlQiL5pRvdCPVcJslC4JIJAkR6uZKAFibhvwgQ G3VY4gwm3RCnlE/qQrgPHgvWYL9Z2/KsKJfoPEoLhRMaKvhw0Nqw6xB5DBg/pf0tJZIY LTzVRT7sypDeyaXbRjsBro6We/LSlxI+oIyDlQkfeCm7sk39Q3XXUHP3jdJ8eYKIrZu5 vTTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@networkplumber-org.20150623.gappssmtp.com header.s=20150623 header.b=bcRPv8Sb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3si28499398pgj.104.2019.05.22.16.13.36; Wed, 22 May 2019 16:13:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@networkplumber-org.20150623.gappssmtp.com header.s=20150623 header.b=bcRPv8Sb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729534AbfEVXMd (ORCPT + 99 others); Wed, 22 May 2019 19:12:33 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:46752 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726215AbfEVXMd (ORCPT ); Wed, 22 May 2019 19:12:33 -0400 Received: by mail-pf1-f196.google.com with SMTP id y11so2099058pfm.13 for ; Wed, 22 May 2019 16:12:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iSR8c9EVVMV5FED+J4MEd80CoCczKyDqKguuCikzCt4=; b=bcRPv8SbXfnc3aWiBfcJjRixhyibAaJ7pTxpJpGNUkX0ndBLwk1bEUzGjbBqyI/n1V rWYzEdTMzleXOY980msJKO3cfh6rLyLQQ/m7FSaPQWk2WrywrcgC4Bi+P0zjKu7PPFRq Te8nfKbTGXhfcm4zlyTnXyvgkgoUDTNOFIrS0OssPvLA3RsbwwBvwf6pHKjVfwV8fUL+ 0V+R22DTTMSajQZ0IpREZ3aoOQ5au8IKdi5HpQRi/9zP4CI6H/F6SJf2JkrsjnZyNPVZ 5rH68pV9Ex4i/PWW6zsXNui3rljvuS9Xs2yyTngEKSlEnyS6n6VuhRB2ykaTjpWZu1hR 6G7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iSR8c9EVVMV5FED+J4MEd80CoCczKyDqKguuCikzCt4=; b=ffq7BJBzdZ3e4qOwCvj68k+QZ0vmsVKivv6zrgZv3ac60IXGVSm0nG7k815IkJyu0V QnwlK+Pc/WIGJPahGtIpcoWhWVifsrVWYH8Nxnz7OhY2XwkA2uwLZaxL3mb4YcQXqjl2 rlwcNnfrffzI74RdB/AgZC4uMSVdkYW0pEmz6ZaAb9hkf51Bk6k9jMKS4qIrW5HKpIjd 22X0/eacf7f5snhCeYSwjhU61yZx3JCnxbCRN3ZQ+HPsW2yCsrL3LY+USEEVz4PSHlJh Jnn3Z3Fyb7kvloLIcIoGlcmtkhLK0PVzNDcV71TNeJoZ8B6mZFUM1wE1bg8fssGt9760 GjDA== X-Gm-Message-State: APjAAAVhEWGBdCb6ZmnmfpJBJ03DoGT7GLVquNgN/dh48YjnFjn/+jxE TZtQaNv0me5tymF6Z418FoTI6Q== X-Received: by 2002:aa7:8d81:: with SMTP id i1mr71762713pfr.244.1558566752498; Wed, 22 May 2019 16:12:32 -0700 (PDT) Received: from xps13.lan (204-195-22-127.wavecable.com. [204.195.22.127]) by smtp.gmail.com with ESMTPSA id 7sm7426438pfo.90.2019.05.22.16.12.32 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 22 May 2019 16:12:32 -0700 (PDT) Date: Wed, 22 May 2019 16:12:30 -0700 From: Stephen Hemminger To: Sunil Muthuswamy Cc: KY Srinivasan , Haiyang Zhang , Stephen Hemminger , Sasha Levin , "David S. Miller" , Dexuan Cui , Michael Kelley , "netdev@vger.kernel.org" , "linux-hyperv@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH net-next] hv_sock: perf: Allow the socket buffer size options to influence the actual socket buffers Message-ID: <20190522161230.7daf840c@xps13.lan> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 22 May 2019 22:56:07 +0000 Sunil Muthuswamy wrote: > Currently, the hv_sock buffer size is static and can't scale to the > bandwidth requirements of the application. This change allows the > applications to influence the socket buffer sizes using the SO_SNDBUF and > the SO_RCVBUF socket options. > > Few interesting points to note: > 1. Since the VMBUS does not allow a resize operation of the ring size, the > socket buffer size option should be set prior to establishing the > connection for it to take effect. > 2. Setting the socket option comes with the cost of that much memory being > reserved/allocated by the kernel, for the lifetime of the connection. > > Perf data: > Total Data Transfer: 1GB > Single threaded reader/writer > Results below are summarized over 10 iterations. > > Linux hvsocket writer + Windows hvsocket reader: > |---------------------------------------------------------------------------------------------| > |Packet size -> | 128B | 1KB | 4KB | 64KB | > |---------------------------------------------------------------------------------------------| > |SO_SNDBUF size | | Throughput in MB/s (min/max/avg/median): | > | v | | > |---------------------------------------------------------------------------------------------| > | Default | 109/118/114/116 | 636/774/701/700 | 435/507/480/476 | 410/491/462/470 | > | 16KB | 110/116/112/111 | 575/705/662/671 | 749/900/854/869 | 592/824/692/676 | > | 32KB | 108/120/115/115 | 703/823/767/772 | 718/878/850/866 | 1593/2124/2000/2085 | > | 64KB | 108/119/114/114 | 592/732/683/688 | 805/934/903/911 | 1784/1943/1862/1843 | > |---------------------------------------------------------------------------------------------| > > Windows hvsocket writer + Linux hvsocket reader: > |---------------------------------------------------------------------------------------------| > |Packet size -> | 128B | 1KB | 4KB | 64KB | > |---------------------------------------------------------------------------------------------| > |SO_RCVBUF size | | Throughput in MB/s (min/max/avg/median): | > | v | | > |---------------------------------------------------------------------------------------------| > | Default | 69/82/75/73 | 313/343/333/336 | 418/477/446/445 | 659/701/676/678 | > | 16KB | 69/83/76/77 | 350/401/375/382 | 506/548/517/516 | 602/624/615/615 | > | 32KB | 62/83/73/73 | 471/529/496/494 | 830/1046/935/939 | 944/1180/1070/1100 | > | 64KB | 64/70/68/69 | 467/533/501/497 | 1260/1590/1430/1431 | 1605/1819/1670/1660 | > |---------------------------------------------------------------------------------------------| > > Signed-off-by: Sunil Muthuswamy It looks like Exchange mangled you patch. It doesn't apply clean. >