Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754607Ab3HVXYa (ORCPT ); Thu, 22 Aug 2013 19:24:30 -0400 Received: from mail-qe0-f48.google.com ([209.85.128.48]:46856 "EHLO mail-qe0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753964Ab3HVXY2 (ORCPT ); Thu, 22 Aug 2013 19:24:28 -0400 MIME-Version: 1.0 In-Reply-To: References: From: Drunkard Zhang Date: Fri, 23 Aug 2013 07:24:07 +0800 Message-ID: Subject: Re: ipvsadm: One-packet scheduling with UDP service is unstable To: Julian Anastasov Cc: Wensong Zhang , Simon Horman , Pablo Neira Ayuso , Patrick McHardy , Jozsef Kadlecsik , "David S. Miller" , netdev@vger.kernel.org, lvs-devel@vger.kernel.org, netfilter-devel@vger.kernel.org, netfilter@vger.kernel.org, coreteam@netfilter.org, linux-kernel Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4480 Lines: 91 2013/8/22 Julian Anastasov : > > Hello, > > On Thu, 22 Aug 2013, Drunkard Zhang wrote: > >> 2013/8/22 Julian Anastasov : >> > >> > No kernel options should be related to OPS. I assume >> > you are not using the SH scheduler. Make sure the OPS mode >> > is properly applied to the virtual service, check for "ops" >> > in the configuration: >> > >> > cat /proc/net/ip_vs >> >> Still no lucky here, ops is set in running config, but it's not like >> that in real world. >> >> vs3 ~ # cat /proc/net/ip_vs >> IP Virtual Server version 1.2.1 (size=1024) >> Prot LocalAddress:Port Scheduler Flags >> -> RemoteAddress:Port Forward Weight ActiveConn InActConn >> UDP 96A46478:0202 wrr ops > >> -> 96A46450:0202 Route 25 0 1 > > The OPS connections are accounted in InActConn > for a very short period, they live up to 1 jiffie, eg. 10ms. > Also, WRR should be reliable for OPS while other > schedulers (eg. *LC) are not suitable. I noticed this too. While ops working, the InActConn is always changing too, if it's fixed, the ops is not working. >> And the traffic routed to each realserver didn't following weight I >> set, it's routed pretty much one to one. I got 17 udp sources sending >> to 16 different realservers, the others are bonding to another VIP. >> >> Prot LocalAddress:Port CPS InPPS OutPPS InBPS OutBPS >> -> RemoteAddress:Port >> UDP x.x.x.120:514 0 67622 0 12339373 0 >> -> x.x.x.65:514 0 29 0 2895 0 >> -> x.x.x.66:514 0 225 0 21850 0 > > Do you see the same problem with ipvsadm -Ln --stats ? > ipvsadm -Z may be needed to zero the stats after restoring all > rules. "Conns" counter in stats should be according to WRR > weights, it shows the scheduler decisions. After every restore, the stats also zeroed, right? While, ops still not working. vs3 ~/pkgs # ./ipvsadm -Z vs3 ~/pkgs # ./ipvsadm -ln --stats -u [snipped] Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes -> RemoteAddress:Port UDP x.x.x.120:514 0 12497040 0 2572M 0 -> x.x.x.65:514 0 3975 0 394171 0 -> x.x.x.66:514 0 48466 0 4835716 0 -> x.x.x.67:514 0 407051 0 58479621 0 -> x.x.x.68:514 0 561120 0 85289892 0 -> x.x.x.69:514 0 30958 0 3120506 0 -> x.x.x.70:514 0 645475 0 100552K 0 -> x.x.x.71:514 0 147228 0 14560649 0 -> x.x.x.72:514 0 535693 0 84069390 0 -> x.x.x.73:514 0 564787 0 88165140 0 -> x.x.x.74:514 0 346734 0 53256088 0 -> x.x.x.75:514 0 47232 0 4801578 0 -> x.x.x.76:514 0 1175288 0 192699K 0 -> x.x.x.77:514 0 254915 0 25939720 0 -> x.x.x.78:514 0 2701531 0 652417K 0 -> x.x.x.79:514 0 2426686 0 573897K 0 -> x.x.x.80:514 0 2599901 0 629793K 0 -> x.x.x.81:514 0 0 0 0 0 -> x.x.x.82:514 0 0 0 0 0 -> x.x.x.83:514 0 0 0 0 0 -> x.x.x.84:514 0 0 0 0 0 -> x.x.x.85:514 0 0 0 0 0 -> x.x.x.86:514 0 0 0 0 0 -> x.x.x.87:514 0 0 0 0 0 -> x.x.x.88:514 0 0 0 0 0 -> x.x.x.89:514 0 0 0 0 0 > In your rates listing CPS 0 is confusing, even for OPS. > Is it from the new ipvsadm? Yes, latest git version. When CPS is changing, the ops works, or it's not. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/