Received: by 2002:ab2:620c:0:b0:1ef:ffd0:ce49 with SMTP id o12csp732460lqt; Tue, 19 Mar 2024 02:00:11 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCUZcfIo6y59Bxl+zOK/zWcIjmCV/ommBi2cHkNlBWK1OKwB2ZTZsBAmB7xzBseXUVOKrQU39WoqA0iAenWAdlfM9dC8IYzylyzW5g+UFQ== X-Google-Smtp-Source: AGHT+IFcuqCP4jVyXW6TgBo3ODP+KwwNolcaX0NOwmrKjjPDcHoNemn/Ne+sLY4BHKUry8W/d0vO X-Received: by 2002:a05:6a20:c6c8:b0:1a3:1246:c43e with SMTP id gw8-20020a056a20c6c800b001a31246c43emr16963275pzb.28.1710838811110; Tue, 19 Mar 2024 02:00:11 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1710838811; cv=pass; d=google.com; s=arc-20160816; b=BUY7H8KW3jkEX0mZvPzaNDZA8rYnQMq6pEzwjVe33gYAXA6BaqhB3pH4VpxEq39TWi 3ZKI7lZyIWxtpg9BPI1U1Nw1SR2s6rwSuVKA8gfBka7cQQldObVNqLk07p19VZKF947q /xYcGG2rCKZoVVCw6+ovDIy2w1JyxBIntbay0Dr6Jowp2JyRRCEsGMsz0mcBgL6Yj67d a1frTe1z8zLqK1iFAq5uJ6oCfLLEM75ol43ti2eqKM8BoGw6y444nLEDeF1Qx1Vhu8Fs C+YKWtkKYHeT5oQfg0Yu9pMHpK8vxaNeKcAsUf1/WibljxMku1KKf2Av0IWOw53BCV3m wTgg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:message-id:references:in-reply-to:subject :cc:to:from:date:mime-version:list-unsubscribe:list-subscribe :list-id:precedence:dkim-signature; bh=LUbKQi10BhRRdxGcjFBVGH4lzCf1fHSjHP8L558ZR/U=; fh=S7NkAICIL15kzjy9NIe1LOoZKLumFIGQGgpp8AAFH/o=; b=fzNqeuHx/NQ/TMxqsdUKGEEBWB/T88RNkp2rEfkljnwibDaM3I56iYdsisL+f2p0Gn YGM4n1E+EoOjoak6xdiD5QtqjRX8iBYWF6vhLXf/qmoMIkWwmooGGL8VX1QZ3xriaXGA sRtjTDqzM4Kw+IjC10tpIftR6ZWmaQvI/VbyPnPBVx7f9oJ13C0HuMgyvLUNXpSR/LfF dyP2SxENAlUJkzmLAZSIHN+mBlXHxMPNcjzPB7GEQA9noqPeYYq/Octy67k4fyI9ac5r uozgied2v4T7XBx8uHypX4hzRX8z29dkKiZx3cubCpIA73pURm+R07RlAV7xO4LJOoG5 xknA==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=hoh38pHS; arc=pass (i=1 spf=pass spfdomain=linux.ibm.com dkim=pass dkdomain=ibm.com dmarc=pass fromdomain=linux.ibm.com); spf=pass (google.com: domain of linux-kernel+bounces-107272-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-107272-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id lk6-20020a17090308c600b001e026c06c21si3664715plb.340.2024.03.19.02.00.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Mar 2024 02:00:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-107272-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=hoh38pHS; arc=pass (i=1 spf=pass spfdomain=linux.ibm.com dkim=pass dkdomain=ibm.com dmarc=pass fromdomain=linux.ibm.com); spf=pass (google.com: domain of linux-kernel+bounces-107272-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-107272-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=NONE dis=NONE) header.from=ibm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id C8F88281E40 for ; Tue, 19 Mar 2024 09:00:10 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 58FBA7C0BD; Tue, 19 Mar 2024 08:59:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="hoh38pHS" Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EEEB54BE2; Tue, 19 Mar 2024 08:59:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710838794; cv=none; b=HWmKQWTg45KdaB7FvustEpD0Y6nEzvOYpqVbb4nBUs6CWjQbYWCbelTDy3Mjdnjd0HMAVHWgD8x8GxTJfDF8XP7f+7qbnkV4D5QavPqY/ItrSdj9hq/0fDhPFHUYGneMMCzrDuynGcMhgsjA1q5dI0v/vCTA5TFsZ/1/lq669nQ= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710838794; c=relaxed/simple; bh=zsdYz3jN/6SotqZabyYZjBZs275r8CvOSdZhCKiQW8Q=; h=MIME-Version:Date:From:To:Cc:Subject:In-Reply-To:References: Message-ID:Content-Type; b=Vgs+zO/Nc7LjODemseNa6/gTc6/HQDZEee+eMxVvlw0afDeo+wGnOyg3FioPco1Cswf6uDT/nNDk1r1RWZT+EH/xqhTarMJ9wkQ4U7ZOHIoQIpFYzdSLO0VI4ZaMgXBvWnx3+9qgZKRtseo3E4ftKZyMLbCOSx3wZyh47Si8QBg= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=hoh38pHS; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 42J8UJxc019881; Tue, 19 Mar 2024 08:59:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=mime-version : date : from : to : cc : subject : in-reply-to : references : message-id : content-type : content-transfer-encoding; s=pp1; bh=LUbKQi10BhRRdxGcjFBVGH4lzCf1fHSjHP8L558ZR/U=; b=hoh38pHSBNRV6IF4S14G4mLMwItWsnfMYQ5lOLWbBp3l5kWcgGAdlYsYsBZnfjNerXUK ld6eBCyKGmQppMiZAjdQr1RRsuS0oQZRT+dhlHQyOpgR1T5sZ0ROXENtGLP4bPQh1oG5 xR2MpM5lh8yDwNQ1EPnNkvbzLxIxJIZDcQeuvT599qG+YvaxZ7VawCiiC8q2bUEHf98B EZX9P9GPSyzVtxmPlKGx30Y1tejWCsEU1Hb/L37PbExBZ5m8g/M/7B0+Nnxrx01T5nIn /Nx1sqbjiTi+nkwjSQiovjq2W0Z16uk2kAB0yORTA3V7KYrjfeSFM1iI2DsFhRfALGkP Kg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wy74u87er-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Mar 2024 08:59:38 +0000 Received: from m0353725.ppops.net (m0353725.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 42J8pns8029795; Tue, 19 Mar 2024 08:59:37 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3wy74u87eg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Mar 2024 08:59:37 +0000 Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 42J8c63x011543; Tue, 19 Mar 2024 08:59:36 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3wwq8kx5nk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 19 Mar 2024 08:59:36 +0000 Received: from smtpav05.wdc07v.mail.ibm.com (smtpav05.wdc07v.mail.ibm.com [10.39.53.232]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 42J8xXKJ45089124 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 19 Mar 2024 08:59:35 GMT Received: from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C99E458067; Tue, 19 Mar 2024 08:59:31 +0000 (GMT) Received: from smtpav05.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1C5CA58061; Tue, 19 Mar 2024 08:59:31 +0000 (GMT) Received: from ltc.linux.ibm.com (unknown [9.5.196.140]) by smtpav05.wdc07v.mail.ibm.com (Postfix) with ESMTP; Tue, 19 Mar 2024 08:59:30 +0000 (GMT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Date: Tue, 19 Mar 2024 09:59:30 +0100 From: Tobias Huschle To: "Michael S. Tsirkin" Cc: Luis Machado , Jason Wang , Abel Wu , Peter Zijlstra , Linux Kernel , kvm@vger.kernel.org, virtualization@lists.linux.dev, netdev@vger.kernel.org, nd Subject: Re: EEVDF/vhost regression (bisected to 86bfbb7ce4f6 sched/fair: Add lag based placement) In-Reply-To: <20240319042829-mutt-send-email-mst@kernel.org> References: <07974.124020102385100135@us-mta-501.us.mimecast.lan> <20240201030341-mutt-send-email-mst@kernel.org> <89460.124020106474400877@us-mta-475.us.mimecast.lan> <20240311130446-mutt-send-email-mst@kernel.org> <73123.124031407552500165@us-mta-156.us.mimecast.lan> <20240314110649-mutt-send-email-mst@kernel.org> <84704.124031504335801509@us-mta-515.us.mimecast.lan> <20240315062839-mutt-send-email-mst@kernel.org> <20240319042829-mutt-send-email-mst@kernel.org> Message-ID: <4808eab5fc5c85f12fe7d923de697a78@linux.ibm.com> X-Sender: huschle@linux.ibm.com Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: cHsvNv2I2RC7P1ZN_n6udKs2m42vkLbJ X-Proofpoint-GUID: rV_1Qssz0DqjdkKHbbVWtMepfcwTjcBL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-18_12,2024-03-18_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 mlxscore=0 mlxlogscore=843 impostorscore=0 spamscore=0 bulkscore=0 suspectscore=0 priorityscore=1501 phishscore=0 adultscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403140000 definitions=main-2403190068 On 2024-03-19 09:29, Michael S. Tsirkin wrote: > On Tue, Mar 19, 2024 at 09:21:06AM +0100, Tobias Huschle wrote: >> On 2024-03-15 11:31, Michael S. Tsirkin wrote: >> > On Fri, Mar 15, 2024 at 09:33:49AM +0100, Tobias Huschle wrote: >> > > On Thu, Mar 14, 2024 at 11:09:25AM -0400, Michael S. Tsirkin wrote: >> > > > >> > >> > Could you remind me pls, what is the kworker doing specifically that >> > vhost is relying on? >> >> The kworker is handling the actual data moving in memory if I'm not >> mistaking. > > I think that is the vhost process itself. Maybe you mean the > guest thread versus the vhost thread then? My understanding was that vhost writes data into a file descriptor which then triggers eventfd. That's at least how I read the vhost code if I remember correctly. The handler beneath (the kworker) then runs the actual instructions that move the data to the receiving vhost on the other end of the connection. Again, I might be wrong here.