Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932974AbdIHXDq (ORCPT ); Fri, 8 Sep 2017 19:03:46 -0400 Received: from mail.kernel.org ([198.145.29.99]:40770 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932954AbdIHXDn (ORCPT ); Fri, 8 Sep 2017 19:03:43 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AECB321EAB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=sstabellini@kernel.org Date: Fri, 8 Sep 2017 16:03:42 -0700 (PDT) From: Stefano Stabellini X-X-Sender: sstabellini@sstabellini-ThinkPad-X260 To: Boris Ostrovsky cc: Stefano Stabellini , xen-devel@lists.xen.org, linux-kernel@vger.kernel.org, jgross@suse.com, Stefano Stabellini Subject: Re: [PATCH v3 11/13] xen/pvcalls: implement poll command In-Reply-To: <702cfa9c-5f14-07a3-63ba-93648ff66d9b@oracle.com> Message-ID: References: <1501541855-7354-1-git-send-email-sstabellini@kernel.org> <1501541855-7354-11-git-send-email-sstabellini@kernel.org> <702cfa9c-5f14-07a3-63ba-93648ff66d9b@oracle.com> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5452 Lines: 171 On Tue, 15 Aug 2017, Boris Ostrovsky wrote: > > @@ -736,6 +755,104 @@ int pvcalls_front_accept(struct socket *sock, struct socket *newsock, int flags) > > return ret; > > } > > > > +static unsigned int pvcalls_front_poll_passive(struct file *file, > > + struct pvcalls_bedata *bedata, > > + struct sock_mapping *map, > > + poll_table *wait) > > +{ > > + int notify, req_id, ret; > > + struct xen_pvcalls_request *req; > > + > > I am a bit confused by the logic here. > > > + if (test_bit(PVCALLS_FLAG_ACCEPT_INFLIGHT, > > + (void *)&map->passive.flags)) { > > + uint32_t req_id = READ_ONCE(map->passive.inflight_req_id); > > + if (req_id != PVCALLS_INVALID_ID && > > + READ_ONCE(bedata->rsp[req_id].req_id) == req_id) > > + return POLLIN; > > This is successful accept? Yes: if an accept reply is pending, there must be something to read on the passive socket. > Why is it returning POLLIN only and not > POLLIN | POLLRDNORM (or POLLOUT, for that matter)? Keep in mind that this is a passive socket. There is no writing to them. But yes, POLLIN and POLLRDNORM are equivalent, so it is best to set both. I'll do that. > > + > > + poll_wait(file, &map->passive.inflight_accept_req, wait); > > + return 0; > > + } > > + > > + if (test_and_clear_bit(PVCALLS_FLAG_POLL_RET, > > + (void *)&map->passive.flags)) > > + return POLLIN; > > This is previous poll request completing? Yes, a previous POLL request completing. > > + > > + /* > > + * First check RET, then INFLIGHT. No barriers necessary to > > + * ensure execution ordering because of the conditional > > + * instructions creating control dependencies. > > + */ > > + > > + if (test_and_set_bit(PVCALLS_FLAG_POLL_INFLIGHT, > > + (void *)&map->passive.flags)) { > > + poll_wait(file, &bedata->inflight_req, wait); > > + return 0; > > + } > > This I don't understand, could you explain? If none of the conditions above apply (there isn't an accept reply ready, and there isn't a poll reply ready) then it means that we actually need to tell the caller to wait. This is done by calling poll_wait to add a wait_queue to the poll table. Then we need to tell the backend that we are polling, that is done by the code below that sends a PVCALLS_POLL message to the backend. The check on PVCALLS_FLAG_POLL_INFLIGHT here is to avoid sending multiple PVCALLS_POLL requests to the backend. Thus, if there is one already in-flight, we can just call poll_wait and return immediately without sending a second request. > > + > > + spin_lock(&bedata->pvcallss_lock); > > + ret = get_request(bedata, &req_id); > > + if (ret < 0) { > > + spin_unlock(&bedata->pvcallss_lock); > > + return ret; > > + } > > + req = RING_GET_REQUEST(&bedata->ring, req_id); > > + req->req_id = req_id; > > + req->cmd = PVCALLS_POLL; > > + req->u.poll.id = (uint64_t) map; > > + > > + bedata->ring.req_prod_pvt++; > > + RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(&bedata->ring, notify); > > + spin_unlock(&bedata->pvcallss_lock); > > + if (notify) > > + notify_remote_via_irq(bedata->irq); > > + > > + poll_wait(file, &bedata->inflight_req, wait); > > + return 0; > > +} > > + > > +static unsigned int pvcalls_front_poll_active(struct file *file, > > + struct pvcalls_bedata *bedata, > > + struct sock_mapping *map, > > + poll_table *wait) > > +{ > > + unsigned int mask = 0; > > + int32_t in_error, out_error; > > + struct pvcalls_data_intf *intf = map->active.ring; > > + > > + out_error = intf->out_error; > > + in_error = intf->in_error; > > + > > + poll_wait(file, &map->active.inflight_conn_req, wait); > > + if (pvcalls_front_write_todo(map)) > > + mask |= POLLOUT | POLLWRNORM; > > + if (pvcalls_front_read_todo(map)) > > + mask |= POLLIN | POLLRDNORM; > > + if (in_error != 0 || out_error != 0) > > + mask |= POLLERR; > > + > > + return mask; > > +} > > + > > +unsigned int pvcalls_front_poll(struct file *file, struct socket *sock, > > + poll_table *wait) > > +{ > > + struct pvcalls_bedata *bedata; > > + struct sock_mapping *map; > > + > > + if (!pvcalls_front_dev) > > + return POLLNVAL; > > + bedata = dev_get_drvdata(&pvcalls_front_dev->dev); > > + > > + map = (struct sock_mapping *) READ_ONCE(sock->sk->sk_send_head); > > I just noticed this --- why is it READ_ONCE? Are you concerned that > sk_send_head may change? No, but I wanted to avoid partial reads. A caller could call pvcalls_front_accept and pvcalls_front_poll on newsock almost at the same time (it is probably not the correct way to use the API), I wanted to make sure that "map" is either read correctly, or not read at all. > > + if (!map) > > + return POLLNVAL; > > + if (map->active_socket) > > + return pvcalls_front_poll_active(file, bedata, map, wait); > > + else > > + return pvcalls_front_poll_passive(file, bedata, map, wait); > > +} > > + > > static const struct xenbus_device_id pvcalls_front_ids[] = { > > { "pvcalls" }, > > { "" } > > diff --git a/drivers/xen/pvcalls-front.h b/drivers/xen/pvcalls-front.h > > index de24041..25e05b8 100644 > > --- a/drivers/xen/pvcalls-front.h > > +++ b/drivers/xen/pvcalls-front.h > > @@ -20,5 +20,8 @@ int pvcalls_front_recvmsg(struct socket *sock, > > struct msghdr *msg, > > size_t len, > > int flags); > > +unsigned int pvcalls_front_poll(struct file *file, > > + struct socket *sock, > > + poll_table *wait); > > > > #endif >