Hi!
I am working on implementing the Sockets Direct Protocol (SDP) for InfiniBand on
Linux. SDP uses the regular IPv4/IPv6 addresses and utilizes the IPv4/IPv6 layer
on top of InfiniBand for address resolution. However, all data is transferred by
means of an infiniband reliable connection.
Some existing SDP implementations posted on the openib.org subversion tree
create a new address family in a free slot, for this purpose.
Would it make sense to move SDP from using a separate address family to
a separate protocol under AF_INET and AF_INET6?
Something like IPPROTO_SDP?
The main advantages of this approach are
- IPv6 support will come more naturally and without further extending
to a yet another address family
- We could use a protocol number > 255 to avoid conflicting
with any IP based protocol.
There are much more free protocol numbers that free family numbers
(which only go up to 32 in linux for now).
- I could reuse more code for creating connections from af_inet.c
I also have a hunch this might make getaddrinfo and friends work better on sdp
selecting IPv4/IPv6 as appropriate but I'm not sure.
Comments? Are there disadvantages to this approach that someone can see?
Thanks,
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
----- End forwarded message -----
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
I don't know any details about SDP, but if there are no differences at the
protocol layer, then neither the address family nor the protocol is
appropriate. If it's just an API change, the socket type is the right
selector. So, maybe SOCK_DIRECT to go along with SOCK_STREAM,
SOCK_DGRAM, etc.
+-DLS
Quoting r. David Stevens <[email protected]>:
> Subject: Re: RFC: move SDP from AF_INET_SDP to IPPROTO_SDP
>
> I don't know any details about SDP, but if there are no differences at the
> protocol layer, then neither the address family nor the protocol is
> appropriate. If it's just an API change, the socket type is the right
> selector. So, maybe SOCK_DIRECT to go along with SOCK_STREAM,
> SOCK_DGRAM, etc.
No, the API SDP implements is the regular SOCK_STREAM semantics.
The difference is in the way connections are established with
infiniband connection management messages, and data is transferred
with infiniband reliable connection send messages.
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of David Stevens
> Sent: Monday, March 06, 2006 11:49 AM
> To: Michael S. Tsirkin
> Cc: Linux Kernel Mailing List; [email protected]
> Subject: Re: RFC: move SDP from AF_INET_SDP to IPPROTO_SDP
>
> I don't know any details about SDP, but if there are no
> differences at the protocol layer, then neither the address
> family nor the protocol is appropriate. If it's just an API
> change, the socket type is the right selector. So, maybe
> SOCK_DIRECT to go along with SOCK_STREAM, SOCK_DGRAM, etc.
> +-DLS
That wouldn't work either. The whole point of SDP, or TOE,
is that the API is either totally unchanged or at least
essentially unchanged.
Whenever an IP Address is used (SDP/iWARP, TOE and potentially
SDP/IB) changing from AF_INET* is wrong.
For both SDP/iWARP and SDP/IB you could argue that a different
wire protocol is in use so IPPROTO_SDP is acceptable.
That's probably the best answer as long as we are stuck
under the restriction that the selection of an alternate
stack cannot be done in the exact manner that the consumer
wants it done (that is transparently to the application).
There are even some corner case scenarios where the
application might care whether their SOCK_STREAM was
carried over SDP or plain TCP. So a protocol based
distinction is probably the least misleading of all
the explicit selection options.
IPPROTO_* should match the protocol field on the wire, which
I gather isn't different. And I'm assuming there is no standard API
defined already...
What about using a socket option?
+-DLS
> -----Original Message-----
> From: David Stevens [mailto:[email protected]]
> Sent: Monday, March 06, 2006 12:32 PM
> To: Caitlin Bestler
> Cc: Linux Kernel Mailing List; Michael S. Tsirkin;
> [email protected]
> Subject: RE: RFC: move SDP from AF_INET_SDP to IPPROTO_SDP
>
> IPPROTO_* should match the protocol field on the wire, which
> I gather isn't different. And I'm assuming there is no
> standard API defined already...
>
SDP uses the existing standard sockets API.
That was the intent in its design, and it is
the sole justification for its use. If you are
not using the existing sockets API then your
application would be *far* better off coding
directly to RDMA.
The wire protocol *is* different, it uses RDMA.
There is some justification for the application
knowing this, albeit slight ones. For example
you need to know if the peer supports SDP and
it might effect how intermediate firewalls
need to be configured.