Hi folks -
Introduction
------------
I have been working on a GPL project called Penumbra
http://penumbra.warmcat.com
The idea is simple, to allow unencrypted wireless broadcast packets to
be generated and received in userspace, while the normal traffic remains
privately encrypted as before. This allows direct intercommunication
between boxes on differently protected private networks (on the same
channel) while not disturbing or reducing security of a normal encrypted
association.
Due to the broadcast nature of the protocol, there are no station
identities at all. All broadcast packets use the same MAC and there is
no concept of addressability, at least not at the protocol level. There
are no retries at the transport protocol level either since there may be
many listeners -- situations where there are multiple listeners in
range of the broadcasts are very well served. Higher levels on protocol
can implement tunneling other protocols and retransmission requests, but
that is out of scope of what is needed in the 80211 stack.
We have a userspace daemon that allows anonymous filesharing using the
protocol, but there are many other applications possible for machines to
be able to remain secure and private for normal traffic, yet be able to
also communicate locally unencrypted when they choose without going
through the Internet, an ISP or other paid -- and metered -- gateway.
There is a start on a formal RFC for the broadcast transport protocol here:
http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt
Status
------
Currently I have this working for several wireless devices using the old
stack (IPW2200, IPW3945, and ZD1211RW -- BCM43XX is also done but does
not always receive for reasons I didn't discover), and I have it working
on the new stack for iwlwifi and RT73USB (RT2571 chipset) too now. The
old stack drivers are patched without any changes to the old stack, but
for the new 80211 stack, the additional code is mostly tucked away in
the stack. You can see examples of the patching required in both the
stack and the driver for the rt73usb case here:
http://penumbra.warmcat.com/rt2x00-cvs-2007021403-penumbra-0.0.3.patch
There is a related userspace daemon also available which communicates
via an embedded https server with the user and allows arbitrary files to
be shared around using the broadcast protocol. The daemon bridges
traffic between multiple wireless devices on different channels seamlessly.
All patches and the penumbrad daemon are available here:
http://penumbra.warmcat.com/_twk/tiki-index.php?page=Downloads
Performance at the moment for a iwlwifi <-> IPW2200 testcase is between
100KBytes/sec and 200KBytes/sec transfer, tested while downloading
simultaneously at 1MByte/sec using a WPA encrypted association to a
Belkin AP. So any questions about whether it is possible, or can work
on a range of devices, or can operate simultaneously with WPA are
answered: it works great.
Implementation
--------------
The unencrypted broadcast packets are indicated by having a "Magic MAC"
address in their IEEE80211 Header Addr<n> fields. The Magic MAC for
Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our
original choice of 11:22.. :-O ).
Here is the path for a packet using the new stack patched to support the
protocol.
- Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and
prepending an Ethernet header with the Magic MAC in it and send()ing it.
Reed-Solomon coding is optionally used in userspace in case the
receiver allows to see damaged packets.
- The stack detects the magic MAC, and prepends a fixed IEEE80211
header to the skb, basically addr1-> FF:FF:FF:FF:FF:FF, addr2 and addr3
-> 13:22:33:44:55:66 and encryption is turned off.
- The wireless driver gets the packet for transmission, recognizes the
Magic MAC, disables retries and sets the transmission rate (currently
fixed 54Mbps, but this will change) and transmits the packet as a broadcast
For receive:
- The wireless device is set to hardware promiscuous
- When an incoming packet is seen with the magic MAC it has a fake
fixed Ethernet, IP and UDP header prepended to it. IP and UDP checksums
are inserted so the packet is clean. The packet always looks like it is
coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to
255.255.255.255:61442 (port 0xF002). The packet is subject to iptables
rules as usual.
- Userspace listens on the right interface at UDP 61442 with a normal
socket and receives the packets as usual.
There is some more detail and information here (it is about the old
stack implementation currently though)
http://penumbra.warmcat.com/_twk/tiki-index.php?page=Implementation
Issues
------
I hope there is some interest in this new capability for wireless
devices in Linux. To get any kind of widespread use, the capability
needs to be already available in stock kernels and drivers so that the
user only needs to open iptables and run a userspace daemon rather than
patch his wireless drivers and stack. So this is my reason for
presenting this Request For Comments, to see if it can be considered.
Some of the issues that need working on/consensus if there is interest:
- Hardware promisc is needed to catch the packets because they are not
tagged as coming from the local AP. All the devices I looked at can do
hardware promisc separately from the logical promisc for Monitor mode.
But perhaps this needs to be enabled and disabled in some modal way.
- The way the packets are captured does not integrate at the moment
with the rx handler list concept, because it needs access to args that
do not come to the "pre" rx handlers where it has to capture. No doubt
there are multiple other implementation issues that can be hammered out
easily enough if there is interest.
- Right this second there are two patches, one for the iwlwifi case
and one for rt2x00, since they are using different versions of the new
80211 stack.
- The first byte of the payload is reserved for holding the desired TX
rate sent up from userspace, as well as indications about any ECC coding
that is present. But today only a fixed 54Mbps packet rate is
implemented. Despite all the stations are anonymous, the upper levels
of the protocol can guess the best rate to send a particular packet
depending on what has been going on, and so allow the system to work
over larger distances than 54Mbps only will allow.
... no doubt there will be more assuming there is any interest ;-)
Thanks for reading this far and I hope it will be possible by adding
these relatively small changes to the stack to enable some cool
additional uses for existing wireless devices.
-Andy
Jouni Malinen wrote:
Hi Jouni -
>> There is a start on a formal RFC for the broadcast transport protocol here:
>>
>> http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt
>
> Could you please extend the description of "Wireless Packet Format" to
> define all fields in the 802.11 header. I would especially be interested
> in seeing what is in the FromDS and ToDS fields of the frame control.
> I'm assuming this is using normal data frames, but that (type and
> subtype fields) should also be described clearly.
I extended it to this:
frame_control: 08 01 (FCTL_DATA/STYPE_DATA/TODS)
duration_id: 00 00
addr1: FF:FF:FF:FF:FF:FF
addr2: 13:22:33:44:55:66
addr3: 13:22:33:44:55:66
seq_ctrl: 00:00
The TODS will always be accepted for transmission by even a pedantic
device in INFRA mode, and any combination will be accepted in MONITOR
mode. Whereas one of the devices (IPW3945? I forget) would not issue
packets with FROMDS in INFRA mode IIRC.
>> - The wireless driver gets the packet for transmission, recognizes the
>> Magic MAC, disables retries and sets the transmission rate (currently
>> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
>
> So this frame is to be sent by the non-AP STAs directly as a broadcast
> frame, not through the AP which is the normal way of transmitting
> frames(?)
Yes -- even further there is nothing in the broadcast frame to identify
the sender or recipient. The broadcast frame is not directed at
anything either: any number of stations are either in range to accept it
or not. If you're not in range, but through one or more hops you are in
range of someone that is, a store-and-forward system that rides of top
of this enables intercommunication anyway. So your street or building
or carpark or train or airplane can be a more interesting place,
hopefully even if you are not associated or have nothing to associate to.
>> I hope there is some interest in this new capability for wireless
>> devices in Linux. To get any kind of widespread use, the capability
>> needs to be already available in stock kernels and drivers so that the
>> user only needs to open iptables and run a userspace daemon rather than
>> patch his wireless drivers and stack. So this is my reason for
>> presenting this Request For Comments, to see if it can be considered.
>
> You would need to add "and configure the wlan driver/802.11 stack to
> enable this". I don't think this has to be disabled by default.
Yes if I understood it I can bring up the monitor interface when the
daemon is started and take it down when it is stopped.
> Is this only for Linux at the moment or is someone looking at supporting
> this with other OSes?
It will really become more useful with Windows support. I looked a
little way into it and found they seem to have a hookable minidriver
idea for XP SP2 and Vista. I am hoping somebody with experience on that
side will write it once the system is easily available on Linux.
>> - Hardware promisc is needed to catch the packets because they are not
>> tagged as coming from the local AP. All the devices I looked at can do
>> hardware promisc separately from the logical promisc for Monitor mode.
>> But perhaps this needs to be enabled and disabled in some modal way.
>
> This has to be disabled by default. There is a reason for such a
> filtering in wlan designs and just enabling promiscuous mode at 802.11
> level is going to considerable increase power needs for the device which
> is not acceptable for many low-power devices.
Fair enough: it seems that creating the logical Monitor interface for rx
will and using that instead of the current system of private monitoring
with hardware promisc will take care of that as well, it will only be
doing promisc while the daemon runs and the interface is up.
-Andy
On Tue, Feb 27, 2007 at 11:48:46AM +0000, Andy Green wrote:
> There is a start on a formal RFC for the broadcast transport protocol here:
>
> http://penumbra.warmcat.com/penumbra-rfc-0.0.3.txt
Could you please extend the description of "Wireless Packet Format" to
define all fields in the 802.11 header. I would especially be interested
in seeing what is in the FromDS and ToDS fields of the frame control.
I'm assuming this is using normal data frames, but that (type and
subtype fields) should also be described clearly.
> - The wireless driver gets the packet for transmission, recognizes the
> Magic MAC, disables retries and sets the transmission rate (currently
> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
So this frame is to be sent by the non-AP STAs directly as a broadcast
frame, not through the AP which is the normal way of transmitting
frames(?)
> I hope there is some interest in this new capability for wireless
> devices in Linux. To get any kind of widespread use, the capability
> needs to be already available in stock kernels and drivers so that the
> user only needs to open iptables and run a userspace daemon rather than
> patch his wireless drivers and stack. So this is my reason for
> presenting this Request For Comments, to see if it can be considered.
You would need to add "and configure the wlan driver/802.11 stack to
enable this". I don't think this has to be disabled by default.
Is this only for Linux at the moment or is someone looking at supporting
this with other OSes?
> - Hardware promisc is needed to catch the packets because they are not
> tagged as coming from the local AP. All the devices I looked at can do
> hardware promisc separately from the logical promisc for Monitor mode.
> But perhaps this needs to be enabled and disabled in some modal way.
This has to be disabled by default. There is a reason for such a
filtering in wlan designs and just enabling promiscuous mode at 802.11
level is going to considerable increase power needs for the device which
is not acceptable for many low-power devices.
--
Jouni Malinen PGP id EFC895FA
Johannes Berg wrote:
Hi -
Thanks for your comments.
> Just a few comments. I'll leave aside the issues with 802.11 here hoping
> you've thought about that. There are issues with admission control and
> similar things, for example.
No I didn't consider these elements, because in effect the broadcasts
are no different from uncontrolled traffic from another network on the
same channel.
> First, allow me to comment on some things from other sources:
>
>> The payload is protected using a 255:223 Reed-Solomon? Forward Error
>> Correction coding. This is capable to correct any 16 symbol errors
>> over each 223 bytes of payload. To maximize the benefit of this
>> protection, bulk data packets are limited to 213 bytes of actual
>> payload (and a 10-byte header), giving a 255 byte encoded packet
>> payload so each fragment fits inside a single error correction coding
>> cycle. The Reed-Solomon? coding also means there is no need for a
>> payload CRC.
>
> 802.11 frames are always protected by a 32-bit CRC and will be discarded
> by hardware (in most cases) if that doesn't match. This is unnecessary
> overhead.
That's right that some devices are hiding frames with failed CRCs, but
it isn't true of all of them. In the case where they aren't hiding the
broken frames, the Reed-Solomon coding allows the broadcast to be
recovered in many cases. One of the bits on the first byte of the
packet is reserved to show if the sending station has ever seen a broken
packet being received, so a responding station can decide whether to use
the ECC or not in reply.
In the case where a broadcasting station has decided to "use the
airtime", it makes sense to try to recover the packet if possible since
the airtime was already blocked. I found that typical errors were runs
of bits getting trashed, the R-S coding can recover up to 16 bytes of
damage in such runs. The benefit of it is greater as the number of
listeners in range increases, or the bitrate used for the transmission
decreases (and the potentially lost airtime therefore goes up).
>> The unencrypted broadcast packets are indicated by having a "Magic MAC"
>> address in their IEEE80211 Header Addr<n> fields. The Magic MAC for
>> Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our
>> original choice of 11:22.. :-O ).
>
> How about registering a OUI or getting someone to donate a MAC address
> instead of using a locally administered one?
I contacted the IEEE about it some weeks ago, their response was
basically to tell me not to use the 11: space but this locally
administered one. They did not respond to my requests for a donation of
one measly MAC address, although I did not exhaust all the contacts
there yet. I can't afford to register a OUI myself, but I have a
customer with an allocation that can probably spare one. An advantage
of using locally administered though is that the LSByte of the MAC could
be overloaded to indicate other broadcast protocols using the same
technique, if only the first 5 bytes of the MAC are matched.
>> - Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and
>> prepending an Ethernet header with the Magic MAC in it and send()ing it.
>
> I don't see why you couldn't use the packet injection stuff we'll be
> needing anyway for userspace MLME.
Well that would be fine :-) The current implementation is just what I
could make work in a reasonable way with the old 80211 stack and then I
ported it to the new one.
The old 80211 stack method uses a kernel module that hooks the
hard_start_xmit() for network interfaces that have wireless extensions
on them, using the network interface notification API in Linux. I chose
to come at it like that because this is similar to the method that I
hope will work on Windows.
>> - The wireless driver gets the packet for transmission, recognizes the
>> Magic MAC, disables retries and sets the transmission rate (currently
>> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
>
> You'd be able to control these parameters then.
Sounds good! Does this exist yet? How does one use it?
>> - When an incoming packet is seen with the magic MAC it has a fake
>> fixed Ethernet, IP and UDP header prepended to it. IP and UDP checksums
>> are inserted so the packet is clean. The packet always looks like it is
>> coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to
>> 255.255.255.255:61442 (port 0xF002). The packet is subject to iptables
>> rules as usual.
>
> Similarly, why not have userspace use a monitor interface directly?
The answer is again because the old stack did not offer the concept of a
logical Monitor action on the same device that is associated as INFRA.
But allowing "out of the box" operation is critical so I will try this
method also. I saw it mentioned that you can create these by writing
down /sys.
>> To get any kind of widespread use, the capability
>> needs to be already available in stock kernels and drivers so that the
>> user only needs to open iptables and run a userspace daemon rather than
>> patch his wireless drivers and stack.
>
> I don't think that once we have packet injection in place for userspace
> MLME (well, we even have it now) you'll need to do any modifications at
> all. You'll just need to do more stuff in userspace. I also don't see
> why iptables should see these packets that are explicitly not IP. In
> fact, I think such packets should not be seen by the networking stack at
> all.
Well this is a perfectly fine solution for me if the need for all my
patches should evaporate and things "just work" for endusers, so I am
interested about the state of the injection stuff.
-Andy
Hi,
Just a few comments. I'll leave aside the issues with 802.11 here hoping
you've thought about that. There are issues with admission control and
similar things, for example.
First, allow me to comment on some things from other sources:
> The payload is protected using a 255:223 Reed-Solomon? Forward Error
> Correction coding. This is capable to correct any 16 symbol errors
> over each 223 bytes of payload. To maximize the benefit of this
> protection, bulk data packets are limited to 213 bytes of actual
> payload (and a 10-byte header), giving a 255 byte encoded packet
> payload so each fragment fits inside a single error correction coding
> cycle. The Reed-Solomon? coding also means there is no need for a
> payload CRC.
802.11 frames are always protected by a 32-bit CRC and will be discarded
by hardware (in most cases) if that doesn't match. This is unnecessary
overhead.
> The unencrypted broadcast packets are indicated by having a "Magic MAC"
> address in their IEEE80211 Header Addr<n> fields. The Magic MAC for
> Penumbra is 13:22:33:44:55:66 (the IEEE had something to say about our
> original choice of 11:22.. :-O ).
How about registering a OUI or getting someone to donate a MAC address
instead of using a locally administered one?
> - Userspace transmits by creating a PF_PACKET / SOCK_RAW socket and
> prepending an Ethernet header with the Magic MAC in it and send()ing it.
I don't see why you couldn't use the packet injection stuff we'll be
needing anyway for userspace MLME.
> - The wireless driver gets the packet for transmission, recognizes the
> Magic MAC, disables retries and sets the transmission rate (currently
> fixed 54Mbps, but this will change) and transmits the packet as a broadcast
You'd be able to control these parameters then.
> - When an incoming packet is seen with the magic MAC it has a fake
> fixed Ethernet, IP and UDP header prepended to it. IP and UDP checksums
> are inserted so the packet is clean. The packet always looks like it is
> coming from UDP 0.0.0.0:61441 (port 0xF001) and is directed to
> 255.255.255.255:61442 (port 0xF002). The packet is subject to iptables
> rules as usual.
Similarly, why not have userspace use a monitor interface directly?
> To get any kind of widespread use, the capability
> needs to be already available in stock kernels and drivers so that the
> user only needs to open iptables and run a userspace daemon rather than
> patch his wireless drivers and stack.
I don't think that once we have packet injection in place for userspace
MLME (well, we even have it now) you'll need to do any modifications at
all. You'll just need to do more stuff in userspace. I also don't see
why iptables should see these packets that are explicitly not IP. In
fact, I think such packets should not be seen by the networking stack at
all.
johannes
On Thursday 01 March 2007 16:11, Andy Green wrote:
> Hi folks -
>
> As suggested here I migrated my daemon to using a per-physical device
> monitor mode interface it creates and destroys from usermode, together
> with libpcap for filtering and receiving broadcasts and that seems to be
> working okay. So that is half the battle to get the broadcast system
> working with mac80211 out of the box without patches.
>
> However, if I inject a captured packet - with a PRISM header prepended -
> down the Monitor mode interface using pcap_inject(), I see it appear on
> the libpcap captured output okay, but it does not go out on the air AFAICT.
>
> Is this in fact the correct way to try to inject a packet with fine
> control over its encryption and rate? If so is it expected to currently
> work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?
Note that rt73usb is currently broken, I know that RX isn't working
because scanning fails. Chances are quite high that TX is failing too.
I am still looking into that issue, but it might be the reason why
packet injection is failing for you.
Ivo
Ivo van Doorn wrote:
> On Thursday 01 March 2007 16:11, Andy Green wrote:
>> Hi folks -
>>
>> As suggested here I migrated my daemon to using a per-physical device
>> monitor mode interface it creates and destroys from usermode, together
>> with libpcap for filtering and receiving broadcasts and that seems to be
>> working okay. So that is half the battle to get the broadcast system
>> working with mac80211 out of the box without patches.
>>
>> However, if I inject a captured packet - with a PRISM header prepended -
>> down the Monitor mode interface using pcap_inject(), I see it appear on
>> the libpcap captured output okay, but it does not go out on the air AFAICT.
>>
>> Is this in fact the correct way to try to inject a packet with fine
>> control over its encryption and rate? If so is it expected to currently
>> work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?
>
> Note that rt73usb is currently broken, I know that RX isn't working
> because scanning fails. Chances are quite high that TX is failing too.
> I am still looking into that issue, but it might be the reason why
> packet injection is failing for you.
The 2007-02-14 rt73usb CVS works okay for RX, it does iwlist wlan0 scans
alright here anyway with the AP shown. I have had that version issuing
unencrypted broadcasts too by hacking the stack and the driver slightly.
Maybe it was some Valentines Day magic :-) But it doesn't associate
in a stable way with the AP using WPA on wpa_supplicant with the dscape
patch.
Anyway it seems the problem is one does not inject for TX on the Monitor
mode interface (/me goes away to grep and google what a Management mode
interface does, thanks Johannes)
-Andy
Hi folks -
As suggested here I migrated my daemon to using a per-physical device
monitor mode interface it creates and destroys from usermode, together
with libpcap for filtering and receiving broadcasts and that seems to be
working okay. So that is half the battle to get the broadcast system
working with mac80211 out of the box without patches.
However, if I inject a captured packet - with a PRISM header prepended -
down the Monitor mode interface using pcap_inject(), I see it appear on
the libpcap captured output okay, but it does not go out on the air AFAICT.
Is this in fact the correct way to try to inject a packet with fine
control over its encryption and rate? If so is it expected to currently
work on iwlwifi 0.0.8 or rt2x00 (rt73usb) from CVS 2008-02-14?
-Andy
On Thu, 2007-03-01 at 15:11 +0000, Andy Green wrote:
> Is this in fact the correct way to try to inject a packet with fine
> control over its encryption and rate?
Right you need to create a management interface and dump it into there I
think. Subject to change, still under discussion though.
johannes