This post deals with the technical challenges we’ve encountered while trying to establish reliable communications while staying in Rural Kenya. Some background information is necessary to understand the efforts we’ve gone through to remain connected.
This year the prestigious Hacker Beach event is taking place on the island of Lamu off the eastern coast of Kenya. The island is serviced by a single UMTS tower located above the hospital in the main town of Lamu City. However, our accommodation is on the other side of the island.
Our accommodation had a previously installed directional antenna on the roof to provide internet access. Unfortunately the access was very slow, with only 14% signal strength. This was complicated by strong winds blowing against antenna, causing it to be pointed in a wrong direction. This further reduced the cellular reception, sometimes making it disconnect completely.
An additional problem was that WiFi was served by a single Cradlepoint MBR1000 router in a corner of the fort, making it inaccessible through the impenetrably thick fort walls. This meant we were limited to camping in the upstairs dining hall, which worked well enough due to all the seating, but there was some desire to branch out to work from other areas of the fort, such as the knights-of-the-round-table-esque meeting room.
For a group of 18 hackers, this level of connectivity was unacceptable. Many of us were making excursions into town to work at cafes with better reception. This was a problem because it threatened to undermine the spontaneous collaborative nature of Hacker Beach. The way we saw it there were two problems to fix:
- Reception of the antenna was abysmal. Was this an inherent problem with the location?
- WiFi reception was limited to only one corner of the house. Ideally the house should have WiFi everywhere.
We attempted to fix this by purchasing local SIM cards and installing them in portable WiFi Hotspot devices. Oddly enough we were able to receive some 3G reception if the devices were placed in some rather random areas of the fort. Unfortunately the connectivity of these devices wasn’t reliable enough for full-time hacking. So we began efforts in earnest to fix the connectivity problem.
We determined that the most appropriate solution to the WiFi problem was to employ PowerLine Ethernet adapters throughout the Fort to distribute connectivity. Simply repeating wireless signal was not a good option because of the lack of strategic locations to place wireless repeaters. The thick walls meant that the signal would be stopped between floors as well. We took a gamble and assumed that most outlets would be on the same power phase (if circuits are on different phases the PowerLine throughput will be severely limited, or likely not work at all). Since we had some new hackers approaching in a few days shipping was out of the question. Thankfully we were able t source some units in Athens, which (after some begging) our gracious friends were kind enough to pick up and bring for us.
The pairing part was easy, with WiFi SSID/password being copied using WPS. After pairing the devices could be moved anywhere in the fort to increase coverage. We installed two devices which are able to blanket the whole fort with connectivity. Problem solved.
Next was a trickier bit that required more calibration and special equipment. While inspecting the old antenna we found that the connectors had been tortured by the elements for several years. This meant that the antenna pigtail connectors were rusted, which was likely causing reception issues. Another problem was that the pigtail was being run through a window, which was then closing on it. We feared this was crushing the cable, which could have easily caused our antenna to become useless.
There were several more hackers arriving from Nairobi in a few days, so we asked them to bring some antenna gear to hopefully help improve our connectivity. In total a questionably-EDGE amplifier, directional antenna, and some cabling was delivered when the hackers arrived early yesterday morning. It didn’t take long for us to tear it all open and start installing it.
Equipped with a laptop, an antenna, and a downstairs accomplice we disconnected the old antenna and threw a new line down to connect to the 3G modem. Next I had opened the router’s modem status page to measure signal strength while another hacker determined the direction the antenna should face to get the best reception. Our best direction was pressed against the old antenna; the people who installed the last one must have known what they were doing.
Unfortunately we were only armed with my multitool which meant that proper mounting was going to be impossible. We tried wrenching the existing nuts that held the antenna in place, but they proved to be well stuck with a decade of rust and generally brutal African elements. Not even cooking oil (our improvised WD-40) would help loosen the offending nuts. Ultimately we ended up doing a bodge job to keep the antenna in place. One of the hackers had brought string with him, which we used to tie the new antenna’s base plate to the old antenna. This worked surprisingly well, although is a horrifyingly temporarily solution. The string will not stand up to more than a few days outside here. Next we plan to source some tools locally and perform a permanent installation of the antenna.
With the old antenna the signal strength would consistently be about 14%, which resulted in throughput of about 200 kbit. After out new antenna was installed and calibrated we were able to see signal strength of up to 80%, which gave us upwards of 1800 kbit of throughput with consistent pings of about 250 ms. Hooray!
After applying liberal traffic shaping on the router we are now able to comfortably surf the internet, download packages, and use IRC.11.21.13
I’ve been as terrible about updating this as I expected to be. Nevertheless, I am struck with inspiration (or maybe it’s just energy from coffee), so another post must be written!
For the next week (and the previous week) I’m spending time in Paris. This turned out to be largely a convenient set of circumstances, since I had an excellent experience when I was here two weeks ago, and I wished I could spend more time here.
Read the rest of this entry »
I’m excited to hear that Hackerbeach will be happening again this year. Last year was an amazing and unique experience, and I can’t wait to go again. This year the village hosting us will be Lamu, Kenya.
For the uninitiated, Hackerbeach involves a group of hackers (historically 15-20) gathering in a tropical location for a month to hack on various open source projects. It can be thought of as a month-long hackathon or code sprint for nomadic open source developers. All of the code so far has been focused on the open web ecosystem.09.28.13
Writing this log reluctantly at the request of a coworker. last time I tried while travelling resulted in three large rambling posts (only 2 of which were published). This time I’ll try to write smaller posts of a more personal nature.
Woke up with a sore throat. A bad omen for a long travel stint. I hadn’t packed the night before, so it was all done day-of, which surprisingly didn’t result in me not packing important items (that I can think of yet). Maybe I’m getting used to this, or maybe I’m just being more reserved in what I consider necessary.05.22.13
Flying in India is a bit different than in other countries I’ve flown. All processes are more strict than at least southeast Asia, Europe, and the US. Some of the differences include the amount of documentation required. Read the rest of this entry »05.22.13
Over the past week I’ve spent in Bangalore, I’ve come to appreciate the humble auto rickshaw, or ‘auto’ for short. For the uninitiated they are similar to a Tuk Tuk in other countreies. That is to say they are small three-wheeled covered vehicles used to ferry two to three passengers around busy streets and through traffic. This is half due to their diminutive size and the other half due to their driver’s propensity for disregarding both traffic laws and courtesy to their fellow road-sharers. Bangalore has the busiest (and narrowest) streets in India, and the auto drivers are notorious for trying to take advantage of unwitting customers.06.7.12
After notifo seemingly stopped pushing notifications to my devices, I began to look for an alternative that more transparently used Google’s C2DM service for notifications. It didn’t take me long to find NMA, short for NotifyMyAndroid. This is a very simple http(s) API to push notifications to one/many devices. I’ve broken this down into a few easy steps.
- Sign up for an account on www.notifymyandroid.com. Log in, click on ‘My Account‘, then ‘Generate New Key’
- Install the NMA Android app (around $3 USD) onto your android device. Launch the app and login
- Install pynma.py and nma.py to $HOME/.weechat/python.
- (Optional) Symlink nma.py to $HOME/.weechat/python/autoload
- In weechat:
/python load python/nma.py /set plugins.var.python.nma.apikey "$myapikey"
- (Optional) To inly send notifications when detached from the screen: install and load the screen_away.py, then
/python load python/screen_away.py /set plugins.var.python.nma.only_away on
Now you too can be harassed by inane IRC highlights no matter where in the world you are! Thankfully, the NMA Android app has a ‘quiet hours’ in case you’d rather not be woken up in the middle of the night.06.6.12
As an exercise on how lightweight and extensible LXC can be I decided to create a large installation of LXC containers on my laptop. I wanted to see if LXC would allow me to create large-scale testing environments often needed for testing software with several moving parts.12.24.11
Several people have asked me recently how I forward IRSSI notifications to my desktop and Android devices. The solution could be basically described as follows:
Install these scripts into $HOME/.irssi/scripts (and symlink it in $HOME/.irssi/scripts/autorun) to have them autoloaded when IRSSI starts.
Sign up for a notifo account, download the android app, and sign in with it. Make note of the username and API key, as you’ll be configuring irssi to use them.
You’ll be loading the scripts in IRSSI and setting 2 notifo parameters:
/script load screen_away.pl
/script load socketnotify.pl
/script load notifonotify.pl
/set notifo_username $USER_NAME
/set notifo_api_secret $API_KEY
Now receiving highlights while you’re away (detached from the screen) will send them to your android device.
Add an entry similar to this in your $HOME/.ssh/config file (creating it if it doesn’t exist yet). This will forward your traffic being received by the IRC server (on localhost) on port 4443 to your SSH client on port 4443 (IE: your laptop):
RemoteForward 4443 localhost:4443
Now all we need is a daemon running on your client to listen on localhost:4443 and receive the messages, then insert them into some sort of notification service. My method for doing this is to have a script in $HOME/bin run in a screen and display them using pynotify.
I’ve written a script that I call notifywatch that does exactly that. Save it to $HOME/bin/notifywatch.py and mark it executable.
Add screen -d -m $HOME/bin/notifywatch into your startup scripts, and desktop notifications should work as well (even while connected to your IRSSI screen)!04.22.11
This morning at Mozilla, we had out first ganeti node failure, which meant we got to learn how to deal with a node failure! So far all of our ganeti clusters have been 2 node clusters, so while relatively simple, it meant an extra quirk in bringing everything up. Thankfully we used the iallocator so as to not overallocate nodes on the cluster. Here’s how the process went.