When I initially received my new X1 Carbon I was very excited at the new hardware. More pixels, more lumens, more cores.

After I got over my initial euphoria, I noticed that some things didn’t work, and confirmed such by a very helpful Archwiki page.

Some of the things that didn’t work include ACPI suspend-to-RAM sleep (since corrected in a BIOS update), the the fingerprint sensor (still outstanding), and the WWAN LTE Cat9 card that Lenovo seems so proud about. It has all the LTE channels you could ever want and can achieve speeds of up to 150Mbit. It’s a pity that the Linux drivers are not up to snuff.

It was doubly disappointing because I had gotten used to the always-connected nature of having a WWAN card in my primary machine.

The WWAN card included in USA-bound X1 Carbons is a Fibocom L580-GL. The card itself is primarily comprised of an Intel modem and chipset, which does have support in the Linux kernel (it is used in several Android phones). The reason it is unsupported on Linux is because of the interface they chose to use. Normally WWAN cards electrically use a USB interface, even if they are primarily found in a M.2 form factor inside a laptop.

To support the higher data rate Lenovo decided to use the card’s PCI Express interface. This interface does not have support in the Linux kernel. Although the card itself enumerates (via lspci) as a device on the PCI bus, no driver will bind to it. Hopefully someday someone will boost the Linux support.

In an attempt to get it to work, some enterprising owners even taped over the PCI Express pins on the card so that it would force connection over USB. Unfortunately this reveals another problem: Lenovo’s card whitelist. Lenovo disallows any card that is not in a list embedded in their signed system firmware.

My online research had revealed that some folks had luck using some Sierra Wireless cards that had been used in previous ThinkPad models as well as other Dell laptops.

So I did it. I went on eBay and purchased a Sierra Wireless 7455 LTE WWAN card. I removed the original Fibocom part and put it in. Unfortunately I was greeted with the very same whitelist screen.

Typical whitelist error message

I knew from having previous Sierra Wireless devices that when running correctly it produced several serial console devices (/dev/ttyUSB[0-2]). Details in a Reddit post showed which commands I would have to run to make it work. The commands would alter the USB vendor and product IDs and set “fastboot” behavior for the card.

The trick to bypassing the whitelist protection turned out to be a timeout. The card spend so long booting that the whitelist protection in the laptop’s system firmware times out.

I spent the better part of a jetlagged morning trying to find any device in my house with a M.2 Key B slot. It turns out I had none other than the X1 Carbon discussed in this article. Frustrated, I went back to sleep and had a think.

At some point in my rest I had an epiphany: if the card itself simply interfaces via USB, couldn’t I hotplug it after the laptop had booted?

I immediately got up and gave it a shot: I removed the Fibocom card, booted the laptop, ran journalctl -xf to see log output, and closed the lid to put it to sleep. Then I took the bottom panel off and slotted in the new Sierra Wireless card, attaching both of the antennas.

I closed it back up and opened my laptop. I watched delightfully as the card enumerated itself and was recognized by my system.

I proceeded to flash the card via instructions that a very helpful person wrote, including links to native Linux binaries! Afterwards I reloaded the card, connected to the card’s serial console, and ran the AT commands from the earlier-mentioned reddit post. This disabled the card’s fastboot from cold system boot (but enabled for a warm-boot), enabled LTE (disabled low-power mode), and set the USB IDs.

I crossed my fingers and rebooted my system. To my relief it Just Worked (TM). My system booted with nary an error and I was soon back in my regular desktop environment. I was able to add the card to NetworkManager, and was online. I did a SpeedTest and found the result to be perfectly sufficient: 60/30Mbit. Not as good as the original Fibocom card, but I’ll take it.

Given that we got this card to work by simply relying on slow card bootup to bypass the whitelist, I wonder if the same can be done with the original Fibocom card. If we simply taped over the PCI-Express pins to force USB, and inserted the card after boot, could we also find a setting to slow boot enumeration so that it would cause the whitelist checking to timeout? Further research is needed.

Attending bkero

I’m looking forward to attending the 2018 IndieWebCamp. It’s a small 2-day event happening in Portland and is exploring the topics of independent web hosting and technologies to knit them together.

If you’re in Portland, you should attend too!


Your old pal syslinux is there to greet you
Your old pal syslinux is there to greet you

After initially running RemixOS, the new Android build for PCs, I decided that I would rather play with booting it natively from my SSD instead of from a USB device. Performance should be better, it would free my USB thumb drive up for other duties, and it would make booting more convenient.

This turned out to be a relatively simple operation. What follows is my methodology for doing that. Please note that these instructions assume you are running Linux.

Continue reading “Installing RemixOS to an internal drive”


Whenever I encounter people as I travel, they are often curious about my luggage. It seems to be invisible. They’ll often ask where my bag is, assuming that it must have gotten lost in transit. Their eyes go wide and confusion sets in when I tell them that the bag on my back is the only one.

It is my estimation that at least some people would be curious about what gear I travel with. They ask how I’m able to pack all the necessities into such a small space. There is no great secret to traveling light. All it takes is a little research and compromise in creature comforts. If you have browsed the postings of other nomadic hackers, there might be little to be gleaned from this post. Here’s a basic rundown, with almost each article deserving its own article.

It should go without saying that nobody paid for me to write this post, and likewise nobody as sent me any products to test.

Continue reading “My Gear Post”

In many parts of the world a reliable internet connection is hard to come by. Ethernet is essentially non-existent and WiFi is king. As any digital nomad can testify, this is ‘good enough’ for doing productive work.

Unfortunately not all WiFi connections work perfectly all the time. They’re fraught with unexpected problems including dropping out entirely, abruptly killing connections, and running into connection limits.

Thankfully with a little knowledge it is possible to regain productivity that would otherwise be lost to a flaky internet connection. These techniques are applicable to coffee shops, hotels, and other places with semi-public WiFi.

Continue reading “Working around flaky internet connections”

One idea that’s been bouncing around in my head for the last few years has been a laptop with an E-Ink display. I would have thought this would be a niche that had been carved out already, but it doesn’t seem that any companies are interested in exploring it.

I use my laptop in some non-traditional environments, such as outdoors in direct sunlight. Almost all laptops are abysmal in a scenario like this. E-Ink screens are a natural response to this requirement. Unlike traditional TFT-LCD screens, E-Ink panels are meant to be viewed with an abundance of natural light. As a human, I too enjoy natural light.

Continue reading “Attempts source large E-Ink screens for a laptop-like device”

A week ago I was fortunate enough to attend the latest code sprint of the Mercurial project. This was my second sprint with this project, and took away quite a bit from the meeting. The attendance of the sprint was around 20 people and took the form of a large group, with smaller groups splitting out intermittently to discuss particular topics. I had seen a few of the attendees before at a previous sprint I attended.

Joining me at the sprint were two of my colleagues Gregory Szorc (gps) and Mike Hommey (glandium). They took part in some of the serious discussions about core bugfixes and features that will help Mozilla scale its use of Mercurial. Impressively, glandium had only been working on the project for mere weeks, but was able to make serious contributions to the bundle2 format (an upcoming feature of Mercurial). Specifically, we talked to Mercurial developers about some of the difficulties and bugs we’ve encountered with Mozilla’s “try” repository due to the “tens of thousands of heads” and the events that cause a serving request to spin forever.

By trade I’m a sysadmin/DevOps person, but I also do have a coder hat that I don from time to time. Still though, the sprint was full of serious coders who seemingly worked on Mercurial full-time. There were attendees who had big named employers, some of whom would probably prefer that I didn’t reveal their identities here.

Unfortunately due to my lack of familiarity with a lot of the deep-down internals I was unable to contribute to some of the discussions. It was primarily a learning experience for me for both the process which direction-driving decisions are made for the project (mpm’s BDFL status) and all of the considerations that go into choosing a particular method to implement an idea.

That’s not to say I was entirely useless. My knowledge of systems and package management meant I was able to collaborate with another developer (kiilerix) to improve the Docker package building support, including preliminary work for building (un)official Debian packages for the first time.

I also learned about some infrequently used features or tips about Mercurial. For example, folks who come from a background of using git often complain about Mercurial’s lack of interactive rebase functionality. The “histedit” extension provides this feature. Much like many other features of Mercurial, this is technically “in core”, but not enabled by default. Adding a line in the “[extensions]” section your “hgrc” file such as “histedit =” enables it. It allows all the expected picking, folding, dropping, editing, or modifying commit messages.

Changeset evolution is another feature that’s been coming for a long time. It enables developers to safely modify history and be able to propagate those changes to any down/upstream clones. It’s still disabled by default, but is available as an extension. Gregory Szorc, a colleague of mine, has written about it before. If you’re curious you can read more about it here.

One of the features I’m most looking forward to is sparse checkouts. Imagine a la Perforce being able to only check out a subtree or subtrees of a repository using ‘–include subdir1/’ and –exclude subdir2/’ arguments during cloning/updating. This is what sparse checkouts will allow. Additionally, functionality is being planned to enable saved ‘profiles’ of subdirs for different uses. For instance, specifying the ‘–enable-profile mobile’ argument will allow a saved list of included and excluded items. This seems like a really powerful way of building lightweight build profiles for each different type of build we do. Unfortunately to be properly implemented it is waiting on some other code to be developed such as sharded manifests.

One last thing I’d like to tell you about is an upcoming free software project for Mercurial hosting named Kallithea. It was borne from the liberated code from the RhodeCode project. It is still in its infancy (version 0.1 as of the writing of this post), but has some attractive features for viewing repositories, such visualizations of changelog graphs, diffs, code reviews, a built-in editor, LDAP support, and even a JSON-RPC API for issue tracker integration.

In all I feel it was a valuable experience for me to attend that benefited both the Mercurial project and myself. I was able to lend some of my knowledge about building packages and familiarity with operations of large-scale hgweb serving, and was able to learn a lot about the internals of Mercurial and understand that even the deep core code of the project isn’t very scary.

I’m very thankful for my ability to attend and look forward to attending the next Sprint in the following year.