Note

• We're going to cover a lot today
• If you've ever wanted to know what building a kernel is about, or why you might want to do it, you're in the right place.
• Prior knowledge: Linux command line, installing packages, general compiling commands like 'make'.

Note

• https://www.youtube.com/watch?v=RaW-opIh7GE
• It's the "system command processor...It's the brain!"
• It's the first program that runs once your computer is booted
• It primarily consists of an image file and (optionally) many kernel module files, each adding new features

Note

• The kernel is responsible for handling device drivers and providing resources for user space
• Device drivers are the pieces of software that make all the separate hardware components work
• It also handles memory management. From a user's perspective that means your program can simply request a chunk of memory from the kernel. It doesn't have to worry about where that memory comes from, or if it could be inadvertantly used by another program.
• Maybe most importantly it provides services to userland. These are things like a filesystem to organize your files, network connections to be able to communicate between programs and to other systems, and scheduling.
• There are many schedulers for resources that run better when ordered. Popular examples are a disk scheduler, which has a queue of read/write requests, and can reorder them to keep a hard drive from thashing, and a process scheduler, which allows your computer to share the processor between multiple processes. That's very useful.

Note

• These are the files of the Linux kernel that live on your disk
• The first is the linux image itself. This is the image file that our bootloader runs to start our system.
• The 'vm' is a throwback to old Unix, when the kernel was called /unix. This was before virtual memory. When virtual memory was introduced, kernels that supported it were prepended with vm-. Thus, vmlinux. The 'z' at the end means it's compressed. This has some advantages to booting on systems with low memory, but is now the standard for x86 PCs.
• The /lib/firmware directory is a place for firmware blobs to live. These blobs are required to initialize a lot of the hardware pieces on your system. These make me grumble because although they're released under a free license, they're difficult to understand or reverse engineer.
• The /lib/modules directory contains all the kernel modules and a few more files. Besides the image itself and firmware blobs, everything that is responsible for the runtime of the kernel lives in here. If you compiled everything in, this is optional.
• We'll get into where these files come from and how to make them ourselves later

Note

• These are some reasons for building a kernel.
• Your distribution might not package the version of the kernel you want. If you want a newer version for trying new features, or have hardware that relies on an older kernel, you might want to consider it.
• Custom kernels are smaller in size and in some cases can be significantly faster. If you're in a situation where performance or footprint are critical, it can be useful to build a custom kernel.
• You might also need to build a new kernel if the distribution-provided one doesn't have the feature that you need. Sometimes you'll be able to simply compile a new module into your existing kernel, but not always.
• You could run across a bug in an existing kernel and would like to try to fix it, or have a new piece of hardware that you'd like to try to add support for.

Note

• Scenario: You just bought a new laptop. You install Fedora on it, and are disappointed to find that WiFi is not working out of the box. You read online that card is supported under Linux, but must be used with a separate manufacturer-supplied driver.
• Your first order of business is to install the linux headers for your particular kernel version. You can usually grab these from your districution as a package called linux-headers (or maybe linux-source). These are necessary for your external modules to know how to build against your kernel.
• You'll also need to go to your manufacturer's web site to download the driver source. This part can vary between manufacturers, but it should come with a README file. Follow the README.
• You'll also need to install the driver after successfully building it.
• Let's look at how to do each step next.

Note

• First we install our headers
• Then we grab our wifi driver from the manufacturer's site
• Then we unzip the driver source
• Afterwards we run it's make script. Sometimes you have to include the "-C" argument to the kernel you want to build against.
• Finally you run make install to install the module.
• To test it out, you should try to modprobe it. Silent usually means it succeeded. Otherwise it's probably an error. You can verify by checking your kernel ring buffer (dmesg)

Note

• First we'll need to grab the kernel source. We can do this a few days. The first way is to do a simple git clone. You can either use Linus's official repository, or his Github mirror (github.com/torvalds/linux).
• Or you can point your web browser to kernel.org and grab the source from there. Afterwards you need to unzip it. If you're only doing a one-off kernel it might be worth grabbing the source.
• If you're going to make a hobby of this, maybe grab the git repository instead. It'll be bigger, but is easier to keep up to date. It's also your choice if you want code newer than any release and you like to live on the bleeding edge.

Note

• Here's a small interactive portion. This is what a config file looks like. $(zcat /proc/config.gz). It's just a bunch of OPTION=(|Y|N|M)
• Even though it's named oldconfig you can give it any config, and it will massage it to work.
• Defconfigs are existing configurations that exist inside the linux source tree. They're located in arch/$ARCH/configs/. The x86 one is pretty generic, but the arm ones are really useful. This one is called corgi. I don't know what that is, but it's probably a little SBC board.

Note

• You need build-essential for a compiler. The requirements list is very small
• Do a demo of make menuconfig

Note

• An initramfs is a small filesystem that's used to boot your system. It's all contained in a single file. Older versions used a filesystem, newer versions use an archive format like zip or tar called cpio.
• cd $HOME/kernel
• zcat initramfs-linux.img | cpio -idmv

Note

• First we need to fetch and build it. Apt-get, the package manager for Debian and friends (including Ubuntu) has a feature called build-dep, which installs the build dependencies for the package we specify. In this case that's the linux kernel.
• Then the next command, apt-get source downloads not only the source for the package, but the requirements for building the package.
• demo SSHing in to bkero-general, cd linux; debuild -b -i -uc -us; ls *.deb

Note

• You can use the install target for installing the kernel image, or if you like as little 'magic' as possible you can simply copy the kernel image from its directory into /boot. This is ideal if you want to change the filename, as by default it's simply 'vmlinuz'.

Note

• This is what a snippet from grub.cfg. This is grub's primary config file, and where autodetected kernels appear. You can also manually add things here, but if you run grub-mkconfig afterwards, your manual changes will be overridden. You might have more luck putting things in /etc/grub.d/40_custom

Note

• You're going to want to test that everything works as expected. The most common things to fail are graphics, networking, hard disk, and input devices.
• If this is a mobile device, you'll want to make sure that your power management works as expected, or else you'll set your lap on fire and burn through your battery in a matter of minutes. This can be done with tools like working sensory organs in combination with tools like powertop
• The first time doing this is always the most difficult. But you should have the old kernel to fall back on, so you should never be without a working system.

Note

• First you'll be setting up email to work locally. There are instructions on KernelNewbies.org about doing this, and you can even set up your local system to work with GMail. This is because all patch submissions are done via e-mail to a mailing list.
• Next you'll obviously be needing a local copy of the code in a git repository. We've covered this. Git is out of the scope of this talk. I hope you either have used it before or have attended an introductory talk here at SeaGL and are looking forward to trying it out.
• Next you edit a file that you'd like to change. To demo here we're going to edit the e1000, which is a driver for the very common Intel Ethernet chip.
• Afterwards you'll need to either compile your standalone module, or compile (and boot into) a new kernel if the symbols that your module needs aren't available.
• Now you'll need to test your changes. This is easier if you're working on hardware that you actually own and can test. Believe it or not, most kernel developers don't actually own the hardware that they build and fix drivers for. Print debugging helps a lot here. That is, using the printk() call to print messages to the kernel ring buffer about the state and events relating to the driver.

Note

• cd code/linux
• zcat /proc/config.gz > .config; make modules_prepare
• ls; cd drivers; ls; cd net; ls; cd ethernet; ls; cd intel; ls; cd e1000; ls
• vim e1000_main.c
• Edit the driver ID line to say 'HELLO WORLD'
• cd /usr/lib/modules/uname -r/build
• make modules M=/home/bkero/code/linux/drivers/net/ethernet/intel/e1000/ (be sure it ends with a trailing /, if you're running the kernel from the dir, can cd to that one)
• sudo modprobe -r e1000
• sudo insmod the resulting file
• run dmesg, observe new output
• SAY okay, we like our "bug fix", now to use the kernel's checkpatch script to see if there's anything we can fix
• cd code/linux; git diff | scripts/checkpatch.pl
• Okay, there are some things to fix

Note

• To comprehensively cover distribution would require more time than we have, so we're only going to go over the basics.
• Sorry you guys, but I just wanted to cover the systems I'm most familiar, and what I feel might be the most popular

Note

• You'll always hear these referred to as PPAs, not the longer name.
• If you're familiar with debian you can think of these as some magic around repo files
• The model has users uploading source (control) files, launchpad takes care of the building and publishing of it
• As we'll see there are command-line helpers to do this, namely dput

Note

• First you need to upload your key to keyserver.ubuntu.com. You can do this through gpg or through the web interface.
• Next you'll need to download, sign, and upload Ubuntu's code of conduct
• Then you'll need to log into the web interface (launchpad.net) to create the PPA on their end
• Afterwards you'll run debuild -k$KEY -S to build and sign the source packages
• Finally you'll execute dput and tell it what to upload, and which PPA on launchpad it should go to
• Ubuntu will then dutifully build and test the packages, then advertise and publish them

Note

• KernelNewbies.org is a site full of resources for people who want to improve their kernel hacking skills, or to get involved with the Linux kernel community.
• Kernel Janitors has a TODO list of things that can be done organized by the skills required. That means that even though you might not be the world's best C hacker yet, you can still contribute. The site has gone through a migration, but didn't really fare well. It still has an active mailing list though.
• LKML is the Linux Kernel Mailing List. For new contributors this is mostly a lurk-only list. It's had a reputation of not being the most friendly place in the world.
• Lastly there's a most excellent scripts/ directory inside the linux kernel source.

Note

• Submitting kernel bugs can be done a few places. First is to the Linux kernel bug tracker itself. Read KernelNewbies to find out how to format a bug for submission. This usually involves creating a bug and attaching your dmesg, kernel version, and kernel config, along with any other context or resources that might be valuable in reproducing.
• You can also submit to your distribution's bug tracker. For CentOS, RHEL, and Fedora you can submit to bugs.redhat.com, and an engineer will investigate and might open a bug on bugs.kernel.org on your behalf. Launchpad is used for Ubuntu, and the first thing they're going to tell you to do is to try reproducing your problem on a vanilla kernel (that is, a kernel without their patches). If you can't, then it's an Ubuntu problem. If you can, then they'll likely tell you to go submit a patch upstream.