bl k_main

Hello!

Welcome to my blog. Over the past week I’ve become completely obsessed with how to write a kernel for the raspberry pi. Please keep in mind, I do not necessarily mean writing a complete OS. I am merely interested in understanding low-level embedded programming concepts. (merely, ha!)

To help others who face the same issue as me, I hope to document my journey. I’ve already learned a lot – some of which I will document here! It’s quite easy to find “bare bones” “linux” kernels for the raspberry pi, but if you’re interested in going deeper down the rabbit hole than that – then hopefully I can help you understand the fundamentals necessary to achieve your goals!

My background is actually web software. I specialize in C#/ASP.NET MVC and a handful of JavaScript frameworks. So taking on the beast of kernel writing (complicated even more by the lightly documented status of the Raspberry PI!) is quite a challenge indeed. My goal is to learn, so I will be discovering things at the same time you are!

I am doing this in my spare time and for fun, so don’t expect constant maintenance. But hopefully together we can experience some of the fundamental concepts necessary to achieve bare-metal programming!

Advertisements
Tagged with:
Posted in Introduction

Getting Started

I figured I would spend a bit of time explaining the environment you should use when developing for the raspberry pi. Let’s start by explaining my current development environment:

As a C#/ASP.NET developer, I pretty much have to have Windows. And as a bleeding-edge adopter, I pretty much have to have Windows 8! That being said, I’ve discovered that Linux is SO MUCH BETTER for the type of developing that we will be doing. If you already have linux, that’s great. If not – I recommend you look at the following:

 

General Software

===================================

Virtual Box – This software allows you to emulate other operating systems within your host environment. for example, I can use VirtualBox to emulate linux while running Windows 8. I highly recommend this flavor of emulators, because it’s free and very reliable. https://www.virtualbox.org/

Crunch Bang (#!) Linux – Personally, I am a fan of Arch Linux. But please note: the raspberry pi is 32bit. This means you have to make sure that you can compile 32 bit applications. Since Arch is only 64 bit, you will have a hard time setting everything up properly. Personally, I gave up on trying to be cool and opted for an already-32-bit operating system. http://crunchbang.org/download/

Linux Applications

===================================

Here is a list of software I recommend using with linux for raspberry pi development.

GIT –   Although not necessary, this is a really useful tool for downloading source code examples including the latest raspberry pi firmware! The command for this:   sudo apt-get install git

WGET – I don’t remember if this comes with crunchbang or not. I use it all the time because if you know the url, it’s usually faster and easier to download to specific locations.   sudo apt-get install wget

The ARM GCC Toolchain – There are many options for compiling ARM-Compatible code. I randomly chose this one and it’s served me well. All of my code will be compiled using this compiler, so all bets are off if you try to use my scripts with other toolchains! Download information can be found here: https://pixhawk.ethz.ch/imu_toolchain/sourcery

That’s pretty much it! I think that’s all I’ve had to download in order to start developing. Keep in mind this list has evolved over time. I’ve tried multiple compiler toolchains, different operating systems, etc. This is what I’ve settled on for now.

Posted in Introduction

Hello, kernel! Part 2

Now that you have your project structure, let’s take a look at the tools we will use to create our final image.

Compiler

As discussed earlier, I use the arm-none-eabi-* compiler. This is a “tool chain” which means if you followed my download instructions, you will have a variety of different programs that all start with that prefix. This tool chain includes a C Compiler, C++ Compiler, Assembly Compiler, Dis-assembler, and a few other things. So how the heck do we get access to all that neat stuff? Well, it’s pretty complicated and well documented. To setup your environment, I recommend following some of this tutorial: http://hertaville.com/2012/09/28/development-environment-raspberry-pi-cross-compiler/

NOTE: That post shows you how to install a different tool chain than I use. You can try it if you want, but the important thing to glean from that site is the PATH setup instructions for pointing your local PATH variable to the compiler bin directory. Please note you’ll have to do this for the root user as well – if you plan on compiling under root.

I’m going to assume you have the compiler setup and configured properly. So how do we actually use it? Well, let’s start by creating a program! Now I don’t intend to teach you assembly language, but I do want to show you the basics of how to use the compiler. Plus, the entry point for our kernel has to be assembly. So you will learn a tiny bit about it. Let’s start by creating a new  file called init.asm within our /project/kernel/ directory. I’ll post a screenshot of my code and step you through it.

code

Okay, that is technically assembly but it doesn’t look too scary.

  • .global – This is basically defining a function that we plan to implement later on which can be linked by the compiler and called from other files. I don’t think it actually has to be global, in this case, but I wanted to show you the code anyway.
  • _start: – This line defines what we call a “label”. Labels are basically functions.
  • MOV sp,#0x8000 – In assembly, there are many “global” variables that you have access to. sp means “stack pointer”, I believe. What this line of code does is specify a location in memory where the stack memory will start to be allocated. It is pretty much required to be the first thing in your program.
  • b hang – This is two things. First, ‘b’ means ‘branch’ (essentially “call function”). There are multiple ways to call a function and I am not entirely sure of the difference between them. ‘bx’ means “branch and then return”, I think. So ‘b’ probably means you’re going away but you aren’t planning on returning!
  • hang: – Again, this just defines a new function called ‘hang’.
  • b hang – Again, this just calls the function ‘hang’ thus putting us in an infinite loop.

And that’s it! So now that we have a lovely program, how do we compile it specifically for the raspberry pi? Well, before we jump ahead of ourself, let’s take a look at some of the tools that our tool chain provides us.

Tools

Here is a list of some really useful commands and their descriptions:

arm-none-eabi-objump

This utility is capable of disassembling the output of your compiled files (amongst many other things). Use the -D flag to tell it to disassemble.

arm-none-eabi-objcopy

This utility is used to copy binary files (and possibly preform transformations on them in the process). We use it to copy our linked program into an IMG file.

arm-none-eabi-as

This is the assembler. We will be using it to compile the bootstrap.S file into a common format that can eventually be linked with C/C++.

arm-none-eabi-gcc

This is the C compiler. There are a number of flags we’ll be talking about later which we will be using during our compilation. NOTE: if you change that last part to g++ it comes a C++ compiler. All the same command line switches are the same.

arm-none-eabi-ld

And finally our linker! This is what takes all of the compiled binaries that we’ve developed and merges them into one output. There are some very important command line switches we need to learn about to properly use this one. So I’ll be explaining those in the next section.

The BUILD Script

Technically you’re supposed to use makefiles, but I prefer bash scripting. So let me show you a possible build.sh script looks like and we will go over some of the options.

#!/bin/sh
compiler=arm-none-eabi
code=../code
boot=../boot

# First let's compile our assembly file.
$compiler-as $code/init.asm -o $code/init.o

# Then let's call the linker to generate a binary output.
$compiler-ld $code/init.asm -o $code/kernel.elf

# Then let's dump some debug info to disk.
$compiler-objdump -D $code/init.o > $code/init.disassembled.asm
$compiler-objdump -D $code/kernel.elf > $code/kernel.disassembled.asm

# Lastly let's copy our binary into an img format.
$compiler-objcopy $code/kernel.elf -O binary $boot/kernel.img

echo "Finished compiling!"

Using the script above, your program should be compiled, disassembled, linked, converted to an IMG file, and then copied to the boot directory! If you just move all the files in that boot directory onto a FAT32 formatted SD Card (or just call that make image script we made in the other post) and you’ll be good! Give it a whirl.

DISCLAIMER: I wrote this code by hand without testing it. I will be doing some tests later (at which point I’ll remove this disclaimer). So be aware that I may have made a mistake somewhere. Feel free to post questions as they arise.

Tagged with: , , , , , ,
Posted in Beginner, Introduction

Hello Kernel!

To start off, I’m going to explain just what is required to be on the SD Card when the raspberry pi boots up. This is a fairly complicated process in actuality, and it took me forever to fully figure it out. From your perspective, however, it’s quite simple. The bare requirements are as follows:

  1. The first partition in your SD Card is formatted under Fat32.
  2. The SD Card contains some pre-loaded boot files provided by the raspberry pi foundation.
  3. A kernel.img file is provided.

Excellent! That’s it. One of the first questions I had when getting setup was what exactly is a kernel.img file and how does that need to be formatted? I’ll get to that in more detail later, for now just realize that it is a raw ELF file dumped into a file with an img extension.

Before starting anything, you’ll need to get the raspberry-pi specific bootloader. Note: you cannot use any old bootloader. So GRUB is out of the question. You have to specifically use the bootloader.bin and startup.elf files provided by the raspberry pi foundation. Assuming you have git installed, let’s snag those files first…

So first, let’s setup our project directory structure. Here is a series of commands you can run to setup the structure.

  • mkdir ~/rpi-kernel
  • mkdir ~/rpi-kernel/boot
  • mkdir ~/rpi-kernel/kernel
  • mkdir ~/rpi-kernel/output
  • mkdir ~/rpi-kernel/make

Now that we have the directory structure setup, let’s look at the bootloader files and put them inside our ~/rpi-kernel/boot directory. The official firmware project can be accessed from this URL: http://github.com/raspberrypi/firmware/tree/master You can git clone the whole repository if you want, but it’s actually pretty large. So I’ll just list the files you specifically need and you can download those directly into your boot directory.

  • boot/bootcode.bin
  • boot/start.elf
  • boot/fixup.dat

And that’s it! These are the necessary files required for the little green light to turn on when booting the raspberry pi from your own image.

Now that we have the bare minimum required, let’s look at making a bash script to automatically generate our IMG file. Now, I am not a linux expert – so I apologize if this seems pretty silly. After quite some time, I was able to write my very own script to automatically generate a bootable IMG file, however, the commands I use require root privileges. I don’t know how to do it any other way, but I’m sure it’s possible…

I’ll walk you through the relevant commands. But first, let’s create the actual script file.

  • nano ~/rpi-kernel/make/genimage.sh

Okay, now that we have a file let’s start writing it! The first thing your bash script needs is this line: #!/bin/sh. This tells the computer it’s an executable script (I think). To make life easy, we’re going to setup a few variables next.

Capture

Yeah, no text. Sorry 🙂 I am a firm believer in typing things out yourself. You’ll thank me later.

All we’re doing here is defining variables that can be used later. For example: $OUTPUT would reference the output name that we’ve set. Note you have to have the dollar-sign in front of the variable name when accessing it. The interesting thing to note here is the variable OUTPUSIZEmb. Yes, we are outputting a 34MB file. This is important because FAT32 requires ~34mb of space to properly generate the file structure. I don’t know exactly why, but it looks like this is the minimum output size we can generate.

Capture2

Now this looks scary, but it’s not all that bad. You can do the math ahead of time if you want – but I like to have it all configured from variables. Basically we are generating some details about the output file that we must use when creating the file. I probably do this the hard way, but after much testing – it all works. So let’s roll with it.

Capture3

This next line uses some of the math we calculated above to generate a completely blank IMG file.

Capture4

Now for a big scary one. Let’s take this one in sections… First, the command is fdisk. This is a utility that allows us to specify some meta-data about the IMG file. Specifically things like how many bytes per sector, how many sectors, whether it’s bootable, etc. Next, you’ll see I have this line > /dev/null 2>&1  This part is just for convenience. It basically says “redirect all the output from this command to a fake console”. That makes it so we don’t have to see the nitty gritty details of what happens when we run the script. Lastly, we do << EOF – this allows us to specify how the IMG file is formatted by sending in command line characters that get processed by the utility. I recommend you do this part by hand first, just to see the output and to better understand the fdisk utility.

Capture5

This is the last part of our script. I’ll try to explain it in detail…

losetup is a utility that lets you mount a sector of an IMG file. This is important because our IMG file has 4 sectors, and due to the boot table information – those sectors are not aligned to 0x0. This means we have to offset the starting point of our FAT32 partition, which is what the command line args are doing for us. The math that I am using in there is just something I’ve determined ahead of time that will work. Basically our FAT32 file system starts at an offset of 63 sectors. Each sector is 512 bytes. So we offset at 512b * 63s. Then we tell it to virtually mount that partition to the location /dev/loop1. You may need to create this directory if it doesn’t already exist.

mkfs.msdos is a utility that will format the virtual drive we just mounted, and passing in the command line args -F 32, it will format it under FAT32, which is what the raspberry pi requires.

mount is a utility that will let us mount the newly formatted partition of our IMG file as though it were a regular folder on our hard drive. Once we’ve mounted it, we can write files to it by copying them.

chmod 777 this is probably unnecessary. I didn’t even know I still had this in here… I would recommend not doing this.

cp this is the linux command to copy files. So basically we are copying files from our BOOT directory into the IMG file.

umount this is a utility to un-mount the IMG file. PLEASE NOTE: it is umount. NOT UNmount. There is no ‘n’.

And that’s it! If you run this script, hopefully you will have an IMG file that you can then write to an SD card and boot from. In later posts, I’ll explain how to generate the kernel.IMG file. We will be writing it to the /boot/ directory of our project folder, which means it will be automatically written to the SD card when we call this script.

Tagged with: , , , , ,
Posted in Beginner, Introduction

My github repository

Samples from my raspberry pi adventures can be found at my github repository:

https://github.com/SharpCoder/rpi-kernel

Please note that my code is not necessarily stable, per se, at any given point in time. Use at your own risk!

Posted in N/A