1. Denx Training Topics

DENX Software Engineering GmbH
Kirchenstraße 5
82194 Gröbenzell

2. About This Document

This document is compiler from various sources. The Embedded Linux Internals part has largely been derived from Detlev Zundel's training material.

The Linux Device Driver part is strongly influenced by the Book "Linux Device Drivers" Third Edition and therefore licenced under the Creative Commons Attribution-NonCommercial-ShareAlike 2.0 License

The newest version of this document can be found at http://www.denx.de/twiki/bin/view/Training2/GenericOverview.

3. General Introduction

This section gives an overview of who we are and what services we provide.

Contents of this section:

3.1. DENX Company Profile

DENX == two companies:

3.2. DENX People

3.3. DENX Projects

Customer Projects:

Public Projects:

What gives Unix users that smug expression?

4.1.1. History of Unix: Unix is born

4.1.2. History of Unix: Unix spreads

4.1.3. History of Unix: hard times

4.1.4. History of Unix: Free Software

4.1.5. Origin of Free Software

4.1.6. History of Linux

4.1.7. History of Unix: GNU/Linux

4.2. Background Information

4.2.1. Licensing Models

4.2.2. Standards and Portability in the Unix world

"The nice thing about standards is that there are so many of them to choose from." A. S. Tanenbaum

C

portability of other languages:

Unix

relevant today:

* FHS (Filesystem Hierarchy Standard)

4.2.3. Working with Free Software

4.2.4. Finding help

4.3. The Unix Philosophy

"The Unix philosophy is a set of cultural norms and philosophical
approaches to developing software based on the experience of leading
developers of the Unix operating system.", from Wikipedia.

-> It is more than simply best practices

Some important aspects:

(mostly from "The Art of Unix Programming", Eric S. Raymond)

4.4. Real-time Linux

This section gives a very general introduction to real-time Linux with special focus on the Xenomai real-time framework

Contents of this section:

4.4.1. Can Linux handle real-time requirements?

4.4.2. Real time with Adeos/Xenomai (Introduction)

4.4.3. Adeos

4.4.4. The Xenomai real time framework

4.4.5. Case study: Migrating from RTOS (VRTXsa) to Xenomai-Linux

4.5. Host Linux Setup

4.5.1. Installing the ELDK

Installation:

ELDK:

4.5.2. Working with the ELDK

4.5.3. Tftp Server Configuration

4.5.4. Setup of the NFS Server

4.5.5. Setup of the DHCP Server

subnet 10.0.0.0 netmask 255.0.0.0 {
        option routers          10.0.0.2;
        option subnet-mask      255.0.0.0;

        option domain-name      "local.net";
        option domain-name-servers ns.local.net;

        host trgt {     hardware ethernet       00:30:BF:01:02:D0;
                        fixed-address           10.0.0.99;
                        option root-path        "/opt/eldk/ppc_8xx";
                        option host-name        "tqm";
                        next-server             10.0.0.2;
                        filename                "/tftpboot/TQM8xxL/uImage";
        }
}

4.5.6. Terminal Emulation Setup

set line /dev/ttyS0
set speed 115200
set carrier-watch off
set handshake none
set flow-control none
robust
set file type bin
set file name lit
set rec pack 1000
set send pack 1000
set window 5

4.5.7. Test the host setup

4.6. U-Boot and Linux

DULG: Booting Embedded Linux on the target

4.6.1. The powerful U-Boot command line interface

DULG chapter

sequoia:

serial# MAC adress
52192 ethaddr 00:10:ec:00:cb:e0
eth1addr 00:10:ec:80:cb:e0
50453 ethaddr 00:10:ec:00:c5:15
eth1addr 00:10:ec:80:c5:15
52173 ethaddr 00:10:ec:00:cb:cd
eth1addr 00:10:ec:80:cb:cd

kilauea:

serial# MAC adress
082KLM2028Z ethaddr 00:06:4B:10:20:72
eth1addr 00:06:4B:10:20:73
082KLM2064Z ethaddr 00:06:4B:10:20:8c
eth1addr 00:06:4B:10:20:8d
082KLM2027Z ethaddr 00:06:4B:10:20:82
eth1addr 00:06:4B:10:20:83
082KLM2045Z ethaddr 00:06:4B:10:20:9a
eth1addr 00:06:4B:10:20:9b
081KLM2002Z ethaddr 00:06:4B:10:1b:0b
eth1addr 00:06:4B:10:1b:0c

4.6.2. An Introduction to git

git basics

Configuration

introduce yourself: 

$ git config --global user.name "Your Name Comes Here"
$ git config --global user.email you@yourdomain.example.com

Create a new repository 

$ git-init-db
$ git-add .
$ git-commit

Working with existing repositories

Best practice: work on a branch and leave master branch unmodified (track upstream)

clone a repository: 

$ git-clone git://www.denx.de/git/linux-2.6-denx.git

look at the history 

$ git-log     # simple
$ tig         # tui

add a branch for hacking 

$ git-checkout -b hacking HEAD

edit some files and look at changes in working dir 

$ git diff

add modified files to the index: 

$ git add file1 file2

delete some and mark them as deleted: 

$ git rm file3

look at diffs of the modified files added to the index 

$ git diff --cached

get some general statistics of which files were modified: 

$ git status

to throw away the changes git reset HEAD, to reset the working directory git-checkout -f

commit the changes added to the index (will be prompted for commit message) 

$ git commit    # -s adds Signed-off-by: ... line

alternatively without using the index as an intermediate stage: 

$ git commit -a

unhappy with commit message? 

$ git commit --amend

local branches

what branches have we got? 

git branch

create a branch as a clone of the current branch: 

$ git branch hacking

check out the contents of this branch: 

git checkout hacking

hack around, make a mess of it and decide to throw it away 

git checkout -f
git checkout master
git branch -d hacking

git for developers

Example: rework last 5 commits 

$ git-rebase --interactive HEAD~5

move around lines and change pick into squash in order to merge commit into previous one. 

$ git rebase --abort

goodies

4.6.3. Building U-Boot

   make distclean
   make sequoia_config
   make all

4.6.4. Configuring and building the Linux kernel

4.6.5. Booting Linux

4.6.6. U-Boot Tips and Tricks: preboot, update, etc.

preboot
This variable can be used to run specific code before the normal startup (i.e. CONFIG_BOOTDELAY loop, autostart or interactive mode) begins. This is especially useful when its contents are generated automatically, i.e. by checking for keypresses, etc.

update
[todo: explain update procedures using U-Boot]

Loading Compressed Images
Sometimes you may want to store some data in compressed format (for example to save flash memory), while your application needs the data in uncompressed form. You can trick U-Boot to do this uncompressing like this:

Generate a compressed U-Boot image of type "standalone" (="mkimage ... -T standalone -C gzip ..."=) and make sure that the envrionment variable autostart is set to no (i. e. enter "setenv autostart no;saveenv"). If you then use "bootm" for such an image, U-Boot will uncompress the contents of the image and store it at the "load address" ("-a" option for mkimage), but not attempt to start it yet. If the image contains executable code, you can omit the setting of "autostart", and U-Boot will automagically start the image by jumping to the entry point address ("-e" option for mkimage).

4.7. Designing Unix Applications

4.7.1. Unix Design Patterns

Definition:

"In software engineering, a design pattern is a general reusable
solution to a commonly occurring problem in software design", Wikipedia

Recall:

4.7.2. Simple Unix Interface Design Patterns

(mostly from "The Art of Unix Programming", Eric S. Raymond)

4.7.3. Advanced Unix Interface Design Patterns

some of the more important, there are more:

4.7.4. Unix Inter Process Communication - Simple Techniques

4.7.5. Unix Inter Process Communication - Advanced Techniques

POSIX IPC

Sockets:

4.7.6. IPC techniques to avoid

4.7.7. Use case: board test software

Requirements:

What we did ...

What we should have done...

4.7.8. Domain specific languages (Minilanguages)

4.8. Application debugging

4.9. Optimization Techniques

4.10. Using FLASH filesystems

Using FLASH file systems

Designing and building root file systems

4.11. Using the BDM/JTAG Debugger, BDI2000, Part 1

4.11.1. BDI2000 Installation and Configuration

Build Linux command line config tool:

Usage: 

[mrk@sokrates bdisetup]$ bdisetup 
Usage of BDI setup program V1.16:
bdisetup -v [-pP] [-bB] [-s]
  -v  Read current versions
   P  Port e.g. /dev/ttyS0
   B  Baudrate 9, 19, 38, 57 or 115
  -s  if present, exit loader and start firmware

bdisetup -e [-pP] [-bB]
  -e  Erase firmware and logic
   P  Port e.g. /dev/ttyS0
   B  Baudrate 9, 19, 38, 57 or 115

bdisetup -u [-pP] [-bB] [-aA] [-tT] [-dD]
  -u  Update firmware and/or logic
   P  Port e.g. /dev/ttyS0
   B  Baudrate 9, 19, 38, 57 or 115
   A  Application type STD,GDB,ADA,TOR,ACC
   T  Target type: PPC400,MPC500,PPC600,PPC700,MPC800
                   MPC7400,MPC7450,MPC8200,MPC8300,MPC8500,MPC8641
                   ARM,ARM11,XSCALE,MIPS,MIPS64
                   CPU32,MCF,HC12,MCORE
   D  Directory with the firmware/logic files

bdisetup -c [-pP] [-bB] [-iI] [-hH] [-mM] [-gG] [-fF]
  -c  Program network configuration
   P  Port e.g. /dev/ttyS0
   B  Baudrate 9, 19, 38, 57 or 115
   I  BDI IP address e.g. 100.100.100.100
   H  Host IP address
   M  Subnet mask (default: 255.255.255.255)
   G  Gateway IP address (default: 255.255.255.255)
   F  Configuration file name

Example: Query the current bdi configuration:

(Note: this is a simple tool. It does not perform any device locking, so root permissions are required to access the serial port.) 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# bdisetup -v -p/dev/ttyS0 -b115
BDI Type : BDI2000 Rev.C (SN: 93202220)
Loader   : V1.05
Firmware : V1.05 bdiGDB for MPC85xx
Logic    : V1.05 PPC6xx/PPC7xx
MAC      : 00-0c-01-93-20-22
IP Addr  : 192.168.10.6
Subnet   : 255.255.0.0
Gateway  : 255.255.0.255
Host IP  : 192.168.1.1
Config   : bdi6.cfg

Example: Install firmware for ppc4xx processors 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# bdisetup -u -p/dev/ttyS0 -b115 -aGDB -tPPC400 -d.
Connecting to BDI loader
Erasing CPLD
Programming firmware with ./b20pp4gd.116
Erasing firmware flash ....
Erasing firmware flash passed
Programming firmware flash ....
............................................................................................................................................
Programming firmware flash passed
Programming CPLD with ./pp4jed21.103
............................................................................................................................................................................................................................................................................
Programming CPLD passed
[root@sokrates gdbpp421-1.16-4xx-2006-11-03]#

Check everything went ok: 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# bdisetup -v -p/dev/ttyS0 -b115
BDI Type : BDI2000 Rev.C (SN: 93202220)
Loader   : V1.05
Firmware : V1.16 bdiGDB for PPC400
Logic    : V1.03 PPC400
MAC      : 00-0c-01-93-20-22
IP Addr  : 192.168.10.6
Subnet   : 255.255.0.0
Gateway  : 255.255.0.255
Host IP  : 192.168.1.1
Config   : bdi6.cfg

Setting Configuration Parameters: 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# bdisetup -c -p/dev/ttyS0 -h192.168.5.1 -i192.168.5.21 \
-m255.255.0.0 -g255.255.255.255 -fsequoia/sequoia.cfg
Connecting to BDI loader
Writing network configuration
Configuration passed

then check if it went ok (note the -s to exit loader and start the firmware): 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# bdisetup -v -p/dev/ttyS0 -b115 -s
BDI Type : BDI2000 Rev.C (SN: 93202220)
Loader   : V1.05
Firmware : V1.16 bdiGDB for PPC400
Logic    : V1.03 PPC400
MAC      : 00-0c-01-93-20-22
IP Addr  : 192.168.5.21
Subnet   : 255.255.0.0
Gateway  : 255.255.255.255
Host IP  : 192.168.5.1
Config   : /tftpboot/sequoia/sequoia.cfg

Quick Test: 

[root@sokrates gdbpp421-1.16-4xx-2006-11-03]# telnet 192.168.5.21
Trying 192.168.5.21...
Connected to bdi (192.168.5.21).
Escape character is '^]'.
BDI Debugger for Embedded PowerPC
=================================

MD    [<address>] [<count>]  display target memory as word (32bit)
MDH   [<address>] [<count>]  display target memory as half word (16bit)
MDB   [<address>] [<count>]  display target memory as byte (8bit)
DUMP  <addr> <size> [<file>] dump target memory to a file
MM    <addr> <value> [<cnt>] modify word(s) (32bit) in target memory
MMH   <addr> <value> [<cnt>] modify half word(s) (16bit) in target memory
MMB   <addr> <value> [<cnt>] modify byte(s) (8bit) in target memory
MC    [<address>] [<count>]  calculates a checksum over a memory range
MV                           verifies the last calculated checksum
RD    [<name>]               display general purpose or user defined register
RDUMP [<file>]               dump all user defined register to a file
RDSPR <number>               display special purpose register
RDDCR <number>               display device control register
RM    {<nbr>&#65533;<name>} <value> modify general purpose or user defined register
RMSPR <number>   <value>     modify special purpose register
RMDCR <number>   <value>     modify device control register
TLB   <from> [<to>]          display TLB entry
WTLB  <idx> <epn> <rpn>      write TLB entry (only PPC440)
DFLUSH [<addr>]              flush data cache (addr = cached memory address)
IFLUSH                       invalidate instruction cache
DCACHE <from> [<to>]         display L1 data cache (440: lines, 405: sets)
ICACHE <from> [<to>]         display L1 inst cache (440: lines, 405: sets)
BOOT                         reset the BDI and reload the configuration
RESET [HALT | RUN [time]]    reset the target system, change startup mode
BREAK [SOFT | HARD]          display or set current breakpoint mode
GO    [<pc>]                 set PC and start target system
GO    <n> <n> [<n>[<n>]]     start multiple cores in requested order
TI    [<pc>]                 trace on instuction (single step)
TC    [<pc>]                 trace on change of flow
HALT                         stop all cores via HALT pin
STOP  [<n>[<n>[<n>[<n>]]]]   stop core(s) via JTAG port (n = core number)
BI  <addr>                   set instruction breakpoint
CI  [<id>]                   clear instruction breakpoint(s)
BD  [R|W] <addr>             set data breakpoint (32bit access)
BDH [R|W] <addr>             set data breakpoint (16bit access)
BDB [R|W] <addr>             set data breakpoint ( 8bit access)
CD [<id>]                    clear data breakpoint(s)
INFO                         display information about the current state
LOAD   [<offset>] [<file> [<format>]] load program file to target memory
VERIFY [<offset>] [<file> [<format>]] verify a program file to target memory
PROG   [<offset>] [<file> [<format>]] program flash memory
                                      <format> : SREC or BIN or AOUT or ELF
ERASE  [<address> [<mode>]]  erase a flash memory sector, chip or block
                   <mode>  : CHIP, BLOCK or SECTOR (default is sector)
ERASE  <addr> <step> <count> erase multiple flash sectors
UNLOCK [<addr> [<delay>]]    unlock a flash sector
UNLOCK <addr> <step> <count> unlock multiple flash sectors
FLASH  <type> <size> <bus>   change flash configuration
DELAY  <ms>                  delay for a number of milliseconds
SELECT <core>                change the current core
HOST   <ip>                  change IP address of program file host
PROMPT <string>              defines a new prompt string
CONFIG                       display or update BDI configuration
CONFIG <file> [<hostIP> [<bdiIP> [<gateway> [<mask>]]]]
HELP                         display command list
JTAG                         switch to JTAG command mode
QUIT                         terminate the Telnet session

- TARGET: waiting for target Vcc
440EPx>quit
Connection closed by foreign host.
[root@sokrates gdbpp421-1.16-4xx-2006-11-03]#

4.11.2. Downloading U-Boot to flash

440EPx>info
    Core number       : 0
    Core state        : debug mode
    Debug entry cause : JTAG stop request
    Current PC        : 0xfffffffc
    Current CR        : 0x28000082
    Current MSR       : 0x00000000
    Current LR        : 0x0ff91484
440EPx>


440EPx>erase 0xfffa0000
Erasing flash at 0xfffa0000
Erasing flash passed
440EPx>erase 0xfffc0000
Erasing flash at 0xfffc0000
Erasing flash passed
440EPx>erase 0xfffe0000
Erasing flash at 0xfffe0000
Erasing flash passed


440EPx>prog
# enter filename and host IP address
440EPx>host 192.168.5.1
Host IP address is 192.168.5.1
440EPx>prog 0xfffa0000 /tftpboot/sequoia/u-boot.bin bin
Programming /tftpboot/sequoia/u-boot.bin , please wait ....
Programming flash passed
440EPx>

4.12.1. Debugging U-Boot

4.13.1. Building and installing Xenomai

4.13.1.1. Building and installing using the ELDK tree 

cd /opt/eldk-4.1/ppc_4xxFP/usr/src/linux-2.6.19.2-xenomai/
make mrproper
make ARCH=ppc sequoia_defconfig

configure additional options from "Real-time subsystem" toplevel menu: 

make ARCH=ppc menuconfig

and then run make: 

make ARCH=ppc uImage

the resulting uImage is a xenomai / adeos enable kernel

4.13.1.2. Building and installing from scratch 

$ scripts/prepare-kernel.sh --linux=<linux-srctree> [--adeos=<adeos-patch>] [--arch=<target-arch>]

example: 

../../git/xenomai/scripts/prepare-kernel.sh  --linux=. --arch=i386 --adeos=../adeos-ipipe-2.6.23-i386-1.10-10.patch

4.13.1.3. Running the cyclictest:

Options are: 

cyclictest -t 3 -p 80 -i 1000 -l 10000

4.14.1. Installing and using Embedded WEB Servers

4.14.2. Tools, Scripts, Tipps and Tricks

There are quite a few scripts/tools out there that will help in automatically generating a toolchain, a root filesystem, a ROM image, and other components of an embedded Linux system.

This is a collection of such tools, scripts, tipps and tricks:

[From: Karim Yaghmour et. al. on the etux mailing list.]

4.14.3. More information

Pointers to more information are colleted in the DULG document in the MoreIformation topic.

4.15. Happy Hacking

Well, that's all for now folks.

Happy Hacking! and may the source be with you.

5.1. Introduction to Linux Device Drivers

5.1.1. History of Linux

5.1.2. The Linux versioning scheme

5.1.3. What is a Device Driver?

Definition:

Characteristics:

Kernel components a driver interacts with:

5.1.4. Kernel vs. Application Development

5.1.5. Alternatives to (kernel) drivers

5.1.6. Licensing Models

5.1.7. Linux kernel / Linux device drivers resources

5.1.8. An Introduction to git

git basics

Configuration

introduce yourself: 

$ git config --global user.name "Your Name Comes Here"
$ git config --global user.email you@yourdomain.example.com

Create a new repository 

$ git-init-db
$ git-add .
$ git-commit

Working with existing repositories

Best practice: work on a branch and leave master branch unmodified (track upstream)

clone a repository: 

$ git-clone git://www.denx.de/git/linux-2.6-denx.git

look at the history 

$ git-log     # simple
$ tig         # tui

add a branch for hacking 

$ git-checkout -b hacking HEAD

edit some files and look at changes in working dir 

$ git diff

add modified files to the index: 

$ git add file1 file2

delete some and mark them as deleted: 

$ git rm file3

look at diffs of the modified files added to the index 

$ git diff --cached

get some general statistics of which files were modified: 

$ git status

to throw away the changes git reset HEAD, to reset the working directory git-checkout -f

commit the changes added to the index (will be prompted for commit message) 

$ git commit    # -s adds Signed-off-by: ... line

alternatively without using the index as an intermediate stage: 

$ git commit -a

unhappy with commit message? 

$ git commit --amend

local branches

what branches have we got? 

git branch

create a branch as a clone of the current branch: 

$ git branch hacking

check out the contents of this branch: 

git checkout hacking

hack around, make a mess of it and decide to throw it away 

git checkout -f
git checkout master
git branch -d hacking

git for developers

Example: rework last 5 commits 

$ git-rebase --interactive HEAD~5

move around lines and change pick into squash in order to merge commit into previous one. 

$ git rebase --abort

goodies

5.1.9. Linux source code organisation

5.2. First steps towards a Linux module

5.2.1. Configuring and building the Linux kernel

5.2.2. A first hello_world module

Code: 

/*                                                                                                                                               
 * A first simple hello_world kernel module                                                                                                      
 */

#include <linux/init.h>
#include <linux/module.h>

static int hello_init(void)
{
        printk(KERN_ALERT "Hello world!\n");
        return 0;
}

static void hello_exit(void)
{
        printk(KERN_ALERT "Goodbye, cruel world\n");
}

module_init(hello_init);
module_exit(hello_exit);

Makefile: 

obj-m := hello.o

Build it: 

make -C ~/git/linux-2.6-denx/ M=`pwd` modules ARCH=powerpc

loading

Extentions:

5.2.3. Linux kernel makefiles

Makefile: 

#
# Slightly more advanced Makefile for building the hello world module
#

ifneq ($(KERNELRELEASE),)
      obj-m       := hello.o
else
   KDIR        := /home/mk/git/linux-2.6-denx
   PWD         := $(shell pwd)

default:
   $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
endif

clean:
   rm -rf .*.cmd .tmp* *.ko *~ *.o *.mod.c *.symvers

5.2.4. Kernel message logging and printk

5.3. Character Drivers

5.3.1. Introduction to character drivers

5.3.2. Device numbers

API: Device numbers

type: 

dev_t

include/linux/types.h

creation, accessors: 

MKDEV(int major, int minor)
MAJOR(dev_t dev)
MINOR(dev_t dev)

include/linux/kdev_t.h

register a (known) range of numbers: 

int register_chrdev_region(dev_t first, unsigned int count, char *name);

allocate a range of free numbers: 

int alloc_chrdev_region(dev_t *dev, unsigned int firstminor, unsigned int count, char *name);

free allocated or registered range of numbers: 

void unregister_chrdev_region(dev_t first, unsigned int count);

include/linux/fs.h

5.3.3. Registering a character driver

API: Character driver registration

type: 

struct cdev

allocate at runtime: (need to add fops by hand) 

struct cdev* cdev_alloc(void);
...
my_cdev->ops = &my_fops;

allocate statically and only initalize at runtime: 

void cdev_init (struct cdev* cdev, const struct file_operations *fops);

for both cases initalize owner field to THIS_MODULE: 

my_cdev.owner = THIS_MODULE;

register the character driver with the kernel: 

int cdev_add(struct cdev *dev, dev_t num, unsigned int count);

remove (and if necessary free memory): 

void cdev_del(struct cdev *dev);

include/linux/cdev.h

old way (do not use anymore): 

int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);
void unregister_chrdev(unsigned int major, const char *name);

5.3.4. The open and release methods

API

open and release: 

int open(struct inode *i, struct file *filp)
int release(struct inode *i, struct file *filp)

find device numbers given an inode: 

unsigned int imajor(struct inode* i);
unsigned int iminor(struct inode* i);

find containing structure: 

container_of(container_field_ptr, container_type, name_of_container_field);

5.3.5. The read and write system calls

API

read and write file operations: 

ssize_t read(struct file *filp, char __user *buff, size_t count, loff_t *f_pos);
ssize_t write(struct file *filp, const char __user *buff, size_t count, loff_t *f_pos);

userspace access: 

unsigned long copy_to_user(void __user *to, const void *from, unsigned long count);
unsigned long copy_from_user(void *to, const void __user *from, unsigned long count);

5.3.6. Non-Blocking IO

API: waitqueues

type: 

wait_queue_head_t

linux/wait.h

Declaration: 

/* compile time */
DECLARE_WAIT_QUEUE_HEAD(name); 

/* run time */
wait_queue_head_t q;
init_waitqueue_head(&q);

Sleeping: 

wait_event(queue, condition)
wait_event_interruptible(queue, condition)
wait_event_timeout(queue, condition, timeout)
wait_event_interruptible_timeout(queue, condition, timeout)

waking up sleeping processes: 

void wake_up(wait_queue_head_t *queue);
void wake_up_interruptible(wait_queue_head_t *queue);

specialities:

deprecated: 

void sleep_on(wait_queue_head_t *q);
void interruptible_sleep_on(wait_queue_head_t *q);

5.3.7. poll and select

View from Userspace:

View from Kernelspace:

API: poll

header and poll operation 

#include <linux/poll.h>
unsigned int poll(struct file *filp, poll_table *wait);

step 1: add waitqueues 

void poll_wait(struct file*, wait_queue_head_t *, poll_table *);

step 2: return appropriate status (more in asm/poll.h) 

/* readable */
return (POLLIN | POLLRDNORM);

/* writeable */
return (POLLOUT | POLLWRNORM);

5.3.8. Memory mapping

API: mmap

file operation 

int *mmap (struct file *filp, struct vm_area_struct *vma);

build page tables 

int remap_pfn_range(struct vm_area_struct *vma, unsigned long virt_addr,
          unsigned long pfn, unsigned long size, pgprot_t prot);

/* identical to above on most plattforms: */
int io_remap_pfn_range(struct vm_area_struct *vma, unsigned long virt_addr, 
             unsigned long phys_addr, unsigned long size, pgprot_t prot);

5.4. Debugging Techniques

5.4.1. Debugging with printk

simple 

#define DEBUG

#ifdef DEBUG
# define _DBG(fmt, args...) printk(KERN_DEBUG "%s: " fmt "\n", __FUNCTION__, ##args)
#else
# define _DBG(fmt, args...) do { } while(0);
#endif

/* example usage */
void read_this(int count)
{
   _DBG("reading count=%d bytes", count);
}

Debug levels: 

#define DEBUG
#define DEBUG_LEVEL_LOW
#define DEBUG_LEVEL_MEDIUM
#define DEBUG_LEVEL_HIGH

static int debug = DEBUG_LEVEL_MEDIUM; /* change with ioctl or proc */

#ifdef DEBUG
# define _DBG(x, fmt, args...) do{ if (debug>=x) printk(KERN_CRIT "%s: " fmt "\n", __FUNCTION__, ##args); } while(0);
#else
# define _DBG(x, fmt, args...) do { } while(0);
#endif

/* example usage */
void function1()
{
        _DBG(DEBUG_LEVEL_LOW, "addr1=0x%x, addr2=0x%x", addr1, addr2);
}

void fatal_error()
{
   _DBG(DEBUG_LEVEL_HIGH, "this is the end");
}

Debug selective subsystems: 

#define DEBUG

#define DEBUG_READ   (1<<0)
#define DEBUG_WRITE   (1<<1)
#define DEBUG_LOCKS   (1<<2)

static unsigned int debug_mask = 0;

#ifdef DEBUG
# define _DBG(x, fmt, args...) do{ if (debug_mask & x) printk(KERN_DEBUG "%s: " fmt "\n", __FUNCTION__, ##args); } while(0);
#else
# define _DBG(x, fmt, args...) do { } while(0);
#endif

/* example usage */
void read()
{
   _DBG(DEBUG_READ, "inside read function");
}

void write(int count)
{
   _DBG(DEBUG_WRITE, "writing count=%d bytes", count);
}

5.4.2. Debugging Oopses

Unable to handle kernel paging request for data at address 0x00000000
Faulting instruction address: 0xd107c0c8
Oops: Kernel access of bad area, sig: 11 [#1]
Sequoia
Modules linked in: faulty chrdrv [last unloaded: faulty]
NIP: d107c0c8 LR: c0072834 CTR: 0fe368a4
REGS: cf9cfe40 TRAP: 0300   Not tainted  (2.6.25-rc2-00102-g0640ebb)
MSR: 00029000 <EE,ME>  CR: 48242028  XER: 00000007
DEAR: 00000000, ESR: 00800000
TASK = cfa5d000[950] 'bash' THREAD: cf9ce000
GPR00: d107c0c0 cf9cfef0 cfa5d000 00000000 4801f000 00000004 cf9cff20 00000000 
GPR08: 00000000 00000000 00000004 00000000 00000004 100fd4b8 00000000 00000000 
GPR16: 10125058 1009e4f0 100f0000 100f7998 00000000 00000000 101219a8 00000000 
GPR24: 00000001 100ac1ac bfaf4918 cf9cff20 4801f000 cfa2ba00 0ff448a8 00000004 
NIP [d107c0c8] faulty_write+0x8/0x10 [faulty]
LR [c0072834] vfs_write+0xcc/0x16c
Call Trace:
[cf9cfef0] [c0072804] vfs_write+0x9c/0x16c (unreliable)
[cf9cff10] [c0072ec8] sys_write+0x4c/0x90
[cf9cff40] [c000cf20] ret_from_syscall+0x0/0x3c
Instruction dump:
38810008 7cbdf850 480000d1 7c7d1a14 4bffffc4 7fc3f378 38810008 7fe5fb78 
480000b9 4bffffb0 39200000 38600000 <91290000> 4e800020 7c0802a6 9421fff0 
---[ end trace 4d8ed6ffb404bf6e ]---

5.4.3. Debugging with Hardware Debuggers

5.4.4. Profiling techniques

simple session (multiply example taken from [1]) 

# initalize oprof
$ opcontrol --init

# where's our kernel?
$ opcontrol --vmlinux=/tmp/vmlinux

# clear old data
$ opcontrol --reset

# available events?
$ opcontrol -l

# start daemon and start collecting data
$ opcontrol --start

# run application
$ while ./multiply; do ./multiply; echo -n .; done
......... 

# dump data to daemon
$ opcontrol --dump

# stop daemon
$ opcontrol --stop

# examine
$ opcontrol -l ./multiply

# annotate
$ opannotate -s ./multiply

5.5. The Linux driver model

5.5.1. Driver model and sysfs

example 

$ tree -L 1  /sys/
/sys/
|-- block
|-- bus
|-- class
|-- devices
|-- firmware
|-- fs
|-- kernel
|-- module
`-- power

explore with tree(1)

5.5.2. How does it all fit together?

Simplified example:

  1. bus driver is loaded (Platform, USB, PCI...)
  2. bus driver detects devices:
  3. somewhere during startup some drivers get loaded
  4. matching:
  5. if match:

5.5.3. Driver model and flattened device tree

Resources

5.6. Interfacing real hardware

5.6.1. Accessing devices

API:

avoid conflicts with other drivers: 

struct resource *request_mem_region(unsigned long start, unsigned long len, char *name);
void release_mem_region(unsigned long start, unsigned long len);

this shows up in /proc/iomem: 

-bash-3.2# cat /proc/iomem 
90000000-97ffffff : /plb/pciex@0a0000000
98000000-9fffffff : /plb/pciex@0c0000000
ef600200-ef600207 : serial
ef600300-ef600307 : serial
ef600400-ef600411 : ibm_iic
ef600500-ef600511 : ibm_iic
ef6c0000-ef6cffff : dwc_otg.0
  ef6c0000-ef6cffff : dwc_otg
f8000000-f8001fff : ndfc-nand.0
fc000000-ffffffff : fc000000.nor_flash

map physical address into kernel address space: 

void *ioremap(unsigned long phys_addr, unsigned long size);
void iounmap(void *addr);

avoid compiler optimization problems: 

void barrier();

avoid compiler and hardware reordering problems: 

mb(void)
rmb(void)
wmb(void)

be portable and avoid above problems: 

void io(write|read)(8|16|32)[be] functions

be versions for big endian registers!

5.6.2. Interrupt Handling

irqreturn_t handler(int irq, void *dev_id);


int request_irq(unsigned int irq,
          irq_handler_t handler,
       unsigned long irqflags,
       const char *devname,
       void * dev_id);


int fiddle_with_registers();

5.6.3. Deferring work

5.7. Miscellaneous topics

5.7.2. The proc file system

API

see procfs-guide in kernel DocBook documentation

5.7.3. The ioctl system call

API example: 


#define DIGIO_IOC_MAGIC         'C'

/* parameter for DIGIO_SET_VAL ioctl */
struct digio_par {
   unsigned int pinnr;
   unsigned int val;
};

/*
 * ioctl definitions
 */
#define DIGIO_GET_VAL   _IOR(DIGIO_IOC_MAGIC, 0, struct digio_par *)
#define DIGIO_SET_VAL   _IOW(DIGIO_IOC_MAGIC, 1, struct digio_par *)
#define DIGIO_RESET_ALL _IO(SCULL_IOC_MAGIC, 2)
#define DIGIO_IOC_MAXNR 2

static int digio_ioctl (struct inode *inode, struct file *file, uint cmd, unsigned long arg)
{
   int err;
   struct digio_par par;

   /* sanity checks */
   if(_IOC_TYPE(cmd) != DIGIO_IOC_MAGIC) return -ENOTTY;
   if(_IOC_NR(cmd) > DIGIO_IOC_MAXNR) return -ENOTTY;

   switch(cmd) {
   case DIGIO_GET_VAL:
      if((err = digio_get_pin(par.pinnr, &par.val)) < 0)
         goto out;

      if(copy_to_user((unsigned int *)arg, &par, sizeof(struct digio_par)) != 0) {
         printk(KERN_CRIT "%s: copy_to_user failed", __FUNCTION__);
         err = -EFAULT;
         goto out;
      }
      break;

   case DIGIO_SET_VAL:
      if(copy_from_user(&par, (unsigned int *)arg, sizeof(struct digio_par))) {
         err = -EFAULT;
         printk(KERN_CRIT "%s: copy_from_user failed", __FUNCTION__);
         goto out;
      }

      digio_set_pin(par.pinnr, par.val);
      break;

   case DIGIO_RESET_ALL:
      digio_reset_all();
      break;

   default:
      /* this is defined by POSIX */
      return -ENOTTY;
   }
   
   /* all ok */
   err = 0;
   
 out:
   return err;
}

5.7.4. Concurrency and Race Conditions

Terms

Concurrency in the Linux kernel

=> we need to protect our critical paths!

Advice:

5.7.4.1. Atomic operations

example:

initalize: 

atomic_t counter ATOMIC_INIT(0); /* or */
atomic_set(&counter, 0);

use: 

atomic_add(3, &counter);
atomic_inc(&counter);
atomic_sub(2, &counter);
atomic_dec(&counter);
val = atomic_read(&counter);

operate and test: 

if (atomic_sub_and_test(1, &counter) {
        printk(KERN_INFO "counter is zero!\n");
}

...

5.7.4.2. Mutexes

API: Kernel Mutexes

initalize: 

#include <linux/mutex.h>


DEFINE_MUTEX(name);

at runtime: 

mutex_init(struct mutex *lock);

locking: 

void mutex_lock(struct mutex *lock);
int mutex_lock_interruptible(struct mutex *lock);
int mutex_trylock(struct mutex *lock);

unlocking: 

void mutex_unlock(struct mutex *lock);

testing state: 

int mutex_is_locked(struct mutex *lock);

5.7.4.3. Spinlocks

5.7.4.4. Other locking techniques

reader/writer semaphores and spinlocks:

completions:

RCU: read-copy-update:

PI mutexes:

seqlocks:

5.7.5. Allocating Memory

5.7.6. How to delay execution

bad: burning CPU cycles (don't use) 

#define DELAY_MS 100
unsigned long j = jiffies + DELAY_MS * HZ / 1000;

while (time_before(jiffies, j1))
      cpu_relax();

slightly better but still wasteful: 

while (time_before(jiffies, j1))
      schedule();

recommended for small (busy waiting) delays 

void ndelay(unsigned long nsecs);
void udelay(unsigned long usecs);
void mdelay(unsigned long msecs);

better: sleep instead of busy waiting if possible: 

void msleep(unsigned int ms);

/* 
 * interruptible version (for waitqueues)
 * return 0 or if interrupted remaining time
 */
unsigned long msleep_interruptible(unsigned int ms);

void ssleep(unsigned int sec);

longer delays: use waitqueues!: 

wait_queue_head_t wait;
init_waitqueue_head(&wait);

/* condition is always false=0, delay in jiffies  */
wait_event_interruptible_timeout(wait, 0, delay);

5.7.7. Kernel library functions

      make htmldocs
      firefox Documentation/DocBook/index.html

-> kernel api

API: linked lists

type: 

struct list_head {
       struct list_head *next, *prev;
};

Initalize head pointer: 

struct list_head mempool;
INIT_LIST_HEAD(&mempool);   /* at runtime or ... */
LIST_HEAD(mempool);      /* at compile time */

Embedd struct list_head in list element: 

struct mem_chunk {
       unsigned char *ptr;
       int len;
       struct list_head list;
}

add element: 

struct mem_chunk mc;
mc.ptr = this;
mc.len = 128;

list_add(&mc.list, &mempool);

remove element: 

struct list_head myelement = list_del(&mempool.next);

find superstructure: 

mc = list_entry(myelement, struct mem_chunk, list);

Functions:

see kernel-api documentation and include/list.h

5.7.8. Userspace kernel communication

5.8.1. Proc, workqueue, Irq

Task:

Write a small module that creates a proc entry that can be read or written. Text written is rot13 scrambled and can be read back with the same proc entry.

Tips: 

static void rot_buf(char *buf, int rotate)
{
        int x;

        for (x=0; buf[x] != '\0'; x++) {
                if (islower (buf[x]))
                        buf[x] = ((buf[x] + rotate - 'a') % 26) + 'a';
                else if (isupper (buf[x]))
                        buf[x] = ((buf[x] + rotate - 'A') % 26) + 'A';
                else if (isdigit (buf[x]))
                        buf[x] = ((buf[x] + rotate - '0') % 10) + '0';
        }
}