How to optimize boot time on a MPC8260 system
Author
This document was written by Wolfgang Denk (wd {at} denx {dot} de).Context
The purpose of this document is to explain which factors have impact on the total boot time of Embedded Linux systems, and what can be done to reduce this time. In the scope of this documents the term "total boot time" describes the time between power-on of the system until a certain application program starts running. For demonstration purposes a TQM8260DBA0C7A-PJDA.201A will be used; this is a MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) based system with 292 MHz CPU clock, 208 MHz CPM clock and 83 MHz bus clock. It is equipped with 64 MB of SDRAM (on the 60x bus) and 32 MB of flash memory. The board is running the U-Boot boot loader (version 1.1.2) and a Linux kernel version 2.4.25. The U-Boot environment is stored in a 2 kB EEPROM. For the purposes of this test a standard, SELF based root file system was used; to get more realistic timings, several binary data files were added to fill up the root file system to result in a total size of 23 MB.Step 1: Measuring boot time in standard configuration:
3.757 3.757 3.797 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 3.797 3.857 MPC8260 Reset Status: External Soft, External Hard 3.857 3.887 MPC8260 Clock Configuration 3.967 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 4.037 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 4.097 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 4.167 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 4.167 4.237 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 5.447 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 5.467 I2C: ready 5.527 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 7.417 FLASH: 32 MB 7.837 In: serial 7.857 Out: serial 7.877 Err: serial 7.907 Net: FCC2 ETHERNET 7.917 7.977 Type "run flash_nfs" to mount root filesystem over NFS 7.977 11.027 Hit any key to stop autoboot: 0 11.047 ###START### 11.137 ## Booting image at 40040000 ... 11.167 Image Name: Linux-2.4.25 11.227 Image Type: PowerPC Linux Kernel Image (gzip compressed) 11.277 Data Size: 640126 Bytes = 625.1 kB 11.297 Load Address: 00000000 11.327 Entry Point: 00000000 11.607 Verifying Checksum ... OK 12.807 Uncompressing Kernel Image ... OK 12.857 ## Loading RAMDisk Image at 40200000 ... 12.897 Image Name: Test root filesystem 12.967 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 13.007 Data Size: 9642067 Bytes = 9.2 MB 13.037 Load Address: 00000000 13.067 Entry Point: 00000000 16.746 Verifying Checksum ... OK 17.466 Loading Ramdisk to 03487000, end 03db9053 ... OK 19.546 Memory BAT mapping: BAT2=64Mb, BAT3=0Mb, residual: 0Mb 19.686 Linux version 2.4.25 (wd@atlas.denx.de) (gcc version 3.2.2 20030217 (Yellow Dog Linux 3.0 3.2.2-2a_1)) #1 Sun Nov 7 21:07:03 MET 2004 19.726 On node 0 totalpages: 16384 19.746 zone(0): 16384 pages. 19.766 zone(1): 0 pages. 19.786 zone(2): 0 pages. 19.966 Kernel command line: root=/dev/ram rw ip=192.168.3.76:192.168.3.1::255.255.0.0:tqm8260:eth0:off panic=1 mtdparts=TQM8260-0:256k(U-Boot),1792k(Linux),-(Root-FS) ramdisk=32768 20.006 Warning: real time clock seems stuck! 20.246 Calibrating delay loop... 194.15 BogoMIPS 20.356 Memory: 53780k available (1116k kernel code, 388k data, 52k init, 0k highmem) 20.426 Dentry cache hash table entries: 8192 (order: 4, 65536 bytes) 20.496 Inode cache hash table entries: 4096 (order: 3, 32768 bytes) 20.546 Mount cache hash table entries: 512 (order: 0, 4096 bytes) 20.616 Buffer cache hash table entries: 4096 (order: 2, 16384 bytes) 20.686 Page-cache hash table entries: 16384 (order: 4, 65536 bytes) 20.726 POSIX conformance testing by UNIFIX 20.756 Linux NET4.0 for Linux 2.4 20.806 Based upon Swansea University Computer Society NET3.039 20.846 Initializing RT netlink socket 20.856 Starting kswapd 20.906 JFFS2 version 2.2. (C) 2001-2003 Red Hat, Inc. 20.946 CPM UART driver version 0.01 20.976 ttyS0 on SMC1 at 0x0000, BRG7 21.306 ttyS1 on SMC2 at 0x0040, BRG8 21.336 ttyS2 on SCC1 at 0x8000, BRG1 21.376 ttyS3 on SCC2 at 0x8100, BRG2 21.406 pty: 256 Unix98 ptys configured 21.456 eth0: FCC2 ENET Version 0.4, 00:D0:93:00:4F:88 21.536 RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize 21.596 TQM8260-0: Found 4 x16 devices at 0x0 in 64-bit mode 21.636 Amd/Fujitsu Extended Query Table at 0x0040 21.706 TQM8260-0: CFI does not contain boot bank location. Assuming top. 21.736 number of CFI chips: 1 21.806 cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness. 21.866 3 cmdlinepart partitions found on MTD device TQM8260-0 21.936 TQM8260 flash bank 0: Using command line partition definition 21.976 Creating 3 MTD partitions on "TQM8260-0": 22.016 0x00000000-0x00040000 : "U-Boot" 22.046 0x00040000-0x00200000 : "Linux" 22.086 0x00200000-0x02000000 : "Root-FS" 22.116 NET4: Linux TCP/IP 1.0 for NET4.0 22.146 IP Protocols: ICMP, UDP, TCP 22.206 IP: routing cache hash table of 512 buckets, 4Kbytes 22.266 TCP: Hash tables configured (established 4096 bind 8192) 23.786 IP-Config: Complete: 23.866 device=eth0, addr=192.168.3.76, mask=255.255.0.0, gw=255.255.255.255, 23.916 host=tqm8260, domain=, nis-domain=(none), 23.986 bootserver=192.168.3.1, rootserver=192.168.3.1, rootpath= 24.036 NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. 24.086 RAMDISK: Compressed image found at block 0 27.646 Freeing initrd memory: 9416k freed 27.696 VFS: Mounted root (ext2 filesystem). 27.736 Freeing unused kernel memory: 52k init 30.915 30.916 30.985 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 31.005 ### Application running ...Oops - even with a pretty fast processor it takes a disappointing 31 seconds between power on and start of the applications program.
Step 2: Reducing Boot Time by Simple Configuration Changes
Let's analyze where the time is spent, and discuss methods to avoid unnecessary delays:2.1 U-Boot startup:
3.797 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57)It takes nearly 4 seconds before U-Boot prints it's first message on this system. On other systems U-Boot is much faster. Let's for example test a PM826 board (MPC8260 CPU at 192 MHz):
0.000 0.040 U-Boot 0.4.8 (Sep 17 2003 - 13:15:34) 0.040 0.110 MPC8260 Reset Status: Check Stop, External Soft, External Hard 0.110 0.140 MPC8260 Clock Configuration 0.220 - Bus-to-Core Mult 3x, VCO Div 2, 60x Bus Freq 33-100, Core Freq 100-300 0.280 - dfbrg 0, corecnf 0x08, busdf 3, cpmdf 1, plldf 0, pllmf 1 0.350 - vco_out 256000000, scc_clk 64000000, brg_clk 64000000 0.420 - cpu_clk 192000000, cpm_clk 128000000, bus_clk 64000000 0.420 0.500 CPU: MPC8260 (HiP4 Rev 14, Mask unknown [immr=0x0064,k=0x002d]) at 192 MHz 0.510 Board: PM826 0.530 I2C: ready 0.550 DRAM: (60x:9COL) 128 MB 0.600 FLASH: 8 MBHere the delay until the first console message is printed is much, much smaller (0.040 vs. 3.797 seconds).
The explanation for this difference is in the way how U-Boot stores and accesses it's so-called "environment variables": on the TQM8260 board they are stored in a 2 kB EEPROM device attached to the I2C bus, while on the PM826 board the environment is stored in flash memory.
Due to the internal organization and initialization sequence of U-Boot, reading the environment from the I2C device takes a considerable amount of time. This can be avoided by storing the environment variables in flash memory.
[Storing the environment in flash is also more reliable: EEPROM chips on a I2C bus may suffer from "I2C Edge Conditions" problems; in flash you can even use redundant storage of the environment data.]
Lesson learned: use flash memory to store the U-Boot environment
Time savings: 3.757 seconds
2.2 Boot Delay:
7.977 Type "run flash_nfs" to mount root file system over NFS 7.977 11.027 Hit any key to stop autoboot: 0 11.047 ###START###The 3 seconds delay (7.977 to 11.027 = 3.050 seconds) is easily explained by the variable settings in U-boot:
=> printenv bootdelay bootdelay=3Here, an initial count down (boot delay) of 3 seconds is selected which causes the delay. On production systems it is trivial to adjust this delay to 0. Lesson learned: pay attention to your system configuration Time savings: 3 seconds
2.3 Console Baud Rate:
The whole boot procedure is astonishingly slow. One factor for this is the console output. The TQM8260 board uses a standard console baud rate of 9600 bps. The output above consists of approx. 100 lines of text, totaling in about 4200 characters. Printing 4200 characters at 9600 bps takes approx. 4.4 seconds. To verify this calculation we re-run the test:2.3.1 using a higher console baud rate:
In U-Boot, we set the console baud rate to 115200 bps:4.857 4.857 4.867 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 4.867 4.867 MPC8260 Reset Status: External Soft, External Hard 4.867 4.867 MPC8260 Clock Configuration 4.877 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 4.887 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 4.887 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 4.897 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 4.897 4.897 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 6.077 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 6.077 I2C: ready 6.077 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 7.967 FLASH: 32 MB 8.377 In: serial 8.377 Out: serial 8.387 Err: serial 8.397 Net: FCC2 ETHERNET 8.407 8.417 Type "run flash_nfs" to mount root filesystem over NFS 8.417 11.417 Hit any key to stop autoboot: 0 11.427 ###START### 11.477 ## Booting image at 40040000 ... 11.477 Image Name: Linux-2.4.25 11.487 Image Type: PowerPC Linux Kernel Image (gzip compressed) 11.487 Data Size: 640126 Bytes = 625.1 kB 11.487 Load Address: 00000000 11.497 Entry Point: 00000000 11.737 Verifying Checksum ... OK 12.907 Uncompressing Kernel Image ... OK 12.927 ## Loading RAMDisk Image at 40200000 ... 12.927 Image Name: Test root filesystem 12.927 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 12.937 Data Size: 9642067 Bytes = 9.2 MB 12.937 Load Address: 00000000 12.937 Entry Point: 00000000 16.586 Verifying Checksum ... OK 17.266 Loading Ramdisk to 03487000, end 03db9053 ... OK 19.296 Memory BAT mapping: BAT2=64Mb, BAT3=0Mb, residual: 0Mb 19.306 Linux version 2.4.25 (wd@atlas.denx.de) (gcc version 3.2.2 20030217 (Yellow Dog Linux 3.0 3.2.2-2a_1)) #1 Sun Nov 7 21:07:03 MET 2004 19.316 On node 0 totalpages: 16384 19.316 zone(0): 16384 pages. 19.316 zone(1): 0 pages. 19.316 zone(2): 0 pages. 19.336 Kernel command line: root=/dev/ram rw ip=192.168.3.76:192.168.3.1::255.255.0.0:tqm8260:eth0:off panic=1 mtdparts=TQM8260-0:256k(U-Boot),1792k(Linux),-(Root-FS) ramdisk=32768 19.336 Warning: real time clock seems stuck! 19.536 Calibrating delay loop... 194.15 BogoMIPS 19.566 Memory: 53780k available (1116k kernel code, 388k data, 52k init, 0k highmem) 19.576 Dentry cache hash table entries: 8192 (order: 4, 65536 bytes) 19.586 Inode cache hash table entries: 4096 (order: 3, 32768 bytes) 19.586 Mount cache hash table entries: 512 (order: 0, 4096 bytes) 19.596 Buffer cache hash table entries: 4096 (order: 2, 16384 bytes) 19.596 Page-cache hash table entries: 16384 (order: 4, 65536 bytes) 19.606 POSIX conformance testing by UNIFIX 19.606 Linux NET4.0 for Linux 2.4 19.606 Based upon Swansea University Computer Society NET3.039 19.616 Initializing RT netlink socket 19.616 Starting kswapd 19.616 JFFS2 version 2.2. (C) 2001-2003 Red Hat, Inc. 19.626 CPM UART driver version 0.01 19.626 ttyS0 on SMC1 at 0x0000, BRG7 19.926 ttyS1 on SMC2 at 0x0040, BRG8 19.926 ttyS2 on SCC1 at 0x8000, BRG1 19.936 ttyS3 on SCC2 at 0x8100, BRG2 19.936 pty: 256 Unix98 ptys configured 19.936 eth0: FCC2 ENET Version 0.4, 00:D0:93:00:4F:88 19.946 RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize 19.956 TQM8260-0: Found 4 x16 devices at 0x0 in 64-bit mode 19.966 Amd/Fujitsu Extended Query Table at 0x0040 19.966 TQM8260-0: CFI does not contain boot bank location. Assuming top. 19.966 number of CFI chips: 1 19.976 cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness. 19.986 3 cmdlinepart partitions found on MTD device TQM8260-0 19.986 TQM8260 flash bank 0: Using command line partition definition 19.986 Creating 3 MTD partitions on "TQM8260-0": 19.996 0x00000000-0x00040000 : "U-Boot" 19.996 0x00040000-0x00200000 : "Linux" 19.996 0x00200000-0x02000000 : "Root-FS" 20.006 NET4: Linux TCP/IP 1.0 for NET4.0 20.006 IP Protocols: ICMP, UDP, TCP 20.006 IP: routing cache hash table of 512 buckets, 4Kbytes 20.016 TCP: Hash tables configured (established 4096 bind 8192) 21.516 IP-Config: Complete: 21.526 device=eth0, addr=192.168.3.76, mask=255.255.0.0, gw=255.255.255.255, 21.526 host=tqm8260, domain=, nis-domain=(none), 21.536 bootserver=192.168.3.1, rootserver=192.168.3.1, rootpath= 21.536 NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. 21.546 RAMDISK: Compressed image found at block 0 25.076 Freeing initrd memory: 9416k freed 25.086 VFS: Mounted root (ext2 filesystem). 25.086 Freeing unused kernel memory: 52k init 28.276 28.276 28.286 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 28.286 Enter 'help' for a list of built-in commands. 28.286 28.296 # ### Application running ...Now this is interesting for several reasons: first, we can see that the total boot time went down from 31.005 to 28.296 seconds; at the same time, the initial delay until the first boot message from U-Boot went up from 3.757 to 4.857 seconds. Why that? All we changed was the console baud rate?!? To understand this behavior, one has to know that U-Boot must read the console baud rate very early during initialization to be able to print to the serial console port. And U-Boot variables are stored as a list, where new (or changed) values always get added at the end. The consequence of our changing the console baud rate was that U-Boot now must search until the very end of the environment variables, which simply takes longer. To verify this effect, we will re-enter the U-Boot environment to make sure the baud rate definition comes as early as possible: U-Boot environment before the re-ordering:
=> printenv
loads_echo=1
preboot=echo;echo Type "run flash_nfs" to mount root filesystem over NFS;echo
nfsargs=setenv bootargs root=/dev/nfs rw nfsroot=$(serverip):$(rootpath)
ramargs=setenv bootargs root=/dev/ram rw
addip=setenv bootargs $(bootargs) ip=$(ipaddr):$(serverip):$(gatewayip):$(netmask):$(hostname):$(netdev):off panic=1
flash_nfs=run nfsargs addip;bootm $(kernel_addr)
kernel_addr=40040000
serial#=TQM8260DBA0C7A-PJDA.201A 10399642 6
ethaddr=00:D0:93:00:4F:87
hostname=tqm8260
netdev=eth0
load=tftp 100000 $(u-boot)
update=protect off 40000000 4003FFFF;era 40000000 4003FFFF;cp.b 100000 40000000 $(filesize)
u-boot=/tftpboot/TQM8260/u-boot.bin-wd
mb=setenv clocks_in_mhz;setenv bootfile TQM8260/myuriy/linux;run net_nfs
rootpath=/opt/eldk/ppc_82xx
wd=setenv bootfile /tftpboot/TQM8260/uImage-wd;setenv clocks_in_mhz;run net_nfs
rtai=setenv clocks_in_mhz;setenv bootfile TQM8260/uImage-rtai; run net_nfs
bootfile=TQM8260/uImage-wd
test=tftp 200000 $(bootfile);run nfsargs addip addpart;bootm
addpart=setenv bootargs $(bootargs) mtdparts=TQM8260-0:256k(U-Boot),1792k(Linux),-(Root-FS)
filesize=932093
netmask=255.255.0.0
ipaddr=192.168.3.76
serverip=192.168.3.1
net_nfs=tftp 200000 $(bootfile);run nfsargs addip addpart;bootm
bootcmd=run flash_self
ramdisk_addr=40200000
bootdelay=3
rd_size=32768
addrdsz=setenv bootargs $(bootargs) ramdisk=$(rd_size)
flash_self=echo ###START###;run ramargs addip addpart addrdsz;bootm $(kernel_addr) $(ramdisk_addr)
==> baudrate=115200
stdin=serial
stdout=serial
stderr=serial
mem=63488k
bootcount=16
Environment size: 1532/2044 bytes
After re-ordering:
serial#=TQM8260DBA0C7A-PJDA.201A 10399642 6
ethaddr=00:D0:93:00:4F:87
bootdelay=3
==> baudrate=115200
loads_echo=1
preboot=echo;echo Type "run flash_nfs" to mount root filesystem over NFS;echo
nfsargs=setenv bootargs root=/dev/nfs rw nfsroot=$(serverip):$(rootpath)
ramargs=setenv bootargs root=/dev/ram rw
addip=setenv bootargs $(bootargs) ip=$(ipaddr):$(serverip):$(gatewayip):$(netmask):$(hostname):$(netdev):off panic=1
flash_nfs=run nfsargs addip;bootm $(kernel_addr)
kernel_addr=40040000
hostname=tqm8260
netdev=eth0
load=tftp 100000 $(u-boot)
update=protect off 40000000 4003FFFF;era 40000000 4003FFFF;cp.b 100000 40000000 $(filesize)
u-boot=/tftpboot/TQM8260/u-boot.bin-wd
rootpath=/opt/eldk/ppc_82xx
bootfile=TQM8260/uImage-wd
test=tftp 200000 $(bootfile);run nfsargs addip addpart;bootm
addpart=setenv bootargs $(bootargs) mtdparts=TQM8260-0:256k(U-Boot),1792k(Linux),-(Root-FS)
netmask=255.255.0.0
ipaddr=192.168.3.76
serverip=192.168.3.1
net_nfs=tftp 200000 $(bootfile);run nfsargs addip addpart;bootm
bootcmd=run flash_self
ramdisk_addr=40200000
rd_size=32768
addrdsz=setenv bootargs $(bootargs) ramdisk=$(rd_size)
flash_self=echo ###START###;run ramargs addip addpart addrdsz;bootm $(kernel_addr) $(ramdisk_addr)
stdin=serial
stdout=serial
stderr=serial
mem=63488k
bootcount=17
Environment size: 1288/2044 bytes
Result:
1.312 1.312 1.312 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.313 1.313 MPC8260 Reset Status: External Soft, External Hard 1.313 1.322 MPC8260 Clock Configuration 1.322 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.332 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.342 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.342 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.343 1.352 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.562 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.562 I2C: ready 1.572 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.162 FLASH: 32 MB 3.572 In: serial 3.572 Out: serial 3.582 Err: serial 3.592 Net: FCC2 ETHERNET 3.602 3.602 Type "run flash_nfs" to mount root filesystem over NFS 3.602 6.612 Hit any key to stop autoboot: 0 6.612 ###START### 6.662 ## Booting image at 40040000 ... 6.662 Image Name: Linux-2.4.25 6.662 Image Type: PowerPC Linux Kernel Image (gzip compressed) 6.672 Data Size: 640126 Bytes = 625.1 kB 6.672 Load Address: 00000000 6.672 Entry Point: 00000000 6.922 Verifying Checksum ... OK 8.092 Uncompressing Kernel Image ... OK 8.102 ## Loading RAMDisk Image at 40200000 ... 8.102 Image Name: Test root filesystem 8.112 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 8.112 Data Size: 9642067 Bytes = 9.2 MB 8.122 Load Address: 00000000 8.122 Entry Point: 00000000 11.772 Verifying Checksum ... OK 12.441 Loading Ramdisk to 03487000, end 03db9053 ... OK 14.481 Memory BAT mapping: BAT2=64Mb, BAT3=0Mb, residual: 0Mb 14.491 Linux version 2.4.25 (wd@atlas.denx.de) (gcc version 3.2.2 20030217 (Yellow Dog Linux 3.0 3.2.2-2a_1)) #1 Sun Nov 7 21:07:03 MET 2004 14.492 On node 0 totalpages: 16384 14.501 zone(0): 16384 pages. 14.502 zone(1): 0 pages. 14.502 zone(2): 0 pages. 14.511 Kernel command line: root=/dev/ram rw ip=192.168.3.76:192.168.3.1::255.255.0.0:tqm8260:eth0:off panic=1 mtdparts=TQM8260-0:256k(U-Boot),1792k(Linux),-(Root-FS) ramdisk=32768 14.521 Warning: real time clock seems stuck! 14.721 Calibrating delay loop... 194.15 BogoMIPS 14.751 Memory: 53780k available (1116k kernel code, 388k data, 52k init, 0k highmem) 14.761 Dentry cache hash table entries: 8192 (order: 4, 65536 bytes) 14.761 Inode cache hash table entries: 4096 (order: 3, 32768 bytes) 14.771 Mount cache hash table entries: 512 (order: 0, 4096 bytes) 14.772 Buffer cache hash table entries: 4096 (order: 2, 16384 bytes) 14.781 Page-cache hash table entries: 16384 (order: 4, 65536 bytes) 14.782 POSIX conformance testing by UNIFIX 14.791 Linux NET4.0 for Linux 2.4 14.791 Based upon Swansea University Computer Society NET3.039 14.792 Initializing RT netlink socket 14.801 Starting kswapd 14.801 JFFS2 version 2.2. (C) 2001-2003 Red Hat, Inc. 14.802 CPM UART driver version 0.01 14.811 ttyS0 on SMC1 at 0x0000, BRG7 15.111 ttyS1 on SMC2 at 0x0040, BRG8 15.111 ttyS2 on SCC1 at 0x8000, BRG1 15.111 ttyS3 on SCC2 at 0x8100, BRG2 15.121 pty: 256 Unix98 ptys configured 15.121 eth0: FCC2 ENET Version 0.4, 00:D0:93:00:4F:88 15.131 RAMDISK driver initialized: 16 RAM disks of 32768K size 1024 blocksize 15.141 TQM8260-0: Found 4 x16 devices at 0x0 in 64-bit mode 15.141 Amd/Fujitsu Extended Query Table at 0x0040 15.151 TQM8260-0: CFI does not contain boot bank location. Assuming top. 15.151 number of CFI chips: 1 15.161 cfi_cmdset_0002: Disabling erase-suspend-program due to code brokenness. 15.161 3 cmdlinepart partitions found on MTD device TQM8260-0 15.171 TQM8260 flash bank 0: Using command line partition definition 15.171 Creating 3 MTD partitions on "TQM8260-0": 15.172 0x00000000-0x00040000 : "U-Boot" 15.181 0x00040000-0x00200000 : "Linux" 15.181 0x00200000-0x02000000 : "Root-FS" 15.182 NET4: Linux TCP/IP 1.0 for NET4.0 15.182 IP Protocols: ICMP, UDP, TCP 15.191 IP: routing cache hash table of 512 buckets, 4Kbytes 15.201 TCP: Hash tables configured (established 4096 bind 8192) 16.701 IP-Config: Complete: 16.701 device=eth0, addr=192.168.3.76, mask=255.255.0.0, gw=255.255.255.255, 16.711 host=tqm8260, domain=, nis-domain=(none), 16.711 bootserver=192.168.3.1, rootserver=192.168.3.1, rootpath= 16.721 NET4: Unix domain sockets 1.0/SMP for Linux NET4.0. 16.721 RAMDISK: Compressed image found at block 0 20.251 Freeing initrd memory: 9416k freed 20.261 VFS: Mounted root (ext2 filesystem). 20.271 Freeing unused kernel memory: 52k init 23.451 23.451 23.461 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 23.471 Enter 'help' for a list of built-in commands. 23.471 23.471 # ### Application running ...Wow! Simply changing the order of the environment variables in U-Boot caused the total boot time to go down from 28.296 to 23.471 seconds - 4.8 seconds or 20% saved!! Lesson learned: little changes may have big effects or:
you better know exactly what's going on.
2.3.2 Suppressing console output:
By simply adding "quiet" to the Linux boot arguments string we can suppress the console output in Linux. This gives:1.307 1.307 1.307 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.307 1.317 MPC8260 Reset Status: External Soft, External Hard 1.317 1.317 MPC8260 Clock Configuration 1.327 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.327 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.337 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.347 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.347 1.347 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.567 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.567 I2C: ready 1.567 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.227 FLASH: 32 MB 3.637 In: serial 3.637 Out: serial 3.637 Err: serial 3.657 Net: FCC2 ETHERNET 3.667 3.667 Type "run flash_nfs" to mount root filesystem over NFS 3.667 6.667 Hit any key to stop autoboot: 0 6.677 ###START### 6.727 ## Booting image at 40040000 ... 6.727 Image Name: Linux-2.4.25 6.737 Image Type: PowerPC Linux Kernel Image (gzip compressed) 6.737 Data Size: 640126 Bytes = 625.1 kB 6.747 Load Address: 00000000 6.747 Entry Point: 00000000 6.997 Verifying Checksum ... OK 8.157 Uncompressing Kernel Image ... OK 8.177 ## Loading RAMDisk Image at 40200000 ... 8.177 Image Name: Test root filesystem 8.187 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 8.187 Data Size: 9642067 Bytes = 9.2 MB 8.187 Load Address: 00000000 8.187 Entry Point: 00000000 11.846 Verifying Checksum ... OK 12.516 Loading Ramdisk to 03487000, end 03db9053 ... OK 23.306 23.306 23.316 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 23.316 Enter 'help' for a list of built-in commands. 23.316 23.316 # ### Application running ...Save: 0.155 seconds. Not exactly as much as expected.
Step 3: Reducing Boot Time by Linux Kernel Configuration:
When Linux starts, it calculates the speed of the processor. This is necessary to calibrate low-level delay loops. A visible indication of this step is the "bogomips" boot message:Calibrating delay loop... 194.15 BogoMIPSOn an embedded system with a fixed configuration this value will never change, so the calculation can be saved by simply inserting a pre-calculated value instead. This is known as "using preset loops_per_jiffy" or "preset lpj". In the DENX Linux kernel, there is a kernel configuration option for this; see "General setup" => "Embedded options" => "Configure kernel for faster booting"; in the first step, leave "Use preset loops_per_jiffy" unselected; the kernel will then run the standard calibration sequence and report the results as part of the boot messages:
Calibrating delay loop... 194.15 BogoMIPS Use 'Preset loops_per_jiffy'=970752 for preset lpj.Now you can select the ""Use preset loops_per_jiffy" configuration option and enter the calculated LPJ value as "Loops per jiffy":
[*] Configure kernel features for embedded systems [*] Configure kernel for faster booting [*] Use preset loops_per_jiffy (970752) Loops per jiffy (NEW)Result:
1.314 ~ 1.314 1.314 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.315 1.315 MPC8260 Reset Status: External Soft, External Hard 1.315 1.315 MPC8260 Clock Configuration 1.315 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.315 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.315 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.315 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.315 1.315 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.544 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.544 I2C: ready 1.544 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.164 FLASH: 32 MB 3.575 In: serial 3.575 Out: serial 3.575 Err: serial 3.575 Net: FCC2 ETHERNET 3.605 3.605 Type "run flash_nfs" to mount root filesystem over NFS 3.605 6.606 Hit any key to stop autoboot: 0 6.606 ###START### 6.777 ## Booting image at 40040000 ... 6.777 Image Name: Linux-2.4.25 6.777 Image Type: PowerPC Linux Kernel Image (gzip compressed) 6.777 Data Size: 640102 Bytes = 625.1 kB 6.777 Load Address: 00000000 6.777 Entry Point: 00000000 6.956 Verifying Checksum ... OK 8.097 Uncompressing Kernel Image ... OK 8.127 ## Loading RAMDisk Image at 40200000 ... 8.128 Image Name: Test root filesystem 8.128 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 8.128 Data Size: 9642067 Bytes = 9.2 MB 8.128 Load Address: 00000000 8.128 Entry Point: 00000000 11.829 Verifying Checksum ... OK 12.539 Loading Ramdisk to 03487000, end 03db9053 ... OK 22.996 22.996 22.996 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 22.996 Enter 'help' for a list of built-in commands. 22.996 22.996 # ### Application running ...Time savings: approx 0.3 seconds.
Step 4: Trading Robustness and Cost for Speed
In the given setup, we still use a standard configuration of U-Boot, which will automatically calculate a CRC32 checksum over all images (Linux kernel image and root file system image), and we use compressed images that need to be uncompressed before they can be used. Both the calculation of the CRC and the uncompressing takes time. However, turning off the CRC verification means that we give up some of the built-in robustness of U-Boot, while using uncompressed images take about three times as much space in flash memory, which means the cost for the flash memory might go up.4.1 Turning off CRC32 checksumming:
To turn off the CRC32 verification it is sufficient in U-Boot to set the "verify" environment variable to "no":=> setenv verify no => saveenv Saving Environment to EEPROM...Result:
1.207 1.208 1.208 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.208 1.208 MPC8260 Reset Status: External Soft, External Hard 1.208 1.208 MPC8260 Clock Configuration 1.208 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.208 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.209 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.209 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.209 1.209 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.407 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.407 I2C: ready 1.407 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.067 FLASH: 32 MB 3.488 In: serial 3.488 Out: serial 3.488 Err: serial 3.488 Net: FCC2 ETHERNET 3.488 3.488 Type "run flash_nfs" to mount root filesystem over NFS 3.488 6.519 Hit any key to stop autoboot: 0 6.519 ###START### 6.590 ## Booting image at 40040000 ... 6.590 Image Name: Linux-2.4.25 6.590 Image Type: PowerPC Linux Kernel Image (gzip compressed) 6.590 Data Size: 640102 Bytes = 625.1 kB 6.590 Load Address: 00000000 6.590 Entry Point: 00000000 7.750 Uncompressing Kernel Image ... OK 7.780 ## Loading RAMDisk Image at 40200000 ... 7.780 Image Name: Test root filesystem 7.781 Image Type: PowerPC Linux RAMDisk Image (gzip compressed) 7.781 Data Size: 9642067 Bytes = 9.2 MB 7.781 Load Address: 00000000 7.781 Entry Point: 00000000 8.450 Loading Ramdisk to 03487000, end 03db9053 ... OK 18.996 18.997 18.997 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 18.997 Enter 'help' for a list of built-in commands. 18.997 18.997 # ### Application running ...Time savings: approx 4 seconds or 20% !
4.2 Using uncompressed images:
Instead of using gzip compressed images for the Linux kernel and the root file system image, we now use uncompressed images:1.216 1.217 1.217 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.217 1.217 MPC8260 Reset Status: External Soft, External Hard 1.217 1.217 MPC8260 Clock Configuration 1.217 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.217 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.217 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.217 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.218 1.218 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.446 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.446 I2C: ready 1.446 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.086 FLASH: 32 MB 3.497 In: serial 3.497 Out: serial 3.497 Err: serial 3.497 Net: FCC2 ETHERNET 3.517 3.517 Type "run flash_nfs" to mount root filesystem over NFS 3.517 6.528 Hit any key to stop autoboot: 0 6.529 ###START### 6.609 ## Booting image at 40040000 ... 6.609 Image Name: Linux-2.4.25 LPJ uncompressed 6.609 Image Type: PowerPC Linux Kernel Image (uncompressed) 6.609 Data Size: 1392176 Bytes = 1.3 MB 6.609 Load Address: 00000000 6.609 Entry Point: 00000000 6.698 OK 6.729 ## Loading RAMDisk Image at 40200000 ... 6.729 Image Name: Test root filesystem 6.729 Image Type: PowerPC Linux RAMDisk Image (uncompressed) 6.729 Data Size: 25093120 Bytes = 23.9 MB 6.729 Load Address: 00000000 6.729 Entry Point: 00000000 8.469 Loading Ramdisk to 025cb000, end 03db9400 ... OK 16.324 16.324 16.324 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 16.324 Enter 'help' for a list of built-in commands. 16.325 16.325 # ### Application running ...Time savings: approx 2.7 seconds
Step 5: Choosing the Optimal Root File system Type
So far, all experiments were carried out using a standard ram-disk image as root file system. Obviously, it has to be loaded into RAM as a whole before execution of the first user application can start. This memory copy operation takes time. To avoid it, a flash file system can be used. We change the Linux command line arguments to include "root=/dev/mtdblock2 ro"; then we put a (uncompressed, read-only) ext2 file system image into the corresponding MTD partition. Result:1.152 1.152 1.153 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.153 1.153 MPC8260 Reset Status: External Soft, External Hard 1.153 1.153 MPC8260 Clock Configuration 1.153 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.153 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.153 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.153 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.153 1.154 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.471 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.472 I2C: ready 1.472 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.182 FLASH: 32 MB 3.592 In: serial 3.593 Out: serial 3.593 Err: serial 3.593 Net: FCC2 ETHERNET 3.622 3.623 Type "run flash_nfs" to mount root filesystem over NFS 3.623 6.664 Hit any key to stop autoboot: 0 6.664 ###START### 6.785 ## Booting image at 40040000 ... 6.785 Image Name: Linux-2.4.25 LPJ uncompressed 6.785 Image Type: PowerPC Linux Kernel Image (uncompressed) 6.785 Data Size: 1392176 Bytes = 1.3 MB 6.785 Load Address: 00000000 6.785 Entry Point: 00000000 6.804 OK 10.897 10.897 10.897 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 10.897 Enter 'help' for a list of built-in commands. 10.897 10.897 # ### Application running ...Time savings: approx. 5.4 sec. For a comparison, we try the same using a cramfs root file system image instead: Result:
1.173 1.173 1.173 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.173 1.173 MPC8260 Reset Status: External Soft, External Hard 1.173 1.173 MPC8260 Clock Configuration 1.174 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.174 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.174 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.174 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.174 1.174 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.401 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.401 I2C: ready 1.401 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.193 FLASH: 32 MB 3.602 In: serial 3.602 Out: serial 3.602 Err: serial 3.603 Net: FCC2 ETHERNET 3.632 3.632 Type "run flash_nfs" to mount root filesystem over NFS 3.633 6.654 Hit any key to stop autoboot: 0 6.654 ###START### 6.734 ## Booting image at 40040000 ... 6.735 Image Name: Linux-2.4.25 LPJ uncompressed 6.735 Image Type: PowerPC Linux Kernel Image (uncompressed) 6.735 Data Size: 1392176 Bytes = 1.3 MB 6.735 Load Address: 00000000 6.735 Entry Point: 00000000 6.989 OK 11.057 11.057 11.057 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 11.057 Enter 'help' for a list of built-in commands. 11.057 11.057 # ### Application running ...This is slower, so we stick with the uncompressed ext2 file system (mounted read-only) as root file system.
Step 6: Target File System Optimizations:
So far, we've been using a standard SELF image (in the last examples only with minor modifications to make it work with a read-only root file system). No attempt was made to optimize the startup sequence in the root file system itself. remove xinetd:1.218 1.218 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.218 1.219 MPC8260 Reset Status: External Soft, External Hard 1.219 1.219 MPC8260 Clock Configuration 1.219 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.219 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.219 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.219 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.219 1.219 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.437 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.438 I2C: ready 1.438 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.238 FLASH: 32 MB 3.649 In: serial 3.649 Out: serial 3.649 Err: serial 3.649 Net: FCC2 ETHERNET 3.669 3.669 Type "run flash_nfs" to mount root filesystem over NFS 3.669 6.680 Hit any key to stop autoboot: 0 6.680 ###START### 6.751 ## Booting image at 40040000 ... 6.751 Image Name: Linux-2.4.25 LPJ uncompressed 6.751 Image Type: PowerPC Linux Kernel Image (uncompressed) 6.751 Data Size: 1392176 Bytes = 1.3 MB 6.751 Load Address: 00000000 6.751 Entry Point: 00000000 6.840 OK 10.943 10.943 10.943 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 10.943 Enter 'help' for a list of built-in commands. 10.943 10.943 # ### Application running ...
Step 7: Final Test:
In this test, we set the "bootdelay" variable in U-Boot to 0. (note that this means that there is no standard way to interrupt the automatic boot sequence. You better do not do this if you don't know how to recover from such a situation.)1.254 1.255 1.255 U-Boot 1.0.0 (Oct 29 2003 - 15:42:57) 1.255 1.255 MPC8260 Reset Status: External Soft, External Hard 1.255 1.255 MPC8260 Clock Configuration 1.255 - Bus-to-Core Mult 3.5x, VCO Div 2, 60x Bus Freq 30-85 , Core Freq 100-300 1.255 - dfbrg 0, corecnf 0x1e, busdf 4, cpmdf 1, plldf 1, pllmf 4 1.256 - vco_out 416665000, scc_clk 104166250, brg_clk 104166250 1.256 - cpu_clk 291665500, cpm_clk 208332500, bus_clk 83333000 1.256 1.256 CPU: MPC8260 (HiP4 Rev 14, Mask A.0(A) 2K25A) at 291.665 MHz 1.514 Board: TQM8260DBA0C7A-PJDA.201A 10399642 6 1.514 I2C: ready 1.515 DRAM: (60x:9COL - 64 MB, Local:8COL - 16 MB) using 64 MB 3.154 FLASH: 32 MB 3.564 In: serial 3.565 Out: serial 3.565 Err: serial 3.565 Net: FCC2 ETHERNET 3.594 3.595 Type "run flash_nfs" to mount root filesystem over NFS 3.595 3.606 Hit any key to stop autoboot: 0 3.606 ###START### 3.665 ## Booting image at 40040000 ... 3.665 Image Name: Linux-2.4.25 LPJ uncompressed 3.666 Image Type: PowerPC Linux Kernel Image (uncompressed) 3.666 Data Size: 1355248 Bytes = 1.3 MB 3.666 Load Address: 00000000 3.666 Entry Point: 00000000 3.765 OK 7.868 7.868 7.868 BusyBox v0.60.5 (2004.08.27-12:12+0000) Built-in shell (msh) 7.868 Enter 'help' for a list of built-in commands. 7.868 7.868 # ### Application running ...
Potential for further optimizations:
- The TQM8260 board used in the tests had the U-Boot environment stored in EEPROM; switching to a flash based environment would allow to reduce the initial delay (until the "U-Boot" startup message gets printed) by at least 1.2 seconds.
- It takes more than 1.6 seconds to initialize the SDRAM on this board; on other, similar boards this takes typically less than 0.05 seconds - optimizing the code here would save at least 1.5 seconds.
- Initialization of the flash memory takes over 0.41 seconds on this board, but typically less than 0.05 seconds on other, similar boards - optimizing the code would save at least 0.3 seconds.