Debugging < PPCEmbedded

%SECTION0{name=Debugging}% Debugging

%SECTION1{name=BDM}% BDM

BDM is way cool, especially if you are bringing up a custom board and don't
want to invest in an In Circuit Emulator. You should definitely include a BDM
connector on any custom hardware, and all off-the-shelf boards have it. For a
broad introduction, see [[http://www.macraigor.com/zenofbdm.pdf]]

Be careful to ensure that the CPU watchdog is disabled in the configuration
file for your BDM probes, otherwise it will continually reset the CPU and
nothing will work. In particular, the watchdog is enabled at reset and must be
disabled if you wish to single step from reset. Also check that DER is zero'd
when running with the debugger. Leaving the BDM probe connected can interfere
with the target if it is not configured absolutely correctly, which usually
involves the BDM probe being completely passive once the kernel is running.
Symptoms are that the kernel crashes with the BDM probe connected, but runs
fine without it. If you are using BDM and experience unexpected kernel crashes,
try disconnecting the BDM probe.

If you're using an off-the-shelf board which already has a working
[[ROMMonitor]], you generally won't need to use BDM at all, as
you can get by fine with just the 
%REF{type=Section,name=SerialConsole,text=serialconsole}%.
Many developers, particularly those bringing up a custom board from scratch, find
BDM invaluable; to others it can be more trouble than it's worth. One advantage
of a BDM interface over a kernel debugger is that you can really "freeze" the
CPU including all timers, interrupts etc.

%SECTION2{name=MMUSupport}% MMU Support

Some BDM debuggers are capable of performing [[MMU]] table walks, which
is essential for debugging in virtual memory environments such as Linux because
otherwise the BDM port merely deals with raw physical addresses. However, many
BDM systems can't do this, so check with the vendor that they support MMU table
walks on the particular CPU you're interested in before committing to one;
otherwise it will be almost impossible to use once the kernel has turned the
MMU on. See
[[http://lists.linuxppc.org/listarcs/linuxppc-embedded/199909/msg00021.html]]

Once the kernel is running you can
%REF{type=Section,name=GDB,text="use gdb remotely"}%.
or even run it
natively on the target hardware if you have enough RAM. It may be worth
providing extra RAM on some of your development boards to allow for this.

%SECTION2{name=GDBonBDM}% GDB on BDM

See "Using GDB for Remote Debugging: BDM Support" in the CrossGCC FAQ at
[[http://www.objsw.com/CrossGCC/FAQ-7.html]]

BDM debugging under *gdb* is not supported by all BDM hardware vendors. See the
thread
[[http://www.oarcorp.com/rtems/maillistArchives/rtems-users/1999/july/msg00057.html]]

To support BDM debugging under *gdb* requires the appropriate remote-<nop>* device
in *gdb*, and possibly a kernel driver. If you are cross-developing, you must
configure *gdb* as follows to include the appropriate devices:
<verbatim>
% configure --target=powerpc-linux
</verbatim>                                                                                      

The following systems currently support BDM debugging on *gdb*:

%SECTION3{name=BDM4GDB}% BDM4GDB

   * [[http://bdm4gdb.sourceforge.net/]]

An open user-supported project to build your own PowerPC BDM interface at
extremely low cost. Supports extensible Flash programming and implements
software-tablewalk so you have
%REF{type=Section,name=MMUSupport,text=MMUSupport}%.
You can use *gdb* to
debug kernel code (single stepping, breakpoints, etc.) and/or inspect kernel
data.

If you don't want to build it yourself, the adapter is available at cost from
%REF{type=Section,topic=CompilerToolset,name=DENX,text=DENX}%,
subject to availability.

%SECTION3{name=MPCBDM}% MPCBDM

   * [[http://www.vas-gmbh.de/software/mpcbdm/]]

The precursor to the 
%REF{type=Section,name=BDM4GDB,text=BDM4GDB}%
project, courtesy of Frank Przybylski.

%SECTION3{name=PPCBDM}% PPCBDM

   * [[http://cyclone.parad.ru/ppcbdm/]]

The ultra low hardware cost precursor to
%REF{type=Section,name=MPCBDM,text=MPCBDM}%
and
%REF{type=Section,name=BDM4GDB,text=BDM4GDB}%,
courtesy of Sergey Drazhnikov.

%SECTION3{name=Abatron}% Abatron

   * [[http://www.abatron.ch/BDI/bdiGDB.html]]

Has an ideal BDM/JTAG emulation solution for Linux hosting, with Linux
%REF{type=Section,name=MMUSupport,text=MMUSupport}%
specifically in mind. It is an Ethernet-based unit
which has a telnet interface with which one can program various popular flash
parts. This is ideal for Linux since one can build a kernel and expect script
the programming process automagically.

Their external box implements the standard *gdb* remote protocol allowing you
to host debugging on any platform that can host *gdb*.

%SECTION3{name=AvocetSystems}% Avocet Systems (formerly Huntsville Microsystems)

   * [[http://www.hmi.com/bmd.htm]]

Patches to add support to *gdb* are available in [[ftp://ftp.hmi.com/pub/gdb/]]

%SECTION3{name=MacraigorSystems}% Macraigor Systems

   * [[http://www.macraigor.com/]]

Currently only support Windows platform, but say Linux support for the Kestrel
device (but not the Wiggler, Raven, etc) should be available sometime in
September 2000.

OCD Commander is a free assembly level debugger for Windows, which may work
under VMWare. I couldn't get it to work with the Raven/Blackbird interface on
the Embedded Planet CLLF under Windows; it wouldn't single-step correctly.

Beware that the 1999 version of OCD Commander silently truncates S-records
longer than 20 bytes during download, so you'll need to patch =objcopy= or
write your own S-record generator to work around this.

They have flash programming software available for download, but the Web site
doesn't mention that it isn't actually free: you have to pay for the key to use
it.

If you're prepared to do your debugging under Windows, you can use the Cygwin
tools from [[http://sources.redhat.com/cygwin/]] to build a *gdb*
cross-debugger which runs under Windows and uses the =Wigglers.dll= to talk to
the target. Note that =Wigglers.dll= also loads other DLLs, so copy all the
DLLs that come with OCD Commander from [[http://www.macraigor.com/]] into your
gdb directory and use the command:
<verbatim>
(gdb) target ocd wiggler
</verbatim>
                                                                                              
You can also do this remotely, by using %REF{type=Section,name=rproxy,text=rproxy}%
described below.

%SECTION3{name=RISCWatch}% RISCWatch Debugger for PowerPC Processors

   * [[http://www.chips.ibm.com/products/powerpc/riscwtch/]]

RISCWatch is a hardware and software development tool for the PowerPC 600/700
Family of microprocessors and the PowerPC 400Series of Embedded Controllers.
The source-level debugger and processor-control features provide developers
with the tools needed to develop and debug hardware and software.

%SECTION3{name=WindRiver}% WindRiver (formerly EST Corporation)

   * [[http://www.windriver.com/products/html/embed_dev_tools.html]]

They've ported their VisionXD UNIX debugger to Linux and it is available with
their parallel port of Ethernet-based probe. It can program flash parts and do
source level debugging. It costs $6k or $8k depending on parport or Ethernet
connection. The Windows version is only $4k. :-/

Beware that the EST tools don't handle the Linux zImage properly. See
[[http://lists.linuxppc.org/listarcs/linuxppc-embedded/200002/msg00073.html]]

%SECTION2{name=OtherDebuggers}% Other debuggers

There are many other sources of BDM hardware probes and debuggers, but many
vendors still lag way behind as far as Linux support. Commercial solutions
which do not support a Linux development host include:

%SECTION3{name=Mentor}% Mentor (X-ray)

   * [[http://www.mentor.com/embedded/xray/index.html]]

%SECTION3{name=AvocetSystems}% Avocet Systems (SourceGate)

   * [[http://www.hmi.com/bmd.htm]]

%SECTION3{name=Tasking}% Tasking (CrossView Pro)

   * [[http://www.tasking.com/technology/xvwp.html]]

%SECTION3{name=WindRiverSingleStep}% WindRiver (SingleStep)

   * [[http://www.windriver.com/products/html/singlestep_suite_ds.html]]

%SECTION3{name=AppliedMicrosystems}% Applied Microsystems (PowerTap)

   * [[http://www.amc.com/products/powertap.html]]

%SECTION3{name=Lauterbach}% Lauterbach

   * [[http://www.lauterbach.com/]]

%SECTION1{name=SerialConsole}% Serial Console

Virtually all boards use a serial console on
[[SMC][SMC1]]
for boot messages
and general debugging. Connect it to a serial port on your Linux development
machine, where you can run minicom to interact with the board.

%SECTION1{name=GDB}% GDB

Once the kernel is running, you can use *gdb* in several different ways to
debug user space programs:

%SECTION2{name=1gdbserver}% gdbserver

You can run gdbserver on your target and run *gdb* back on your development
machine, even if you're cross-developing. This requires far less resources than
running all of *gdb* on your target. See
[[http://qslinux.org/docs/cross/gdb/index.html]]

If you're cross-developing, remember to configure your *gdb* as described
%REF{type=Section,name=GDBonBDM,text=earlier}%.

%SECTION2{name=rproxy}% rproxy

   * [[http://www.std.com/qqi/labslave/rproxy.html]]

This in an extended/enhanced gdbserver, which can also run on Windows and talk
to BDM devices not yet supported by Linux.

%SECTION2{name=native}% native

If you have lots of RAM, you can run *gdb* directly on your target. If you are
cross-developing, you need to configure *gdb* with:
<verbatim>
% configure --host=powerpc-linux
</verbatim>

%SECTION1{name=Kernel}% Kernel

Some kernels include *kgdb* support for using *gdb* for kernel debugging,
enabled by configuring with =CONFIG_KGDB=.

%SECTION1{name=OopsMessages}% Oops Messages

You get these whenever something truly bad happens in the kernel. Learn to know
and respect them -- they are your friends, not your enemies. For general info
on how to understand them, see the file =Documentation/oops-tracing.txt= in the
kernel source tree.

You'll get a long way just by looking up the instructions at the address
indicated by NIP on the first line of the Oops in the output of:
<verbatim>
objdump --disassemble vmlinux
</verbatim>
This will show you the instruction causing the fault. Work backwards to find
the line of C source code associated with it, and add printk's around it to
find what is going wrong.

For more help with decoding kernel panic messages, see
[[http://lists.linuxppc.org/listarcs/linuxppc-embedded/199912/msg00090.html]]

%SECTION1{name=printk}% printk

=printk= is an indispensible tool. You can use it to add checkpointing, print
kernel values that you can't get to via =/proc=, etc. It can be called
anywhere, including interrupt routines, provided you're prepared for some
interesting output.

Note that during the boot process, the kernel "prints" lots of stuff, and it
all goes into a buffer, to emerge quite late in the boot process when the
serial console port is initialized with the call to =console_init=. This
eventually calls =register_console= which will dump out the logged messages. So
you can't necessarily assume that the kernel didn't get to your checkpoint just
because the printk message didn't appear on the serial port during this part of
the boot sequence.
17. Applications		1. Introduction		19. Performance
Prev		Home		Next