[Crash-utility] problem with crash utility in ubuntu

Dave Anderson anderson at redhat.com
Thu Feb 24 14:35:55 UTC 2011



----- Original Message -----
> Hi ,
> 
> I have tried to use crash 4.1.0 in ubuntu 10.04 kernel (2.6.32.25),
> when I don't specify the memory dump file ,it works perfectly as it
> reads the memory from crash driver (crash model), it shows below:
> 
> root at o:/# crash /boot/System.map-2.6.32-25-generic /home/amer/ubuntu-lucid/debian/build/build-generic/vmlinux
> 
> crash 4.1.0
> Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
> 
> GNU gdb 6.1
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for details.
> This GDB was configured as "i686-pc-linux-gnu"...
> 
> SYSTEM MAP: /boot/System.map-2.6.32-25-generic
> DEBUG KERNEL: /home/amer/ubuntu-lucid/debian/build/build-generic/vmlinux (2.6.32-25-generic)
> DUMPFILE: /dev/crash
> CPUS: 1
> DATE: Wed Feb 23 11:40:40 2011
> UPTIME: 09:17:34
> LOAD AVERAGE: 0.89, 0.74, 0.78
> TASKS: 258
> NODENAME: o
> RELEASE: 2.6.32-25-generic
> VERSION: #45-Ubuntu SMP Sat Oct 16 19:48:22 UTC 2010
> MACHINE: i686 (1339 Mhz)
> MEMORY: 511.5 MB
> PID: 2703
> COMMAND: "crash"
> TASK: cf27bfc0 [THREAD_INFO: dfbc0000]
> CPU: 0
> STATE: TASK_RUNNING (ACTIVE)

OK.  

Note that you don't have to use the /boot/System.map-2.6.32-25-generic
file on the command line, presuming that the live kernel is actually running
that same /home/amer/ubuntu-lucid/debian/build/build-generic/vmlinux kernel.

> 
> but when I entered the dump file argument I got error (the memory
> image is obtained from /dev/crash by dd tool), it shows below:
> 
> root at o:/# crash /boot/System.map- 2.6.32 -25-generic /home/amer/ubuntu-lucid/debian/build/build-generic/vmlinux home/amer/image.dd
> crash 4.1.0
> Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License ,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
> 
> crash: home/amer/image.dd: not a supported file format
> 
> Usage:
> crash [-h [opt]][-v][-s][-i file][-d num] [-S] [mapfile] [namelist]
> [dumpfile]
> 
> Enter "crash -h" for details.
> 
> 
> could you please give some advice to resolve this problem.

Yes -- don't do that.  

Running "dd /dev/crash > image.dd" just dumps a bunch of bytes into
the image.dd file.  Dumpfiles by definition need some kind of bookkeeping
header in order for the crash utility to know where the contents of a 
particular page are located in the dumpfile.  Crash supports over
a dozen different dumpfile formats. 

If you want to take a snapshot of your live system without crashing
it, you can try the "snap" extension module:

  # cd <path-to>/crash-5.1.2
  # make extensions
  ...
  #

That will build the "snap.so" extension module (along with a few
others found in the crash-5.1.2/extensions subdirectory).

The run live -- note that normally you don't have to enter anything
on the command line if the debuginfo vmlinux file of the running
kernel is in a "known" location.  I don't know how Ubuntu sets things
up, so it's possible that you at least need to enter the path to
your vmlinux file (and the System.map if that vmlinux file is different
from the running kernel).

Anyway, get a live session going, load the "snap.so" extension module,
and dump kernel memory into an ELF-style dumpfile.  For example:
 
  # crash

  crash 5.1.2
  Copyright (C) 2002-2011  Red Hat, Inc.
  Copyright (C) 2004, 2005, 2006  IBM Corporation
  Copyright (C) 1999-2006  Hewlett-Packard Co
  Copyright (C) 2005, 2006  Fujitsu Limited
  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
  Copyright (C) 2005  NEC Corporation
  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
  This program is free software, covered by the GNU General Public License,
  and you are welcome to change it and/or distribute copies of it under
  certain conditions.  Enter "help copying" to see the conditions.
  This program has absolutely no warranty.  Enter "help warranty" for details.
   
  GNU gdb (GDB) 7.0
  Copyright (C) 2009 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "x86_64-unknown-linux-gnu"...
  
        KERNEL: /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux
      DUMPFILE: /dev/crash
          CPUS: 8
          DATE: Thu Feb 24 09:21:24 2011
        UPTIME: 37 days, 00:57:58
  LOAD AVERAGE: 0.29, 0.38, 0.18
         TASKS: 262
      NODENAME: crash.usersys.redhat.com
       RELEASE: 2.6.18-128.el5
       VERSION: #1 SMP Wed Dec 17 11:41:38 EST 2008
       MACHINE: x86_64  (1995 Mhz)
        MEMORY: 1 GB
           PID: 27743
       COMMAND: "crash"
          TASK: ffff81002750a820  [THREAD_INFO: ffff81003db8c000]
           CPU: 2
         STATE: TASK_RUNNING (ACTIVE)
  
  crash> extend snap.so
  ./extensions/snap.so: shared object loaded
  crash> snap live.dump
  live.dump: [100%] 
  -rw-r--r-- 1 root root 1071686392 Feb 24 09:18 live.dump
  crash> q
  #
  
Then use the "live.dump" dumpfile:

  # crash /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux live.dump
  
  crash 5.1.2
  Copyright (C) 2002-2011  Red Hat, Inc.
  Copyright (C) 2004, 2005, 2006  IBM Corporation
  Copyright (C) 1999-2006  Hewlett-Packard Co
  Copyright (C) 2005, 2006  Fujitsu Limited
  Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
  Copyright (C) 2005  NEC Corporation
  Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
  Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
  This program is free software, covered by the GNU General Public License,
  and you are welcome to change it and/or distribute copies of it under
  certain conditions.  Enter "help copying" to see the conditions.
  This program has absolutely no warranty.  Enter "help warranty" for details.
   
  GNU gdb (GDB) 7.0
  Copyright (C) 2009 Free Software Foundation, Inc.
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  and "show warranty" for details.
  This GDB was configured as "x86_64-unknown-linux-gnu"...
  
        KERNEL: /usr/lib/debug/lib/modules/2.6.18-128.el5/vmlinux
      DUMPFILE: live.dump
          CPUS: 8
          DATE: Thu Feb 24 09:18:13 2011
        UPTIME: 37 days, 00:54:47
  LOAD AVERAGE: 0.31, 0.11, 0.06
         TASKS: 257
      NODENAME: crash.usersys.redhat.com
       RELEASE: 2.6.18-128.el5
       VERSION: #1 SMP Wed Dec 17 11:41:38 EST 2008
       MACHINE: x86_64  (1995 Mhz)
        MEMORY: 1 GB
         PANIC: ""
           PID: 27603
       COMMAND: "crash"
          TASK: ffff81001a57f100  [THREAD_INFO: ffff81001ad22000]
           CPU: 2
         STATE: TASK_RUNNING (PANIC)
  
  crash> 

You may see some command errors because the system is not frozen when
the dump is taken, so vital things may be changing while the memory is
being copied.  For example, "bt" on the active tasks may show nonsensical
data, given that they were running when the memory was being copied.

Hope this helps,
  Dave
   
 




More information about the Crash-utility mailing list