[Crash-utility] Re: Viewing module symbols using crash
Durga Prasad
durgaprasadsharma at gmail.com
Sat Aug 9 14:55:15 UTC 2008
> Message: 1
> Date: Fri, 8 Aug 2008 16:10:34 +0530
> From: "Durga Prasad" <durgaprasadsharma at gmail.com>
> Subject: [Crash-utility] Viewing module symbols using crash
> To: crash-utility at redhat.com
> Message-ID:
> <4295d5fd0808080340h3996b141oacb4eee4eb9ee00e at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> [1]:
> I need to debug a kernel module (a device driver).
> I added the desired module to crash using mod -S <path to module>. The
> driver is compiled with -g option.
> I could not see the structures defined within the module.
>
> Am I missing something?
>
> System info:
> KERNEL: vmlinux.sles10sp2.smp.x86_64
> DUMPFILE: 2008-08-08-18:57/vmcore
> CPUS: 4
>
> RELEASE: 2.6.16.60-0.21-smp
> VERSION: #1 SMP Tue May 6 12:41:02 UTC 2008
> MACHINE: x86_64 (1995 Mhz)
> MEMORY: 3.9 GB
>
>
>
> [2]:
> I could get into live kernel debugging using crash on RHEL 5 after
> installing relevant kernel-debug packages. How do I do the same on SLES 10?
> As of now, I have been compiling the kernel with CONFIG_DEBUG_INFO set. I
> can use the resulting vmlinux for post crash analysis, but not for live
> analysis. Any light on how to accomplish this would be appreciated.
>
>
> Thanks
> Durga Prasad
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> https://www.redhat.com/archives/crash-utility/attachments/20080808/bd7e28cf/attachment.html
>
> ------------------------------
>
> Message: 2
> Date: Fri, 08 Aug 2008 09:32:09 -0400
> From: Dave Anderson <anderson at redhat.com>
> Subject: Re: [Crash-utility] Viewing module symbols using crash
> To: "Discussion list for crash utility usage, maintenance and
> development" <crash-utility at redhat.com>
> Cc: Bernhard Walle <bwalle at suse.de>
> Message-ID: <489C4AD9.4010309 at redhat.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Durga Prasad wrote:
> > Hi,
> >
> > [1]:
> > I need to debug a kernel module (a device driver).
> > I added the desired module to crash using mod -S <path to module>. The
> > driver is compiled with -g option.
> > I could not see the structures defined within the module.
> >
> > Am I missing something?
>
> Accessing data in kernel modules can be spotty at best,
> but the embedded gdb module should typically be able
> to pick up structure definitions in modules.
>
> For example, these two structures are defined in ./fs/ext3/xattr.c
> like so:
>
> struct ext3_xattr_info {
> int name_index;
> const char *name;
> const void *value;
> size_t value_len;
> };
>
> struct ext3_xattr_search {
> struct ext3_xattr_entry *first;
> void *base;
> void *end;
> struct ext3_xattr_entry *here;
> int not_found;
> };
>
> Without the ext3 module debuginfo, crash (gdb) doesn't know what
> they are:
>
> # crash -s
> crash> struct ext3_xattr_search
> struct: invalid data structure reference: ext3_xattr_search
> crash> struct ext3_xattr_info
> struct: invalid data structure reference: ext3_xattr_info
> crash>
>
> Until after adding the module debuginfo data:
>
> crash> mod -s ext3
> MODULE NAME SIZE OBJECT FILE
> ffffffff8806be00 ext3 168017
> /lib/modules/2.6.18-102.el5.perfmon2/kernel/fs/ext3/ext3.ko
> crash> struct ext3_xattr_search
> struct ext3_xattr_search {
> struct ext3_xattr_entry *first;
> void *base;
> void *end;
> struct ext3_xattr_entry *here;
> int not_found;
> }
> SIZE: 40
> crash> struct ext3_xattr_info
> struct ext3_xattr_info {
> int name_index;
> const char *name;
> const void *value;
> size_t value_len;
> }
> SIZE: 32
> crash>
>
> Then you may be able to "p" individual instances of the structures
> if there are symbolic/static declaration of them. Otherwise you
> can always cast the address of the structure to the structure type,
> as in:
>
> crash> struct ext3_xattr_info <address>
>
> If you cannot get the structure declaration after using "mod -[sS]",
> then you'll have to get primitive, and just dump the data at the
> relevant address with "rd".
>
> >
> > System info:
> > KERNEL: vmlinux.sles10sp2.smp.x86_64
> > DUMPFILE: 2008-08-08-18:57/vmcore
> > CPUS: 4
> >
> > RELEASE: 2.6.16.60-0.21-smp
> > VERSION: #1 SMP Tue May 6 12:41:02 UTC 2008
> > MACHINE: x86_64 (1995 Mhz)
> > MEMORY: 3.9 GB
> >
> >
> >
> > [2]:
> > I could get into live kernel debugging using crash on RHEL 5 after
> > installing relevant kernel-debug packages. How do I do the same on SLES
> 10?
> > As of now, I have been compiling the kernel with CONFIG_DEBUG_INFO set.
> > I can use the resulting vmlinux for post crash analysis, but not for
> > live analysis. Any light on how to accomplish this would be appreciated.
>
> For SLES I'm guessing that you could use the post-built (CONFIG_DEBUG_INFO)
> kernel,
> but also add the System.map file of the live kernel to the command line, as
> in:
>
> $ crash /boot/System.map vmlinux-post-built-with-debuginfo
>
> Crash will use the debuginfo data from the vmlinux file, but then go in
> and back-patch all the vmlinux file's minimal_symbols with the correct
> symbol values from the System.map file.
>
> But Bernhard Walle (bwalle at suse.de) can answer that part of your question.
> He's active on this list, but I've added him to the cc: list to make
> sure he sees this.
>
> Dave Anderson
>
>
> >
> > Thanks
> > Durga Prasad
> >
>
Hi Dave,
I attempted to try your suggested commands, but it tends to fail on my
machine:
libl041:/var/log/dump # crash -s vmlinux.sles10sp2.smp.x86_64
2008-08-08-23\:27/vmcore
crash> struct ext3_xattr_search
struct: invalid data structure reference: ext3_xattr_search
crash> mod -s ext3
MODULE NAME SIZE OBJECT FILE
ffffffff8856a500 ext3 153360
/lib/modules/2.6.16.60-0.21-smp/kernel/fs/ext3/ext3.ko
crash> struct ext3_xattr_search
struct: invalid data structure reference: ext3_xattr_search
crash> struct ext3_xattr_info
struct: invalid data structure reference: ext3_xattr_info
crash> sys
KERNEL: vmlinux.sles10sp2.smp.x86_64
DUMPFILE: 2008-08-08-23:27/vmcore
CPUS: 4
DATE: Fri Aug 8 23:20:09 2008
UPTIME: 01:31:48
LOAD AVERAGE: 2.48, 1.32, 0.56
TASKS: 157
NODENAME: libl041
RELEASE: 2.6.16.60-0.21-smp
VERSION: #1 SMP Tue May 6 12:41:02 UTC 2008
MACHINE: x86_64 (1995 Mhz)
MEMORY: 3.9 GB
PANIC: "Oops: 0000 [1] SMP " (check log for details)
crash version: 4.0-4.10 gdb version: 6.1
I tried the same after compiling and installing the latest crash version
(crash version: 4.0-6.3 gdb version: 6.1) with no difference in results.
What would you suggest?
Thanks
Durga
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/crash-utility/attachments/20080809/5503687c/attachment.htm>
More information about the Crash-utility
mailing list