[Crash-utility] crash cannot read the symbols
Koornstra, Reinoud
koornstra at hp.com
Tue Sep 20 22:28:56 UTC 2011
Hmm, the /dev/mem does not reflect the kernel and symbols I am trying to read, because I do not have a core dump of the crash.
I just tried to read the kernel and modules in crash to read it.
crash 5.1.8
Copyright (C) 2002-2011 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
get_live_memory_source: /dev/mem
NOTE: stdin: not a tty
Symbol list
[SNIP}
GNU gdb (GDB) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
GETBUF(128 -> 0)
GETBUF(1500 -> 1)
FREEBUF(1)
FREEBUF(0)
GETBUF(128 -> 0)
GNU_GET_DATATYPE[cpumask_t]: returned via gdb_error_hook (1 buffer in use)
FREEBUF(0)
GETBUF(4 -> 0)
FREEBUF(0)
<readmem: c032e840, KVADDR, "xtime", 8, (FOE), 851c5d8>
addr: c032e840 paddr: 32e840 cnt: 8
/dev/mem: Operation not permitted
crash: read(/dev/mem, 32e840, 0): 8 (ffffffff)
crash: read error: kernel virtual address: c032e840 type: "xtime"
-----Original Message-----
From: crash-utility-bounces at redhat.com [mailto:crash-utility-bounces at redhat.com] On Behalf Of Dave Anderson
Sent: Tuesday, September 20, 2011 1:54 PM
To: Discussion list for crash utility usage, maintenance and development
Subject: Re: [Crash-utility] crash cannot read the symbols
----- Original Message -----
> Hi Dave, thanks for your answer.
> The basic procedure forgotten, I assumed -g would be included.
> Never mind, I compile the kernel with -g.
> However, crash still doesn't like it:
It seems to be having a problem reading /dev/mem. I don't know if
2.4.37.9 allows the restriction of /dev/mem to the first 256 pages of memory when the kernel is configured with CONFIG_STRICT_DEVMEM.
What is the output of:
$ sudo crash -d8 vmlinux System.map
Dave
>
> einoud at superchi:~/Pretoria/pretoria/kernel2.4/build/emPC/GREYHOUND/lin
> ux-2.4.37.9$
>
>
> crash 5.1.8
> Copyright (C) 2002-2011 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C)
> 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu
> Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License, and you are welcome to change it and/or distribute copies of
> it under certain conditions. Enter "help copying" to see the
> conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.0
> Copyright (C) 2009 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "i686-pc-linux-gnu"...
>
> crash: read error: kernel virtual address: c032e840 type: "xtime"
>
> I did some stepping through crash, maybe that'll be helpful as to see
> what happened?
>
> crash 5.1.8
> Copyright (C) 2002-2011 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C)
> 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu
> Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License, and you are welcome to change it and/or distribute copies of
> it under certain conditions. Enter "help copying" to see the
> conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.0
> Copyright (C) 2009 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "i686-pc-linux-gnu"...
>
> please wait... (patching 13083 gdb minimal_symbol values) Breakpoint
> 1, readmem (addr=3224561728, memtype=1, buffer=0x851c5d8, size=8,
> type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:1849
> 1849 {
> (gdb) step
> 1856 if (CRASHDEBUG(4))
> (gdb) step
> 1849 {
> (gdb) step
> 1856 if (CRASHDEBUG(4))
> (gdb) step
> 1863 if (size <= 0) {
> (gdb) step
> 1861 bufptr = (char *)buffer;
> (gdb) step
> 1863 if (size <= 0) {
> (gdb) step
> 1870 fd = REMOTE_MEMSRC() ? pc->sockfd : (ACTIVE() ?
> pc->mfd : pc->dfd);
> (gdb) step
> 1875 switch (memtype)
> (gdb) p fd
> $1 = 6
> (gdb) step
> 1891 if (LKCD_DUMPFILE())
> (gdb) step
> 1894 if (!IS_KVADDR(addr)) {
> (gdb) step
> generic_is_kvaddr (addr=3224561728) at memory.c:1739
> 1739 {
> (gdb) step
> 1741 }
> (gdb) step
> generic_is_kvaddr (addr=3224561728) at memory.c:1739
> 1739 {
> (gdb) step
> generic_is_kvaddr (addr=3224561728) at memory.c:1741
> 1741 }
> (gdb) step
> readmem (addr=3224561728, memtype=1, buffer=0x851c5d8, size=8,
> type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:1921
> 1921 if (!kvtop(CURRENT_CONTEXT(), addr,
> &paddr, 0)) {
> (gdb) step
> 1910 switch (memtype)
> (gdb) step
> 1921 if (!kvtop(CURRENT_CONTEXT(), addr,
> &paddr, 0)) {
> (gdb) step
> kvtop (tc=0x0, kvaddr=3224561728, paddr=0xbfffdc20, verbose=0) at
> memory.c:2615
> 2615 {
> (gdb) step
> 2618 return (machdep->kvtop(tc ? tc : CURRENT_CONTEXT(),
> kvaddr,
> (gdb) step
> 2615 {
> (gdb) step
> 2618 return (machdep->kvtop(tc ? tc : CURRENT_CONTEXT(),
> kvaddr,
> (gdb) step
> x86_kvtop (tc=0x0, kvaddr=3224561728, paddr=0xbfffdc20, verbose=0) at
> x86.c:2753
> 2753 if (!IS_KVADDR(kvaddr))
> (gdb) step
> 2743 {
> (gdb) step
> 2753 if (!IS_KVADDR(kvaddr))
> (gdb) step
> generic_is_kvaddr (addr=3224561728) at memory.c:1739
> 1739 {
> (gdb) step
> 1741 }
> (gdb) step
> generic_is_kvaddr (addr=0) at memory.c:1739
> 1739 {
> (gdb) step
> generic_is_kvaddr (addr=0) at memory.c:1741
> 1741 }
> (gdb) step
> x86_kvtop (tc=0x0, kvaddr=3224561728, paddr=0xbfffdc20, verbose=0) at
> x86.c:2756
> 2756 if (XEN_HYPER_MODE()) {
> (gdb) step
> 2763 if (!vt->vmalloc_start) {
> (gdb) step
> 2764 *paddr = VTOP(kvaddr);
> (gdb) step
> 2765 return TRUE;
> (gdb) step
> 2862 }
> (gdb) step
> kvtop (tc=0x0, kvaddr=3224561728, paddr=0xbfffdc20, verbose=0) at
> memory.c:2620
> 2620 }
> (gdb) step
> readmem (addr=3336256, memtype=1, buffer=0x851c5d8, size=8,
> type=0x840c9c6 "xtime", error_handle=1) at memory.c:1947
> 1947 cnt = PAGESIZE() - PAGEOFFSET(paddr);
> (gdb) step
> 1952 if (CRASHDEBUG(8))
> (gdb) step
> 1956 if (memtype == KVADDR)
> (gdb) step
> 1957 pc->curcmd_flags |= MEMTYPE_KVADDR;
> (gdb) step
> 1961 switch (READMEM(fd, bufptr, cnt,
> (gdb) step
> read_dev_mem (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728,
> paddr=3336256) at memory.c:2027
> 2027 {
> (gdb) step
> 2030 if (!machdep->verify_paddr(paddr)) {
> (gdb) step
> generic_verify_paddr (paddr=3336256) at memory.c:2020
> 2020 }
> (gdb) step
> read_dev_mem (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728,
> paddr=3336256) at memory.c:2043
> 2043 if (vt->high_memory && (paddr >=
> (physaddr_t)(VTOP(vt->high_memory)))) {
> (gdb) step
> 2049 if (lseek(fd, (off_t)paddr, SEEK_SET) == -1)
> (gdb) step
> 2026 read_dev_mem(int fd, void *bufptr, int cnt, ulong addr,
> physaddr_t paddr)
> (gdb) step
> 2053 errno = 0;
> (gdb) step
> 2054 readcnt = read(fd, bufptr, cnt);
> (gdb) step
> read (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728, paddr=3336256)
> at /usr/include/bits/unistd.h:45
> 45 return __read_alias (__fd, __buf, __nbytes);
> (gdb) step
> read_dev_mem (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728,
> paddr=3336256) at memory.c:2056
> 2056 if ((readcnt != cnt) && CRASHDEBUG(4)) {
> (gdb) step
> 2054 readcnt = read(fd, bufptr, cnt);
> (gdb) step
> read (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728, paddr=3336256)
> at /usr/include/bits/unistd.h:45
> 45 return __read_alias (__fd, __buf, __nbytes);
> (gdb) step
> read_dev_mem (fd=6, bufptr=0x851c5d8, cnt=8, addr=3224561728,
> paddr=3336256) at memory.c:2056
> 2056 if ((readcnt != cnt) && CRASHDEBUG(4)) {
> (gdb) step
> 2089 if (readcnt != cnt)
> (gdb) step
> 2079 if ((readcnt != cnt) && readcnt && (machdep->flags &
> DEVMEMRD) &&
> (gdb) step
> 2093 }
> (gdb) step
> readmem (addr=<value optimized out>, memtype=1, buffer=0x851c5d8,
> size=8, type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:1962
> 1962 (memtype == PHYSADDR) || (memtype ==
> XENMACHADDR) ? 0 : addr, paddr))
> (gdb) step
> 1970 if (PRINT_ERROR_MESSAGE)
> (gdb) step
> 1971 error(INFO, READ_ERRMSG,
> memtype_string(memtype, 0), addr, type);
> (gdb) step
> memtype_string (memtype=1, debug=0) at memory.c:2325
> 2325 switch (memtype)
> (gdb) step
> 2331 sprintf(membuf, debug ? "KVADDR" : "kernel
> virtual");
> (gdb) step
> 2340 sprintf(membuf, debug ? "FILEADDR" :
> "dumpfile");
> (gdb) step
> sprintf (memtype=<value optimized out>, debug=0) at
> /usr/include/bits/stdio2.h:34
> 34 return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL -
> 1,
> (gdb) step
> memtype_string (memtype=<value optimized out>, debug=<value optimized
> out>) at memory.c:2351
> 2351 }
> (gdb) step
> __error (type=1, fmt=0x83c90d8 "read error: %s address: %llx type:
> \"%s\"\n") at tools.c:41
> 41 {
> (gdb) step
> 48 if (CRASHDEBUG(1) || (pc->flags & DROP_CORE)) {
> (gdb) step
> 41 {
> (gdb) step
> 45 void *retaddr[NUMBER_STACKFRAMES] = { 0 };
> (gdb) step
> 48 if (CRASHDEBUG(1) || (pc->flags & DROP_CORE)) {
> (gdb) step
> 45 void *retaddr[NUMBER_STACKFRAMES] = { 0 };
> (gdb) step
> 48 if (CRASHDEBUG(1) || (pc->flags & DROP_CORE)) {
> (gdb) step
> 40 __error(int type, char *fmt, ...)
> (gdb) step
> 55 (void)vsnprintf(buf, BUFSIZE, fmt, ap);
> (gdb) step
> vsnprintf (type=1, fmt=0x83c90d8 "read error: %s address: %llx type:
> \"%s\"\n") at /usr/include/bits/stdio2.h:78
> 78 return __builtin___vsnprintf_chk (__s, __n,
> __USE_FORTIFY_LEVEL - 1,
> (gdb) step
> __error (type=1, fmt=0x83c90d8 "read error: %s address: %llx type:
> \"%s\"\n") at tools.c:58
> 58 if (!fmt && FATAL_ERROR(type)) {
> (gdb) step
> 63 end_of_line = FATAL_ERROR(type) && !(pc->flags &
> RUNTIME);
> (gdb) step
> 65 if ((new_line = (buf[0] == '\n')))
> (gdb) step
> 67 else if (pc->flags & PLEASE_WAIT)
> (gdb) step
> 70 if (type == CONT)
> (gdb) step
> 67 else if (pc->flags & PLEASE_WAIT)
> (gdb) step
> 70 if (type == CONT)
> (gdb) step
> 75 if (pc->stdpipe) {
> (gdb) step
> 85 fprintf(stdout, "%s%s%s %s%s",
> (gdb) step
> fprintf (type=1, fmt=0x83c90d8 "read error: %s address: %llx type:
> \"%s\"\n") at /usr/include/bits/stdio2.h:98
> 98 return __fprintf_chk (__stream, __USE_FORTIFY_LEVEL - 1,
> __fmt,
> (gdb) step
> crash: read error: kernel virtual address: c032e840 type: "xtime"
> __error (type=1, fmt=0x83c90d8 "read error: %s address: %llx type:
> \"%s\"\n") at tools.c:92
> 92 fflush(stdout);
> (gdb) step
> 95 if ((fp != stdout) && (fp != pc->stdpipe)) {
> (gdb) step
> 105 if ((pc->flags & DROP_CORE) && (type != NOTE)) {
> (gdb) step
> 111 switch (type)
> (gdb) step
> 133 }
> (gdb) step
> readmem (addr=3224561728, memtype=1, buffer=0x851c5d8, size=8,
> type=0x840c9c6 "xtime",
> error_handle=<value optimized out>) at memory.c:1992
> 1992 switch (error_handle)
> (gdb) step
> 1995 if ((pc->flags & DEVMEM) && (kt->flags &
> PRE_KERNEL_INIT) &&
> (gdb) step
> 1996 devmem_is_restricted() &&
> switch_to_proc_kcore())
> (gdb) step
> devmem_is_restricted (addr=3224561728, memtype=1, buffer=0x851c5d8,
> size=8, type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:2137
> 2137 if (STREQ(pc->live_memsrc, "/dev/mem") &&
> (gdb) step
> readmem (addr=3224561728, memtype=1, buffer=0x851c5d8, size=8,
> type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:1996
> 1996 devmem_is_restricted() &&
> switch_to_proc_kcore())
> (gdb) step
> devmem_is_restricted (addr=3224561728, memtype=1, buffer=0x851c5d8,
> size=8, type=0x840c9c6 "xtime", error_handle=1)
> at memory.c:2138
> 2138 kernel_symbol_exists("devmem_is_allowed") &&
> (gdb) step
> kernel_symbol_exists (symbol=0x83c63dc "devmem_is_allowed") at
> symbols.c:4722
> 4722 if ((sp = symname_hash_search(symbol)))
> (gdb) step
> symname_hash_search (name=0x83c63dc "devmem_is_allowed") at
> symbols.c:889
> 889 sp = st->symname_hash[SYMNAME_HASH_INDEX(name)];
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step
> 892 if (STREQ(sp->name, name))
> (gdb) step
> 894 sp = sp->name_hash_next;
> (gdb) step 10
> 891 while (sp) {
> (gdb) step 20
> 894 sp = sp->name_hash_next;
> (gdb) step
> 891 while (sp) {
> (gdb) step 100
>
> Program exited with code 01.
> Thanks,
>
> Reinoud.
>
> -----Original Message-----
> From: crash-utility-bounces at redhat.com
> [mailto:crash-utility-bounces at redhat.com] On Behalf Of Dave Anderson
> Sent: Thursday, September 15, 2011 6:07 AM
> To: Discussion list for crash utility usage, maintenance and
> development
> Subject: Re: [Crash-utility] crash cannot read the symbols
>
>
>
> ----- Original Message -----
> > Hi Everyone,
> >
> > I am just trying load a kernel and System.map file into crash.
> >
> > crash vmlinux System.map
> >
> > I get:
> >
> > crash 5.1.7
> > Copyright (C) 2002-2011 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation Copyright (C)
> > 1999-2006
> > Hewlett-Packard Co Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright
> > (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License, and you are welcome to change it and/or distribute copies
> > of it under certain conditions. Enter "help copying" to see the
> > conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > GNU gdb (GDB) 7.0
> > Copyright (C) 2009 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later
> > <http://gnu.org/licenses/gpl.html>
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "i686-pc-linux-gnu"...
> >
> > crash: vmlinux: no debugging data available
> >
> > However, the System.map file reads fine and “nm vmlinux” shows the
> > symbols as well.
> >
> > A side note that this is a linux 2.4 kernel.
> > Is linux 2.4 not supported or what am I doing wrong?
>
> Linux 2.4 is supported, but either the vmlinux file was not originally
> built with the -g CFLAG, or it had the debuginfo data stripped.
> Without the debuginfo data, the vmlinux is useless.
>
> If by chance it is a RHEL3 2-4-based kernel, there is an associated
> vmlinux.debug file that comes with the separate kernel-debuginfo rpm
> package, so for example, you'd end up with two files:
>
> vmlinux-2.4.21-47.EL and vmlinux-2.4.21-47.EL.debug
>
> that would be installed here:
>
> /boot/vmlinux-2.4.21-47.EL
> /usr/lib/debug/boot/vmlinux-2.4.21-47.EL.debug
>
> If the files are found in locations above, running on a live system
> they would be found automatically when you enter:
>
> $ crash
>
> Alternatively you could just throw both files on the command line:
>
> $ crash <path-to>/vmlinux-2.4.21-47.EL
> <path-to>/vmlinux-2.4.21-47.EL.debug
>
> That all being said, since you are trying "crash vmlinux" I'm guessing
> that you are using a vmlinux file from another distro, or perhaps
> building your own kernel.
>
> If building your own kernel, the CFLAGS in the top-level Makefile
> needs to contain -g. In linux 2.4, I don't believe that there was a
> CONFIG_DEBUG_INFO option that could be configured to do it
> automatically, so you may have to change it manually, and then rebuild
> the kernel. Presuming that the original kernel that you're trying to
> analyze was booted from the same kernel version but was not built with
> -g, then you would also have to put the System.map file from the
> original kernel on the command line as you have done.
>
> If the vmlinux file is from another distro, mention that, and perhaps
> someone from the list can help.
>
> Dave
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
--
Crash-utility mailing list
Crash-utility at redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
More information about the Crash-utility
mailing list