[Crash-utility] Re: Fix for source line numbers for x86_64 modules

[Dave Anderson]

----- "Bob Montgomery" <bob.montgomery at hp.com> wrote:

> This patch allows the dis -l command to show real source line numbers
> for module code, instead of this sort of thing:
> 
> crash-4.0.9> dis -l bnx2_poll_msix
> include/linux/cpumask.h: 612
> 0xffffffffa008acc3 <bnx2_poll_msix>:    push   %rbp
> 0xffffffffa008acc4 <bnx2_poll_msix+1>:  mov    %rsp,%rbp
> 0xffffffffa008acc7 <bnx2_poll_msix+4>:  push   %r15
> 0xffffffffa008acc9 <bnx2_poll_msix+6>:  push   %r14
> 0xffffffffa008accb <bnx2_poll_msix+8>:  push   %r13
> 0xffffffffa008accd <bnx2_poll_msix+10>: mov    %esi,%r13d
> 0xffffffffa008acd0 <bnx2_poll_msix+13>: push   %r12
> 0xffffffffa008acd2 <bnx2_poll_msix+15>: xor    %r12d,%r12d
> 0xffffffffa008acd5 <bnx2_poll_msix+18>: push   %rbx
> 0xffffffffa008acd6 <bnx2_poll_msix+19>: mov    %rdi,%rbx
> 0xffffffffa008acd9 <bnx2_poll_msix+22>: sub    $0x8,%rsp
> 
> The problem has to do with a symbol table in the kernel containing an
> abnormally huge text range because of a goofy vsyscall reference in the
> file  "arch/x86/kernel/hpet.c".
> 
> (gdb) p *(struct block *)0x8d664b0
> $41 = {startaddr = 0xffffffff80225130, endaddr = 0xffffffffff6001b3,
> 
> This range pretty much covers the entire module address space and since
> the kernel file is first in the object_files list, the search stops
> there and returns the closest symbol from the kernel, instead of looking
> on through the object list to the module of actual interest.
> 
> This patch causes the code to keep looking through all modules and
> return the real best symbol thingy (you can tell how much I really know
> about gdb's symtab stuff, ahem).  Thanks (and blame) to John Wright for
> suggesting the fix once I found the problem.
> 
> The patch applies to the gdb patch file in the crash directory.
> 
> John and I think that this code in gdb searches things too many times,
> particularly with this patch, but it's a start since it seems to fix the
> problem.  Mayhaps some gdb experts can now help do it right.
> 
> With the patch, the above example becomes:
> 
> crash-4.0.9> dis -l bnx2_poll_msix
> /build/buildd/linux-2.6-clim-2.6.29-clim/debian/build/build_amd64_none_amd64/drivers/net/bnx2.c: 3215
> 0xffffffffa008acc3 <bnx2_poll_msix>:    push   %rbp
> 0xffffffffa008acc4 <bnx2_poll_msix+1>:  mov    %rsp,%rbp
> 0xffffffffa008acc7 <bnx2_poll_msix+4>:  push   %r15
> 0xffffffffa008acc9 <bnx2_poll_msix+6>:  push   %r14
> 0xffffffffa008accb <bnx2_poll_msix+8>:  push   %r13
> 0xffffffffa008accd <bnx2_poll_msix+10>: mov    %esi,%r13d
> 0xffffffffa008acd0 <bnx2_poll_msix+13>: push   %r12
> /build/buildd/linux-2.6-clim-2.6.29-clim/debian/build/build_amd64_none_amd64/drivers/net/bnx2.c: 3219
> 0xffffffffa008acd2 <bnx2_poll_msix+15>: xor    %r12d,%r12d
> /build/buildd/linux-2.6-clim-2.6.29-clim/debian/build/build_amd64_none_amd64/drivers/net/bnx2.c: 3215
> 0xffffffffa008acd5 <bnx2_poll_msix+18>: push   %rbx
> 0xffffffffa008acd6 <bnx2_poll_msix+19>: mov    %rdi,%rbx
> 0xffffffffa008acd9 <bnx2_poll_msix+22>: sub    $0x8,%rsp

Outstanding! 

BTW, I don't know if there *are* any gdb experts on this list...
If there are, I hope they can speak up!

I haven't tested this patch or taken the time to understand it, 
but I'm taking it on good faith that it solves this bug, which
has been an elephant in the room since (I believe) the 2.6.21
timeframe.  

In fact, Mike Snitzer had narrowed it down to the kernel patch that 
seemingly caused it, and you've confirmed it:

  https://www.redhat.com/archives/crash-utility/2009-January/msg00063.html

which was this one:

  commit 7460ed2844ffad7141e30271c0c3da8336e66014
  Author: john stultz <johnstul at us.ibm.com>
  Date:   Fri Feb 16 01:28:21 2007 -0800

    [PATCH] time: x86_64: re-enable vsyscall support for x86_64

    Cleanup and re-enable vsyscall gettimeofday using the generic clocksource
    infrastructure.

      [akpm osdl org: cleanup]
      Signed-off-by: John Stultz <johnstul us ibm com>
      Cc: Ingo Molnar <mingo elte hu>
      Cc: Thomas Gleixner <tglx linutronix de>
      Cc: Andi Kleen <ak muc de>
      Cc: Roman Zippel <zippel linux-m68k org>
      Signed-off-by: Andrew Morton <akpm linux-foundation org>
      Signed-off-by: Linus Torvalds <torvalds linux-foundation org>

   arch/x86_64/Kconfig              |    4 +
   arch/x86_64/kernel/hpet.c        |    6 ++
   arch/x86_64/kernel/time.c        |    6 --
   arch/x86_64/kernel/tsc.c         |    7 ++
   arch/x86_64/kernel/vmlinux.lds.S |   28 ++++------
   arch/x86_64/kernel/vsyscall.c    |  119 ++++++++++++++++++++++---------------
   include/asm-x86_64/proto.h       |    2 -
   include/asm-x86_64/timex.h       |    1 -
   include/asm-x86_64/vsyscall.h    |   29 ++--------
   9 files changed, 104 insertions(+), 98 deletions(-)

Anyway, this is awesome -- I (and countless others) really appreciate it...

Thanks to you and John,
  Dave