[Crash-utility] crash can't read vmlinux on live system?

[Dave Anderson]

Kris Corwin wrote:

> > First, is the vmlinuz that the kernel is running created from the ./vmlinux
> > file that you're using as an argument?
>
> yes.
>
> > I'm guessing that it continued on and complained
> > about not being able to read the the linux_banner string, and then died?
>
> crash exited right after what I posted.
>
> I can run crash on 2.6.9-11 kernels, but not 2.6.9-22.
>
> Here's the -d run.  It looks like I have a memory read issue.
>
> [root at f14 linux-2.6.9-22.0.2.EL]# crash -d7 ./vmlinux
>
> crash 4.0-2
> Copyright (C) 2002, 2003, 2004, 2005  Red Hat, Inc.
> Copyright (C) 2004, 2005  IBM Corporation
> Copyright (C) 1999-2005  Hewlett-Packard Co
> Copyright (C) 1999, 2002  Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions.  Enter "help copying" to see the conditions.
> This program has absolutely no warranty.  Enter "help warranty" for details.
>
> get_live_memory_source: /dev/mem
> /proc/version:
> Linux version 2.6.9-prep.qp2.2.5.11.3qsnet (root at f15) (gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-20)) #3 SMP Tue Feb 21 15:12:39 EST 2006
> ./vmlinux:
> Linux version 2.6.9-prep.qp2.2.5.11.3qsnet (root at f15) (gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-20)) #3 SMP Tue Feb 21 15:12:39 EST 2006
> <readmem: c0153373, KVADDR, "x86_omit_frame_pointer", 4, (ROE), bff71420>
> /dev/mem: Operation not permitted
> crash: read(/dev/mem, 153373, 0): 4 (ffffffff)
> crash: read error: kernel virtual address: c0153373  type: "x86_omit_frame_pointer"
> <readmem: c0439d10, KVADDR, "xtime", 8, (FOE), 8349634>
> /dev/mem: Operation not permitted
> crash: read(/dev/mem, 439d10, 0): 8 (ffffffff)
> crash: read error: kernel virtual address: c0439d10  type: "xtime"
> [root at f14 linux-2.6.9-22.0.2.EL]# whoami
> root
> [root at f14 linux-2.6.9-22.0.2.EL]# ls -l /dev/mem
> crw-r-----  1 root kmem 1, 1 Feb 22 06:31 /dev/mem

Ok, a couple of things.

Red Hat kernels have restricted /dev/mem drivers which only allow
reads of the first 1MB of physcal address space.  For that reason
RHEL (and Fedora) kernels have an additional /dev/crash driver
(in ./drivers/char/crash.c) that replaces /dev/mem.  It's a "misc"
driver, only used for x86 and x86_64 kernels, and is dynamically
modprobe'd by crash.

The debug output from your crash session attempt shows two
things of interest:

> get_live_memory_source: /dev/mem

so the /dev/crash driver is not available from that kernel, which
is the second point of interest:

> /proc/version:
> Linux version 2.6.9-prep.qp2.2.5.11.3qsnet (root at f15)...

So the question is whether the mem.c driver in that kernel is
based upon the Red Hat restricted mem.c, or is restricted
otherwise.  AFAIK, the upstream drivers/char/mem.c does
not return EPERM from its read_mem() function.

> /dev/mem: Operation not permitted

but the restricted Red Hat mem.c does.  In any case, it all
comes down to why /dev/mem is not allowing read attempts.

Dave