[Crash-utility] RFC: Improving crash's search speed

Dave Anderson anderson at redhat.com
Wed Jan 26 14:47:35 UTC 2011



----- Original Message -----
> On Tue, 2011-01-25 at 21:50 +0000, Dave Anderson wrote:
> 
> > >
> > > The attached patch implements the -K option that is the same as doing
> > > "-k -e <vmalloc_start_addr>".
> > >
> > > Comments?
> > > Bob Montgomery
> >
> > I like the idea...
> >
> > But it won't work on ia64 machines, where the vmalloc space is contained
> > within the region starting at a000000000000000, which is below the unity-mapped
> > region starting at e000000000000000.
> ...
> >
> > So it looks like it might need a hack for ia64 anyway, which
> > already has a few. I don't think any of the other arches have
> > their vmalloc regions below their unity-mapped regions.
> >
> > Dave
> 
> So -K should really be defined as "search the identity-mapped region",
> maybe? Perhaps a mach-specific utility routine to return start and end
> for that region which become the defaults for -K search??

Yeah, I think so.  In fact, in looking at the ia64 anomoly, I note that
the similarly-mapped kernel text/data region in x86_64 is being skipped
entirely:

  crash>  mach
         MACHINE TYPE: x86_64
          MEMORY SIZE: 1 GB
                 CPUS: 8
      PROCESSOR SPEED: 1995 Mhz
                   HZ: 1000
            PAGE SIZE: 4096
  KERNEL VIRTUAL BASE: ffff810000000000
  KERNEL VMALLOC BASE: ffffc20000000000
     KERNEL START MAP: ffffffff80000000
  KERNEL MODULES BASE: ffffffff88000000 
    KERNEL STACK SIZE: 8192
  crash>

The x86_64 search starts at the unity-mapped region at ffff810000000000,
then jumps to the vmalloc base region at ffffc20000000000, and then when
it reaches the end of that section, it calls next_vmalloc_vaddr(), which
skips to the modules part of of the vmalloc region at ffffffff88000000.
So it leaps over the kernel text/data mapped area at ffffffff80000000.  
The kernel text/data mapped region, similar to the vmalloc region, is 
"double-mapped", so you would see any search matches in that memory, but
they would be shown by their unity-mapped reference only.  But that's not
the intent, so it's a bug.
 
> My own tools are really "search physical pages", but I didn't see that
> mechanism in crash (though it might be there?).

No it's not, but it probably should be.  That would be easy to shoe-horn
in there since all of the search addresses (user or kernel) are first
turned into physical addresses before being read (except for the
Xen hypervisor).  So yeah, I think a "-p" argument would be pretty
useful.

> 
> It's been a while since ia64 was my main architecture :-)

Yeah, since RHEL6 dropped it, it's beginning to leak out of my brain cells,
but I remember running into the oddity of its mapped kernel and vmalloc
region location. 
 
Anyway, I'm thinking now that "search" should be reworked to add these
new variants:

  search -K  unity-mapped kernel only
  search -V  vmalloc memory only
  search -M  kernel text/data mapped region (arch-dependent)
  search -p  physical memory

and leave the current mechanism as it is now:

  search -u  user-space of current context 
  search -k  all of kernel virtual space 

where "search -k" would be equivalent to "search -KVM".

Let me tinker around with this -- thanks for the suggestion
(and the inadvertant x86_64 bug report).

Dave




More information about the Crash-utility mailing list