[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[linux-lvm] lvm kernel module OOPS



I'm running RedHat 9 fully patched (2.4.20-31.9smp) with mostly default
settings.  In order to get consistent backups with dump(1), I take
advantage of LVM snapshots (lvm-1.0.3-12).  I have nightly backups to a
remote tapedrive, and disk-to-disk backups a few times during the day.
It works pretty well, usually.  But then there's the occasional kernel
crash.  :(

The first time this happened, the server (astro) was under fairly high
load, as a result of a memory-intensive program causing the machine to
swap.  While that was going on, my 18:00 backups ran, and this was the
result:

May 25 18:15:00 astro logger:  18:15:00  up 41 days,  1:57, 33 users,  load average: 5.13, 5.72, 4.01
May 25 18:22:50 astro kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000004
May 25 18:22:51 astro kernel:  printing eip:
May 25 18:22:51 astro kernel: f883ff8b
May 25 18:22:51 astro kernel: *pde = 00000000
May 25 18:22:51 astro kernel: Oops: 0002
May 25 18:22:51 astro kernel: vfat fat es1371 gameport ac97_codec soundcore ide-cd cdrom parport_pc lp parport nfsd lockd sunrpc e100 ipt_REJECT ipt_LOG ipt_limit ipt_mac ipt_state ip_conn
May 25 18:22:51 astro kernel: CPU:    1
May 25 18:22:51 astro kernel: EIP:    0060:[<f883ff8b>]    Not tainted
May 25 18:22:51 astro kernel: EFLAGS: 00010202
May 25 18:22:51 astro kernel:
May 25 18:22:51 astro kernel: EIP is at lvm_find_exception_table [lvm-mod] 0x7b (2.4.20-30.9smp)
May 25 18:22:51 astro kernel: eax: f8b19cd8   ebx: f8b2c4e8   ecx: f8b14770   edx: 00000000
May 25 18:22:51 astro kernel: esi: 05144f80   edi: 00000001   ebp: 00000802   esp: c48dfd68
May 25 18:22:51 astro kernel: ds: 0068   es: 0068   ss: 0068
May 25 18:22:51 astro kernel: Process dump (pid: 18145, stackpage=c48df000)
May 25 18:22:51 astro kernel: Stack: 000007ff f884bcbf f5678a00 00000000 00000000 c48dfdee f883efde 00000802
May 25 18:22:51 astro kernel:        05144f80 f5678a00 05144f80 00000000 0279cd00 05144280 f5678a00 00000001
May 25 18:22:51 astro kernel:        f883b66d c48dfdee c48dfde8 05144280 f5678a00 e6907f38 f884bcbf c915c700
May 25 18:22:51 astro kernel: Call Trace:   [<f884bcbf>] do_get_write_access [jbd] 0x31f (0xc48dfd6c))
May 25 18:22:51 astro kernel: [<f883efde>] lvm_snapshot_remap_block [lvm-mod] 0x7e (0xc48dfd80))
May 25 18:22:51 astro kernel: [<f883b66d>] lvm_map [lvm-mod] 0x20d (0xc48dfda8))
May 25 18:22:51 astro kernel: [<f884bcbf>] do_get_write_access [jbd] 0x31f (0xc48dfdc0))
May 25 18:22:51 astro kernel: [<f883ba67>] lvm_make_request_fn [lvm-mod] 0x17 (0xc48dfe00))
May 25 18:22:51 astro kernel: [<c01b612a>] generic_make_request [kernel] 0xda (0xc48dfe0c))
May 25 18:22:51 astro kernel: [<c01b61d7>] submit_bh [kernel] 0x57 (0xc48dfe34))
May 25 18:22:51 astro kernel: [<c0156927>] block_read_full_page [kernel] 0x257 (0xc48dfe50))
May 25 18:22:51 astro kernel: [<c013c108>] add_to_page_cache_unique [kernel] 0x68 (0xc48dfea0))
May 25 18:22:51 astro kernel: [<c013c24a>] page_cache_read [kernel] 0xda (0xc48dfeb4))
May 25 18:22:51 astro kernel: [<c015a950>] blkdev_get_block [kernel] 0x0 (0xc48dfebc))
May 25 18:22:51 astro kernel: [<c013cb2b>] generic_file_readahead [kernel] 0xdb (0xc48dfedc))
May 25 18:22:51 astro kernel: [<c013ce33>] do_generic_file_read [kernel] 0x1c3 (0xc48dff0c))
May 25 18:22:51 astro kernel: [<c013d450>] file_read_actor [kernel] 0x0 (0xc48dff38))
May 25 18:22:51 astro kernel: [<c013d600>] generic_file_read [kernel] 0xb0 (0xc48dff58))
May 25 18:22:51 astro kernel: [<c013d450>] file_read_actor [kernel] 0x0 (0xc48dff68))
May 25 18:22:51 astro kernel: [<c0152f27>] sys_read [kernel] 0x97 (0xc48dff94))
May 25 18:22:51 astro kernel: [<c01098cf>] system_call [kernel] 0x33 (0xc48dffc0))
May 25 18:22:51 astro kernel:
May 25 18:22:51 astro kernel:
May 25 18:22:51 astro kernel: Code: 89 42 04 8b 03 89 48 04 89 01 89 59 04 89 0b 89 c8 eb ce 0f
May 25 18:23:28 cayuga kernel: nfs: server astro not responding, still trying
May 25 18:23:34 cayuga modprobe: modprobe: Can't locate module char-major-10-134
May 25 18:24:47 cayuga modprobe: modprobe: Can't locate module char-major-10-134

I chalked it up a rare event due to the extreme load the machine was
experiencing (I think the load was above 10 at the time, and it was
using a full gig of swap), and went on with life.  But it just happened
again, when nobody was on the server, during my 12:00 backups:

Jun  9 12:02:36 astro kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000004
Jun  9 12:02:36 astro kernel:  printing eip:
Jun  9 12:02:36 astro kernel: f883ff8b
Jun  9 12:02:36 astro kernel: *pde = 00000000
Jun  9 12:02:36 astro kernel: Oops: 0002
Jun  9 12:02:36 astro kernel: es1371 gameport ac97_codec soundcore ide-cd cdrom parport_pc lp parport nfsd lockd sunrpc e100 ipt_REJECT ipt_LOG ipt_limit ipt_mac ipt_state ip_conntrack ipt
Jun  9 12:02:36 astro kernel: CPU:    3
Jun  9 12:02:36 astro kernel: EIP:    0060:[<f883ff8b>]    Not tainted
Jun  9 12:02:36 astro kernel: EFLAGS: 00010202
Jun  9 12:02:36 astro kernel:
Jun  9 12:02:36 astro kernel: EIP is at lvm_find_exception_table [lvm-mod] 0x7b (2.4.20-31.9smp)
Jun  9 12:02:36 astro kernel: eax: f8900510   ebx: f891c3a8   ecx: f8900090   edx: 00000000
Jun  9 12:02:36 astro kernel: esi: 03373b80   edi: 00000001   ebp: 00000802   esp: c0d2dd68
Jun  9 12:02:36 astro kernel: ds: 0068   es: 0068   ss: 0068
Jun  9 12:02:36 astro kernel: Process dump (pid: 13098, stackpage=c0d2d000)
Jun  9 12:02:36 astro kernel: Stack: 000007ff c01b618a f02db400 00000000 00000000 c0d2ddee f883efde 00000802
Jun  9 12:02:36 astro kernel:        03373b80 f02db400 03373b80 f883b9d0 009cb900 03372280 f02db400 00000001
Jun  9 12:02:36 astro kernel:        f883b66d c0d2ddee c0d2dde8 03372280 f02db400 c722de00 f884bcbf d41862c0
Jun  9 12:02:36 astro kernel: Call Trace:   [<c01b618a>] generic_make_request [kernel] 0xda (0xc0d2dd6c))
Jun  9 12:02:36 astro kernel: [<f883efde>] lvm_snapshot_remap_block [lvm-mod] 0x7e (0xc0d2dd80))
Jun  9 12:02:36 astro kernel: [<f883b9d0>] lvm_push_callback [lvm-mod] 0xa0 (0xc0d2dd94))
Jun  9 12:02:36 astro kernel: [<f883b66d>] lvm_map [lvm-mod] 0x20d (0xc0d2dda8))
Jun  9 12:02:36 astro kernel: [<f884bcbf>] do_get_write_access [jbd] 0x31f (0xc0d2ddc0))
Jun  9 12:02:36 astro kernel: [<f883ba67>] lvm_make_request_fn [lvm-mod] 0x17 (0xc0d2de00))
Jun  9 12:02:36 astro kernel: [<c01b618a>] generic_make_request [kernel] 0xda (0xc0d2de0c))
Jun  9 12:02:36 astro kernel: [<c01b6237>] submit_bh [kernel] 0x57 (0xc0d2de34))
Jun  9 12:02:36 astro kernel: [<c0156937>] block_read_full_page [kernel] 0x257 (0xc0d2de50))
Jun  9 12:02:36 astro kernel: [<c013c108>] add_to_page_cache_unique [kernel] 0x68 (0xc0d2dea0))
Jun  9 12:02:36 astro kernel: [<c013c24a>] page_cache_read [kernel] 0xda (0xc0d2deb4))
Jun  9 12:02:36 astro kernel: [<c015a960>] blkdev_get_block [kernel] 0x0 (0xc0d2debc))
Jun  9 12:02:36 astro kernel: [<c013cb2b>] generic_file_readahead [kernel] 0xdb (0xc0d2dedc))
Jun  9 12:02:36 astro kernel: [<c013cffa>] do_generic_file_read [kernel] 0x38a (0xc0d2df0c))
Jun  9 12:02:36 astro kernel: [<c013d450>] file_read_actor [kernel] 0x0 (0xc0d2df38))
Jun  9 12:02:36 astro kernel: [<c013d600>] generic_file_read [kernel] 0xb0 (0xc0d2df58))
Jun  9 12:02:36 astro kernel: [<c013d450>] file_read_actor [kernel] 0x0 (0xc0d2df68))
Jun  9 12:02:36 astro kernel: [<c0152f37>] sys_read [kernel] 0x97 (0xc0d2df94))
Jun  9 12:02:36 astro kernel: [<c01098cf>] system_call [kernel] 0x33 (0xc0d2dfc0))
Jun  9 12:02:36 astro kernel:
Jun  9 12:02:36 astro kernel:
Jun  9 12:02:36 astro kernel: Code: 89 42 04 8b 03 89 48 04 89 01 89 59 04 89 0b 89 c8 eb ce 0f

Debugging kernel crashes is slightly beyond my skill level, but I wanted
to report the problem.  I can provide additional information if needed.

Damian Menscher
-- 
-=#| Physics Grad Student & SysAdmin @ U Illinois Urbana-Champaign |#=-
-=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=-
-=#| 4602 Beckman, VMIL/MS, Imaging Technology Group:(217)244-3074 |#=-
-=#| <menscher uiuc edu> www.uiuc.edu/~menscher/ Fax:(217)333-9819 |#=-
-=#| The above opinions are not necessarily those of my employers. |#=-

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]