[dm-devel] Question about work queues

Fri Nov 16 19:11:05 UTC 2007

Hi all, 

I'm working on the dm-stripe.c driver, adding in the workqueue calls to setup eventing for use with dmeventd. I'm hitting a strange problem though, its dumping a call trace in the workqueue.c file at line 166 where it checks the work->entry list to see if its empty:

Nov 14 20:26:20 dmraid-devhost kernel: ------------[ cut here ]------------
Nov 14 20:26:20 dmraid-devhost kernel: kernel BUG at kernel/workqueue.c:166!
Nov 14 20:26:20 dmraid-devhost kernel: invalid opcode: 0000 [1] SMP 
Nov 14 20:26:20 dmraid-devhost kernel: CPU 1 
Nov 14 20:26:20 dmraid-devhost kernel: Modules linked in: autofs4 hidp rfcomm l2cap bluetooth sunrpc e1000 nf_conntrack_netbios_ns ipt_REJECT nf_conntrack_ipv4 xt_state nf_conntrack nfnetlink iptable_filter ip_tables xt_tcpudp ip6t_REJECT ip6table_filter ip6_tables x_tables cpufreq_ondemand acpi_cpufreq dm_mirror dm_multipath dm_mod sbs dock battery ac radeon drm ipv6 floppy snd_hda_intel snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd sr_mod e100 button cdrom parport_pc soundcore mii parport rtc_cmos ata_generic snd_page_alloc serio_raw i2c_i801 i2c_core sg ahci libata sd_mod scsi_mod ext3 jbd mbcache ehci_hcd ohci_hcd uhci_hcd
Nov 14 20:26:20 dmraid-devhost kernel: Pid: 3462, comm: dmraid Not tainted 2.6.23-rc9 #12
Nov 14 20:26:20 dmraid-devhost kernel: RIP: 0010:[<ffffffff81045064>]  [<ffffffff81045064>] queue_work+0x1b/0x40
Nov 14 20:26:20 dmraid-devhost kernel: RSP: 0000:ffff81001c5a3c28  EFLAGS: 00010283
Nov 14 20:26:20 dmraid-devhost kernel: RAX: ffff81003a0d63b0 RBX: ffff81003a0d63a8 RCX: 00000000ffffffff
Nov 14 20:26:20 dmraid-devhost kernel: RDX: 0000000000000000 RSI: ffff81003a0d63a8 RDI: ffff8100280ae2c0
Nov 14 20:26:20 dmraid-devhost kernel: RBP: 0000000000000000 R08: ffff81003d266000 R09: ffffffff812653a0
Nov 14 20:26:20 dmraid-devhost kernel: R10: 0000000000000000 R11: ffff81003d198970 R12: ffffc2000073a080
Nov 14 20:26:20 dmraid-devhost kernel: R13: 0000000000000002 R14: ffff810027a04820 R15: 0000000000000002
Nov 14 20:26:20 dmraid-devhost kernel: FS:  00002b7923cabb40(0000) GS:ffff810037ce1300(0000) knlGS:0000000000000000
Nov 14 20:26:20 dmraid-devhost kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Nov 14 20:26:20 dmraid-devhost kernel: CR2: 00007fffa78d13a0 CR3: 000000001c61a000 CR4: 00000000000006e0
Nov 14 20:26:20 dmraid-devhost kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Nov 14 20:26:20 dmraid-devhost kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Nov 14 20:26:20 dmraid-devhost kernel: Process dmraid (pid: 3462, threadinfo ffff81001c5a2000, task ffff81001c40e810)
Nov 14 20:26:20 dmraid-devhost kernel: Stack:  ffff81003a0d6380 ffffffff882d23d0 000000001d1c4500 0000000000000002
Nov 14 20:26:20 dmraid-devhost kernel:  0000000000000002 ffff810027a04c00 0000000000000000 ffffc20000735165
Nov 14 20:26:20 dmraid-devhost kernel:  0000000000000000 000000003a388a00 ffffc2000073a080 ffff810027a04c00
Nov 14 20:26:20 dmraid-devhost kernel: Call Trace:
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff882d23d0>] :dm_mod:stripe_ctr+0x2c1/0x2d2
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff882d1607>] :dm_mod:dm_table_add_target+0x152/0x29f
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff882d2f9e>] :dm_mod:table_load+0xfb/0x1c9
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff882d2ea3>] :dm_mod:table_load+0x0/0x1c9
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff882d3a5f>] :dm_mod:ctl_ioctl+0x223/0x26e
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff81097ef5>] do_ioctl+0x55/0x6b
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff8109814e>] vfs_ioctl+0x243/0x25c
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff810981c0>] sys_ioctl+0x59/0x79
Nov 14 20:26:20 dmraid-devhost kernel:  [<ffffffff8100bd7c>] tracesys+0xdc/0xe1
Nov 14 20:26:20 dmraid-devhost kernel: 
Nov 14 20:26:20 dmraid-devhost kernel: 
Nov 14 20:26:20 dmraid-devhost kernel: Code: 0f 0b eb fe 65 8b 34 25 24 00 00 00 e8 a6 f9 ff ff 48 89 de 
Nov 14 20:26:20 dmraid-devhost kernel: RIP  [<ffffffff81045064>] queue_work+0x1b/0x40
Nov 14 20:26:20 dmraid-devhost kernel:  RSP <ffff81001c5a3c28>

Have I forgot something in initializing my work_struct? Below is what I've added to the driver for using the workqueue model:

I've added the work_struct to the stripe_c structure and the calls to workqueue and trigger_event function like multipath
(I also added a dm_target struct pointer to record the target table for later use in the dm_table_event() call):

struct stripe_c {
      uint32_t stripes;

      /* The size of this target / num. stripes */
      sector_t stripe_width;

      /* stripe chunk size */
      uint32_t chunk_shift;
      sector_t chunk_mask;

      struct stripe stripe[0];

      /* Needed for handling events */
      struct dm_target *ti;

      /* New work_queue setup for triggering events*/
      struct work_struct trigger_event;

};

struct workqueue_struct *kstriped;
static void trigger_event(struct work_struct *work);

/*
 * An event is triggered whenever a drive drops out of a 
 * stripe volume.
 */
static void trigger_event(struct work_struct *work)
{
      struct stripe_c *sc = container_of(work, struct stripe_c, trigger_event);

      dm_table_event(sc->ti->table);
}

Added the calls to stripe_ctr() for Initialization and creation of the struct & queue:

      /* Create the workqueue */
      kstriped = create_singlethread_workqueue("kstriped");
      if (!kstriped) {
             DMERR("failed to create workqueue kstriped");
             return -ENOMEM;
      }
      /* Initialize the work_struct */
      INIT_WORK(&sc->trigger_event, trigger_event);
      /* Set pointer to dm target; used in trigger_event */
      sc->ti = ti;

In the stripe_dtr() function I put the flush and destroy workqueue:

      flush_workqueue(kstriped);
      destroy_workqueue(kstriped);
      kfree(sc);

And for my test of all this to make sure its setup correctly I added a test call of queue_work() in the stripe_ctr() function at the bottom before the "return 0;"

      queue_work(kstriped, &sc->trigger_event);

It's when it hits this call that it is generating the error above; saying the &work->entry is empty (as if it didn't get initialized)? I thought that the INIT_WORK() call sets this list pointer? Do I need a call to INIT_LIST_HEAD()? I've tried reading up some in my "Kernel Driver" O'reilly book and my "Understanding the Linux Kernel" Book and I can't find what is missing. 

Any help would be appreciated, thanks everyone.

Brian Wood
Software Engineer
Intel Corp., Manageability & Platform Software Division
brian.j.wood at intel.com