[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Cluster-devel] fenced logsys/cman/ccs setup

On Wed, Jul 02, 2008 at 06:31:10AM +0200, Fabio M. Di Nitto wrote:
> >@@ -9,18 +11,62 @@ static int open_ccs(void)
> >		sleep(1);
> >		if (++i > 9 && !(i % 10))
> >			log_error("connect to ccs error %d, "
> >-				  "check ccsd or cluster status", cd);
> >+				  "check cluster status", cd);
> >+
> >+		/* FIXME: do we want this infinite? */
> >+		if (i > 10)
> >+			break;
> I think we want this to be infinite (and consistent across the board) or 
> configurable the same way other daemons/tools do so that users can decide 
> how long to wait.

OK, I think the following logic supports that:

- During startup, a program first connects to cman and waits for it to be
  ready (this should be a consistent and finite retry period).

- Then the program connects to ccs.  I think we can assume that since cman
  is running, the ccs connection will work within a short time.  So,
  it's safe to just put an infinite loop around the ccs_connect.

> >@@ -122,7 +165,6 @@ int read_ccs(struct fd *fd)
> >
> >	log_debug("added %d nodes from ccs", count);
> > out:
> >-	ccs_disconnect(cd);
> >	return 0;
> >}
> I don't see any call to ccs_disconnect around. We still need to invoke it 
> on exit to close the connection to the objdb when we exit and release 
> aisexec resources.

OK, I need to rework the exit path for the daemon to close/cleanup things.
I've just been lazy in the past because an exit will almost always clean
everything up automatically anyway.

> You can either do:
> or just use SYSLOGFACILITY instead.
> SYSLOGFACILITY is defined by the build system as default build option and 
> can be set by packagers according to distro rules/best practice/etc.

OK, that works for fenced.  What about things that aren't a daemon,
though?  Perhaps they'd like a different default?

> I don't have a DEFAULT_PRIORITY in the build system (maybe i can add it) 
> but it should be LOG_LEVEL_INFO. At least this is the best practise we 
> used so far.

OK, just to be clear, this setting is used:
- during startup before reading the user preferences in cluster.conf
- during running if the user has set no preference in cluster.conf

and the defaults should assume that the majority of people will never add
logging preferences to cluster.conf.  (Both because we picked good
defaults, and because they're lazy to look up and write the xml.)

> >+#define DEFAULT_FILE		NULL
> #define DEFAULT_FILE LOGDIR "/fenced.log"
> LOGDIR is set by the build system (same reasons as SYSLOGFACILITY). We 
> want files by default consistently across the board.

I think that by default we should probably have all logging go to
/var/log/messages like it has in the past.  Or, if we're daring, maybe
have it all go to /var/log/cluster.log.  But, I really doubt that people
want all programs to log to different files.

> >+#define LEVEL_PATH 
> >"/cluster/logging/logger_subsys[ subsys=\"FENCED\"]/@syslog_level"
> >+#define DEBUG_PATH 
> >"/cluster/logging/logger_subsys[ subsys=\"FENCED\"]/@debug"
> I am kind of curious.. why do you add defines for some queries but not for 
> others?

I'm still in limbo about what coding style I like there; in this case it's
if the line goes over 80 chars.

> >+/* Read cluster.conf settings and convert them into logsys values.
> >+   If no cluster.conf setting exists, the default that was used in
> >+   logsys_init() is used.
> >+
> >+   mode from
> >+   "/cluster/logging/@to_stderr"
> >+   "/cluster/logging/@to_syslog"
> >+   "/cluster/logging/@to_file"
> >+
> >+   facility from
> >+   "/cluster/logging/@syslog_facility"
> >+
> >+   priority from
> >+   "/cluster/logging/logger_subsys[ subsys=\"prog_name\"]/@syslog_level"
> >+
> >+   file from
> >+   "/cluster/logging/@filename"
> >+
> >+   debug from
> >+   "/cluster/logging/logger_subsys[ subsys=\"prog_name\"]/@debug"
> >+*/
> This logic is almost ok.
> You need to check for:
> /cluster/logging/@debug - global debug on/off switch.
> the other daemons respects globla debug setting if no subsystem debug 
> setting is available.

OK, so subsystem setting has priority over global setting.

> You also want to allow debugging to be set via cmdline and envvar.
> This allows a great control of debugging.
> The use cases are:
> - set /cluster/logging/@debug to on:
>   all daemons across the whole cluster are in debug mode (and logging
>   debug info). You don't need to add X subsystem lines to achieve this.


> - /cluster/logging/logger_subsys[ subsys=\"prog_name\"]/@debug:
>   users either want to debug only this system or disable debugging only
>   for this subsystem across the whole cluster when master debug is on.


> - cmdline debug:
>   allows the user to enable debugging manually only on that specific
>   daemon on that specific node.
> - envvar debug on (see for example CMAN_DEBUG or CCSD_DEBUG or QDISK_...):
>   these are easy to set within init scripts and /etc/defaults/... without
>   changing the daemon invokation in the init script itself that is not
>   always trivial and it is not a recommended practise.
>   the use case is similar (but more flexible) to the cmdline debug.
>   It allows the users to enable debugging for one specific node.
> cmdline and envvar debugging overrides whatever is in the config.

This is taking it too far for me (overengineering).  You'll need a new
debugging system to debug your complicated debugging system.

There two debug schemes:

1. for users/customers:  uses logsys, configured in cluster.conf
2. for me:  does not use logsys, controlled via command line

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]