[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

[Pulp-list] configuration files, validation and standards



All,

So, I got derailed on something yesterday (related to this, but not tihs) and decided "why waste a perfectly hosed up day" and decided to finish up the day on something that has been bugging me about pulp v1. Jay, don't kill me .. I mostly worked on this last night ;)

One area in which Pulp is inconsistent is in configuration file handling (reading/parsing/accessing/converting). The server uses a module based on ConfigParser and the client (originally) took the route of classes derived from INIConfig. These subclasses represented the configuration file and defined a schema used for self validation. The validation provided in common/config.py.

As we move forward in v2, I would like to consolidate on a single approach. That we pass configuration as (dict) objects instead of INIConfig or ConfigParser objects. Further, that we consistently leverage the validation that we invested in long ago but only adopted (completely) in the client. This approach has several benefits.

1. Configuration file parsing/formatting is independent of how configuration is used by our code.

2. By representing/passing configuration as a dictionary (actually a dict of dict), cobbling up configuration for testing is easier.

3. Consistent usage of (existing) configuration validation provides the following benefits:
  - define/document configuration in one place.
  - validate and report errors in one place with consistent error
    messages instead of splattering validation everywhere the
    configuration(s) are used.
  - combined usage of validation and companion type conversion functions
    makes for safe (and easy) conversion.  Eg: values for bool
    properties are consistently verified and converted.

4. A dict is well documented and understood. It can be easily pickled and converted to json.

I GC content handlers (in the agent) follow this approach. When loaded, the handler configuration is passed as a dictionary and the loader using validation to ensure that /standard/ parts of the configuration (descriptor) are valid. I would suggest the same for server plugin configurations.

I suppose Im guilty of adding yet one more way of dealing with configuration files but unlike some of the one-off solutions that have been popping up (I think), this is an attempt to standardize on a single approach. Just something to consider as we move forward.

--------------------


To help facilitate (and demonstrate) this approach:

In common/gc_config.py, I added a Config class that can be used to easily read INI config files into a dict (graph). It supports construction with a variety of inputs that are merged together to provide: a composite representation; property/section overrides; easy defaults. I also converted the validation to work on dictionaries. Further, I added a method that renders an object graph representation of the dict (graph) for easy access using (.) dot notation for those of you that liked INIConfig for this reason.

A Few Examples:

This just show validation and simple dict access to the configuration.

<snip>
[server]
host=myhost
port=80
</snip>

>>> config = Config(path)
>>> config.validate(schema)
>>> print config['server']['port']
80

>>> config = Config(path)
>>> obj = config.graph()
>>> print obj.server.port
80

# graph

>>> print obj.server.port
80

# not defined

>>> print obj.server.notdefined
None

>>> obj = config.graph(strict=True)
>>> print obj.server.notdefined
KeyError('notdefinded')

----

This example shows configuration defaults (dict) that is overlaid with file at path (A) and then file at path (B). The precedence defined by ordering.

defaults = { 'port' : 443 }

pathA = /etc/pulp/admin/admin.conf
<snip>
[server]
host=myhost
</snip>

pathB = ~/.pulp/admin.conf
<snip>
[server]
host=redhat.com
</snip>

>>> config = Config(defaults, pathA, pathB)
>>> config.validate(schema)
server = config['server']
>>> print server['host']
redhat.com

>>> print server['port']
443

----

Section filtering provided but not demonstrated.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]