[augeas-devel] load specific lens

[David Lutterkort]

On Sun, 2010-03-14 at 16:35 +0100, Frederik Wagner wrote:
> On Fri, Mar 12, 2010 at 2:15 AM, David Lutterkort <lutter at redhat.com> wrote:
> > On Wed, 2010-03-10 at 16:25 +0100, Frederik Wagner wrote:
> >> On Wed, Mar 10, 2010 at 9:46 AM, Frederik Wagner <fnerdwq at googlemail.com> wrote:
> >>
> > Where are you seeing the slowdown ? With augtool or augparse ?
> 
> in both. It is due to the this line:
> 
>  let key_re = /[A-Za-z0-9_]+(\[[0-9]+\])?/ - "unset" - "export" - key_wa
> 
> each additional word which is 'substracted' (so each additional word
> in "key_wa"), it becomes slower, using up more and more memory (an
> makes my poor old laptop swap like mad)
> 
> I do not find a way around this construct...

Yes, regexp subtraction is pretty expensive, since it leads to pretty
big regular expressions; you can make it a little lighter by writing
        
        let key_re = /[A-Za-z0-9_]+(\[[0-9]+\])?/ - /unset|export|
        key_wa/
        
Not sure how much of a performance gain it is, though.

> > Generally, these kinds of lenses cause the typechecker to work very hard
> > - the typechecker is only run by augparse. You can bypass it with
> > 'augparse --notypecheck'; though before declaring any changes 'done',
> > you need to typecheck the lens at least once.
> >
> >> To filter or the keywords I'm using - just showing the relevant adjustments:
> >>
> >> ...
> >>   (* keywords from files which should be treaded as array of words *)
> >>   let key_wa =
> >>                (* from /etc/sysconfig/kernel *)
> >>                "MODULES_LOADED_ON_BOOT"
> >>              | "DOMU_INITRD_MODULES"
> >>              | "INITRD_MODULES"
> >>                (* from /etc/sysconfig/bootloader *)
> >>              | "DEFAULT_APPEND"
> >>              | "FAILSAFE_APPEND"
> >>              | "XEN_KERNEL_APPEND"
> >>              | "XEN_APPEND"
> >>              | "RT_APPEND"
> >>
> >>   let key_re = /[A-Za-z0-9_]+(\[[0-9]+\])?/ - "unset" - "export" - key_wa
> >> ...
> >>
> >> and later in the lens I'm applying extras steps to the filtered keywords:
> >>
> >> ...
> >>   let char_wa = /[^ "\t\n]/
> >>
> >>   (* for the keywords in key_wa treat the values as an array of words *)
> >>   let wa_array = f
> >>     let wa_array_value = store char_wa+ in
> >>     del /\"[ \t]*/ "\"" . counter "wa_val" .
> >>       [ seq "wa_val" .wa_array_value ] .
> >>       [ del /[ \t\n]+/ " "  . seq "wa_val" . wa_array_value ] *
> >>       . del /[ \t]*\"/ "\""
> >>
> >>   (* lens for the array of words, extra handling of empty strings "" *)
> >>   let wa_kv = [ key key_wa . eq . (del "\"\"" "\"\""|wa_array) . eol ]
> >>
> >>   let lns = ( comment | empty | source | kv | unset | wa_kv ) *
> >> ...
> >>
> >> Since I change the fast working original shellvars lens, I suppose the
> >> filtering construct is the problem.
> >>
> >> Am I doing something wrong? Or is the adjustment of the shellvars lens
> >> generally not usefull anyway? (if it is I would submit the patch, when
> >> I'm ready).
> >
> > Another way to structure this is to make the current lens 'lns' a
> > function that takes the special variables as an argument, so that you'd
> > have
> >
> >  let wa (keys:regexp) =
> >    ( comment | empty | source | kv | unset | wa_kv keys) *
> >
> > and change wa_kv to pass keys respectively (key_re - key_wa) to the
> > right parts of the lens.
> >
> 
> I don't completely understand what you mean by 'the right parts of the
> lens', but this does not solve the problem above, does it?

What I meant was that you pass keys down to wa_kv, so that it only
matches the variables mentioned in keys and splits them, and pass
(key_re - keys) into the existing kv lens.

> > Then, write separate transforms and filters for each file, e.g.
> >
> >  let kernel_xfm =
> >    let wa_keys = /MODULES_LOADED_ON_BOOT|DOMU_INITRD_MODULES|INITRD_MODULES/ in
> >    transform (wa wa_keys) (incl "/etc/sysconfig/kernel")
> >
> > There's one minor hitch though: Augeas right now only allows one
> > autoload statement per module. That will force you to have separate
> 
> may I ask, what does the autoload stand for? (xfm?)

The way Augeas knows what files to process with what lens is through a
transform, which combines a filter and a lens - the transform will send
all files matching the filter through the lens and put the result into
the overall tree.

When you mark a transform for autoload, it tells Augeas that it should
process the transform when it starts up automatically. The autoload
statements are responsible for filling the Augeas tree on initial
startup.

David