[Freeipa-devel] [Freeipa-users] Auto membership plugin

Nathan Kinder nkinder at redhat.com
Mon Apr 11 22:11:00 UTC 2011 

On 04/11/2011 02:53 PM, Dmitri Pal wrote:
> On 04/11/2011 05:21 PM, Nathan Kinder wrote:
>> On 04/11/2011 11:59 AM, Dmitri Pal wrote:
>>> On 04/11/2011 02:32 PM, Nathan Kinder wrote:
>>>> On 04/11/2011 10:58 AM, Nathan Kinder wrote:
>>>>> On 04/11/2011 10:25 AM, Dmitri Pal wrote:
>>>>>> On 04/11/2011 11:27 AM, Nathan Kinder wrote:
>>>>>>> On 04/08/2011 09:07 AM, Dmitri Pal wrote:
>>>>>>>> On 04/08/2011 11:49 AM, JR Aquino wrote:
>>>>>>>>> Is there any way to capture a description associated with the
>>>>>>>>> regex
>>>>>>>>> ->     group mapping?
>>>>>>>>>
>>>>>>>>> I was thinking that after time, it would be important to look back
>>>>>>>>> on rules and know why they were put there.
>>>>>>>>>
>>>>>>>>> Particularly in the case of regex, since it may not be completely
>>>>>>>>> obvious by looking back at alphabet soup.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Freeipa-users mailing list
>>>>>>>>> Freeipa-users at redhat.com
>>>>>>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>> The more I think about current design the more I want to normalize
>>>>>>>> things.
>>>>>>>> I would rather instead of:
>>>>>>>>
>>>>>>>> dn: cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>>>>>> objectclass: autoMemberDefinition
>>>>>>>> autoMemberScope: dc=example,dc=com
>>>>>>>> autoMemberFilter: objectclass=ipaHost
>>>>>>>> autoMemberExclusiveRegex:
>>>>>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^www5\.example\.com
>>>>>>>>
>>>>>>>>
>>>>>>>> autoMemberInclusiveRegex:
>>>>>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^www[1-9]+\.example\.com
>>>>>>>>
>>>>>>>>
>>>>>>>> autoMemberInclusiveRegex:
>>>>>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^web[1-9]+\.example\.com
>>>>>>>>
>>>>>>>>
>>>>>>>> autoMemberInclusiveRegex:
>>>>>>>> cn=mailservers,cn=hostgroups,dc=example,dc=com:fqdn=^mail[1-9]+\.example\.com
>>>>>>>>
>>>>>>>>
>>>>>>>> autoMemberDefaultGroup: cn=orphans,cn=hostgroups,dc=example,dc=com
>>>>>>>> autoMemberGroupingAttr: member:dn
>>>>>>>>
>>>>>>>>
>>>>>>>> Have something like:
>>>>>>>>
>>>>>>>> dn: cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>>>>>> objectclass: autoMemberDefinition
>>>>>>>> objectclass: cnContainer
>>>>>>>> autoMemberScope: dc=example,dc=com
>>>>>>>> autoMemberFilter: objectclass=ipaHost
>>>>>>>> autoMemberRegexRule: cn=Webserver Inclusion
>>>>>>>> Rule,cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>>>>>> autoMemberRegexRule: cn=Mailserver Inclusion
>>>>>>>> Rule,cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>>>>>> autoMemberRegexRule: cn=Desktop exclusion
>>>>>>>> Rule,cn=Hostgroups,cn=Auto
>>>>>>>> Membership Plugin,cn=plugins,cn=config
>>>>>>>> autoMemberDefaultGroup: cn=orphans,cn=hostgroups,dc=example,dc=com
>>>>>>>> autoMemberGroupingAttr: member:dn
>>>>>>>>
>>>>>>>>
>>>>>>>> dn: cn=Webserver Inclusion Rule,cn=Hostgroups,cn=Auto Membership
>>>>>>>> Plugin,cn=plugins,cn=config
>>>>>>>> objectclass: autoMemberDefinitionRegexRule
>>>>>>>> cn: Webserver Inclusion Rule
>>>>>>>> description: Rule contains regular expression to include webserver
>>>>>>>> hosts into the webserver group.
>>>>>>>> include: yes<- include or exclude
>>>>>>>> memberGroup: cn=webservers,cn=hostgroups,dc=example,dc=com
>>>>>>>> arrtibuteToMath: fgdn
>>>>>>>> expressionToMatch: ^www[1-9]+\.example\.com
>>>>>>>>
>>>>>>>>
>>>>>>>> Or something along those lines...
>>>>>>> It's a nice logical layout, but it would be hard for an
>>>>>>> administrator
>>>>>>> to figure out what exactly would happen if they were to add a host
>>>>>>> with a specific hostname.  Since the config is spread over so many
>>>>>>> entries, one would have to look at the top level config entry to
>>>>>>> find
>>>>>>> each rule DN, fetch each rule DN to look at the regexes.  All of the
>>>>>>> information is so spread out that you can't just look in one
>>>>>>> place to
>>>>>>> see the rules that will be used.  This could make things difficult
>>>>>>> from a troubleshooting perspective.
>>>>>> This should not be viewed in raw. THe UI and CLi should come to the
>>>>>> rescue.
>>>>>> I am not sure that this is a right approach to mix readability and
>>>>>> normalization.
>>>>>> To follow this logic no-one would ever normalize data in any DB
>>>>>> due to
>>>>>> the claim that it would be hard to join tables.
>>>>> The feature is implemented outside of the UI, and I would like to be
>>>>> able to troubleshoot it without the UI without a ton of hassle.
>>>>> Ultimately, it is a DS plug-in, and we should be able to troubleshoot
>>>>> it easily as a single component instead of requiring the UI to
>>>>> understand what it is doing.  We have run into this same issue with
>>>>> things like access control since ACI attributes can live anywhere in
>>>>> the database.  It is for this reason that we have things like the get
>>>>> effective rights control for evaluating settings.
>>>>>
>>>>> If one adds an inclusion rule, but doesn't realize that an exclusion
>>>>> rule is overriding it, they could easily get confused.  I understand
>>>>> that we want the UI to be able to notice this and present it to the
>>>>> user, but there is plenty of room for error in the UI as we add more
>>>>> and more logic into it.
>>>>>
>>>>> I am not totally opposed to this approach, but I want to point out
>>>>> some of the downsides in going in this direction.  I usually like to
>>>>> make the internal representation of plug-in configuration closely
>>>>> mirror the layout of the configuration entries/attributes
>>>>> themselves.  In this case, it would be highly inefficient to do so.
>>>>> Internally to the plug-in, we will be representing the config for a
>>>>> specific object type as a single struct with a list of all inclusive
>>>>> and exclusive rules.  I can convert multiple config entries into this
>>>>> same internal format, but it is something I generally like to avoid.
>>>>>
>>>>> I will look into this approach further.  I have the current approach
>>>>> from the design document implemented and working at this point, so
>>>>> some surgery will be required to go back to a different configuration
>>>>> layout.  Parsing and loading the configuration when it is split
>>>>> across so many dependent entries is not going to be trivial.  I can
>>>>> see plenty of corner cases in configuration validation cropping up.
>>>>> Dynamic config changes will also be more difficult to handle, as we
>>>>> will have to back track to find what config entries reference a
>>>>> modified rule.
>>>>>>> The description issue is a tough one to deal with if we have the
>>>>>>> config in the form that is currently described in the design doc.
>>>>>>> Since we want a description per regex rule, we should need to
>>>>>>> make the
>>>>>>> description be a part of the regex rule value instead of a separate
>>>>>>> description attribute.  I don't necessarily like this approach,
>>>>>>> as the
>>>>>>> readability of the config will not be nice.
>>>>>>>
>>>>>> I think this tips the scale towards the approach I proposed.
>>>>> Yes, it does.
>>>> I started writing up some notes to map out how we would represent the
>>>> configuration with this split out approach.  We would have one top
>>>> level config entry per filter/scope.  Child entries would be
>>>> inclusive/exclusive rule entries.  We sould load all child entries as
>>>> regex rules, so we don't need pointers to each regex entry in the
>>>> top-level entry.  Here is an example:
>>>>
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> dn: cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberDefinition
>>>> autoMemberScope: dc=example,dc=com
>>>> autoMemberFilter: objectclass=ipaHost
>>>> autoMemberGroupingAttr: member:dn
>>>> autoMemberDefaultGroup: cn=orphans,cn=hostgroups,dc=example,dc=com
>>>>
>>>> dn: cn=www 5,cn=Hostgroups,cn=Auto Membership
>>>> Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberRegexRule
>>>> description: exclude www5 from being added to webservers
>>>> autoMemberRegexType: exclusive
>>>> autoMemberRegexAttr: fqdn
>>>> autoMemberTargetGroup: cn=webservers,cn=hostgroups,dc=example,dc=com
>>>> autoMemberRegex: ^www5\.example\.com
>>>>
>>>> dn: cn=www hosts,cn=Hostgroups,cn=Auto Membership
>>>> Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberRegexRule
>>>> description: include www hosts in webservers
>>>> autoMemberRegexType: inclusive
>>>> autoMemberRegexAttr: fqdn
>>>> autoMemberTargetGroup: cn=webservers,cn=hostgroups,dc=example,dc=com
>>>> autoMemberRegex: ^www[1-9]+\.example\.com
>>>>
>>>> dn: cn=web hosts,cn=Hostgroups,cn=Auto Membership
>>>> Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberRegexRule
>>>> description: include web hosts in webservers
>>>> autoMemberRegexType: inclusive
>>>> autoMemberRegexAttr: fqdn
>>>> autoMemberTargetGroup: cn=webservers,cn=hostgroups,dc=example,dc=com
>>>> autoMemberRegex: ^web[1-9]+\.example\.com
>>>>
>>>> dn: cn=mail hosts,cn=Hostgroups,cn=Auto Membership
>>>> Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberRegexRule
>>>> description: include mail hosts in mailservers
>>>> autoMemberRegexType: inclusive
>>>> autoMemberRegexAttr: fqdn
>>>> autoMemberTargetGroup: cn=mailservers,cn=hostgroups,dc=example,dc=com
>>>> autoMemberRegex: ^mail[1-9]+\.example\.com
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> What I don't like about this approach is that the first 3 regex rules
>>>> really belong together, as they all have to do with webservers.  It
>>>> makes sense to split out the mail hosts regex since it is logically
>>>> different.  If you have the case where your hosts have hostnames that
>>>> don't follow any sort of format, you may end up having many inclusive
>>>> rule entries for all of your different webservers.  I see that as
>>>> being a headache in many cases.  Keep in mind that the above is a very
>>>> simple example.  You would likely have many more entries in an actual
>>>> deployment.
>>>>
>>>> To alleviate this problem, we could make the autoMemberRegex attribute
>>>> multi-valued, but you would only have one description per regex rule
>>>> entry, not per regex.  This is more flexible, as administrators could
>>>> choose when to group regexes together vs. having them separate.  The
>>>> only real advantage is to add a description to a group of regex rules
>>>> though.
>>>>
>>>> This is all equivalent to the following config with the method
>>>> described in the design doc (without the ability to have descriptions):
>>>>
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> dn: cn=Hostgroups,cn=Auto Membership Plugin,cn=plugins,cn=config
>>>> objectclass: autoMemberDefinition
>>>> autoMemberScope: dc=example,dc=com
>>>> autoMemberFilter: objectclass=ipaHost
>>>> autoMemberExclusiveRegex:
>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^www5\.example\.com
>>>> autoMemberInclusiveRegex:
>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^www[1-9]+\.example\.com
>>>>
>>>> autoMemberInclusiveRegex:
>>>> cn=webservers,cn=hostgroups,dc=example,dc=com:fqdn=^web[1-9]+\.example\.com
>>>>
>>>> autoMemberInclusiveRegex:
>>>> cn=mailservers,cn=hostgroups,dc=example,dc=com:fqdn=^mail[1-9]+\.example\.com
>>>>
>>>> autoMemberDefaultGroup: cn=orphans,cn=hostgroups,dc=example,dc=com
>>>> autoMemberGroupingAttr: member:dn
>>>> -------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>> Or you can go the other way around and eliminate the top level entry:
>>>
>>> dn: cn=www 5,cn=Hostgroups,cn=Auto Membership
>>> Plugin,cn=plugins,cn=config
>>> objectclass: autoMemberRegexRule
>>> description: exclude www5 from being added to webservers
>>> autoMemberScope: dc=example,dc=com
>>> autoMemberFilter: objectclass=ipaHost
>>> autoMemberGroupingAttr: member:dn
>>> autoMemberRegexType: exclusive
>>> autoMemberRegexAttr: fqdn
>>> autoMemberTargetGroup: cn=webservers,cn=hostgroups,dc=example,dc=com
>>> autoMemberRegex: ^www5\.example\.com
>>>
>>>
>>> If you need to express the default entry you can say
>>>
>>> dn: cn=default,cn=Hostgroups,cn=Auto Membership
>>> Plugin,cn=plugins,cn=config
>>> objectclass: autoMemberRegexRule
>>> description: default catch all rule for Hosts
>>> autoMemberScope: dc=example,dc=com
>>> autoMemberFilter: objectclass=ipaHost
>>> autoMemberGroupingAttr: member:dn
>>> autoMemberRegexType: default
>>> autoMemberTargetGroup: cn=orpahns,cn=hostgroups,dc=example,dc=com
>>>
>>> Then you read all the entries, sort them buy "exclusive, inclusive,
>>> default" and handle accordingly.
>>> That will create a little bit more duplication of data but will be
>>> readable and manageable in a raw form.
>> Why not just leave the config the way I have it now then?  There's
>> nothing preventing you from creating multiple config entries for the
>> same scope/filter/grouping attr.
>>
>> Right now, I use a single struct for each config entry, even if you
>> defined multiple entries with the same scope/filter/grouping attr.
>> Each config entry is processed independently of other config entries.
>> You can't have an exclusion in one config entry that overrides an
>> inclusion from another config entry.  If we want multiple config
>> entries to interact in this way, I would need to group and order
>> inclusion rules from multiple config entries that have the same exact
>> scope/filter/grouping attr values into a single struct when parsing
>> and loading the config.  This can be done, but we need to handle
>> things such as duplicate regex rules.
>>
>> The reason I like the single struct approach is that I only need to do
>> a filter check against an entry being added for each config entry.  If
>> we don't handle it this way internally, we would end up checking the
>> filter and scope from each and every config entry against every entry
>> added to DS.  This would mean 100 filter checks for 100 regex rules.
>> This obviously can't be done this way for performance reasons.  We can
>> make the internal representation differ from the presentation as
>> config entries, but I find that this introduces room for confusion in
>> how the plug-in will work for a given configuration.
>>
>> I still feel that this is adding a lot of complexity (and room for
>> bugs) for very little value.  The main downside with the current
>> approach is that we don't have a way to have a description per regex,
>> right?  Is there some other downside here?  Maybe we can solve the
>> description issue without the additional complexity if that is the
>> only requirement that is lacking in the current approach.
> So how you are going to deal with two colliding config entries now or
> you think there will be single config entry for the whole plugin?
> If not than there can be colliding config entries that you would have to
> process anyways.
Each config entry is independent.  Collisions are possible, but they 
should be minimized since the approach minimizes the need to create 
multiple config entries for the same type of object.  A collision is not 
horrible to deal with, as the plug-in will gracefully handle the case 
where it adds the same entry as a member when it is already a member.
> But I am not insisting. I understand your arguments, I am not sure I
> agree with all of them but you have much more experience in the area
> than I do.
> Let us not fix something that is not broken. We can add description to
> the Regex attribute. It won't be the first time. ACI is like that.
I'm not insisting on my way either.  I think we can still have the UI 
represent the data the way you are thinking without any additional work 
over your suggested approach.  Since I have the implementation nearly 
finished, I'll get a test copy out for developers to play with later 
this week.  If we find that we want to change it up, we can do that.
>
>
>>>
>>>> -NGK
>>>>>>> _______________________________________________
>>>>>>> Freeipa-users mailing list
>>>>>>> Freeipa-users at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/freeipa-users
>>>>>>>
>>>>>>>
>>>>> _______________________________________________
>>>>> Freeipa-devel mailing list
>>>>> Freeipa-devel at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel
>>>> _______________________________________________
>>>> Freeipa-devel mailing list
>>>> Freeipa-devel at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/freeipa-devel
>>>>
>>>>
>> _______________________________________________
>> Freeipa-devel mailing list
>> Freeipa-devel at redhat.com
>> https://www.redhat.com/mailman/listinfo/freeipa-devel
>>
>>
>

More information about the Freeipa-devel mailing list