[katello-devel] Foreman integration and Elastic search

[Petr Chalupa]

On 26.11.12 17:42, Dmitri Dolguikh wrote:
> On 26/11/12 04:15 PM, Mike McCune wrote:
>> On 11/22/2012 01:08 AM, Tomas Strachota wrote:
>>> On 11/21/2012 04:22 PM, Dmitri Dolguikh wrote:
>>>> On 21/11/12 03:13 PM, Bryan Kearney wrote:
>>>>> On 11/20/2012 02:28 PM, Lukas Zapletal wrote:
>>>>>> On Tue, Nov 20, 2012 at 05:04:54PM +0200, Ohad Levy wrote:
>>>>>>> On 11/20/2012 04:05 PM, Dmitri Dolguikh wrote:
>>>>>>>> We are about to start using Elastic search for lookups on
>>>>>>>> Foreman data
>>>>>>>> on Katello side. Up until this moment Katello didn't persist any
>>>>>>>> Foreman-related data at all.
>>>>>>>>
>>>>>>>> This introduces what is essentially a long-term cache, but
>>>>>>>> without any
>>>>>>>> form of cache control or cache validation. Another issue is this
>>>>>>>> breaks
>>>>>>>> Foreman modularization, with Katello now being responsible for
>>>>>>>> some of
>>>>>>>> the Foreman data.
>>>>>>>>
>>>>>>>> I think this functionality should be moved to Foreman (or Foreman's
>>>>>>>> existing functionality improved) and exposed via introduction of
>>>>>>>> search
>>>>>>>> functionality in Foreman's API.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>> If we introduce ES updates as callback filters would that work?
>>>>>>>
>>>>>>> if that works, it sounds like a plan for foreman rails engine /
>>>>>>> plugin which adds more callbacks?
>>>>>>
>>>>>> I like this more than integrating ES with Foreman in Katello, but
>>>>>> please
>>>>>> let's do NOT use HTTP/REST for that and leverage messaging for this.
>>>>>> This is typical case for fire-and-forget pattern which is much more
>>>>>> reliable than playing around with http requests.
>>>>>>
>>>>> Dmitri, is the issue here that we have no object inthe katello db?
>>>>
>>>> One of the issues, yes. The larger problem is that we are losing a
>>>> clear
>>>> separation between Katello and Foreman (if only on UI side). It appears
>>>> that we might need tighter integration (Justin pointed out that
>>>> combining elastic search results in the model I proposed would be hard)
>>>> -- perhaps we should look into adopting Foreman-as-rails-engine
>>>> approach: integration would be simpler (faster too, I'd wager).
>>>>
>>>> -d
>>>
>>> Pardon my ignorance but I'm afraid I still didn't understand the issue.
>>> How is it different from what we do with records in Pulp? Some models
>>> are also stored completely on Pulp's side without having a bit in
>>> Katello db but still we index them with ES.
>>>
>>
>> I had the exact same question .. why is what we are doing with
>> Foreman's searchable data any different than what we have done in the
>> past with Pulp's?  We decided some time ago that ES is, in fact, a
>> long-term cache of data we need to make searchable and display very
>> quickly in the UI using elasticsearch.
> Our integration with foreman is such that we have no foreman-related
> data in Katello local db (unlike with pulp) and are essentially caching
> remote system data ignoring any cache-controls mandated by that remote
> system. I think this breaks ES usage pattern that was established
> earlier, and we should revise how ES is being used/interfaced with.
>
>> I'm also curious what it really means to have foreman-as-rails-engine
>> as an alternate approach. Do you have something we could look at that
>> explains how that would work?
>>
> http://guides.rubyonrails.org/engines.html. In short, rails engine is a
> way to embed a slice of functionality into another rails application. If
> I understood this correctly, foreman team has actually started looking
> into this.

I think to use Foreman as a rails engine is not easy thing to do. Rails 
application are typically not developed as embeddable - they pollute 
Ruby global namespace a lot. There would be a lot of collisions in 
classes (e.g. User in Katello and Foreman), in database tables and other 
problems.

I would rather not go this way.

Petr

> -d
>
>> Mike
>
>
>
>
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel
>