[Ovirt-devel] [R&D] Breaking the Browser

[Jeff Schroeder]

On Fri, Jul 4, 2008 at 7:33 PM, mark wagner <mwagner at redhat.com> wrote:
>
>
> Jeff Schroeder wrote:
>>
>> On Thu, Jul 3, 2008 at 2:14 PM, Jason Guiditta <jguiditt at redhat.com>
>> wrote:
>>>
>>> On Thu, 2008-07-03 at 13:59 -0700, Jeff Schroeder wrote:
>>>
>>>> Q: How do you get realtime graphs with a webapp?
>>>> A: SVG + AJAX
>>>>
>>>> A good example of how to do this would be the m0n0wall firewall.
>>>> Here is a screenshot of their interface graphs:
>>>> http://m0n0.ch/wall/images/screens/status_graph.png
>>>>
>>>> The svg graphs are drawn by the javascript. It gets the data from a
>>>> xmlhttprequest
>>>> feed directly from the server. It is really sexy technology and most
>>>> browsers
>>>> support it. Is asking IE users to install the Adobe SVG plugin or user
>>>> Firefox
>>>> too much?
>>>>
>>>> Take a look at how easy this is to do:
>>>> http://www.browserland.org/scripts/svgclock/
>>>>
>>>> If you compress that svg down, the whole thing is 4k. I don't even think
>>>> flash
>>>> could do something that smooth and clean looking in 4k. Were you aware
>>>> of this?
>>>>
>>> Agreed, and we are using svg for the graphs right now, though they do
>>> need some optimization and better javascript hooks.  I am not a
>>> proponent of us using flash for this, but others disagree.  Also, since
>>> adobe is discontinuing support for their svg viewer, there is another
>>> one I have read good things about called renesis:
>>> http://www.examotion.com/Downloads.product_player_download.0.html
>>
>> Besides internet explorer, a better question would be why not use
>> javascript + svg?
>> If someone that isn't you disagrees, can they give a solid technical
>> reason for this?
>>
>> What graph _absolutely has to be_ realtime? In all of the cases I
>> could come up with
>> not a one wouldn't be serviced by near-realtime graphs. Do you expect
>> users to have
>> ovirt graph screens up on a bank of plasma displays watching node
>> stats in realtime?
>
> So if your job is to monitor the 25 hosts in your pool and take immediate
> and
> effective action to mitigate any performance or catastrophic issues in said
> pool within one minute and 26 secs (3 nines, assuming this is the only issue
> that day) of the event, how are you going to monitor your pool ?

That is what monitoring software is for. If you expect someone to _look_ at
a webgui 24/7 you are approaching the problem from the absolute wrong angle.

> If we don't provide the ability to notify immediately when a host goes down,
> you are rapidly eating into time specified to resolve a problem.
You are agreeing with me on this one. This is what monitoring software is for,
not a website. Something like nagios could call a pager or run a script to do
something much faster than a human could.

> The thing that people seem to be missing or ignoring is that this a
> Systems Management Tool. Often times, there will be Service Level Agreements
> associated with with running a NOC.  Not making data available in real
> time is going to exclude ovirt from sites.
No, you missed my point entirely. SLAs in a NOC are perfectly reasonable.

> We need to look at making the nav bar have near realtime capabilities.
> I need to know if a system is getting close to capacity (change the color
> of the icon?) or is offline.
Changing the color of the icon would be good, but the admin should be able
to set threshholds. When X reaches X%, send alert to foo at bar.com or run
script /usr/local/bin/foo-alert.sh

> If I have a graph of CPU utilization in my browser for the last two
> hours of "my pool" it should automatically get updated as data becomes
> available or my finger will get tired from hitting refresh. Never
> underestimate the strain on a server caused by people hitting refresh
> to see if the data has changed, if I know it will auto update, I cease
> that behavior.
> (How often to update the data from the host and guests is also an important
> part of this equation)
If you have to hit refresh over and over you are doing this completely wrong.

>> Even in a big fortune 100 NOC this seems unreasonable and a bad use case.
>
> So are you saying we should focus our attention on providing real time
> alerts
> / event notification?
> How do the admins get notified as quickly as possible?
> Do they sit in front of a terminal and hit refresh on there browser?
> Set their mail clients to fetch every 10 secs ?
> This is clearly a case where time is money, if you are in a big NOC and
> miss your SLA it could cost big bucks, not to mention a job or two.

Mark, without causing any bad blood, this conversation is basicly over. You
agree with me that notification should happen as soon as possible. You disagree
with me on how it should be done. For true autonomy, the human aspect
needs to be taken out of the picture as much as possible. This is why I'd
argue a script should automatically do alerting and NOT a human. What
the human does with that alert is up to the business.

>> Once oVirt a policy engine (like VMotion)  the graphs updating in
>> realtime don't seem
>> near as useful.
>
> This does not make the case not to have the ability *now*. Just that it may
> not
> be needed later.

Yes we are both aware of this and that it will take a few years for
this to materialize.


> Can you think of a reason that demands absolute realtime graphs?
>
> I agree that no graph "has to be realtime".  We could do things via a "top"
> like tool or other means. However, graphs tend to be easier to read,
> although "top"
> updates more frequently....
>
> The argument you make is actually a not meaningful here because there is
> really
> nothing that "demands ovirt". Its a tool to (hopefully) make management of a
> virtualized easier, but its not an absolute to deploying virtualization. You
> could
> do the same management with existing tools, just be a lot harder and costly.
>
> In my mind, realtime graphs in some limited form could be a useful feature.
> For instance, I don't think that we need a realtime graph to show the
> monthly
> data (sigh, but now I know that some customers will want it, double sigh )
>
>
>
>>
>> Here is how I envision oVirt with a 2-3 major releases under it's belt.
>
> You realize that two or three major releases are at least several years
> away.
> Do we have the functionality required now to ensure that there is will be
> sufficient demand for ovirt that far out ?
> The internet is littered with many projects that never deliver the
> functionality
> needed to survive until their third major release is ready...
>
>>
>> AMQP bus message:  OH NOES, Node3 is about to bust out the oom killer to
>> lay the
>>                                 smack down on vm1 and vm2. Run
>> around, scream, and panic!
>> oVirt Policy Engine:    Live migrates vm1 and vm2 from Node3 to Node4
>> because Node4 is idle.
>> AMQP bus message:  Disaster diverted by policy engine. All systems
>> nominal.
>> oVirt Policy Engine:    Goes back to saving the world and drinking it's
>> latte
>> Systems Admin:         Hey look at that, something bad almost happened but
>> this
>>                                 cool software fixed itsself. /goes
>> back to reading xkcd
>>
>> What do you think?
>
> This seems to require real time data to do its job effectively.  Doing it
> after the fact will interrupt your comics...
>
>
> We do need to be concerned about overtaxing the servers with polling from
> many clients. The correct solution would most likely involve a more
> distributed architecture that distributes the load and can sustain growth
> to meet the needs of the customers. A push model for certain types of data
> is also essential.
>
> Keep in mind that we are trying to solve problems that have been resolved
> many times over the years. The basic architecture of network management
> apps is that a server polls for stats and events / alerts get pushed to
> the server. With a distributed system, the servers need to push the events
> amongst themselves. Stats can be stored centrally or in a distributed
> fashion, the important aspect is that the data is available to all the
> servers .
>
> -mark
>



-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com