[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [augeas-devel] improving performance of aug_get() and aug_match() with large datasets

On 10/05/2015 12:31 PM, David Lutterkort wrote:
On Mon, Oct 5, 2015 at 6:54 AM, Laine Stump <laine redhat com> wrote:

augeas netcf  libvirt list   dump    virt-manager
------ -----  ------- ------ ------- ------------
1.4.0  0.2.8  1.2.20  1:37.6 13:46.6 15:37
upstrm 0.2.8  1.2.20  1:04.7 07:34.8 08:41
upstrm upstrm 1.2.20  0:03.7 06:40.3 06:46
upstrm upstrm upstrm  0:02.0 06:39.5 06:39

With your suggested query implemented in netcf:

upstrm upstrm upstrm  0:02.1 0:17.7 00:17

so nearly identical to the timing when I *removed that query completely*!

This all looks very encouraging - are these times satisfactory for users or do we need to look more into speeding them up further ?

I would say that 60x better performance looks impressive to anyone :-). Especially since:

1) The number of users with this many host interfaces is very small, and

2) upstream virt-manager becomes usable as soon as it has the list of all interface names is returned (which now takes 2sec rather than 1m37s), with the remaining 15 sec simply consuming one CPU core while the user goes merrily on their way.

So I think this much improvement is good enough for now. (It would be nice if aug_load() could take advantage of inotify or something similar to avoid the large overhead of checking all files' mtimes though - even though it works in practice, I'm still not comfortable with the idea that we could occasionally have information that is up to 1 second out of date, as this could lead to race conditions if some other entity is modifying the config on disk directly.)

Thanks for all your help with this. The resulting improvement has been *much* better than I ever expected.

(BTW - please feel free to review/ACK the netcf patch that I posted to netcf-devel yesterday :-); you surely know the details behind it better than anyone else, and I posted it mainly so that you could verify I am implementing what you said. (Note that your current address has been whitelisted, so you don't need to subscribe)

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]