[katello-devel] Reduce elasticsearch memory

[Mike McCune]

On 02/08/2012 04:05 AM, Lukas Zapletal wrote:
> Hey,
>
> there is a BZ in regard to elastic search memory consumption.
>
> Until it is fixed you can easily tune memory setting with:
>
> # grep MEM /etc/sysconfig/elasticsearch
> ES_MIN_MEM=32m
> ES_MAX_MEM=64m
>
> I recommend to check -Xms and -Xmx options after restart:
>
> # ps axuwww | grep ElasticSearch
> 996        780 59.3  6.3 1023540 75608 ?       Sl   13:02   0:01
> /usr/bin/java -Xms32m -Xmx64m .............
>
> ps - not sure if 64 MB is enough, I will see :-)
>
> LZ
>

FYI, we had a bug to increase it (786572) which I just pushed a fix for 
as outlined below.  We were seeing OOME errors with the old limit of 1G 
so I bumped it to 1.5G. You probably don't need that much during 
development so the above stands true.  ES's preferred way of setting it 
is by the use of their sourced environment variable script here:

/usr/share/java/elasticsearch/bin/elasticsearch.in.sh

we prepend the MX/MS settings there at the top:

ES_MIN_MEM=1512m
ES_MAX_MEM=1512m

we also are following the advice in the configuration files that 
indicate you should set the MAX/MIN to be the same.

commit d0ea23ae941f6d02fff7cdcc82d05d26953e1f46
Author: Mike McCune <mmccune at redhat.com>
Date:   Tue Feb 7 17:00:09 2012 -0800

     786572 - force in max/min heap sizes to 1.5G vs the current 1G limit

diff --git a/puppet/modules/elasticsearch/manifests/config.pp 
b/puppet/modules/elasticsearch/manifests/config.pp
index 181ec15..5313693 100644
--- a/puppet/modules/elasticsearch/manifests/config.pp
+++ b/puppet/modules/elasticsearch/manifests/config.pp
@@ -10,4 +10,12 @@ class elasticsearch::config {
       owner => "elasticsearch",
       group => "elasticsearch";
    }
+
+  # Set elasticsearch's heap sizes
+  exec { "/bin/sed -i '1i ES_MIN_MEM=1512m' 
/usr/share/java/elasticsearch/bin/elasticsearch.in.sh":
+       unless => "/bin/grep -qFx 'ES_MIN_MEM=1512m' 
/usr/share/java/elasticsearch/bin/elasticsearch.in.sh"
+  }
+  exec { "/bin/sed -i '1i ES_MAX_MEM=1512m' 
/usr/share/java/elasticsearch/bin/elasticsearch.in.sh":
+       unless => "/bin/grep -qFx 'ES_MAX_MEM=1512m' 
/usr/share/java/elasticsearch/bin/elasticsearch.in.sh"
+  }
  }


commit d0ea23ae941f6d02fff7cdcc82d05d26953e1f46
Author: Mike McCune <mmccune at redhat.com>
Date:   Tue Feb 7 17:00:09 2012 -0800

     786572 - force in max/min heap sizes to 1.5G vs the current 1G limit



-- 
Mike McCune
mmccune AT redhat.com
Red Hat Engineering       | Portland, OR
Systems Management        | 650.254.4248