[Jenkins-infra] [support.osuosl.org #22394] eggplant down (140.211.15.101)

[Kohsuke Kawaguchi]

Thanks. If you are going to give us more RAMs, I am not going to refuse :-)

The last time this situation happened was last October. I don't think I 
looked into syslog back then, but the failure mode was quite similar, so 
it's probably the same cause. I don't know every twice a year qualifies 
as "OOM'ing a lot".

I just finished adjusting oom_adj for those two daemons.

We do no track VM memory usage at all. If you have an infrastructure to 
do that, your adding this machine to your setup would be greatly 
appreciated.


On 05/01/2013 10:28 AM, Lance Albertson via RT wrote:
> On Wed, May 1, 2013 at 10:04 AM, kk at kohsuke.org via RT
> <support at osuosl.org>wrote:
>
>> Thank you for a quick response.
>>
>> Yes, indeed syslog indicates OOM.
>>
>> This box runs two heavy-weight Java processes. I guess what I should do
>> is to make these services a better target for OOM killer so that when
>> the available memory gets low we won't make the whole system unresponsive?
>>
>
> We probably should increase the RAM the VM gets if its OOM'ing a lot or
> see if you can tune the java processes running to use less ram (if its
> possible). Are you guys graphing the VM's memory usage at all using munin?
> We could just add it to our munin graphing if you don't. That will help us
> out quite a bit on determining how much ram you need.
>
> Thanks-
>


-- 
Kohsuke Kawaguchi                          http://kohsuke.org/