Monitor RIAK process with Supervisord

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Monitor RIAK process with Supervisord

vmalhotra
This post was updated on .
We run a 8 nodes RIAK cluster in our Prod environment. Lot of time, RIAK process stops and we also noticed out of memory issues. Typically, we run restart the affected node to recover from the issue. I thought of using Supervisor to control the RIAK processes so the idea is if any of the process crash SupervisorD daemon will automatically restart that process on a crash.

Wanted to know what you guys think? Can it cause any other issue or it should work fine?

Thanks in advance.
Reply | Threaded
Open this post in threaded view
|

Re: Monitor RIAK process with Supervisord

Magnus Kessler

On 29 October 2016 at 19:59, vmalhotra <[hidden email]> wrote:
We run 8 nodes RIAK cluster in our Prod environment. Lot of time, RIAK
process stops and we also noticed out of memory issues. Typically, we run
restart the affected node to recover from the issue. I thought of using
Supervisor to control the RIAK processes so the idea is if any of the
process crash SupervisorD daemon will automatically restart that process on
a crash.

Wanted to know what you guys think? Can it cause any other issue or it
should work fine?

Hi Varun,

I would recommend against blindly restarting Riak nodes, in particular if these were shut down uncleanly, as may happen in out of memory situations. There is a risk that an unclean shutdown leaves behind corrupted files and that a subsequent restart is unsuccessful.

You should instead investigate why Riak stops being responsive.

Please have a look at the documentation, in particular memory requirements[0], and OS tuning[1].

Kind Regards,

Magnus



 

Thanks in advance.



--
View this message in context: http://riak-users.197444.n3.nabble.com/Monitor-RIAK-process-with-Supervisord-tp4034655.html
Sent from the Riak Users mailing list archive at Nabble.com.

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com