cluster clear without stopping node

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

cluster clear without stopping node

Daniel Miller

According to the docs “Running [riak-admin cluster clear] will also stop the current node in addition to clearing any staged changes.

Is there a way to clear the current cluster plan without stopping the current node?

Even better (shameless feature request) would be to have a way to “replan” the current plan. That is, clear the plan and then redo the currently planned actions with a single command. Often I have a plan that ends up with an uneven ring allocation and I’ve noticed that I can sometimes get a better allocation if I clear and re-plan several times, but this is tedious.

Thanks!
Daniel


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cluster clear without stopping node

Magnus Kessler
On 8 February 2017 at 21:33, Daniel Miller <[hidden email]> wrote:

According to the docs “Running [riak-admin cluster clear] will also stop the current node in addition to clearing any staged changes.

Is there a way to clear the current cluster plan without stopping the current node?

Hi Daniel,

That statement from the docs is wrong. [riak-admin cluster clear] does not stop the current node in general. It may stop the node if the command is run on a joining node, and the join is cancelled, but should be safe to run on all other nodes. I have reached out to the documentation team to get this fixed.

Even better (shameless feature request) would be to have a way to “replan” the current plan. That is, clear the plan and then redo the currently planned actions with a single command. Often I have a plan that ends up with an uneven ring allocation and I’ve noticed that I can sometimes get a better allocation if I clear and re-plan several times, but this is tedious.

You can run  [riak-admin cluster plan] more than once before  [riak-admin cluster commit], and it may generate a different transition plan every time depending on cluster state.
 

Thanks!
Daniel

Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cluster clear without stopping node

Daniel Miller

Hi Magnus,

Thanks, this is great news! I was worried that clearing a planned node removal would stop the node from which I was issuing the clear command, which would be bad. Sounds like I have nothing to worry about in that case.

On running cluster plan multiple times to generate a new plan: this is a little surprising since I think I’ve done that before but I haven’t observed it generating a new plan. I’ll try it again.

Thanks for very quick response.
Daniel


On Thu, Feb 9, 2017 at 9:27 AM, Magnus Kessler <[hidden email]> wrote:
On 8 February 2017 at 21:33, Daniel Miller <[hidden email]> wrote:

According to the docs “Running [riak-admin cluster clear] will also stop the current node in addition to clearing any staged changes.

Is there a way to clear the current cluster plan without stopping the current node?

Hi Daniel,

That statement from the docs is wrong. [riak-admin cluster clear] does not stop the current node in general. It may stop the node if the command is run on a joining node, and the join is cancelled, but should be safe to run on all other nodes. I have reached out to the documentation team to get this fixed.

Even better (shameless feature request) would be to have a way to “replan” the current plan. That is, clear the plan and then redo the currently planned actions with a single command. Often I have a plan that ends up with an uneven ring allocation and I’ve noticed that I can sometimes get a better allocation if I clear and re-plan several times, but this is tedious.

You can run  [riak-admin cluster plan] more than once before  [riak-admin cluster commit], and it may generate a different transition plan every time depending on cluster state.
 

Thanks!
Daniel

Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: cluster clear without stopping node

Magnus Kessler
On 9 February 2017 at 14:46, Daniel Miller <[hidden email]> wrote:

Hi Magnus,

Thanks, this is great news! I was worried that clearing a planned node removal would stop the node from which I was issuing the clear command, which would be bad. Sounds like I have nothing to worry about in that case.

The behaviour of `riak-admin cluster plan` is quite complex, depending on the staged changes. If a `leave` operation has been staged, it will just undo the staged change; no node will be stopped. However, if a `join` operation has been staged, the joining node will be shut down after its ring has been cleared. When this node restarts, it will behave like a fresh unjoined node and can be joined again. If `riak-admin cluster clear` was run from a node that remains in the cluster, this node will be unaffected.
 

On running cluster plan multiple times to generate a new plan: this is a little surprising since I think I’ve done that before but I haven’t observed it generating a new plan. I’ll try it again.

You may see the effect more pronounced if you use `claim_v3`, which is more aggressive. For a discussion about how to rebalance the ring without adding or removing nodes, please see http://lists.basho.com/pipermail/riak-users_lists.basho.com/2016-November/018815.html
 

Thanks for very quick response.
Daniel


Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Loading...