[Riak KV] Adding new algorithm

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[Riak KV] Adding new algorithm

nikhilrane
Hello all,

I am trying to understand Riak Key Value store as a whole to add new replication algorithms. I moved around a lot of files for the "put" flow: riak_vnode_master, riak_kv_put_fsm, riak_ensemble_*, etc. but still do not understand it well.

Could anyone please suggest an approach or important files/findings?

I would really appreciate any help! Thanks a lot.
Reply | Threaded
Open this post in threaded view
|

Re: [Riak KV] Adding new algorithm

Christopher Meiklejohn

> On Feb 13, 2015, at 5:03 AM, nikhilrane <[hidden email]> wrote:
>
> Hello all,
>
> I am trying to understand Riak Key Value store as a whole to add new
> replication algorithms. I moved around a lot of files for the "put" flow:
> riak_vnode_master, riak_kv_put_fsm, riak_ensemble_*, etc. but still do not
> understand it well.
>
> Could anyone please suggest an approach or important files/findings?

Hi there,

It’s hard to point you to just one particular resource that will answer your
question.  Maybe we can attempt a different approach.  What in particular
are you looking to change?  With that information, we may be able to point
you in the right direction.

- Chris

Christopher Meiklejohn
Senior Software Engineer
Basho Technologies, Inc.
[hidden email]


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: [Riak KV] Adding new algorithm

nikhilrane
Hello Chris,

thank you for the quick reply. To put simply, I am planning to change Riak in a way that it can support pluggable replication strategies. For instance, as Riak supports backends selection through a simple property file, I would implement some replication algorithms and switch them using such a property.

My main motto is towards strict consistency algorithms. Hence excavating the "put" flow to start with. Will it be possible for you to guide me in some direction?

Thank you,
Nikhil.
Reply | Threaded
Open this post in threaded view
|

Re: [Riak KV] Adding new algorithm

Christopher Meiklejohn

> On Feb 13, 2015, at 11:59 AM, nikhilrane <[hidden email]> wrote:
>
> Hello Chris,
>
> thank you for the quick reply. To put simply, I am planning to change Riak
> in a way that it can support pluggable replication strategies. For instance,
> as Riak supports /backends/ selection through a simple property file, I
> would implement some replication algorithms and switch them using such a
> property.
>
> My main motto is towards strict consistency algorithms. Hence excavating the
> "put" flow to start with. Will it be possible for you to guide me in some
> direction?
>
> Thank you,
> Nikhil.
>

Hi Nikhil,

Can you clarify which consistency models you’d like to support?

For instance, riak_ensemble’s integration with riak_kv shows a straightforward
example of getting single-key serializability, however, if you wanted to provide
something along the lines of causal consistency (or non-monotonic snapshot
isolation), you’ll most likely need to write a new coordinator mechanism and
perform significant rewrites of the kv_vnode, as some of these mechanisms
require locking, and the ability to store multiple versions of keys in the data
store.

- Chris

Christopher Meiklejohn
Senior Software Engineer
Basho Technologies, Inc.
[hidden email]


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: [Riak KV] Adding new algorithm

nikhilrane
Hello Chris,

I plan to add some Chaining replication mechanisms & Paxos variants. The biggest advantage I have is that the nature of data in my case is simple CREATE-DELETE. So, this means, the <key, value> pairs would just be created, replicated and at some point, deleted; but  never  updated. Hence I presume the refactoring required should be less.

To start with, and get a hang-on, I will go with simple chain replication mechanism.

- Thank you,
Nikhil.