Associating Riak CRDT Sets to Buckets / Keys via Erlang Client

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Associating Riak CRDT Sets to Buckets / Keys via Erlang Client

Vikram Lalit
Hi - I am trying to leveraging CRDT sets to store chat messages that my distributed Riak infrastructure would store. Given the intrinsic conflict-resolution, I thought this might be more beneficial than me putting together a merge implementation based on the causal context.

However, my data model requires each chat message to be associated to something like a post, hence I was thinking of having the post reference as the bucket, and chat references as keys in that bucket. With of course the bucket-type datasource equated to 'set'. Unfortunately though, from the documentation, I'm not able to ascertain how to associate a created set with an existing bucket and a new key reference if I use the Erlang client. This seems possible for other languages but not for Erlang, with the Basho doc mentioning  "%% Sets in the Erlang client are opaque data structures that collect operations as you mutate them. We will associate the data  structure with a bucket type, bucket, and key later on.".

Subsequent code only seems to fetch the set from the bucket / key but where exactly is the allocation happening?

{ok, SetX} = riakc_pb_socket:fetch_type(Pid, {<<"sets">>,<<"travel">>}, <<"cities">>).

Perhaps I'm missing something, or is there a code snippet that I can leverage?

Thanks!

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Associating Riak CRDT Sets to Buckets / Keys via Erlang Client

Magnus Kessler
On 16 November 2016 at 17:40, Vikram Lalit <[hidden email]> wrote:
Hi - I am trying to leveraging CRDT sets to store chat messages that my distributed Riak infrastructure would store. Given the intrinsic conflict-resolution, I thought this might be more beneficial than me putting together a merge implementation based on the causal context.

However, my data model requires each chat message to be associated to something like a post, hence I was thinking of having the post reference as the bucket, and chat references as keys in that bucket. With of course the bucket-type datasource equated to 'set'. Unfortunately though, from the documentation, I'm not able to ascertain how to associate a created set with an existing bucket and a new key reference if I use the Erlang client. This seems possible for other languages but not for Erlang, with the Basho doc mentioning  "%% Sets in the Erlang client are opaque data structures that collect operations as you mutate them. We will associate the data  structure with a bucket type, bucket, and key later on.".

Subsequent code only seems to fetch the set from the bucket / key but where exactly is the allocation happening?

{ok, SetX} = riakc_pb_socket:fetch_type(Pid, {<<"sets">>,<<"travel">>}, <<"cities">>).

Perhaps I'm missing something, or is there a code snippet that I can leverage?

Thanks!


Hi Vikram,

Please have a look at the following snippet, that shows the complete set of operations used to update a CRDT set with the Erlang client:

update_crdt_set(Server, BType, Bucket, Key, Val) ->
        T = unicode:characters_to_binary(BType),
        B = unicode:characters_to_binary(Bucket),
        K = unicode:characters_to_binary(Key),

        {ok, Pid} = riakc_pb_socket:start_link(Server, 8087),

        Set = case
                riakc_pb_socket:fetch_type(Pid, {T, B}, K)
        of
                {ok, O} -> O;
                {error, {notfound, set}} -> riakc_set:new()
        end,

        Set1 = riakc_set:add_element(unicode:characters_to_binary(Val), Set),

        {ok, {set, Vals, _Adds, _Dels, _Ctx}} = riakc_pb_socket:update_type(
                Pid, {T, B}, K, riakc_set:to_op(Set1), [return_body]),
        Vals.

The set is updated with riakc_set:add_element/2, and sent back to the server with riakc_pb_socket:update_type/5, which in turn takes an argument returned from riakc_set:to_op/1.

More information and samples can be found in the Riak documentation [0] and the Riak Erlang client API docs [1][2].

Please let me know if this answered your question.

Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Associating Riak CRDT Sets to Buckets / Keys via Erlang Client

Vikram Lalit
This is awesome...! Many thanks Magnus - much appreciated...

Must have overlooked some of these details in my initial analysis but am sure I have a very good starting point / details now!

Thanks again!

On Thu, Nov 17, 2016 at 7:48 AM, Magnus Kessler <[hidden email]> wrote:
On 16 November 2016 at 17:40, Vikram Lalit <[hidden email]> wrote:
Hi - I am trying to leveraging CRDT sets to store chat messages that my distributed Riak infrastructure would store. Given the intrinsic conflict-resolution, I thought this might be more beneficial than me putting together a merge implementation based on the causal context.

However, my data model requires each chat message to be associated to something like a post, hence I was thinking of having the post reference as the bucket, and chat references as keys in that bucket. With of course the bucket-type datasource equated to 'set'. Unfortunately though, from the documentation, I'm not able to ascertain how to associate a created set with an existing bucket and a new key reference if I use the Erlang client. This seems possible for other languages but not for Erlang, with the Basho doc mentioning  "%% Sets in the Erlang client are opaque data structures that collect operations as you mutate them. We will associate the data  structure with a bucket type, bucket, and key later on.".

Subsequent code only seems to fetch the set from the bucket / key but where exactly is the allocation happening?

{ok, SetX} = riakc_pb_socket:fetch_type(Pid, {<<"sets">>,<<"travel">>}, <<"cities">>).

Perhaps I'm missing something, or is there a code snippet that I can leverage?

Thanks!


Hi Vikram,

Please have a look at the following snippet, that shows the complete set of operations used to update a CRDT set with the Erlang client:

update_crdt_set(Server, BType, Bucket, Key, Val) ->
        T = unicode:characters_to_binary(BType),
        B = unicode:characters_to_binary(Bucket),
        K = unicode:characters_to_binary(Key),

        {ok, Pid} = riakc_pb_socket:start_link(Server, 8087),

        Set = case
                riakc_pb_socket:fetch_type(Pid, {T, B}, K)
        of
                {ok, O} -> O;
                {error, {notfound, set}} -> riakc_set:new()
        end,

        Set1 = riakc_set:add_element(unicode:characters_to_binary(Val), Set),

        {ok, {set, Vals, _Adds, _Dels, _Ctx}} = riakc_pb_socket:update_type(
                Pid, {T, B}, K, riakc_set:to_op(Set1), [return_body]),
        Vals.

The set is updated with riakc_set:add_element/2, and sent back to the server with riakc_pb_socket:update_type/5, which in turn takes an argument returned from riakc_set:to_op/1.

More information and samples can be found in the Riak documentation [0] and the Riak Erlang client API docs [1][2].

Please let me know if this answered your question.

Kind Regards,

Magnus


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com