A bunch of Erlang client questions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

A bunch of Erlang client questions

Alex Rønne Petersen
Hi list,

I have a bunch of random questions about the Erlang API that aren't
quite answered by the documentation (at least from what I could find):

* Is the proper way to replace the object associated with a bucket/key
pair to simply do a riakc_pb_socket:put/2 with the same bucket/key
pair (with a new object)?
* What exactly is the purpose of riakc_obj:update_value/2 (with
regards to the above question)?
* How does one associate a secondary index with an object? Is this
done via riakc_obj:update_metadata/2?
* Is there any particular reason the Erlang client doesn't yet support
all bucket properties?
* What exactly do the various get and put options do? Specifically,
if_modified, notfound_ok, basic_quorum, head, and deletedvclock for
the former and return_body, return_head, if_not_modified, and
if_none_match for the former.
* How will riakc_pb_socket:get/3 behave when queue_if_disconnected is
enabled? Will it fail immediately if the socket is disconnected or
block until it has reconnected?

Thanks in advance!

Regards,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A bunch of Erlang client questions

bryan-basho
Administrator
Hi, Alex. I'm sorry you weren't able to find your answers in our docs.
Some of them assume that you've read all of the other docs as well.
I'll see what I can do to clear things up inline.

On Mon, Nov 26, 2012 at 7:23 PM, Alex Rønne Petersen
<[hidden email]> wrote:
> * Is the proper way to replace the object associated with a bucket/key
> pair to simply do a riakc_pb_socket:put/2 with the same bucket/key
> pair (with a new object)?

This one is answered in the "Modifying Data" section of the README:

https://github.com/basho/riak-erlang-client/blob/master/README.md#modifying-data

In short, the proper way to replace an object is to first fetch the
existing object, then update the value of what you fetched, then use
riakc_pb_socket:put to write the new value. The reason for doing this
is to comply with Riak's consistency system (the vector clocks) - it
needs to know what value you were starting from in order to overwrite
it correctly.

> * What exactly is the purpose of riakc_obj:update_value/2 (with
> regards to the above question)?

The riakc_obj structure stores two different things: the
value+metadata you read, and the value+metadata you want to write. The
update_value and update_metadata functions set the value+metadata you
want to write. The same can be said about get_value/get_metadata and
get_update_value/get_update_metadata - they each access one part of
the riakc_obj structure. Although, as was discussed recently on this
list, get_update_value will return the result of get_value if no
update value has been set.

> * How does one associate a secondary index with an object? Is this
> done via riakc_obj:update_metadata/2?

Yes, you need to set the "index" metadata entry to associate a
secondary index with an object. For example, to set values the
"myidx_bin" and "youridx_int" indexes:

    Obj = riakc_obj:new(Bucket, Key, Value).
    Idx = [{"myidx_bin", "hello"},{"youridx_int","9876"}].
    MD = dict:store(<<"index">>, Idx, dict:new()).
    ObjIndexed = riakc_obj:update_metadata(Obj, MD).
    riakc_pb_socket:put(Socket, ObjIndexed).

If overwriting a value, you would use the result of
riakc_obj:get_metadata/1 instead of dict:new/0 for building your new
metadata.

> * Is there any particular reason the Erlang client doesn't yet support
> all bucket properties?

Yes, it does not support the bucket properties that the Protocol
Buffers interface to Riak does not support. It's a known problem. You
can follow work on it here:

https://github.com/basho/riak_pb/issues/5

In the meantime, the riak-erlang-http-client can be used to set other
bucket properties:

https://github.com/basho/riak-erlang-http-client

> * What exactly do the various get and put options do? Specifically,
> if_modified, notfound_ok, basic_quorum, head, and deletedvclock for
> the former

These answers can be found in the docs for the Protocol Buffers get message:

http://docs.basho.com/riak/latest/references/apis/protocol-buffers/PBC-Fetch-Object/

 and return_body, return_head, if_not_modified, and
> if_none_match for the former.

And these answers are in the docs for the Protocol Buffers put message:

http://docs.basho.com/riak/latest/references/apis/protocol-buffers/PBC-Store-Object/

> * How will riakc_pb_socket:get/3 behave when queue_if_disconnected is
> enabled? Will it fail immediately if the socket is disconnected or
> block until it has reconnected?

With queue_if_disconnected enabled, riakc_pb_socket:get/3 will block
until the socket is reconnected.

HTH,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A bunch of Erlang client questions

Alex Rønne Petersen
Hi Bryan,

Thanks for the answers! A few more questions inline:

On Tue, Nov 27, 2012 at 2:16 PM, Bryan Fink <[hidden email]> wrote:

> Hi, Alex. I'm sorry you weren't able to find your answers in our docs.
> Some of them assume that you've read all of the other docs as well.
> I'll see what I can do to clear things up inline.
>
> On Mon, Nov 26, 2012 at 7:23 PM, Alex Rønne Petersen
> <[hidden email]> wrote:
>> * Is the proper way to replace the object associated with a bucket/key
>> pair to simply do a riakc_pb_socket:put/2 with the same bucket/key
>> pair (with a new object)?
>
> This one is answered in the "Modifying Data" section of the README:
>
> https://github.com/basho/riak-erlang-client/blob/master/README.md#modifying-data

I don't know how I managed to miss that. Thanks!

>
> In short, the proper way to replace an object is to first fetch the
> existing object, then update the value of what you fetched, then use
> riakc_pb_socket:put to write the new value. The reason for doing this
> is to comply with Riak's consistency system (the vector clocks) - it
> needs to know what value you were starting from in order to overwrite
> it correctly.

OK, makes sense. Just out of curiosity, though, is updating values via
put without an update a supported scenario in some way or is it
generally considered a Bad Idea (tm)?

>
>> * What exactly is the purpose of riakc_obj:update_value/2 (with
>> regards to the above question)?
>
> The riakc_obj structure stores two different things: the
> value+metadata you read, and the value+metadata you want to write. The
> update_value and update_metadata functions set the value+metadata you
> want to write. The same can be said about get_value/get_metadata and
> get_update_value/get_update_metadata - they each access one part of
> the riakc_obj structure. Although, as was discussed recently on this
> list, get_update_value will return the result of get_value if no
> update value has been set.
>
>> * How does one associate a secondary index with an object? Is this
>> done via riakc_obj:update_metadata/2?
>
> Yes, you need to set the "index" metadata entry to associate a
> secondary index with an object. For example, to set values the
> "myidx_bin" and "youridx_int" indexes:
>
>     Obj = riakc_obj:new(Bucket, Key, Value).
>     Idx = [{"myidx_bin", "hello"},{"youridx_int","9876"}].
>     MD = dict:store(<<"index">>, Idx, dict:new()).
>     ObjIndexed = riakc_obj:update_metadata(Obj, MD).
>     riakc_pb_socket:put(Socket, ObjIndexed).
>
> If overwriting a value, you would use the result of
> riakc_obj:get_metadata/1 instead of dict:new/0 for building your new
> metadata.

Does Riak figure out based on the name of the index what type it is?
That is, *_int = integer, *_bin = string?

Is there any physical limit to how many secondary indexes can be
created for an object? Or perhaps a recommended limit?

>
>> * Is there any particular reason the Erlang client doesn't yet support
>> all bucket properties?
>
> Yes, it does not support the bucket properties that the Protocol
> Buffers interface to Riak does not support. It's a known problem. You
> can follow work on it here:
>
> https://github.com/basho/riak_pb/issues/5
>
> In the meantime, the riak-erlang-http-client can be used to set other
> bucket properties:
>
> https://github.com/basho/riak-erlang-http-client
>
>> * What exactly do the various get and put options do? Specifically,
>> if_modified, notfound_ok, basic_quorum, head, and deletedvclock for
>> the former
>
> These answers can be found in the docs for the Protocol Buffers get message:
>
> http://docs.basho.com/riak/latest/references/apis/protocol-buffers/PBC-Fetch-Object/
>
>  and return_body, return_head, if_not_modified, and
>> if_none_match for the former.
>
> And these answers are in the docs for the Protocol Buffers put message:
>
> http://docs.basho.com/riak/latest/references/apis/protocol-buffers/PBC-Store-Object/

When if_none_match is enabled, will riakc_pb_socket:put/3 return ok
regardless of whether the value was stored?

>
>> * How will riakc_pb_socket:get/3 behave when queue_if_disconnected is
>> enabled? Will it fail immediately if the socket is disconnected or
>> block until it has reconnected?
>
> With queue_if_disconnected enabled, riakc_pb_socket:get/3 will block
> until the socket is reconnected.
>
> HTH,
> Bryan

Regards,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: A bunch of Erlang client questions

bryan-basho
Administrator
On Tue, Nov 27, 2012 at 11:41 AM, Alex Rønne Petersen
<[hidden email]> wrote:

>> In short, the proper way to replace an object is to first fetch the
>> existing object, then update the value of what you fetched, then use
>> riakc_pb_socket:put to write the new value. The reason for doing this
>> is to comply with Riak's consistency system (the vector clocks) - it
>> needs to know what value you were starting from in order to overwrite
>> it correctly.
>
> OK, makes sense. Just out of curiosity, though, is updating values via
> put without an update a supported scenario in some way or is it
> generally considered a Bad Idea (tm)?

This depends on each bucket's configuration of allow_mult. In the
default setup, with allow_mult=false, writing without fetching first
causes Riak to simply store the new value instead of the old value,
with conflict resolution (during read-repair/etc.) done simply by
comparing modification timestamps. With allow_mult=true, writing
without fetching first causes Riak to store sibling objects.

> Does Riak figure out based on the name of the index what type it is?
> That is, *_int = integer, *_bin = string?
>
> Is there any physical limit to how many secondary indexes can be
> created for an object? Or perhaps a recommended limit?

Answers to both of these are on the poorly-named "Configuring
Secondary Indexes" page:

http://docs.basho.com/riak/latest/cookbooks/Secondary-Indexes---Configuration/

> When if_none_match is enabled, will riakc_pb_socket:put/3 return ok
> regardless of whether the value was stored?

No, it looks like it returns an error in that case:

    6> O = riakc_obj:new(<<"one">>, <<"two">>, <<"three">>).
    {riakc_obj,<<"one">>,<<"two">>,undefined,[],undefined,
               <<"three">>}
    7> riakc_pb_socket:put(S, O, [if_none_match]).
    ok
    8> riakc_pb_socket:put(S, O, [if_none_match]).
    {error,<<"match_found">>}

-Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Loading...