Storing relationship data

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Storing relationship data

Bryan Nagle
Hi,

For the project that I am currently working on, we are trying to decide on the best way to store relational data in riak.  At first glance, the obvious choice looks to be links, however one of our requirements is that this relationship information has to be sent to a client along with actual data.  The client has to be able to map the relationships between the data solely from the information it receives while being completely outside of Riak.

So, I was wondering if anyone had any suggestions?  We are considering either encapsulating our relationship data within the riak store itself (in the value tied to the key), or using riak links.  However, if we use riak links, then we have to convert those links into data that the client can receive & understand when sending data, and then convert the data we get back from the client into riak links;  we are wondering if this extra step is worth implementing this kind of a translation.

Bryan Nagle
Liquid Analytics
[hidden email]

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Storing relationship data

Justin Sheehy
Hi, Bryan.

The link data is embedded in the riak_object metadata, so you can
easily observe it from outside Riak even when not performing
link-walking queries.

To see this in action, check out the "Link" headers when using the
HTTP interface.

-Justin



On Thu, Dec 30, 2010 at 6:36 PM, Bryan Nagle
<[hidden email]> wrote:

> Hi,
> For the project that I am currently working on, we are trying to decide on
> the best way to store relational data in riak.  At first glance, the obvious
> choice looks to be links, however one of our requirements is that this
> relationship information has to be sent to a client along with actual data.
>  The client has to be able to map the relationships between the
> data solely from the information it receives while being completely outside
> of Riak.
> So, I was wondering if anyone had any suggestions?  We are considering
> either encapsulating our relationship data within the riak store itself (in
> the value tied to the key), or using riak links.  However, if we use riak
> links, then we have to convert those links into data that the client can
> receive & understand when sending data, and then convert the data we get
> back from the client into riak links;  we are wondering if this extra step
> is worth implementing this kind of a translation.
> Bryan Nagle
> Liquid Analytics
> [hidden email]
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Storing relationship data

Bryan Nagle
Hi Justin,

Our current setup, is we are using webmachine;  Client connects to webmachine, and webmachine connects to riak via the erlang pcb client.  So, if we use links, and we want the client to be aware of the relationships, we would still have to translate the links into the http response from webmachine back to the client;  or am I missing something?  

Thanks,

Bryan Nagle
Liquid Analytics
[hidden email]


On Thu, Dec 30, 2010 at 6:47 PM, Justin Sheehy <[hidden email]> wrote:
Hi, Bryan.

The link data is embedded in the riak_object metadata, so you can
easily observe it from outside Riak even when not performing
link-walking queries.

To see this in action, check out the "Link" headers when using the
HTTP interface.

-Justin



On Thu, Dec 30, 2010 at 6:36 PM, Bryan Nagle
<[hidden email]> wrote:
> Hi,
> For the project that I am currently working on, we are trying to decide on
> the best way to store relational data in riak.  At first glance, the obvious
> choice looks to be links, however one of our requirements is that this
> relationship information has to be sent to a client along with actual data.
>  The client has to be able to map the relationships between the
> data solely from the information it receives while being completely outside
> of Riak.
> So, I was wondering if anyone had any suggestions?  We are considering
> either encapsulating our relationship data within the riak store itself (in
> the value tied to the key), or using riak links.  However, if we use riak
> links, then we have to convert those links into data that the client can
> receive & understand when sending data, and then convert the data we get
> back from the client into riak links;  we are wondering if this extra step
> is worth implementing this kind of a translation.
> Bryan Nagle
> Liquid Analytics
> [hidden email]
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Storing relationship data

Justin Sheehy
Hi, Bryan.

On Thu, Dec 30, 2010 at 8:02 PM, Bryan Nagle
<[hidden email]> wrote:

> Our current setup, is we are using webmachine;  Client connects to
> webmachine, and webmachine connects to riak via the erlang pcb client.  So,
> if we use links, and we want the client to be aware of the relationships, we
> would still have to translate the links into the http response from
> webmachine back to the client;  or am I missing something?

You are correct with regard to standard behavior -- if there is a
layer between the client application and Riak, and you wish for the
actual links (as opposed to just the ability to traverse them) to be
visible to that client, then the intermediary must pass along the link
data as well.

There is a rarely-used alternative that might suit the scenario you
described if you find it too annoying to carry the metadata to your
client: you could set the "linkfun" bucket property to examine the
content of objects instead of metadata, and define your own custom
link serialization format to match your custom link storage format.
This would allow you to embed the links directly inside the objects
and still have mapreduce link queries work, but might break some other
things such as the HTTP interface to link walking.  I don't generally
recommend this path, as you'd be well outside the realm of "normal"
usage, but it is possible.

-Justin

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com