Link walking with Erlang riak client

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Link walking with Erlang riak client

Morgan Segalis
Hi everyone !

I'm looking for a NoSQL solution to get ride of MySQL.
In this particular issue, I think that Riak is one of the top of my choices in order to do what I'm looking for.

I would like to store all my users into Riak.
I understand that Riak is document oriented, so that's great for getting every information really fast from one user.
One of the feature I'm really interested in, are Links.

Links seems to be a great way for me to create association between users…

When A adds B, I create a link from A to B.
When B accepts A's invitation, I create a link from B to A.

If links are not both ways, e.g. A to B to A, message can not go through (B who didn't accept A should not get its message yet)

My server is coded in Erlang, I have checked the Erlang riak client, and I didn't see something as simple as the HTTP use of Links in the demo.

1 - Are Link walking in Erlang fast & easy ?
2 - Is it possible to check fast if A -> B -> A (still in Erlang)? ('->' represents Links)

Thank you for your help !

Morgan.
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

Eric Moritz
Links may not be the most appropriate way to model a set of associated
users.  They have a limited length and both the user data and the
links have to be saved at the same time.

What you can do is emulate a set using a separate bucket and secondary indexes.

So basically the bucket would look like this:

So if I'm a friend of yours,

PUT /buckets/friends/keys/eric-morgan
x-riak-index-friend_id_bin: morgan
x-riak-index-user_id_bin: eric
Content-Type: application/json

{"user_id": "eric", "friend_id": "morgan"}

Then you can fetch a set of my friends by doing a 2i query for the
bucket friends and the index
user_id.

If you want to a set of all the people that have friended you, do a 2i
query on friend_id with the value "morgan".

You can quickly see if we're mutual friends by fetching:

/buckets/friends/keys/eric-morgan and /buckets/friends/keys/morgan-eric.

If both are found, then we're mutual friends.

Eric.

On Tue, Jun 19, 2012 at 12:35 PM, Morgan Segalis <[hidden email]> wrote:

> Hi everyone !
>
> I'm looking for a NoSQL solution to get ride of MySQL.
> In this particular issue, I think that Riak is one of the top of my choices in order to do what I'm looking for.
>
> I would like to store all my users into Riak.
> I understand that Riak is document oriented, so that's great for getting every information really fast from one user.
> One of the feature I'm really interested in, are Links.
>
> Links seems to be a great way for me to create association between users…
>
> When A adds B, I create a link from A to B.
> When B accepts A's invitation, I create a link from B to A.
>
> If links are not both ways, e.g. A to B to A, message can not go through (B who didn't accept A should not get its message yet)
>
> My server is coded in Erlang, I have checked the Erlang riak client, and I didn't see something as simple as the HTTP use of Links in the demo.
>
> 1 - Are Link walking in Erlang fast & easy ?
> 2 - Is it possible to check fast if A -> B -> A (still in Erlang)? ('->' represents Links)
>
> Thank you for your help !
>
> Morgan.
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

bryan-basho
Administrator
In reply to this post by Morgan Segalis
On Tue, Jun 19, 2012 at 12:35 PM, Morgan Segalis <[hidden email]> wrote:
> Hi everyone !

Hi, Morgan.

> 1 - Are Link walking in Erlang fast & easy ?

Link-walking in the Erlang client is provided via the MapReduce
interface. Instead of a 'map' or 'reduce' phase, specify a 'link'
phase like:

    {link, Bucket, Tag, Keep}

Bucket is either a binary name of a bucket to match, or the atom '_',
which matches any bucket. Tag is either a binary tag to match, or the
atom '_', which matches any tag. Keep is a boolean with the same
meaning as in map and reduce phases.

http://wiki.basho.com/MapReduce.html#Erlang-Query-Syntax

> 2 - Is it possible to check fast if A -> B -> A (still in Erlang)? ('->' represents Links)

The fastest way to do this is to fetch both A and B and look through
the links they contain.

HTH,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

Morgan Segalis
In reply to this post by Eric Moritz
Hi Eric,

Thank you for your answer.

Yeah apparently, Links are not the Eldorado I though it was.
In my head it was a pointer to another record, and accessing it would be faster than regular requests…

I'm not really looking for more than 100 friends in one user's list really. (Except the rare user that will have 1000 or more friends).
When you say "a limited length", I'm not sure what you are referring… ?

Actually, more than checking if A and B are friends, I was looking for getting fast Information about friends, e.g. :

- retrieve all friend's information by Link Walking, like : A/B/name A/C/name A/D/name …

But it seems that I would not be as fast as I though ?

Thank you,

Morgan.

Le 19 juin 2012 à 19:48, Eric Moritz a écrit :

> Links may not be the most appropriate way to model a set of associated
> users.  They have a limited length and both the user data and the
> links have to be saved at the same time.
>
> What you can do is emulate a set using a separate bucket and secondary indexes.
>
> So basically the bucket would look like this:
>
> So if I'm a friend of yours,
>
> PUT /buckets/friends/keys/eric-morgan
> x-riak-index-friend_id_bin: morgan
> x-riak-index-user_id_bin: eric
> Content-Type: application/json
>
> {"user_id": "eric", "friend_id": "morgan"}
>
> Then you can fetch a set of my friends by doing a 2i query for the
> bucket friends and the index
> user_id.
>
> If you want to a set of all the people that have friended you, do a 2i
> query on friend_id with the value "morgan".
>
> You can quickly see if we're mutual friends by fetching:
>
> /buckets/friends/keys/eric-morgan and /buckets/friends/keys/morgan-eric.
>
> If both are found, then we're mutual friends.
>
> Eric.
>
> On Tue, Jun 19, 2012 at 12:35 PM, Morgan Segalis <[hidden email]> wrote:
>> Hi everyone !
>>
>> I'm looking for a NoSQL solution to get ride of MySQL.
>> In this particular issue, I think that Riak is one of the top of my choices in order to do what I'm looking for.
>>
>> I would like to store all my users into Riak.
>> I understand that Riak is document oriented, so that's great for getting every information really fast from one user.
>> One of the feature I'm really interested in, are Links.
>>
>> Links seems to be a great way for me to create association between users…
>>
>> When A adds B, I create a link from A to B.
>> When B accepts A's invitation, I create a link from B to A.
>>
>> If links are not both ways, e.g. A to B to A, message can not go through (B who didn't accept A should not get its message yet)
>>
>> My server is coded in Erlang, I have checked the Erlang riak client, and I didn't see something as simple as the HTTP use of Links in the demo.
>>
>> 1 - Are Link walking in Erlang fast & easy ?
>> 2 - Is it possible to check fast if A -> B -> A (still in Erlang)? ('->' represents Links)
>>
>> Thank you for your help !
>>
>> Morgan.
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

Morgan Segalis
In reply to this post by bryan-basho
Hi Bryan,

Thank you for your answer, that is what I was looking for at first !

Why would you think that fetch both user and look through the links is fastest than A -> B -> A ? Are links really that slow ?

Thanks,

Morgan.

Le 20 juin 2012 à 13:41, Bryan Fink a écrit :

> On Tue, Jun 19, 2012 at 12:35 PM, Morgan Segalis <[hidden email]> wrote:
>> Hi everyone !
>
> Hi, Morgan.
>
>> 1 - Are Link walking in Erlang fast & easy ?
>
> Link-walking in the Erlang client is provided via the MapReduce
> interface. Instead of a 'map' or 'reduce' phase, specify a 'link'
> phase like:
>
>    {link, Bucket, Tag, Keep}
>
> Bucket is either a binary name of a bucket to match, or the atom '_',
> which matches any bucket. Tag is either a binary tag to match, or the
> atom '_', which matches any tag. Keep is a boolean with the same
> meaning as in map and reduce phases.
>
> http://wiki.basho.com/MapReduce.html#Erlang-Query-Syntax
>
>> 2 - Is it possible to check fast if A -> B -> A (still in Erlang)? ('->' represents Links)
>
> The fastest way to do this is to fetch both A and B and look through
> the links they contain.
>
> HTH,
> Bryan


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

bryan-basho
Administrator
On Wed, Jun 20, 2012 at 8:00 AM, Morgan Segalis <[hidden email]> wrote:
> Why would you think that fetch both user and look through the links is fastest than A -> B -> A ? Are links really that slow ?

It's not that links are that slow, but that they're not that focused.
Unless you're using B's key as the tag on the link from A, and A's key
as the tag on the link from B, you'll have to follow all "person" or
"friend" (or whatever) links from A, and the same from B.  So, what
you'd actually be walking is A -> [any person/friend] -> A.  You could
put a map or reduce phase between the two link phases to filter out
everything but B, but my bet is that simply fetching A and B
individually will be much faster than setting up and tearing down the
MR framework.

YMMV, always benchmark real data, etc.

-Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Link walking with Erlang riak client

Morgan Segalis
Actually this is what I'm doing…

A as an unique ID, lets call it A-UID

A and B are stored in the bucket with keys A-UID and B-UID

The link in B for A will be named A-UID
And link in A for B will be named B-UID

Thanks,

Morgan.

Le 20 juin 2012 à 15:28, Bryan Fink a écrit :

> On Wed, Jun 20, 2012 at 8:00 AM, Morgan Segalis <[hidden email]> wrote:
>> Why would you think that fetch both user and look through the links is fastest than A -> B -> A ? Are links really that slow ?
>
> It's not that links are that slow, but that they're not that focused.
> Unless you're using B's key as the tag on the link from A, and A's key
> as the tag on the link from B, you'll have to follow all "person" or
> "friend" (or whatever) links from A, and the same from B.  So, what
> you'd actually be walking is A -> [any person/friend] -> A.  You could
> put a map or reduce phase between the two link phases to filter out
> everything but B, but my bet is that simply fetching A and B
> individually will be much faster than setting up and tearing down the
> MR framework.
>
> YMMV, always benchmark real data, etc.
>
> -Bryan


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com