link graph traversal

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

link graph traversal

Colin-2
Hi,

If I create a linked list of many items (unknown size) in a bucket,
for example, item1 links to item2, item2 links to item3, and so on, is
there a way to retrieve the complete set of items in a single
link-walking query or mapreduce query??

It looks like it only gathers the item direct childs and not the whole graph.

Thanks for your help,
Colin

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: link graph traversal

Alexander Sicular
Colin,

Link walking http calls are non exhaustive. So if you wanted to follow, for example, two levels of depth you would have to do something like:

http://localhost:8098/raw/bucket/key/_,_,_/_,_,_

Check here, http://blog.basho.com/2010/02/24/link-walking-by-example/, and here, https://wiki.basho.com/display/RIAK/REST+API#RESTAPI-Linkwalking, if you haven't already.

Otherwise you need to build your own recursive function. I'm not exactly sure what that would look like in Riak M/R.

-Alexander


On Apr 12, 2010, at 5:07 PM, Colin Surprenant wrote:

> Hi,
>
> If I create a linked list of many items (unknown size) in a bucket,
> for example, item1 links to item2, item2 links to item3, and so on, is
> there a way to retrieve the complete set of items in a single
> link-walking query or mapreduce query??
>
> It looks like it only gathers the item direct childs and not the whole graph.
>
> Thanks for your help,
> Colin
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: link graph traversal

Colin-2
Thanks for the info. This is what I understood from the doc, the
examples and the tests I did. I was wondering if there was something I
was missing to be able to traverse a complete graph using a simple
call.

I don't know how the mapreduce link query is actually executed in the
mapreduce engine, if its distributed (i think not) but, AFAIK a
function cannot be defined, only patterns. Only the map and reduce
queries can have functions. Unless we can find a way to daisychain
mapreduce jobs for each link step? Otherwise, traversal can be done at
the application level I guess, requiring new mapreduce for each "next"
link.

But it would be really nice to be able to run a mapreduce job over a
complete graph defined with links.

Thanks,
Colin


On Mon, Apr 12, 2010 at 5:18 PM, Alexander Sicular <[hidden email]> wrote:

> Colin,
>
> Link walking http calls are non exhaustive. So if you wanted to follow, for example, two levels of depth you would have to do something like:
>
> http://localhost:8098/raw/bucket/key/_,_,_/_,_,_
>
> Check here, http://blog.basho.com/2010/02/24/link-walking-by-example/, and here, https://wiki.basho.com/display/RIAK/REST+API#RESTAPI-Linkwalking, if you haven't already.
>
> Otherwise you need to build your own recursive function. I'm not exactly sure what that would look like in Riak M/R.
>
> -Alexander
>
>
> On Apr 12, 2010, at 5:07 PM, Colin Surprenant wrote:
>
>> Hi,
>>
>> If I create a linked list of many items (unknown size) in a bucket,
>> for example, item1 links to item2, item2 links to item3, and so on, is
>> there a way to retrieve the complete set of items in a single
>> link-walking query or mapreduce query??
>>
>> It looks like it only gathers the item direct childs and not the whole graph.
>>
>> Thanks for your help,
>> Colin
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: link graph traversal

John Lynch-2
Colin, I believe the link phase is just a prebuilt map phase, and the  
link data is available to your own MR jobs to do whatever you like with.

- John

Sent from my iPhone

On Apr 12, 2010, at 8:26 PM, Colin Surprenant <[hidden email]>  
wrote:

> Thanks for the info. This is what I understood from the doc, the
> examples and the tests I did. I was wondering if there was something I
> was missing to be able to traverse a complete graph using a simple
> call.
>
> I don't know how the mapreduce link query is actually executed in the
> mapreduce engine, if its distributed (i think not) but, AFAIK a
> function cannot be defined, only patterns. Only the map and reduce
> queries can have functions. Unless we can find a way to daisychain
> mapreduce jobs for each link step? Otherwise, traversal can be done at
> the application level I guess, requiring new mapreduce for each "next"
> link.
>
> But it would be really nice to be able to run a mapreduce job over a
> complete graph defined with links.
>
> Thanks,
> Colin
>
>
> On Mon, Apr 12, 2010 at 5:18 PM, Alexander Sicular  
> <[hidden email]> wrote:
>> Colin,
>>
>> Link walking http calls are non exhaustive. So if you wanted to  
>> follow, for example, two levels of depth you would have to do  
>> something like:
>>
>> http://localhost:8098/raw/bucket/key/_,_,_/_,_,_
>>
>> Check here, http://blog.basho.com/2010/02/24/link-walking-by- 
>> example/, and here, https://wiki.basho.com/display/RIAK/REST+API#RESTAPI-Linkwalking 
>> , if you haven't already.
>>
>> Otherwise you need to build your own recursive function. I'm not  
>> exactly sure what that would look like in Riak M/R.
>>
>> -Alexander
>>
>>
>> On Apr 12, 2010, at 5:07 PM, Colin Surprenant wrote:
>>
>>> Hi,
>>>
>>> If I create a linked list of many items (unknown size) in a bucket,
>>> for example, item1 links to item2, item2 links to item3, and so  
>>> on, is
>>> there a way to retrieve the complete set of items in a single
>>> link-walking query or mapreduce query??
>>>
>>> It looks like it only gathers the item direct childs and not the  
>>> whole graph.
>>>
>>> Thanks for your help,
>>> Colin
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com