I'm trying to figure out whether there's an easy way to accomplish
something like a "recursive/looping map-reduce query" in Riak.
A use case for this is when the objects in the database form a linked
list, tree, or graph of some kind, and you want to look up a set of
objects whose depth from the initial key set isn't known in advance.
For a simple example, say you have a log of events that are stored as
a linked list of blocks of some maximum size, and you want to retrieve
the most recent n events that have some property (which for whatever
reason isn't practical to index in advance). It's easy to write a
function that looks up each block in order and stops when it has n
matches, but it would be nice to do this without O(n) roundtrips to
Riak and possibly having to discard a large fraction of the results on
the client side.
One hack solution would be to write a map-reduce query with some large
number of identical pairs of map and reduce phases, where once the
stopping condition is met, the subsequent phases will get a flag in
their input which makes them a cheap no-op.
Are there any other options short of running custom query-coordinating
code at the level of riak_pipe?
On Wed, Feb 22, 2012 at 8:23 PM, Michael Radford <[hidden email]> wrote:
> I'm trying to figure out whether there's an easy way to accomplish
> something like a "recursive/looping map-reduce query" in Riak.
> Are there any other options short of running custom query-coordinating
> code at the level of riak_pipe?
Hi, Mike. There are no direct options for open-ended recursive input
in MR at this time. Pipe does provide a very limited ability to do
so, but there is no external way to access Pipe directly at the