riak_core application design questions

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

riak_core application design questions

Chris Hicks-3
Hello everyone,

I'm in the beginning phases of building a riak_core application and am curious as to how others might recommend I set up my application. Every single bit of my application will be computation only, that is that all storage and data will be managed by Riak. Basically, a client sends a command to my riak_core application, it pings Riak to get the data, does something with it and then saves the results back to Riak before letting the user know what happened. If I planned on losing a few commands which were in flight when a node goes down this would be very simple, but I would like to avoid that (the is kinda the point of all this, after all). However, sending the command off to, say, 3 nodes to process means that each of my nodes will then ping Riak (if I need 2 responses per query suddenly that means I hit Riak nodes a total of 6 times), process, and then all try to save the data back to Riak (again possibly hitting 6+ nodes to save the data). Is that wasteful? Is there a better way to approach this? How can I make sure that the results are consistent with multiple nodes doing the computation?

Would it better, if it is even possible, to have one node doing the computation with another node holding a copy of the command as a fail-over? Once the 'primary' computational node finishes it's work and the data is saved the secondary one could be informed and just drop the command since it isn't necessary anymore. I'm still trying to grok all the implications of the way riak_core does things so any advice would be greatly appreciated.

Chris

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: riak_core application design questions

bryan-basho
Administrator
On Wed, Sep 26, 2012 at 4:38 PM, Chris Hicks
<[hidden email]> wrote:
> However, sending
> the command off to, say, 3 nodes to process means that each of my nodes will
> then ping Riak (if I need 2 responses per query suddenly that means I hit
> Riak nodes a total of 6 times), process, and then all try to save the data
> back to Riak (again possibly hitting 6+ nodes to save the data). Is that
> wasteful?

Hi, Chris. Yes, there is waste here. But, as many implementers in this
space have pointed out[1], redundant work can make distributed queries
faster (as long as you have the excess processing power available).

[1] http://www.bailis.org/blog/doing-redundant-work-to-speed-up-distributed-queries/

In fact, this is part of the reason that Riak hits N nodes for each
read/write: it's a bet that R of them will reply faster than the rest,
and thus you don't have to wait on the slowest.

> Would it better, if it is even possible, to have one node doing the
> computation with another node holding a copy of the command as a fail-over?
> Once the 'primary' computational node finishes it's work and the data is
> saved the secondary one could be informed and just drop the command since it
> isn't necessary anymore. I'm still trying to grok all the implications of
> the way riak_core does things so any advice would be greatly appreciated.

This is still "wasteful" in the sense that you have to tell a node
some information that, in the happy-path case, it's never going to
care about. The hard part here will be deciding at what point the
"first" node failed to complete the computation, and failover to the
next. Waiting to little time means you may have the same waste as your
earlier setup, but waiting too much time means your latency will
skyrocket.

However, this is still a common setup. Some dynamo implementations
even do a similar thing where they first ask the node they expect to
reply the fastest, and then only ask other nodes if that first one
does not reply fast enough. It's all in the accuracy of the modeling
and prediction at that point, as to whether this makes things faster
or slower.

Riak Pipe, Riak's internal distributed processing system for it's
scatter-gather query system, takes this approach as well. It sends an
input to the first node of a preflist, and then falls back to others
if needed. (Though its current use case also gets to assume that the
node that started the query stays up for the entire duration of the
query, so there is a point that can tell when an input has been lost.)

HTH,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com