Tuning parameters for reduce phases introduced in 1.0

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Tuning parameters for reduce phases introduced in 1.0

Elias Levy
Riak 1.0 introduced two reduce phase tuning parameters: reduce_phase_batch_size and reduce_phase_only_1. Both of these can be specified on a per phase basis using the arg parameter, passing them in as a map.  This appears to overload the arg parameter, which was already being used to pass static arguments to the phase functions.

Is there some mechanism to pass both a static function argument, say to the built-in Riak.reduceSort, while at the same time passing one of the new tuning parameters?

Elias

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Tuning parameters for reduce phases introduced in 1.0

bryan-basho
Administrator
On Wed, Oct 26, 2011 at 6:48 PM, Elias Levy <[hidden email]> wrote:
> Riak 1.0 introduced two reduce phase tuning
> parameters: reduce_phase_batch_size and reduce_phase_only_1. Both of these
> can be specified on a per phase basis using the arg parameter, passing them
> in as a map.  This appears to overload the arg parameter, which was already
> being used to pass static arguments to the phase functions.
> Is there some mechanism to pass both a static function argument, say to the
> built-in Riak.reduceSort, while at the same time passing one of the new
> tuning parameters?

Unfortunately, no, it's not possible to pass both Riak.reduceSort's
expected static argument and also one of the new reduce tuning
parameters in 1.0.

If the same reduce batch size is applicable to all of your queries,
then you can still use the global tuning parameter
mapred_reduce_phase_batch_size in the riak_kv section of your
app.config.

Riak.reduceSort is a very simple function, so it should also be easy
to provide a custom version of it that looks for the function within a
map.

We'll also be re-evaluating the method for providing these new tuning
parameters in a future version.

Hope that helps,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com