Reduce phase behavior clarification

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Reduce phase behavior clarification

Brian Rowe
Hello,

I see the following on the Basho wiki for Riak [1]:

  The important thing to understand is that the function defining the
reduce phase may be evaluated multiple times, and the input of later
evaluations will include the input of earlier evaluations.
  For example, a reduce phase may implement the set-union function. In
that case, the first set of inputs might be [1,2,2,3], and the output
would be [1,2,3]. When the phase receives more inputs, say [3,4,5],
the function will be called with the concatentation of the two lists:
[1,2,3,4,5].

This description isn't clear to me as the behavior I'm seeing is that
the _output_ of earlier evaluations is included as input to later
evaluations. Unfortunately, the example yields the same results
irrespective of whether it is the input or the output of the first
evaluation, so it doesn't clear anything up.

Thanks,
Brian


[1] http://wiki.basho.com/MapReduce.html

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Reduce phase behavior clarification

Bob Ippolito
It sounds like there is a typo, I would assume that it should be "the
input of later evaluations will include the output of earlier
evaluations". It's not really a reduction otherwise.

On Mon, Jul 25, 2011 at 3:03 PM, Brian Rowe <[hidden email]> wrote:

> Hello,
>
> I see the following on the Basho wiki for Riak [1]:
>
>  The important thing to understand is that the function defining the
> reduce phase may be evaluated multiple times, and the input of later
> evaluations will include the input of earlier evaluations.
>  For example, a reduce phase may implement the set-union function. In
> that case, the first set of inputs might be [1,2,2,3], and the output
> would be [1,2,3]. When the phase receives more inputs, say [3,4,5],
> the function will be called with the concatentation of the two lists:
> [1,2,3,4,5].
>
> This description isn't clear to me as the behavior I'm seeing is that
> the _output_ of earlier evaluations is included as input to later
> evaluations. Unfortunately, the example yields the same results
> irrespective of whether it is the input or the output of the first
> evaluation, so it doesn't clear anything up.
>
> Thanks,
> Brian
>
>
> [1] http://wiki.basho.com/MapReduce.html
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Reduce phase behavior clarification

Alexander Sicular
And wouldn't the concatenation be [1,2,3,3,4,5] ?

-alexander
Sent from my Verizon Wireless BlackBerry

-----Original Message-----
From: Bob Ippolito <[hidden email]>
Sender: [hidden email]
Date: Mon, 25 Jul 2011 15:13:36
To: Brian Rowe<[hidden email]>
Cc: riak-users<[hidden email]>
Subject: Re: Reduce phase behavior clarification

It sounds like there is a typo, I would assume that it should be "the
input of later evaluations will include the output of earlier
evaluations". It's not really a reduction otherwise.

On Mon, Jul 25, 2011 at 3:03 PM, Brian Rowe <[hidden email]> wrote:

> Hello,
>
> I see the following on the Basho wiki for Riak [1]:
>
>  The important thing to understand is that the function defining the
> reduce phase may be evaluated multiple times, and the input of later
> evaluations will include the input of earlier evaluations.
>  For example, a reduce phase may implement the set-union function. In
> that case, the first set of inputs might be [1,2,2,3], and the output
> would be [1,2,3]. When the phase receives more inputs, say [3,4,5],
> the function will be called with the concatentation of the two lists:
> [1,2,3,4,5].
>
> This description isn't clear to me as the behavior I'm seeing is that
> the _output_ of earlier evaluations is included as input to later
> evaluations. Unfortunately, the example yields the same results
> irrespective of whether it is the input or the output of the first
> evaluation, so it doesn't clear anything up.
>
> Thanks,
> Brian
>
>
> [1] http://wiki.basho.com/MapReduce.html
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Reduce phase behavior clarification

Sean Cribbs-2
In reply to this post by Bob Ippolito
Bob is right, it's a simple typo. Please send a pull request or open an issue on the tracker.  https://github.com/basho/riak_wiki/issues

On Mon, Jul 25, 2011 at 6:13 PM, Bob Ippolito <[hidden email]> wrote:
It sounds like there is a typo, I would assume that it should be "the
input of later evaluations will include the output of earlier
evaluations". It's not really a reduction otherwise.

On Mon, Jul 25, 2011 at 3:03 PM, Brian Rowe <[hidden email]> wrote:
> Hello,
>
> I see the following on the Basho wiki for Riak [1]:
>
>  The important thing to understand is that the function defining the
> reduce phase may be evaluated multiple times, and the input of later
> evaluations will include the input of earlier evaluations.
>  For example, a reduce phase may implement the set-union function. In
> that case, the first set of inputs might be [1,2,2,3], and the output
> would be [1,2,3]. When the phase receives more inputs, say [3,4,5],
> the function will be called with the concatentation of the two lists:
> [1,2,3,4,5].
>
> This description isn't clear to me as the behavior I'm seeing is that
> the _output_ of earlier evaluations is included as input to later
> evaluations. Unfortunately, the example yields the same results
> irrespective of whether it is the input or the output of the first
> evaluation, so it doesn't clear anything up.
>
> Thanks,
> Brian
>
>
> [1] http://wiki.basho.com/MapReduce.html
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



--
Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Reduce phase behavior clarification

Brian Rowe
https://github.com/basho/riak_wiki/pull/139


On Tue, Jul 26, 2011 at 9:35 AM, Sean Cribbs <[hidden email]> wrote:

> Bob is right, it's a simple typo. Please send a pull request or open an
> issue on the tracker.  https://github.com/basho/riak_wiki/issues
>
> On Mon, Jul 25, 2011 at 6:13 PM, Bob Ippolito <[hidden email]> wrote:
>>
>> It sounds like there is a typo, I would assume that it should be "the
>> input of later evaluations will include the output of earlier
>> evaluations". It's not really a reduction otherwise.
>>
>> On Mon, Jul 25, 2011 at 3:03 PM, Brian Rowe <[hidden email]> wrote:
>> > Hello,
>> >
>> > I see the following on the Basho wiki for Riak [1]:
>> >
>> >  The important thing to understand is that the function defining the
>> > reduce phase may be evaluated multiple times, and the input of later
>> > evaluations will include the input of earlier evaluations.
>> >  For example, a reduce phase may implement the set-union function. In
>> > that case, the first set of inputs might be [1,2,2,3], and the output
>> > would be [1,2,3]. When the phase receives more inputs, say [3,4,5],
>> > the function will be called with the concatentation of the two lists:
>> > [1,2,3,4,5].
>> >
>> > This description isn't clear to me as the behavior I'm seeing is that
>> > the _output_ of earlier evaluations is included as input to later
>> > evaluations. Unfortunately, the example yields the same results
>> > irrespective of whether it is the input or the output of the first
>> > evaluation, so it doesn't clear anything up.
>> >
>> > Thanks,
>> > Brian
>> >
>> >
>> > [1] http://wiki.basho.com/MapReduce.html
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > [hidden email]
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> --
> Sean Cribbs <[hidden email]>
> Developer Advocate
> Basho Technologies, Inc.
> http://www.basho.com/
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com