Timeout during reduce phase

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Timeout during reduce phase

Matthew A. Brown
I'm running a map/reduce operation with fairly complex map and reduce
phases against about 2000 keys. Each key also has a (fairly small)
payload arg. Currently just testing it locally against a single Riak
node. I've started consistently seeing this error during the reduce
phase:

error:{case_clause,{error,timeout}} reducing:
(then a dump of a bunch of the data for the phase)

The map/reduce itself has a timeout of 10 minutes, but this is
happening after about 30 seconds. Any idea what could be causing this?

Thanks!
Mat

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Timeout during reduce phase

Reid Draper
Mat,

Are you sure you're using milliseconds[1] to specify the timeout?


Reid

On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown <[hidden email]> wrote:
I'm running a map/reduce operation with fairly complex map and reduce
phases against about 2000 keys. Each key also has a (fairly small)
payload arg. Currently just testing it locally against a single Riak
node. I've started consistently seeing this error during the reduce
phase:

error:{case_clause,{error,timeout}} reducing:
(then a dump of a bunch of the data for the phase)

The map/reduce itself has a timeout of 10 minutes, but this is
happening after about 30 seconds. Any idea what could be causing this?

Thanks!
Mat

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Timeout during reduce phase

Matthew A. Brown
Yes, I am. This does not seem to be a timeout of the full map/reduce
operation, as it does not cause the m/r to fail -- rather certain
portions of the reduce phase fail so I get back semi-reduced results.

On Mon, Dec 12, 2011 at 15:24, Reid Draper <[hidden email]> wrote:

> Mat,
>
> Are you sure you're using milliseconds[1] to specify the timeout?
>
> [1]: http://wiki.basho.com/MapReduce.html#HTTP-Query-Syntax
>
> Reid
>
> On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown <[hidden email]>
> wrote:
>>
>> I'm running a map/reduce operation with fairly complex map and reduce
>> phases against about 2000 keys. Each key also has a (fairly small)
>> payload arg. Currently just testing it locally against a single Riak
>> node. I've started consistently seeing this error during the reduce
>> phase:
>>
>> error:{case_clause,{error,timeout}} reducing:
>> (then a dump of a bunch of the data for the phase)
>>
>> The map/reduce itself has a timeout of 10 minutes, but this is
>> happening after about 30 seconds. Any idea what could be causing this?
>>
>> Thanks!
>> Mat
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Timeout during reduce phase

bryan-basho
Administrator
In reply to this post by Matthew A. Brown
On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown <[hidden email]> wrote:
> The map/reduce itself has a timeout of 10 minutes, but this is
> happening after about 30 seconds. Any idea what could be causing this?

Hi, Mat.  Is your reduce phase implemented in Javascript?  If so, it's
subject to a hardcoded five second timeout in erlang_js.  Each
Javascript call a MapReduce query makes must finish in under five
seconds, regardless of the timeout set for the whole query.  Phases
implemented in Erlang are not subject to this restriction.

HTH,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Timeout during reduce phase

Matthew A. Brown
Hi Bryan,

That sounds like the problem.

Thanks,
Mat

On Tue, Dec 13, 2011 at 07:30, Bryan Fink <[hidden email]> wrote:

> On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown <[hidden email]> wrote:
>> The map/reduce itself has a timeout of 10 minutes, but this is
>> happening after about 30 seconds. Any idea what could be causing this?
>
> Hi, Mat.  Is your reduce phase implemented in Javascript?  If so, it's
> subject to a hardcoded five second timeout in erlang_js.  Each
> Javascript call a MapReduce query makes must finish in under five
> seconds, regardless of the timeout set for the whole query.  Phases
> implemented in Erlang are not subject to this restriction.
>
> HTH,
> Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

SV: Timeout during reduce phase

Jens Rantil
In reply to this post by bryan-basho
Hi,

Sorry for bringing this up for attention at such late notice. However, shouldn't this be a configurable setting in riak?

Regards,
Jens

-----Ursprungligt meddelande-----
Från: [hidden email] [mailto:[hidden email]] För Bryan Fink
Skickat: den 13 december 2011 13:31
Till: Matthew A. Brown
Kopia: [hidden email]
Ämne: Re: Timeout during reduce phase

On Mon, Dec 12, 2011 at 1:12 PM, Matthew A. Brown <[hidden email]> wrote:
> The map/reduce itself has a timeout of 10 minutes, but this is
> happening after about 30 seconds. Any idea what could be causing this?

Hi, Mat.  Is your reduce phase implemented in Javascript?  If so, it's subject to a hardcoded five second timeout in erlang_js.  Each Javascript call a MapReduce query makes must finish in under five seconds, regardless of the timeout set for the whole query.  Phases implemented in Erlang are not subject to this restriction.

HTH,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com