odd error on mapreduce

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

odd error on mapreduce

Giancarlo Frison
Hi all,

I'm performing a mapreduce call for retrieving objects with a peculiar tag link:

{"inputs": "minos","query": [{"map": {"language": "javascript","source": "function(v){ if(v.values[0].metadata.Links!=null && v.values[0].metadata.Links[0][0]==\"eclass-com_chelab_core_minos\" &&v.values[0].metadata.Links[0][1]==\"State\" &&v.values[0].metadata.Links[0][2]==\"iof\")  {return [v.key]; } else return [];}"}}]}

I have an object stored in binary format(protobuf) that the body is (json format):

{
    "name": "2login",
    "type": "start",
    "transitions": [
        {
            "name": "logon",
            "nextState": "logged",
            "guard": "request.user!=null && request.password!=null",
            "activity": {
                "name": "call-login",
                "serviceName": "http://apigeoname",
                "condition": "this.password==request.password"
            }
        }
    ]
}

and the mapreduce works fine.

IF I change JUST 'serviceName' element to 'http://apigeonames' adding the last char 's' I encounter and error in the mapreduce above. 
the error log is:



Error JSON encoding arguments: [{struct,
                                 [{<<"bucket">>,<<"minos">>},
                                  {<<"key">>,<<"2login">>},
                                  {<<"vclock">>,
                                   <<"a85hYGBgzGDKBVIsLNdPbMpgSmTMY2XYIXvmOF8WAA==">>},
                                  {<<"values">>,
                                   [{struct,
                                     [{<<"metadata">>,
                                       {struct,
                                        [{<<"Links">>,
                                          [[<<"eclass-com_chelab_core_minos">>,
                                            <<"State">>,<<"iof">>]]},
                                         {<<"X-Riak-VTag">>,
                                          <<"1uFeJm9wytHzl66Pxjsunx">>},
                                         {<<"content-type">>,
                                          <<"application/octet-stream">>},
                                         {<<"X-Riak-Last-Modified">>,
                                          <<"Fri, 26 Aug 2011 13:38:00 GMT">>}]}},
                                      {<<"data">>,
                                       <<26,128,1,10,5,108,111,103,111,
                                         110,18,6,108,111,103,103,101,
                                         100,26,44,114,101,113,117,101,
                                         115,116,46,117,115,101,114,33,
                                         61,110,117,108,108,32,38,38,32,
                                         114,101,113,117,101,115,116,46,
                                         112,97,115,115,119,111,114,100,
                                         33,61,110,117,108,108,34,65,18,
                                         18,104,116,116,112,58,47,47,97,
                                         112,105,103,101,111,110,97,109,
                                         101,115,34,10,99,97,108,108,45,
                                         108,111,103,105,110,42,31,116,
                                         104,105,115,46,112,97,115,115,
                                         119,111,114,100,61,61,114,101,
                                         113,117,101,115,116,46,112,97,
                                         115,115,119,111,114,100,42,6,
                                         50,108,111,103,105,110,64,6>>}]}]}]},
                                undefined,none]

{error,
    {no_candidate_nodes,exhausted_preflist,
        [{riak_kv_mapred_planner,claim_keys,3},
         {riak_kv_map_phase,handle_info,2},
         {luke_phase,handle_info,3},
         {gen_fsm,handle_msg,7},
         {proc_lib,init_p_do_apply,3}],
        [{{5680,'[hidden email]'},
          [{pid,<0.12932.0>},
           {keys,
               {{730750818665451459101842416358141509827966271488,
                 '[hidden email]'},
                [{riak_kv_map_input,
                     {<<"minos">>,<<"2login">>},
                     [{name,<<"minos">>},
                      {n_val,3},
                      {allow_mult,false},
                      {last_write_wins,false},
                      {precommit,[]},
                      {postcommit,[]},
                      {chash_keyfun,{riak_core_util,chash_std_keyfun}},
                      {linkfun,
                          {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},
                      {old_vclock,86400},
                      {young_vclock,20},
                      {big_vclock,50},
                      {small_vclock,10},
                      {r,quorum},
                      {w,quorum},
                      {dw,quorum},
                      {rw,quorum}],
                     undefined,
                     [{730750818665451459101842416358141509827966271488,
                       '[hidden email]'}]}]}}]},
         {<0.12932.0>,{5680,'[hidden email]'}}]}}


have you any idea about this problem?

Thanks,


Giancarlo Frison


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: odd error on mapreduce

Giancarlo Frison
If i serialize the body as Json I catch errors any more instead of protobuf object representation (application/octet-stream). 
Would be an mapreduce bug?

Is it possibile to skirt any body parsing during the mapreduce? Just for evaluating headers values?

Thanks,

Giancarlo Frison



On Thu, Aug 25, 2011 at 3:48 PM, Giancarlo Frison <[hidden email]> wrote:
Hi all,

I'm performing a mapreduce call for retrieving objects with a peculiar tag link:

{"inputs": "minos","query": [{"map": {"language": "javascript","source": "function(v){ if(v.values[0].metadata.Links!=null && v.values[0].metadata.Links[0][0]==\"eclass-com_chelab_core_minos\" &&v.values[0].metadata.Links[0][1]==\"State\" &&v.values[0].metadata.Links[0][2]==\"iof\")  {return [v.key]; } else return [];}"}}]}

I have an object stored in binary format(protobuf) that the body is (json format):

{
    "name": "2login",
    "type": "start",
    "transitions": [
        {
            "name": "logon",
            "nextState": "logged",
            "guard": "request.user!=null && request.password!=null",
            "activity": {
                "name": "call-login",
                "serviceName": "http://apigeoname",
                "condition": "this.password==request.password"
            }
        }
    ]
}

and the mapreduce works fine.

IF I change JUST 'serviceName' element to 'http://apigeonames' adding the last char 's' I encounter and error in the mapreduce above. 
the error log is:



Error JSON encoding arguments: [{struct,
                                 [{<<"bucket">>,<<"minos">>},
                                  {<<"key">>,<<"2login">>},
                                  {<<"vclock">>,
                                   <<"a85hYGBgzGDKBVIsLNdPbMpgSmTMY2XYIXvmOF8WAA==">>},
                                  {<<"values">>,
                                   [{struct,
                                     [{<<"metadata">>,
                                       {struct,
                                        [{<<"Links">>,
                                          [[<<"eclass-com_chelab_core_minos">>,
                                            <<"State">>,<<"iof">>]]},
                                         {<<"X-Riak-VTag">>,
                                          <<"1uFeJm9wytHzl66Pxjsunx">>},
                                         {<<"content-type">>,
                                          <<"application/octet-stream">>},
                                         {<<"X-Riak-Last-Modified">>,
                                          <<"Fri, 26 Aug 2011 13:38:00 GMT">>}]}},
                                      {<<"data">>,
                                       <<26,128,1,10,5,108,111,103,111,
                                         110,18,6,108,111,103,103,101,
                                         100,26,44,114,101,113,117,101,
                                         115,116,46,117,115,101,114,33,
                                         61,110,117,108,108,32,38,38,32,
                                         114,101,113,117,101,115,116,46,
                                         112,97,115,115,119,111,114,100,
                                         33,61,110,117,108,108,34,65,18,
                                         18,104,116,116,112,58,47,47,97,
                                         112,105,103,101,111,110,97,109,
                                         101,115,34,10,99,97,108,108,45,
                                         108,111,103,105,110,42,31,116,
                                         104,105,115,46,112,97,115,115,
                                         119,111,114,100,61,61,114,101,
                                         113,117,101,115,116,46,112,97,
                                         115,115,119,111,114,100,42,6,
                                         50,108,111,103,105,110,64,6>>}]}]}]},
                                undefined,none]

{error,
    {no_candidate_nodes,exhausted_preflist,
        [{riak_kv_mapred_planner,claim_keys,3},
         {riak_kv_map_phase,handle_info,2},
         {luke_phase,handle_info,3},
         {gen_fsm,handle_msg,7},
         {proc_lib,init_p_do_apply,3}],
        [{{5680,'[hidden email]'},
          [{pid,<0.12932.0>},
           {keys,
               {{730750818665451459101842416358141509827966271488,
                 '[hidden email]'},
                [{riak_kv_map_input,
                     {<<"minos">>,<<"2login">>},
                     [{name,<<"minos">>},
                      {n_val,3},
                      {allow_mult,false},
                      {last_write_wins,false},
                      {precommit,[]},
                      {postcommit,[]},
                      {chash_keyfun,{riak_core_util,chash_std_keyfun}},
                      {linkfun,
                          {modfun,riak_kv_wm_link_walker,mapreduce_linkfun}},
                      {old_vclock,86400},
                      {young_vclock,20},
                      {big_vclock,50},
                      {small_vclock,10},
                      {r,quorum},
                      {w,quorum},
                      {dw,quorum},
                      {rw,quorum}],
                     undefined,
                     [{730750818665451459101842416358141509827966271488,
                       '[hidden email]'}]}]}}]},
         {<0.12932.0>,{5680,'[hidden email]'}}]}}


have you any idea about this problem?

Thanks,


Giancarlo Frison



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: odd error on mapreduce

bryan-basho
Administrator
On Fri, Aug 26, 2011 at 6:32 AM, Giancarlo Frison <[hidden email]> wrote:
> If i serialize the body as Json I catch errors any more instead of protobuf
> object representation (application/octet-stream).
> Would be an mapreduce bug?
> Is it possibile to skirt any body parsing during the mapreduce? Just for
> evaluating headers values?

All objects passed through Javascript map or reduce phases must be
valid UTF-8-encoded JSON.  There is no way to change that at this
time.  Erlang map/reduce phases do not have such requirements.

The fact that your earlier ProtocolBuffer-encoded object made it
through a Javascript phase was probably a fluke, based on the encoding
containing nothing that was completely invalid UTF-8.  It probably
came through as a string, maybe with one or two odd characters
prepended.

Hope that helps,
Bryan

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: odd error on mapreduce

Giancarlo Frison
Thanks Bryan,

I skirt the problem using base64.

Giancarlo Frison



On Fri, Aug 26, 2011 at 1:11 PM, Bryan Fink <[hidden email]> wrote:
On Fri, Aug 26, 2011 at 6:32 AM, Giancarlo Frison <[hidden email]> wrote:
> If i serialize the body as Json I catch errors any more instead of protobuf
> object representation (application/octet-stream).
> Would be an mapreduce bug?
> Is it possibile to skirt any body parsing during the mapreduce? Just for
> evaluating headers values?

All objects passed through Javascript map or reduce phases must be
valid UTF-8-encoded JSON.  There is no way to change that at this
time.  Erlang map/reduce phases do not have such requirements.

The fact that your earlier ProtocolBuffer-encoded object made it
through a Javascript phase was probably a fluke, based on the encoding
containing nothing that was completely invalid UTF-8.  It probably
came through as a string, maybe with one or two odd characters
prepended.

Hope that helps,
Bryan


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com