Riak mapreduce error

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Riak mapreduce error

raghuveer sj
Hi Team,

I am trying to run mapreduce in erlang.

curl -XPUT http://localhost:8098/buckets/training/keys/foo -H 'Content-Type: text/plain' -d 'caremad data goes here'
curl -XPUT http://localhost:8098/buckets/training/keys/bar -H 'Content-Type: text/plain' -d 'caremad caremad caremad caremad'
curl -XPUT http://localhost:8098/buckets/training/keys/baz -H 'Content-Type: text/plain' -d 'nothing to see here'
curl -XPUT http://localhost:8098/buckets/training/keys/bam -H 'Content-Type: text/plain' -d 'caremad caremad caremad'

Running in erlang shell :

ReFun = fun(O, _, Re) -> case re:run(riak_object:get_value(O), Re, [global]) of
    {match, Matches} -> [{riak_object:key(O), length(Matches)}];
    nomatch -> [{riak_object:key(O), 0}]
end end.

code:which(riakc_pb_socket).
"./ebin/riakc_pb_socket.beam"

{ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.36.0>}

riakc_pb_socket:ping(Pid).
pong

{ok, Re} = re:compile("caremad").
{ok,{re_pattern,0,0,0,
                <<69,82,67,80,85,0,0,0,0,0,0,0,81,0,0,0,255,255,255,255,
                  255,255,...>>}}

{ok, Riak} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.42.0>}

riakc_pb_socket:mapred_bucket(Riak, <<"training">>, [{map, {qfun, ReFun}, Re, true}]).
* 1: variable 'ReFun' is unbound

Trying to run the famous erlang sample program sample. I am stuck at this error. Kindly help me out.

Regards,
Raghuveer

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Riak mapreduce error

Magnus Kessler
On 3 February 2017 at 18:31, raghuveer sj <[hidden email]> wrote:
Hi Team,

I am trying to run mapreduce in erlang.

curl -XPUT http://localhost:8098/buckets/training/keys/foo -H 'Content-Type: text/plain' -d 'caremad data goes here'
curl -XPUT http://localhost:8098/buckets/training/keys/bar -H 'Content-Type: text/plain' -d 'caremad caremad caremad caremad'
curl -XPUT http://localhost:8098/buckets/training/keys/baz -H 'Content-Type: text/plain' -d 'nothing to see here'
curl -XPUT http://localhost:8098/buckets/training/keys/bam -H 'Content-Type: text/plain' -d 'caremad caremad caremad'

Running in erlang shell :

ReFun = fun(O, _, Re) -> case re:run(riak_object:get_value(O), Re, [global]) of
    {match, Matches} -> [{riak_object:key(O), length(Matches)}];
    nomatch -> [{riak_object:key(O), 0}]
end end.

code:which(riakc_pb_socket).
"./ebin/riakc_pb_socket.beam"

{ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.36.0>}

riakc_pb_socket:ping(Pid).
pong

{ok, Re} = re:compile("caremad").
{ok,{re_pattern,0,0,0,
                <<69,82,67,80,85,0,0,0,0,0,0,0,81,0,0,0,255,255,255,255,
                  255,255,...>>}}

{ok, Riak} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.42.0>}

riakc_pb_socket:mapred_bucket(Riak, <<"training">>, [{map, {qfun, ReFun}, Re, true}]).
* 1: variable 'ReFun' is unbound

Trying to run the famous erlang sample program sample. I am stuck at this error. Kindly help me out.

Regards,
Raghuveer


Hi Raghuveer,

I have run the steps you provided, and found that they work fine for me. Can you let me know which version of Riak you are running this against, and which version of Erlang is used on the client side? Has the riak-erlang-client been compiled with the same Erlang version?

Kind Regards,

Magnus

--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Riak mapreduce error

raghuveer sj
Hi Magnus,

Previously i had used developer branch as master build was failing. Now with the latest changes i see :

10> riakc_pb_socket:mapred_bucket(Riak, <<"training">>, [{map, {qfun, ReFun}, Re, true}]).
{error,<<"{\"phase\":0,\"error\":\"{badfun,#Fun<erl_eval.18.50752066>}\",\"input\":\"{ok,{r_object,<<\\\"training\\\">>,<<\\\"bar\\\">>"...>>}

Please help me out.

Regards,
Raghuveer

On Mon, Feb 6, 2017 at 9:18 PM, Magnus Kessler <[hidden email]> wrote:
On 3 February 2017 at 18:31, raghuveer sj <[hidden email]> wrote:
Hi Team,

I am trying to run mapreduce in erlang.

curl -XPUT http://localhost:8098/buckets/training/keys/foo -H 'Content-Type: text/plain' -d 'caremad data goes here'
curl -XPUT http://localhost:8098/buckets/training/keys/bar -H 'Content-Type: text/plain' -d 'caremad caremad caremad caremad'
curl -XPUT http://localhost:8098/buckets/training/keys/baz -H 'Content-Type: text/plain' -d 'nothing to see here'
curl -XPUT http://localhost:8098/buckets/training/keys/bam -H 'Content-Type: text/plain' -d 'caremad caremad caremad'

Running in erlang shell :

ReFun = fun(O, _, Re) -> case re:run(riak_object:get_value(O), Re, [global]) of
    {match, Matches} -> [{riak_object:key(O), length(Matches)}];
    nomatch -> [{riak_object:key(O), 0}]
end end.

code:which(riakc_pb_socket).
"./ebin/riakc_pb_socket.beam"

{ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.36.0>}

riakc_pb_socket:ping(Pid).
pong

{ok, Re} = re:compile("caremad").
{ok,{re_pattern,0,0,0,
                <<69,82,67,80,85,0,0,0,0,0,0,0,81,0,0,0,255,255,255,255,
                  255,255,...>>}}

{ok, Riak} = riakc_pb_socket:start_link("127.0.0.1", 8087).
{ok,<0.42.0>}

riakc_pb_socket:mapred_bucket(Riak, <<"training">>, [{map, {qfun, ReFun}, Re, true}]).
* 1: variable 'ReFun' is unbound

Trying to run the famous erlang sample program sample. I am stuck at this error. Kindly help me out.

Regards,
Raghuveer


Hi Raghuveer,

I have run the steps you provided, and found that they work fine for me. Can you let me know which version of Riak you are running this against, and which version of Erlang is used on the client side? Has the riak-erlang-client been compiled with the same Erlang version?

Kind Regards,

Magnus

--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Riak mapreduce error

Magnus Kessler
On 7 February 2017 at 03:50, raghuveer sj <[hidden email]> wrote:
Hi Magnus,

Previously i had used developer branch as master build was failing. Now with the latest changes i see :

10> riakc_pb_socket:mapred_bucket(Riak, <<"training">>, [{map, {qfun, ReFun}, Re, true}]).
{error,<<"{\"phase\":0,\"error\":\"{badfun,#Fun<erl_eval.18.50752066>}\",\"input\":\"{ok,{r_object,<<\\\"training\\\">>,<<\\\"bar\\\">>"...>>}

Please help me out.

Regards,
Raghuveer



Hi Raghuveer,

I have been able to reproduce the error message you are seeing after I built riak-erlang-client with OTP-19. It appears that there are some incompatibilities between newer OTP versions and the map-reduce code. I haven't yet had the time to dig deeper, but suspect that changes in Erlang's string handling may play a role.

Please try to re-compile the Erlang client with OTP-16, and let me know if you can successfully run the example code you posted earlier under OTP-16 (e.g. start Erlang with "$(riak ertspath)/erl").

Kind Regards,

Magnus 


--
Magnus Kessler
Client Services Engineer
Basho Technologies Limited

Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Loading...