Updating Riak values in batch

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Updating Riak values in batch

ricardo.ekm
Hi all,
I've a large dataset in Riak (about 20 million keys), storing JSON documents. I'd like to update those documents to remove a JSON attribute. What's the best way to approach this problem in Riak?

Thanks.

--
Ricardo Mayerhofer

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Updating Riak values in batch

Luke Bakken
Hi Ricardo,

If you know your keys in advance, you can fetch the keys in parallel,
update them, and write them back to Riak in parallel.

Other options include map/reduce jobs that iterate over all the keys,
but keep in mind that any key listing operation will be resource
intensive.
--
Luke Bakken
Engineer
[hidden email]


On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
<[hidden email]> wrote:

> Hi all,
> I've a large dataset in Riak (about 20 million keys), storing JSON
> documents. I'd like to update those documents to remove a JSON attribute.
> What's the best way to approach this problem in Riak?
>
> Thanks.
>
> --
> Ricardo Mayerhofer
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Updating Riak values in batch

ricardo.ekm
Hi Luke,
Thanks for your reply! Can I use map/reduce operation to update documents? Do you have any link with an example?

On Tue, Jun 14, 2016 at 2:44 PM, Luke Bakken <[hidden email]> wrote:
Hi Ricardo,

If you know your keys in advance, you can fetch the keys in parallel,
update them, and write them back to Riak in parallel.

Other options include map/reduce jobs that iterate over all the keys,
but keep in mind that any key listing operation will be resource
intensive.
--
Luke Bakken
Engineer
[hidden email]


On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
<[hidden email]> wrote:
> Hi all,
> I've a large dataset in Riak (about 20 million keys), storing JSON
> documents. I'd like to update those documents to remove a JSON attribute.
> What's the best way to approach this problem in Riak?
>
> Thanks.
>
> --
> Ricardo Mayerhofer
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>



--
Ricardo Mayerhofer

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Updating Riak values in batch

Luke Bakken
Hi Ricardo,

I don't have examples right at hand, here is a link to our advanced
map/reduce examples:

http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/advanced-mapreduce/

Our own Joe Caswell provided this example: http://stackoverflow.com/a/23633491

If you run into issues after following our examples, feel free to
share what you've done via a gist and we can continue to help out.

--
Luke Bakken
Engineer
[hidden email]

On Tue, Jun 14, 2016 at 10:51 AM, Ricardo Mayerhofer
<[hidden email]> wrote:

> Hi Luke,
> Thanks for your reply! Can I use map/reduce operation to update documents?
> Do you have any link with an example?
>
> On Tue, Jun 14, 2016 at 2:44 PM, Luke Bakken <[hidden email]> wrote:
>>
>> Hi Ricardo,
>>
>> If you know your keys in advance, you can fetch the keys in parallel,
>> update them, and write them back to Riak in parallel.
>>
>> Other options include map/reduce jobs that iterate over all the keys,
>> but keep in mind that any key listing operation will be resource
>> intensive.
>> --
>> Luke Bakken
>> Engineer
>> [hidden email]
>>
>>
>> On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
>> <[hidden email]> wrote:
>> > Hi all,
>> > I've a large dataset in Riak (about 20 million keys), storing JSON
>> > documents. I'd like to update those documents to remove a JSON
>> > attribute.
>> > What's the best way to approach this problem in Riak?
>> >
>> > Thanks.
>> >
>> > --
>> > Ricardo Mayerhofer
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > [hidden email]
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>
>
>
>
> --
> Ricardo Mayerhofer

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com