Key Expiry in Riak

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Key Expiry in Riak

Abhishek Kona
Hi

Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
Or is there any other way that I am missing.

This would be greatly useful for maintaining Session data in RIAK, as
expired session data can be auto cleared.

-Abhishek Kona

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Sean Cribbs-2
Abhishek,

Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak

If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):

{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"bitcask">>},
{multi_backend, [
        {<<"bitcask">>, riak_kv_bitcask_backend, []},
        {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
]},

The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/


P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb <search terms>".

On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:

> Hi
>
> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
> Or is there any other way that I am missing.
>
> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>
> -Abhishek Kona
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Abhishek Kona
Thanks for the help.
Will look up the knowledge base from next time before asking a question.
-Abhishek Kona

On 02/03/11 7:43 PM, Sean Cribbs wrote:

> Abhishek,
>
> Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>
> If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):
>
> {storage_backend, riak_kv_multi_backend},
> {multi_backend_default,<<"bitcask">>},
> {multi_backend, [
> {<<"bitcask">>, riak_kv_bitcask_backend, []},
>          {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
> ]},
>
> The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.
>
> Sean Cribbs<[hidden email]>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
>
> P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb<search terms>".
>
> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>
>> Hi
>>
>> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
>> Or is there any other way that I am missing.
>>
>> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>>
>> -Abhishek Kona
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Sean Cribbs-2
Sorry, I wasn't clear on how you can select the "sessions" backend that you set up.  It's a simple PUT to the bucket URL, e.g.

curl -X PUT -H "content-type: application/json" -d '{"props":{"backend":"sessions"}}' http://localhost:8098/riak/sessions

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Mar 2, 2011, at 9:23 AM, Abhishek Kona wrote:

> Thanks for the help.
> Will look up the knowledge base from next time before asking a question.
> -Abhishek Kona
>
> On 02/03/11 7:43 PM, Sean Cribbs wrote:
>> Abhishek,
>>
>> Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>>
>> If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):
>>
>> {storage_backend, riak_kv_multi_backend},
>> {multi_backend_default,<<"bitcask">>},
>> {multi_backend, [
>> {<<"bitcask">>, riak_kv_bitcask_backend, []},
>>         {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
>> ]},
>>
>> The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.
>>
>> Sean Cribbs<[hidden email]>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>>
>>
>> P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb<search terms>".
>>
>> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>>
>>> Hi
>>>
>>> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
>>> Or is there any other way that I am missing.
>>>
>>> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>>>
>>> -Abhishek Kona
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Tristan Sloughter
In reply to this post by Sean Cribbs-2
Is this possible when using innodb? Or would I use innodb for my persistent data and a bitcask backend (like your sessions example) for data I want to expire?

Thanks,
Tristan

On Wed, Mar 2, 2011 at 8:13 AM, Sean Cribbs <[hidden email]> wrote:
Abhishek,

Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak

If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):

{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"bitcask">>},
{multi_backend, [
       {<<"bitcask">>, riak_kv_bitcask_backend, []},
       {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
]},

The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/


P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb <search terms>".

On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:

> Hi
>
> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
> Or is there any other way that I am missing.
>
> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>
> -Abhishek Kona
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Sean Cribbs-2
Innostore does not have an expiration option.  In general, you'll want to use bitcask unless your operational requirements specifically call for innostore.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.

On Mar 2, 2011, at 11:53 AM, Tristan Sloughter wrote:

Is this possible when using innodb? Or would I use innodb for my persistent data and a bitcask backend (like your sessions example) for data I want to expire?

Thanks,
Tristan

On Wed, Mar 2, 2011 at 8:13 AM, Sean Cribbs <[hidden email]> wrote:
Abhishek,

Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak

If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):

{storage_backend, riak_kv_multi_backend},
{multi_backend_default, <<"bitcask">>},
{multi_backend, [
       {<<"bitcask">>, riak_kv_bitcask_backend, []},
       {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
]},

The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/


P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb <search terms>".

On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:

> Hi
>
> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
> Or is there any other way that I am missing.
>
> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>
> -Abhishek Kona
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Jason J. W. Williams
Are there any plans to make expiry setable via an HTTP header?

-J

On Wed, Mar 2, 2011 at 9:57 AM, Sean Cribbs <[hidden email]> wrote:

> Innostore does not have an expiration option.  In general, you'll want to
> use bitcask unless your operational requirements specifically call for
> innostore.
> Sean Cribbs <[hidden email]>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
> On Mar 2, 2011, at 11:53 AM, Tristan Sloughter wrote:
>
> Is this possible when using innodb? Or would I use innodb for my persistent
> data and a bitcask backend (like your sessions example) for data I want to
> expire?
> Thanks,
> Tristan
>
> On Wed, Mar 2, 2011 at 8:13 AM, Sean Cribbs <[hidden email]> wrote:
>>
>> Abhishek,
>>
>> Mostly answered here:
>> https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>>
>> If you use multi_backend with several bitcask setups you can have both
>> expiring data and normal persistent data. Here's an example of what you
>> might put in app.config (inside the riak_kv section):
>>
>> {storage_backend, riak_kv_multi_backend},
>> {multi_backend_default, <<"bitcask">>},
>> {multi_backend, [
>>        {<<"bitcask">>, riak_kv_bitcask_backend, []},
>>        {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200},
>> {data_root, "data/sessions"}]}
>> ]},
>>
>> The key pieces there is expiry_secs = 7200 (2 hours), and the separate
>> root for the data files.
>>
>> Sean Cribbs <[hidden email]>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>>
>>
>> P.S. If you haven't seen the Knowledge Base yet, check it out:
>> https://help.basho.com/forums/270314-riak. You can also search the KB from
>> the #riak IRC channel using "!kb <search terms>".
>>
>> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>>
>> > Hi
>> >
>> > Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per
>> > key).
>> > Or is there any other way that I am missing.
>> >
>> > This would be greatly useful for maintaining Session data in RIAK, as
>> > expired session data can be auto cleared.
>> >
>> > -Abhishek Kona
>> >
>> > _______________________________________________
>> > riak-users mailing list
>> > [hidden email]
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Alexander Sicular
i know i remember seeing a filed issue on issues.basho.com regarding
key expiration via headers. the comments debated a mark and sweep
approach versus a delete on read. tried searching for it but i cant
find it via 'expire' 'expiration' or 'ttl'. maybe someone from basho
knows which issue number i'm referring to here?

-alexander

On Wed, Mar 2, 2011 at 12:22, Jason J. W. Williams
<[hidden email]> wrote:

> Are there any plans to make expiry setable via an HTTP header?
>
> -J
>
> On Wed, Mar 2, 2011 at 9:57 AM, Sean Cribbs <[hidden email]> wrote:
>> Innostore does not have an expiration option.  In general, you'll want to
>> use bitcask unless your operational requirements specifically call for
>> innostore.
>> Sean Cribbs <[hidden email]>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>> On Mar 2, 2011, at 11:53 AM, Tristan Sloughter wrote:
>>
>> Is this possible when using innodb? Or would I use innodb for my persistent
>> data and a bitcask backend (like your sessions example) for data I want to
>> expire?
>> Thanks,
>> Tristan
>>
>> On Wed, Mar 2, 2011 at 8:13 AM, Sean Cribbs <[hidden email]> wrote:
>>>
>>> Abhishek,
>>>
>>> Mostly answered here:
>>> https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>>>
>>> If you use multi_backend with several bitcask setups you can have both
>>> expiring data and normal persistent data. Here's an example of what you
>>> might put in app.config (inside the riak_kv section):
>>>
>>> {storage_backend, riak_kv_multi_backend},
>>> {multi_backend_default, <<"bitcask">>},
>>> {multi_backend, [
>>>        {<<"bitcask">>, riak_kv_bitcask_backend, []},
>>>        {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200},
>>> {data_root, "data/sessions"}]}
>>> ]},
>>>
>>> The key pieces there is expiry_secs = 7200 (2 hours), and the separate
>>> root for the data files.
>>>
>>> Sean Cribbs <[hidden email]>
>>> Developer Advocate
>>> Basho Technologies, Inc.
>>> http://basho.com/
>>>
>>>
>>> P.S. If you haven't seen the Knowledge Base yet, check it out:
>>> https://help.basho.com/forums/270314-riak. You can also search the KB from
>>> the #riak IRC channel using "!kb <search terms>".
>>>
>>> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>>>
>>> > Hi
>>> >
>>> > Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per
>>> > key).
>>> > Or is there any other way that I am missing.
>>> >
>>> > This would be greatly useful for maintaining Session data in RIAK, as
>>> > expired session data can be auto cleared.
>>> >
>>> > -Abhishek Kona
>>> >
>>> > _______________________________________________
>>> > riak-users mailing list
>>> > [hidden email]
>>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Anthony Molinaro
In reply to this post by Sean Cribbs-2
Hi,

  Is it possible to add an expiry_secs after the fact?  In that if I didn't
set one originally for a bitcask backend can I add one now, or will I have
to dump data if I do?

  Since my understanding is that this value is used when bitcasks are
compacted in order to remove old fields, I have a feeling you can set it
at any point, but I want to make sure.

Thanks,

-Anthony

On Wed, Mar 02, 2011 at 09:13:27AM -0500, Sean Cribbs wrote:

> Abhishek,
>
> Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>
> If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):
>
> {storage_backend, riak_kv_multi_backend},
> {multi_backend_default, <<"bitcask">>},
> {multi_backend, [
> {<<"bitcask">>, riak_kv_bitcask_backend, []},
>         {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
> ]},
>
> The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.
>
> Sean Cribbs <[hidden email]>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
>
> P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb <search terms>".
>
> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>
> > Hi
> >
> > Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
> > Or is there any other way that I am missing.
> >
> > This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
> >
> > -Abhishek Kona
> >
> > _______________________________________________
> > riak-users mailing list
> > [hidden email]
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

--
------------------------------------------------------------------------
Anthony Molinaro                           <[hidden email]>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Key Expiry in Riak

Sean Cribbs-2
Anthony,

Yes, you can set it at any point but your Riak nodes might need to be restarted.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Mar 2, 2011, at 3:50 PM, Anthony Molinaro wrote:

> Hi,
>
>  Is it possible to add an expiry_secs after the fact?  In that if I didn't
> set one originally for a bitcask backend can I add one now, or will I have
> to dump data if I do?
>
>  Since my understanding is that this value is used when bitcasks are
> compacted in order to remove old fields, I have a feeling you can set it
> at any point, but I want to make sure.
>
> Thanks,
>
> -Anthony
>
> On Wed, Mar 02, 2011 at 09:13:27AM -0500, Sean Cribbs wrote:
>> Abhishek,
>>
>> Mostly answered here: https://help.basho.com/entries/466512-how-can-i-automatically-expire-a-key-from-riak
>>
>> If you use multi_backend with several bitcask setups you can have both expiring data and normal persistent data. Here's an example of what you might put in app.config (inside the riak_kv section):
>>
>> {storage_backend, riak_kv_multi_backend},
>> {multi_backend_default, <<"bitcask">>},
>> {multi_backend, [
>> {<<"bitcask">>, riak_kv_bitcask_backend, []},
>>        {<<"sessions>>, riak_kv_bitcask_backend, [{expiry_secs, 7200}, {data_root, "data/sessions"}]}
>> ]},
>>
>> The key pieces there is expiry_secs = 7200 (2 hours), and the separate root for the data files.
>>
>> Sean Cribbs <[hidden email]>
>> Developer Advocate
>> Basho Technologies, Inc.
>> http://basho.com/
>>
>>
>> P.S. If you haven't seen the Knowledge Base yet, check it out: https://help.basho.com/forums/270314-riak. You can also search the KB from the #riak IRC channel using "!kb <search terms>".
>>
>> On Mar 2, 2011, at 8:21 AM, Abhishek Kona wrote:
>>
>>> Hi
>>>
>>> Is it possible to have a expiry (ttl) on RIAK keys per bucket (or per key).
>>> Or is there any other way that I am missing.
>>>
>>> This would be greatly useful for maintaining Session data in RIAK, as expired session data can be auto cleared.
>>>
>>> -Abhishek Kona
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> --
> ------------------------------------------------------------------------
> Anthony Molinaro                           <[hidden email]>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com