bucket props question

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

bucket props question

Bryce Verdier
Hi All,

Is it at all possible to have default bucket props be based on the
bucket names? I ask because I'm trying to use Riak to store key/value
data and use the buckets to separate the data based on day and for a
couple of different projects, for example:

buckets/project-year-month-day/key

The issue becomes I have one project that needs 'allow_mult' to true,
and another that needs 'last_write_wins' set to true (requiring that
'allow_mult' be false). So using the default_buckets_props will only get
me half way there.

Does anyone know of a way I can do this? The only solution I can think
of off the top of my head is to create a cron job that creates the
bucket sooner than when the clients start writing to it.

Thank you in advance,

Bryce

PS -- can anyone tell me what:
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]}}
does?

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: bucket props question

Tom Santero
Hi Bryce,

You generally want to avoid creating too many buckets with custom bucket properties, as this gets stored in Riak's ring data. A large number of custom buckets will degrade cluster performance. 

Is there any reason why you don't just create two buckets, with the desired LWW/allow_mult=true settings and then prepend the timestamp to your key? 

Tom 


On Wed, Dec 11, 2013 at 1:47 PM, Bryce Verdier <[hidden email]> wrote:
Hi All,

Is it at all possible to have default bucket props be based on the bucket names? I ask because I'm trying to use Riak to store key/value data and use the buckets to separate the data based on day and for a couple of different projects, for example:

buckets/project-year-month-day/key

The issue becomes I have one project that needs 'allow_mult' to true, and another that needs 'last_write_wins' set to true (requiring that 'allow_mult' be false). So using the default_buckets_props will only get me half way there.

Does anyone know of a way I can do this? The only solution I can think of off the top of my head is to create a cron job that creates the bucket sooner than when the clients start writing to it.

Thank you in advance,

Bryce

PS -- can anyone tell me what:
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]}}
does?

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: bucket props question

Jordan West-3
Hi Bryce,

Unfortunately Riak 2.0 final is not yet available but I would be curious to know if the upcoming Bucket Types [1] [2] feature would help you model your problem. You could create a Bucket Type for your allow_mult=true projects and another for lww=true. so you would have something like (extending the notation you used in your email): 

* bucket_types/allowsibs/buckets/project-year-month-day/keys/key
* bucket_types/nosibs/buckets/project-year-month-day/keys/key

where "allowsibs" and "nosibs" are the two bucket types, respectively.

Cheers,

Jordan



On Wed, Dec 11, 2013 at 11:45 AM, Tom Santero <[hidden email]> wrote:
Hi Bryce,

You generally want to avoid creating too many buckets with custom bucket properties, as this gets stored in Riak's ring data. A large number of custom buckets will degrade cluster performance. 

Is there any reason why you don't just create two buckets, with the desired LWW/allow_mult=true settings and then prepend the timestamp to your key? 

Tom 


On Wed, Dec 11, 2013 at 1:47 PM, Bryce Verdier <[hidden email]> wrote:
Hi All,

Is it at all possible to have default bucket props be based on the bucket names? I ask because I'm trying to use Riak to store key/value data and use the buckets to separate the data based on day and for a couple of different projects, for example:

buckets/project-year-month-day/key

The issue becomes I have one project that needs 'allow_mult' to true, and another that needs 'last_write_wins' set to true (requiring that 'allow_mult' be false). So using the default_buckets_props will only get me half way there.

Does anyone know of a way I can do this? The only solution I can think of off the top of my head is to create a cron job that creates the bucket sooner than when the clients start writing to it.

Thank you in advance,

Bryce

PS -- can anyone tell me what:
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]}}
does?

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: bucket props question

Jeremiah Peschka
Based on my understanding of the Bucket Types feature - yes, this feature would solve the problem.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Wed, Dec 11, 2013 at 12:39 PM, Jordan West <[hidden email]> wrote:
Hi Bryce,

Unfortunately Riak 2.0 final is not yet available but I would be curious to know if the upcoming Bucket Types [1] [2] feature would help you model your problem. You could create a Bucket Type for your allow_mult=true projects and another for lww=true. so you would have something like (extending the notation you used in your email): 

* bucket_types/allowsibs/buckets/project-year-month-day/keys/key
* bucket_types/nosibs/buckets/project-year-month-day/keys/key

where "allowsibs" and "nosibs" are the two bucket types, respectively.

Cheers,

Jordan



On Wed, Dec 11, 2013 at 11:45 AM, Tom Santero <[hidden email]> wrote:
Hi Bryce,

You generally want to avoid creating too many buckets with custom bucket properties, as this gets stored in Riak's ring data. A large number of custom buckets will degrade cluster performance. 

Is there any reason why you don't just create two buckets, with the desired LWW/allow_mult=true settings and then prepend the timestamp to your key? 

Tom 


On Wed, Dec 11, 2013 at 1:47 PM, Bryce Verdier <[hidden email]> wrote:
Hi All,

Is it at all possible to have default bucket props be based on the bucket names? I ask because I'm trying to use Riak to store key/value data and use the buckets to separate the data based on day and for a couple of different projects, for example:

buckets/project-year-month-day/key

The issue becomes I have one project that needs 'allow_mult' to true, and another that needs 'last_write_wins' set to true (requiring that 'allow_mult' be false). So using the default_buckets_props will only get me half way there.

Does anyone know of a way I can do this? The only solution I can think of off the top of my head is to create a cron job that creates the bucket sooner than when the clients start writing to it.

Thank you in advance,

Bryce

PS -- can anyone tell me what:
{multi_backend_prefix_list, [{<<"0b:">>, be_blocks}]}}
does?

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com