How many keys inside a bucket ?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How many keys inside a bucket ?

Alex Feng
Hi,

I am wondering if there are some best practice or recommendation for how many keys inside a single bucket?

Let's say I have some sensors reporting data every 5 seconds, I can put all these data under one single bucket, or I can dynamically generate a bucket every day.

My question is,   is there any performance difference for "insert", "query" etc. between them?  for example,  1 million records under one bucket and 10k records under 100 buckets ? 

Thank you.

Br,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How many keys inside a bucket ?

Alexander Sicular-2
Hi, you should consider using Riak TS for this use case.

-Alexander


@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Jan 27, 2017, at 01:54, Alex Feng <[hidden email]> wrote:
>
> Hi,
>
> I am wondering if there are some best practice or recommendation for how many keys inside a single bucket?
>
> Let's say I have some sensors reporting data every 5 seconds, I can put all these data under one single bucket, or I can dynamically generate a bucket every day.
>
> My question is,   is there any performance difference for "insert", "query" etc. between them?  for example,  1 million records under one bucket and 10k records under 100 buckets ?
>
> Thank you.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How many keys inside a bucket ?

Alex Feng
Hi Alexander,

Yes, I should consider the possibility of switching to Riak TS.
But I guess the question still valid, does it ?  Should I divide millions of keys to different buckets, does it make any difference in performance, memory, space ?


Br,
Alex

2017-01-27 2:50 GMT+01:00 Alexander Sicular <[hidden email]>:
Hi, you should consider using Riak TS for this use case.

-Alexander


@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Jan 27, 2017, at 01:54, Alex Feng <[hidden email]> wrote:
>
> Hi,
>
> I am wondering if there are some best practice or recommendation for how many keys inside a single bucket?
>
> Let's say I have some sensors reporting data every 5 seconds, I can put all these data under one single bucket, or I can dynamically generate a bucket every day.
>
> My question is,   is there any performance difference for "insert", "query" etc. between them?  for example,  1 million records under one bucket and 10k records under 100 buckets ?
>
> Thank you.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How many keys inside a bucket ?

Alexander Sicular-2
Hello Alex,

As long as each bucket does not have its own properties but rather shares one or a handful of bucket types you should be fine and it wouldn't make a difference. 

One way to record data temporally, aka in a time series fashion, from a data model perspective is via a pattern called deterministic materialized keys. You basically record the sensor name (known id) and a base time at some granularity into the key name and push all data captured in that time frame into that key. This works well if you expect to record data at regular intervals. If, however, your data is sparse you'll need some additional model to know when you've recorded data. 

-Alexander 


@siculars

Sent from my iRotaryPhone

On Jan 27, 2017, at 14:32, Alex Feng <[hidden email]> wrote:

Hi Alexander,

Yes, I should consider the possibility of switching to Riak TS.
But I guess the question still valid, does it ?  Should I divide millions of keys to different buckets, does it make any difference in performance, memory, space ?


Br,
Alex

2017-01-27 2:50 GMT+01:00 Alexander Sicular <[hidden email]>:
Hi, you should consider using Riak TS for this use case.

-Alexander


@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

> On Jan 27, 2017, at 01:54, Alex Feng <[hidden email]> wrote:
>
> Hi,
>
> I am wondering if there are some best practice or recommendation for how many keys inside a single bucket?
>
> Let's say I have some sensors reporting data every 5 seconds, I can put all these data under one single bucket, or I can dynamically generate a bucket every day.
>
> My question is,   is there any performance difference for "insert", "query" etc. between them?  for example,  1 million records under one bucket and 10k records under 100 buckets ?
>
> Thank you.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Loading...