Does Riak support Range Queries over binary safe strings?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Does Riak support Range Queries over binary safe strings?

gbrits
First, hello all!

Coming from Redis, I love that you can just put any binary blob in Redis which is just treated as a string. This is possible because Redis strings are what they call 'binary safe'. This makes it possible to return slices of string-encoded binary data, which is super useful for bitset-operations, etc.

I'm investigating Riak and I like it a lot so far. Riak seems to have range queries (on values, as it seems I must make that distinction with column-stores), but I'm not sure if strings in Riak are "Binary safe" in the above sense. If not, is there another way to store binary data in Riak and still do range queries over them quickly?

To be exact: I want to do multi-key lookups in Riak, where each returned result should be of format: <key,slice(featureX,start,end)>

Thanks,
Geert-Jan
Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

Alexander Sicular
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

Jeremiah Peschka
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

gbrits
Wow, high speed on this list!

I wanted it for near realtime anyway so Map/reduce is out of the question. Thought somehow it could be done through Riak Search or directly on secondary indices instead of map/reduce. 
Guess not. Oh well, can't have it all. 

Thanks


2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]>
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]

> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

Jeremiah Peschka
Not a problem.

MapReduce across an entire keyspace is slow.

MapReduce when provided with a few bucket/key pairs is the same as a multi-get + processing.

You can combine 2i + MR to get quick processing of data. Although, at that point, you might as well just process your data on the client side. Especially if you're just pulling out a slice of bytes.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 4:13 PM, gbrits <[hidden email]> wrote:
Wow, high speed on this list!

I wanted it for near realtime anyway so Map/reduce is out of the question. Thought somehow it could be done through Riak Search or directly on secondary indices instead of map/reduce. 
Guess not. Oh well, can't have it all. 

Thanks


2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]>
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]

> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML



View this message in context: Re: Does Riak support Range Queries over binary safe strings?

Sent from the Riak Users mailing list archive at Nabble.com.

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

Alexander Sicular
To the extent you limit your sliced data via an m/r you will reap those savings on the wire when transferring back to the client

You can feed an m/r from riak search, 2i or enumerated keys thereby skipping a costly bucket scan. 


@siculars

Sent from my iRotaryPhone

On Jul 16, 2013, at 19:46, Jeremiah Peschka <[hidden email]> wrote:

Not a problem.

MapReduce across an entire keyspace is slow.

MapReduce when provided with a few bucket/key pairs is the same as a multi-get + processing.

You can combine 2i + MR to get quick processing of data. Although, at that point, you might as well just process your data on the client side. Especially if you're just pulling out a slice of bytes.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 4:13 PM, gbrits <[hidden email]> wrote:
Wow, high speed on this list!

I wanted it for near realtime anyway so Map/reduce is out of the question. Thought somehow it could be done through Riak Search or directly on secondary indices instead of map/reduce. 
Guess not. Oh well, can't have it all. 

Thanks


2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]>
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]

> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML



View this message in context: Re: Does Riak support Range Queries over binary safe strings?
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

gbrits
Sounds good to me. Just to confirm: the value which I want to get a slice from can be a bytearray encoded as a string? (base 255)?

Oh and almost forgot, I DO need to be able to parameterize the m/r (with param startSliceAt, endSliceAt + some other parameters that manage how aggregation of values is performed) . 
Being an absolute noob to m/r is this possible?




2013/7/17 Alexander Sicular [via Riak Users] <[hidden email]>
To the extent you limit your sliced data via an m/r you will reap those savings on the wire when transferring back to the client

You can feed an m/r from riak search, 2i or enumerated keys thereby skipping a costly bucket scan. 


@siculars

Sent from my iRotaryPhone

On Jul 16, 2013, at 19:46, Jeremiah Peschka <[hidden email]> wrote:

Not a problem.

MapReduce across an entire keyspace is slow.

MapReduce when provided with a few bucket/key pairs is the same as a multi-get + processing.

You can combine 2i + MR to get quick processing of data. Although, at that point, you might as well just process your data on the client side. Especially if you're just pulling out a slice of bytes.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 4:13 PM, gbrits <[hidden email]> wrote:
Wow, high speed on this list!

I wanted it for near realtime anyway so Map/reduce is out of the question. Thought somehow it could be done through Riak Search or directly on secondary indices instead of map/reduce. 
Guess not. Oh well, can't have it all. 

Thanks


2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]>
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]

> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML



View this message in context: Re: Does Riak support Range Queries over binary safe strings?
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028365.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Does Riak support Range Queries over binary safe strings?

Sean Cribbs-2

On Wed, Jul 17, 2013 at 5:31 AM, gbrits <[hidden email]> wrote:
Sounds good to me. Just to confirm: the value which I want to get a slice from can be a bytearray encoded as a string? (base 255)?


Bytes are bytes to Riak. Your value is completely opaque.
 
Oh and almost forgot, I DO need to be able to parameterize the m/r (with param startSliceAt, endSliceAt + some other parameters that manage how aggregation of values is performed) . 
Being an absolute noob to m/r is this possible?


Each phase of a MapReduce job can receive arbitrary additional arguments. I highly suggest you check out our tutorials on the matter: http://docs.basho.com/riak/latest/tutorials/querying/MapReduce/




2013/7/17 Alexander Sicular [via Riak Users] <[hidden email]>
To the extent you limit your sliced data via an m/r you will reap those savings on the wire when transferring back to the client

You can feed an m/r from riak search, 2i or enumerated keys thereby skipping a costly bucket scan. 


@siculars

Sent from my iRotaryPhone

On Jul 16, 2013, at 19:46, Jeremiah Peschka <[hidden email]> wrote:

Not a problem.

MapReduce across an entire keyspace is slow.

MapReduce when provided with a few bucket/key pairs is the same as a multi-get + processing.

You can combine 2i + MR to get quick processing of data. Although, at that point, you might as well just process your data on the client side. Especially if you're just pulling out a slice of bytes.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 4:13 PM, gbrits <[hidden email]> wrote:
Wow, high speed on this list!

I wanted it for near realtime anyway so Map/reduce is out of the question. Thought somehow it could be done through Riak Search or directly on secondary indices instead of map/reduce. 
Guess not. Oh well, can't have it all. 

Thanks


2013/7/17 Jeremiah Peschka [via Riak Users] <[hidden email]>
Following up on Alex's comments -

If you know which bytes you need to slice, you can store this in a secondary index. You can perform range queries across secondary indices (as well as keys).

As long as you're storing your data in a way that allows it to be read by either Erlang or JavaScript, you should be able to query over it in MapReduce. This is typically regarded as a Bad Idea™ since an MR query will need to scan all keys in a bucket (which effectively means scanning the entire cluster) and is best done as an infrequent activity to transform data.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Tue, Jul 16, 2013 at 3:45 PM, Alexander Sicular <[hidden email]> wrote:
I would say no. Riak is generally oblivious as to the content of your data. Any ranges or other method you would use to query needs to be explicitly indexed via riak search or secondary indexes. Once you have found your data you could operate over that data in a map reduce, but I can't speak to "binary safe" blob operations in either erlang or JavaScript although I'm inclined to say yes, you would be able to operate over it in m/r.

So searching for keys with certain data in the binblob is probably not gonna happen but once you have a key to feed an m/r you could get a slice of that value.

Make sense?
-Alexander

@siculars
http://siculars.posthaven.com

Sent from my iRotaryPhone

On Jul 16, 2013, at 18:17, gbrits <[hidden email]> wrote:

> First, hello all!
>
> Coming from Redis, I love that you can just put any binary blob in Redis
> which is just treated as a string. This is possible because Redis strings
> are what they call 'binary safe'. This makes it possible to return slices of
> string-encoded binary data, which is super useful for bitset-operations,
> etc.
>
> I'm investigating Riak and I like it a lot so far. Riak seems to have range
> queries (on values, as it seems I must make that distinction with
> column-stores), but I'm not sure if strings in Riak are "Binary safe" in the
> above sense. If not, is there another way to store binary data in Riak and
> still do range queries over them quickly?
>
> To be exact: I want to do multi-key lookups in Riak, where each returned
> result should be of format: <key,slice(featureX,start,end)>
>
> Thanks,
> Geert-Jan
>
>
>
> --
> View this message in context: http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356.html
> Sent from the Riak Users mailing list archive at Nabble.com.
>
> _______________________________________________
> riak-users mailing list
> [hidden email]

> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028359.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML



View this message in context: Re: Does Riak support Range Queries over binary safe strings?
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



If you reply to this email, your message will be added to the discussion below:
http://riak-users.197444.n3.nabble.com/Does-Riak-support-Range-Queries-over-binary-safe-strings-tp4028356p4028365.html
To unsubscribe from Does Riak support Range Queries over binary safe strings?, click here.
NAML


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




--
Sean Cribbs <[hidden email]>
Software Engineer
Basho Technologies, Inc.

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com