Secondary indexes or Riak search ?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Secondary indexes or Riak search ?

Alex Feng
Hello Riak-users,

I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well. 
So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.

Many thanks in advance.

Br,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Jason Voegele
Hi Alex,

There is some info on this page that can help you decide:


See the sections titled "When to Use Secondary Indexes" and " When Not to Use Secondary Indexes".

Sent from my iPad

On Feb 2, 2017, at 4:43 AM, Alex Feng <[hidden email]> wrote:

Hello Riak-users,

I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well. 
So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.

Many thanks in advance.

Br,
Alex
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Russell Brown-4
In reply to this post by Alex Feng
It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.

Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
Secondary indexes are written atomically with the object they index.
Operationally they don’t require you run a JVM and Solr alongside your riak nodes.

You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.

Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.

I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.

Cheers

Russell

On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:

> Hello Riak-users,
>
> I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
>
> Many thanks in advance.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Alex Feng
Hi Russell,

It is really helpful, thank you a lot.
We are suffering from solr crash now, are considering to switch to 2i.

Br,
Alex

2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.

Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
Secondary indexes are written atomically with the object they index.
Operationally they don’t require you run a JVM and Solr alongside your riak nodes.

You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.

Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.

I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.

Cheers

Russell

On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:

> Hello Riak-users,
>
> I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
>
> Many thanks in advance.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Alex Feng
Hi Russell,

In your reply, you mentioned this,

>> There is also the feature that can return the actual riak objects for a $keys index search,
>>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)

If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.


Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?  

Many thanks in advance.

Br,
Alex

2017-02-06 19:02 GMT+08:00 Alex Feng <[hidden email]>:
Hi Russell,

It is really helpful, thank you a lot.
We are suffering from solr crash now, are considering to switch to 2i.

Br,
Alex

2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.

Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
Secondary indexes are written atomically with the object they index.
Operationally they don’t require you run a JVM and Solr alongside your riak nodes.

You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.

Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.

I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.

Cheers

Russell

On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:

> Hello Riak-users,
>
> I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
>
> Many thanks in advance.
>
> Br,
> Alex
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Russell Brown-4
If you look at the riak-erlang-client here https://github.com/basho/riak-erlang-client/blob/develop/src/riakc_pb_socket.erl#L1129 there is a client API call that was implemented for CS (riak s2, or whatever it is called, the large object store thingy that basho had) that will use the $keys index, but in the back end calls fold_objects rather than fold_keys on eleveldb, and so returns the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum read here, but a single vnode per-value only. I don’t know if other clients added this API, but it would not be hard to add to any client that supports 2i. Like I say, originally it was for riakCS, but it’s open source and part of the release for 4 years now, so hardly a secret.

Cheers

Russell

On 13 Feb 2017, at 06:18, Alex Feng <[hidden email]> wrote:

> Hi Russell,
>
> In your reply, you mentioned this,
>
> >> There is also the feature that can return the actual riak objects for a $keys index search,
> >>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)
>
> If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.
>
>
> Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?  
>
> Many thanks in advance.
>
> Br,
> Alex
>
> 2017-02-06 19:02 GMT+08:00 Alex Feng <[hidden email]>:
> Hi Russell,
>
> It is really helpful, thank you a lot.
> We are suffering from solr crash now, are considering to switch to 2i.
>
> Br,
> Alex
>
> 2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
> It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.
>
> Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
> You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
> Secondary indexes are written atomically with the object they index.
> Operationally they don’t require you run a JVM and Solr alongside your riak nodes.
>
> You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.
>
> Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.
>
> I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.
>
> Cheers
>
> Russell
>
> On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:
>
> > Hello Riak-users,
> >
> > I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> > So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
> >
> > Many thanks in advance.
> >
> > Br,
> > Alex
> > _______________________________________________
> > riak-users mailing list
> > [hidden email]
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Alex Feng
Hi Russell,

We have tried this, but it returns a reference(some number) only.  We have googled and haven't found any clue about how to use this API.
The API user guide doesn't say much about the "reference".  What is the next step with the returned reference ?


cs_bucket_fold(Pid::pid(), Bucket::bucket() | bucket_and_type(), Opts::cs_opts()) -> {ok, reference()} | {error, term()}

secret function, do not use, or I come to your house and keeel you.

cs_opt() = {timeout, timeout()} | {continuation, binary()} | {max_results, non_neg_integer() | all} | {start_key, binary()} | {start_incl, boolean()} | {end_key, binary()} | {end_incl, boolean()}


Br,
Alex

2017-02-13 16:34 GMT+08:00 Russell Brown <[hidden email]>:
If you look at the riak-erlang-client here https://github.com/basho/riak-erlang-client/blob/develop/src/riakc_pb_socket.erl#L1129 there is a client API call that was implemented for CS (riak s2, or whatever it is called, the large object store thingy that basho had) that will use the $keys index, but in the back end calls fold_objects rather than fold_keys on eleveldb, and so returns the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum read here, but a single vnode per-value only. I don’t know if other clients added this API, but it would not be hard to add to any client that supports 2i. Like I say, originally it was for riakCS, but it’s open source and part of the release for 4 years now, so hardly a secret.

Cheers

Russell

On 13 Feb 2017, at 06:18, Alex Feng <[hidden email]> wrote:

> Hi Russell,
>
> In your reply, you mentioned this,
>
> >> There is also the feature that can return the actual riak objects for a $keys index search,
> >>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)
>
> If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.
>
>
> Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?
>
> Many thanks in advance.
>
> Br,
> Alex
>
> 2017-02-06 19:02 GMT+08:00 Alex Feng <[hidden email]>:
> Hi Russell,
>
> It is really helpful, thank you a lot.
> We are suffering from solr crash now, are considering to switch to 2i.
>
> Br,
> Alex
>
> 2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
> It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.
>
> Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
> You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
> Secondary indexes are written atomically with the object they index.
> Operationally they don’t require you run a JVM and Solr alongside your riak nodes.
>
> You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.
>
> Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.
>
> I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.
>
> Cheers
>
> Russell
>
> On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:
>
> > Hello Riak-users,
> >
> > I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> > So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
> >
> > Many thanks in advance.
> >
> > Br,
> > Alex
> > _______________________________________________
> > riak-users mailing list
> > [hidden email]
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Russell Brown-4
Hi Alex,
A quick look at riak_test it seems that the `return_body` parameter is generally supported for both HTTP and PB now. I wasn’t aware of that.

This test exercises the API.

https://github.com/basho/riak_test/blob/develop/tests/verify_2i_returnbody.erl

The `ref` returned from the link I gave you is a way to correlate messages received with the query. If you have a look at the code in the test here

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L204

You can get an idea of the general pattern. I guess line 227

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L227

Is the pertinent one for you.

Cheers

Russell

On 13 Feb 2017, at 11:08, Alex Feng <[hidden email]> wrote:

> Hi Russell,
>
> We have tried this, but it returns a reference(some number) only.  We have googled and haven't found any clue about how to use this API.
> The API user guide doesn't say much about the "reference".  What is the next step with the returned reference ?
>
>
> cs_bucket_fold(Pid::pid(), Bucket::bucket() | bucket_and_type(), Opts::cs_opts()) -> {ok, reference()} | {error, term()}
> secret function, do not use, or I come to your house and keeel you.
>
> cs_opt() = {timeout, timeout()} | {continuation, binary()} | {max_results, non_neg_integer() | all} | {start_key, binary()} | {start_incl, boolean()} | {end_key, binary()} | {end_incl, boolean()}
>
>
> Br,
> Alex
>
> 2017-02-13 16:34 GMT+08:00 Russell Brown <[hidden email]>:
> If you look at the riak-erlang-client here https://github.com/basho/riak-erlang-client/blob/develop/src/riakc_pb_socket.erl#L1129 there is a client API call that was implemented for CS (riak s2, or whatever it is called, the large object store thingy that basho had) that will use the $keys index, but in the back end calls fold_objects rather than fold_keys on eleveldb, and so returns the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum read here, but a single vnode per-value only. I don’t know if other clients added this API, but it would not be hard to add to any client that supports 2i. Like I say, originally it was for riakCS, but it’s open source and part of the release for 4 years now, so hardly a secret.
>
> Cheers
>
> Russell
>
> On 13 Feb 2017, at 06:18, Alex Feng <[hidden email]> wrote:
>
> > Hi Russell,
> >
> > In your reply, you mentioned this,
> >
> > >> There is also the feature that can return the actual riak objects for a $keys index search,
> > >>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)
> >
> > If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.
> >
> >
> > Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?
> >
> > Many thanks in advance.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 19:02 GMT+08:00 Alex Feng <[hidden email]>:
> > Hi Russell,
> >
> > It is really helpful, thank you a lot.
> > We are suffering from solr crash now, are considering to switch to 2i.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
> > It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.
> >
> > Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
> > You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
> > Secondary indexes are written atomically with the object they index.
> > Operationally they don’t require you run a JVM and Solr alongside your riak nodes.
> >
> > You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.
> >
> > Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.
> >
> > I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.
> >
> > Cheers
> >
> > Russell
> >
> > On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:
> >
> > > Hello Riak-users,
> > >
> > > I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> > > So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
> > >
> > > Many thanks in advance.
> > >
> > > Br,
> > > Alex
> > > _______________________________________________
> > > riak-users mailing list
> > > [hidden email]
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
> >
>
>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

Alex Feng
Hi Russell,

We will look into the links, thank you for the great help.

Br,
Aelx

2017-02-13 20:05 GMT+08:00 Russell Brown <[hidden email]>:
Hi Alex,
A quick look at riak_test it seems that the `return_body` parameter is generally supported for both HTTP and PB now. I wasn’t aware of that.

This test exercises the API.

https://github.com/basho/riak_test/blob/develop/tests/verify_2i_returnbody.erl

The `ref` returned from the link I gave you is a way to correlate messages received with the query. If you have a look at the code in the test here

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L204

You can get an idea of the general pattern. I guess line 227

https://github.com/basho/riak_test/blob/develop/tests/secondary_index_tests.erl#L227

Is the pertinent one for you.

Cheers

Russell

On 13 Feb 2017, at 11:08, Alex Feng <[hidden email]> wrote:

> Hi Russell,
>
> We have tried this, but it returns a reference(some number) only.  We have googled and haven't found any clue about how to use this API.
> The API user guide doesn't say much about the "reference".  What is the next step with the returned reference ?
>
>
> cs_bucket_fold(Pid::pid(), Bucket::bucket() | bucket_and_type(), Opts::cs_opts()) -> {ok, reference()} | {error, term()}
> secret function, do not use, or I come to your house and keeel you.
>
> cs_opt() = {timeout, timeout()} | {continuation, binary()} | {max_results, non_neg_integer() | all} | {start_key, binary()} | {start_incl, boolean()} | {end_key, binary()} | {end_incl, boolean()}
>
>
> Br,
> Alex
>
> 2017-02-13 16:34 GMT+08:00 Russell Brown <[hidden email]>:
> If you look at the riak-erlang-client here https://github.com/basho/riak-erlang-client/blob/develop/src/riakc_pb_socket.erl#L1129 there is a client API call that was implemented for CS (riak s2, or whatever it is called, the large object store thingy that basho had) that will use the $keys index, but in the back end calls fold_objects rather than fold_keys on eleveldb, and so returns the whole object. NOTE: this is equivalent to r=1, you don’t get a quorum read here, but a single vnode per-value only. I don’t know if other clients added this API, but it would not be hard to add to any client that supports 2i. Like I say, originally it was for riakCS, but it’s open source and part of the release for 4 years now, so hardly a secret.
>
> Cheers
>
> Russell
>
> On 13 Feb 2017, at 06:18, Alex Feng <[hidden email]> wrote:
>
> > Hi Russell,
> >
> > In your reply, you mentioned this,
> >
> > >> There is also the feature that can return the actual riak objects for a $keys index search,
> > >>You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see >>return_terms in docs.)
> >
> > If I understood correctly, it is possible to fetch object (value) by 2i in one time. But, we have tried using "return_term = true", it returns the index with key comparing only key when not using "return_term=true".  It doesn't help much with extra index, what we want to achieve is to fetch the object in one time.
> >
> >
> > Our use case,  client search DB every 10 seconds by 2i, Riak will return a list of around 5000 results(Keys), then client will query DB to fetch value for each key, basically it is around 5000 times,  client is easy to run into some issues most of the time. Any suggestion here ?
> >
> > Many thanks in advance.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 19:02 GMT+08:00 Alex Feng <[hidden email]>:
> > Hi Russell,
> >
> > It is really helpful, thank you a lot.
> > We are suffering from solr crash now, are considering to switch to 2i.
> >
> > Br,
> > Alex
> >
> > 2017-02-06 16:53 GMT+08:00 Russell Brown <[hidden email]>:
> > It’s worth noting that secondary indexes (2i) has some other advantages over solr search. If you _can_ model your queries in 2i then I'd recommend it.
> >
> > Secondary indexes  have a richer API than is currently documented, if you look at https://docs.basho.com/riak/1.4.7/dev/using/2i/ you’ll see that documents a feature that allows the index terms to be filtered via reg ex. There is also the feature that can return the actual riak objects for a $keys index search,
> > You can pack the index terms with data and return the terms in a query so that you don’t need a further object fetch (see return_terms in docs.)
> > Secondary indexes are written atomically with the object they index.
> > Operationally they don’t require you run a JVM and Solr alongside your riak nodes.
> >
> > You have the tools with basho_bench to answer the question about performance and overhead for your workload. I suspect for “overhead” 2i wins, as there is no JVM-per-node.
> >
> > Modelling for 2i is perhaps harder, in the classical nosql way, you have to do more work upfront when designing your querying.
> >
> > I hope that helps a little. I worked quite a lot on 2i and never really understood why riak-search was seen as a replacment, imo they’re complementary, and you pick the one that best fits.
> >
> > Cheers
> >
> > Russell
> >
> > On 2 Feb 2017, at 09:43, Alex Feng <[hidden email]> wrote:
> >
> > > Hello Riak-users,
> > >
> > > I am currently using Riak search to do some queries, since my queries are very simple, it should be fulfilled by secondary indexes as well.
> > > So, my question is which one has better performance and less overhead, let's say both can fulfill the query requirement.
> > >
> > > Many thanks in advance.
> > >
> > > Br,
> > > Alex
> > > _______________________________________________
> > > riak-users mailing list
> > > [hidden email]
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
> >
> >
>
>



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Secondary indexes or Riak search ?

John Daily
In reply to this post by Russell Brown-4

A quick look at riak_test it seems that the `return_body` parameter is generally supported for both HTTP and PB now. I wasn’t aware of that.

I’m not sure into which releases the feature has made it, but bewarned: if it works at all, return_body only works for system indexes: $bucket and $key.

-John

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com