RiakTS and Multi-Backend

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

RiakTS and Multi-Backend

Junk, Damion A
Hello all -

I was wondering if it is possible to use RiakTS with a multi-backend configuration.

I have an existing set of applications using RiakKV and Bitcask, but we're now wanting to start using some of the TS features on a new project. Setting up the multi-backend configuration with TS seems to work fine (our app existing app reads the Bitcask buckets and SOLR indices without issue), and we can even create a TS schema. We have LevelDB as the default backend, so presumably this is what was used during the "create table" query.

When trying to query the TS table, we're seeing:

2016-10-12 10:02:41.372 [error] <0.1410.0> gen_fsm <0.1410.0> in state active terminated with reason: call to undefined function riak_kv_multi_backend:range_scan/4 from riak_kv_vnode:list/7 line 1875

And then in the client code, no return until finally an Exception (Riak Java Client).


Thanks for any assistance!


Damion
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: RiakTS and Multi-Backend

John Daily
We have not done any work to support the multi-backend, hence the error you’re seeing. TS depends exclusively on leveldb.

We’re not recommending the use of KV functionality in the TS product yet, because the latter is still changing rapidly and we will need to go back and fix some basic KV mechanisms. We’re also not yet sure of the performance characteristics if both KV and TS are in use under heavy load.

In short: I apologize, but we’re not really ready to support your use case yet.

-John

> On Oct 12, 2016, at 12:17 PM, Junk, Damion A <[hidden email]> wrote:
>
> Hello all -
>
> I was wondering if it is possible to use RiakTS with a multi-backend configuration.
>
> I have an existing set of applications using RiakKV and Bitcask, but we're now wanting to start using some of the TS features on a new project. Setting up the multi-backend configuration with TS seems to work fine (our app existing app reads the Bitcask buckets and SOLR indices without issue), and we can even create a TS schema. We have LevelDB as the default backend, so presumably this is what was used during the "create table" query.
>
> When trying to query the TS table, we're seeing:
>
> 2016-10-12 10:02:41.372 [error] <0.1410.0> gen_fsm <0.1410.0> in state active terminated with reason: call to undefined function riak_kv_multi_backend:range_scan/4 from riak_kv_vnode:list/7 line 1875
>
> And then in the client code, no return until finally an Exception (Riak Java Client).
>
>
> Thanks for any assistance!
>
>
> Damion
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: RiakTS and Multi-Backend

Junk, Damion A
So are you suggesting that it's not advisable to use even KV backed by LevelDB in a TS instance, or is it more that performance is unknown and therefore not guaranteed / supported?  If the data is otherwise "safe", this may still be a better option for us than running separate clusters.

Are others who need both a general KV store and the TS functionality running multiple clusters to handle this use case?

Thanks for the quick reply!


Damion

> On Oct 12, 2016, at 11:53 AM, John Daily <[hidden email]> wrote:
>
> We have not done any work to support the multi-backend, hence the error you’re seeing. TS depends exclusively on leveldb.
>
> We’re not recommending the use of KV functionality in the TS product yet, because the latter is still changing rapidly and we will need to go back and fix some basic KV mechanisms. We’re also not yet sure of the performance characteristics if both KV and TS are in use under heavy load.
>
> In short: I apologize, but we’re not really ready to support your use case yet.
>
> -John
>
>> On Oct 12, 2016, at 12:17 PM, Junk, Damion A <[hidden email]> wrote:
>>
>> Hello all -
>>
>> I was wondering if it is possible to use RiakTS with a multi-backend configuration.
>>
>> I have an existing set of applications using RiakKV and Bitcask, but we're now wanting to start using some of the TS features on a new project. Setting up the multi-backend configuration with TS seems to work fine (our app existing app reads the Bitcask buckets and SOLR indices without issue), and we can even create a TS schema. We have LevelDB as the default backend, so presumably this is what was used during the "create table" query.
>>
>> When trying to query the TS table, we're seeing:
>>
>> 2016-10-12 10:02:41.372 [error] <0.1410.0> gen_fsm <0.1410.0> in state active terminated with reason: call to undefined function riak_kv_multi_backend:range_scan/4 from riak_kv_vnode:list/7 line 1875
>>
>> And then in the client code, no return until finally an Exception (Riak Java Client).
>>
>>
>> Thanks for any assistance!
>>
>>
>> Damion
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: RiakTS and Multi-Backend

John Daily
There are several important KV components such as AAE and Search with which TS integration has not been sufficiently tested.  At this time we still recommend running multiple clusters as mixed use workloads in Riak TS are currently not supported.

Your internal testing may reveal that small supporting KV datasets may work as expected and allow you to augment your TS data in a single cluster.  We are interested in your results as this will help us to improve and expand Riak TS moving forward.

-John

> On Oct 12, 2016, at 1:04 PM, Junk, Damion A <[hidden email]> wrote:
>
> So are you suggesting that it's not advisable to use even KV backed by LevelDB in a TS instance, or is it more that performance is unknown and therefore not guaranteed / supported?  If the data is otherwise "safe", this may still be a better option for us than running separate clusters.
>
> Are others who need both a general KV store and the TS functionality running multiple clusters to handle this use case?
>
> Thanks for the quick reply!
>
>
> Damion
>
>> On Oct 12, 2016, at 11:53 AM, John Daily <[hidden email]> wrote:
>>
>> We have not done any work to support the multi-backend, hence the error you’re seeing. TS depends exclusively on leveldb.
>>
>> We’re not recommending the use of KV functionality in the TS product yet, because the latter is still changing rapidly and we will need to go back and fix some basic KV mechanisms. We’re also not yet sure of the performance characteristics if both KV and TS are in use under heavy load.
>>
>> In short: I apologize, but we’re not really ready to support your use case yet.
>>
>> -John
>>
>>> On Oct 12, 2016, at 12:17 PM, Junk, Damion A <[hidden email]> wrote:
>>>
>>> Hello all -
>>>
>>> I was wondering if it is possible to use RiakTS with a multi-backend configuration.
>>>
>>> I have an existing set of applications using RiakKV and Bitcask, but we're now wanting to start using some of the TS features on a new project. Setting up the multi-backend configuration with TS seems to work fine (our app existing app reads the Bitcask buckets and SOLR indices without issue), and we can even create a TS schema. We have LevelDB as the default backend, so presumably this is what was used during the "create table" query.
>>>
>>> When trying to query the TS table, we're seeing:
>>>
>>> 2016-10-12 10:02:41.372 [error] <0.1410.0> gen_fsm <0.1410.0> in state active terminated with reason: call to undefined function riak_kv_multi_backend:range_scan/4 from riak_kv_vnode:list/7 line 1875
>>>
>>> And then in the client code, no return until finally an Exception (Riak Java Client).
>>>
>>>
>>> Thanks for any assistance!
>>>
>>>
>>> Damion
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: RiakTS and Multi-Backend

Damien Krotkine-3

For what it's worth, I've successfully used KV features inside of Riak
TS and tested it quite a lot, including with a heavy load. As John said,
I didn't use multi-backend and I disabled AAE.

Riak TS was happy when using Gets, Sets, bucket types, and including
CRDTs (I tested only the Set CRDTs). So you can go ahead and test these
features, they should work fine, at least for now.

That said, I would not recommend using this in production in the long
term. Instead, use a different cluster for KV. However, different
cluster doesn't have to mean different hardware. You could setup Riak KV
and Riak TS alongside on the same nodes (you need some work to have
separate configuration, data directory, different cookies, etc, but it's not a
lot of work). Granted, both clusters will share resources, but having
them running on the same nodes has a lot of benefits in term of
maintenance and administration.

John Daily writes:

> There are several important KV components such as AAE and Search with which TS integration has not been sufficiently tested.  At this time we still recommend running multiple clusters as mixed use workloads in Riak TS are currently not supported.
>
> Your internal testing may reveal that small supporting KV datasets may work as expected and allow you to augment your TS data in a single cluster.  We are interested in your results as this will help us to improve and expand Riak TS moving forward.
>
> -John
>
>> On Oct 12, 2016, at 1:04 PM, Junk, Damion A <[hidden email]> wrote:
>>
>> So are you suggesting that it's not advisable to use even KV backed by LevelDB in a TS instance, or is it more that performance is unknown and therefore not guaranteed / supported?  If the data is otherwise "safe", this may still be a better option for us than running separate clusters.
>>
>> Are others who need both a general KV store and the TS functionality running multiple clusters to handle this use case?
>>
>> Thanks for the quick reply!
>>
>>
>> Damion
>>
>>> On Oct 12, 2016, at 11:53 AM, John Daily <[hidden email]> wrote:
>>>
>>> We have not done any work to support the multi-backend, hence the error you’re seeing. TS depends exclusively on leveldb.
>>>
>>> We’re not recommending the use of KV functionality in the TS product yet, because the latter is still changing rapidly and we will need to go back and fix some basic KV mechanisms. We’re also not yet sure of the performance characteristics if both KV and TS are in use under heavy load.
>>>
>>> In short: I apologize, but we’re not really ready to support your use case yet.
>>>
>>> -John
>>>
>>>> On Oct 12, 2016, at 12:17 PM, Junk, Damion A <[hidden email]> wrote:
>>>>
>>>> Hello all -
>>>>
>>>> I was wondering if it is possible to use RiakTS with a multi-backend configuration.
>>>>
>>>> I have an existing set of applications using RiakKV and Bitcask, but we're now wanting to start using some of the TS features on a new project. Setting up the multi-backend configuration with TS seems to work fine (our app existing app reads the Bitcask buckets and SOLR indices without issue), and we can even create a TS schema. We have LevelDB as the default backend, so presumably this is what was used during the "create table" query.
>>>>
>>>> When trying to query the TS table, we're seeing:
>>>>
>>>> 2016-10-12 10:02:41.372 [error] <0.1410.0> gen_fsm <0.1410.0> in state active terminated with reason: call to undefined function riak_kv_multi_backend:range_scan/4 from riak_kv_vnode:list/7 line 1875
>>>>
>>>> And then in the client code, no return until finally an Exception (Riak Java Client).
>>>>
>>>>
>>>> Thanks for any assistance!
>>>>
>>>>
>>>> Damion
>>>> _______________________________________________
>>>> riak-users mailing list
>>>> [hidden email]
>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


--
Cheers,
Damien Krotkine

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com