Model design and atomic transactions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Model design and atomic transactions

Amirouche
Héllo,

At work, we are studying the offering in the distributed database
space and my eyes stumbled upon riak (via erlang).

I am wondering how does riak compare to leveldb (bsddb, wiredtiger
and the like) when it comes down to schema design.

Does riak support orderred records based on the key?

If the records are ordered, is it possible to change the 'order'
function/procedure?

Regarding CAP I understand, that Riak is AP which means that some
time, some node can see old version the database. During a partition,
is it possible to do writes at this point?

I don't understand what ACID guarantees riak does provide. What's
the position of riak regarding ACID?

How do you compare riak and mnesia?

Thanks!


Amirouche ~ http://hyperdev.fr


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Model design and atomic transactions

Fred Dushin-2
Okay, I'll bite, but I don't know if anything I say will be satisfying.

In general Riak doesn't make any assumptions about ordering of data, particularly around back ends.  While Riak does support levelDB as a back end, and does order keys (and other data, such as secondary indices), until very recently not much leveraged this ordering.  We did a little work in Riak/KV to leverage ordering to optimize bucket and key listing, but that work was never released.  If you are interested, the work is on a few PRs and branches [1]

Riak data is generally schema-less.  Eventually-consistent Key, value, get, put, delete, with some more advanced features around the edges (secondary indexing, CRDTs, semi-structured search), as well a "enterprise-level" features, mostly focused around multi-datacenter replication.  I am sure I am butchering the feature set of Riak, the product.  But at it's core, that's the way I look at it.

Re: writes during network partitions, yes, of course, this part of the motivation of DynamoDB (not necessarily the product), and Riak faithfully implements the architecture, along with what guarantees can be provided for write availability in the presence of machine|network|software failure.

I can't really speak to ACID properties, except to say there is no transactional integrity in Riak, other than some tunable guarantees around reads and writes.

Can't say much about mnesia, except that Riak can store 100s of terabytes of data across a cluster (if not more).

Hope that helps.

-Fred


On Jul 4, 2017, at 1:49 PM, Amirouche <[hidden email]> wrote:

Héllo,

At work, we are studying the offering in the distributed database
space and my eyes stumbled upon riak (via erlang).

I am wondering how does riak compare to leveldb (bsddb, wiredtiger
and the like) when it comes down to schema design.

Does riak support orderred records based on the key?

If the records are ordered, is it possible to change the 'order'
function/procedure?

Regarding CAP I understand, that Riak is AP which means that some
time, some node can see old version the database. During a partition,
is it possible to do writes at this point?

I don't understand what ACID guarantees riak does provide. What's
the position of riak regarding ACID?

How do you compare riak and mnesia?

Thanks!


Amirouche ~ http://hyperdev.fr


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Model design and atomic transactions

Christopher Meiklejohn-2
Referring to ACID: ACID is a property of transactions.

Riak doesn't provide any transactional facilities: there's no notion
of isolation (the 'I' in ACID; a concurrency control mechanism, such
as serializability), nor the ability to perform multiple updates
atomically against the database in an all-or-nothing fashion (the 'A'
in ACID.)

The closest you can get to having something with similar, but weaker,
guarantees in Riak, is to use the strong consistency mechanism that
provides an atomic compare-and-swap for a single register: by reading
and supplying your read context with your write you can ensure a form
of isolation (or, concurrency control) by failing write operations
that have been invalidated by a concurrent writer operating on the
same register.  These operations are CP, not AP, and will not be
available if a majority of the cluster is not reachable by the node
performing the operation.

Thanks,
Christopher

On Wed, Jul 5, 2017 at 2:55 PM, Fred Dushin <[hidden email]> wrote:

> Okay, I'll bite, but I don't know if anything I say will be satisfying.
>
> In general Riak doesn't make any assumptions about ordering of data,
> particularly around back ends.  While Riak does support levelDB as a back
> end, and does order keys (and other data, such as secondary indices), until
> very recently not much leveraged this ordering.  We did a little work in
> Riak/KV to leverage ordering to optimize bucket and key listing, but that
> work was never released.  If you are interested, the work is on a few PRs
> and branches [1]
>
> Riak data is generally schema-less.  Eventually-consistent Key, value, get,
> put, delete, with some more advanced features around the edges (secondary
> indexing, CRDTs, semi-structured search), as well a "enterprise-level"
> features, mostly focused around multi-datacenter replication.  I am sure I
> am butchering the feature set of Riak, the product.  But at it's core,
> that's the way I look at it.
>
> Re: writes during network partitions, yes, of course, this part of the
> motivation of DynamoDB (not necessarily the product), and Riak faithfully
> implements the architecture, along with what guarantees can be provided for
> write availability in the presence of machine|network|software failure.
>
> I can't really speak to ACID properties, except to say there is no
> transactional integrity in Riak, other than some tunable guarantees around
> reads and writes.
>
> Can't say much about mnesia, except that Riak can store 100s of terabytes of
> data across a cluster (if not more).
>
> Hope that helps.
>
> -Fred
>
> [1] https://github.com/basho/riak_kv/pull/1634 and
> https://github.com/basho/riak_kv/compare/jv-optimize_list_buckets...develop
>
> On Jul 4, 2017, at 1:49 PM, Amirouche <[hidden email]> wrote:
>
> Héllo,
>
> At work, we are studying the offering in the distributed database
> space and my eyes stumbled upon riak (via erlang).
>
> I am wondering how does riak compare to leveldb (bsddb, wiredtiger
> and the like) when it comes down to schema design.
>
> Does riak support orderred records based on the key?
>
> If the records are ordered, is it possible to change the 'order'
> function/procedure?
>
> Regarding CAP I understand, that Riak is AP which means that some
> time, some node can see old version the database. During a partition,
> is it possible to do writes at this point?
>
> I don't understand what ACID guarantees riak does provide. What's
> the position of riak regarding ACID?
>
> How do you compare riak and mnesia?
>
> Thanks!
>
>
> Amirouche ~ http://hyperdev.fr
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Model design and atomic transactions

Amirouche
Thanks for your replies.


Le 07/07/2017 à 01:21, Christopher Meiklejohn a écrit :
> Referring to ACID: ACID is a property of transactions.

You are right.
> Riak doesn't provide any transactional facilities: there's no notion
> of isolation (the 'I' in ACID; a concurrency control mechanism, such
> as serializability), nor the ability to perform multiple updates
> atomically against the database in an all-or-nothing fashion (the 'A'
> in ACID.)
Ok. My application has very low writes number, with low chances
of concurrent writes against the same keys.

That said, there is a backup restore feature, that loads the configuration
of the whole application in one go. I guess that when backup restore is
running the whole database must be locked via REDIS or something. WDYT?
> The closest you can get to having something with similar, but weaker,
> guarantees in Riak, is to use the strong consistency mechanism that
> provides an atomic compare-and-swap for a single register: by reading
> and supplying your read context with your write you can ensure a form
> of isolation (or, concurrency control) by failing write operations
> that have been invalidated by a concurrent writer operating on the
> same register.  These operations are CP, not AP, and will not be
> available if a majority of the cluster is not reachable by the node
> performing the operation.
Are you refering to
http://docs.basho.com/riak/kv/2.2.3/developing/app-guide/strong-consistency/

>
> Thanks,
> Christopher
>
> On Wed, Jul 5, 2017 at 2:55 PM, Fred Dushin <[hidden email]> wrote:
>> Okay, I'll bite, but I don't know if anything I say will be satisfying.

You describe well my understanding of Riak.

>>
>> In general Riak doesn't make any assumptions about ordering of data,
>> particularly around back ends.  While Riak does support levelDB as a back
>> end, and does order keys (and other data, such as secondary indices), until
>> very recently not much leveraged this ordering.  We did a little work in
>> Riak/KV to leverage ordering to optimize bucket and key listing, but that
>> work was never released.  If you are interested, the work is on a few PRs
>> and branches [1]
Actually I don't think I need ordered keys. I understand that this is
supported
using the Secondary Indices, but AFAIU that feature is deprecated in
favor of search
feature.

>> Riak data is generally schema-less.  Eventually-consistent Key, value, get,
>> put, delete, with some more advanced features around the edges (secondary
>> indexing, CRDTs, semi-structured search),
Another question, when I don't use CDRTs (aka. Riak data types) how does
the application handle reconciliation? Otherwise said, how one does
implement
it's own data types?

>> as well a "enterprise-level" features, mostly focused around multi-datacenter replication.

I have a question regarding multi datacenter replication. Do all nodes
must know
each other?

>>    I am sure I
>> am butchering the feature set of Riak, the product.  But at it's core,
>> that's the way I look at it.
That what I understand about Riak without being able to say it.
>>
>> Re: writes during network partitions, yes, of course, this part of the
>> motivation of DynamoDB (not necessarily the product), and Riak faithfully
>> implements the architecture, along with what guarantees can be provided for
>> write availability in the presence of machine|network|software failure.
>>
>> I can't really speak to ACID properties, except to say there is no
>> transactional integrity in Riak, other than some tunable guarantees around
>> reads and writes.
Somehow you must "work around" eventual consistency somehow. I will read
dynamodb documentation to know more.
>> Can't say much about mnesia, except that Riak can store 100s of terabytes of
>> data across a cluster (if not more).
>>
>> Hope that helps.

Thanks.

>>
>> -Fred
>>
>> [1] https://github.com/basho/riak_kv/pull/1634 and
>> https://github.com/basho/riak_kv/compare/jv-optimize_list_buckets...develop
>>
>> On Jul 4, 2017, at 1:49 PM, Amirouche <[hidden email]> wrote:
>>
>> Héllo,
>>
>> At work, we are studying the offering in the distributed database
>> space and my eyes stumbled upon riak (via erlang).
>>
>> I am wondering how does riak compare to leveldb (bsddb, wiredtiger
>> and the like) when it comes down to schema design.
>>
>> Does riak support orderred records based on the key?
>>
>> If the records are ordered, is it possible to change the 'order'
>> function/procedure?
>>
>> Regarding CAP I understand, that Riak is AP which means that some
>> time, some node can see old version the database. During a partition,
>> is it possible to do writes at this point?
>>
>> I don't understand what ACID guarantees riak does provide. What's
>> the position of riak regarding ACID?
>>
>> How do you compare riak and mnesia?
>>
>> Thanks!
>>
>>
>> Amirouche ~ http://hyperdev.fr
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com