are Siblings ordered?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

are Siblings ordered?

Alex Rice
Are they ordered by timestamp, and is the ordering guaranteed? (within
the clock accuracy of course). Using the C# client Thanks,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Jeremiah Peschka

There's no guarantee of return order as far as I know. Since you can't count on clocks anyway...

Are you trying to determine which data modifications to apply from multiple siblings?

---
sent from a tiny portion of the hive mind...
in this case, a phone

On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email]> wrote:
Are they ordered by timestamp, and is the ordering guaranteed? (within
the clock accuracy of course). Using the C# client Thanks,
Alex

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Alex Rice
Yes, exactly that's what I'm working on. By knowing which sibling is
the oldest and which is the newest it seems like I can usually figure
out how to marge/apply the modifications. Things like Player profiles,
friends lists, etc.

On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
<[hidden email]> wrote:

> There's no guarantee of return order as far as I know. Since you can't count
> on clocks anyway...
>
> Are you trying to determine which data modifications to apply from multiple
> siblings?
>
> ---
> sent from a tiny portion of the hive mind...
> in this case, a phone
>
> On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email]> wrote:
>>
>> Are they ordered by timestamp, and is the ordering guaranteed? (within
>> the clock accuracy of course). Using the C# client Thanks,
>> Alex
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Alex Rice
Yeah so it seems oldest/newest is a flawed notion I have to get over!
Siblings are so *ZEN* :)

On Mon, Oct 7, 2013 at 12:47 PM, Alex Rice <[hidden email]> wrote:

> Yes, exactly that's what I'm working on. By knowing which sibling is
> the oldest and which is the newest it seems like I can usually figure
> out how to marge/apply the modifications. Things like Player profiles,
> friends lists, etc.
>
> On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> <[hidden email]> wrote:
>> There's no guarantee of return order as far as I know. Since you can't count
>> on clocks anyway...
>>
>> Are you trying to determine which data modifications to apply from multiple
>> siblings?
>>
>> ---
>> sent from a tiny portion of the hive mind...
>> in this case, a phone
>>
>> On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email]> wrote:
>>>
>>> Are they ordered by timestamp, and is the ordering guaranteed? (within
>>> the clock accuracy of course). Using the C# client Thanks,
>>> Alex
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Sam Elliott
In reply to this post by Alex Rice
Each sibling has it's own last-modified date, which you should be able to use to sort them when you get them back.

However, I'd suggest the following: if they're siblings, they were created from concurrent edits. Thus, create a merge function that is entirely deterministic without using the timestamp. This should save you from any clock skew issues, and also from the fact that an edit may not have been performed with the most up-to-date information.

I guess people can't wait for our CRDTs to launch.

Sam

--
Sam Elliott
Engineer
[hidden email]
--


On Monday, 7 October 2013 at 2:47PM, Alex Rice wrote:

> Yes, exactly that's what I'm working on. By knowing which sibling is
> the oldest and which is the newest it seems like I can usually figure
> out how to marge/apply the modifications. Things like Player profiles,
> friends lists, etc.
>
> On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> <[hidden email] (mailto:[hidden email])> wrote:
> > There's no guarantee of return order as far as I know. Since you can't count
> > on clocks anyway...
> >
> > Are you trying to determine which data modifications to apply from multiple
> > siblings?
> >
> > ---
> > sent from a tiny portion of the hive mind...
> > in this case, a phone
> >
> > On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email] (mailto:[hidden email])> wrote:
> > >
> > > Are they ordered by timestamp, and is the ordering guaranteed? (within
> > > the clock accuracy of course). Using the C# client Thanks,
> > > Alex
> > >
> > > _______________________________________________
> > > riak-users mailing list
> > > [hidden email] (mailto:[hidden email])
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email] (mailto:[hidden email])
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Jeremiah Peschka
I was about to add something similar :)

There is CRDT support present in the develop branch of CorrugatedIron, that operates at a relatively low level - you're left to construct the operations yourself. The current API works great for things like friends lists but it's rough for something like modifying the properties in an existing object. 

Expect more developments as time allows.

---
Jeremiah Peschka - Founder, Brent Ozar Unlimited
MCITP: SQL Server 2008, MVP
Cloudera Certified Developer for Apache Hadoop


On Mon, Oct 7, 2013 at 12:04 PM, Sam Elliott <[hidden email]> wrote:
Each sibling has it's own last-modified date, which you should be able to use to sort them when you get them back.

However, I'd suggest the following: if they're siblings, they were created from concurrent edits. Thus, create a merge function that is entirely deterministic without using the timestamp. This should save you from any clock skew issues, and also from the fact that an edit may not have been performed with the most up-to-date information.

I guess people can't wait for our CRDTs to launch.

Sam

--
Sam Elliott
Engineer
[hidden email]
--


On Monday, 7 October 2013 at 2:47PM, Alex Rice wrote:

> Yes, exactly that's what I'm working on. By knowing which sibling is
> the oldest and which is the newest it seems like I can usually figure
> out how to marge/apply the modifications. Things like Player profiles,
> friends lists, etc.
>
> On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> <[hidden email] (mailto:[hidden email])> wrote:
> > There's no guarantee of return order as far as I know. Since you can't count
> > on clocks anyway...
> >
> > Are you trying to determine which data modifications to apply from multiple
> > siblings?
> >
> > ---
> > sent from a tiny portion of the hive mind...
> > in this case, a phone
> >
> > On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email] (mailto:[hidden email])> wrote:
> > >
> > > Are they ordered by timestamp, and is the ordering guaranteed? (within
> > > the clock accuracy of course). Using the C# client Thanks,
> > > Alex
> > >
> > > _______________________________________________
> > > riak-users mailing list
> > > [hidden email] (mailto:[hidden email])
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email] (mailto:[hidden email])
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com





_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Pedram Nimreezi
In reply to this post by Sam Elliott

I'd like to see the dotted version vectors get launched, even with merging siblings high concurrency = high amount of unnecessary siblings,
Which wastes space and impacts performance. Dotted version vectors helps improve this. Proof of concept and paper available...

On Oct 7, 2013 3:06 PM, "Sam Elliott" <[hidden email]> wrote:
Each sibling has it's own last-modified date, which you should be able to use to sort them when you get them back.

However, I'd suggest the following: if they're siblings, they were created from concurrent edits. Thus, create a merge function that is entirely deterministic without using the timestamp. This should save you from any clock skew issues, and also from the fact that an edit may not have been performed with the most up-to-date information.

I guess people can't wait for our CRDTs to launch.

Sam

--
Sam Elliott
Engineer
[hidden email]
--


On Monday, 7 October 2013 at 2:47PM, Alex Rice wrote:

> Yes, exactly that's what I'm working on. By knowing which sibling is
> the oldest and which is the newest it seems like I can usually figure
> out how to marge/apply the modifications. Things like Player profiles,
> friends lists, etc.
>
> On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> <[hidden email] (mailto:[hidden email])> wrote:
> > There's no guarantee of return order as far as I know. Since you can't count
> > on clocks anyway...
> >
> > Are you trying to determine which data modifications to apply from multiple
> > siblings?
> >
> > ---
> > sent from a tiny portion of the hive mind...
> > in this case, a phone
> >
> > On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email] (mailto:[hidden email])> wrote:
> > >
> > > Are they ordered by timestamp, and is the ordering guaranteed? (within
> > > the clock accuracy of course). Using the C# client Thanks,
> > > Alex
> > >
> > > _______________________________________________
> > > riak-users mailing list
> > > [hidden email] (mailto:[hidden email])
> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email] (mailto:[hidden email])
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Russell Brown
I'd very much like to see the same thing.

I have a working branch and test here https://github.com/basho/riak_kv/pull/688 and https://github.com/basho/riak_test/tree/feature/rdb-sib-ex

This isn't using the DVVSets but a sort of rough hack, where we store the event dot for each write in the metadata dictionary for that value. As you can see from the test it works. There is some work to be done to productionise it, I really hope I get time for that. Feel free to pitch in (the riak_object tests that are broken would be the first place to help.)

My only concern is with client id vclocks (which are still supported in Riak.)

Cheers

Russell

On 8 Oct 2013, at 05:40, Pedram Nimreezi <[hidden email]> wrote:

> I'd like to see the dotted version vectors get launched, even with merging siblings high concurrency = high amount of unnecessary siblings,
> Which wastes space and impacts performance. Dotted version vectors helps improve this. Proof of concept and paper available...
>
> On Oct 7, 2013 3:06 PM, "Sam Elliott" <[hidden email]> wrote:
> Each sibling has it's own last-modified date, which you should be able to use to sort them when you get them back.
>
> However, I'd suggest the following: if they're siblings, they were created from concurrent edits. Thus, create a merge function that is entirely deterministic without using the timestamp. This should save you from any clock skew issues, and also from the fact that an edit may not have been performed with the most up-to-date information.
>
> I guess people can't wait for our CRDTs to launch.
>
> Sam
>
> --
> Sam Elliott
> Engineer
> [hidden email]
> --
>
>
> On Monday, 7 October 2013 at 2:47PM, Alex Rice wrote:
>
> > Yes, exactly that's what I'm working on. By knowing which sibling is
> > the oldest and which is the newest it seems like I can usually figure
> > out how to marge/apply the modifications. Things like Player profiles,
> > friends lists, etc.
> >
> > On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> > <[hidden email] (mailto:[hidden email])> wrote:
> > > There's no guarantee of return order as far as I know. Since you can't count
> > > on clocks anyway...
> > >
> > > Are you trying to determine which data modifications to apply from multiple
> > > siblings?
> > >
> > > ---
> > > sent from a tiny portion of the hive mind...
> > > in this case, a phone
> > >
> > > On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email] (mailto:[hidden email])> wrote:
> > > >
> > > > Are they ordered by timestamp, and is the ordering guaranteed? (within
> > > > the clock accuracy of course). Using the C# client Thanks,
> > > > Alex
> > > >
> > > > _______________________________________________
> > > > riak-users mailing list
> > > > [hidden email] (mailto:[hidden email])
> > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> >
> >
> >
> > _______________________________________________
> > riak-users mailing list
> > [hidden email] (mailto:[hidden email])
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: are Siblings ordered?

Pedram Nimreezi
Excellent!


On Tue, Oct 8, 2013 at 3:24 AM, Russell Brown <[hidden email]> wrote:
I'd very much like to see the same thing.

I have a working branch and test here https://github.com/basho/riak_kv/pull/688 and https://github.com/basho/riak_test/tree/feature/rdb-sib-ex

This isn't using the DVVSets but a sort of rough hack, where we store the event dot for each write in the metadata dictionary for that value. As you can see from the test it works. There is some work to be done to productionise it, I really hope I get time for that. Feel free to pitch in (the riak_object tests that are broken would be the first place to help.)

My only concern is with client id vclocks (which are still supported in Riak.)

Cheers

Russell

On 8 Oct 2013, at 05:40, Pedram Nimreezi <[hidden email]> wrote:

> I'd like to see the dotted version vectors get launched, even with merging siblings high concurrency = high amount of unnecessary siblings,
> Which wastes space and impacts performance. Dotted version vectors helps improve this. Proof of concept and paper available...
>
> On Oct 7, 2013 3:06 PM, "Sam Elliott" <[hidden email]> wrote:
> Each sibling has it's own last-modified date, which you should be able to use to sort them when you get them back.
>
> However, I'd suggest the following: if they're siblings, they were created from concurrent edits. Thus, create a merge function that is entirely deterministic without using the timestamp. This should save you from any clock skew issues, and also from the fact that an edit may not have been performed with the most up-to-date information.
>
> I guess people can't wait for our CRDTs to launch.
>
> Sam
>
> --
> Sam Elliott
> Engineer
> [hidden email]
> --
>
>
> On Monday, 7 October 2013 at 2:47PM, Alex Rice wrote:
>
> > Yes, exactly that's what I'm working on. By knowing which sibling is
> > the oldest and which is the newest it seems like I can usually figure
> > out how to marge/apply the modifications. Things like Player profiles,
> > friends lists, etc.
> >
> > On Mon, Oct 7, 2013 at 12:43 PM, Jeremiah Peschka
> > <[hidden email] (mailto:[hidden email])> wrote:
> > > There's no guarantee of return order as far as I know. Since you can't count
> > > on clocks anyway...
> > >
> > > Are you trying to determine which data modifications to apply from multiple
> > > siblings?
> > >
> > > ---
> > > sent from a tiny portion of the hive mind...
> > > in this case, a phone
> > >
> > > On Oct 7, 2013 11:40 AM, "Alex Rice" <[hidden email] (mailto:[hidden email])> wrote:
> > > >
> > > > Are they ordered by timestamp, and is the ordering guaranteed? (within
> > > > the clock accuracy of course). Using the C# client Thanks,
> > > > Alex
> > > >
> > > > _______________________________________________
> > > > riak-users mailing list
> > > > [hidden email] (mailto:[hidden email])
> > > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> > >
> >
> >
> >
> > _______________________________________________
> > riak-users mailing list
> > [hidden email] (mailto:[hidden email])
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




--
/* Sincerely
-------------------------------------------------------------- 
Pedram Nimreezi - Chief Technology Officer  */

// The hardest part of design … is keeping features out. - Donald Norman



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com