Riak Development Environments

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Riak Development Environments

Mark Phillips-4
I was going to make this part of today's Recap, but upon further
thinking I concluded that it might deserve its own thread...

@wmacgyver, @russelldb, and some others exchanged a few messages on
Twitter yesterday about what dev setup they are using for their
respective Riak work.

wmacgyver's response was: "3 macmini server 2.66ghz with 8gb, raid 1
500gb=your own ec2 7.5gb node with 500gb ebs. Every dev has a set."
[1]

Anyone else care to share their setup? Or have any questions about
what your dev setup should look like?

Mark

1 - http://twitter.com/wmacgyver/statuses/52117875980386304

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Ryan Zezeski
At my current job I've had the luxury of 4 dell poweredge servers (CentOS, 128GB, 1-2TB disk, 16-core).

However, I've been seriously considering either buying some old 1Us and building my own rack in the basement OR 3 used mac mini's from eBay (I'm single so I can do stuff like this).  The nice thing about mini's is they travel...plus they're so darn cute.  The nice thing about an actual rack is it is (probably?) closer to real production deployments.

I'm also tempted to use older hardware just b/c if you can make it more efficient there then the shinier stuff _should_ automatically get a boost.  Although I supposed stuff like NUMA vs. SMP could bite you?  I'm curious to hear what others think.

Mark, are you allowed to share any statistics related to the setups of actual production users?

-Ryan

On Mon, Mar 28, 2011 at 3:27 PM, Mark Phillips <[hidden email]> wrote:
I was going to make this part of today's Recap, but upon further
thinking I concluded that it might deserve its own thread...

@wmacgyver, @russelldb, and some others exchanged a few messages on
Twitter yesterday about what dev setup they are using for their
respective Riak work.

wmacgyver's response was: "3 macmini server 2.66ghz with 8gb, raid 1
500gb=your own ec2 7.5gb node with 500gb ebs. Every dev has a set."
[1]

Anyone else care to share their setup? Or have any questions about
what your dev setup should look like?

Mark

1 - http://twitter.com/wmacgyver/statuses/52117875980386304

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Alexander Sicular
I just spin up brand new ubuntu 32 or 64 bit instances on my mac pro w 16gb ram. That's all I need for dev and/or prototyping. Obviously load testing is a different animal.


On Mar 28, 2011, at 1:30 PM, Ryan Zezeski wrote:

> At my current job I've had the luxury of 4 dell poweredge servers (CentOS, 128GB, 1-2TB disk, 16-core).
>
> However, I've been seriously considering either buying some old 1Us and building my own rack in the basement OR 3 used mac mini's from eBay (I'm single so I can do stuff like this).  The nice thing about mini's is they travel...plus they're so darn cute.  The nice thing about an actual rack is it is (probably?) closer to real production deployments.
>
> I'm also tempted to use older hardware just b/c if you can make it more efficient there then the shinier stuff _should_ automatically get a boost.  Although I supposed stuff like NUMA vs. SMP could bite you?  I'm curious to hear what others think.
>
> Mark, are you allowed to share any statistics related to the setups of actual production users?
>
> -Ryan
>
> On Mon, Mar 28, 2011 at 3:27 PM, Mark Phillips <[hidden email]> wrote:
> I was going to make this part of today's Recap, but upon further
> thinking I concluded that it might deserve its own thread...
>
> @wmacgyver, @russelldb, and some others exchanged a few messages on
> Twitter yesterday about what dev setup they are using for their
> respective Riak work.
>
> wmacgyver's response was: "3 macmini server 2.66ghz with 8gb, raid 1
> 500gb=your own ec2 7.5gb node with 500gb ebs. Every dev has a set."
> [1]
>
> Anyone else care to share their setup? Or have any questions about
> what your dev setup should look like?
>
> Mark
>
> 1 - http://twitter.com/wmacgyver/statuses/52117875980386304
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Jeremiah Peschka
In reply to this post by Ryan Zezeski
You can pick up some HP Micro servers for dirt cheap. http://h10010.www1.hp.com/wwpc/us/en/sm/WF05a/15351-15351-4237916-4237918-4237917-4248009.html

I haven't tried setting them up yet (no room in the current domicile) but I suspect it'd be great for development.

Jeremiah Peschka
Microsoft SQL Server MVP
MCITP: Database Developer, DBA


On Mon, Mar 28, 2011 at 1:30 PM, Ryan Zezeski <[hidden email]> wrote:
At my current job I've had the luxury of 4 dell poweredge servers (CentOS, 128GB, 1-2TB disk, 16-core).

However, I've been seriously considering either buying some old 1Us and building my own rack in the basement OR 3 used mac mini's from eBay (I'm single so I can do stuff like this).  The nice thing about mini's is they travel...plus they're so darn cute.  The nice thing about an actual rack is it is (probably?) closer to real production deployments.

I'm also tempted to use older hardware just b/c if you can make it more efficient there then the shinier stuff _should_ automatically get a boost.  Although I supposed stuff like NUMA vs. SMP could bite you?  I'm curious to hear what others think.

Mark, are you allowed to share any statistics related to the setups of actual production users?

-Ryan


On Mon, Mar 28, 2011 at 3:27 PM, Mark Phillips <[hidden email]> wrote:
I was going to make this part of today's Recap, but upon further
thinking I concluded that it might deserve its own thread...

@wmacgyver, @russelldb, and some others exchanged a few messages on
Twitter yesterday about what dev setup they are using for their
respective Riak work.

wmacgyver's response was: "3 macmini server 2.66ghz with 8gb, raid 1
500gb=your own ec2 7.5gb node with 500gb ebs. Every dev has a set."
[1]

Anyone else care to share their setup? Or have any questions about
what your dev setup should look like?

Mark

1 - http://twitter.com/wmacgyver/statuses/52117875980386304

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Wilson MacGyver
In reply to this post by Ryan Zezeski
In our case, because we use ec2 7.5gb instance with ebs. The macmini setup I tweeted is a good match to production nodes.
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Mårten Gustafson
In reply to this post by Mark Phillips-4
On 28 March 2011 21:27, Mark Phillips <[hidden email]> wrote:
> Anyone else care to share their setup? Or have any questions about
> what your dev setup should look like?

For the general and common cases of testing - ie development,
verifying application and cluster behavior in case of failures etc a
simple "make devrel" works fine for me. I run Riak directly on my MBP.

As others have pointed out - load testing and tuning for production is
another story.



/mårten.

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Les Mikesell
On 3/29/2011 5:33 PM, Mårten Gustafson wrote:

> On 28 March 2011 21:27, Mark Phillips<[hidden email]>  wrote:
>> Anyone else care to share their setup? Or have any questions about
>> what your dev setup should look like?
>
> For the general and common cases of testing - ie development,
> verifying application and cluster behavior in case of failures etc a
> simple "make devrel" works fine for me. I run Riak directly on my MBP.
>
> As others have pointed out - load testing and tuning for production is
> another story.

What would have to be changed from the devrel version to get something
where you could simply add other nodes to migrate to production - and
maybe eventually remove the originals?

--
   Les Mikesell
    [hidden email]

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Jason J. W. Williams
In reply to this post by Mark Phillips-4
Anyone else care to share their setup? Or have any questions about
what your dev setup should look like?

We use 4GB RAM, 1x 2.0GHz core Cloudlayer instances at Softlayer for testing. Usually spin up 4-5 nodes.

-J

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Alexander Sicular
In reply to this post by Les Mikesell
"devrel" is not a version. It's a makefile setting that makes 3  
installations with the correct cinfig settings such that the 3 can be  
joined together.


@siculars on twitter
http://siculars.posterous.com

Sent from my iPhone

On Mar 29, 2011, at 15:52, Les Mikesell <[hidden email]> wrote:

> On 3/29/2011 5:33 PM, Mårten Gustafson wrote:
>> On 28 March 2011 21:27, Mark Phillips<[hidden email]>  wrote:
>>> Anyone else care to share their setup? Or have any questions about
>>> what your dev setup should look like?
>>
>> For the general and common cases of testing - ie development,
>> verifying application and cluster behavior in case of failures etc a
>> simple "make devrel" works fine for me. I run Riak directly on my  
>> MBP.
>>
>> As others have pointed out - load testing and tuning for production  
>> is
>> another story.
>
> What would have to be changed from the devrel version to get  
> something where you could simply add other nodes to migrate to  
> production - and maybe eventually remove the originals?
>
> --
>  Les Mikesell
>   [hidden email]
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Les Mikesell
Yes, the question is, what would you need to change in that initial setup to
have something that would migrate, still running to some larger number of nodes
on other machines?  A larger number of partitions?


On 3/29/11 6:27 PM, Alexander Sicular wrote:

> "devrel" is not a version. It's a makefile setting that makes 3 installations
> with the correct cinfig settings such that the 3 can be joined together.
>
>
> @siculars on twitter
> http://siculars.posterous.com
>
> Sent from my iPhone
>
> On Mar 29, 2011, at 15:52, Les Mikesell <[hidden email]> wrote:
>
>> On 3/29/2011 5:33 PM, Mårten Gustafson wrote:
>>> On 28 March 2011 21:27, Mark Phillips<[hidden email]> wrote:
>>>> Anyone else care to share their setup? Or have any questions about
>>>> what your dev setup should look like?
>>>
>>> For the general and common cases of testing - ie development,
>>> verifying application and cluster behavior in case of failures etc a
>>> simple "make devrel" works fine for me. I run Riak directly on my MBP.
>>>
>>> As others have pointed out - load testing and tuning for production is
>>> another story.
>>
>> What would have to be changed from the devrel version to get something where
>> you could simply add other nodes to migrate to production - and maybe
>> eventually remove the originals?
>>
>> --
>> Les Mikesell
>> [hidden email]
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak Development Environments

Alexander Sicular
In reply to this post by Mark Phillips-4
ring_creation_size is the key setting. Once you set it for a cluster it is a lock. Check the deets on the wiki.

cookie : erlang clusters share a cookie that they all need to have in the config. Make it sUp3rSp3c!aL!.

name : each host needs its own unique name. Something like riak@private-ip-addr will work.

That's about all you need to keep in mind.
------Original Message------
From: Les Mikesell
To: Alexander Sicular
Cc: [hidden email]
Subject: Re: Riak Development Environments
Sent: Mar 29, 2011 18:19

Yes, the question is, what would you need to change in that initial setup to
have something that would migrate, still running to some larger number of nodes
on other machines?  A larger number of partitions?


On 3/29/11 6:27 PM, Alexander Sicular wrote:

> "devrel" is not a version. It's a makefile setting that makes 3 installations
> with the correct cinfig settings such that the 3 can be joined together.
>
>
> @siculars on twitter
> http://siculars.posterous.com
>
> Sent from my iPhone
>
> On Mar 29, 2011, at 15:52, Les Mikesell <[hidden email]> wrote:
>
>> On 3/29/2011 5:33 PM, Mårten Gustafson wrote:
>>> On 28 March 2011 21:27, Mark Phillips<[hidden email]> wrote:
>>>> Anyone else care to share their setup? Or have any questions about
>>>> what your dev setup should look like?
>>>
>>> For the general and common cases of testing - ie development,
>>> verifying application and cluster behavior in case of failures etc a
>>> simple "make devrel" works fine for me. I run Riak directly on my MBP.
>>>
>>> As others have pointed out - load testing and tuning for production is
>>> another story.
>>
>> What would have to be changed from the devrel version to get something where
>> you could simply add other nodes to migrate to production - and maybe
>> eventually remove the originals?
>>
>> --
>> Les Mikesell
>> [hidden email]
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


Sent from my Verizon Wireless BlackBerry
_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com