What is the advantage of using luwak?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

What is the advantage of using luwak?

Joshua Partogi
From the wiki:
"Luwak works by breaking large files into blocks and storing each block as a separate Riak object"

But what does not get mentioned is the advantage of it. Does it make serving files faster? Or does it save more space in HD?

Thanks heaps for your assistance.

--
http://twitter.com/jpartogi

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: What is the advantage of using luwak?

Jeremiah Peschka
As I see it, there are a few advantages -

You get Riak's built-in redundancy. The various chunks of your file will be written in multiple locations across your cluster. Even in the event of a single server failure, you'll still be able to read data out of your luwak cluster.

Performance - I would think that there's a performance boost since you can, in theory, serve more files through the same luwak cluster than you could serve through a single file server (unless that file server itself were also clustered and attached to a SAN).

If you're storing text (or some other easily understood format) you can use MapReduce to perform queries across your files instead of having to write some kind of file system crawler.

Jeremiah Peschka
Microsoft SQL Server MVP
MCITP: Database Developer, DBA


On Tue, Jan 25, 2011 at 8:56 AM, Joshua Partogi <[hidden email]> wrote:
From the wiki:
"Luwak works by breaking large files into blocks and storing each block as a separate Riak object"

But what does not get mentioned is the advantage of it. Does it make serving files faster? Or does it save more space in HD?

Thanks heaps for your assistance.

--
http://twitter.com/jpartogi

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: What is the advantage of using luwak?

Thomas Burdick
It looks like luwak allows you stream data as well instead of just doing a put or get on the entirety of it. Thats a huge bonus if the files are very large such as video files.

Tom Burdick

On Tue, Jan 25, 2011 at 8:05 AM, Jeremiah Peschka <[hidden email]> wrote:
As I see it, there are a few advantages -

You get Riak's built-in redundancy. The various chunks of your file will be written in multiple locations across your cluster. Even in the event of a single server failure, you'll still be able to read data out of your luwak cluster.

Performance - I would think that there's a performance boost since you can, in theory, serve more files through the same luwak cluster than you could serve through a single file server (unless that file server itself were also clustered and attached to a SAN).

If you're storing text (or some other easily understood format) you can use MapReduce to perform queries across your files instead of having to write some kind of file system crawler.

Jeremiah Peschka
Microsoft SQL Server MVP
MCITP: Database Developer, DBA


On Tue, Jan 25, 2011 at 8:56 AM, Joshua Partogi <[hidden email]> wrote:
From the wiki:
"Luwak works by breaking large files into blocks and storing each block as a separate Riak object"

But what does not get mentioned is the advantage of it. Does it make serving files faster? Or does it save more space in HD?

Thanks heaps for your assistance.

--
http://twitter.com/jpartogi

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: What is the advantage of using luwak?

Justin Sheehy
Yes, Jeremiah and Thomas have hit on the reasons.  The biggest single
one is for streaming up a relatively static but very large content
object such as a video or a virtual machine image.

Luwak handles very large content much more nicely than if you stored
that same content in plain riak objects.

-Justin

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com