Riak as Binary File Store

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Riak as Binary File Store

Praveen Baratam
Hello Everybody!

I have read abundantly over the web that Riak is very well suited to store and retrieve small binary objects such as images, docs, etc.

In our scenario we are planning to use Riak to store uploads to our portal which is a Social Network. Uploads are mostly images with maximum size of 2 MB and typical size ranges between few KBs to few 100 KBs.

Does this usage pattern fit Riak? What are the caveats if any?

Thank you!

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Justin Sheehy
Hi, Praveen.

Nothing about what you have said would cause a problem for Riak. Go for it!

Justin



On May 29, 2012, at 8:36 AM, Praveen Baratam <[hidden email]> wrote:

> Hello Everybody!
>
> I have read abundantly over the web that Riak is very well suited to store and retrieve small binary objects such as images, docs, etc.
>
> In our scenario we are planning to use Riak to store uploads to our portal which is a Social Network. Uploads are mostly images with maximum size of 2 MB and typical size ranges between few KBs to few 100 KBs.
>
> Does this usage pattern fit Riak? What are the caveats if any?
>
> Thank you!
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Shuhao Wu
In reply to this post by Praveen Baratam
It'll be interesting if you can write a filesystem on top of Riak.

That would be a cool project to see on github :P

Shuhao


On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam <[hidden email]> wrote:
Hello Everybody!

I have read abundantly over the web that Riak is very well suited to store and retrieve small binary objects such as images, docs, etc.

In our scenario we are planning to use Riak to store uploads to our portal which is a Social Network. Uploads are mostly images with maximum size of 2 MB and typical size ranges between few KBs to few 100 KBs.

Does this usage pattern fit Riak? What are the caveats if any?

Thank you!

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Alvaro Videla-2
Like this perhaps: https://github.com/johnthethird/riak-fuse *cough* *cough*

On Tue, May 29, 2012 at 2:49 PM, Shuhao Wu <[hidden email]> wrote:
It'll be interesting if you can write a filesystem on top of Riak.

That would be a cool project to see on github :P

Shuhao


On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam <[hidden email]> wrote:
Hello Everybody!

I have read abundantly over the web that Riak is very well suited to store and retrieve small binary objects such as images, docs, etc.

In our scenario we are planning to use Riak to store uploads to our portal which is a Social Network. Uploads are mostly images with maximum size of 2 MB and typical size ranges between few KBs to few 100 KBs.

Does this usage pattern fit Riak? What are the caveats if any?

Thank you!

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Vlad Gorodetsky
I've read somewhere here on the mailing list that storing blobs that
are more than 50KB isn't recommended.
Is that correct? If so, is it something specific to storage backend?

~Vlad

On Tue, May 29, 2012 at 3:51 PM, Alvaro Videla <[hidden email]> wrote:

> Like this perhaps: https://github.com/johnthethird/riak-fuse *cough* *cough*
>
>
> On Tue, May 29, 2012 at 2:49 PM, Shuhao Wu <[hidden email]> wrote:
>>
>> It'll be interesting if you can write a filesystem on top of Riak.
>>
>> That would be a cool project to see on github :P
>>
>> Shuhao
>>
>>
>> On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam
>> <[hidden email]> wrote:
>>>
>>> Hello Everybody!
>>>
>>> I have read abundantly over the web that Riak is very well suited to
>>> store and retrieve small binary objects such as images, docs, etc.
>>>
>>> In our scenario we are planning to use Riak to store uploads to our
>>> portal which is a Social Network. Uploads are mostly images with maximum
>>> size of 2 MB and typical size ranges between few KBs to few 100 KBs.
>>>
>>> Does this usage pattern fit Riak? What are the caveats if any?
>>>
>>> Thank you!
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Mark Phillips-4


On Tue, May 29, 2012 at 12:51 PM, Vlad Gorodetsky <[hidden email]> wrote:
I've read somewhere here on the mailing list that storing blobs that
are more than 50KB isn't recommended.
Is that correct? If so, is it something specific to storage backend?


Riak can probably handle objects up to about 10MBs. That said, depending on your hardware and network, the functional limit is probably smaller than that. Also, keep in mind n_val when writing large values. (i.e., 5MB with an n_val of 3 = 15MB across the wire).

Mark  
 
~Vlad

On Tue, May 29, 2012 at 3:51 PM, Alvaro Videla <[hidden email]> wrote:
> Like this perhaps: https://github.com/johnthethird/riak-fuse *cough* *cough*
>
>
> On Tue, May 29, 2012 at 2:49 PM, Shuhao Wu <[hidden email]> wrote:
>>
>> It'll be interesting if you can write a filesystem on top of Riak.
>>
>> That would be a cool project to see on github :P
>>
>> Shuhao
>>
>>
>> On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam
>> <[hidden email]> wrote:
>>>
>>> Hello Everybody!
>>>
>>> I have read abundantly over the web that Riak is very well suited to
>>> store and retrieve small binary objects such as images, docs, etc.
>>>
>>> In our scenario we are planning to use Riak to store uploads to our
>>> portal which is a Social Network. Uploads are mostly images with maximum
>>> size of 2 MB and typical size ranges between few KBs to few 100 KBs.
>>>
>>> Does this usage pattern fit Riak? What are the caveats if any?
>>>
>>> Thank you!
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Reply | Threaded
Open this post in threaded view
|

Re: Riak as Binary File Store

Praveen Baratam
Thank you everybody for the feedback.

I am planning to write binary data to Riak from the application server which handles upload. On the read side I am planning to use Apache with mod_proxy and mod_rewrite to directly fetch from Riak and serve the files without any other application server / scripting overhead.

I feel this is the least overhead path to using Riak as a file store.

Comments please.

On Thu, May 31, 2012 at 2:48 PM, Praveen Baratam <[hidden email]> wrote:
Thank you everybody for the feedback.

I am planning to write binary data to Riak from the application server which handles upload. On the read side I am planning to use Apache with mod_proxy and mod_rewrite to directly fetch from Riak and serve the files without any other application server / scripting overhead.

I feel this is the least overhead path to using Riak as a file store.

Comments please.


On Tue, May 29, 2012 at 10:28 PM, Mark Phillips <[hidden email]> wrote:


On Tue, May 29, 2012 at 12:51 PM, Vlad Gorodetsky <[hidden email]> wrote:
I've read somewhere here on the mailing list that storing blobs that
are more than 50KB isn't recommended.
Is that correct? If so, is it something specific to storage backend?


Riak can probably handle objects up to about 10MBs. That said, depending on your hardware and network, the functional limit is probably smaller than that. Also, keep in mind n_val when writing large values. (i.e., 5MB with an n_val of 3 = 15MB across the wire).

Mark  
 
~Vlad

On Tue, May 29, 2012 at 3:51 PM, Alvaro Videla <[hidden email]> wrote:
> Like this perhaps: https://github.com/johnthethird/riak-fuse *cough* *cough*
>
>
> On Tue, May 29, 2012 at 2:49 PM, Shuhao Wu <[hidden email]> wrote:
>>
>> It'll be interesting if you can write a filesystem on top of Riak.
>>
>> That would be a cool project to see on github :P
>>
>> Shuhao
>>
>>
>> On Tue, May 29, 2012 at 8:36 AM, Praveen Baratam
>> <[hidden email]> wrote:
>>>
>>> Hello Everybody!
>>>
>>> I have read abundantly over the web that Riak is very well suited to
>>> store and retrieve small binary objects such as images, docs, etc.
>>>
>>> In our scenario we are planning to use Riak to store uploads to our
>>> portal which is a Social Network. Uploads are mostly images with maximum
>>> size of 2 MB and typical size ranges between few KBs to few 100 KBs.
>>>
>>> Does this usage pattern fit Riak? What are the caveats if any?
>>>
>>> Thank you!
>>>
>>> _______________________________________________
>>> riak-users mailing list
>>> [hidden email]
>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>>
>>
>>
>> _______________________________________________
>> riak-users mailing list
>> [hidden email]
>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>>
>
>
> _______________________________________________
> riak-users mailing list
> [hidden email]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com




_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com