Fwd: Limit to the number of links an object can have?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Limit to the number of links an object can have?

Sean Cribbs-2
Forgot to reply-all.

Begin forwarded message:

From: Sean Cribbs <[hidden email]>
Date: March 23, 2010 1:02:11 PM EDT
To: Magnus Enarsson <[hidden email]>
Subject: Re: Limit to the number of links an object can have?

Magnus,

This goes down deep into the way mochiweb is implemented, in that it opens the TCP socket with a receive buffer of 8192 bytes.  It looks like if the header doesn't fit into that size, it will fail to parse the request.  This _might_ be solved by sending multiple Link headers instead of a single one.

I don't have time today to test this theory, but it should be easy to do so.

Sean Cribbs <[hidden email]>
Developer Advocate
Basho Technologies, Inc.
http://basho.com/

On Mar 23, 2010, at 12:04 PM, Magnus Enarsson wrote:

Hi,

Besides the scaling and availability I find the linkage feature of
Riak to be one of the most exciting ones. To me it is a really killer
feature as it makes it possible to model relations, and fetch objects
based on those relations in one single query!

However, the number of links an object can have seems to be somewhat
limited, and that make the feature much less usefull:

At https://wiki.basho.com/display/RIAK/Riak+compared+to+Neo4J I read
"In a pinch, this can substitute as a lightweight graph database, as
long as the number of links is kept reasonably low; think dozens, not
thousands."

On the other hand, at https://wiki.basho.com/display/RIAK/Links I read
"There is no artificial limit to the number of links an object can
have. But, as adding links to an object does increase that object's
size, the same guidelines that apply to your data should also apply to
your links: strike a balance between size and usability."

When I try to create an object with many links using Ripple Ruby gem,
I get a 400 error after adding a little more than 70-250 links
depending on key sizes. The limit seems to be when the Link header go
beyond 8200 or so characters.

I understand that many links will make it more expensive to handle the
object, but it is still better to follow many links from one object to
another than to have to do a full bucket scan with map/reduce. I would
not expect a tight limit on links on a Big Data database like Riak.

So what are really the limits with the number of links you can have
for one object? Are the limits that I encounter related to Ripple
rather than Riak? How can I go around this limit?

Kind regards,
Magnus

_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com



_______________________________________________
riak-users mailing list
[hidden email]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com