Created
February 11, 2011 22:48
-
-
Save PharkMillups/823211 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
18:02 <stodge> Does write performance degrade as the size of the database increases? | |
I'm talking millions of writes. | |
18:03 <justinsheehy> stodge: generally, no. bitcask is designed to have roughly constant | |
write time with regard to database size. | |
18:03 <stodge> Thanks | |
18:04 <stodge> I'm researching nosql solutions for a data warehouse. I want to | |
dump data into it and then use map/reduce to retrieve data later for statistics etc | |
18:05 <stodge> I need something that will happily whirr away in the background | |
storing data with writes every second | |
18:07 <justinsheehy> so far, sounds like a plausibly riak-shaped kind of thing -- | |
though there are some analytics systems that may be better optimized for the data warehouse | |
style querying, riak can do some of that and can certainly consumer lots of continuous writes. | |
18:08 <stodge> ok | |
18:09 <stodge> I also want to provide 1..n web servers to let users access and query | |
historical data, so I need a distributed db | |
18:10 <stodge> I have a requirement that may not be suitable to nosql solutions. | |
A lot of my data is essentially a snapshot; a customer can wind the clock back and retrieve | |
data every second starting from say 7-Feb-2011 08:00:00 and every second the browser will | |
request the current snapshot at that time. | |
18:11 <stodge> The customer can recreate the 'system' state at 08:00 on the 7th and | |
can playback the system state at 1s increments. | |
18:11 <justinsheehy> well, there's nothing built-in to riak (or most systems I know) | |
to do exactly that, but there are different ways to build it | |
18:12 <justinsheehy> I have to run, hit up the mailing list if you don't find people | |
here sometime when you need help. | |
18:12 <stodge> Ok thanks, will do | |
18:19 <Wuher> stodge: No idea if it would really be suited for your needs, | |
but have a look at logstash | |
18:19 <stodge> Will do thanks |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment