Skip to main content

Relational DataStores - How good?

An interesting updated by the twitter team about the scalability probs they are facing : http://dev.twitter.com/2008/05/twittering-about-architecture.html.

 

They say that twitter was not exactly architected as a messaging system but for a content management system. But users started using it to a messaging system. Another example of an application used differently from what it was initially designed. Previously there were social networking sites, which were intended initially to help people to connect, but then started being used to convey full blown information.

 

Anyway back to the topic. The twitter blog does mention some points worth considering. The queries they ran seem to be executing for too long. Ya, I know what they r talking about. The example they cite “consider 1000 persons following a user”, and the user updates her status multiple times. Whew…. Must be quite a challenge for the team at twitter.

 

So one thing I picked up is “stay away from relational databases for real loads of data. But then what is the solution? Need to look around first thing tomorrow morning.

 

 

 

Comments

Post a Comment

Popular posts from this blog

My first problem with AJAX

I was using Callback (AJAX) to retrieve a page i had developed . Lets start all over, I was using AJAX to retrieve a part of the contents of a web page. Cool, simple, it downloaded without a pain, then I innerHTMLed it to a span tag. Worked fine, successful, clapped my hands etc.etc. Clicked on the button which had seemed to appear out of nowhere. And then, it all fell apart. The feared exclamatory symbol appeared on the status bar.... And for the newbies, thats an indication of a JavaScript error in IE. Of course, as usual i was all curses at my IE window......Anyway to make a long story short (i AM quite sleepy u know). the scripts which i seemed to be innerHTMLing to the span tag are not executed. And what i mean is anything between the script tags which have been innerHTMLed to that span tag are not executed. Of course i can see a ray of hope in the distance. I mean the distance till tomorrow morning. And just for kicks, check out a hi-fundoo flash file titled EPIC 2004 . And for...
4 comments
Read more

Imagine...an internet based world

Was reading "The Google Story", when i got this wierd idea. Consider a stripped down version of Linux, meant only for doing simple things such as browsing the internet, to read from the various drives (which is primarily meant only for storage of personal data), to play music and some single-player games. All the rest provided by the internet. Information searching/Chatting/Mailing/Documentation editing/journal maintenance/daily schedule maintenance/news/rss/shopping, etc. What else will you use for your computer for? Ok, actually i have been working on mainframes for some time now and am getting adjusted to the "dumb terminal" concept. Then i started associating the concept with scenario we are observing all over the net. As a matter of fact this will reduce the power of processing required on the client side also!!! Looks like the world is changing.... for the bettter, yup, i am sure. technorati tags: New internet age , dumb terminal , internet based world
3 comments
Read more

Changing JAVA_HOME on a HDP 1.1 Hadoop on Windows Setup

We were setting up a small cluster of  Hadoop for some requirements, and decided to go with the HortonWorks release of Hadoop. ( HDP 1.1 for Windows ). After installing, we realized that for some reason the Oozie service was not starting up . (we tried both the CLI command provided with the HDP installation, as well as trying to manually start the service via services.msc). Looking through the log files (<HDP 1.1 installation path>\oozie-<version>\Service directory, specifically the oozieservice.out.log file) we found the error -    and so realized that Oozie server is dependent on JDK. It does not work with only JRE. To correct this we uninstalled JRE, and installed JDK. We also manually modified the JAVA_HOME environment variable to point to the new installation path. And then Oozie server started working all fine. But all the rest of the services (hadoop jobtracker, hadoop datanode etc.) stopped starting. Checking the logs of the services, we fou...
2 comments
Read more