Twitter’s scale problems stem from the fact that the workload increases geometrically as every message people send is forwarded to each follower. The more followers people have, the slower twitter is. The follow-spammers just increase that load. Some of these spammers follow 25,000 people. That’s 25,000 extra message each time any of those 25,000 tweet. There are hundreds of follow-spammers, so that’s hundreds of thousands, maybe millions, of extra tweets.
Did Twitter Spammers Cause Downtime on Twitter?
May 30, 2008 – 12:54 am
Comments #
Twitter provided another update on their developer blog today. In this latest update, Twitter answered questions about their recent stability issues. One of the questions asked if there is anything that Twitter users can do to lighten the load.
Twitter’s response:
The events that hit our system the hardest are generally when “popular” users - that is, users with large numbers of followers and people they’re following - perform a number of actions in rapid succession. This usually results in a number of big queries that pile up in our database(s). Not running scripts to follow thousands of users at a time would be a help, but that’s behavior we have to limit on our side.
So who are the people who are running scripts to follow thousands of people at a time? I’m willing to bet that most of these people are spammers or over-zealous marketers who are trying to follow as many people as possible in a short amount of time. It’s not really clear from the blog post what Twitter is doing to limit these scripts but it’s good news that they are taking steps to limit this type of behavior.
Looking back on this, it looks like one of the key features that triggered Twitter’s massive growth over the past year - their open API - has also bitten them in the ass because it enabled things like the mass-follow scripts, which have caused stability problems. This left their platform wide open for abuse by “black hat” marketers who have been treating the Twitter user base like a free mass mailing list. And the Twitter community paid the price with frequent downtime. It’s pretty clear that whatever throttles and checkpoints Twitter had in place to limit their exposure to rogue applications was not enough. They learned this lesson the hard way.
This is a critical time for Twitter. Many users are moving over to other tools like FriendFeed, Jaiku, and Pownce while Twitter sorts out their technical problems. Is this a temporary move or are these users gone for good? I’m really hoping that this turns out to be a temporary glitch and not a repeat of what Friendster went through a few years ago.
Update:
Below is a screenshot of a tool that is designed to follow massive amounts of Twitter users. There are other many other tools similar to this one.
