YouTomb is a research project designed by the MIT Free Culture group to track video take downs on YouTube. To succeed, the team needed to track every single video on YouTube… which is close to impossible. Instead, they built several “explorer” scripts to track what videos were interesting. One explorer tracks all of YouTube’s lists: recommended, featured, most active, and more. Another explorer picks up every video submitted to YouTube, and a third crawls Technorati.

The explorers just find the videos; a separate group of scanner scripts checks the current status of videos. It checks both the new videos and ones that have been killed to see if they return. YouTomb archives every video it finds. They display the thumbnail of the video under fair use, but they’re still determining whether they can display each video in full.

YouTomb is tracking a little more than 282,000 videos right now and maintain a public MySQL snapshot for anyone that wants to build their own tools. The code is also open source. They’ve been archiving all their historical data too, all 70 million rows of it.

They’ve started trending country censorship. Germany, Poland, and France all have hate speech bans, so any video with a swastika can’t be viewed there. Thailand blocks anything that impugns the king. Crank That is blocked in 200+ countries.

YouTomb got a lot of press when it was initially released. The team feels that this is the result of a clear interface. They encourage others to take the time to present data clearly. As a final note, they pointed out that you can always file a DMCA counterclaim to get your videos restored.