Virtual Raid 5 Internet Storage


[wonder] sent in an interesting proof of concept how-to on setting up a virtual raid 5 drive that uses free ftp servers to store data. The technique is a bit round-about, but he says it works pretty decently. The shares are mounted under windows with netdrive, then raided by FreeNAS under VMware. I haven’t tried it myself, but I’d probably try to do the same thing, but use LUFS and do everything under one operating system.

35 thoughts on “Virtual Raid 5 Internet Storage

  1. But why?

    In order for a RAID 5 array to work properly at least n-1 disks need to be available. FTP servers that you don’t own/control may not always be available and online. If you violate some part of a TOS (too much data transfer, etc) your account could be suspended or removed. If that happens to 2 of your accounts, ALL of your data is effectively lost.

    RAID 5 is often used so that you can combine a bunch of similarly sized disks into 1 big disk, and/or for the benefits on the data-reading side where multiple disks are all simultaneously fetching parts of a file.

    This seems to negate the read benefits, because the ftp servers are going to be too slow to have any real benefit, and the amount of space you get with any ftp account is so trivial compared to the cost of hard drives, combining 4 50GB ftp accounts into a virtual 150GB drive doesn’t seem to have much value.

    One use for this that could be HIGHLY valuable though, is in storing various bits of illegal data. If you are saving the data in a true RAID 5 striped fashion, then any 1 account would probably not have enough data to extract incriminating evidence. Multiple accounts would all need to be seized and re-combined into the RAID array in order for “The Authorities” to get any useful data.

  2. @brk:
    you first mentioned that RAID-5 ist fault tolerant only n-1. Sure. But you could use it as I figured out in the Picture 3 different and independant Accounts OR MORE!
    Your objection on the read speed I have also, but my tests speak a other language. It is fast enough for Video Streams (if you have enough bandwidth of course!) ald also for lets say backup some documents. My concept should be for single users and no prof. Workgroups or so ;-)
    You can get FTP-Space up to some GB for free, VMWare Server is free, NetDrive is for free, FreeNAS is free. Yes, you can also BUY some stuff (I had a cheap PC based NAS up to a SAN Meldody).
    Your last point is one of the main reasons for that. Nor for the illegal, but for security Stuff you want to have outside your office or home or so, so you can have it with you … over the web…

  3. now that brk mentions it, I’d like to see some sort sort of HDD self destruct system hack.

    (I’ve always thought it would be cool to rig up an HDD with a 20 gauge shotgun shell that would be triggered if a certain cord was removed from the back of the computer without first disabling something, ie, if someone tries to take your computer for evidence while your gone, all they’ll end up with is a drive case full platter shards. (thermite could work too, though it would definetly be a fire hazzard)

  4. Without a barrel of some kind a shotgun shell wouldn’t have enough power to punch thru the flimsy aluminum cover on most hard drives, much less frag the platters. Thermite would the do the job nicely but you would need a large amount of current to ignite it. Getting seriously off-topic here… :-)

  5. wolf –

    A shotgun shell would likely destroy the entire innards of your PC in that manner. Probably not exactly the solution you’re after :)

    A small alarm backup battery inside the computer case hooked to an electromagnet coil that can change intensity and polarity, while keeping the drive spinning, would probably be a more “implementable” solution. And at least then a power failure or someone tripping over a cable wouldn’t set off an unexpected explosion inside your computer.

    You would need the electromagnet to be powerful and in some sort of state of flux, just a strong, but static, magnetix field alone wouldn’t make the drive unrecoverable.

    Even better would probably be some sort of modified drive that had a metal scraper destroy the platter surfaces.

  6. My “But why?” comment is really asking what problem does this solve, or what function does it make better?

    It involves a fair bit of complexity and lots of software, for seemingly little gain.

    Many things are possible, and make for novel experiments. But if they don’t lead to some overall improvements, then the novelty wears off quickly (at least to me).

    All I was really asking, is what is the practical real-world application of this? Because if the point is just to get more disk space, it’s probably not worth the effort. If the point is to have some distributed fault-tolerant “storage in the sky”, then it’s pretty neat.

  7. I don’t see how this works… in order to implement a RAID 5 you’d need block-level access to the disks, which FTP isn’t giving you. How does the Windows FTP filesystem give you block-level access to an FTP account?

    Seeing as you’re already running linux in a VM, why not install FUSE under Linux, ftpfs, loop-mount some disk images off FUSE (sometimes possible… depends on FS implementation) then RAID the loop devices?

  8. I wonder if anyones actually verified the electromagnet approach? I’d imagine it would be even less practical than the thermite, mainly because it would require a pretty beefy backup battery in case they unplug the computer before it’s triggered.

    rob-

    your probably right about the shotgun shell, though that wouldn’t be very hard to test…

    As for the thermite, you could use a more volatile pilot mixture to set it off, but the main problem would be keeping the molten slag from melting through the bottom and sides of the case and potentially causing a house fire.

    The only commercial self destructing drive I’ve been able to find is this one

    http://www.deadondemand.com/products/enhancedhdd/

    which apparently self destructs by flooding the drives internals with an acid mist.

  9. Although interesting in concept, this is really a bit of an *anti* hakc. A hack is a simple (if a bit rough) solution to a (typically) complex problem, or one that normally requires a complex solution. This spins the definition on its head, using a baroque solution to a relatively simple problem that typically has a relatively simple solution (or range therof). Still interesting, just not a hack in my book.

    That said, I could see it being used, as another poster mentioned, in a security context rather than one which takes advantage of the other usual reasons to implement RAID-5, however to do so reliably it would need many more “spindles” as well as many, many extra parity disks per stripe. The alternative would be to do something akin to CD encoding where a massive number of bad bytes can be corrected. This all assumes of course that several of your ftp servers are down for whatever reason at the same time.

    Finally, regarding the self-destructing hdd, I’d think that a simple mechanical pucture device would suffice so long as it shattered the platter. Unless you are a truly high-profile prize, no law enforcement agency will spend the $$$ to reconstruct a physically broken platter.

  10. This seems like an interesting idea, unfortunately I am having trouble finding free ftp hosts, especially ones that don’t have file size limits etc that would make this useless. Anyone have any links handy?

  11. As far as using an electromagnet to auto-degauss your hard drive, that would likely fail. I’ve actually done some testing (see my link) and even a commercial “tape media” degausser doesn’t have the power to wipe a hard drive. Commercial hard drive degaussers are very expensive because, essentially, you are purchasing a small MRI machine. The lines of flux must be very intense to even start to affect the tiny bits written on the platters. After my testing the most efficient way I could think of to wipe the drive was to introduce a corrosive element into the cavity that would remove the ferromagnetic material from the platters, aka turpentine or something similar.

    http://www.packetsniffers.org/bitbucket/degaussing.html

  12. @Exparrot: To get Access to the FTP you need (as in the text mentioned) NetDrive.
    The Thing with GMail FS was that under Windoof you get only a Drive “without a Drive Letter” in the network Drive Folder. So netdrive does the job to connect FTP to a localy mapped drive (matt mentioned it well). So you have locally 3 (or more) Drives wich can be used für the RAID. an THEN you have 1 Drive wich could be encrypted or could be used for Backup or whatever. Use your Imagination :-)

  13. this is a AWESOME idea but i wish it was simple and could be implemented by a simple 50 gmail accounts mounted using shell extension and then software raid 5 and this would defeat the 10mb limit do to the stripping if im correct right? and you would get 313GB of space (and rising) and the increased transfer speed due to multiple accounts and defeating the bandwidth cap since each account would not be used alot since the bandwidth is shared.

    I am very interested in this idea if you have any ideas please comment and tell me what you think!

  14. I can fix the self distructing idea for you… The key word is aluminum… hd’s are mostly aluminum. So, What chemical agent is cheap and readily available almost anywhere? Muritic Acid!!!! and there are lots of ways to dispence it to/on your hd to distroy it!!! mist, funnel, dip….. you get the idea!!

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.