The Problem with “Big Data”

If your business has geographically diverse offices -- whether they span a state, a country, or the globe -- then you inevitably have a need to share data across those locations. Since the Internet went mainstream, it has been the de facto way of exchanging information with each other, but it has always been limited in its capacity to do so. Yes, it works really well for networking apps, email, and document sharing, but what if you need to migrate a lot of data that is generated locally, such as a massive application database on a volume spanning terabytes? And what if you need to do this routinely… or even daily… because the data generated in one location drives productivity at the other? Moving several terabytes of data over a traditional WAN connection could require days, during which time the data may not be accessible to either party. This IS the problem with “big data”. What are the options? Assuming the problem won’t be solved on the WAN side anytime soon, the impetus is purely on the data side of the issue. Fortunately, there has been some innovation on this front in recent years.

Migration Meets Magic

data-migration.jpgOne modern storage company, Reduxio®, has devised a brilliant system that is built into the operating system of all their arrays. Internally they call it “NoMigrate™”, but resellers prefer to call it Follow the Sun Data. Whatever you call it, it does seem to solve the big-data migration problem within the constraints of the WAN. The approach involves a little slight-of-hand that can make it appear as if hundreds of terabytes were transferred across two remote storage systems in a matter of minutes. This is physically impossible of course, but NoMigrate relies on the principle that perception is reality. In other words, if you can see the migrated data locally, and your applications can access it as if it were local, then who is to say it isn’t local?

What’s the Trick?

This feat is accomplished by allowing remote applications full access to the data even while it is being migrated from one location to the other. Normally, applications must wait until the last byte of data is physically copied over, so you can clearly see the benefit in this approach. You are probably thinking, “Won’t performance be sluggish if the application has to access data remotely?” Well yes… it should be, but once again, innovation comes to the rescue! The system makes sure that the most-requested data is prioritized in the migration process, so thot data gets copied before cold data, and the application gets to access most of what it needs locally. The rate of data transmission is facilitated by the fact that data stored in the array is already deduplicated, compressed, and tiered for fast-access, so the data travels extremely efficiently over the WAN.

Get on Board

Implementing a modern storage system with these capabilities is not like installing a new app on your existing network. It is part of a tightly integrated storage area network that was designed from the ground up to to meet the demands of modern companies and solve their data issues. For this reason, you won’t see this data migration capability entering the market as an add-in feature from other vendors, because it would require them to scrap their core platform and start developing from scratch.

Fortunately, the process of migrating your existing storage onto a modern system is about as fast and easy as you would expect it to be.

It’s a New Day for Storage

There is a natural limit to moving data, based on the existing topologies and bandwidth limits of modern networking. And like the laws of physics, they cannot be broken. But just because Isaac Newton developed the Law of Gravitation, it didn’t stop others from inventing the airplane.

True innovation finds a way around the problems of the day, and in the process, changes the world. After decades of nearly stagnant development, there are modern storage vendors who are putting forth solutions that are changing the storage industry and allowing it to benefit from some fresh ideas and innovations.

To learn more about the amazing capabilities of modern storage solutions, contact the storage professionals at Core Evolution Group. If you are planning a storage upgrade or just want to learn how new technologies may change the way you think about what is possible, then please download the eBook titled The Essentials when Upgrading a Storage Solution

learn the basics of upgrading your storage solution