Edge Mass Data Transfer
Lots of data is generated in the field, at the network edge, and then needs centralised for analysis-based decision making. But when searching for a solution to get the job done, challenges abound. Find out what they are.
Today more than ever, data gets generated at the edge, out in the field. From IoT devices and sensors to other endpoints, all these things collect big data. And that big data is only going to grow. In fact, it’s estimated that edge devices will generate 175 ZB of data by 2025.
So how do you capitalise on all this data? Typically, you’re faced with tonnes of machine-generated data, which creates small files — millions of them. And it's unstructured. You don't know what's there; you don't know what's being captured. It's all over the place. That’s data sprawl.
When capturing data in the field, chances are you face the physical network limitations that make data difficult to access. Mobile networks are the most obvious solution. But maybe you don’t have any connectivity at all on a mountain or out on the ocean. And if you do have a mobile network available, it could be unreliable, limited or shared. Or the pipe is too small to give you the fast speeds you need. Or it’s just super expensive.
In this case, you usually end up using a physical enclosure. Then you face the challenge of getting it out to the field and back, over and over.
If your solution is a typical rack-mountable storage server, most aren’t suited for the purpose of moving your data from the field. A giant server or enclosure that needs to be secured and moved on a pallet with a forklift and then travel in a vehicle big enough to accommodate it won’t work if you’re shooting a documentary on a mountain. An appliance that’s not ruggedized won’t hold up if you’re collecting data in a harsh or wet environment. Even if you can get an appliance where you need it, it often won’t have the capacity you need.
There are cloud storage vendors who supply data transfer appliances, but you’ll probably be forced to use the vendor’s platform. Sometimes, these options only enable one-directional data flow, or they’re designed to keep your data in and not let it out. You then end up having to consolidate on whatever other storage devices you have and load it onto this vendor’s appliance. The appliance then travels to the facility designated by your vendor as your upload server. It's a one-run trip. You can't move that physical piece of hardware anywhere else.
If you’re looking for flexibility, you may not find it with a vendor-specific appliance.
Whether you have data moving around on physical devices or on mobile networks, you face security risks. Standard storage servers or appliances can be tampered with or stolen. And mobile networks may not meet the data security requirements with which you need to comply. This is true for government, healthcare, and other regulation-heavy industries, but also oil and gas companies can be targets of cyberattacks as well. Let’s face it, no matter which industry you’re in, having your data compromised doesn’t fly.
And then there’s always the cost. If you want to buy mobile storage appliances, you’ll deal with the initial capital expense along with ongoing hardware maintenance. You’ll need to purchase for maximum capacity, even if you only need it occasionally. And while there are managed service solutions, that could mean paying over a period of years, again, even if you only use the maximum capacity every now and then.
Until recently, there hasn’t been an option to pay only for what you use in either case.
The solution to your biggest data transfer challenges should provide you with ruggedized, high-capacity, truly mobile, and truly secure appliances — when you need them, where you need them, and at a predictable cost, whether you’re doing a one-time lift-and-shift or a weekly data transfer. Pay only for what you need with no vendor lock-in.
Curious about the solution that makes all this possible? Download our free ebook, “Solving Edge Storage Challenges” You’ll find out how government, media and entertainment, biomedical research, and other industries can collect, move, and store data more easily, faster, and more cost-effectively than ever.