Making today’s data ready for tomorrow’s AI

In manufacturing, it’s common to use machine vision and sensor readings to inspect products coming off a production line and verify that they meet strict specifications. Like most data, that information is generated for a specific purpose.

What's harder to know is how that same data might become useful later. Years from now, a new AI model may detect a failure pattern in production records that nobody knew to look for when the data was collected. Visual data captured for one application may become useful to computer vision models. Operational logs may improve predictive maintenance. Scientific observations may yield new insights as analytical techniques advance.

AI is creating new ways to extract value from existing data. But realizing that value depends on whether the data remains understandable and usable when those new applications emerge.

What makes data ready for AI

AI systems working across enterprise data need information to establish what the data represents, where it came from, how it relates to other information and how it can be used.

That remains a real challenge. In Seagate's Data Infrastructure Readiness Report, 53% of respondents identified data quality and readiness as a leading challenge to AI deployment.

No single technology solves the problem. Different parts of the data architecture play different roles.

Metadata can describe characteristics of stored data. Provenance can help establish its origins, while version information can help distinguish one iteration from another. Classifications and permissions help govern access and use. Data catalogs and other management tools can help people and applications locate relevant information.

The exact mix will vary with the data and the organization. However, the practical requirements are fairly constant: retain data worth keeping, preserve the information needed to interpret it, make it discoverable, maintain appropriate controls and store it at a sustainable cost. Together, those choices determine how much of an organization’s data estate remains useful as new AI applications emerge. 

The payoff comes when someone finds a new use for older data, because more of the data the organization has already created can go back to work. That starts with information that can be located and evaluated without having to piece together where it came from, what it represents and whether it can be used.

The missing layer: Keeping data available

Object storage plays a specific role in maintaining that data estate, particularly as organizations accumulate images, video, documents, telemetry, logs and other forms of unstructured data. Objects can be stored with metadata that helps describe them, while other information and controls may reside in catalogs, governance systems, applications and identity and access-management systems.

These systems serve different purposes. Object storage keeps large datasets and associated metadata available while the systems around that data help people and applications find, manage, govern and use it.

As organizations retain more data for longer periods, the economics of that storage become increasingly important. 

Determining what you keep

A decision that reduces storage costs today can remove data that could have supported a future application. Tiered architectures give organizations more flexibility by matching data with storage suited to its requirements. Frequently accessed data may require higher-performance storage, while data accessed less often can remain available on storage optimized for scale and cost.

For large volumes of data that need to be retained over time, HDDs can help make the cost of that retention practical. That gives organizations more room to preserve information whose value may extend beyond its original use. The storage economics can help determine how much potentially valuable data remains available when a new use emerges.

Put more data to work

An AI-ready data architecture starts with choices made long before the next use for the data is known: what to retain, what context to preserve and how to store growing volumes economically. By keeping valuable data usable for longer, more of it will be available when the next opportunity arrives.

READ: Explore how cloud and AI workloads are redefining primary storage.