Virtual Stores at NASA

Unifying access to NASA datasets

⚠️ UNDER DEVELOPMENT ⚠️

Vision: NASA datasets accessible through a single entrypoint

Virtual stores deliver a single entrypoint to a dataset comprised of many files. For NASA datasets this enables:

• Less pre-processing to be “analysis-ready”.
• Users do not have to know about the underlying data format or storage location.
• Greater interoperability through a common API for reading, writing and analyzing complex and heterogeneous NASA datasets.

What are virtual stores and what do they enable?

Core to virtual store technology is lightweight metadata pointing to data byte ranges in existing files. Virtual store technology enables data users to access subsets of large scientific datasets without downloading, scanning, or pre-processing any files.

Benefits for data users

• Logical dataset access across a large number of files, without downloading any data, and regardless of local or in-region compute location
• Consistent access patterns across diverse data types

Benefits for data providers and NASA

• Cost savings through reduced egress and compute
• Providing analysis-ready access to existing archives without reformatting