Unified Data Access Across Hybrid Cloud

Shared Metadata and Cost-Effective Storage: watsonx.data uses the Apache Iceberg open table format, shared metadata, and cost-effective object storage to maintain a single copy of data accessible by multiple query engines. This approach enhances collaboration, reduces data duplication, and mitigates security and governance risks by minimizing the number of data copies required for various users and tools.

• Powerful Query Engines: watsonx.data integrates with multiple query engines, including:

• Presto: An open-source, distributed SQL query engine designed for fast, reliable, and efficient analytic queries on large datasets.

• Apache Spark: An open-source, unified analytics engine for large-scale data processing and machine learning pipelines, offering capabilities for data engineering, data wrangling, data science, and machine learning with data parallelism and fault tolerance.

Support for External Engines: External engines that support the Iceberg open table format can also interact directly with data in watsonx.data object storage. For instance:

• Db2 and Netezza: Enhanced to read from and write to the Iceberg format, enabling them to participate in the lakehouse ecosystem.
• IBM Data Gate for watsonx (2Q24): Facilitates the replication of Db2 for z/OS, IMS, and VSAM data from IBM Z (mainframe) into Iceberg tables in object storage. This integration allows enterprises to build AI models with transactional data from mainframes, fostering revenue growth, productivity enhancement, and cost management.