Enable real-time, zero-copy API data access in Incorta—eliminating ingestion, reducing complexity, and accelerating insights.

[WOB]

🚀 Enhancement Request: Real-Time API Virtual Layer (Zero Data Copy)

🔍 Problem

Current API connectors (Salesforce, ServiceNow, Jira) require data ingestion into schemas, leading to:

• Stale dashboards (refresh delays)
• Duplicate data across systems
• Ongoing load/configuration overhead

---

💡 Proposed Solution

Introduce a Virtual API Layer that enables:

• ✅ Real-time access via APIs (no ingestion)
• ✅ Query-time data retrieval (virtual tables/views)
• ✅ Optional caching for performance

---

🎯 Business Value

• Real-time insights → no stale dashboards or AI outputs
• Zero data duplication → reduced storage & governance effort
• Faster time to value → no pipelines or load management
• Simpler architecture → less ETL, fewer moving parts

---

🧩 Use Cases

• Salesforce → live pipeline & deal insights
• ServiceNow → real-time incident/SLA visibility
• Jira → up-to-date sprint & backlog analytics
• Other APIs

---

⚖️ Why It Matters

Incorta already delivers near-real-time data updates, but this takes it further to true zero-copy, real-time access—aligned with modern data and AI architectures.

 

[Adetweiler]

 Release Notes Incorta 2025.12.0 (Preview)

Real-time data access

Query data directly from Snowflake, providing instant access to source data for ad-hoc analysis
No-copy read from Delta Lake (S3, ADLS, GCS), reducing storage overhead and operational complexity

I think your idea they had a premonition and put it in their last release.... does this get you there?

[JoeM]

Adding @mhelmy  and @emanmoussa  and @anurag  for review. 

@WOB -  As @Adetweiler  mentioned, we have no-copy ingestion today for delta lake tables. These benefit users who want to blend internal datasets with external datasets.

In an upcoming release, we are also introducing AI applications. With these AI applications, users can query API endpoints through the context of the application. Do you need the virtual data sets persisted outside of a presentation layer @WOB ?