Query business schema data from PySpark using the incorta_sql_pg() function

[dylanwan]

Introduction

The incorta_sql_pg() function lets you directly query business schema data in a PySpark MV without going through the old and cumbersome way. 

What you need to know before reading this article

Previously, you could use spark.sql() to query the data from physical schema data by creating temp views. 

Let's review the previously tedious steps: 

Go to Business Schema -> Explore Data -> SQL generation -> Copy the Reference SQL to Notebook -> Load data and create temp view -> Copy and edit the Reference SQL in spark.sql() function 

Incorta Version

Incorta 2022.10.0 or later

Let's Go

df = incorta_sql_pg("""
SELECT
    *
FROM
    BUSINESS_SCHEMA_NAME.BUSINESS_VIEW_NAME
""")

Example: 

Related Material

• Query physical schema data from PySpark using the incorta_sql() function
• Run SQL Queries from PySpark and Incorta Notebook