Use SparkSQL with SparkR

[suxinji]

Overview

In this recipe, you’ll learn how to run SQL queries from SparkR. After using the read() Incorta Notebook extension function to load data in Incorta Notebook, the data is in a SparkR DataFrame. You can use sql() to write SQL queries after creating a temporary view and a new DataFrame will be created.

You can use the display() Incorta Notebook extension function to explore the data. 

Solution

# load data from incorta
df <- read("Titanic_R.Titanic_train")
# create a temporary view
createOrReplaceTempView(df, "titanic_data")
# use sql query 
df_sql <- sql("SELECT * FROM titanic_data WHERE Survived = 0 AND age <= 19")
# show data 
display(df_sql)

After running the display in a separate paragraph, the data is shown in a table. You can use different visualization tools provided by Incorta Notebook to explore and visualize the data.