on 04-12-2022 09:00 AM
In this recipe, you'll learn how to convert R DataFrame into Spark DataFrame. You can use as.DataFrame() or createDataFrame()to convert R data frame into Spark DataFrame.
We assume that you would like to use R to transform the data and the R DataFrame df_r_select was already created:
The df_spark is now a Spark DataFrame that can be saved as a MV.
# convert R data frame into a Spark data frame
df_spark <- as.DataFrame(df_r_select)
# convert R data frame into a Spark data frame - method 2
df_spark <- createDataFrame(df_r_select)