I am trying to create a table reading the json object and dynamically getting the table columns from the json objects. I am able to display the data in notebook and getting the expected result but unable to save the schema when I am dynamically getting the table columns. If I hard code the values I am able to save it. Even added a property spark.dataframe.sampling.enabled to false.
Failed to generate schema
ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
