How do I convert Julian date (stored as Integer) to proper date/timestamp format

msinha8 — Wed, 22 Mar 2023 01:29:06 GMT

How do I convert Julian date (stored as Integer) to proper date/timestamp format?

Re: How do I convert Julian date (stored as Integer) to proper date/timestamp format

msinha8 — Wed, 22 Mar 2023 11:32:12 GMT

Hello Gurus,

I tried to convert the Julian date to a calendar date with below

from pyspark.sql.functions import from_unixtime, unix_timestamp

# Create a DataFrame with a Julian date column

df = spark.createDataFrame([('2459603',), ('2459604',), ('2459605',), ('112001',)], ["julian_date"])

# Convert Julian date to calendar date

df = df.withColumn("calendar_date", from_unixtime(unix_timestamp("julian_date", "yyyyDDD"), "yyyy-MM-dd"))

# Save the result to a new table

save(df)

and the result is as below :

which is not the expected result. Any pointers on this?

Regards,

Mritunjay Sinha