How do I convert Julian date (stored as Integer) to proper date/timestamp format
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-21-2023 06:29 PM
How do I convert Julian date (stored as Integer) to proper date/timestamp format?
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
03-22-2023 04:32 AM
Hello Gurus,
I tried to convert the Julian date to a calendar date with below
from pyspark.sql.functions import from_unixtime, unix_timestamp
# Create a DataFrame with a Julian date column
df = spark.createDataFrame([('2459603',), ('2459604',), ('2459605',), ('112001',)], ["julian_date"])
# Convert Julian date to calendar date
df = df.withColumn("calendar_date", from_unixtime(unix_timestamp("julian_date", "yyyyDDD"), "yyyy-MM-dd"))
# Save the result to a new table
save(df)
and the result is as below :
which is not the expected result. Any pointers on this?
Regards,
Mritunjay Sinha
