Can you read/write directly to hard disk from a spark job?
Hope that helps Fundamentally, no, you cannot use spark's native writing
APIs (e.g. df.write.parquet) to write to local filesystem files. When
running in spark local mode (on your own computer, not a cluster), you will
be reading/writing from your lo
Date : October 22 2020, 03:08 PM , By : Ellie Bibi