Jul 18, 2022
0
Level 1
Load CSV file with Spark
I'm new to Spark and I'm trying to read CSV data from a file with Spark. Here's what I am doing :
sc.textFile('file.csv') .map(lambda line: (line.split(',')[0], line.split(',')[1])) .collect() I would expect this call to give me a list of the two first columns of my file but I'm getting this error :
File "", line 1, in IndexError: list index out of range
although my CSV file as more than one column.
Please or to participate in this conversation.