Problem
When you attempt to create an import job, you receive the following error upon pointing to your intended Parquet files:
Can not parse input: Can not read value at 1 in block 0 in file hdfs://<path_to_file(s)/<file_name>.parquet.snappy
Cause
The above error is typically presented when Datameer is unable to read the target file correctly. For additional details, please review Datameer's "conductor.log" for the full stack trace. Included in this log should be enhanced details such as mismatched schemas or invalid entries.
Solution
Attempt to read these files into an external application. This helps to validate the integrity of your target files.
If found to be corrupt, attempt to recreate your source data or consider pointing Datameer directly to your source so that Datameer can create the parquet files as part of its normal flow.
Comments
0 comments
Please sign in to leave a comment.