I have an import job using a hive connection.
The table is stores as parquet and is using GZIP compression.
Recently the schema for the backing table change, some columns were removed.
Now from a hive shell I can query this table without any issue.
But the import job broke.
When trying to recreate the import job to reflect the new schema, on the "Define Fields" tab.
The Message box reads.
"Not all preview records parsed successfully"
And a handful of errors all reading
"java.lang.UnsupportedOperationException: Cannot inspect org.apache.hadoop.io.LongWritable"
I can still save the job, but when I run it I get 0 records In, 0 out and 0 dropped.
Another thing to note is that, if I copy entire contents of the table to a temp table and back, I
can create the import job no problem.
So it would seem that datameer is having some issue reconciling the hive and parquet schema's
Has anyone seen this kind of problem before?
Please sign in to leave a comment.