You try to export data into an existing Hive table but the export job fails with the following exception:
java.lang.IllegalStateException: Existing table does not have the same columns as the current set of columns.
Names or data types of columns in the Datameer workbook don't match names or data types of corresponding columns in the target Hive table.
According to Datameer documentation Exporting to Hive and Data Field Types in Datameer,"Big Integers, Big Decimals, and Dates data types are treated differently in Datameer than in Hive and use a larger range of values. Because of this they are written as Strings into a Hive table on export".
When one uses an existing Hive table to export data into, it's required that column names in this table match appropriate column names from the export source. Also, data type mapping should be introduced according to the approach described in the Exporting to Hive section of the Data Field Types in Datameer documentation page.
- There is a workbook that has 2 columns -
- On export, Datameer's
Datedata types are mapped to Hive's
STRINGdata types accordingly.
- The table that the existing Hive table wants to import this data into should have two columns -
hive> describe test_table; OK name bigint time string
As of Datameer 6.4, it is possible to leverage a specific export mode for the Hive plugin that allows the export of date values into Hive TIMESTAMP, DATE or STRING data type columns. Please get in touch with Datameer support to get more details on this issue.
Please sign in to leave a comment.