Export to Hive: Existing Table Does Not Have the Same Columns as the Current Set of Columns

Problem

You try to export data into an existing Hive table but the export job fails with the following exception:

java.lang.IllegalStateException: Existing table does not have the same columns as the current set of columns.

Cause

Names or data types of columns in the Datameer workbook don't match names or data types of corresponding columns in the target Hive table.

Solution 

According to Datameer documentation Exporting to Hive and Data Field Types in Datameer,"Big Integers, Big Decimals, and Dates data types are treated differently in Datameer than in Hive and use a larger range of values. Because of this they are written as Strings into a Hive table on export".

When one uses an existing Hive table to export data into, it's required that column names in this table match appropriate column names from the export source. Also, data type mapping should be introduced according to the approach described in the Exporting to Hive section of the Data Field Types in Datameer documentation page.

Example

• There is a workbook that has 2 columns - name (Integer) and time (Date).
• On export, Datameer's Integer and Date data types are mapped to Hive's BIGINT and STRING data types accordingly.
• The table that the existing Hive table wants to import this data into should have two columns -  name (BIGINT) and time (STRING).

hive> describe test_table;
OK
name                bigint                                      
time                string    

As of Datameer 6.4, it is possible to leverage a specific export mode for the Hive plugin that allows the export of date values into Hive TIMESTAMP, DATE or STRING data type columns. Please get in touch with Datameer support to get more details on this issue.