We have a custom script that build out source data on HDFS and then creates a Hive table from this location. When we create this metadata, we are using a mixture of upper and lower case characters "Variable_Name".
Datameer throws errors when attempting to read from these tables.
The cause is due to Datameer only having access to information returned through the Hive connection. Case sensitivity is not enforced at the Hive layer and as such it will always return values in lower case when queried. Datameer takes the returned details and attempts to tie into the data directly on HDFS, however now the path names do not match. We are looking for "variable_name" when the actual path on HDFS is "Variable_Name".
Ensure that when you create such paths on HDFS that you observe Hive standard practice and keep all path names restricted to lower case.
Please sign in to leave a comment.