Datameer automatically converts dates to another time zone during the import

Question

Datameer application server's time zone is UTC. Datameer application is configured to use the ET time zone via system.property.das.default-timezone=America/New_York. When I import a parquet file from HDFS, Datameer changes the time from e.g. 2020-07-07 08:10:30 to 2020-07-07 04:10:30. What is the reason behind this?

Answer

• Datameer application is configured to use timezone different from the application server via a configuration property system.property.das.default-timezone=America/New_York. This means that while the application server is running with the UTC time zone, Datameer is configured to display ET time in UI. This is implemented for users' convenience to cover situations when the application server and actual users are located in different time zones.
• With this configuration, users see ET time in the following cases:
  • At the top right corner of the Datameer page.
  • Jobs execution time at System Dashboard and Job History sections of the Admin Tab.
  • In JobTrace files (e.g. job.log) loaded via UI.
• However, it is important to understand that the ET timezone time is only an abstraction layer and Datameer automatically converts timestamps to the desired time zone for users' convenience.
  
  
• When one configures an ImportJob, Datameer automatically adjusts the time zone for date/timestamps columns from UTC to ET, assuming that in case one had configured a custom timezone for Datameer, it wants to see date values displayed in this time zone.
• As well, Datameer highlights the time zone in which values are displayed at the ImportJob configuration step.

To have date values displayed in the UTC time zone, one could either consider changing Datameer application timezone to UTC or convert values to the desired timezone already in a Workbook, via SHIFTTIMEZONE function.