Datameer Redshift Performance in 5.9.x

Problem

When using the native Redshift JDBC drivers provided by Amazon, export performance can be slow and large exports might hang or fail after running for some time.

Cause

This issue stems from how batch exports are handled and committed through the native Redshift JDBC driver.

Solution

The current workaround is the use the PostgreSQL driver. Download the driver and add it to Datameer using the steps in the How to Add Database Drivers documentation.

Note that commit/batch sizes need to be tuned after observing performance of the PostgreSQL driver. Typically, larger batch sizes are more performant, but this needs to be adjusted on a case-by-case basis.