Split Data import by COALESCE Column?
Hi,
I'm wondering what option I have when I want to enable concurrent importing from a DB2 connection where I have written a custom query. Basically it is a FULL OUTER JOIN between two tables containing the same but opposite variables (one is for online usage by customers by month and the other is offline usage). I use COALESCE to combine the customer keys and the time keys in the Custom SQL, and I would like to enable concurrent importing by the combined time key from both tables.
My solution was to put this into a sub query and SELECT * from that sub-query and put the time key in $$. This runs, but at the end of the job all records fail.
Am I doing something wrong? Is there another way of doing this? (without splitting it up into two tasks)
Thanks in advance
-
Artur, thanks for your question. The only additional consideration that comes to my mind is to create a View in the DB2 system if it is available. I haven't tested this directly as I don't have quick access to a suitable system to replicate -- this idea may not be compatible with DB2 directly or may not be compatible with a particular Datameer version. In short, if you can create a view that has the columns pre-defined as a join you may be able to select that as the key for splitting when defining the Import Job or Data Link within Datameer.
Speaking more generally, I would recommend splitting the task in two and performing the join within a Datameer workbook downstream. This allows your users to be more flexible when selecting the data set. They could select the left set, the right set, or a pre-joined set from a workbook.
Please sign in to leave a comment.
Comments
1 comment