Import Job / Data Link problem - Help please...
Hello, I am trying to perform regular import jobs from S3 (connection is established), these are weblogs where the Logs are stored in Folders under S3 by the %year%, %month%, %day%, and %hour%.
How do I schedule a regular pull from S3, where the weblogs in a day, or a particular month is sitting at different workbooks, or different sheets within a workbook?
-
Hello Anson.
Any Datameer artifact except a FileUpload could be scheduled for execution at a certain time - just open an artifact's configuration, go to the Schedule tab, and set the Trigger according to your requirements. Please refer to the Scheduling Job Runtime section of our documentation for more details.
An ImportJob or a DataLink could be used as a source for one or multiple Workbooks. To have "a data for a particular month is sitting at different workbooks, or different sheets within a workbook" you need to create each Workbook or each Sheet within a single Workbook manually.
To have a separate Workbook for each month's data you could create a single partitioned DataLink and then create a separate Workbook for each month with an appropriate partition filter that will narrow the data Workbook will process to a certain month only.
In theory, you could add the same DataLink with different partition filters into one Workbook as a separate source Sheets, but this should be tested based on further data transformation you need to apply.
Please sign in to leave a comment.
Comments
2 comments