Import all data IDs from a sheet (not just current)

Some of my workbooks are kind of inefficient

They process a lot of the same historic data every day

I can prevent this by setting filters so that they only look at a (more recent) time period

But by doing so the output will only include that time period and so I lose the comprehensiveness that running the workbook on all the data gives me.

I thought I'd found a way around this - through the workbook configuration screen, data retention, setting 'Never delete historical data'.

I am now able to see all the individual 'runs' of the workbook as different data ids under 'current data' on the workbook details page. I can also see them under 'history' when building an infographic.

However I can't see a way to import anything other than the latest set of data into another workbook - is there a means of doing this so that I can get a consolidated view of the data produced by the historic runs of the workbook?

Thanks

Stephen

Gido

September 01, 2017 15:25

Hello Stephen,

Right now the functionality you are looking isn't implemented. We've already a feature request open to have the possibility of using historical data directly in Workbooks. Please understand that I don't have have timeline or final release version when this will be implemented yet.

Stephen Waters

September 01, 2017 15:27

Hi Gido

Thanks for your reply

With that in mind, would my best bet be to export the daily output of the workbook to a location elsewhere (HDFS/Hive/etc), then data link to that? I guess it wouldn't be so quick given the data going in & out of Datameer but probably quicker than running the processing in full daily?

September 06, 2017 12:03

Right Stephen,

This is a possible approach and which is used in other setups too.

Import all data IDs from a sheet (not just current)

Comments

Didn't find what you were looking for?