Datameer partition filter expression to specify 2 Months

Follow

Charlie Bronson

• February 09, 2016 16:54

I need dynamic data partition for my datameer workbook. I need two months of data from today. So if i go to Advance partition filtering and specify something like $partition > TODAY()-60d && $partition <= TODAY() , it isn't getting me full 60 days of data. Couldn't see any examples on how to use MONTH in datameer documentation. Any help is appreciated. I use datameer version 4

0

• Official comment

  

  Joel Stewart

  • February 11, 2016 18:07

  Thanks for the clarification. This behavior is consistent with the design of the partitions. Monthly partitions are tagged for the first of the month (which was longer than 60 days ago) and so it was not included.

  For this particular use case, I'd recommend the following Advanced Partition Filter:

  $partition >= STARTOFMONTH(ADDTODATE(TODAY();"-2M"))

  To break this down, the formula's core starts with the same TODAY() calculation and then uses the ADDTODATE function to go to the date 2 months ago ("-2M"). After this, the STARTOFMONTH function is used to ensure that the first of the month is referenced to align with the partition definition. 

   

  Comment actions Permalink
• 

  Joel Stewart

  • February 09, 2016 22:04

  What granularity is the data partitioned over? 

  In the last 60 days, we would not capture the "Dec. 1, 2015" partition for example. Is that consistent with the data that is missing from your current implementation? 

  0

  Comment actions Permalink
• 

  Charlie Bronson

  • February 11, 2016 09:48

  Every second there are 100's of data records, total data was about 26GB, i missed the whole december data when i used the previously mentioned expression from last week's date.

  All i want to know is examples of partition filter using MONTH($Partition) . In this case is it best to use MONTH ? or should the previous expressed worked correctly ?

  0

  Comment actions Permalink
• 

  Charlie Bronson

  • February 12, 2016 10:22

  Great , Thanks Joel, Datameer Advanced partition filter documenation page has only basic examples. Datameer should include better examples and cover all use cases.

  0

  Comment actions Permalink

Please sign in to leave a comment.