Datameer partition filter expression to specify 2 Months

Comments

4 comments

  • Official comment
    Joel Stewart

    Thanks for the clarification. This behavior is consistent with the design of the partitions. Monthly partitions are tagged for the first of the month (which was longer than 60 days ago) and so it was not included.

    For this particular use case, I'd recommend the following Advanced Partition Filter:

    $partition >= STARTOFMONTH(ADDTODATE(TODAY();"-2M"))

    To break this down, the formula's core starts with the same TODAY() calculation and then uses the ADDTODATE function to go to the date 2 months ago ("-2M"). After this, the STARTOFMONTH function is used to ensure that the first of the month is referenced to align with the partition definition. 

     

    Comment actions Permalink
  • Joel Stewart

    What granularity is the data partitioned over? 

    In the last 60 days, we would not capture the "Dec. 1, 2015" partition for example. Is that consistent with the data that is missing from your current implementation? 

    0
    Comment actions Permalink
  • Charlie Bronson

    Every second there are 100's of data records, total data was about 26GB, i missed the whole december data when i used the previously mentioned expression from last week's date.

    All i want to know is examples of partition filter using MONTH($Partition) . In this case is it best to use MONTH ? or should the previous expressed worked correctly ?

    0
    Comment actions Permalink
  • Charlie Bronson

    Great , Thanks Joel, Datameer Advanced partition filter documenation page has only basic examples. Datameer should include better examples and cover all use cases.

    0
    Comment actions Permalink

Please sign in to leave a comment.