Cloudera Navigator - Datameer

Follow

Nikhil Srinidhi

• September 22, 2016 18:55

Hi,

We just set up cloudera navigator on our sandbox environment. I'm curious to know if it's possible to use navigator to view data lineage of jobs run within datameer.

Example:

1) I have a workbook with data

2) I do groupby, groupcount transformations, and add a filter

3) I execute the job

 

I'd like to see the lineage in cloudera navigator showing the transformations (groupby, groupcount, filter). Is this even possible?

 

Thanks!

Nikhil

0

• 

  Joel Stewart

  • September 22, 2016 20:43

  The Datameer jobs are compiled in such a way that the individual functions that occur in your workbook definitions may not execute in the same way. Datameer optimizes the job based on the scope of the entire work that is expected. 

  You'll be able to view the same information with Datameer jobs as you would with any other YARN application, but you will not likely be able to see the individual functions along the way due to the compilation that takes place first. 

  0

  Comment actions Permalink

Please sign in to leave a comment.