Decision tree algorithm

Follow

enaven

• December 21, 2015 11:59

What algorithm does datameer use for 'decision tree' feature under smart analytics?
what method does it employ to prevent overfitting?
does datameer provide option to specify 'stopping rules' for decision tree classification?

1

• 

  Jason Arrigo

  • December 21, 2015 16:12

  Hello enaven!
  The Smart Analytics Decision Tree module uses a Classification and Regression Trees (CART) algorithm. Datameer offers pruning and validates against a fraction of the sample data when building the Decision Tree, but doesn't offer a way to provide stopping criteria.
  The point of the Smart Analytics modules are to offer pretty powerful algorithms to business analysts, out of the box. If there is a need to have more advanced functionality, you could write what you need in R or PMML and use the appropriate plugin to integrate it into your Datameer workflow.
  I hope this helps!
  - Jason

  1

  Comment actions Permalink
• 

  Gido

  • December 28, 2015 10:48

  Hi Enaven,

  The decision tree algorithm prevents overfitting by pruning the tree. Pruning is done on a validation set. The size of the validation set is specified by the user in the advanced tab. When pruning is switched off, then the only way to reduce overfitting is by limiting the maximum tree depth, though this is a less accurate way of avoiding overfitting.

  I assume you mean "stopping rules" when to stop pruning? The rule is: prune as long it improves the accuracy of the tree on the validation set. Pruning is stopped when accuracy on validation set gets worse.

  0

  Comment actions Permalink
• 

  enaven

  • January 11, 2016 08:53

  Hi Jason,

  Thanks for the help. Could you provide me the material for using the plugin in datameer to incorporate advanced functionalities(like loops). I have never used a plugin before so I am not acquainted with the technicalities of it.

  Thanks

   

  0

  Comment actions Permalink
• 

  Gido

  • January 11, 2016 10:04

  Hello Enaven,

  Some public available information regarding the R plugin you will find under How to Install the R Plug-in. 

  After installing R within the cluster and the R plugin in Datameer you have new functions available. One is i.e 

  =RMapWrapper("<RCode>";#Column1;#Cloumn2;...)

  Please take note that R-Integration and PMML Support are not part of the Enterprise Edition. It is recommended to speak with your sales person or Customer Success Manager (CSM) about obtaining the R or PMML plug-in ZIP file for your Datameer version.

  0

  Comment actions Permalink

Please sign in to leave a comment.