Local to Hadoop mode

Follow

Aleksandar Razmovski

• September 16, 2016 08:57

Dear,

What needs to be done if Datameer is already running in local mode with Connections, Data Links, Workbooks, Infographics already existing there, to be changed to a Hadoop distribution? . Also can the artifacts be preserved in some way? 

Best Regards,

AR

0

• 

  Konsta Danyliuk

  • September 16, 2016 09:48

  Hello Aleksandar.

  In order to better understand your goal and suggest the best approach, may I ask you to address the following questions, please?

  - What Datameer version you are using?

  - What is your final goal - just start to use cluster and move your artefacts into HDFS?

   

  0

  Comment actions Permalink
• 

  Aleksandar Razmovski

  • September 16, 2016 10:26

  Dear Konsta,

  Txs for quick reply.

  To answer Your questions:

  1. ATM Datameer 5.6 is used.
  2. Precisely what You described.

  Also can the transition somehow be tested before the actual/final moving? 

  Best Regards,

  AR

  0

  Comment actions Permalink
• 

  Konsta Danyliuk

  • September 16, 2016 11:00

  Aleksandar.

  In this case common approach should be the following:

  1. If you use HSQL for Datameer database - upgrade it to MySQL.

  https://documentation.datameer.com/documentation/display/QA/Database+Migration

  2. Connect Datameer to a cluster and set all required settings.

  For example: https://documentation.datameer.com/documentation/display/DAS50/Set+Up+Datameer+with+YARN+Cluster

  3. Move your artefacts from local store (/das-data folder) to HDFS. Note that DM should be stopped during this action.

  4. Update paths to artefacts in database. Note that DM should be stopped during this action.

  https://documentation.datameer.com/documentation/display/DASSB60/Migrating+from+One+Hadoop+Cluster+to+Another 

   

  Concerning 

  >>Also can the transition somehow be tested before the actual/final moving?

  I would suggest to perform full database dump, backup full datameer folder (if possible) and note all changes you are doing - this helps to roll back to initial state, in case of any issues.

   

  0

  Comment actions Permalink
• 

  Aleksandar Razmovski

  • September 16, 2016 11:05

  Dear Konsta,

  Will try to follow all the steps You've provided and hopefully migrate successfully. If something comes up along the way I will post/ask again here. Thanks again. 

  Best Regards,

  AR

  0

  Comment actions Permalink

Please sign in to leave a comment.