How do I retrieve additional pages from the REST API?



  • Official comment
    Saurabh Agashe

    If you take a look in conf/, there is a value for the maximum rest download records. The default for this is 100000. You can adjust the following property to raise this limit:

    I have a relatively standard Datameer install, and it's located on line 230, but the exact line may vary.

    Hope this helps!

    Comment actions Permalink
  • Brian Junio

    I would just like to add that increasing this value over the 100k default configuration can result in slowness felt throughout Datameer. The 'conductor' is responsible for not only managing API calls, but for every process that Datameer must manage.

    It is recommended that you create an ExportJob and offload that work to your cluster.

    Comment actions Permalink

Please sign in to leave a comment.