Need a special offer?Find out if your project fits.
+
Get Free Trial
Get Free Trial

Flexmonster Dataserver API to refresh data instead of using DataRefreshTime

posted on August 5th, 2020

Answered
Kim Li Foo asked on August 5, 2020

Hi, recently I was trying the Flexmonster Dataserver on our ECS Server with Windows OS, but notice some issue as below:
 

  1. Memory Usage is growing, I set the DataRefreshTime to 1 minute as I need the report to be the latest once the user goes to our reporting module to generate the report. But after the period of time, I notice the Memory keeps on growing from 100 MB to over 800 MB. Not sure whether there are any memory leak on the Flexmonster Dataserver application. F.Y.I. I did change the CacheSizeLimit from 0 to some other number, but no luck.
  2. After a few reports generated, the Flexmonster Dataserver application console showed out of memory exception, then after few exceptions the Flexmonster Dataserver application freeze, no Data refresh from the console, and also not responding to the client-side request.

I would like to propose to have few APIs for below:

  1. API for client-side to call to refresh data, which will reduce the frequency of data refresh, and also avoid the unnecessary data query to reduce the server loading.
  2. API to restart/reset the Flexmonster Dataserver, when it goes to an error state.

thanks.
 

4 answers

Public
Kim Li Foo ⋅ August 5, 2020

attached is the log file for your reference

Attachments:
fmlog-2020-08-04.7z

Public
Mykhailo Halaida Mykhailo Halaida ⋅ Flexmonster ⋅ August 5, 2020

Hi Pang,
 
Thank you for posting your question.
 
Regarding your suggestions for the API, please find our answers below:

  1. An option to manually reload the data to the Data Server is something that we have actually been working on for some time now – we are planning to introduce this feature in one of our future releases.
  2. It would be somewhat problematic to implement a full restart of the Data Server on a specific API call due to particular technical specificities, so we are currently not planning to introduce this feature in the nearest future.  

Speaking of memory issues you've mentioned, please see our response in the related thread.
 
Please let us if we've managed to answer your questions in full or if there is still anything we can help you with.
 
Best regards,
Mykhailo

Public
Kim Li Foo ⋅ August 6, 2020

Hi Mykhailo,
Thanks for the prompt reply. I am looking forward to having the option to reload the data through the client-side request. Which is more applicable to us, as AWS EC2 we subscribed has limited CPU and Memory, which also shared among many of our projects and application. Refresh data by time will consume too many resources if too frequently, but less frequency will also cause the report pulled can not reflect the latest.
Please also help to implement GC properly, try to release the memory ASAP once the request fulfilled. We may want to have few indexes in Flexmonster DataServer for large data reports, we do not need it to cache the report data in memory, this is because AWS EC2 and RDS in the internal network with large bandwidth, and thanks to Flexmonster Pivot we no need the execute complex store procedure at RDS to group the data, so it is very fast for Flexmonster DataServer to retrieve the data from RDS(no need to preload or cache data).

Public
Mykhailo Halaida Mykhailo Halaida ⋅ Flexmonster ⋅ August 7, 2020

Hi Pang,
 
Thank you for providing further details on your use case.
 
We also wanted to clarify a couple of things about how the Data Server works. More specifically, once started, the data sets for each of the indexes specified in the configurations are loaded to the memory – this is necessary for the Data Server to be able to build the query structure based on the corresponding data set.
 
Speaking of the CacheSizeLimit Data Server configuration, it defines how many responses are cached for every index – when set to 0, the caching is disabled and the responses will not be saved.
 
We hope we've managed to explain the above specificities of Flexmonster Data Server clearly – please let us know if there are still any questions on this that we can help you with. 
 
Kind regards,
Mykhailo

Please login or Register to Submit Answer