Get Free Trial

Problem on handling large dataset with custom filter

Ken Wong asked on July 5, 2022

I have a situation where I am suppose to load the data based on the custom date range into the Pivot table.
The approach that is being used now is just returning the JSON data from the Web API server and load into the Pivot table.
However, this is resulting in huge data sizes (as shown in the screen shot) being transferred over to the client side.
I am wondering if there is any recommended approach to avoid this problem.

network sizes.png

3 answers

Maksym Diachenko Maksym Diachenko Flexmonster July 6, 2022

Hello, Ken!

Thank you for reaching out to us.

While working with JSON data sources, the component loads all data and performs all calculations on the client-side. This can lead to some difficulty in achieving better performance, especially when the data files exceed 100 MB. That’s why we recommend using Flexmonster Data Server with large JSON datasets.  

Flexmonster Data Server is our tool for delegating data aggregation to the server-side. Flexmonster Data Server is based on the custom data source API. It allows you to retrieve already aggregated data from the server and pass it to the Flexmonster pivot table. Kindly note that using the data server requires having a JSON file with data stored on the machine where it is launched. You are welcome to check our connecting to JSON with FDS guide for more information.

Alternatively, you can implement your server-side tool for data aggregation with the custom data source API. Custom API is a communication protocol used by Flexmonster to retrieve already aggregated data from the server. This approach will also improve the performance with large datasets. 

Please let us know if our recommendations were helpful for the case.

Best Regards,

Ken Wong July 12, 2022

Hi Maksym, 
Thanks for the pointer to Custom API, it helps us to solve the addressed issue.

Maksym Diachenko Maksym Diachenko Flexmonster July 12, 2022

Hi, Ken!

Thank you for your feedback.
We are glad to hear that using the custom API data source worked for you.
Feel free to write us if more questions arise.

Best Regards,

Please login or Register to Submit Answer