Need a special offer?Find out if your project fits.
+

Best way to perform hierarchy and 2 million rows.

Answered
William Silveira asked on October 4, 2019

Hi guys,

First of all, great job on this component!!

I am using Flexmonster (trial), because I am trying to perform all I need.

My ambient its, HTML (Flexmonster), AJAX POST ---> PHP (WITH JSON HEADER RESPONSE) + MYSQL, because I need to parse my DATA to use hierarchy.

The problem is when component stats load the data, it read like 1000 rows per second, so it takes like 40 minutes to show 1.5kk ~ 2kk rows and a lot of browser memory.

There is other way to perform it better? remember keeping the hierarchy.
When I report 20K rows only works perfectly, example on attach.
 
Thanks guys!!

6 answers

Public
Vera Didenko Vera Didenko Flexmonster October 4, 2019

Hello, William,
 
Thank you for writing to us.
 
We would like to point out that loading CSV/JSON data with two million rows is not efficient regarding loading time and browser memory.
Please see our recommendations in the following thread: https://www.flexmonster.com/question/load-field-data-when-it-is-needed/
 
You are welcome to contact us in case of questions.
 
Best Regards,
Vera

Public
William Silveira October 4, 2019

Hi Vera,
I read the thread, but I can't find how to do hierarchy there.
In your 1 million rows example you are using CSV, right?
Is possible in other way to use something like Mondrian example for Ajax requests?
Best Regards
 

Public
Vera Didenko Vera Didenko Flexmonster October 4, 2019

Hello, William,
 
Thank you for your reply.
 
We would like to explain that at the moment multilevel data structures are not supported for Elasticsearch and the custom data source API.
 
1) In case you would like to use a multilevel data structure, we kindly suggest using Microsoft Analysis Services or Mondrian as mentioned in point 1 in https://www.flexmonster.com/question/load-field-data-when-it-is-needed/.
2) Also, in Flexmonster the same output can be achieved by simply adding a few fields to the slice instead of using levels.
Here is a JSFiddle example for illustration: https://jsfiddle.net/flexmonster/1gk3v8zx/
 
Please let us know if you have any questions.
 
Best Regards,
Vera

Public
William Silveira October 6, 2019

Hello Vera,
I am trying to use the Data Source API from your Git Hub project and its going very well,
But I have two problems:
1 - When I use a filter and I will search on it, doesn't work, https://youtu.be/3FQgEG7VSDE
2 - I don't find example how to specify the FIELDs for use on drill through view. I need to inform it when I create Flexmonster instance in my HTML file?
Thanks!

Public
Vera Didenko Vera Didenko Flexmonster October 7, 2019

Hello, William,
 
Thank you for your reply.
Your feedback is very important to us.
 

  1. Thank you for providing a screen recording of the problem.
    This is a known issue and our team will update the beta version with the fix shortly.
    We will inform you once the fix is ready.
  2. We would like to explain that specifying the fields for the drill-through is not supported for the custom data source API at the moment.
    However, this feature is included in our custom data source API roadmap and our team is working on it. 
    We will keep you updated.

 
You are welcome to contact us in case questions arise.
 
Best Regards,
Vera

Public
Illia Yatsyshyn Illia Yatsyshyn Flexmonster March 18, 2020

Hello, William,
 
We are happy to inform you that the custom data source API is released with Flexmonster version 2.8.
 
The version is stable, and the mentioned bugs were fixed as well (including searching through filters).
Also, you may be interested in our tutorial dedicated to implementing the drill-through feature.
 
Moreover, we recommend checking out our blog post in order to get familiar with the idea of custom data source API and find out more about the advantages it brings in.
 
Our team has prepared two sample servers, which can be found on our GitHub. Its source code is open, and you are welcome to use it or modify it in the way it fits your requirements.
A quick overview of the mentioned examples is placed in our documentation:

 
Please contact us in case any additional questions on this point occur.
 
Best regards,
Illia

Please login or Register to Submit Answer