Despite the COVID-19 outbreak, our team continues operating at full speed. We are always here to support and answer all your questions.

Feel free to reach out by filling this quick form.

Fill the form
Get Free Trial

Dataset is tool large (on expandAllData) error message and behavior issues

Re-Open
Dmitry Shtefan asked on December 9, 2020

Hello,
 
We have found a problem with the data display when the “Dataset is too large” error happens on expandAllData function call.
 
We are getting the following message: “Dataset is too large. Some fields cannot be expanded. Please narrow down the dataset.”
 
The problem here is that the part of the dataset is missing in this case, that is not very clear from the message above. There are only successfully expanded rows displayed, but all the failed to expand rows are missing. At the same time, the grand totals are the same before and after the failed expandAllData action.
Is there any chance we can have more consistent pivot behavior here?
 
Another problem is that there is a significant difference between the datasets size for this error to happen across Google Chrome and Firefox browsers: in FF we can see this error for multidimension dataset with 30K rows (report_01.csv dataset example is attached) vs in Chrome it happens only for way bigger datasets (400K rows).
 
Attaching the flexmonster config along with dataset csv file to reproduce the expandAllData error in FF for a relatively small dataset with 30K rows. Could you please check if there is anything can be done to fix this, so this functionality works the same way it does in Google Chrome?
 
Flexmonster version: 2.8.21;
Firefox version: 83.0;
Google Chrome version: 87.0.4280.88;
 
Thanks in advance,
Dmitriy

5 answers

Public
Mykhailo Halaida Mykhailo Halaida Flexmonster December 11, 2020

Hi Dmitriy,
 
Thank you for posting your question.
 
Please note that this warning is triggered when multiple expanding operations are likely to overload the user’s browser. 
 
While Flexmonster does not impose any programmatical limitations on the size of the data set and the number of expands, the component still uses the resources available to the client’s browser to perform calculations. While we’re at it, it is also worth mentioning that different browser might manage their resources differently. This explains the difference in acceptable data set sizes between Chrome and Firefox that you’ve mentioned.
 
Expansion of cells is a heavy operation, sometimes requiring many browser resources. When performed over large data sets, it can lead to exceptions and page crashes.
 
This is what the mentioned pop-up is for – to notify the user when Flexmonster aborts the operation in order to avoid crashing the page & triggering exceptions. Naturally, this also explains why some of the fields have been left unexpanded.
 
With that in mind, we would suggest reducing the number of expands in the report.
 
Please let us know if this helps or if there is still anything else we can help you with.
 
Best regards,
Mykhailo

Public
Dmitry Shtefan December 14, 2020

Hello Mykhailo,
 
Thanks for the details!
 
The problem with the mentioned pop-up is that it says that some fields cannot be expanded.
But what happens is that some data is missing.
 
For example, in case we have a dataset with 10 collapsed rows by default (Level1Dimension1…10 and aggregated Metric1..10 values):
Level1Dimension1, Metric1
Level1Dimension2, Metric2

Level1Dimension10, Metric10

and when we try to expand all data, we get:

Level1Dimension1, Level2Dimension11, Metric11 (successfully expanded)
Level1Dimension2, Level2Dimension11, Metric12 (successfully expanded)

-Level1Dimension10, Metric10- (not displayed any more even in the collapsed form, not like the fields are left unexpanded)
 
We understand the problem with limited browser resources, but the concern is to have clear pivot behavior in case of exceptions like this, so it doesn’t lead to a false data analysis assumptions.
 
Is there any chance we can have the fields that failed to expand still displayed unexpanded?
 
Thank you,
Dmitriy
 

Public
Mykhailo Halaida Mykhailo Halaida Flexmonster December 15, 2020

Hi Dmitriy,
 
Thank you for providing an additional explanation of the issue you’ve experienced.
 
We have now managed to reproduce the described behavior and can confirm it is indeed an issue. Our team will take a deeper look at this and we’ll return to you with a fix on ETA Jan 11th.
 
Please let us know if there is anything else we can help you with in the meantime.
 
Kind regards,
Mykhailo

Public
Mykhailo Halaida Mykhailo Halaida Flexmonster 6 days ago

Hi Dmitry,
 
Thank you for giving us time to look into the issue in question.
 
We’re glad to inform you that the issue with expanding the data has been fixed – all unexpanded members are still available even if the expanding process is terminated.
 
This is available in the latest minor release of Flexmonster: https://www.flexmonster.com/release-notes/. You are welcome to update the component. 
 
Also, we’ve done some tests, and it seems like Firefox is the only popular browser that is incapable of executing the mentioned expansion with your data set (due to the resources allocation we’ve mentioned earlier).
 
After thorough research on our side, we’ve concluded that to optimize the expanding process in a way that it executes within similar time limits regardless of the browser would require significant modifications to the expandAllData() algorithm and overall pivot table architecture. Therefore, we cannot provide you with a precise ETA for this, but the issue has been added to our backlog and will be reviewed once our team can put enough effort into it.
 
In the meantime, we would suggest considering the expandExecutionTimeout option, which lets you specify the execution timeout for expandAllData(). It is set to 9000 milliseconds by default – you can increase this limit so that Firefox has enough time to fully execute the expansion (note that a large execution timeout can cause an unresponsive script alert).
 
We hope you find the information above helpful. Please let us know if there is anything else we can help you with.
 
Best regards,
Mykhailo

Public
Dmitry Shtefan 6 days ago

Hello Mykhailo,
 
Thank you for fixing the problem and support.
We will be planning checks and changes within the next two weeks.
 
Best regards,
Dmitriy

Please login or Register to Submit Answer