Problems with PostgreSQL large data

posted on January 23rd, 2020

Answered

Ingenieria Especializada asked on January 23, 2020

Hello Support Team,

We use Flexmonter since 6 months ago in our software SIG+ and we are happy with this product. SIG+ uses as database PostgreSQL and we have noticed that Flexmonster with some few data works perfect, however, when we send it a query that returns 20 columns and 726935 rows and other that return 107 columns and 5499 rows it mixed the data (put data from one row into another) and the plugin show things like 'undefined' with data that clearly works fine.

We like to know if the plugin has some restrictions in quantity of data or if there is a restriction with characters that data could contain like ", &, etc, please have in consideration that our native language is spanish, so data can contain á, é, í, ó, ú, ñ, etc.

Thank you for your help in this issue.

1 answer

Public

Illia Yatsyshyn ⋅ Flexmonster ⋅ January 24, 2020

Hello,

Thank you for reaching out to us.

Being a client-side component, Flexmonster relies on resources available to the browser, and this affects the loading time and the maximum size of the data that can be handled on every particular machine.
This means that the client computer's RAM determines how much data can be loaded at once, and CPU capabilities affect how much time is spent on the data analyzation.

In case the required data set appears to be too big for the specific machine and the browser of the user, our team kindly recommends considering using MS Analysis Server or Elasticsearch as a data source.

We would like to provide some explanations regarding the difference between MS Analysis Server & Elasticsearch and other data sources (e.g., PostgreSQL).

The key point is when the browser loads data:

If the PostgreSQL data set is used, all data has to be loaded before the component starts displaying any report.
In the case of using MS Analysis Server or Elasticsearch, Flexmonster can handle much larger data because not all data is loaded into the browser at once. Only the part of the data needed for the current representation in Flexmonster is sent to the client-side when using MS Analysis Server or Elasticsearch. Every time more data needs to be shown, a query is formed, and a different set of data is sent to Flexmonster. This way, memory is used in moderation.

As for mixing the data and "undefined" alert, you have mentioned, it would be useful for our investigation if you could provide us with detailed information about the problems you are facing.

Concerning specific characters contained in the data set, there are no restrictions. However, in order to display it correctly, the encoding has to be specified. It is achievable using the following meta-tag:

<meta http-equiv="content-type" content="text/html; charset=utf-8" />

or defining encoding of a separate JS file in case the HTML content of the file can not be changed:

<script src="yourfile.js" charset="UTF-8"></script>

Also, our team would like to kindly suggest considering the custom API data source.
Flexmonster custom data source API is designed for summarized data retrieval from a server to Flexmonster Pivot.
This API will work for projects with their own data access layer, where filtering and aggregation are delegated to a server, and Flexmonster receives ready-to-show data. Using the custom API data source can have a favorable effect on loading large data sets.

More information about custom API data source and sample projects can be found following the link: API data source.

Please contact us in case of additional questions.

Best regards,
Illia

Problems with PostgreSQL large data

1 answer

Please login or Register to Submit Answer