asked on February 20, 2020
I noticed that in the new version v2.8 there are quite a few usefull new properties for the mapping. The only thing is that they can only be used for "csv" and "json" data sources.
hierarchy – the hierarchy name. This property is necessary to specify only if the field is a level of a hierarchy ("type": "level") or if the field is a member property of a hierarchy ("type": "property"). Only for "csv" and "json" data sources.level – the level caption. This property can be specified only if the field is a level of hierarchy ("type": "level"). Only for "csv" and "json" data sources.parent – the unique name of the parent level. This property can be specified only if the field is a level of hierarchy ("type": "level"). Only for "csv" and "json" data sources.dimensionUniqueName – the dimension unique name. This property can be used to group several fields under one dimension. Only for "csv" and "json" data sources.dimensionCaption – the dimension caption. This property specifies the name of a folder in the Field List under which several fields are grouped. Only for "csv" and "json" data sources.
Is it possible to make these properties also available for the "ocsv" datasource.
Regards Jaco
⋅ February 21, 2020
Hi Jaco,
Thank you for writing to us.
Kindly note that at the moment it is only possible to use the mentioned properties with CSV and JSON data sources.
The thing is, since version 2.7.14, the Flexmonster Data Compressor no longer compresses the data as before (the output format is no longer OCSV). The OCSV format is therefore not actively supported anymore and not recommended for use.
Please let us know if this helps.
Best regards,
Mykhailo
⋅ February 21, 2020
Hi Mykhailo,
So, if you say that ocsv is no longer actively supported, why does the example on the bottom of the page https://www.npmjs.com/package/flexmonster-compressor?activeTab=readme still uses the dataSourceType "ocsv"?
If the flexmonster data compressor no longer compresses the data as before version 2.7.14 (in the releasenotes i dont see anything about this), what would you recommend? I want a lot of data from MSSQL Server go to Flexmonster, preferably compressed to reduce the load AND i want to use the new mapping options for creating groups in the field selector.
Regards Jaco
⋅ February 21, 2020
Hi Jaco,
Thank you for reporting this inconsistency in our example. We will update it to make sure it only contains the most relevant information.
For MSSQL Server connection we would suggest considering the custom data source API, introduced in the latest version 2.8.0. The custom data source API is a communication protocol that allows you to retrieve the aggregated data from the server with the data source of your choice. This approach is much more efficient in terms of handling large data and using the browser's resources.
More information on the custom data source API can be found in our latest blog post: https://www.flexmonster.com/blog/connect-to-any-data-source-with-flexmonster/
Speaking of mapping options, only a few of them are currently available with the custom API (caption, aggregations & visible). However, we plan to keep extending the mapping options for custom API in the nearest minor releases.
Also, we've put out a sample .NET Core implementation of the custom data source API, so feel free to check it out for your reference: https://www.flexmonster.com/doc/pivot-table-with-dot-net-core-server/
Please let us know if this helps.
Regards,
Mykhailo
⋅ February 26, 2020
Hi Mykhailo,
The queries used for retrieving the data are quite time-expensive. Although i understand the need of aggregating the data on the server side (Just like Elastic Search does), but i doubt it will be faster, because for every action the same time-expensive query needs to be run.
But we will try to set up a test-case to see whether or not it is an improvement for us.
2. too bad the new mapping options are not available for all types and for the custom API. Since we have lots of fields our users can choose from, it would have made life easier if we could group these fields.
I hope, that they will be included soon for all types and the custom API.
Regards Jaco
⋅ February 27, 2020
Hi Jaco,
Thank you for your response.
Please note that you don't have to build the API implementation from scratch, as the sample we've sent you earlier is essentially ready to use – only the ConnectionStrings & Indexes properties have to be specified to connect your database to Flexmonster.
Speaking of data retrieval time, the key aspect of the custom data source API is that all the heavy calculations are carried out by the server instead of the browser. This approach ensures the browser works more smoothly while the server performs the aggregations much faster, especially on large volumes of data than it would have been with the browser's capabilities. This, of course, depends on each individual server machine, but is true as a general rule.
We hope this helps.
Best regards,
Mykhailo
⋅ April 20, 2020
Hi Mykhailo,
1.Regarding my original question:
I noticed that in the new version v2.8 there are quite a few usefull new properties for the mapping. The only thing is that they can only be used for “csv” and “json” data sources.
Is it possible to make these properties also available for the “ocsv” datasource.
Why isnt it possible to use these properties for the "elasticsearch" type?
2. You answered with:
The thing is, since version 2.7.14, the Flexmonster Data Compressor no longer compresses the data as before (the output format is no longer OCSV). The OCSV format is therefore not actively supported anymore and not recommended for use.
What does this mean for the compression? Will the type "ocsv" become obsolete in the near future? If yes, do you have a date for that? Since that forces us to make changes before that date.
Hope to hear from you.
Regards Jaco
⋅ April 23, 2020
Hi Jaco,
The reason why many mapping options are currently only available for the CSV & JSON data sources is the difference in how Elasticsearch & CSV/JSON data is handled by Flexmonster. The data structure and transfer process are rather different in these two scenarios – with Elasticsearch Flexmonster supports dynamic queries for retrieving partial data instead of loading all data at once, as it is in the case of JSON and CSV. This makes it difficult to support many mapping options for Elasticsearch.
Speaking of your second question, while we do not recommend using OCSV format, it is still supported by Flexmonster, even though no new OCSV-related features will be added in the future anymore. Therefore, no immediate actions are required from your side to ensure compatibility with Flexmonster.
Please let us know if this helps.
Best regards,
Mykhailo