asked on August 19, 2020
Hello, we have seen the way that hierarchies can be described when using a CSV of JSON data source together with a mapping object, for example in here: https://jsfiddle.net/flexmonster/yb20apuL/
We haven't seen any mention of hierarchies in the Custom Data Source API documentation for the /fields call. Is that something that is possible just using the Mapping object on the client side, is there more that the api needs to provide to enable the hierarchy functionality.
Cheers,
Ezequiel
⋅ August 19, 2020
Hello, Ezequiel,
Thank you for writing to us.
We would like to explain that for multilevel hierarchies to work for the custom data source API, advanced hierarchal filters need to be implemented on your server. For this, the following aspects need to be considered and handled in your custom data source API implementation:
filter property in the exclude and include parameters. Specifying this filter property will allow filtering multilevel hierarchies./members request ( please see: https://www.flexmonster.com/api/members-request/ ) will contain a Filter Object. This filter will need to be taken into account on your server when preparing the data for the response.
We noticed that the Filter Object is missing in our documentation for the /members request. Our team will provide an update and add this detail to our documentation as well.
advanced property needs to be set to true in your response to the /fields request: https://www.flexmonster.com/api/fields-request/.
Setting the advanced parameter to true will indicate that your server supports hierarchies and can filter them, so multilevel hierarchies can be configured in the component via the mapping property. By default, the advanced parameter is set to false, meaning the server cannot filter the hierarchical data. In this case, if multilevel hierarchies were configured in the Mapping Object, these configurations will be ignored.
When all of the above points are considered in your custom data source API implementation, then multilevel hierarchies can be defined via the mapping object.
For a better understanding of the required response structure, please see the requests and responses when a hierarchy level is selected in the following JSFiddle example: https://jsfiddle.net/flexmonster/o4jywt38/ (by inspecting the page and navigating to the network tab).
This example uses the Flexmonster Data Server - a ready to use implementation of the custom data source API created by our team. The Flexmonster Data Server enables handling large data sets from different data sources (JSON, CSV, and SQL databases).
For more information about the Flexmonster Data Server and its usage please see the following guide.
Please let us know if this helps or if any further questions arise.
Looking forward to your response.
Kind regards,
Vera
⋅ September 1, 2020
That is great, thank you for the detailed response. Will let you know if any problems arise.
Cheers,
Eze
⋅ September 3, 2020
Hi there, just getting back to you because I've run into an issue.
First, in addition to implementing what you had outlined, I also had to make sure that the /select call could deal with the filter.include and filter.exclude having nested filter conditions.
I completed those and now can display the hierarchies on the front end component. However when I expand a level, the component doesn't render the expanded rows, instead showing the total. There's an example of what that looks like in the attached gif.
I checked the server responses from our server and compared them to the ones that olap.flexmonster.com returns, and found no obvious differences that would account for this (I used the fm-product-sales.json file with our server to make the comparison easier.
The only difference I can see between the two /select responses when the level is expanded is to do with the order in which the aggregate values appear in the aggs field of the response. Here I list them:
In both cases the request is the same:
POST /select
{
"type": "select",
"index": "fm-product-sales",
"query": {
"aggs": {
"by": {
"rows": [{ "uniqueName": "Country" }, { "uniqueName": "Category" }]
},
"values": [{ "func": "sum", "field": { "uniqueName": "Price" } }]
},
"filter": [
{
"field": { "uniqueName": "Country" },
"include": [{ "member": "Australia" }]
}
]
},
"page": 0
}
The response from olap.flexmonster.com which results in proper expansion of the level:
{
"aggs": [
{
"values": { "Price": { "sum": 1372281 } },
"keys": { "Country": "Australia" }
},
{
"values": { "Price": { "sum": 3452 } },
"keys": { "Country": "Australia", "Category": "Accessories" }
},
{
"values": { "Price": { "sum": 74493 } },
"keys": { "Country": "Australia", "Category": "Bikes" }
},
{
"values": { "Price": { "sum": 4228 } },
"keys": { "Country": "Australia", "Category": "Clothing" }
},
{
"values": { "Price": { "sum": 72159 } },
"keys": { "Country": "Australia", "Category": "Components" }
},
{
"values": { "Price": { "sum": 1217949 } },
"keys": { "Country": "Australia", "Category": "Cars" }
},
{ "values": { "Price": { "sum": 1372281 } } }
],
"page": 0,
"pageTotal": 1
}
Response from our server, which results in no expansion as shown on the attached gif:
{
"aggs": [
{
"values": { "Price": { "sum": 3452 } },
"keys": { "Category": "Accessories", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 74493 } },
"keys": { "Category": "Bikes", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 1217949 } },
"keys": { "Category": "Cars", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 4228 } },
"keys": { "Category": "Clothing", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 72159 } },
"keys": { "Category": "Components", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 1372281 } },
"keys": { "Country": "Australia" }
},
{ "values": { "Price": { "sum": 1372281 } } }
],
"page": 0,
"pageTotal": 1
}
The order of the keys object is reversed, but that should not be a problem as this is an object and key order is irrelevant.
The order of the items in the aggs array is different, but each array contains the same results.
The question then is: Does the order of this array matter? What order is needed for the expansion to render correctly? Is there something else that I may be doing wrong?
Regards,
Ezequiel
⋅ September 3, 2020
Just so you know, I am using this jsfiddle as the basis for the test: https://jsfiddle.net/ezk84/r08dkf6b/
⋅ September 4, 2020
Hello, Ezequiel,
Thank you for your response.
We would like to confirm that this is not the expected behavior: the order of the keys entries' fields should not matter.
Our team will provide the fix to this issue in the minor release version with the ETA 5th of October.
Currently, for the expansion to render correctly, it seems the keys entries' fields should come in the same order as in the request.
For example, in your case in the request in the rows parameter the order is - first "Country" then "Category":
...
"aggs": {
"by": {
"rows": [{ "uniqueName": "Country" }, { "uniqueName": "Category" }]
},
...
},
...
Our team kindly recommends following this order in the response as well.
Please let us know if this helps. If any questions arise, please don't hesitate to reach out.
Looking forward to your reply.
Kind regards,
Vera
⋅ September 4, 2020
Thank you! I can confirm that indeed altering the order of the keys items fields fixed it. Thanks for the workaround. Looking forward to the fix come Oct.
Cheers,
Ezequiel
⋅ October 6, 2020
Hello, Ezequiel,
We are glad to inform you that the issue with the order of the keys entries’ fields in the custom API data source was fixed.
This is available in the latest version of Flexmonster: https://www.flexmonster.com/release-notes/.
You are welcome to update the component. Here is our updating to the latest version tutorial for guidance: https://www.flexmonster.com/doc/updating-to-the-latest-version/.
Please let us know if everything works.
Kind regards,
Vera
⋅ October 21, 2020
Hello Vera,
That seems to work great! Thank you,
Ezequiel
⋅ October 24, 2020
Hey there,
We noticed a side issue related to the order of the keys. After removing the key ordering workaround from our data API, when opening the filter for a hierarchy from the web component, the order of the levels for hierarchies is broken, so that some values that were meant to be in level 3 are at level 1, and vice-versa.
Included screenshots with example
left) with key ordering workaround (correct)
right) when I remove the key ordering workaround
I will keep the workaround for now, but I expect you'll want to fix this so that the order of the keys doesn't affect the filtering.
Cheers,
Eze
⋅ October 26, 2020
Hello, Ezequiel,
Thank you for reporting this issue.
Our team will look into this issue and we will provide the fix in the minor release version with the ETA 30th of November.
Please let us know if this would work for you.
If further questions arise, feel free to reach out.
Kind regards,
Vera
⋅ November 25, 2020
Hello, Ezequiel,
We are happy to inform you that our team has found a way to solve the issue with hierarchy members rendering incorrectly in the filter pop-up.
Looking deeper, it turns out that the issue occurs if levelName is specified in the slice, and the /select endpoint does not handle this case:
We would like to kindly explain that if levelName (the level from which to show the hierarchy) is specified in the slice, an additional /select request is initiated.
This /select request is needed to extract the members from the specified level.
It is possible that this /select request scenario is not handled on the server, therefore, leading to unexpected results.
With this in mind, our team kindly recommends trying the following solutions:
levelName property from the slice.levelName property is needed, please add the implementation for the /select response for loading the required levels.Please let us know if this helps to resolve the issue on your end.
Looking forward to hearing from you soon.
Kind regards,
Vera
⋅ November 25, 2020
Aha! Didn't notice that the extra request was being sent out. I will check that this case is being covered in our implementation of /select.
Thanks for the help, I'll see how I go.
Cheers,
Ezequiel
⋅ December 8, 2020
Hello, Ezequiel,
Just checking in to ask if our suggestions worked for you.
Did adding the additional /select request scenario to your /select implementation resolve the issue?
If any questions remain, please feel free to reach out.
Looking forward to your feedback.
Kind regards,
Vera
⋅ December 17, 2020
Hello, Ezequiel,
Our team is wondering if you managed to resolve the issue.
Did the suggestions help to fix the order of the levels for hierarchies in the filter pop-up?
We would be grateful for your feedback.
Kind regards,
Vera