Despite the COVID-19 outbreak, our team continues operating at full speed. We are always here to support and answer all your questions.

Feel free to reach out by filling this quick form.

Fill the form
Get Free Trial

Is it possible to describe hierarchies with the Custom Data Source API?

Answered
eze@wejugo.com asked on August 19, 2020

Hello, we have seen the way that hierarchies can be described when using a CSV of JSON data source together with a mapping object, for example in here: https://jsfiddle.net/flexmonster/yb20apuL/

We haven’t seen any mention of hierarchies in the Custom Data Source API documentation for the /fields call. Is that something that is possible just using the Mapping object on the client side, is there more that the api needs to provide to enable the hierarchy functionality. 
 
Cheers,
Ezequiel

10 answers

Public
Vera Didenko Flexmonster August 19, 2020

Hello, Ezequiel,
 
 
Thank you for writing to us. 
 
We would like to explain that for multilevel hierarchies to work for the custom data source API, advanced hierarchal filters need to be implemented on your server. For this, the following aspects need to be considered and handled in your custom data source API implementation:
 

  1. The Filter Object contains the filter property in the exclude and include parameters. Specifying this filter property will allow filtering multilevel hierarchies.
    Please see the following guide for more information: https://www.flexmonster.com/api/filter-object-for-requests/.
  2. When a hierarchy level is selected, the corresponding /members request ( please see: https://www.flexmonster.com/api/members-request/ ) will contain a Filter Object. This filter will need to be taken into account on your server when preparing the data for the response. 

    We noticed that the Filter Object is missing in our documentation for the /members request. Our team will provide an update and add this detail to our documentation as well. 

  3. Also, the advanced property needs to be set to true in your response to the /fields request: https://www.flexmonster.com/api/fields-request/.

    Setting the advanced parameter to true will indicate that your server supports hierarchies and can filter them, so multilevel hierarchies can be configured in the component via the mapping property. By default, the advanced parameter is set to false, meaning the server cannot filter the hierarchical data. In this case, if multilevel hierarchies were configured in the Mapping Object, these configurations will be ignored.
     

When all of the above points are considered in your custom data source API implementation, then multilevel hierarchies can be defined via the mapping object.

For a better understanding of the required response structure, please see the requests and responses when a hierarchy level is selected in the following JSFiddle example: https://jsfiddle.net/flexmonster/o4jywt38/ (by inspecting the page and navigating to the network tab). 
This example uses the Flexmonster Data Server – a ready to use implementation of the custom data source API created by our team. The Flexmonster Data Server enables handling large data sets from different data sources (JSON, CSV, and SQL databases).
For more information about the Flexmonster Data Server and its usage please see the following guide.
 
 
Please let us know if this helps or if any further questions arise. 
Looking forward to your response. 
 
Kind regards, 
Vera

Public
eze@wejugo.com September 1, 2020

That is great, thank you for the detailed response. Will let you know if any problems arise. 
 
Cheers,
Eze

Public
eze@wejugo.com September 3, 2020

Hi there, just getting back to you because I’ve run into an issue.
First, in addition to implementing what you had outlined, I also had to make sure that the /select call could deal with the filter.include and filter.exclude having nested filter conditions.
I completed those and now can display the hierarchies on the front end component. However when I expand a level, the component doesn’t render the expanded rows, instead showing the total. There’s an example of what that looks like in the attached gif.
I checked the server responses from our server and compared them to the ones that olap.flexmonster.com returns, and found no obvious differences that would account for this (I used the fm-product-sales.json file with our server to make the comparison easier.

The only difference I can see between the two /select responses when the level is expanded is to do with the order in which the aggregate values appear in the aggs field of the response. Here I list them:

In both cases the request is the same:

POST /select
{
"type": "select",
"index": "fm-product-sales",
"query": {
"aggs": {
"by": {
"rows": [{ "uniqueName": "Country" }, { "uniqueName": "Category" }]
},
"values": [{ "func": "sum", "field": { "uniqueName": "Price" } }]
},
"filter": [
{
"field": { "uniqueName": "Country" },
"include": [{ "member": "Australia" }]
}
]
},
"page": 0
}

The response from olap.flexmonster.com which results in proper expansion of the level:

  {
"aggs": [
{
"values": { "Price": { "sum": 1372281 } },
"keys": { "Country": "Australia" }
},
{
"values": { "Price": { "sum": 3452 } },
"keys": { "Country": "Australia", "Category": "Accessories" }
},
{
"values": { "Price": { "sum": 74493 } },
"keys": { "Country": "Australia", "Category": "Bikes" }
},
{
"values": { "Price": { "sum": 4228 } },
"keys": { "Country": "Australia", "Category": "Clothing" }
},
{
"values": { "Price": { "sum": 72159 } },
"keys": { "Country": "Australia", "Category": "Components" }
},
{
"values": { "Price": { "sum": 1217949 } },
"keys": { "Country": "Australia", "Category": "Cars" }
},
{ "values": { "Price": { "sum": 1372281 } } }
],
"page": 0,
"pageTotal": 1
}

Response from our server, which results in no expansion as shown on the attached gif:

  {
"aggs": [
{
"values": { "Price": { "sum": 3452 } },
"keys": { "Category": "Accessories", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 74493 } },
"keys": { "Category": "Bikes", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 1217949 } },
"keys": { "Category": "Cars", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 4228 } },
"keys": { "Category": "Clothing", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 72159 } },
"keys": { "Category": "Components", "Country": "Australia" }
},
{
"values": { "Price": { "sum": 1372281 } },
"keys": { "Country": "Australia" }
},
{ "values": { "Price": { "sum": 1372281 } } }
],
"page": 0,
"pageTotal": 1
}

The order of the keys object is reversed, but that should not be a problem as this is an object and key order is irrelevant.

The order of the items in the aggs array is different, but each array contains the same results. 
The question then is: Does the order of this array matter? What order is needed for the expansion to render correctly? Is there something else that I may be doing wrong?

Regards,
Ezequiel

Attachments:
L4uJfFd7Q7.gif

Public
eze@wejugo.com September 3, 2020

Just so you know, I am using this jsfiddle as the basis for the test: https://jsfiddle.net/ezk84/r08dkf6b/

Public
Vera Didenko Flexmonster September 4, 2020

Hello, Ezequiel,
 
Thank you for your response. 
 
We would like to confirm that this is not the expected behavior: the order of the keys entries’ fields should not matter.
Our team will provide the fix to this issue in the minor release version with the ETA 5th of October.
 
Currently, for the expansion to render correctly, it seems the keys entries’ fields should come in the same order as in the request.
For example, in your case in the request in the rows parameter the order is – first "Country" then "Category":

...
"aggs": {
"by": {
"rows": [{ "uniqueName": "Country" }, { "uniqueName": "Category" }]
},
...
},
...

Our team kindly recommends following this order in the response as well. 
 
 
Please let us know if this helps. If any questions arise, please don’t hesitate to reach out. 
Looking forward to your reply.
 
Kind regards, 
Vera

Public
eze@wejugo.com September 4, 2020

Thank you! I can confirm that indeed altering the order of the keys items fields fixed it. Thanks for the workaround. Looking forward to the fix come Oct.
 
Cheers,
Ezequiel

Public
Vera Didenko Flexmonster October 6, 2020

Hello, Ezequiel,
 
We are glad to inform you that the issue with the order of the keys entries’ fields in the custom API data source was fixed.
 
This is available in the latest version of Flexmonster: https://www.flexmonster.com/release-notes/.
You are welcome to update the component. Here is our updating to the latest version tutorial for guidance: https://www.flexmonster.com/doc/updating-to-the-latest-version/.
 
Please let us know if everything works.
 
Kind regards, 
Vera

Public
eze@wejugo.com 7 days ago

Hello Vera,
That seems to work great! Thank you,
Ezequiel

Public
eze@wejugo.com 4 days ago

Hey there, 
We noticed a side issue related to the order of the keys. After removing the key ordering workaround from our data API, when opening the filter for a hierarchy from the web component, the order of the levels for hierarchies is broken, so that some values that were meant to be in level 3 are at level 1, and vice-versa.
 
Included screenshots with example
left) with key ordering workaround (correct)
right) when I remove the key ordering workaround 

I will keep the workaround for now, but I expect you’ll want to fix this so that the order of the keys doesn’t affect the filtering.

Cheers,
Eze

Public
Vera Didenko Flexmonster 2 days ago

Hello, Ezequiel,
 
Thank you for reporting this issue. 
 
Our team will look into this issue and we will provide the fix in the minor release version with the ETA 30th of November.
 
Please let us know if this would work for you. 
If further questions arise, feel free to reach out.
 
Kind regards, 
Vera

Please login or Register to Submit Answer