asked on November 29, 2020
Hi,
When setting CacheSizeLimit=0, server goes up fine.
When the client calls it, it displays the list of fields. when client approves the fields, it calls the index, and then the server log an error:
|ERROR|Flexmonster.DataServer.Middlewares.ErrorHandlingMiddleware|Object reference not set to an instance of an object.
what am I missing?
Thanks,
Moshe
Note: the table is a view.
This is the configuration:
{
"DataSources": [
{
"Type": "database",
"DatabaseType": "mysql",
"ConnectionString":
"Server=~~~~~~~~~~~~~~~~~~~~~~~",
"Indexes": {
"index_1": {
"Query": "SELECT * FROM xxxxx"
}
}
}
],
"Security":
{
"Authorization":
{
"Enabled": false
},
"CORS":
{
"AllowOrigin": "*"
}
},
"DataStorageOptions":
{
"CacheSizeLimit": 0
},
"Port": "9500"
}
⋅ November 30, 2020
Hello,
Thank you for reporting the issue.
We did manage to reproduce it on our side.
Our team will research the error and provide you with results ETA Dec 28.
Please let us know in case any other questions arise.
Regards,
Illia
⋅ November 30, 2020
Thank you!
Maybe a follow-up question (let me know if better to open a standalone question instead):
What would be the best strategic to handle DB with large tables?
In our case, we would like to provide reports based on ~30 tables, which some will hold 1 million or more records. I tried with about 100K (4 times) and got 3GB in memory. Not very scalable (will need about 300GB just for FM).
What is the recommended approach to handle this cases? how to maintain a reasonable amount of RAM? Using custom API instead? Something else?
Will using 0 cashSize do the trick (probably in cost of longer fetch time)?
Thanks!
Moshe
⋅ December 2, 2020
Hello, Moshe,
Thank you for reaching out.
Generally, we recommend opening a separate forum thread if questions are not connected.
Concerning the best strategy to handle large datasets.
Flexmonster Data Server stores the retrieved data in the server's RAM. It allows to significantly reduce the time required to process the client's request. However, it means the whole dataset will be uploaded and stored on your backend.
If the dataset size exceeds the limitation exposed by the server, we suggest developing your implementation of the custom data source API that would operate resources differently. For example, you can query the database every time the user makes a request. After forming and sending the response, memory can be released and reused for further requests.
Please note that this custom server has to implement the custom data source API to handle requests from Flexmonster Pivot.
To provide you with illustrative examples of such a server, we developed two ready-to-check Node.js and .NET Core servers.
Note that these are only the samples, and the implemented logic of data processing may differ from what you need. You can use them as a core for building a custom connector.
As a starting point, we highly recommend jumping into the Quick start guide. This guide provides step-by-step recommendations to help you get going with the API. In the Custom API requests section, you will find all the front-end and backend specifications of the API. There you will also be able to check the examples of requests and responses for different queries.
We hope it helps.
Please let us know if other questions arise.
Regards,
Illia
⋅ December 29, 2020
Hello, Moshe,
We are happy to let you know that the issue with requests when setting the CacheSizeLimit property to 0 was fixed.
You are welcome to update the component.
Here is our updating to the latest version guide for assistance: https://www.flexmonster.com/doc/updating-to-the-latest-version/.
Please let us know if everything works fine for you.
Best regards,
Illia
⋅ January 11, 2021
Hello, Moshe,
We are wondering if the recent fix works for you.
Our team is looking forward to hearing from you.
Kind regards,
Illia
