Dear visitor, despite the COVID-19 outbreak, our team continues operating at full speed.

Also, here is the form where you can apply for a special discount and we will contact you with possible options. Stay safe and continue achieving your business goals.

Fill the form
Get Free Trial
Get Free Trial
  1. API reference
  2. Welcome
    1. Component overview
    2. Quick start
    3. System requirements
    4. Troubleshooting
    5. Managing license keys
    6. Migrating from WebDataRocks to Flexmonster
  3. Connecting to Data Source
    1. JSON
      1. Connecting to JSON
      2. Connecting to JSON using the custom data source API
      3. Data types in JSON
    2. CSV
      1. Connecting to CSV
      2. Connecting to CSV using the custom data source API
      3. Data types in CSV
    3. Database
      1. Connecting to SQL databases
      2. Connecting to a MySQL database
      3. Connecting to a Microsoft SQL Server database
      4. Connecting to a PostgreSQL database
      5. Connecting to an Oracle database
      6. Connecting to other databases
    4. MongoDB
      1. Introduction to the Flexmonster MongoDB Connector
      2. Getting started with the MongoDB Connector
      3. Embedding the MongoDB Connector into the server
    5. Microsoft Analysis Services
      1. Connecting to Microsoft Analysis Services
      2. Getting started with the Accelerator
      3. Installing the Accelerator as a Windows service
      4. Referring the Accelerator as a DLL
      5. Configuring the authentication process
      6. Configuring a secure HTTPS connection
      7. Troubleshooting
    6. Pentaho Mondrian
      1. Connecting to Pentaho Mondrian
      2. Getting started with the Accelerator
      3. Configuring Mondrian roles
      4. Configuring username/password protection
      5. Configuring a secure HTTPS connection
      6. Troubleshooting
    7. Elasticsearch
      1. Connecting to Elasticsearch
      2. Configuring the mapping
    8. Custom data source API
      1. Introduction to the custom data source API
      2. A quick overview of a sample Node.js server
      3. A quick overview of a sample .NET Core server
      4. Implementing the custom data source API server
      5. Implementing filters
      6. Returning data for the drill-through view
      7. Supporting more aggregation functions
  4. Security
    1. Security in Flexmonster
    2. Security aspects of connecting to an OLAP cube
      1. Ways of connecting to an OLAP cube
      2. The data transfer process
      3. Data security
      4. Data access management
  5. Configuring report
    1. What is a report
    2. Data source
    3. Slice
    4. Options
    5. Mapping
    6. Number formatting
    7. Conditional formatting
    8. Set the report for the component
    9. Get the report from the component
    10. Date and time formatting
    11. Configuring global options
    12. Export and print
    13. Calculated values
    14. Custom sorting
  6. Integration with frameworks
    1. Available tutorials
    2. Integration with AngularJS (v1.x)
    3. Integration with Angular
    4. Integration with React
    5. Integration with React Native
    6. Integration with Vue
    7. Integration with Python
      1. Integration with Django
      2. Integration with Jupyter Notebook
    8. Integration with R Shiny
    9. Integration with Webpack
    10. Integration with ASP.NET
    11. Integration with jQuery
    12. Integration with JSP
    13. Integration with TypeScript
    14. Integration with RequireJS
    15. Integration with PhoneGap
  7. Integration with charts
    1. Integration with Highcharts
    2. Integration with Google Charts
    3. Integration with FusionCharts
    4. Integration with any charting library
  8. Customizing
    1. Customizing the Toolbar
    2. Customizing appearance
    3. Customizing the context menu
    4. Localizing the component
  9. Updating to the latest version
    1. Updating to the latest version
    2. Release notes
    3. Migration guide from 2.7 to 2.8
    4. Migration guide from 2.6 to 2.7
    5. Migration guide from 2.5 to 2.6
    6. Migration guide from 2.4 to 2.5
    7. Migration guide from 2.3 to 2.4
    8. Migration guide from 2.2 to 2.3
    9. Documentation for older versions
Table of contents

Connecting to CSV using the custom data source API

In the previous article, we described how to connect the pivot table to the CSV data source. To gain even better performance while working with CSV datasets, use the custom data source API – our custom communication protocol allowing you to retrieve already aggregated data from a server to Flexmonster Pivot.

With our sample .NET Core server implementing the custom data source API, it is possible to work with the data from CSV files. This approach has the following advantages:

  • There’s no limit on the dataset volume. Since all the data processing and aggregating is made on the back end, Flexmonster can visualize reports on larger data volumes. Thus, the maximum size of a file to process depends only on your server’s power.
  • You can implement your own security logic. With the custom data source API, you have full control over the data access and you can manage the data security the way you want.
  • Reduced browser’s memory usage. Flexmonster loads only the data needed for the report, and that data is already summarized, so the client side is less loaded. 

To connect to CSV using the custom data source API, follow these steps:

  1. To run the sample project, you will need .NET Core. Get it here if it’s not already installed on your machine.

    Note that you need at least .NET Core 3.0. To check which versions of .NET Core are installed on your computer, run the following commands in the console:

    dotnet --list-sdks
    dotnet --list-runtimes
  2. Download the .zip archive with the sample project or clone it from GitHub with the following  command:
    git clone https://github.com/flexmonster/api-data-source my-proj
    cd my-proj
  3. By default, the server uses the data stored in JSON format. To make the server use a CSV data, the "DataSourceName" configuration in server-dotnetcore/appsettings.json should be the following:
    "DataSource": {
    "DataSourceName": "csv",
    ...
    }
  4. In the server-dotnetcore/appsettings.json file, create an index for the file with your data. It should be done as follows:
    "DataSource": {
    "DataSourceName": "csv",
    "Indexes": {
    "fm-product-sales": "./data/fm-product-sales.csv",
    "data-types": "./data/data-types.csv",
    "your-index": "./data/data.csv"
    }
    },
    In this key:value pair, key is your index’s name, and value is a path to your file (e.g. ./data/data.csv). You can choose any name for your index.

    As soon as you start the sample .NET Core server, it automatically preloads the data specified in the "Indexes" property. Thus, when Flexmonster requests for the data, the server responses with the already preloaded data.

    Note 1: the preloaded data is kept in the server’s RAM, so the number of indexes you can specify is limited by the amount of RAM available to the server.

    Note 2: data is stored in the server-dotnetcore/data/ folder in this sample. You can store your data either in this folder or in any other folder. In case the folder with your data isn’t on the server, an absolute path to your file must be defined when creating a new index. In addition, you should also make sure that the server has permission to access the file.

  5. (optional) By default, data from a database is reloaded every 60 minutes. You can change the refresh time in the "DataRefreshTime" configuration in server-dotnetcore/appsettings.json. For example:
    "DataStorageOptions": {
    "DataRefreshTime": "1800000"
    }
    Now the data will be reloaded every 30 minutes.

    Note that refresh time is set in milliseconds.

  6. Configure the report on the client side (see client/index.html):
    new Flexmonster({
    container: "#pivot",
    report: {
    dataSource: {
    type: "api",
    url: "http://localhost:3400/api/cube",
    index: "your-index"
    }
    }
    });
    Note 1: index must match the name of the index defined in step 3 (e.g. "your-index").

    Note 2: when Flexmonster requests for the data, the sample .NET Core server sends the response and then caches it. In case the component sends the same request once again, the server responses with the data from its cache. The cache is cleared only when the server is restarted, though the cache has a memory limit: when the limit is reached, and the new response can’t be cached, the .NET Core server deletes one of the previously cached responses from the cache.

  7. To start the .NET Core server, run the following commands in a console:
    cd server-dotnetcore
    dotnet restore
    dotnet run

What’s next?

You may be interested in the following articles: