Connecting to CSV using Flexmonster Data Server

In the previous article, we described how to connect the pivot table to the CSV data source. To gain even better performance while working with CSV datasets, use Flexmonster Data Server – a special server developed by Flexmonster. This server communicates with the client using the custom data source API – our custom communication protocol allowing you to retrieve already aggregated data from a server to Flexmonster Pivot.

Flexmonster Data Server has the following advantages:

Reduced browser’s memory usage. Since loading, processing, filtering, and aggregating of the data is made on the server side, Flexmonster Pivot gets the data in a ready-to-show format. Thus, the data is visualized faster, and the browser’s resources are used more efficiently.
Larger datasets. Despite Flexmonster Pivot can connect to a CSV file directly, the browser’s capability limits the size of a file (on average, up to 100 MB). Flexmonster Data Server can process files more than 1GB — the maximum file size depends on your RAM capacity and the number of unique members in the dataset.
Built-in security configurations. The Data Server allows establishing authorized access to data and managing security in a convenient way.
Cross-platform solution. The Data Server is available for popular operating systems: Windows, Ubuntu/Linux, and macOS.

Prerequisites

To download the Data Server, you will need Flexmonster CLI — a command-line interface tool for Flexmonster. If needed, install the CLI globally using npm:

npm install -g flexmonster-cli

After that, a new flexmonster command will be available in the console. Learn more about Flexmonster CLI and its commands in our documentation.

Now follow the steps below to connect to CSV using the Data Server.

Step 1. Embed the component into your web page

If Flexmonster is not yet embedded, set up an empty component in your web page:

In pure JavaScript

Complete the Quick start guide. Your code should look similar to the following example:

var pivot = new Flexmonster({
    container: "pivotContainer",
    toolbar: true
});

In Angular

Complete the Integration with Angular guide. Your code should look similar to the following example:

<fm-pivot 
 [toolbar]="true">
</fm-pivot>

In React

Complete the Integration with React guide. Your code should look similar to the following example:

<FlexmonsterReact.Pivot
 toolbar={true}
/>

In Vue

Complete the Integration with Vue guide. Your code should look similar to the following example:

<Pivot
 ref="pivot"
 toolbar>
</Pivot>

Step 2. Download Flexmonster Data Server

The previous step demonstrated how to configure Flexmonster Pivot. Now it’s time to set up Flexmonster Data Server.

Get the Data Server with the following CLI command:

flexmonster add fds -r

The flexmonster add fds command will do the following:

Download the .zip archive with Flexmonster Data Server.
Automatically unpack the files in the current folder. As a result, the flexmonster-data-server/ folder will appear in your working directory.

The -r option, which stands for --run, will install the Data Server as a Windows/Unix service and then run it. Besides, it will install and run Flexmonster Admin Panel – a graphical user interface for the Data Server.

Now follow the steps below to connect the Data Server to your CSV file.

Step 3. Add a new index

Now let’s create a new index for the file with your data.

Open Flexmonster Admin Panel and go to Indexes > Add New Index. Then, fill in the following fields:

Name. This field defines the index name. It will be used to configure the connection on the client side.
Type. This drop-down menu contains possible data source types. In this case, select CSV.
Path to file. In this field, specify the path to the file with your data. You can click the folder icon to choose the needed file using Explorer.
Delimiter. Specify the field delimiter if anything other than , separates СSV fields.
Decimal separator. It defines the specific character used to separate decimal parts of numbers. For example, a comma in 3,14. Default value: ..
Thousand separator. Defines the specific character used to separate groups of digits in numbers. For example, a period in 1.000 (one thousand). Default value: ,.
Refresh time. It defines how often the Data Server should reload the data from the file. The refresh time should be specified in minutes.
By default, the refresh time is 0, which means the Data Server will not reload the data.

Your configurations should look similar to the following:

When the index configuration is complete, click Create. The index will be automatically added to your index pool.

Step 4. Configure the report

On the client side, the report should be configured as follows:

var pivot = new Flexmonster({
    container: "pivotContainer",
    toolbar: true,
    report: {
        dataSource: {
            type: "api",
            url: "http://localhost:9500",
            index: "index-csv"
        }
    }
});

index must match the name of the index defined in step 3 (e.g., "index-csv").

When Flexmonster Pivot requests the data, Flexmonster Data Server sends the response and then caches it. In case the component sends the same request once again, the server responds with the data from its cache. The Data Server clears the cache when restarted.

The Data Server’s cache has a memory limit. When the cache does not have enough memory for a new response, the Data Server server deletes one of the previously cached responses.

You can manage the cache size via the Cache size limit configuration.

What’s next?

You may be interested in the following articles: