Visualize your MongoDB data with Pivot Table and Data Compressor

Veronika Rovnik posted on July 31st, 2018

Our team continues to work hard on the component’s performance optimization which has always been of great importance to our users and customers.

Therefore, in the light of a new 2.6 version of Flexmonster release, we are happy to inform you about the new significant feature – Compressor for JSON data source which will boost the performance of your project when loading the JSON data from MongoDB or other NoSQL Database into Flexmonster Pivot Table & Charts.

Now creating fascinating reports has become faster yet not less top-notch.

It suits you if:

you keep your data in JSON files
you are a developer who works with applications that generate massive amounts of structured, semi-structured or unstructured data and uses NoSQL databases such as MongoDB in favor of scalability and productivity
you build Node.js apps. Learn how to configure a connection to a database with the Node.js environment and Flexmonster Data Compressor.

You may ask…

What are the benefits of Flexmonster Data Compressor?

Significant improvement on optimization, data transfer and processing rates due to a reduced transfer size of the data.
Connecting to MongoDB or any other database becomes easier.
Creating interactive visualizations for your business reports becomes faster.

How does it work?

Let us get you acquainted with the principles of work of the JSON Compressor and explain why it’s a great tool for processing your data.

Data can consist of thousands and millions of rows or key-value pairs. Particularly, it’s worth mentioning that JSON format is not efficient and is redundant itself because of probable key repetition. The algorithm under the hood of Flexmonster Data Compressor allows efficient reducing the size of a local/remote file or a dataset from the database you are getting access to. It’s an important step before passing the data to the pivot table component.

As a result, the Compressor increases data loading speed from the server to the browser. Therefore, there is no need to be concerned about using too much RAM. Our special algorithm ensures your data is transferred in a matter of milliseconds.

Being a developer, you definitely know how milliseconds of latency may affect the work of the whole application. We recommend to connect to the database via the Compressor so as not to let that happen.

Data compression process

scheme — The process of data compression

Create a query to your MongoDB instance and pass the resulting stream to the Compressor.

The Compressor immediately returns a readable stream with an optimized structure of your complex data.

The data is returned in OCSV (Optimized CSV) format and is transferred then to the page that has a pivot table component embedded into it.

The loading and compressing process of a JSON file is analogous to the above-mentioned one except for the first step – the file is requested from the local or remote source of your choice.

How to start?

Firstly, embed Flexmonster Pivot Table & Charts into your application or website with the help of the Quick start guide.

Now it’s time to load your data into the component.

Let’s consider two possible scenarios:

Your data is stored in a MongoDB database.

Firstly, install the flexmonster-compressor dependency by using NPM:
```
npm install flexmonster-compressor
```
Then query your database for some data:
```
let dataStream = dbo.collection("my_collection").find().stream();
```
This statement gets all the data from the collection and converts it to the stream.

Compress your data:

const compressor = require('flexmonster-compressor');
let outputStream = compressor.compressJsonStream(dataStream);

And write the response:

outputStream.on('data', data => {
    res.write(data);
});
outputStream.on('end', () => {
    res.end();
});

Your data is stored in a JSON file.

The process is the same but instead of querying the database you can read data from the local file system:
```
let dataStream = compressor.compressJsonFile('./data.json');
```

Or you can pass a JSON array directly:

let jsonData = [{...}, {...}, ...];
let dataStream = compressor.compressJson(jsonData);

Find an example of a project on GitHub.

Try it yourself

We hope you are as excited about this new feature as we are.

Download the latest version of Flexmonster to make sure our tool works rapidly not only with a test data but with a real one and demonstrates high-performance indicators.

Stay tuned to new updates!