Visualize your MongoDB business data with Pivot Table using a new JSON Compressor

Veronika Rovnik posted on August 1st, 2018

Our team continues to work hard on the component’s performance optimization which has always been crucial for our customers.

Therefore, in light of a new 2.6 version of Flexmonster release, we are happy to inform you about the new significant feature – Compressor for JSON data source which will boost the performance of your project when loading the JSON data from MongoDB or other NoSQL Database into Flexmonster Pivot Table.

Now creating fascinating reports has become faster yet not less top-notch.

It suits you:

  • If you keep your data in a JSON format;
  • If you are a developer who works with applications that create massive amounts of structured, semi-structured and unstructured data and chooses NoSQL Databases such as MongoDB in favor of scalability and productivity;
  • If you use the Node.js platform to build your apps. The library for Flexmonster JSON Compression is available for it. See the documentation on how to configure the connection to the database with a Node.js environment.

You may ask…

What are the benefits of the novelty?

  • Significant improvement on optimization, data transfer and processing rates due to a reduced transfer size of the data.
  • An easier process of connection to MongoDB.
  • A faster process of interactive visualization for building your business reports.

How does it work?

We consider it our duty to get you acquainted with the principles of work of the JSON Compressor to convince why it is suitable and important for processing your data.

As you know, data can consist of thousands and millions of rows or pairs of key-values. Particularly, it’s worth mentioning that JSON format lacks efficiency and is redundant itself because of probable key repetition. The implementation of the algorithm allows reducing in size your local/remote file or dataset from the database you are using before passing it to our client-side pivot table component. This server-side utility known as Compressor helps to increase data loading speed from the server to the browser. Therefore, there is no need to be concerned about the high server load. Our special algorithm allows to do it in a matter of milliseconds.

As a developer, you may know how milliseconds of latency may affect the work of the whole application! That’s why the connection to the database should be done via Compressor.

A short overview of steps in process of compression when connecting to MongoDB:

scheme

 

  • Create a query to MongoDB and pass the result stream to Compressor.
  • Compressor immediately returns a readable stream with an optimized structure of your complex data.
  • As a result, the data is returned in OCSV (Optimized CSV) format and is transferred to the browser page with your web component of Pivot Table built into it.

The process with the loading of JSON file is similar with the mentioned above except for the first point – the JSON file is loaded from the static or dynamic source you’ve chosen.

How to start?

We hope you’ve already embedded a Flexmonster Pivot Table into your project with the help of out detailed Quickstart.

Now it’s time to load your data into the report.

Let’s consider two possible cases:

  1. Your dataset is stored in the MongoDB.

    2. Firstly, install flexmonster-compressor dependency using NPM:

    npm install flexmonster-compressor

    Then, query your database for some data:

    let dataStream = dbo.collection("my_collection").find().stream();

    Change the name of the collection from the database to the one you need to load. This statement gets all the data from the collection and converts it to the stream.

    Now, you can compress your data:

    const compressor = require('flexmonster-compressor');
let outputStream = compressor.compressJsonStream(dataStream);

    And write the response:

    outputStream.on('data', data => {
    res.write(data);
    });
    outputStream.on('end', () => {
    res.end();
    });

    You can find a full sample on our GitHub.

  2. Your data is stored in the JSON format

    3. The process is similar to the above, but instead of querying the database you can read data from the local file system:

    let dataStream = compressor.compressJsonFile('./data.json');

    or you can pass JSON array directly:

    let jsonData = [{...}, {...}, ...];
let dataStream = compressor.compressJson(jsonData);

    We hope you are excited about this important feature as we are.

    Stay tuned to new updates!

    Try it by yourself and make sure our tool works rapidly not only with test data but with the real one and demonstrates high-performance indicators.

    The new Сompressor is already available in the new version of Flexmonster.