Get Free Trial
Get Free Trial

How to use different separator for flex monster compressor data rather than default comma separator

manoranajan asked on April 5, 2019

Hi Team,
I want use different separator while compressing  flex monster data using flex monster compressor.By default its taking comma (,) as default separator .Due to comma separator i am getting some wrong data while using number format in my project . Suppose if have any number data which contains comma then flex monster treating it as new column.
 
Can you please tell is fieldSeparator option is there for “ocsv” datatype ?Because in my project if it is a csv datatype if it contains ‘~’ as a separator , if i am configuring fieldSeparator as ‘~’ then its working but for ocsv type this option is not working.

Please find the attached screen shot.
Regards,
Manoranjan
 

Attachments:
flex.jpg

21 answers

Public
Tanya Gryshko Flexmonster April 5, 2019

Hello, Manoranjan,

Thank you for writing to us.

If you are compressing the data from the database, Flexmonster Compressor does not need any fieldSeparators specified. In case you want to compress CSV file that contains ‘~’ as a separator, it can be specified as a second argument:

InputStream compressFile(String fileName, char delimiter)

Alternatively, for strings in CSV format and streams:

InputStream compressString(String string, char delimiter)
InputStream compressStream(InputStream inputStream, char delimiter)

Please let us know if it helps.

Regards,
Tanya

Public
manoranajan April 8, 2019

Hi Tanya,
Thank you for the response.
The above solution is not working for “ocsv” dataSourceType .
Please find the attached project in zip format
 

Public
manoranajan April 9, 2019

Hi Team,
Could you please respond on this ASAP ?
Regards,
Manoranjan
 

Public
Vera Didenko Flexmonster April 10, 2019

Hello, Manoranjan,
 
Thank you for your reply and for sharing a code sample.
It helped us a lot to understand the problem with preformatted numbers better and reproduce it on our side.
 

As a result, we noticed that the issue occurs when the decimal separator is “,”.

Our team is currently working on a solution to this issue and the fix will be available in the minor release with the ETA May 6th.

 
We kindly advise passing raw (not preformatted) data to Flexmonster Compressor. 

Please note that raw number data can be formatted on the client side in Flexmonster via the thousandsSeparator and decimalSeparator option in the formats parameter.
This can be achieved the following way:


"formats": [
{
"name": "2sfou03a",

/* SET DESIRED SEPARATOR FOR THOUSANDS */
"thousandsSeparator": ",",

/* SET DESIRED SEPARATOR FOR DECIMALS */
"decimalSeparator": ".",

"decimalPlaces": 2,
"currencySymbol": "$",
"positiveCurrencyFormat": "$1",
"nullValue": "",
"textAlign": "right",
"isPercent": false
}
]

 
 
We have prepared a few JSFiddle examples for illustration:
 
1) In this JSFiddle example, it is shown how the number format can be predefined in Flexmonster via the formats parameter.
2) In this JSFiddle example, you can see how the number format can be changed during runtime.
 
Could you please provide more information on how and where you get your data? It would greatly help us.
 
Please let us know if passing raw data would work for you.
 
We are looking forward to hearing from you.

Best Regards,
Vera

Public
manoranajan April 10, 2019

Hi Vera,
Thank you for quick response.
We are getting data from service call only and in our case we are using compress data only i.e ocsv type . We can not use raw data.
We will wait for the fix in the minor release.
Regards,
Manoranjan

Public
Vera Didenko Flexmonster May 8, 2019

Hello, Manoranjan,
 
We are glad to inform you that the issue with compressing numbers with comma as decimal separator in Java is fixed.
The fix is available on our website in the minor release 2.7.5 of Flexmonster.
You are welcome to update the component.
 
Please let us know if everything works fine for you.
 
Best Regards,
Vera

Public
Vera Didenko Flexmonster May 8, 2019

Hello, Manoranjan,
 

We would like to note that Flexmonster Data Compressor also needs to be updated.
Here is a link to how it is done: https://www.flexmonster.com/doc/updating-to-the-latest-version/#!compressor

Best Regards,
Vera

Public
manoranajan May 8, 2019

Hi Vera,
Thank you so much .
After upgrading compressor jar the issue is resolved
Regards,
Manoranjan

Public
manoranajan May 28, 2019

Hi Team,
We have a found another issue postupgrade as suggested above.
Here we have data that has quotes and commas in raw csv.
Example :
Atelier graphique~“2,003”~3138.46~Shipped~Classic Cars~Hernandez~2015 Aston Martin~103~05/20/2003.
But in the output ,we see that the data is altered as below.
Atelier graphique~%q%2%d%003%q%~3138.46~Shipped~Classic Cars~Hernandez~2015 Aston Martin~103~05/20/2003.
 
Attached the screen pic from running Flexmonster utility.
 
 
 

Attachments:
flexmonster_op.png

Public
manoranajan May 28, 2019

Hi Team,
We suspect this is the root cause why 2,003 is rendered as 2.003 on web pivot.A screen print of webpivot is attached.
 
Please let us know if any other information is needed to analyse the issue further.

Attachments:
webpivot_issue.png

Public
manoranajan May 30, 2019

Hi Team,
 
Could you please provide an update.
 
Regards,
Manoranjan
 

Public
Tanya Gryshko Flexmonster May 30, 2019

Hello, Manoranjan,

Thank you for writing to us.

There are a couple of things we’d ask you to check in order to fix this issue.

  1. We have noticed that the screenshot called flexmonster_op.png pictures the Data Compressor with version 2.312. Our team kindly suggests updating the Data Compressor to the latest version.
  2. The second point is how your OCSV output looks like. Your CSV file has ~ as separators:
    +Customername~Order Year
    Atelier graphique~"2,003"

    Such a file should be compressed by specifying this custom separator:

    InputStream inputStream = Compressor.compressFile(destPath,'~');

    The response from the Data Compressor should look the following way:

    ___ocsv2___2.7.6/2.213
    +Customername,+Order Year
    Atelier graphique,2%d%003

    Please note that ~ is replaced with , in this response. However, the comma inside "2,003" is represented as %d%. This allows Flexmonster to interpret this comma as comma displayed on the grid, not the field separator.

    We suggest making sure your OCSV response looks the same as shown above.

Please let us know if the issue is resolved.

Regards,
Tanya

Public
manoranajan May 31, 2019

Hi Tanya,
 
Thanks for your response.
We have tried by upgrading compressor and i see that comma is replaced with %d.
However we still see 2,003 is rendered as 2.003 on web pivot.I have attached a screenprint webpivot_issue.png in my other comment.
 
Please suggest if anything needed to analyse this further.

Attachments:
flexmonster_op1.png

Public
Vera Didenko Flexmonster June 3, 2019

Hello, Manoranjan,
 
Thank you for writing to us and for providing a screenshot for illustration.
 
We have managed to reproduce the issue on our end.
 
The problem is that the ~ in the ocsv output were supposed to be changed by the Data Compressor to ,.
 
Our development team is working on a solution to the problem and the fix will be available in the minor release with the ETA 18th of June.
 
One more suggestion we recommend applying right now is adding the data types to the CSV data you pass to the Data Compressor.
This means that your CSV header should contain types.

Header without types:
Customername~Order Year~Sale Price

Header with types:
+Customername~+Order Year~-Sale Price
 
Feel free to contact us in case any questions arise.
 
Best Regards,
Vera

Public
manoranajan June 4, 2019

Hi Vera,
Thanks for your response.
Just to make sure that our understanding is same i am reiterating the expectation here.
When there is a comma inside a data field example 2,003 it is being rendered as 2.003.
We will wait for the minor release and then retest.
 
 
Thanks
Manoranjan

Public
Vera Didenko Flexmonster June 4, 2019

Hello,  Manoranjan,
 
Thank you for writing to us.
 
At the moment the custom delimiter is being ignored (that is why the OCSV output is not correct). The fix which is currently in progress should resolve this issue.
 
Please note that adding the types to your CSV header is also a necessary step towards fixing the issue.
 
You are welcome to contact us in case of any questions.
 
Best Regards,
Vera

Public
Vera Didenko Flexmonster June 18, 2019

Hello, Manoranjan,

We are glad to announce that the issue with setting custom delimiter for compressing CSV files was fixed.

This is available in the 2.7.8 version of Flexmonster: https://www.flexmonster.com/release-notes/

You are welcome to update the component: https://www.flexmonster.com/doc/updating-to-the-latest-version/ 

A kind reminder is to update Flexmonster Data Compressor also: https://www.flexmonster.com/doc/updating-to-the-latest-version/#!compressor 

Please let us know if everything works fine.

We are looking forward to hearing from you.

Best Regards,

Vera

Public
manoranajan June 21, 2019

Hi Vera,
We have upgraded compressor jar to 2.7.8 but we were unable to resolve the issue.
We see that comma inside a value is being replaced by a dot(.) on web pivot.We do not want this alter to take place.
Attached zip contains input file to compressor and output from compressor.
 
Thanks
Manoranjan
 

Attachments:
TestRun.zip

Public
Vera Didenko Flexmonster June 21, 2019

Hello, Manoranjan,
 
 
Thank you for writing to us and providing test data to understand the problem better.
 
We would like to point out that the - data type prefix specifies that the type of data provided in the specified column will be of type value (number).
In order for it to be recognized as a dimension (string), the + data prefix needs to be added to the column name instead.
 
Please change the data type prefix for the Order Year in the CSV header from - to +:

...,+Order Year,...

After this, Order Year data should be displayed in Flexmonster as desired. 
 
 
Please let us know if this works.
 
 
We are looking forward to hearing from you.
 
 
Best Regards,
Vera

Public
manoranajan June 24, 2019

Hi Vera,
We are treating Order year filed as a number field so we prefixed it with “-“.But the value is enclosed inside double quotes(“2,003”).We arrive at this value after applying our custom number formatting rules.
We cannot modify the datatype of this field as we may miss some of the calculation functions if i change it to String type.
could you please suggest if  their is a way to  treat it as number and display its value on the grid as is without modifying the data/datatype?
 
Thanks
Manoranjan

Public
Vera Didenko Flexmonster June 25, 2019

Hello, Manoranjan,

 
It seems there may be some sort of misunderstanding so let’s clarify some points.
 
As we understand, Order Year values must be of type number.
Values, such as Sale Price, must be displayed in Flexmonster on the grid with , as the thousands separator.
 
Could you please confirm whether the Order Year values must also be displayed in Flexmonster on the grid with , as the thousands separator?
 
 
Please note that Flexmonster Data Compressor needs to be given raw data without number formatting, otherwise the results can be unexpected.
 
The recommended approach is to pass data without number formatting: 

+Customername, -Order Year, -Sale Price, +Status, -Customernumber
Atelier graphique, 2003, 3138.46, Shipped, 103
Atelier graphique, 2003, 5282.64, Shipped, 103

And then apply number formatting with Flexmonster.
 
We would like to bring to your attention that Flexmonster provides the ability to set number formatting, such as thousands separators.
This means applying custom number formatting rules can be done on the client side. 
 
For example, you can pass the 4515905.51 Sale Price value to Flexmonster. Then you can specify the formats parameter in Flexmonster the following way: 

"formats": [
{
"name": "", //since name is not specified, the format will be applied to all values
"thousandsSeparator": ","
}
]

 
As a result, 4515905.51 will be displayed as 4,515,905.51 in Flexmonster.
 
Here is a JSFiddle for illustration: https://jsfiddle.net/flexmonster/5hdcqs10/.
 
 
Please let us know if we understand your case correctly.
 
We are looking forward to hearing from you.
 
Best Regards,
Vera

Please login or Register to Submit Answer