Precision problems in primitive datatypes due to Javascript

[pedrobranco]

As of the XSD specification allows that primitive datatypes Decimal, Double, Float and Integer to have a precision with more than 15 significant digits (example is via totalDigits).

When we are using parseFloat here / here and parseInt here makes the parsed values lose precision (javascript does not support float/number representation with more than 15 digits).

[mitar]

True, but that is a problem of JSON representation in general, no? There is no standard way to represent more precise numbers.

[pedrobranco]

True, but that is a problem of JSON representation in general, no?

It is not a JSON problem, but a Javascript problem.

There is no standard way to represent more precise numbers.

A lot of people use some modules to represent big numbers such as bignumber.js, but in this case, IMO we should return the value as string in every case that Javascript precision cannot represent the respective value.

[mitar]

So you are saying that we should check if totalDigits exists and if that is larger than 15, we should just keep it as a string?

Feel free to make a pull request for this.

[pedrobranco]

So you are saying that we should check if totalDigits exists and if that is larger than 15, we should just keep it as a string?

Instead of checking totalDigits we should just check if the parsed float / number loses precision comparing to the original value (there are some simple libs that already check this), and if so we should return it as string, or maybe passing via options custom handlers for parseFloat / parseInt. WDYT?

Feel free to make a pull request for this.

Thanks, when possible I will submit a PR.

[mitar]

No, I do not like this approach. The main reason why I made this package was because I wanted that resulting JSON has the same schema always. Not that I have to check if once something is an array, and something not, because xml2js made it not an array because there was only one element.

In this way I would not like that something is a number and in another document from the same collection of inputs it is a string.

So I would be more for approach that if schema is saying that there can be many digits, then we always make strings. Users can change their schema if they want less precision.

[mitar]

or maybe passing via options custom handlers for parseFloat / parseInt

We could do something like this as a workaround. But probably it can also be done manually in a postprocessing step?

[pedrobranco]

We could do something like this as a workaround.

Honestly, I would prefer passing the integerParser and floatParser functions via options with the default functions parseInt and parseFloat.

But probably it can also be done manually in a postprocessing step?

After the value is parsed with parseInt and parseFloat it cannot be recovered to its original state.

[mitar]

After the value is parsed with parseInt and parseFloat it cannot be recovered to its original state.

No, I mean, if there is totalDigits which are larger than what JSON can represent with float, then we keep it as a string. And if you want to have this processing of some values to float anyway, then you can postprocess the resulting object manually, instead of having to provide ways to hook into this step here.

[swftvsn]

I would say that custom integer / float parsers are more than welcome, so we can plug in our own implementations, like big.js or similar. It is really confusing if the data type can vary, like be a number or string. It would be better if it is only one type. (And provide the option to use custom parsers.)