How to deal with Number precision in Actionscript?

I have BigDecimal objects serialized with BlazeDS to Actionscript. Once they hit Actionscript as Number objects, they have values like:

140475.32 turns into 140475.31999999999998

How do I deal with this? The problem is that if I use a NumberFormatter with precision of 2, then the value is truncated to 140475.31. Any ideas?

Solution 1:

This is my generic solution for the problem (I have blogged about this here):

var toFixed:Function = function(number:Number, factor:int) {
  return Math.round(number * factor)/factor;
}

For example:

trace(toFixed(0.12345678, 10)); //0.1

• Multiply 0.12345678 by 10; that gives us 1.2345678.
• When we round 1.2345678, we get 1.0,
• and finally, 1.0 divided by 10 equals 0.1.

Another example:

trace(toFixed(1.7302394309234435, 10000)); //1.7302

• Multiply 1.7302394309234435 by 10000; that gives us 17302.394309234435.
• When we round 17302.394309234435 we get 17302,
• and finally, 17302 divided by 10000 equals 1.7302.

Based on the anonymous answer below, there is a nice simplification for the parameter on the method that makes the precision much more intuitive. e.g:

var setPrecision:Function = function(number:Number, precision:int) {
 precision = Math.pow(10, precision);
 return Math.round(number * precision)/precision;
}

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on

N.B. I added this here just in case anyone sees this as the answer and doesn't scroll down...

Solution 2:

Just a slight variation on Frasers Function, for anyone who is interested.

function setPrecision(number:Number, precision:int) {
 precision = Math.pow(10, precision);
 return (Math.round(number * precision)/precision);
}

So to use:

var number:Number = 10.98813311;
trace(setPrecision(number,1)); //Result is 10.9
trace(setPrecision(number,2)); //Result is 10.98
trace(setPrecision(number,3)); //Result is 10.988 and so on

Solution 3:

i've used Number.toFixed(precision) in ActionScript 3 to do this: http://livedocs.adobe.com/flex/3/langref/Number.html#toFixed%28%29

it handles rounding properly and specifies the number of digits after the decimal to display - unlike Number.toPrecision() that limits the total number of digits to display regardless of the position of the decimal.

var roundDown:Number = 1.434;                                             
// will print 1.43                                                        
trace(roundDown.toFixed(2));                                              

var roundUp:Number = 1.436;                                               
// will print 1.44                                                        
trace(roundUp.toFixed(2));                                                

Solution 4:

I converted the Java of BigDecimal to ActionScript. We had no choices since we compute for financial application.

http://code.google.com/p/bigdecimal/