Problems with precision..

rbuser

Hi all!
I have a problem. We develop financial systems which means a whole lot of
cases where one needs to, in an exact way, multiply for example 1,02 with 1
000 000. Any rounding errors makes a) the customers angry b) for a lot of
future problems.

The usual floating point errors are usually circumvented by using
decimal-type fields like BCD and so forth.
However, it seems that report builder, somewhere in the last steps before
displaying the value(using dbtext), converts the data into a floating point
value again, which in many cases results in rounding errors(like
1.0349999999999 that should be 1.035 and rounded up to 1.04..).
I have not yet found the strength to plow through all RP source code so I am
not exactly where it happens...But it sure feels like a
formatfloat...:hmm -)

Is there any way around this problem or is it a bug that has been fixed?
Since customers might have millions of shares, these small errors soon turns
into quite substantial values. Helluva lot more than i can afford,
anyway..:-)

//Nicklas

PS.
I am using RPEE 7.02
DS.

nicocizik

Hi Nicklas,

You are correct, when no display format is defined for a numerical value,
the FormatFloat method is used. You can see where this is done in the
ppDisplayFormat.pas file in the Format function. To work around this, you
will need to define the display format for each of your dbTexts and/or
change the source of the Format function to work as you need. If you choose
to change the RB source, do not change any of the Interface section code or
RAP will become un-usable.

--

rbuser

Ok. But i use formatting. The big problem is that the underlying BCD-value
seems to be converted into a floating value even though the source field is
a BCD-type field. This, when the report is printed, results in rounding
errors later on...:-(

//Nicklas

PS.
I hope i didn't misunderstand your reply in some way.
DS.

rbuser

Hm.. I just looked into that function.. There i found that the dtCurrency
type, which is supposed to be an exact representation of a value, is
formatted from the variant input parameter with the FloatToStrF function.
This mean that there is an implicit conversion to float which in "taints"
the data. For example 1.0695 could become 1.06949999999999 which obviously
creates a rounding problem.

//Nicklas

nicocizik

Nicklas,

Please send a small example showing this behavior to
support@digital-metaphors.com. I will need to know your exact display
format and numbers you are trying to calculate.

--

rbuser

Hi,

Hrm..... Then try this quick one:

Create a select statement (this is on a MS-SQL server with TADODataset):
SELECT cast(1.06950 as money)

Create a report, connect to the dataset, add a TppDBText-field.
Ok. Without formatting, everything looks normal since no additional
formatting has been performed.
I then apply a "#,0.000" DisplayFormat and looks again. On my system, it
says 1.069, which is wrong!
It should have been 1,070! (Would be right using either normal or bankers'
rounding either)
To check what data it is that makes this happen, i use a
"#,0.00000000000000000" DisplayFormat: 1.0694999999999990.
This is must be wrong. The reason (as i stated earlier) for this problem is
that dtCurrency-type data is formatted using floating point routines which
means that implicit conversions to float are made.
The solution for you (I'd rather not depend on local source changes that are
not in our CVS), at least in this case, would be to use CurrToStrF(BCDToStrF
for decimal data and so forth) or similar functions.

//Nicklas

PS.
A Decimal(28,5)-type generates the same error.
DS.

nicocizik

Hi Nicklas,

Thanks for the explaination. I went ahead and created a patch for RB 7.03
that should solve this problem for you. Please send an e-mail to
support@digital-metaphors.com and we'll send the patch right out to you.

--