Learnem Educational
February 04, 2012, 12:51:15 PM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News:
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: float problem with gcvt(), precision, and Pentium 4??  (Read 2208 times)
Anonymous
Guest
« on: September 19, 2002, 06:59:54 PM »

Assume the following quick code sample:

void AddFloatText(LPSTR a,LPSTR b,LPSTR sum)
{
     double  an,bn;

     an = atof(a);                  // convert number string to float
     bn = atof(b);                  // convert number string to float
     an += bn;                       // add them
     (void) gcvt(an,16,sum);    // convert back to string
}

Seems reasonable, right?  Well, I am getting some unexpected results using Visual C++ 6.0 with latest service packs and libraries on a P4 running Windows XP.

For example, when I try to add  "4905.69" + "4905.69" with the string size set to 16 as above, I get a result of "9811.379999999999" instead of "9811.38."   If I set the size to 15 or lower, I get the correct answer. Huh?!

Similarly, this happens with other string quantities with decimal points to greater or lesser degree depending on how large I set the string size.   If I set the string size to be larger, like 20 or 30, I get many more errors in my datastream.  

The problem does not appear to be with the atof() function.  In the above case using atof, an=4905.6900000000 and bn=4905.6900000000 (according to the debug windows in VC++).  So, where is the precision error coming from?

Is this a floating point math bug in the Pentium 4?  Am I making some other glaring error in my use of these stdclib calls?  I can't imagine what it would be...

Your help is appreciated.

Thanks.
« Last Edit: January 01, 1970, 12:00:00 AM by Anonymous » Logged
Anonymous
Guest
« Reply #1 on: September 19, 2002, 07:21:32 PM »

I should add that the double value before using gcvt() is also correct, according to the debug window.

It is 9811.3800000000.  The problem occurs when I try and use gcvt() to go from double back to string and end up with 9811.379999999999.

Thanks...
« Last Edit: January 01, 1970, 12:00:00 AM by Anonymous » Logged
Anonymous
Guest
« Reply #2 on: September 19, 2002, 11:42:25 PM »

I had a few moments to debug and it looks like the IEEE overflow flag is being set.  Is there a limit for these functions of 16 digits?
« Last Edit: January 01, 1970, 12:00:00 AM by Anonymous » Logged
Anonymous
Guest
« Reply #3 on: September 20, 2002, 01:40:28 AM »

Well...for those like me who have never messed around with the floating point library routines and double data types, here is the answer:

The "double" data type only has a precision of 15 digits.  So, anything more than that will cause an overflow.  I didn't really think about the data type precision when I made the call.  I knew my data had much less precision and I thought it was just being 0-padded based on the supplied buffer size after the calculation on the significant digits.  The solution is not to specify a buffer bigger than 15 characters, since the library routines will attempt to calculate that precision and overflow.

Now you know and so do I.  Just double trouble.  :)
« Last Edit: January 01, 1970, 12:00:00 AM by Anonymous » Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!