Beijer Electronics (formerly QSI Corporation)

Manufacturer of Mobile Data and Human Machine Interface Terminals.
It is currently Fri Nov 24, 2017 12:11 am

All times are UTC - 7 hours




Post new topic Reply to topic  [ 3 posts ] 
Author Message
 Post subject: Floating Point
PostPosted: Fri Dec 19, 2008 1:28 pm 
Offline

Joined: Wed Apr 30, 2008 2:37 pm
Posts: 14
The included workspace contains an interesting issue.
press the down key until the number drops below 1.
number should change by .1 from 1 to .9 but get a .89999 instead.
I am running this on the simulator not sure if it also occurs on G55 device also.


Attachments:
File comment: a G55 workspace
floatroundingissue.qly [24.04 KiB]
Downloaded 163 times
Top
 Profile  
 
 Post subject: Re: Floating Point
PostPosted: Fri Dec 19, 2008 1:41 pm 
Offline
QSI Support
QSI Support
User avatar

Joined: Wed Mar 08, 2006 12:25 pm
Posts: 881
Location: Salt Lake City, Utah
This is, in fact, an normal byproduct of the fact that floating point numbers on a PC cannot represent decimal values with exact precision.

Qlarity devices, including Simulation View and the G55 use the IEEE-758 floating point standard (which is the same standard used by nearly every platform these days, including your PC). Specifically Qlarity devices use 32 bit floating point math (your PC software may use 64 or 80 bit versions of the standard, but the same limitations apply).

Because 0.10000000... cannot be perfectly represented in floating point, when you perform arithmatic operations with it, you eventually see that the number is imperfectly represented as in your example. (As a side note, because of this, financial software uses sophisticated math packages and data representations to ensure that fractions of pennies are neither lost nor created in financial transactions).

Also, because of this, standard programming practice across any software platform recommends against comparing floating point numbers for equality. Instead you compare for near equality.


Code:
Library standard source formatfunctions
constant fpError := 0.00001

func PlayWithX()
    dim x as integer
    x = 1.0
    x = x - 0.1
    if (x >= 0.9-fpError) and (x <= 0.9+fpError) then
        'X is for all intents and purposes equal to 0.9
    endif

    'To display X as a string.  This will properly round
    'causing 0.8999999 to properly display as 0.900
    _print(_FormatFloat(x, 0, 3, _FORMAT_NORMAL))
endfunc

_________________
Jeremy
http://www.beijerinc.com


Top
 Profile  
 
 Post subject: Re: Floating Point
PostPosted: Fri Dec 19, 2008 3:07 pm 
Offline

Joined: Wed Apr 30, 2008 2:37 pm
Posts: 14
Thanks for the info.
Used the _FormatFloat to get the desired results.
For reference to anyone who might use the code _FORMAT_NORMAL should be _FLOAT_NORMAL.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 7 hours


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group