function Int strange behaviour

[Djek]

hi all,

for label calculating i must create a checksum from number.

but i notice that sometimes the function integer (int) is behaving different...?

this is the code:

Code: Select all

let>varA=91
//varA=91
let>varB=%varA%/10
//varb=9.1
let>varC={int(%varB%)}
//varC=9
let>varD=%varB%-%varC%
//should be: 9.1 minus 9 =0.1 
//but result is varD=0.099999999999

as result varD i should get 0.1
but the calculation makes it 0.099999999
Is this also when you try it on your versions of MS ?

Why is this behaviour and how do i solve this ?

kind regards,
Djek

[Marcus Tettmar]

Odd. I'm investigating why this is. In the mean time you could do:

Format>%.1n,varD,res

[Djek]

thanks, Marcus,

the Format command solved the odd glitch,

kind regards,
Djek