nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Problem with rounding

Macamba

In a report I'm developing I have a field containing the average of a downtime of a system. This downtime is expressed in minutes. I want to express it in hours and minutes. So I want to use a calculation like: 
<pre class='_prettyXprint _lang-auto _linenums:0'>
roundDown(minutes/60,0)+":"+roundDown(minutes mod 60,0)

roundDown(223/60,0)+":"+roundDown(223 mod 60,0) = 3:44
</pre>
 
But when I try to accomplish the first part of the equation (with mtrs_minutes is 224) I get 
<pre class='_prettyXprint _lang-auto _linenums:0'>
BirtMath.roundDown(BirtMath.divide(dataSetRow["mtrs_minutes"],60),0) = 3,6666666666666665
</pre>
While devision by 10 gives the correct answer. 
<pre class='_prettyXprint _lang-auto _linenums:0'>
BirtMath.roundDown(BirtMath.divide(dataSetRow["mtrs_minutes"],10),0) = 22
</pre>
 
What am I doing wrong?

Find more posts tagged with

Comments

Hans_vd

Macamba,

I don't want to be rude or anything, but it's hard to believe the results in your example really are the results of the formulas in your example. 224 / 60 equals 3.73333... and not 3.66666...

To get 3.66666... you need to do 220 / 60
In that case (your number of minutes being 220), the fact that when dividing by 10 the result is correct, is also explained.

So I wonder, are you looking to the right numbers? Are the results really the outcome of the expressions you think they are the results from?

Macamba

<blockquote class='ipsBlockquote' data-author="'Hans_vd'" data-cid="68371" data-time="1284387618" data-date="13 September 2010 - 07:20 AM">
Macamba, 
 
I don't want to be rude or anything, but it's hard to believe the results in your example really are the results of the formulas in your example. 224 / 60 equals 3.73333... and not 3.66666... </blockquote>
 
Thanks for pointing out the error in my calculation. 
 
My sample consist of the following three datums: 6, 127 and 540, (being dataSetRow["mtrs_minutes"]). The average of these datums is 224,3333. If I divide by 60, and round down to 0 digits after the dot/comma, I should get 3. If I mod the average by 60, and round down to 0 digits I should get 44 (the average downtime being 3:44). 
 
If I only look to the minutes the formula 
<pre class='_prettyXprint _lang-auto _linenums:0'>
BirtMath.roundDown(BirtMath.divide(dataSetRow["mtrs_minutes"],60),0) = 3,6666666666666665
</pre>
 
So, my calculation results in 3.66666...(, and not 3.73333...) and not 3 as I was expecting. 
 
Besides the difference between the calculation I get on screen, and the result that should be, roundDown does not work either.

Hans_vd

Can you also show the values of dataSetRow["mtrs_minutes"] and BirtMath.divide(dataSetRow["mtrs_minutes"],60) in you report and tell us what they are?

Macamba

<blockquote class='ipsBlockquote' data-author="'Hans_vd'" data-cid="68429" data-time="1284542781" data-date="15 September 2010 - 02:26 AM">
Can you also show the values of dataSetRow["mtrs_minutes"] and BirtMath.divide(dataSetRow["mtrs_minutes"],60) in you report and tell us what they are? </blockquote>
 
<pre class='_prettyXprint _lang-auto _linenums:0'>
dataSetRow DataSetRow/60
550,00 3,67
39,33 3,50
1,50
89,00 116,25
252,00 8,50
62,00 0,00
</pre>
 
The plot seems to thicken. Clearly 550,00/60 <> 3,67. Grouping seems to have its influence too.

Hans_vd

What do you mean by "grouping seems to have its influence too"? 
 
If you have a number and you divide it by 60, the result should be correct, no matter what else you have in your report. 
 
I mean, that's what you did, didn't you? You printed a number and you printed that exact same number divided by 60?