Ch2.1.2 Precision

• Precision provides a measure for how specific a number is.
• Precision explains the level of detail in a measurement.
• As directions increase in precision when traveling, we get closer to destination, without being incorrect at each step.

• It is possible to estimate a value of x with high degree of precision, but this may not be accurate.
• A commonly used approximation of \( \pi \) is \( 22/7 \):

pi

[1] 3.141593

22/7

[1] 3.142857

• This estimate is accurate to two decimal places.
• The estimate given by \( 22/7 \) is a non-terminating decimal.
• Only number of digits shown limits its precision.

pi

[1] 3.141593

22/7

[1] 3.142857

• More accurate estimate of \( \pi \) would be first 8 decimal digits:

options(digits = 9)
pi

[1] 3.14159265

• This is less precise than \( 22/7 \) but has greater accuracy.

22/7

[1] 3.14285714

• While increased precision may sound beneficial, it can cause problems.
• A calculation may induce greater precision in final result than warranted.
• This can cause users to trust calculations more than they should.
• This is called false precision.
• False precision arises when a very precise estimate is presented, but the precision presented does not necessarily reflect precision actually included in number.

• For instance, a note saying a box will hold 5 kilograms might turn into 11.0231 pounds.
• In this case, the precision is generated by the conversion factor, 2.20462 pounds per kilogram.
• But to say the box will hold 11.0231 pounds adds a measure of precision that is probably unwarranted.
• The critical input value was the 5-kilogram figure.
• It has one significant digit, the 5.
• The final result should include only one, and be interpreted as 10 pounds.

• Significant digits, also called significant figures, are the parts of a number that include the precision of that number.
• This is limited to any non-zero digits in the representation of the number.
• The significant digits convey all of the precision of a number.
• For instance, the number \( \pi \) can be represented as 3, 3.1, 3.14, 3.142, 3.1416, 3.14159, and so on with an increasing number of significant digits.
• Accordingly, as the number of significant digits increases, so does the precision of the estimate.

• Significant digits are frequently encoded in scientific notation.
• The number \( 1.5 \times 10^8 \) is the same as 150 million and 150,000,000.
• When discussing the distance from the Earth to the Sun, this estimate was shown to be the center of a range.
• But as an estimate, 150 million kilometers is a good fit and \( 1.5 \times 10^8 \) conveys all of the critical information.
• The 1.5 carries all of the precision and the \( 10^8 \) carries the magnitude of the number.
• As a result, this estimate of the radius of the Earth's orbit contains two significant digits.

• However, when doing mathematics with numbers of limited precision, the lowest precision number determines the final precision.
• That is, the number with the fewest significant digits included carries that number of significant digits into the final calculation.
• That is why in the case of the 5-kilogram holding box, the estimate in pounds should only be 10, no matter how precise a conversion factor is used.

• This holds for multiplication and division of estimated values, and also for addition and subtraction.
• However, unlike with addition and subtraction, multiplication and division can change the magnitude without affecting the number of significant digits in the final result.

(6.4 * 10^(3))/(2.0 * 10^2)

[1] 32

• The process of rounding numbers reduces both the number of significant digits and the precision of an estimated value.
• For each place of rounding, the number of significant digits is reduced by one.
• Rounding \( \pi \) to 6 total digits is 3.14159.

pi

[1] 3.14159265

• Rounded to 5 digits, the value is 3.1416.

• Rounding \( \pi \) to 6 total digits is 3.14159.
• Rounded to 5 digits, the value is 3.1416.
• A significant digit is lost and a digit of precision is lost.
• The correct “5” was replaced by the incorrect “6”.

pi

[1] 3.14159265

• Rounding \( \pi \) to 6 total digits is 3.14159.
• Rounded to 5 digits, the value is 3.1416.
• Accuracy is lost, by a bounded amount.
• Rounding happens in last place and splits input range evenly.
• Upper bound on absolute error is 0.00005.

abs(3.14155 - 3.1416)

[1] 5e-05

• Actual error

abs(3.14159 - 3.1416)

[1] 1e-05

• Worst case scenario (error bound):

abs(3.14155 - 3.1416)

[1] 5e-05

• Actual error smaller than error bound, and in this case we know how by how much because we know the value of \( \pi \).