Rounding and Significant Figures¶
The Story Behind the Math¶
In 1940, aeronautical engineers at the National Advisory Committee for Aeronautics (NACA, precursor to NASA) encountered a disaster. A critical calculation for wing design was off by 10%—not because the math was wrong, but because engineers rounded intermediate results too aggressively.
The problem: Rounding errors compound. A small error in step 1 becomes a larger error in step 10.
This led to the formalization of significant figures—rules for tracking precision through calculations. During WWII, similar issues plagued artillery calculations, ballistics tables, and atomic research. Precision wasn't just academic—it was life or death.
The modern irony: Computers calculate with extreme precision, but humans still need to decide what to report. Should that be 3.14 or 3.14159? The answer depends on what you know, not what your calculator shows.
Why It Matters¶
Rounding and significant figures affect:
- Science: Reporting measurements with appropriate precision
- Engineering: Ensuring calculations maintain required accuracy
- Finance: Currency calculations, exchange rates, interest
- Medicine: Drug dosages, test results, treatment protocols
- Computing: Floating-point arithmetic, numerical stability
- Everyday life: Estimating costs, distances, quantities
Understanding precision prevents false confidence in numbers and catches calculation errors.
Prerequisites¶
- Decimal-Fraction-Conversion — Decimal places and precision
- Place-Value — Understanding digit positions
- Basic arithmetic operations
The Core Insight¶
Why We Round¶
Measurements are never exact. A ruler marked in millimeters can't measure micrometers. Rounding reflects the limit of our knowledge.
If you measure a table as 1.24 meters using a tape measure marked in centimeters: - You know it's about 1.24 meters - You don't know if it's 1.241 or 1.239 meters - Reporting 1.2400 would be dishonest—you don't have that precision
The Fundamental Tension¶
- More digits: Implies more precision (may be misleading)
- Fewer digits: Loses information (may be insufficient)
Significant figures are the compromise: report all digits you know, and no more.
Rounding Rules¶
The Basic Rule¶
To round to a specific place: 1. Identify the digit at the target place 2. Look at the digit immediately to its right 3. If that digit is 5 or greater, round up the target digit 4. If less than 5, keep the target digit unchanged 5. Replace all digits to the right with zeros (or drop them after decimal)
Examples¶
Round 3.14159 to: - 2 decimal places: 3.14 (next digit is 1 < 5) - 3 decimal places: 3.142 (next digit is 5 ≥ 5, round up) - 1 decimal place: 3.1 (next digit is 4 < 5) - Whole number: 3 (next digit is 1 < 5)
Round 2,847 to: - Nearest 10: 2,850 (4 < 5, but we need the 0 placeholder) - Nearest 100: 2,800 - Nearest 1,000: 3,000
The "5" Rule (Banker's Rounding)¶
What about exactly 5? Standard rule: round up.
But this introduces bias—over many calculations, you'll round up slightly more often.
Banker's rounding (IEEE 754 standard): - Round to nearest even digit when exactly halfway - 2.5 → 2, 3.5 → 4
This balances rounding up and down over time.
Significant Figures¶
What Are Significant Figures?¶
Significant figures (sig figs) are the digits in a number that carry meaning about its precision.
Rules for counting significant figures:
- Non-zero digits are always significant: 345 has 3 sig figs
- Zeros between non-zero digits are significant: 3.05 has 3 sig figs
- Leading zeros are NOT significant: 0.0045 has 2 sig figs
- Trailing zeros after decimal ARE significant: 3.400 has 4 sig figs
- Trailing zeros without decimal are ambiguous: 3400 could be 2, 3, or 4 sig figs
Examples¶
| Number | Significant Figures | Explanation |
|---|---|---|
| 123 | 3 | All non-zero |
| 0.0123 | 3 | Leading zeros not significant |
| 100.5 | 4 | Zeros between non-zeros are significant |
| 100 | Ambiguous | Could be 1, 2, or 3 |
| 100. | 3 | Decimal indicates precision |
| 1.00 × 10² | 3 | Scientific notation makes it clear |
Scientific Notation Removes Ambiguity¶
Instead of writing 3400 (ambiguous), write:
- \(3.4 \times 10^3\) = 2 sig figs
- \(3.40 \times 10^3\) = 3 sig figs
- \(3.400 \times 10^3\) = 4 sig figs
The mantissa (the part before × 10ⁿ) always shows significant figures clearly.
Operations with Significant Figures¶
Multiplication and Division¶
The result has the same number of significant figures as the factor with the fewest sig figs.
Example: \(4.56 \times 1.4\)
- 4.56 has 3 sig figs
- 1.4 has 2 sig figs
- Result should have 2 sig figs: 6.4
Why? The least precise measurement limits the precision of the result. If you only know one factor to 2 sig figs, you can't magically gain precision by calculating.
Example: \(100 \div 3.0\)
- 100 (ambiguous, assume 1 sig fig)
- 3.0 has 2 sig figs
- Result should have 1 sig fig: 30
Addition and Subtraction¶
The result has the same number of decimal places as the number with the fewest decimal places.
Example: \(12.345 + 1.2\)
- 12.345 has 3 decimal places
- 1.2 has 1 decimal place
- Result should have 1 decimal place: 13.5
Why the different rule? With addition, decimal place matters, not total sig figs. You can't know the thousandths place if one number is only precise to tenths.
Example: \(1500 + 24.5\)
- 1500 (ambiguous precision, assume ones place)
- 24.5 has tenths place
- Result should be rounded to ones: 1525
Combined Operations¶
When mixing operations, track significant figures at each step, or keep extra digits during calculation and round only at the end.
Example: \((4.5 \times 3.12) + 2.1\)
Step 1: \(4.5 \times 3.12 = 14.04\) → round to 2 sig figs: 14
Step 2: \(14 + 2.1 = 16.1\) → round to ones place: 16
Or with extra precision: \(14.04 + 2.1 = 16.14\) → round to 16
The Mathematical Foundation¶
Error Propagation¶
When we write \(3.14\), we mean "somewhere between 3.135 and 3.145."
The range of uncertainty: - \(3.14 \pm 0.005\) (2 decimal places) - This represents about 0.16% uncertainty
Multiplication error: If \(x\) has uncertainty \(\Delta x\) and \(y\) has uncertainty \(\Delta y\):
The relative errors add. This is why sig figs multiply the way they do.
Logarithms and Significant Figures¶
In logarithms, the number of decimal places in the result equals the number of sig figs in the input:
3.0 has 2 sig figs, so the answer has 2 decimal places.
Exact Numbers vs. Measurements¶
Exact Numbers¶
Some numbers have infinite precision: - Counted quantities: 12 eggs, 5 students - Defined conversions: 1 inch = 2.54 cm (exactly) - Mathematical constants in formulas: \(\pi\), \(e\)
Exact numbers don't limit significant figures.
Example: Calculate circumference of circle with radius \(r = 2.5\) cm
- \(2\) is exact
- \(\pi\) is a mathematical constant (use as many digits as needed)
- \(2.5\) has 2 sig figs
- Result: 16 cm (2 sig figs)
Practical Applications¶
Scientific Reporting¶
Wrong: "The reaction took 45.23847 seconds"
Right: "The reaction took 45 seconds" (if measured with a stopwatch)
Only report digits you actually measured.
Engineering Tolerances¶
A specification says: "Length: \(100 \pm 0.5\) mm"
This means: - Acceptable range: 99.5 mm to 100.5 mm - Measurement should be reported to tenths place - 100.0 mm implies ±0.05 mm precision (too precise!)
Financial Calculations¶
Currency is often rounded to cents (2 decimal places), but:
- Intermediate calculations should keep full precision
- Only round the final result
- Rounding each step creates cumulative errors
Example: Three items at $1.333 each
Step-by-step rounding: $1.33 + $1.33 + $1.33 = $3.99 Exact: $3.999 → $4.00
Computing¶
Floating-point precision: Computers use binary floating point with about 15-16 decimal digits of precision.
Catastrophic cancellation: Subtracting nearly equal numbers loses precision:
Both numbers had 8 sig figs, but the result has only 1!
Common Misconceptions¶
-
"More decimal places = more accurate": No! Extra digits from calculator don't add precision.
-
"Round at each step": Don't! Round only at the end to avoid error accumulation.
-
"Trailing zeros are never significant": They are after a decimal point (3.400 is more precise than 3.4).
-
"Exact conversion factors limit precision": Defined conversions (1 inch = 2.54 cm) are exact—they don't limit sig figs.
-
"All measured digits are significant": Not leading zeros. 0.0045 has 2 sig figs, not 4.
Visual Understanding¶
The Precision Ladder¶
Most Precise Least Precise
↓ ↓
3.14159 → 3.1416 → 3.142 → 3.14 → 3.1 → 3
(6 sf) (5 sf) (4 sf) (3 sf) (2 sf) (1 sf)
Each step loses information but gains clarity.
Measurement Precision¶
Ruler marked in millimeters (mm):
You can estimate to nearest 0.5 mm, so measurements should be reported as: - 23.5 mm (3 sig figs) - Not 23.500 mm (you don't have that precision)
Best Practices¶
For Students¶
- Keep extra digits during calculation, round at the end
- Use scientific notation to avoid ambiguity
- Understand your measuring tool's precision
- Report uncertainty when possible (\(23.5 \pm 0.5\) mm)
For Scientists and Engineers¶
- Always state precision explicitly: "3.14 (3 sig figs)"
- Propagate errors properly for critical calculations
- Consider logarithms for multiplicative processes
- Use interval arithmetic for safety-critical systems
Related Concepts¶
- Decimal-Fraction-Conversion — Precision in different forms
- Percentage-Changes — Error percentages
- Statistics — Standard deviation and confidence intervals
References¶
- Taylor, J. R. (1997). An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements (2nd ed.). University Science Books.
- ISO 80000-1:2009. Quantities and units—Part 1: General.
- Higham, N. J. (2002). Accuracy and Stability of Numerical Algorithms (2nd ed.). SIAM.