This course will become read-only in the near future. Tell us at community.p2pu.org if that is a problem.

Defining precision


All measured data is only known to a certain degree of precision, which stems from experimental limitations, specifically relating to the properties of your measuring equipment.

For example, most analog thermometers will only reliably tell you the temperature to the nearest tenth of a degree.  This means that a temperature of 25 degrees Celsisus should really be recorded as 25.0 degrees Celsius, which in turn implies that your temperature is 25.0 +/- 0.1 degrees Celsius.

With calculated variables you are always limited by your least precise measured variable in determining your level of precision.  If, for example, you know the mass of a piece of metal to the nearest 0.1 grams and its volume to the nearest 0.01 cubic centimeters, then your density will only be reported to the nearest 0.1 grams per cubic centimeter.  As your mass measurement is only know to two significant figures in this case then your calculated density is only reliably known to two significant figures, even though the volume is known to three significant figures.

At this point start listing your precision values for your measured variables.  Then, using these limits, you can determine your precision values for your calculated variables.

Task Discussion