Physics Practical Skills Part 2: Validity, Reliability and Accuracy of Experiments (2024)

Introduction to Validity, Reliability, and Accuracy of Experiments

Practical assessments are designed to test your practical skills: how well you can design and carry out an experiment and analyse results, but also your understanding of the purpose of the experiment and its limitations. One aspect of this is the reliability, validity, and accuracy of the experiment. So what do these terms mean and how do these things affect each other?

In this article we discuss:

• What is an experiment
• Validity
• Reliability
• Accuracy
• Relationship between reliability and accuracy

Want to ace your next Physics Practical Assessment?

Learn how to:

• Assess the validity, reliability and accuracy of any measurements and calculations
• Determine the sources of systematic and random errors
• Identify and apply appropriate mathematical formulae and concepts
• Draw appropriate graphs to convey relationships

with the Matrix Practical Skills Workbook.

What is an experiment?

An experiment is a set of measurements that are analysed to test a link or relationship between different things.The measurements are usually analysed in an experimental report, which is set out in a clear format to help the reader understand different aspects of the experiment, like the aim, the equipment and method used, the results obtained, how they were analysed, and what conclusions can be drawn.

When writing the method of an experiment, you must ensure that each step of the method incorporates reliability, accuracy or validity.

Validity

What is validity?

Validity relates to the experimental method and how appropriate it is in addressing the aim of the experiment:

• “Is my experiment suitable?” or
• “Does it test what it’s meant to test?” or
• “Am I actually measuring what I’m trying to measure?”

Several aspects of the experiment can contribute to validity: the equipment, the experimental method, and the analysis of the results.

Although it may seem obvious, the appropriate equipment needs to be used. The equipment must be suitable for carrying out the experiment and taking the necessary measurements.

The experiment is ultimately testing a relationship between cause and effect: how changing X affects Y. To address this, you must only change X, and see what happens to Y.If you allow other changes at the same time, then you cannot make a valid conclusion about how Xaffected Y, since Y may have been affected by the other changes as well.

The correct way to describe this is in terms of the independent, dependent, and control variables. The independent variable in an experiment is the one you set(X). The dependent variable is the one you measure (Y, because it depends on X). All other variables are called control variables, and they must be kept constant to prevent them from affecting the dependent variable. This forms part of the experimental method.

How do you improve validity?

The method (including the analysis) may contain some assumptions that need to be satisfied, e.g. maybe something has been simplified, or an equation being used is an approximation. The experimental method must ensure that all the assumptions are satisfied, otherwise, you will end up using a method or analysis that is inappropriate, and the result will be invalid. You may be able to identify invalid measurements and discard them from the analysis.

If your experiment is invalid, then the result ismeaningless because either the equipment, method or analysis were not appropriate for addressing the aim.

An example from Module 6 Electromagnetism, is an experiment involving transformers, using the transformer equation: \(\frac{N_p}{N_s} = \frac{Vp}{Vs} \).

• This equation assumes there is no flux leakage, so suitable equipment such as a ferromagnetic coremust be used.
• If this assumption is not satisfied, then the experiment will be invalid.

Reliability

What is reliability?

Reliability is about how close repeated measurements are to each other. You can consider the reliability of a measurement, or of the entire experiment.

How to you improve reliability?

A measurement is reliable if you repeat it and get the same or a similar answer over and over again, and an experiment is reliable if it gives the same result when you repeat the entire experiment.

How do you test reliability?

You can test reliability through repetition. The more similar repeated measurements are, the more reliable the results. However,repetition alone doesn’t make your measurementsreliable, it just allows you to check whether or not they are reliable.

Improving reliability is a different matter to testing it. The reliability of single measurements is not improved through repetition, but through the design of the experiment. Implementing a method that reduces random errors will improve reliability. However, the entire result of the experiment can be improved through repetition and analysis, as this may reduce the effect of random errors.

Summary of reliability

The table below provides a summary of reliability.

To learn more about random errors, read Physics Skills Guide Part 3: Systematic vs Random Errors.

The effect of validity on reliability of an experiment

Reliability can be affected by the validity of the experiment.

If an experiment is invalid because of an inappropriate method being used, the result may still be reliable, it just won’t address the aim of the experiment.

However, if an experiment is invalid because the control variables are not constant, then they may be affecting measurements in an unpredictable way, making the result unreliable.

Accuracy

What is accuracy?

Accuracy is much easier to define: the accuracy of an experiment is how close the final result is to the correct or accepted value.The closer it is, the more accurate the experiment.

How do you improve accuracy?

The accuracy can be improved through the experimental method if each single measurement is made more accurate, e.g. through the choice of equipment. Implementing a method that reduces systematic errors will improve accuracy.

Accuracy is not precision!

Note that, precision is a separate aspect which is not directly related to accuracy. Precision refers to the maximum resolution or the number of significant figures in a measurement. For example, a clock has a precision of 1 s, whereas a stopwatch has a precision of 0.01 s. Whether or not a measurement is accurate does not depend on the precision.

How do you test accuracy?

You can test the accuracy of your results by:

• comparing measurement to the value expected from theory for single measurements
• comparing the final experimental result to the accepted value for entire experiment’s result

Summary of accuracy

The table below provides a summary of accuracy.

To learn more about systematic errors, read Physics Skills Guide Part 3: Systematic vs Random Errors.

Relationship between reliability and accuracy

What is the relationship between reliability and accuracy?

Reliability and accuracy are separate aspects of an experimentandthe relationship between them is sometimes misunderstood.

Consider the following table:

Aresultcan be reliable and inaccurate if you get the same incorrect answer all the time (e.g. your friendis always 10 minutes late), and it can also be accurate and unreliable (e.g. your friend is more or less on time, but sometimes early, sometimes late).

A result can be reliable and inaccurate.

We can use shooting at a target as an example to further clarify our understanding of reliability and accuracy:

In summary, all combinations are possible:

• accurate and reliable
• accurate and unreliable
• inaccurate and reliable
• inaccurate and unreliable

How can you improve both accuracy and reliability?

Some steps can be taken to improve both accuracy and reliability.For example, if you use better quality equipment, your measurements can be more reliable and more accurate.

If the measurement is easier to do, then you’re more likely to get the same result in each repetition.

Considering, for example, an experiment in which youmust measure a short time period (around 1-2 s).

• If you time it by hand, your reaction time will introduce an error in the measurement.
• You can make the measurement more reliable by using light gates and a computer.
• You can also consider re-designing the experiment such that you need to measure a longer time period.Anyerror you then make is a smaller fraction of the measurement, and you are more likely to get the correct measurement (i.e. the same result) each time.

Ultimately though, you cannot make general conclusions about the reliability from the accuracy and vice versa. The reason is that they are affected by different types of experimental errors as mentioned above:

• accuracy is affected by systematic error, and
• reliabilityis affected by random errors.