Reliability, which is a necessary condition for validity, is the extent to which repeated measurements made under the same conditions produce the same results. For many substances, the ESPAD questionnaire contains questions about lifetime use and also age at first use. The latter all include the response option ‘never’, which makes it possible to compare rates of lifetime prevalence for each substance according to these two sets of questions. Five substances are checked for inconsistency in relation to lifetime use in Table I as an indication of reliability. In addition, reported lifetime alcohol intoxication is also checked against reported lifetime use of alcohol.
The lowest inconsistency figures were found for cannabis and ecstasy use, with averages of 1 % each. In nearly all countries, the inconsistency rates are 0 % or 1 %, meaning that 99-100 % gave consistent answers about their consumption of these substances.
For the more common use of cigarettes, the average inconsistency rate was 3 %. Most countries had relatively low figures, with only four countries at 5 % or more; the highest figure (7 %) was found for Portugal. The average inconsistency figures were also relatively low (3 %) for use of tranquillisers and sedatives without a doctor’s prescription. Only just over four countries had a figure that was 5 % or higher. The highest average rate of inconsistency (4 %) is found for lifetime inhalants use. In 10 countries, 5 % or more of the students gave inconsistent answers. Inhalants are also the substance with the highest national rates of inconsistency. The top countries were Croatia (10 %), followed by Latvia and Georgia (9 % each).
With the exception of inhalant use, there are very few cases where the inconsistency rate is above 5 %. It should be recognised, however, that there are some technical discrepancies between the two types of measures of use which might contribute to inconsistency. One is the fact that the question about age at first use did not include a ‘do not remember’ response category. A student who has used a substance but does not remember how old he or she was the first time could conceivably decide to answer ‘never’ instead of guessing an age, especially if he or she has used that substance only once or a few times. Yet another factor contributing to inconsistency might be that students were ambivalent when answering the question about age at ‘first use’ of a substance. If a student had used a substance only once or twice and did not define himself or herself as a ‘user’, it may not have seemed appropriate to give an age when he or she ‘first’ used it (which may have come across as synonymous with the age at which he or she ‘started using’ it). These students may have answered ‘never’ since they think of their consumption as an experiment rather than the beginning of ‘real’ use.
Most substances controlled in Table I are probably relatively familiar to the students in the sense that they have heard about them. If a substance is familiar and mentioned in several questions, the students ought to use the same definition consistently. However, inhalants might be an exception in this respect. The concept includes a great many different agents that can be inhaled. If not all relevant agents are consistently given as examples in the questions that are compared, there is a risk that the students’ frame of reference will not be the same when they answer the two questions, which in turn may explain the lower consistency rates found for inhalants.
The proportion of students reporting having been intoxicated from alcohol use without previously reporting any lifetime use of alcohol was low. The ESPAD average was 0 % and the highest inconsistency rate was only 2 %, found in Albania.
In summary, it can be said that inconsistency figures for all variables controlled are relatively low in nearly all countries, indicating good reliability. Whilst still adequate, inhalant use showed the lowest levels of consistency, which is likely related to the fact that this concept is the most difficult to define. Particularly high scores are relatively uncommon, and no country scored among the highest for all variables. Albania could be mentioned as the country that tends to display the lowest consistency throughout. On the whole, inconsistency rates are not seen as reflecting any major reliability problems.
Table I. Some aspects of reliability. Inconsistency between two questions in a single administration. Students reporting substance use on one question but not on another a. Percentages. 2015
Download XLSCountry | Cigarettes LTP (C06 and C08a) | Cannabis LTP (C22a and C23) | Ecstasy LTP (C26a and C33d) | Inhalants LTP (C31a and C33e) | Tranquillisers or sedatives, non-medical use LTP (C32a and C33a) | Alcohol LTP: intoxication without consumption (C15a and C10a) |
Albania | 6 | 2 | 2 | 3 | 5 | 2 |
Austria | 2 | 1 | 0 | 5 | 2 | 0 |
Belgium (Flanders) | 1 | 0 | 1 | 1 | 2 | 0 |
Bulgaria | 4 | 2 | 3 | 2 | 2 | 1 |
Croatia | 4 | 1 | 1 | 10 | 2 | 0 |
Cyprus | 3 | 1 | 2 | 5 | 3 | 1 |
Czech Republic | 2 | 0 | 1 | 3 | 9 | 0 |
Denmark | 1 | 0 | 0 | 1 | 1 | 0 |
Estonia | 1 | 0 | 1 | 4 | 4 | 0 |
Faroes | 2 | 2 | 0 | 2 | 0 | 0 |
Finland | 1 | 0 | 0 | 2 | 1 | 0 |
FYR Macedonia b | 6 | 1 | 1 | 1 | 7 | 1 |
France | 1 | 0 | 1 | 3 | 2 | 0 |
Georgia | 3 | 1 | 2 | 9 | 9 | 0 |
Greece | 1 | 1 | 1 | 7 | 2 | 0 |
Hungary | 2 | 1 | 1 | 4 | 2 | 0 |
Iceland | 1 | 0 | 0 | 1 | 1 | 0 |
Ireland | 2 | 1 | 1 | 5 | 1 | 0 |
Italy | 2 | 2 | 1 | 2 | 2 | 0 |
Latvia | 4 | 1 | 2 | 9 | 3 | 0 |
Liechtenstein | 2 | 0 | 0 | 3 | 2 | 0 |
Lithuania | 4 | 1 | 1 | 4 | 3 | 1 |
Malta | 1 | 0 | 1 | 3 | 1 | 0 |
Moldova | 4 | 1 | 1 | 1 | 1 | 0 |
Monaco | 2 | 1 | 1 | 3 | 1 | 0 |
Montenegro | 5 | 1 | 2 | 3 | 3 | 0 |
Netherlands | 2 | 1 | 1 | 3 | 3 | 0 |
Norway | 2 | 0 | 0 | 3 | 1 | 0 |
Poland | 1 | 1 | 2 | 5 | 4 | 0 |
Portugal | 7 | 1 | 1 | 3 | 1 | 1 |
Romania | 4 | 1 | 1 | 3 | 1 | 1 |
Slovakia | 4 | 1 | 1 | 5 | 3 | 0 |
Slovenia | 3 | 1 | 1 | 6 | 1 | 0 |
Sweden | 1 | 0 | 1 | 3 | 2 | 0 |
Ukraine | 3 | 1 | 1 | 3 | 1 | 1 |
AVERAGE | 3 | 1 | 1 | 4 | 3 | 0 |
b Official name former Yugoslav Republic of Macedonia.