As part of the development of the survey, Ipsos were commissioned to carry out an evidence review to look at the feasibility of Sport England collecting information from children aged 5-16 through a school-based survey.
This research informed decisions made about the survey such as the sampling, length and mode. Focus groups were also carried out to inform the design of the questionnaire and survey implementation.
Questionnaires were tested using cognitive interviews with children and young people, parents and teachers. Cognitive interviews involve administering survey questions and exploring how respondents understand the questions, recall the information, make judgements about their responses and then respond to the questions.
Following the development stage of the project, a pilot test of the survey was conducted in June-July 2017. The purpose of the pilot was to test the survey methods ahead of the main survey; specifically, to test engagement with schools and the quality and administration of the questionnaires and levels of response.
As a result of the pilot and earlier stages, the survey was implemented in the following way:
- selection of three classes per school and selection of one class per year group
- three main questionnaires but with variants for school year 3-4, 5-6, 7-11 and parents within the main questionnaire
- assumptions made about time spent doing activities at school
- questions about whether indoors or outdoors but no detailed questions on location
- an audio option was included in the school year 1-2 questionnaire but not the school year 3-11 questionnaire
- provision of a detailed information pack to schools in advance of taking part
- provision of online practice questionnaires (which do not collect data) to allow teachers to review the content in advance
- plans for imputation of missing or assumed data to maximise use of cases with don’t know answers and where questions were not asked because the answer could be assumed.
Owing to the wide age range covered by the survey, different questions were asked to different year groups. This reflected:
- whether questions are relevant (e.g. volunteering is not relevant for most school year 1-4 pupils)
- whether children could be expected to understand and answer questions on that topic (e.g. younger pupils were not expected to answer questions about life satisfaction)
- the time it took younger pupils to answer the questionnaire (questions were omitted for school year 3-4 pupils to reduce questionnaire length and burden)
- whether questions can be answered by parents on behalf of their child (e.g. parents were not asked about their child’s wellbeing).
The User Guide includes a table showing which groups were asked each question.
An objective measurement study was carried out by researchers at Sheffield Hallam University during 2017-18 to look at responses from the Active Lives questionnaire against objective data from a waist-worn accelerometer for pupils aged 5-16.
This also included pupils in school years 1 and 2, where parents completed the questionnaire with their child in the objective measurement study.
The study showed that self-reported data included both over and under measurement of activity level compared with the objective measurement.
This could be due to some reported activity sessions including preparation or briefing time and, therefore, while the self-report data has not been adjusted exactly to match the activity measured by an accelerometer, they are adjusted to reflect the known over-reporting in self-report data where appropriate.
More detail on the development of the survey and the objective measurement study can be found in the academic year 2017-18 (Year 1) technical report.
Since academic year 2017-18 (Year 1), further development work has been conducted to improve the survey.
Prior to academic year 2019-20 (Year 3), development work was carried out to design new questions about disability which would be accessible and inclusive, but which could be the same for all age groups and identify children with long-term limiting conditions, rather than the broader definition used in academic years 2017-18 (Year 1) and 2018-19 (Year 2) of the survey.
These new questions were introduced in academic year 2019-20 (Year 3) and continued to be used in academic year 2024-25 (Year 8). More details about this can be found in the academic year 2019-20 (Year 3) technical report.
For academic year 2020-21 (Year 4) of the survey, some new questions about motivations, whether they would like to do more activity, and ability to ride a bike were added to the questionnaire and some existing questions were modified.
In advance of academic year 2020-21 (Year 4), development work was carried out to design these questions so that they were appropriate.
Cognitive testing was carried out with 12 children and four parents using Microsoft Teams to test for understanding and acceptability. Cognitive testing was also carried out to investigate use of materials to encourage participation of parents of school year 1-2 pupils.
Findings from these interviews led to the development of the parent postcard and poster reminders that schools can send out to parents to encourage their participation in the survey. More details about this can be found in the academic year 2020-21 (Year 4) technical report.
Prior to academic year 2024-25 (Year 8), cognitive testing was carried out to test new questions on physical literacy with pupils. Teachers of pupils in years 3-4 were asked for their views on questions for pupils, as a result of feedback that pupils in those year groups found the questions challenging.
Overall, six teachers and 18 pupils were interviewed. Interviews took place between 24 June and 5 July 2024. For pupils, a range of participants of different ages (year 3 to year 11), genders and locations were recruited across four schools (two primary and two secondary).
Following cognitive testing, new physical literacy questions were introduced and changes were made to questions on time spent on activities, swimming, individual and community development, travel to school and ethnicity.
Plans were made for a future, more comprehensive, programme of testing with pupils prior to fieldwork in academic years 2025-26 and 2026-27.