Belgian Political Panel Survey (BPPS), Technical Report

Transcriptie

1 KU Leuven Belgian Political Panel Survey (BPPS), Technical Report Marc Hooghe, Nele Havermans, Ellen Quintelier, Ruth Dassonneville K.U. Leuven Leuven, July 2011

2 Marc Hooghe, Nele Havermans, Ellen Quintelier, Ruth Dassonneville (2011). Belgian Political Panel Survey (BPPS), Technical report. Leuven: K.U. Leuven. K.U. Leuven Overname van de gegevens is toegestaan mits correcte bronvermelding. M. Hooghe, Parkstraat 45/3602, B-3000 Leuven (

3 Content CONTENT 1 INTRODUCTION 1 2 THE BELGIAN YOUTH SURVEY QUESTIONNAIRE SELECTION OF SCHOOLS FIELDWORK: VISITING SCHOOLS CODING WEIGHING 5 3 THE BELGIAN YOUTH SURVEY QUESTIONNAIRE FIELDWORK: VISITING SCHOOLS FIELDWORK: MAIL SURVEYS CODING WEIGHING 11 4 THE BELGIAN POLITICAL PANEL SURVEY ( ) LINKING THE BELGIAN YOUTH SURVEY 2006 AND THE BELGIAN YOUTH SURVEY PANEL ATTRITION WEIGHING WRAP UP: THE BELGIAN POLITICAL PANEL SURVEY THE BELGIAN YOUTH SURVEY QUESTIONNAIRE FIELDWORK 20

4 5.3 CODING INCENTIVES WEIGHING OF THE DATA 28 REFERENCES 31

5 1 Introduction The political socialization of new age cohorts can be considered as a perennial focus of research in political science. In the more recent literature, various authors have expressed concern about what they perceive as a lack of civic engagement and political interest among young age groups. One of the main shortcomings in the current research however, is that most studies are based on cross-sectional observations. These observations often do not allow to disentangle any causal relations. Ideally, determining causality would require access to longitudinal data, and that is exactly why we decided to start a panel study of Belgian youth. The first two waves of this panel study have now been implemented, allowing for a first, full analysis. The data of the first two waves have been assembled in one dataset, the Belgian Political panel Survey (BPPS). The BPPS is the main dataset that will be used for the analyses. The BPPS is a two-wave panel survey of respectively sixteen and eighteen year olds in Belgium (both the French and Dutch-speaking communities). In the first wave, a representative sample of 6,330 young people in high school (4 th year of secondary school or 10 th grade) were surveyed about their school environment, citizenship, political attitudes (political interest, efficacy, trust etc.), political behaviour (future and current engagement) and political socialization agencies. In the first chapter, we will present the data from the first wave of the BPPS : the Belgian Youth Survey This data forms the baseline of the panel, and they contain information from more than 6, year-old respondents. The second chapter describes the second wave of the panel: the Belgian Youth Survey 2008, which also contains data about more than 6, year-old respondents. The respondents of the Belgian Youth Survey 2008 are those that were in the same classes as in 2006 (both new and old students) and those that participated in 2006 but did not participate in the school survey 2008 and filled in the mail survey. In this chapter we explain how we tried to resurvey as much respondents as possible. Finally, in chapter 3 we explain how both datasets are linked and the characteristics of the BPPS

6 2 The Belgian Youth Survey 2006 In this chapter, it will be explained how the Belgian Youth Survey 2006 (e.g. the first wave of the BPPS) was constructed: how the questionnaire was developped, the schools were selected, contacted and visited. Finally, we set forth how the surveys were coded and how the weighs for the data are constructed. 2.1 Questionnaire The data collection was based on a written school survey among 16-year-old respondents. This survey method was chosen in 2006 because it would allow interviewing a group of young people at the same time in their school environment. The Belgian survey was based on the McGill Youth survey, that was previously developed by Prof. dr. M. Hooghe and Prof. dr. D. Stolle (McGill University, Canada). Each survey contained a small card which the students had to complete to allow linking the surveys of 2006 with those that would be filled in later in the panel study. Self-evidently, all measures have been taken to protect the privacy of the respondents, according to the strict Belgian legislation (Law of 8 December 1992). Because we selected both French- and Flemish speaking schools (see further), two versions of the questionnaire were developed: a French and a Dutch one. The French and Dutch version of the questionnaire were read through by native speakers and tested among a sample of students from general and vocational education. The first part of the school survey contained individual level variables as gender, age, socio-economic status, religion, membership of voluntary associations etc. These questions allowed the respondents to go relatively fast through the first pages of the 20 page booklet. The second part was the core module of the questionnaire, and surveyed young people about their political involvement and interest. The third part contained questions about young people s school environment: what is the school climate like, what activities do they do in school, etc. The fourth part contained questions about the family: additional questions of parental socio-economic status and parents political involvement. The final part of the questionnaire dealt with questions about racism and discrimination. Although it was quite a long survey (e.g. 89 questions), 84 percent of the respondents were able to fill in the survey within 50 minutes (or a standard lesson in Belgian schools). If they were unable to this, this had most of the time to do with the available time. For a 2

7 variety of reasons, it was sometimes impossible to start the survey in time: classes that forgot they had to take part in the survey, availability of the rooms etc. 2.2 Selection of schools The schools were selected through a stratified sample, based on the location of the school (province) and educational system (public or private education). In 2006, both in Flanders and the Walloon region, a sample was drawn of respectively 60 and 52 schools. This number of schools was chosen because it allowed to survey about 6000 students, a number that allowed to model also the effects of smaller sub samples (for instance of immigrants). Furthermore, a larger sample size reduces standard deviations. Fewer schools were selected in the Walloon region as there are less schools and inhabitants in that region compared to Flanders. In Table 1 we present the number of schools that were selected in each province and educational system. Though we could, at least theoretically, have stratified our sample even further, we could practically not go that far since the cells would become too small to draw a sample. As can be seen in Table 1, most schools are private (e.g. Catholic) schools in Belgium, which is due to historical circumstances. In Belgium, private schools account for 75 percent of the enrolled students 1. Most schools are in the province of Antwerp, the least in Limburg. In each school, there were at least 50 students selected, according to the offered educational tracks. So if a school had general, technical, artistic and vocational education, one or more classes of each track were surveyed, with a total of at least 50 students

8 Table 2.1. Sampled schools by educational system and province. Educational system Total Private education Public education 1 Public education² Antwerp 11 (144) 3 (47) 2 (35) 16 West-Vlaanderen 8 (120) 2 (36) 1 (3) 11 Limburg 5 (69) 2 (29) 1 (9) 8 Oost-Vlaanderen 8 (117) 3 (44) 1 (18) 12 Flemish Provinces Walloom Provinces Vlaams Brabant + Brussels Capital 8 (79) 4 (37) 1 (13) 13 Subtotal Brabant wallon + Brussels Capital 8 (81) 3 (28) 4 (33) 15 (142) Liège 5 (63) 3 (30) 2 (21) 10 (114) Namur 5 (42) 2 (20) 0 (4) 7 (66) Luxembourg 3 (27) 1 (12) 0 (1) 4 (40) Hainaut 9 (83) 5 (43) 1 (26) 15 (152) Flemish territory³ Subtotal Total Entries are number of selected schools and original number of schools between brackets. 1 : Public education: funded by the Community. ²: Public education: funded by Provinces and municipalities. ³: There is one French school on the Flemish territory that was selected in the sample. In the sample, it was taken together with the neigbouring province of Hainaut. Schools that did not want to participate in the survey were replaced by other schools from the same province and the same educational system. In total, five different samples were drawn to allow replacing the schools. In Flanders, the contacts yielded into a response rate of 72 percent compared to 60 percent among the French-speaking schools. A possible reason for these lower response rates is that they were contacted by a Flemish university (and not a French one), which makes it less important/interesting for them to participate. 2.3 Fieldwork: visiting schools The 112 selected schools were contacted early 2006 to report that they were selected for a two-wave panel survey and that they would be contacted by one of the researchers in the coming weeks to make an appointment. Each time, 10 to 15 letters were (at random) sent to the selected schools to announce the telephone call. The schools were contacted several times by telephone to make an appointment for the school visit. Each school received the same standard introduction: that it was a large-scale, Belgian study to survey the social and political attitudes of the pupils, that the survey took 50 minutes, that it was adapted to 4

9 their level of understanding, etc. Each school was visited by one of the three researchers on the project or some well-trained job students, so that the pupils could ask every question they wanted to ask to the researchers. In the schools, students were surveyed class per class or in a large room, depending on the preference of the school and the number of the students that were surveyed. 2.4 Coding The surveys were coded by job students. First, after an extensive explanation about the project and the coding, the job students were provided with 20 surveys, an excel file and a codebook. After they entered these surveys, they were extensively controlled by one of the researchers on the project. The following times, they were allowed to take more surveys with them and send them back at regular times. It goes without saying that these codings were also controlled. Furthermore, several surveys where randomly coded twice to control for the reliability of the job students. Finally, the whole dataset was controlled for outliers. 2.5 Weighing Afterwards, the data was controlled for the number of pupils according to offered tracks (general, technical and vocational) and gender, and this weighing shows that the data are quite representative for the Belgian school population (Table 2). Girls are somewhat overrepresented in both regions: while they are almost 27 percent in the student population in Flanders, they are only 25 percent in the sample. A similar discrepancy is present in the Walloon data. However, this is not problematic because the weights are still acceptable. 5

10 Table 2.2. Weighing coefficients (BYS 2006) n in population % in population n in sample % in sample Weighing coefficients Boys in Flanders 35, , Girls in Flanders 34, , Boys in the 29, , Walloon region Girls in the 28, , Walloon region Missing Total 128, , Source: (11/03/08), French data received through from the Ministry of Education of the French Community in Belgium. 6

11 3 The Belgian Youth Survey 2008 In 2006, 60 Flemish schools and 52 Walloon schools were surveyed, which yielded a total of 6,330 respondents. In 2008, we tried to resurvey the same schools and the same pupils again. Unfortunately, this was not possible for every school: in Flanders, one of the schools did not exist anymore and one school did not offer the surveyed tracks anymore. In the Walloon region, one school refused to participate. Therefore, these schools were not resurveyed. This resulted in a response rate of 97 percent at the school level. Table 3.1. Number of participating schools in each region Schools 2008 Flanders Walloon region Total (in %) Surveyed (97.32) Refused (0.89) Stopped (1.78) Total (100) 3.1 Questionnaire As this is a panel survey, we relied heavily on the BYS 2006 questionnaire for the creation of the BYS 2008-questionnaire. First, we decided if we should keep the questions based on the quality of the answers in 2006 and the necessity for the different research projects. Some questions were left out because they were insignificant or unnecessary for future analyses. For one question, we changed the answering possibilities. However, we tried to stay as close as possible to the 2006 questionnaire and kept the same order, to be able to compare the answering results. Of the 89 questions, 72 questions were retained for the 2008 questionnaire. Secondly, we created some new questions, which were necessary for future analyses. These questions were added to the end of the questionnaire to keep the influence as low as possible. Again, these questionnaires contained a small card on which they could fill in their name, address, etc so that we could combine the surveys from both waves (see further). 3.2 Fieldwork: visiting schools In Belgium, the end of compulsory education is at the age of 18. So, the sixteen-year-olds which were surveyed two years ago are compelled to stay (part-time) in school. Therefore, it is reasonable to assume that surveying the respondents through schools will still give a 7

12 representative sample, except maybe for those that are older. The schools were contacted in the beginning of October 2007 to indicate that we would visit the schools again from the beginning of January onwards. A second letter was sent to the schools in the beginning of January Not all schools were immediately contacted in January, but the letters to the schools were sent in waves, in the same order as the schools were visited in The letter indicated that we would contact the school in the following week(s) and which classes were surveyed in In these following weeks, we called the schools several times until we could make an appointment to survey the same classes as in We asked the principal, secretary or the responsible teacher to look for the class that was surveyed in So, if we said them that we surveyed the students from economy in 2006, the principal knew that we had to survey economy-languages and economymathematics this year (see Figure 1). Self-evidently not all students were retained in this way: some changed classes or school, some were ill, etc. (see further). Each school was visited by one of the three researchers on the project or some well-trained job students, so that the students could ask every question they wanted to ask to the researchers and to keep it quiet. It turned out, however, that the students were much quieter than two years ago, partly because they were more interested in the subject. In the schools, students were surveyed class per class or in a large room, depending on the preference of the school and the number of the students that were surveyed. Table 3.2. Follow-up of different classes Name School General education 2 nd grade 3 rd grade - Economics Economics-languages Economics-mathematics 1-2 Technical education 2 nd grade 3 rd grade - Mechanische technieken Mechanische vormgevingstechnieken 1-2 Professional education 2 nd grade 3 rd grade - Basismechanica Werktuigmachines 1-2 Source: (in Dutch) 8

13 3.3 Fieldwork: mail surveys Because we were unable to survey every student in their school, we decided to send a mail survey to the respondents which were not in school at the time of the interview (Dillman, 2000:219). Therefore, we developed a shorter, but highly similar survey 2 that we sent to the respondents. We used the addresses from 2006 cards to send the respondents the survey. First, we sent the respondents a survey with the kind request to fill the survey in and return it in an envelope that could be sent back without stamps. We promised to do a raffle among the participants for movie tickets, CD vouchers and an Ipod. After two or three weeks, the respondents were called and asked whether they filled the questionnaire in and had returned it. Most of the time, they remembered to have received a large brown envelope immediately. If they said they threw it away, we asked whether we could send them a new survey to fill in, and send them a new survey. Unfortunately some of them answered that they filled the survey in, although we did not receive the survey. Finally, after two or three weeks we sent a new survey to all those of whom we did not yet receive a completed survey (both those that were called and those we could not reach by telephone). These surveys however were still anonymous: it was stated clearly to the respondents that we only needed the identification number to allow combining the 2006 and 2008 data, and that their names would not be used in the data analysis. By doing so, we oblige to Belgian legislation, stating that information on persons can only be used for the purposes that were stated at the moment of collecting the data. Several response-incentives were used to increase the response rate of the mailing. First, to increase the trust in the survey, the cover letter used the same lay-out and title as those of two years ago, as well as the introduction that stated that they participated in 2006 in the 10 th grade. Second, we stated that we regret that we did not find them back in their school and ask for their help to fill it in. Third, we stressed that the questionnaire is short, and will not take much of their time. To reduce the costs for the respondent, we also included a return envelope. Fourth, respondents that returned the questionnaire 2. In fact, we deleted some questions and grouped all school questions, so that respondents who did not go full-time to school anymore could easily skip these questions. 9

14 participated in a raffle in which they could win film tickets, CD-bons and an Ipod. Fifth, the questionnaire was printed in a nice booklet, which gave it a more professional outlook (Dillman, Smyth, & Christian, 2009). Using a mixed-mode design has some clear advantages and disadvantages (for a discussion see De Leeuw, 2005). The advantages are clear: we collect more data for the panel design and have less drop-out in our panel. However, there are also some disadvantages: mail surveys have a low response rate, there are mode-effects, and people are in a different environment filling out the questionnaires (for instance with respect to the school-questions). The response of this questionnaire might have been higher because the first contact was a face-to-face contact in the schools in 2006 (De Leeuw, 2005). The school setting also could have created the impression that participation in the survey was compulsory, although in various schools pupils declined to participate in the survey for various reasons. However, on the other hand those that are not in school anymore are more difficult to recruit: they might have moved, changed track, left school (e.g. a lower socio-economic status) etc. Unfortunately, the last option seems to be the case: the mail surveys suffer from a large non-response. The response rates of the mail surveys are quite low, especially in the Walloon region. The response rate in Flanders is 50 percent, in the Walloon region 33 percent. These response rates are acceptable, since the contacted people were those that we did not find back in school because they moved, left school, etc. This implies that the group who did receive a mail survey have different characteristics than the general population: because they dropped out of school, changed schools, moved, etc., we can also assume that they will be less easy to target in a survey of this kind. The difference between the two regions can be explained through two reasons: first, the French-speaking students were more skeptical to the survey and less will to provide their address already in It has to be remembered in this respect that for cultural reasons, respect for privacy is stronger in the French community than in the Dutch community of Belgium. Second, the fact that the research was conducted by a university in the Dutch community might have limited the willingness to participate among the French speaking sample of the survey. 10

15 3.4 Coding As well as in 2006, the data were coded by job students. Job students were provided with an excel file, codebook and mail or school surveys. In total 4,780 students were surveyed in their school (either for the first or second time) and 1,291 students were surveyed through mail (for the second time). Again, the quality of the coding was controlled by double codings and controls for outliers. Figure 3.1. Number of surveys in 2006 and Belgium Belgian Youth Survey 2006 School survey: 6,330 students Belgian Youth Survey 2008 School survey: 4,870 students Mail survey: 1,298 students 3.5 Weighing In total, 4,870 respondents were surveyed in school. These are thus both respondents that filled in the 2006 survey and those that did not (e.g. new respondents). These results indicate that there are slightly more surveys from Flemish respondents than those of the French-speaking region. This was already the case in 2006 as we selected less schools in the Walloon region. However, this deviation is by no means problematic as the weights are still between 0.93 and

16 Table 3.3. Weighing coefficients for school surveys (BYSS 2008) n in population % in population n in sample % in sample Weighing coefficients Boys in Flanders 31, , Girls in Flanders 31, , Boys in the 22, Walloon region Girls in the 24, , Walloon region Missing 24 Total 108,575 4,870 Source: (11/03/08) Data from Etnic Elèves de 6ième de l'enseignement secondaire ordinaire par forme, réseau et sexe en , received on 08/12/08 through . Similarly, the weighing coefficients for the mail surveys are also very close to 1: between 0.96 and Therefore, it might be argued that both the mail and school survey data are quite reliable. In total, 6,161 pupils were interviewed in Although this is a similar number than in 2006, this does not mean that we were able to re-interview everyone: the data are weighed on the returned mail surveys, not according to the sent ones.the constitution of the panel data will be analyzed in the following paragraphs. Table 3.4. Weighing coefficients for mail surveys (BYMS 2008) n in population % in population n in sample % in sample Weighing coefficients Boys in Flanders 31, Girls in Flanders 31, Boys in the Walloon region 22, Girls in the Walloon region 24, Missing 5 1 Total 108, Source: (11/03/08) Data from Etnic Elèves de 6ième de l'enseignement secondaire ordinaire par forme, réseau et sexe en , received on 08/12/08 through . 12

17 4 The Belgian Political Panel Survey ( ) Combining the data from the Belgian Youth Survey 2006 & 2008 gave us the BPPS Below we describe how the datasets were merged into the BPPS , and how representative the panel study is. 4.1 Linking the Belgian Youth Survey 2006 and the Belgian Youth Survey 2008 Finally, the datasets of the Belgian Youth Survey 2006 and the Belgian Youth Survey 2008 (school survey) were combined, based on the cards that the respondent filled in in 2006 and/or On these cards, the panel participants filled in their name, address, telephone etc. which allowed us to combine their data from 2006 and If we did not find the respondents back in school, this information was used to send the respondents a mail survey. Figure 4.1. Identification card. This leads to the following three possibilities as the SPSS-file demonstrates. First, the respondent (Sofie) participated in 2006 and 2008 in the school survey. She has a identification number in 2006 and 2008, which both refer to a survey. Second, the respondent (Laura) did participate in the school survey in 2006, but not in She was 13

18 sent a mail survey (if she filled in all necessary information in 2006) to keep her in the panel. The mail surveys were numbered beforehand, so that we could see to who we sent which survey. If the respondent sent the mail survey back, the number of the mail survey was added to the corresponding name. Laura could for instance be sent a mail survey with the number If we received number 4000, we could link this mail survey to her name. The third possibility is that the respondent (Nathan) participated in the 2008 survey, but not in This data can be used for separate analyses on the 2008-dataset. Figure 4.2. SPSS file with the names of the respondents Note that these names are fictitious. So after, the identification number 2006 of the respondents was coupled to the identification number 2008 (either from the mail or school survey), the datasets were merged. First, the 2006 data were added to the file with the information about the respondent. Second, the 2008 data (both school and mail survey) were merged. Third, the name and address of the respondents were deleted, to keep anonymity. Finally, the BPPSdataset contains the identification number from 2006 and 2008 (either mail or school), plus the data from 2006 and

19 Figure 4.3. SPSS file with the 2006 and 2008 data 4.2 Panel attrition Of the 6,330 students that were surveyed in 2008, 4,235 students could be linked based on the data they provided 2006 and This means that 67 percent of the respondents that were surveyed in 2006, also participated in the survey in Finally, we were unable to link 33 percent of the respondent that participated in the initial study. This was the case if the respondent (1) was not in school and did not sent the mail survey back; (2) did not provide at least his/her name in 2006 or 2008; or (3) erased his/her identification number on one of the surveys. Although we lost some respondents during these two years, we also gained 1,896 respondents. These new respondents are those that were in a class or school that was surveyed two years ago but did not participate in the survey at that time because they were sick/absent, in a different class, school, etc. This data is graphically presented in the following figure (e.g. Figure 2.5). These new respondents were still included in the data-set, as they allow us to construct a control group. This way we can ascertain whether the fact that the other respondents have had the experience of filling in the 2006 might have had an effect by itself (e.g., on salience of political issues, or discussion about politics in class). 15

20 Figure 4.4. Overview of the linked respondents and attrition. Belgium Belgian Youth Survey 2006 School survey: 6,330 students Belgian Youth Panel Survey Belgian Youth Survey 2008 School survey: 2,989 students Mail survey: 1,246 students Attrition: 2,095 students New students: 1,896 students Note that these numbers do not completely correspond to the number of surveys in This is because some respondents did not fill in their cards in 2006 or These respondents (3 percent) are not taken into account in this figure. An important issue in panel data is attrition. In this research, there are two different kinds of attrition: in the school survey and in the panel survey. First, the attrition in the school survey is caused by pupils that were not in class, classes that could not be interviewed, or students that changed school. Although this might seem at random, in fact it is not. People from lower educational tracks are more likely to abstain than people from general education (Debruyne, Lips, & Van Causenbroeck, 2006). Furthermore, it was more difficult to retain all classes from technical and professional education, because they have smaller classes, are more likely to have changed class and have traineeships. Finally, some of these students might already have left school. However, the weighing coefficients indicate that this kind of attrition is not worrisome, especially since we will always control for socio-economic status. Second, the attrition in the mail surveys is more problematic: the mail survey allows the respondent him/herself to decide whether or not he/she will participate in the survey, while the school surveys do not allow such a self-selection. Research has indicated that people with certain characteristics and interest in the survey topic are more likely to participate in a survey than others. Often, people with specific characteristics are less likely to survive, as people with lower levels of education for example (Cameron & Trivedi, 2005). Groves et al. (2004) find that people with particular interests (i.e. teachers, new parents) are more likely to participate in surveys about a topic 16

21 that might be more of interest to them (respectively education and children). So, mail respondents that are not interested in politics might be less likely to fill in the survey. One of the solutions for panel attrition is weighing. 4.3 Weighing Looking the Table 2.7, one should certainly mention that the weight factors are still quite acceptable, ranging between 0.9 and 1.2. Therefore, the sample contains slightly higher weights for the French sample, indicating that the French speaking respondents were slightly less willing to participate again in the survey. Table 4.1. Weighing coefficients for the panel data (BPPS ) n in population % in population n in sample % in sample Weighing coefficients Boys in Flanders 31, Girls in Flanders 31, Boys in the Walloon region 22, Girls in the Walloon region 24, Missing 49 1 Total 108, Source: (11/03/08) Data from Etnic Elèves de 6ième de l'enseignement secondaire ordinaire par forme, réseau et sexe en , received on 08/12/08 through Wrap up: the Belgian Political Panel Survey The Belgian Youth Survey 2006 (BYS 2006) was conducted in 2006 among 6,330 students in the 4 th year of higher education. In 2008, we tried to resurvey the same students again. In total, 4,870 students were surveyed in school and 1,298 students through mail. Both the school and mail survey will be referred to as the Belgian Youth Survey 2008 (BYS 2008). If one of both datasets is used, they will be referred to as the Belgian Youth School Survey 2008 (BYSS 2008) and the Belgian Youth Mail Survey 2008 (BYMS 2008). Finally, a third dataset is comprised of those that were surveyed twice: in 2006 in school and 2008 in school or through mail. This survey will be referred to as the Belgian Political Panel Survey (BPPS ), and contains data from 4,235 students. Furthermore, if we refer to Belgian Youth Survey Panel 2006 or 2008 (BYPS 2006/BYPS 2008), we refer to respectively the data from the panel 17

22 respondents in the first/second wave. In Table 2.8, the names, abbreviations and used data look as follows: Table 4.2. Names of the different datasets. Contains data from: 2006 school 2008 school 2008 mail Belgian Youth Survey 2006 BYS 2006 x Belgian Youth Survey 2008 BYS 2008 x x Youth School Survey 2008 BYSS 2008 x Belgian Youth Mail Survey 2008 BYMS 2008 x Belgian Political Panel Survey, BPPS (x) (x) (x) Belgian Youth Panel Survey 2006 BYPS 2006 (x) Belgian Youth Panel Survey 2008 BYPS 2008 (x) (x) Number of cases 6,330 4,870 1,298 Number of cases for panel 4,235 2,989 1,246 An x indicates that this data is completely used in this dataset, (x) indicates that a part of this data is used in the dataset, e.g. only the panel respondents. 18

23 5. The Belgian Youth Survey 2011 The third wave of the Belgian Political Panel Survey took place between February and July The decision was taken to only contact those respondents who had participated both in 2006 and 2008 (n=4,235). This was done for a number of reasons. In the first place, the three-wave panel sample is the most relevant to analyze: three waves allow to model change (for instance using latent growth curve models). Moreover, sending out mail surveys to respondents of the panel (and thus not outside the panel) reduced the survey costs significantly. Contacting those who participated solely in the 2006 wave would be extremely difficult: we were already unable to contact them in 2008 through school and mail, so it would be even more difficult doing so in The money saved from decreasing the sample size was utilized to increase the response rate by engaging in multiple mixed-mode contacts with the respondents. Aiming at a higher level of response is generally seen as a good way to prevent nonresponse errors (Hansen & Hurwitz, 2004; Ryu, Couper & Marans, 2006; Daly, Jones, Gereau, Levy, 2011) and panel attrition (Callegaro & DiSogra, 2008; Castiglioni, Pforr &Krieger, 2008) Questionnaire Contrary to the surveys held in 2006 and 2008, respondents could not be questioned at school in the third mailing, because they were all pursuing further education or were already at work. As the 2011 wave of the BPPS survey was done by mail only, it was important to keep the number of questions in the questionnaire as limited as possible and thus the survey as short as possible. Elaborate questionnaires can have a detrimental effect on the response rate (Dillman, Smyth & Christian, 2009). We have chosen to keep most of the questions identical to the 2006 and 2008 questionnaire to allow for an over-time comparison of respondents answers and to investigate evolutions. Of the 56 questions in the 2011 questionnaire, 25 were asked in an identical way in 2006 and Further, 5 questions had been asked in 2006 and 2008, but in 2011 small changes were made to the questions (e.g. extra items or changes in question due to changes in time). Three questions were only asked in the 2008 questionnaire and one question was taken from the 2006 questionnaire. In total, the 2011 questionnaire contained 22 new questions. Six of these new 19

24 questions concerned the working/educational status of the respondent and they replaced some of the school-related questions in 2006 and For the design of the questionnaire and the cover letter, we followed the recommendations formulated in the Total Design Method by Dillman (1976, 2000, 2009). The cover letter contained a personal salutation of the respondent, a short explanation of the content and aims of the study and the importance of participating in it. At the bottom of the letter, a handwritten signature of the research coordinator was placed. The letters for the French-speaking respondents were professionally translated from the letter in Dutch. Both letters were identical with the exception that the letter for the French-speaking respondents contained two handwritten signatures: one of the Flemish research coordinator and one of a researcher of a French-speaking university. This was done to prevent a larger fall-out of French-speaking respondents, because it is possible that French-speaking respondents are less willing to cooperate in research done by a Flemish university only. Furthermore, the respondents were also referred to a free telephone number in the letter. They were kindly invited to telephone this number in case they had any remarks or questions about the questionnaire or the study in general Fieldwork As already mentioned, because the respondents could no longer be contacted at schools, the third wave was done by mail. Despite the advantage of its low cost, mail surveys are also known for having low response rates compared to face-to-face survey modes (Dillman, Phelps, Tortora, Swift, Kohrell, Berck & Messer, 2009). It is nevertheless crucial for the quality of the data collected that the response rate is as high as possible, especially since we are working on the third wave of a panel study. Therefore in every step of the survey process, a number of measures were taken to boost the response rate to the questionnaire. Figure 5.1. represents the different steps taken in the survey process. These steps will be elaborately discussed in the following paragraphs. 20

25 Figure 5.1.: Survey structure First Survey First contact Second Survey Second Contact Final Survey The survey was held among those respondents who had participated in both the 2006 and 2008 wave of the BPPS-survey (n= 4235). We used the addresses from the 2006 and 2008 identification cards. When information of respondents postal address was missing (house number, postal code) this information was completed if possible. For missing house numbers or incomplete street names this was done by means of a general search on telephone numbers (also given on the identification cards) in the online 1207/1307-phone number-database 1. Further, if no valid phone number was available, incomplete or fault street names were completed or corrected through a Google Maps search. In a first step of the survey process, we have sent out a mail questionnaire to all the respondents. To keep the mailing procedure manageable, about 500 surveys were sent out each week alternating Dutch and French surveys. Each respondent had a unique key number, that was stamped on the back of the questionnaire. By means of those key numbers, we could ascertain which respondents had completed the questionnaire and which had not yet done so. A return envelope was sent to the respondents together with the questionnaire. Sending a pre-paid return envelope to the respondents is also considered a good way of increasing response rates. Finally, 1 This online database consists of public information on people s telephone numbers and addresses. Furthermore, people who do not want their information to be displayed in this information base, can choose to keep this information hidden from others. 21

26 we told the respondents that they could win an ipad if they participated in the survey. Thirty-one percent of the respondents filled in the questionnaire after this first step (see Table 1). As a second step, after three weeks, we tried to contact by telephone those respondents who had not returned their survey yet. Most of the calling was done by job students, who received a thorough training and a sheet with instructions for the telephone contact. Telephone numbers came from the 2006/2008 identification cards or were retrieved via the 1207/1307-phone number-service. Respondents were called and asked if they had received the questionnaire. If respondents had not received the survey, a correct address was asked. If respondents did receive the survey, they were asked to fill in the questionnaire and to return it. Further, respondents who said they had thrown away the survey, were asked if we could send them a new questionnaire. Respondents who explicitly refused to participate to the BPPS-survey, were marked in the data file and no further attempts were made to contact them (according to Belgian privacy laws). Furthermore, respondents who could not be contacted by phone were sent an on-line message in which we asked for a valid mail address. In a first attempt these messages were sent by regular e- mail ( addresses were asked for on the 2006 and 2008 identification cards). If the addresses were not/no longer valid, we looked for a Facebook account of the respondents and sent them a private Facebook message with the same content as the regular s. As mentioned earlier, 30.8% of the respondents had already answered to the first send-out of the questionnaire. We were able to contact 47.8% of the respondents by telephone (Table 5.1.) of which 2.5% refused to participate further. Fourteen percent of the respondents was contacted by mail, internet and Facebook. Respondents who were contacted by mail and did not responded within a week time, were contacted by Facebook. Almost five percent of the respondents could not be reached through mail, phone or internet ( and/or Facebook). A very small percentage of the respondents were staying abroad during the time the survey was held: we opted to send them an online survey by mail and/or Facebook. The proceedings of this first contact can be found in Table

27 Table 5.1. First contact with respondents N % Cumulative n Cumulative % Response by post before first contact Respondents contacted by Telephone Mail Mail and Facebook Facebook Online Refusals Respondents who could not be contacted Total As a third step, another three weeks after this attempt to contact respondents who had not filled in the questionnaire yet, a second mail survey was sent to all respondents who at that time had not returned the survey (for some respondents this was already the third questionnaire they received, if at the time of the phone contact, they agreed to receive a second questionnaire). This second mail survey was accompanied by a different cover letter. The new cover letter once more contained a personal salutation and handwritten signatures at the bottom, but the content was slightly different. In the new cover letter respondents were reminded of the importance of participating in the research and were asked to fill out the questionnaire as soon as possible (Dillman, 2008). Almost fifty-two percent returned the mail survey at this point in time (Table 5.2.), which means that 45.7% of the respondents received a second survey after taking account of refusals. Table 5.2. Second Survey N % Cumulative n Cumulative % Response by post before first contact Response by post after first contact Response online after first contact Refusals after first contact

28 Respondents who received a second survey by post Total As a fourth step, we tried to contact the respondents again in person. After another three weeks waiting, a new attempt was done to contact the respondents who had not yet returned the questionnaire. At this stage, the response rate to the survey was 56.4%. The response rate increased with 4.6% after sending out the second survey, making this a less efficacious method to increase the response rate compared to the previous step (contacting respondents by telephone or internet). Taking account of the 2.5% of refusals after the first contact, 31.2% of the respondents was contacted for the second time by telephone or online. When respondents were contacted by phone, we verified whether the 2006/2008 address was still valid. Respondents who could not be contacted by phone were immediately sent a personalized link to the on-line questionnaire through an or Facebook message. Online surveys were sent out to 13.6% of the respondents, of which 10.8% answered online. An additional 2% of the respondents refused to participate in the survey after this second telephone contact. Table 5.3. Second contact with respondents N % Cumulative n Cumulative % Response by post before first contact Response by post after first contact Response online after first contact Response to second survey by post Refusals after first contact Respondents contacted by Telephone Online Refusals after second contact Respondents who could not be contacted Total

29 Two weeks after this attempt of personal contact a final effort was done to reach the respondents. The response rate before this last contact was 66.8% (see Table 5.4.). All respondents who had not answered the questionnaire yet (and did not refuse to participate) and of whom we had a valid -address got their personalized link to the on-line questionnaire. Those respondents who had already received the link to the on-line questionnaire were sent a reminder. We sent another and final survey to the remaining respondents, this time with written addresses and real post stamps. An additional 38 respondents filled in the online questionnaire in this step. Further, 158 filled-in questionnaires were sent by mail after this final step in the survey process. This brings the final response rate to 71.3%. Table 5.4. Third online and mail survey sent out to respondents N % Cumulative n Cumulative % Response by post before first contact Response by post after first contact Response online after first contact Response to second survey by post Response by mail after second contact Response online after second contact Refusals after first contact Refusals after second contact Respondents who received third survey by Mail Online Refusals after third contact Total

30 During the second and third attempt to contact respondents personally, respondents were given the opportunity to fill in the questionnaire online. Offering a different survey mode to respondents has already proven to be a good way to increase the response rates to surveys (Deutskens, De Ruyter & Wetzels, 2006). The online survey was made as identical as possible to the paper version in order to avoid response effects of using two different survey modes. The sequence and wording of the questions and the answer categories were identical to the mail survey. Further, respondents were able to change their answers to previous questions and to leave questions unanswered. This was all done to increase measurement equivalence of the two survey modes (Dillman & Christian, 2003). Mail and online surveys are two survey modes that can well be combined in a mixed-mode survey design, since the measurement equivalence for these two modes is very high (Dillman, Phelps, Tortora, Swift, Kohrell, Berck & Messer, 2009; Atkeson & Tofaya, 2008; De Beuckelaer & Lievens, 2009). In total, 2.4% (n= 102) of the respondents answered to the online version of the questionnaire. This means that 3.3% of the respondents who answered (n= 3021) did so by means of online survey. Since the online questionnaire was only offered to the respondents who had not yet responded after two trials to contact them, this can be considered as a good mean to boost the response rate in the final stages of the data gathering. The total response rate of the survey is 71.3%. A summary of the number of responses and refusals at every step in the survey process can be found in Table 5.5. It can be seen from Table 5.5 that the highest increases in response rate took place after sending out the first survey and after contacting the respondents for the first and second time. Table 5.5. Response rate per stage in survey process Survey Stage N % Cumulative percentage Cumulative n First Survey First Contact Second Survey Second Contact Final Survey Nonresponse Total

31 5.3. Coding The coding of the surveys was done by a small group of well-trained job students. After a thorough explanation about the questions and the coding, the job students were handed 20 surveys. These surveys had to be coded in an Excel-file that was secured in such a way that no other codes than those in the codebook could be typed into the file. The first 20 surveys were controlled extensively by one of the researchers on the project. If there were too many mistakes, the student was no longer allowed to code surveys. Thereafter, the job students could take more surveys with them and they had to return them at regular times. Each time a job student brought back surveys, one of the researchers on the project controlled a small number of the coded surveys. In case a mistake was found, all the coded surveys of the job student were controlled for mistakes Incentives As mentioned earlier, two ipads (64GB) were distributed among the respondents of the survey as an incentive to participate in the study (e.g. Berk, Mathiowetz, Ward & White, 1987; Dillman, Smyth & Christian, 2009). The ipads were chosen as incentives, because they are very popular among youngsters in Two respondents were randomly chosen of the panel and they received the ipads after the third wave was finished Weighing of the data Panel attrition can be the cause of non-representativeness of a sample. We will control for panel attrition by comparing the distribution of characteristics, such as gender, region 2 and track followed in secondary school, in the 2006 and 2011 sample. Since information on the population of 2011 is no longer available, weighing coefficients will be calculated based on population information in 2006 and Table 5.6. presents the weighing coefficients for the 2011 sample, 2 Considering the region of the respondents, we will differentiate between Flanders (Dutch-speaking respondents) and the Walloon region (French-speaking respondents). 27