The Effect of Game-Assisted Mathematics Education on Academic Achievement in Turkey : A Meta-Analysis Study

In this research, the effects of using game in mathematics teaching process on academic achievement in Turkey were examined by metaanalysis method. For this purpose, the average effect size value and the average effect size values of the moderator variables (education level, the field of education, game type, implementation period and sample size) were calculated. MetaWin and Comprehensive MetaAnalysis (CMA) statistical programs were used for the analysis. Based on the inclusion criteria, 31 effect size values for 26 studies were calculated. Hedges's g coefficient was used when the effect sizes were calculated and the confidence level was accepted as 95%. The average effect size value was 0.792 with 0.077 standard error which was calculated by random-effects model. As a result, the effects of using game on academic achievement is medium and positive in mathematics teaching process.


Introduction
It is the game itself which facilitates the child's interaction with the environment, completely opens the channels related to the communication.The teachers' use of games affects the perspectives of the children towards school and mathematics.The game has an important place in children's thinking on the numbers and initiating and maintaining mathematical communication (Trawick-Smith, Swaminathan, & Liu, 2016).The game improves oral communication, top-level social interaction skills, creative thinking skills, imaginary and divergent thinking skills and problem solving skills of the children (Wood & Attfield, 2005).It can be said that the game presents an environment to the children in which the communication process is practiced densely.From the social point of view, children's speaking, getting feedback to their questions in this process, communicating with their friends and teachers have importance in mathematics learning.The children comprehend the mathematical concepts before they use because mathematical thinking develops before language.Therefore, the proper use of mathematical words can help children to acquire the mathematical concepts.Using a clear and explanatory language during acquisition of mathematical knowledge and skills process of children is crucial (Presser, Clements, Ginsburg, & Ertle, 2015).The game can increase the effectiveness of teaching by generating a collaborative learning environment and creating discussion platforms.It also helps the students having less knowledge to improve their understanding (Ke, 2008).
In game process, the communication set by the children might increase their consciousness about mathematics.The more the variety of the scenarios and situations in educational environments are extended, the more the children gain consciousness about not only about their own but also the other children's mathematics.As long as the game based approaches are used in classrooms, the mathematical consciousness of children is expected to increase (Marcus, Perry, Dockett, & MacDonald, 2016).The students might not be relaxed if they perceive mathematics course as difficult.The game can change the students' perceptions that the mathematics is difficult and contribute them to feel relaxed in the course.The children can improve the informal mathematical knowledge they have acquired in game activities if they attend problem solving process (Brandth, 2013).It can be said that there is a relationship between the children's creating new structures with various materials during the game and their cognitive development.For instance, Wolfgang, Stannard, and Jones (2003) have stated that playing legos and making constructions with them improve cognitive development of children, moreover contribute learning the subjects requiring abstract thinking such as geometry, arithmetic, trigonometry in mathematics learning process.
With the development of the technology, the computer games have been involved in children's game world and the educators started using technology and technologyassisted games in learning process.Real life situations and experiences are learned in the best way in technology education.The students are able to both learn and maintain their learnings by practicing; the teachers are able to teach complex concepts more easily in technological setting (Bellamy & Mativo, 2010).Mind games are also used in learning-instruction process like computer games.Kazemi, Yektayar, and Abad (2012) stated that teaching chess improves the mathematical problem solving abilities of students at different educational level considerably and contribute increasing the students' meta-cognitive abilities.The problem solving ability is a complex interaction between cognition and meta-cognition.The main source of troubles about problem solving might be that the students cannot follow their cognitive activities actively, cannot control them or do not have opportunity to supervise (Artzt & Armour-Thomas, 1992).The children learn new concepts, corroborate them by practicing, strengthen their mathematical skills and develop their problem solving strategies when they play games during the process of learning mathematics (Ernest, 1986).The teachers' use of games and creative pedagogical applications might be helpful for enhancing students' attitudes towards mathematics (Afari, Aldridge, Fraser, & Khine, 2013).
Reviewing the literature, there has not been found any researches examining the effect of using games in mathematics education by meta-analysis method.Therefore, it is thought that this research will provide a holistic perspective towards how teaching mathematics with games affect academic achievement.Thus, the current situation can be interpreted and some suggestions can be presented to future researches.It is aimed to statistically reveal the effect of using games in the process of teaching mathematics on academic achievement of the students.Additionally, it is tried to be determined whether the academic achievement gained as a consequence of using games in mathematics teaching process differs in terms of educational level, learning domain, type of the game, implementation period and sample size.

Research Model
The effect of using games in the process of teaching mathematics on academic achievement has been examined through combining the findings obtained from primary studies.Meta-analysis technique has been used for this purpose.Meta-analysis is the statistical analysis of the quantitative data obtained from a number of independent studies on a specific subject and a method of overall evaluation about these studies (Glass, 1976;Lipsey & Wilson, 2001).The average effect sizes related to primary studies are identified, the relationship among these effect sizes and the relationship among study characteristics are evaluated with meta-analysis (Card, 2012).The effect size refers the degree of the relationship between two related variables, the size of the score emerging or to be found between the groups in an experimental implementation (Ellis, 2010).Meta-analysis includes standardization of various effect size statistics used in order to code different types of quantitative studies (Lipsey & Wilson, 2001).Thus, the numerical findings obtained from primary studies can be interpreted in a statistically coherent way.Moreover, various statistical errors of primary studies can be demonstrated.It can be commented that meta-analysis is a secondary analysis format.
As the findings about a certain research subject are interpreted by being combined in meta-analysis, it can be stated that meta-analysis also provides a basis to theory development (Hunter & Schmidt, 2004).Therefore, it can be determined what kind of researches are needed by making overall evaluations about the existing studies associated with the subject as well.
A series of phases that should be followed exists in metaanalysis studies.First of all, the problem is determined.Then the literature is scanned in accordance with the problem.As a consequence of this, the attained studies are coded in terms of the determined criteria.The statistical analysis of the data is done after this phase.Finally, the findings obtained as a result of the analysis are interpreted (Pigott, 2012;Sánchez-Meca & Marín-Martínez, 2010).The effect of using games in teaching mathematics on academic achievement has been examined within the frame of mentioned phases in this research.

Data Collection
The research data were collected in April 2017.The studies examining the effect of using games in teaching mathematics on academic achievement in Turkey established the data resources.YOK (Higher Education Council), ULAKBIM (National Academic Network and Information Center), Google Scholar databases were used so as to access the studies.For this purpose, the mentioned databases were scanned with the keywords "game and mathematics, mathematics and game, teaching mathematics with game".As a result, 60 works were reached.The studies to be included in the metaanalysis were determined according to the following criteria: 1. Studies should be prepared between the years of 2000-2017.
2. Studies should be written either in Turkish or in English.

3.
Studies should be open to access in YOK, ULAKBIM and Google Academic databases.
4. Studies should be related to preschool, primary school, middle school, high school and university students who are studying in Turkey.
5. The studies should be experimental and the pretest-posttest control group model should be used in the studies.
6.In the studies, the experimental group should be taught with game(s) and the control group should be taught based on the traditional methods.
7. In the studies, statistical values such as sample sizes, arithmetic means, standard deviation etc. of both the experimental and the control groups should be given to calculate effect size.30 studies were selected in accordance with these criteria.It has been noticed that 4 of the studies were the articles generated from dissertations or thesis and the articles were included in the study while 4 dissertations or thesis owning an article were ignored.Consequently, totally 26 studies, 4 articles and 22 dissertations, were included in the meta-analysis.As two different achievement tests were implemented to the same group in a master's thesis from these studies, 2 effect size values of this study were calculated.As three different control groups were found in one of the master's thesis, 3 effect size values of this study were calculated.As there were also 2 different experimental groups in a doctoral dissertation, 2 effect size values were calculated for this study.In order that there is not a confusion, the letters a, b, c were added next to the study year of this type of studies when the analysis results were stated.Eventually, 31 effect size values were calculated related to 26 studies included in the meta-analysis.

Coding the Data
A coding form was constituted in accordance with the inclusion criteria to use at the phases of meta-analysis study.This form includes information about study number, study name, author name, study type, study year, sample size, educational level of sample, learning domain, implementation period, and sample size, arithmetic mean, standard deviation of the experimental and control groups, and validity and reliability information of the utilized assessment instruments.The information about the studies, to be included in the analysis, were coded by the two researchers in order to determine correctly and transfer the data without any error.After the coding, the two researchers compared the forms.As a result of the comparison, there was not found any difference between two researchers' coding.Consequently, the analysis phases were started.

Data Analysis
There are two main statistical approaches, fixed effect model and random effect model, in meta-analysis process (Borenstein, Hedges, Higgins, & Rothstein, 2009;Hunter & Schmidt, 2004).The distribution of the effect sizes of the studies included in meta-analysis is considered when the model to be used in analysis process is determined.In order to determine whether the distributions are homogenous, Q statistics can be used.Zero hypothesis indicating that all the studies included in meta-analysis share a mutual effect size with Q statistics is tested using chi-square distribution (Borenstein, Hedges, Higgins, & Rothstein, 2009).If the value obtained as a result of Q statistics is lower than p significance value and the value of degree of freedom (df) corresponding in chi-square (X 2 ) table, homogenous distribution is provided (Borenstein et. al, 2009).Otherwise (if it is higher than the value corresponding in chi-square table), it can be said that the distribution is heterogonous.If the effect sizes of the studies included in meta-analysis show homogenous distribution according to Q statistics, fixed effect model and if they show heterogonous distribution random effect model is preferred (Ellis, 2010).Another statistics that can be used for determining the distribution is I 2 .I 2 statistics provides a ratio independent from size effect scale and its interpretation is heuristic (Higgins, Thompson, Deeks, & Altman, 2003).I 2 explains the heterogeneity effect in the analysis (Petticrew & Roberts, 2006).If the value calculated as a result of I 2 statistics is higher than 25%, it indicates heterogeneity; if it is 50%, it indicates medium heterogeneity and if it is 75%, it indicates high-level heterogeneity (Cooper, Hedges, & Valentine, 2009;Higgins, Thompson, Deeks, & Altman, 2003).Random effect model was used depending on the results of Q statistics and I 2 statistics (see Table 3) in this study.
Comprehensive Meta-Analysis (CMA) statistical programs were used for MetaWin, forest plot, funnel plot, publication bias, effect sizes and sub-group analysis for determining whether the effect sizes of the studies included in the study show normal distribution.
Funnel plot and Rosenthal's fail-safe N statistics were examined in finding publication bias associated with the studies.The fact that the effect sizes of the studies distribute around overall effect size symmetrically in funnel plot indicates lack of publication bias (Borenstein et. al., 2009).But an asymmetrical distribution might also remark a real heterogeneity (Tang & Liu, 2000).Therefore, it might not mean an absolute publication bias if the distribution is not completely symmetrical.Rosenthal's fail-safe N (FSN) statistics refers the number of new studies that should be added to the analysis in order to zero the effect size reached as a result of meta-analysis (Borenstein et. al., 2009).If FSN value (N) in the analysis is relatively higher than the observed studies, the results are enough resistant to publication bias (Rosenthal, 1991).In addition this, Mullen, Muellerleile, and Bryant (2001) produced N/(5k+10) (k is the number of included studies in meta-analysis) considering Rosenthal's fail-safe N statistics.In case that the value to be reached is higher than 1, they stated that the results are enough resistant to publication bias.
Hedge's g coefficient was used to calculate the effect sizes of the studies.Reliability degree was accepted as 95% in the calculations related to effect sizes.The criteria was taken into consideration that if it is between 0-0.20, it is weak; between 0.21-0.50, it is small; between 0.51-1.00, it is medium; and if it is higher than 1, it is large effect (Cohen, Manion, & Morrison, 2007, p. 521) while the effect sizes were interpreted.Educational level, learning domain, type of game, implementation period and sample size were identified as moderators in the study.Analog ANOVA test was utilized in the analysis of moderators.
The descriptive statistics of the studies examining the effect of game in the process of mathematics teaching on academic achievement in Turkey is presented in Table 1. ), no study was done in the range of 1-20 persons.Among the identified studies, it is seen that most of the studies were implemented for 6-10 hours period (5 studies, 19.23%), there are nine studies (34.61%) in which the implementation period is not mentioned as hours.

Findings
Findings about the Effect of Game on Academic Achievement in Mathematics Normal distribution plot has been observed in order to determine the convenience of combining effect sizes of 26 studies by meta-analysis.Normal distribution of the effect sizes of the studies is presented in Figure 1.Looking at Figure 1, it is seen that the effect sizes of the studies distribute on the right and left sides of the normal distribution line and within the borders of confidence interval showed by dotted lines.Accordingly, it can be said that the effect sizes show normal distribution and they can be combined statistically by meta-analysis.
Before the effect sizes were calculated for the purpose of determining the effect of teaching by using games on academic achievement in mathematics, funnel plot of publication bias probability related to the studies included in meta-analysis was created and it is given in Figure 2. In case of publication bias, it would be asymmetrical.In addition to this, the fact that the distribution is not completely symmetrical indicates publication bias.Therefore, Rosenthal's fail-safe N (FSN) was examined in addition to funnel plot.The related statistical information is given in Table 2. Calculating the effect sizes of the studies to be included in meta-analysis; homogeneity value, average effect sizes and confidence intervals according to fixed effect and random effect models are presented in Table 3 in order to determine the model to be selected.According to this result, it is seen that Q value (64.579) is higher than the critical value corresponding 30 degree of freedom in chi-square table (x 2 = 43.773for df= 30).
Depending on these findings, it can be stated that the studies analysed through meta-analysis show heterogeneous distribution.Additionally, I 2 value with 53.545% indicates heterogeneity over medium.Therefore, random effect model was chosen when the average effect sizes of the studies analysed through meta-analysis were calculated.The average effect size value was calculated as 0.792 with 0.077 standard error according to random effect model.The fact that the calculated effect size is positive indicates that treatment effect is on behalf of experimental groups.0.792 effect size value reflects medium effect according to Cohen et. al. (2007).Depending on this reference, it can be inferred that using games in mathematics teaching affects academic achievement positively and this effect is at medium level.
Forest plot demonstrating the distribution of effect size values of primary studies examined through meta-analysis according to random effect model is presented in Figure 3.The squares seen in Figure 3 reflect the effect sizes of primary studies, the areas of the squares reflect the weight of the effect size of the study it belongs within the overall effect size.The numerical values of these weights are demonstrated at the righmost part of the figure.The lines appearing at two sides of the squares represent upper and lower bounds of these effect sizes within 95% confidence interval.The equilateral quadrangle found at the lowest part of the squares indicates overall effect size.Investigating the effect sizes calculated, it is observed that the lowest effect value is -0.059 and the highest effect value is 1.815.Only 1 effect size value is negative among 31 effect sizes.Consequently, using games in the process of mathematics teaching affected on behalf of experimental groups in 30 studies.

Findings about Effect Sizes in Terms of Educational Level
Calculated effect sizes of the effect of using game in teaching mathematics on academic achievement in terms of educational level are presented in Table 4. Homogeneity value between groups in terms of educational level was calculated as (QB) 7.830.Critical value of 3 degree of freedom is 7.815 at 95% significance level in chi-square.It is seen that Q value is higher than the critical value corresponding to 3 degree of freedom in chi-square table (QB= 7.830, p= 0.050).Accordingly, a statistically significant difference between groups in terms of educational level is found.Regarding findings, using games in mathematics teaching has maximum effect on higher school and minimum effect on middle school in terms of educational level.According to Cohen et. al. (2007), the effect sizes calculated for higher school and preschool are large, the effect sizes calculated for primary school and middle school are medium-sized.

Findings about Effect Sizes in Terms of Learning Domain
Calculated effect sizes of the effect of using game in teaching mathematics on academic achievement in terms of learning domains are presented in Table 5. statistically significant difference between groups in terms of learrning domain is not found.Therefore, the academic achievement attained by using games does not show a statistically significant differentiation in terms of different learning domains.According to Cohen et. al. (2007), the effect sizes calculated for learning domains have medium effect.
Findings about the Effect Sizes in Terms of Game Types Calculated effect sizes of the effect of using game in teaching mathematics on academic achievement in terms of game types are presented in Table 6.
Homogeneity value between groups in terms of game types was calculated as (QB) 6.667.Critical value of 2 degree of freedom is 5.991 at 95% significance level in chisquare.It is seen that Q value is higher than the critical value corresponding to 2 degree of freedom in chi-square table (QB= 6.667, p= 0.036).Accordingly, a statistically significant difference between groups in terms of game types is found.Regarding the findings, musical games have maximum effect on academic achievement in mathematics teaching.According to Cohen et. al. (2007), the effect sizes calculated for computer-assisted games are low, the effect sizes calculated for pedagogical games are medium, and the effect sizes calculated for musical games are large.

Findings about the Effect Sizes in Terms of Implementation Period
Calculated effect sizes of the effect of using game in teaching mathematics on academic achievement in terms of implementation period are presented in Table 7. degree of freedom in chi-square table (QB= 8.313,p= 0.216).Accordingly, a statistically significant difference between groups in terms of implementation period is not found.Therefore, the academic achievement attained by using games in different implementation periods does not show a statistically significant differentiation.According to Cohen et. al. (2007), the effect sizes calculated for implementation periods of 1-5 hours, 26-30 hours and 31 or more hours are low, implementation periods of 11-15 hours, 16-20 hours and unidentified ones are medium, and the effect sizes calculated for implementation periods of 6-10 hours are large.
Findings about the Effect Sizes in Terms of Sample Size Calculated effect sizes of the effect of using game in teaching mathematics on academic achievement in terms of sample size are presented in Table 8. (QB= 1.016, p= 0.602).Accordingly, a statistically significant difference between groups in terms of implementation period is not found.Therefore, the academic achievement attained by using games with different sample sizes does not show a statistically significant differentiation.According to Cohen et al. (2007), the effect sizes calculated for sample size are low.
Accordingly, it can be inferred that using game in mathematics teaching process generally effects academic achievement positively.This result shows similarity with the literature.Ku, Chen, Wu, Lao, & Chan (2014) stated that game based learning is more effective than pencilpaper based (traditional) learning processes and the students feel more comfortable and their performances enhance in mathematics courses assisted by game based learnings.Uğurel (2003) remarked that mathematics teaching by using games and activities increases interest, provides motivation, offers the opportunity of active participation and permanent learning by providing use of all sense organs in learning process.Yılmaz (2014) expressed that teaching through games positively effects the students' academic achievement and their attitudes towards mathematics course.Özgenç (2010) stated that game based mathematics activities affect the interest of the students in the course and their participation positively, enhance their interaction with both their teachers and friends.Fırat (2011) reported that mathematics teaching maintained by computer-assisted educational games has a positive effect on students' learning the concept of probability.
Educational level, learning domain, game type, sample size and implementation period related to the primary studies were determined as moderators and the effect sizes in terms of these values were calculated.Thus, it was examined whether the academic achievement attained by using game in the process of teaching mathematics differs statistically according to moderators.
The effect sizes of four different groups including preschool, primary school, middle school, higher school were calculated in terms of educational level in the analysis.Consequently, it was found out that maximum effect of using game in mathematics teaching process on academic achievement is at higher school while minimum effect is at middle school level.The fact that there is only one study at higher school level might be effective in existence of a result like this.
The effect sizes of three different groups including mathematics, geometry and mathematics and geometry were calculated in terms of learning domains.Consequently, it was found out that the effect of using game in mathematics teaching does not differentiate according to learning domains, has similar effects in all three groups.
The effect sizes of three different groups including computer-assisted, pedagogical and musical games were calculated in terms of game type.Consequently, it was found out that using musical games has the maximum effect on achievement.The fact that there is only one musical game among the studies done by meta-analysis might have effected this result.However, some studies supporting this finding are encountered in the literature.For instance, Yılmaz (2006) stated that musical game activities are effective on 5-6 aged students' acquisition of number and operation concepts in his study.
The effect sizes of seven groups including 1-5 hours, 6-10 hours, 11-15 hours, 16-20 hours, 26-30 hours, 31 or more hours and unidentified were calculated in terms of implication period.Consequently, it was observed that the effect of using game in mathematics teaching does not differentiate according to implementation period.
The effect sizes of three groups including 21-40 persons, 41-60 persons, 61 or more persons were calculated in terms of sample size.Consequently, it was observed that the effect of using game in mathematics teaching does not differentiate according to sample size.
The effect of using game in the process of mathematics education on only academic achievement was examined in terms of different variables in this study.The effect of using game on motivation, attitude, achievement motive, achievement permanency and acquisition of concepts can be investigated in further researches.
The effect size of only one study at higher school level was calculated in terms of educational level in the analysis.No effect size at high school or faculty level was calculated.The effect size of only one study including musical game was calculated in terms of game type, as well.Accordingly, it can be argued that mentioned types of studies are needed to entirely determine the effect of using game in mathematics teaching.Additionally, it can be pointed out that the statistical values needed for metaanalysis should be given completely both in existing and future studies.

Figure 1 .
Figure 1.Normal distribution plot of the effect sizes of the studies included in meta-analysis

Figure 2 .
Figure 2. Funnel plot related to the effect sizes of the studies included in meta-analysis Looking at Figure 2, it is seen that the effect sizes generally show an almost symmetrical distribution at the middle part of the funnel plot and at the right and left sizes of the line indicating combined effect size.In case of publication bias, it would be asymmetrical.In addition to

Table 1 .
The Descriptive Statistics of the Studies Examining the Effects of Game on Academic Achievement in Mathematics

Table 2 .
Calculated Rosenthal FSN Statistics Results Related to Meta-analysis Examining the Effect of Using Game on Academic Achievement in Mathematics

Table 3 .
Average Effect Sizes and Lower and Upper Values of Confidence Interval According to Effect Model

Table 4 .
Effect Size Differences in Terms of Educational Level Figure 3. Forest plot of studies' effect sizes according to random effects model

Table 5 .
Effect Size Differences in Terms of Learning Domain

Table 6 .
Effect Size Differences in Terms of Game Types

Table 7 .
Effect Size Differences in Terms of Implementation Period

Table 8 .
Effect Size Differences in Terms of Sample Size