Development of an Academic Achievement Risk Assessment Scale for Undergraduates: Low, Medium and High Achievers

This research has developed The Academic Achievement Risk Assessment Scale [AARS], for identification of the factors which influence performance of undergraduate (448 students); studying at three universities of Lahore, Pakistan. An 18-item scale, with five distinct factors was developed which included lack of motivation, dysfunctional parental practices, parental involvement in drug abuse or antisocial activities, difficulty with peers, and language barrier. The results revealed differences among low, medium and high academic CGPA groups as all five risk factors were significantly related to the low achieving group. The study has implications for teachers, counselors, and policy makers in the field of learning.

tudents' academic performance plays a vital role in producing the best quality graduates who are responsible for a country's economic and social development. The performance of students in universities is a concern not only to the administrators and educators, but also to corporations in the labour market (Ali, Jusoff, Ali, Mokhtar, & Salamat, 2009). The employers pay great attention to academic achievement level of workers and recent graduates while recruiting. It is important to note that the problem of low academic achievement is one of the great crises of the educational system in third world countries. The problem of low academic achievement has been identified several times as problematic in terms of social and economic waste (Peelo & Wareham, 2002).
Previous statistics indicated that 40% college students leave higher education without getting a degree and 75 % of students leave within their first two years of college (Deberard, Spielmans, & Julka, 2004). Education for All (EFA) Global Monitoring Report (UNESCO, 2005) suggested that only 41.5 % of people older than 15 years of age are literate in Pakistan having the highest dropout rates in South Asian countries, with just over 10 percent of students finishing twelve years of schooling (Akram & Khan, 2007). The recent Barro-Lee's (2010) data indicated that percentage of students who complete college education range 4% to 6 % in Pakistan indicating very low rate in comparison with developed countries (Barro & Lee, 2010).
In Pakistan, the academic achievement is calculated by the CGPA (Cumulative Grade Point Average) that shows the overall academic performance of a student where it considers the average of all examination grades for all semesters during the tenure in a university (Ali et al., 2009). The students performing on the on the low end of the continuum are considered low achievers, with a grade point average below a B (below 70th percentile) on a five-point grading system (e.g., A, B, C, D, and F) while high achievers perform on the high end of the continuum with a grade point average above a B (above 80th percentiles) on a five-point grading system (Cohen, 2001).
Researchers try to relate the constructs of individualist and collectivist culture with specific psychological functioning of the individual (e.g., S attitudes,cognitions,norms,values,goals). In general, group cohesiveness, emotional interdependence, obligation, and group solidarity are characteristics of collectivistic societies whereas personal autonomy, emotional independence, singular actions, and personal goals are related to individualistic societies (Pearson & Child, 2007;Triandis, 1989). As Pakistan is a collectivist culture, the social pattern is characterized by differences in things such as family living arrangements (e.g., collectivism tends to larger families and extended families living under the same roof), social behavior (e.g., collectivists tend to show greater conformity to group norms), beliefs, political ideologies and so on. Because of these trends educational researchers are interested in studying the academic success and adjustment of college students of different societies (Dennis, Phinney, & Chuateco, 2005, p. 223).
Risk factors related to academic achievement are those conditions that increase the likelihood of a student' being of the school dropout or low academic achievers. Of all the personal and psychological factors that have attracted researchers in the area of educational achievement, motivation seems to be gaining more popularity and leading other variables (Awan, Noureen, & Naz, 2011). Motivation is defined as a set of interrelated beliefs and emotions that influence and direct behaviors (Martin, 2009). It has been indicated that low achievers show various motivational problems including a lack of participation in the class, lower self motivation, less goal directed behavior and more negative or non-cooperative attitudes toward institution, teachers or studies than high achievers (Downey & Yuan, 2005;Ma & Xu, 2004;McCoach & Siegle, 2001;2003a;Tella, 2007). Literature documents that positive parental support and nurturance promotes higher academic attainment whereas dysfunctional parental practices have been defined as a potential risk factor for poor academic performance among early and late adolescents (Aunola, Stattin, & Nurmi, 2000;Dennis et al, 2005;Hickman, Kim, & Rohner, 2002;Kordi & Baharudin, 2010). These practices comprise poor parent-child communication, permissive or strict parenting, less acceptance, less supervision, and more conflict towards their children (Moss & St.-Laurent, 2001;Shek, Lee, & Chan, 1998;Stewart, 2007). Further, studies have reported positive relations between peer acceptance or peer support and academic success among both children and adolescents (Fass &Tubman, 2002). It has been found that perceived same-sex and opposite sex peer relationships yield positive direct and indirect links with academic performance and general self-esteem (Liem & Matin, 2011). Moreover, low peer acceptance or peer rejection in adolescence has been identified as a risk indicator for poor school adjustment including academic failure (Buhs & Ladd, 2001). Moreover, parental substance or alcohol abuse also increases a child's risk for behavioral problems that include drug and alcohol abuse, socialskill deficits, and low educational attainment (Fillmore, 1987;Solis, Shadur, Burns, & Hussong, 2012;Winters, 2006). Findings indicate that children from anti-social alcoholic families are most susceptible to relative intellectual, cognitive, and academic deficits. Another individual factor related to low academic performance is language barrier. A number of studies have examined the correlation between language proficiency and academic performance among post-secondary students (Butler & Castellon-Wellington, 2005;Francis & Rivera, 2007;Parker, Louie, & O'Dwyer, 2009). It has been suggested that limited-English-proficient students achieve lower academic grades as well as drop out of school (Rumberger & Larson, 1998).
The main objective of this study was to develop a multidimensional measure of academic achievement risk (personal, familial and peers' related) factors for low academic achievement among Pakistani undergraduate students. Further, to check the validity of the newly developed scale in differentiating low, medium and high academic achievers on identified risk factors in different domains. Therefore, it was hypothesized that low academic achievers are significantly different from high and medium achievers regarding the level of academic achievement motivation, dysfunctional parental practices, parental involvement in drug abuse or antisocial activities, and relationship problems with peers and language-barrier.

Participans and Procedure
The final sample for the present study was comprised of 448 undergraduate students studying at three universities including COMSATS Institute of Information Technology, University of Management Sciences and University of Central Punjab, Lahore, Pakistan (Table 1). Initially, 20 participants were contacted for item generation. The sample included 10 male and 10 female undergraduate students with low CGPA (below 2.51). Afterwards, the clarity and comprehensiveness of the initially formed items was assessed using a separate sample of 30 (22 male and 8 female) undergraduate students. Finally, the exploratory factor analysis was conducted with 448 undergraduate students studying at three universities mentioned above (Table 1).
The average age of students was 20.32 years (SD = 1.83) and it was composed of primarily males (88%) as compared to females (12 %). In terms of income comfort level, on a scale from 1 (not at all satisfied) to 4 (high level of comfort), the mean was 2.1 (SD = 1.0). The mean number of family members in home was 6.5 (SD = 3.27). Seventy three percent of the participants belonged to nuclear and 26% came from joint family living arrangement. In terms of medium of instruction, 38% of the participants had Urdu, while 60% of the participants had English background. For the purpose of this study, three groups were created based on their self-reported CGPAs on a five-point grading system: low (CGPA at or below 2.50 or below 70th percentile), medium (CGPA ranged 2.51 to 3.0 or 70th to 79th percentile) and high (CGPA above 3.1or above 80th percentiles) (Table 1). Note. The numbers do not always lead up to 448 as a result of some missing data

Stages of Scale Development
The Academic Achievement Risk Assessment Scale was developed following the sequential stages given below.
Item generation and content validity. The first step was generation of the items and content validity was the main aim of this step which was accomplished by a theoretical framework and employing a careful sorting process. Through this process, items were matched to construct definition. The literature indicated that different factors affect college students' academic performance (Buddy, 2007;Buhs & Ladd, 2001;Casanova et al., 2005;Kirby & Sharpe, 2001;McCoach & Siegle, 2003b;Rumberger & Larson, 1998). This scale was developed based on combined inductive and deductive approach and therefore, items were derived from two sources: (a) a review of the literature, including studies on low academic achievement factors; and (b) unstructured interviews with undergraduate students with low CGPA. In the current study, the researchers did not establish specific hypothesis regarding the core factor structure of the scale items. Through this process, the researchers came up with 45 items that were then assessed by five subject matter experts including the researcher themselves.
Inter-item correlations and expert feedback. In the second step, the initially generated 45 items were presented to five subject matter experts including the researcher themselves. They assessed the items and provided feedback regarding face and construct validity, comprehensibility and comprehensiveness.
The experts analyzed the items to evaluate its content validity and provided an explanation of the meaning of each item and outlined the objectives, concepts, and definitions of the items. The three steps that were taken were checking for agreement among the experts, discussion, and consensus. The experts ranked each item's priority, deleted or added comments, and provided a level of agreement for each item. Only those items were retained for further analysis when these experts provided 80 % agreement or consensus (Lynn, 1986). As a result of expert feedback, 10 items were excluded and consequently, 35 items remained for the next procedure.
Item categorization and pilot study. In this step, the researchers sorted 35 items into different categories (Churchill, 1979) and applied these items on thirty participants. Different categories included lack of motivation (7items), dysfunctional parental practices (7-items), parental involvement in drugs or antisocial activities (4-items), relationship problems with peers (5items), and language barrier (4-items), and miscellaneous problems (8items). This categorization of items was based on the consensus among three coders (two doctoral students and one researcher herself). The coders independently back translated the 35-items into the different categories to further refine the assignment of the items into categories mentioned above. The only criterion for retaining the item for further analysis was agreement between coders. As a result of participants' feedback, the five point rating scale (1 = not at all, 2 = very less, 3 = less, 4 = mostly, 5 = always) was 32 Jibeen & Khan -Risk Factors Academic Achievement changed to four point scale and the options for the responses were changed into ''Disagree = 1'', "To some extent disagree = 2'', "To some extent agree = 3'', and ''Agree = 4''. Lower scores indicate a lower level of risk factors and higher scores show higher level of risk factors. The purpose of this change was to adjust the opinions of the responses according to the wordings of these items and to get more meaningful responses.
After getting the coders' ratings and pilot study, the researchers eliminated 5 repeated and poorly functioning items leaving a pool of 30items for further analysis. The 30-items were divided into different categories for further analysis including lack of motivation (6-items), dysfunctional parental practices (6-items), parental involvement in drugs or antisocial activities (3-items), relationship problems with peers (4-items), and language barrier (3-items), and miscellaneous problems (8-items). The following stages are related to the validation and refinement of the 30 items on the final sample of 448 participants.

Exploratory Factor Analysis (EFA)
Principal Component Analysis technique was applied on the correlation matrix of the final 30 items. Bartlett's test of sphericity (Bartlett, 1954) was significant (p _ .0001), showing that the data were adequately distributed to allow an evaluation of the potential factor structure. Next, Kaiser-Meyer-Olkin yielded a value of .82, indicating that the ratio of the number of participants to AARS items was sufficient to run a principal-component factor analysis. The factors were based on the following criteria including : (a) an unrotated eigen value>1 with a category factor loadings of at least .35 (b) a simple structure with each factor different from one another and with all items loading highly on one factor (c) and interpretability, that the factor represents a meaningful underlying aspect (Zeller & Carmines, 1980). The Kaiser criterion and the total explained variance criteria were also used for the determination of "meaningful" factors (Kaiser, 1974). The five factor solution most closely corresponded to the best approximation of simple structure with the fewest number of cross-loadings and it yielded the most interpretable solution.
The principal component analysis, item loadings and communality coefficients for the final 18 items are presented in Table 2.  Note. Item 1 to 5 = Lack of motivation; Item 6 to 10 = Dysfunctional parental practices; Item 11 to 12 = Parental involvement in drugs or antisocial activities; item 13 to 16 = Relationship problems with peers; Item 17 to 18 = Language Barrier.
The final components were consisted of those selected items with a factor loading at least 0.50 on a specific component, cross-loadings not exceeding 0.30, and loading on two factors with the difference of less than 15 units. The items with miscellaneous problems (8-items) components did not meet the minimum retaining criteria of 0.50 values and items with cross-loadings with the difference of less than 15 units were deleted.
After item deletion, 18-items with five factors were retained including Lack of motivation, Dysfunctional parental practices, Parental involvement in drugs or antisocial activities, Relationship problems with peers and Language Barrier (see Table 2). The five factors accounted for 25.81%, 10.10%, 7.89%, 6.84%, and 6.13% variance respectively. The overall variance accounted for 57%, while the communalities ranged from .36 to .80 after extraction (see Table 2). The four point Likert-type scales ranging from 1 (completely disagree) to 4 (strongly agree) were used for 18-items (see Appendix A). The resulting total 18-items AARAS had a coefficient alpha of .81 and the lack of motivation, dysfunctional parental practices, parental involvement in drugs or antisocial activities, relationship problems with peers and language-barrier subscales had alphas of .80, .81, .82, .79, and .64, respectively ranging from moderate to high. Although these results were promising, data-driven modifications to instruments may capitalize on chance (Jöreskog, 1993). Thus, further investigation into the reliability of the AARAS with an independent sample is needed. The 18-items were administered to a separate 40 participants (67% male and 32% female) in second reliability analysis study. The five subscale and total inter correlations were moderate to large in size, ranging from r = .64 to r = .82. The results from ANOVA did not indicate any significant gender effect for the lack of motivation (F = .314, p =.57), dysfunctional parental practices (F = 1.42, p = .23), parental involvement in drugs or antisocial activities (F = .29, p =.53), relationship problems with peers (F = .11, p = .73), and language barrier (F = 3.30, p = .07). Five parametric analyses of variance procedures were performed to examine the difference between three academic achievement groups based on CGPA. Bonferroni method of adjustment was utilized such that each statistical analysis had to reach a level of .01 for a result to be considered statistically significant. A one way ANOVA showed that the three groups (low, medium and high CGPAs) were statistically significant regarding lack of motivation, F(2, N = 402) = 15.44, p < .0001, dysfunctional parental practices, F (2, N = 427) = 8.50, p < .001, parental involvement in drugs or antisocial activities, F (2, N = 439) = 7.07, p < .01, relationship problems with peers, F (2, N = 414) = 7.7, p < .0001, and language barrier , F (2, N = 437) = 6.65, p < .01 (Table 3). Because the overall test was significant, post-hoc tests (i.e., Tukey's HSD) were used to decompose and interpret the results of the ANOVA. The post-hoc comparisons revealed that the mean scores of low CGPA group was typically higher than high CGPA regarding all risk factors including lack of motivation, M = 2.154, SD = .3888, p < .0001, dysfunctional parental practices, M = 1.517, SD = .3679, p < .001, parental involvement in drugs or antisocial activities, M = 8.5549, SD =.1495, p < .01, relationship problems with peers, M = 1.328, SD = p < .0001, and language barrier, M = .6422, SD =.1847 , p < .01. Further, the high CGPA group was significantly different from medium CGPA group regarding lack of motivation, M = -1.27481, SD = .39596, p < .01, and relationship problems with peers, M = -.93997, SD =.34892, p < .01. The other factors including language barrier, M = .8792, SD =.3888, p < .05, and parental involvement in drugs or antisocial activities, M = -.37204, SD = .15227, p < .05, did not reach Bonferroni criteria for significance (.01).

Discussion
The main purpose of this multistage investigation was to explore the risk factors associated with low academic achievement and to compare high, medium and low academic achievers on these factors at undergraduate level. The current scale is comprehensive as it focuses on salient factors related to low academic achievement that previously had not been combined in a single measure (Nunnally & Bernstein, 1994). These factors included lack of motivation, dysfunctional parental practices, parental involvement in drugs or antisocial activities, relationship problems with peers and language barrier. The overall variance explained by all of these factors accounted for 57%. The current study provided confirmatory evidence to previously identified themes in literature (Bean, Bush, McKenry, & Wilson, 2003;Buddy, 2007;Buhs & Ladd, 2001;Diaz, 2003;Eamon, 2005;Oliverez & Tierney, 2005;Turner, Chandler, & Heffer, 2009).
The present results are in line with previous studies (Baker et al., 1998;Lufi & Cohen, 2003) indicating that low academic achievers are significantly different from high and medium academic achievers regarding low motivation characteristics. Sarwar, Bashir, Naemullah and Khan (2009) conducted a study with Pakistani secondary school students and found that the high achievers showed better study orientation and study habits than the low achievers. Literature (Jeynes, 2005;Mandara, 2006;Moss & St.-Laurent, 2001;Whitlock, 2006) has emphasized that parental support and warmth and monitoring are the key parental characteristics that enhance student's academic performance even after entering college. The present findings supported the literature (Shek, Lee, & Chan, 1998;Stewart, 2007) indicating that in comparison to students with high academic achievement, the parents of students with low academic achievement significantly indicate higher level of dysfunctional parental practices (e.g., parental strictness, lack of monitoring etc). In another study, Casanova, Garcia-Linares, Torre, and Carpio (2005) found that in the group of students with low achievement, parents were classified as authoritarian, permissive and indifferent. Further, students with problems reported that their parents show lower levels of supervision, support and affection as well as higher levels of conflict than students with no achievement problems.
The incidence of dysfunctional parental practices and low academic performance can be justified by observing a significant gap in the dropout rate between students who have a strong family background and those who have a weak background. It has been suggested that parental involvement activities and family practices are more important for helping students succeed in school than are family structure including socioeconomic status or characteristics such as race, family size, or age of child (Hidalgo, Epstein & Siu, 2002). It is important to note that parenting forms the basis of a family environment and without parental education; it may not possible for them to fulfill their roles and duties in the family and the society (Kordi & Baharudin, 2010;Sinha & Singh, 1998). It seems that educated parents seem to provide all possible support services including coaching, guidance and facilities to their children as they are more competent than uneducated parents. For example, Hidalgo, Epstein and Siu (2002) found that education contributes to improve the parents' capacity to intervene in their children's education, for instance, establishing supportive home environments for children and helping children out with their homework. At the other hand, when the parents have little knowledge about the specific demands of academic fields and their children's lack of potential to succeed in different fields, they are more likely to practice authoritarian parenting to fulfill their own aspirations (Rudy & Grusec, 2006).
Research (Anna & Nattavudh, 2009;Hasnain & Krantz , 2010) indicates that the students from higher socio-economic and more educated backgrounds have lower rates of dropouts whereas those from poor and uneducated background higher rates of dropouts in Pakistan. The family structure affects children through the degree to which family members provide resources or compete for them. As extended family members who live with their children are generally poorer, less healthy, and less educated. Thus, children who live with extend family members (especially grand-parents) typically have lower academic achievement than those who do not live with extended family members (Ainsworth, 2013). A child's parents might give or lend money to poor relatives, thereby reducing the immediate resources available to the child. Moreover, siblings and extended family members share parents' attention, so children with more siblings have lower academic achievement (Chadda & Deb, 2013). Recently, Hasnain and Krantz (2010) investigated the risk factors associated with college dropouts among young adults in Karachi, Pakistan, and found that migrant residential status, living in an extended family and lower socio-economic status were identified as risk factors for college dropouts both for males and females.
The present results indicated that the low academic achievers significantly revealed parental substance abuse or criminal activities than did high achievers (Dallaire, Ciccone, & Wilson, 2010). One important potential explanation is that adult children of substance abusing parents may show cognitive deficits that impact their academic performance in college (Solis, Shadur, Burns, & Hussong, 2013;Winters, 2006). These adult college students (respectively) are typically exposed to negligent or abusive parenting and financial hardships. It is important to note that academic difficulties in children of alcoholics are partly due to less parental involvement in their academic activities, lower levels of family organization and less parental involvement in their college or school educational activities (Gonzalez-DeHass, Willems, & Holbein, 2005).
The current analyses revealed significant peer relationship problems and uncomfortable feelings in coeducational setting in the low and middle achiever students than did high achievers. The present findings are in line with literature (Thompson & Ungerleider, 2004) indicating that students from single-sex schools score higher than students from coeducational schools. It has been noted that single-sex schools actually benefit boys the most-specifically, boys from minority groups and boys from poor families who may need more direct guidance (Guarisco, 2010). For example, Hopkins (1997) found that single sex schooling is particularly effective for low-income African, American and Hispanic boys.
Working from a social psychological perspective, advocates of single sex environment describe concerns about the negative stereotypes, low expectations, and relative lack of student and adult role models in coeducational schools (Singh, Vaught, & Mitchell, 1998). In a recent study, Ogden (2011) found that single-sex environments help to reduce gender stereotypes that students encounter in coeducational settings and they are generally more settled and more relaxed (Sax, 2008;Wills, 2007). Though it is claimed that single-sex schools are superior to coeducational schools, in reducing sex differences, but in most countries, single-sex schools tend to be private, whereas coeducational schools tend to be government; therefore, this hypothesis is very hard to test in an unconfounded way (Thompson & Ungerleider, 2004).
Literature (Carlivati, 2001;Liem & Martin, 2011) suggests that students doing well in school have been found to have a close friend than those rejected by peers. Researchers (Buote, 2002;Martin, 2012;Martin & Dowson, 2009;Stewart, 2007) noted that the involvement with positive peer group activities contributes to academic success, controls violent inclinations and increase the expression of pro-social behavior. In a recent study (Swenson Goguen, Hiester, & Nordstrom, 2010) the importance of peer relationships to academic outcomes of first-year undergraduates was tested and it was found that sharing common interests and having trust in peer was positively related to GPA while the extent of conflict with a new college friend was associated negatively with GPA and persistence to the second college year.
Finally, in current sample, the students revealing low proficiency in English language reported low academic performance as compared to high academic performance. These finding are in line with previous literature (Butler & Castellon-Wellington, 2005;Francis & Rivera, 2007;Kong, Powers, Starr & Williams, 2012;Parker, Louie & O'Dwyer, 2009) suggesting that low language proficiency has been considered a barrier to learning and academic success at the post-secondary level because sufficient level of English language proficiency is needed to be able to demonstrate content knowledge on academic assessments.

Limitations
One of the limitations of the current study is the moderate reliability of subscales as Cronach's alphas for the subscales were moderate. In the resent study, probably the small number of items in each subtest and limited (4-points' scale) width resulted in these "relative moderate coefficients". Indeed, it has been shown that Cronbach's alpha estimation of reliability increases with scale length (Voss, et al., 2000). Other limitations include the use of self-report questionnaires to assess the outcome variables, the lack of temperament and IQ measures to assess how student temperament factors and ability affect the perception of the variables reported, and the cross-sectional nature of the study. Another limitation is related to the lack of information about those students who might have learning disabilities as they need comprehensive assessment separately using appropriate questionnaires.

Implications
The current study has demonstrated the utility of risk-focused ecological model that could be effective in improving academic achievement of students. The academic achievement predictive model is particularly important for college student personnel that are looking for ways to identify students who are at risk for academic difficulties. It is important to note that the college counselors might use these data as an impetus for furthering development of behavior modification of parents and students. For example, there is need for the promotion of parenting programs emphasizing home environments of warmth and autonomy during adolescence to help students be more academically successful throughout their education. These programs would help students develop skills that an authoritative home environment imparts, such as elements of mastery and persistence, which are important for success in college.