Non Experimental Research Design Psychology Essay

Non Experimental Research Design Psychology Essay This chapter details the methodology that will be used to conduct this study. Methodology is an approach, or method used to conduct a research. The aspects that will be discussed in this chapter include the researchs design, approach, sample, instrument, procedure, and also data collection and analysis. 3.2 Researchs Approach and Design 3.2.1 Researchs Approach There are two types of approaches used in research, which are quantitative approach and qualitative approach. According to Blaxter, Hughes and Tight (as cited in Hughes, 2006) Quantitative research is a study that concerned with the collection and analysis of data in numeric form. It emphasizes relatively large-scale and representative sets of data, and is often presented or perceived as being about the gathering of `facts. Qualitative research is concerned with collecting and analysing information in as many forms as possible, mainly non-numerical form. It focuses on exploring smaller numbers of instances or examples which are seen as being interesting or illuminating, and aims to achieve `depth rather than `breadth. In this study, the data collected by researcher will be in numerical form, which obtained from the questionnaires. Thus, the approach used in this study is quantitative research. 3.2.2 Researchs Design According to Chua Yan Piaw (2006), there are three main designs in research, namely true experimental design, non-experimental design, and quasi-experimental design. True experimental design studies the relation between the independent variables and manipulated variables, manipulates the independent variables and observes the changes on manipulated variables. Non-experimental design is used when the treatment is not possible as the independent variables naturally exist, and the respondents will be randomly distributed into every group. This design studies the relation between independent variables and manipulated variables without manipulate the independent variables. Besides that, quasi-experimental design normally used to evaluate the effectiveness of certain program when then the randomly distribution of the respondents in the study is not possible. It studies the relation between independent variables and manipulated variables (Chua, 2006). Non-experimental research design According to Chua Yan Piaw (2006), there are a lot of non-experimental designs, including survey, field research, case study, action research, ethnography, and so on. Survey is one of the most widely used non-experimental research design, which used in different form of media, such as magazine, newspapers, and television to collect data from subjects who respond to a series of questions about behaviors and opinions regarding to certain issue, or to survey the effectiveness of a product or plan. Normally interview or quetionnaires will be used in collecting data. Field research refers to gathering primary data from a natural environment without doing a lab experiment or a survey, where the researcher must be willing to step into new environments and observe, participate, or experience those worlds by themselves. Case study is an in-depth analysis of a single event, situation, or individual, where the researcher examines existing sources like documents and archival records, conducts in terviews, engages in direct observation, and even participant observation, to collect the information in depth regarding to individual behaviours and social conditions. Action research refers to the research initiated to solve an immediate problem or a reflective process of progressive problem solvin lef by individuals working with others in teams or as part of a community of practice to improve the way they address and solve problems. Survey 3.3 Sampling According to Gay and Airasian (2003), sampling is the process of selecting a number of participants for a study in such a way that they represent the larger group from which they were selected (p. 101). Selecting a sample is a very important step in conducting a research study, particularly for quantitative research. The goodness of the sample determines the meaningfulness and generalizability of the research results (Gay Asian, 2003, p. 103). 3.3.1 Population The first step in sampling is to define the population. A sample comprises the individuals, items, or events selected from a larger group referred to as a population. The population is the group of interest to the researcher, the group to which the results of the study will ideally generalize (Gay Airasian, 2003, p. 102). The location of this study is Sabah, and the populations of this study are all the upper form students in secondary schools in Sabah, who are 16 to 17 years old. Initially, the researcher will obtains the information about the upper form students from all the secondary schools in Sabah, and then determine the samples which are involved in this study. The researcher will get the information about the total number of upper form students in each of the schools, because it might influences the outcomes of the study. 3.3.2 Sampling Method According to Chua Yan Piaw (2006), there are two main sampling methods, which are probability sampling and non-probability sampling (p. 189). According to Gay and Airasian (2003), probability sampling, which also known as random sampling, methods are possible for the researcher to specify the probability, or chance, that each member of a defined population will be selected for the sample, and these sampling are all based on randomness in the selection of the sample (p. 103). Non-probability sampling, which is also known as non-random sampling, methods do not have random sampling at any stage of sample selection, it is used when the random sampling is not possible, such as teachers or administrators select students or classes as the samples (p. 114). In this study, random sampling method will be used to select the sample. Gay and Airasian (2003) states that there are four basic random sampling techniques or procedures, namely simple random sampling, stratified sampling, cluster sampling, and systematic sampling, which are also referred to as probability sampling (p. 103). Simple random sampling is the process of selecting a sample in such a way that all individuals in the selected population have an equal and independent chance to be selected for the sample, where the randomness in sampling takes the selection of the sample completely out of the researchers control by letting a random, or chance, procedure select the sample (p. 103). Stratified sampling is the process of selecting a sample in such a way that identified sub-groups in the population are represented in the sample in the same proportion that they exist in the population (p. 106). Besides that, cluster sampling randomly selects groups, not individuals; all the members selected groups have similar characterisics, and it is most useful when the population is very large or spread out over a wide geographic area (p. 108). Lastly, systematic sampling is the sampling technique in which individuals are selected from a list taking every Kth name (p. 110). Although the instrument of students national identity is designed to be applicable to all students in Malaysia, the secondary school students will be selected to test the instrument in this study. The rationale of choosing upper form students as the target population to test the instrument is they might be more mature if compared to the lower form students who are only 13 to 15 years old. They might have a better understanding on the items stated in the questions, and perhaps they will give their response seriously. So, the outcomes which obtained from the upper form students might be more reliable and trustworthy if compared to lower form students. All upper form students from the secondary school in Sabah are over sampled. Two random samples of 200 students and 2500 students will be purchased for the two survey studies by using cluster sampling method. A number of schools will be randomly selected from each of the districts in Sabah as the representatives of those districts. The secondary school students in Sabah are eligible to receive the mailed survey. In the first survey study, the questionnaire will be initially sent to the random sample of 200 students to examine test-retest reliability of the instrument and its subscale. Then, a repeat mailing survey will be sent to the respondents from the initial mailing again. The test-retest reliability of the instrument will be examined using the data from the remaining surveys returned from both the initial and repeat mailings. In the second survey study, the sample needed to be large in order to conduct factor analysis to test the construct validity of the instrument. There is a general agreement among measurement methodologists that large sample sizes are required for the stability of results of factor analysis, the use of larger sample sizes in applications of factors analysis tends to provide results such that sample factor loadings are more precise estimates of population loadings and are also more stable, or less variable, across repeated sampling. The researcher will send the refined questionnaire to the second random sample of 2500 students in order to perform factor analysis, and then to test construct validity and internal consistency reliability of the instrument. 3.4 Researchs Procedures This study will be conducted in two stages. Stage one is the instrument development, and stage two is the instrument testing and refining. Stage one consists of three steps and stage two consists of five steps. Each of these steps is described in the following section. 3.4.1 Stage One: Instrument Development The instrument development stage has three steps: (1) developing conceptual and operational definitions of the construct of students national identity, (2) generating item pool, and (3) determining the format for measurement (or selecting a scaling technique for the measurement). All the three steps in this stage involved determination of content validity of the instrument of students national identity. a. Step 1: Developing Conceptual and Operational Definitions of the Construct. The first step in developing this instrument is to determine clearly what the concept of students national identity is. The definition of the construct, students national identity, is based onà ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦ (need more information from chapter 2) b. Step 2: Generating an Item Pool for the Instrument The focus of second step is to generate a large pool of items for all the dimensions of the construct of students national identity. In the beginning stage, it is better to generate more items. Thus a 60 items scale might be evolve from an item pool over a hundred items. The instrument of students national identity is designed to measure the degree to which a student à ¢Ã¢â€š ¬Ã‚ ¦. Wording of the items is very important and should reflect this goal of the instrument. Appropriate wording can accurately capture the essence of the construct. The items are evaluative in nature and can reflect respondents views about the desirability of something. Moreover, the researcher developed several negatively worded items with careful wording that examine students à ¢Ã¢â€š ¬Ã‚ ¦ besides developing positively worded items that measure students national identity. The purpose of constructing several negatively worded items is to detect respondents with acquiescence bias by their response pattern if there is any. Those respondents will not be included in the data analysis to avoid or minimize the influence caused by acquiescence bias. The issues related to acquiescence bias were discussed in detail in Chapter 2. After a large pool of items that measures students national identity is established, the dissertation committee will be first critique all the items before sending the item pool to the expert panel for content validity review. The researcher will revises the item pool based on comments of the dissertation committee. The items which are repetitive, inappropriate or poorly worded, confusing, or irrelevant to the construct will be eliminated or revised. After this revision, 84 items were retained in the instrument. c. Step 3: Determining the Format for the Instrument The researcher considers the format simultaneously with the generation of items so that the two are compatible. The Likert scales with five responses options is chosen to develop the instrument measuring the students national identity. Each item is presented as a declaration sentence, followed by the response options that indicate varying degrees of agreement with the statement. The five response options are: (1) strongly disagree, (2) disagree, (3) either agree or disagree, (4) agree, and (5) strongly agree. These five points Likert scales are ordinal scale level. Each item in the instrument is classified as two broad categories, which are favourable (positive) or unfavourable (negative). Scoting is reversed for negatively worded items such that disagreement with a negatively worded resulted in high score. At this step, the development stage is completed. Then, the researcher will moves on to the next stage, which is instrument testing and refining. 3.4.2 Stage 2: Instument Testing and Refining The instrument testing and refining stage includes give steps: (1) establishing content validity of the instrument, (2) developing directions for responding, (3) preparing a revised draft of the questionnaire, (4) examining test-retest reliability and pretesting internal consistency reliability, dan (5) testing construct validity and internal consistency reliability. All steps in this stage involve refining the instrument and testing the psychometric properties of the instrument. a. Step 1: Establishing Content Validity of the Instrument Determining the number of experts needed has always been somewhat arbitrary in content validity determination. According to Lynn (1986) (as cited in Wynd, Schmidt, Schaefer, 2003), a minimum of five experts (rates, observers, or judges) and a larger number of categories for data assignment yield greater absolute agreement and increase the risk of chance agreement. The use of more experts may therefore contribute directly to chance agreement (pg 511). The use of only two judges is not only statistically unjustifiable, but also it places the instrument developer at great risk of an erroneous conclusion that content validity has been achieved when it actually has not (Bu, 2005, pg 73). In this study, five experts who either have conducted research related to students national identity or have interest in students national identity will be asked to review the instrument as the content experts. A delineation of the full content dimentions of the construct of students national identity with specific instructions pertaining to the content relevance of each item are provided to the experts by mail or email for review. The five will be asked to return the instrument and their comments within four to five weeks. This review serves multiple purposes related to determining and maximizing the content validity of the instrument. First, having experts review the item pool, confirm or invalidate the definition of the phenomenon of students national identity and the content validity of the instrument is quantified. The experts will be asked to rate how relevant they think each item is related to what the researcher intends to measure. The instrument of students national identity includes three subscales. The general objective of the subscale of à ¢Ã¢â€š ¬Ã‚ ¦ is to measure the degree to which the studentà ¢Ã¢â€š ¬Ã‚ ¦ Under the broad objective, more specific objectives that are assumed to measure relevant items will provided as well. The intensity of students national identity can fluctuate over time but they are considered stable during a certain period of time, typically three to four weeks. These objectives will be sent to experts along with the instrument. The experts will be asked to rate the content relevance of each item to its general objective and its more specific objective. Content validity index (CVI)à ¢Ã¢â€š ¬Ã‚ ¦.. In addition, the experts will be asked to clarify reasons and provide suggestions if they disagree with some items included in the instrument. Second, the experts will be asked to evaluate the items clarity and conciseness. Sometimes, the content of an item may be relevant to the construct but its wording may be problematic. This yields problems on item reliability because an ambiguous or otherwise unclear item, to a greater degree than a clear item, may reflect factors extraneous to the latent variable. Third, in addition to judging each item, the experts will be asked to identify phenomena that were omitted from the instrument as a part of the content validity assessment. Thus, by asking experts to review the instrument in a variety of ways the researcher could capture the phenomenon of interest, and the expert reviewers help the researcher maximize the content validity of the instrument. The researcher will be paid careful attention to all suggestions from these content experts then made an informed decision about how to use their advice. The instrument will be revised to enhance content validity with the consensus of the dissertation committee. b. Step 2: Developing Directions for Responding The procedures for responding to the statements, as well as the meaning of the anchor points on the continuum, will be carefully develop and then reviewed by the dissertation committee chair and colleagues to avoid confusing respondents. Dillman (1978) (as cited in Bu, 2005) proposed some ideas about providing directions to subjects on how to answer questionnaires. He states that the encirclement process results in fewer ambiguous markings and should be encouraged. It is important that the same marking procedure be used throughout the questionnaire. Lower case letters are preferred for directions because of their greater readability (pg 76). The researcher will apply these suggestions when developing directions for responding. c. Step 3: Preparing a Draft of the Questionnaire. This step involves constructing a draft of the questionnaire and assessing the questionnaire. Three activities will be completed in this step. First, a section of questions for gathering demographic information from participants will be designed and included in the survey along with the instrument of students national identity. The purpose of developing this section of questions is to gather information that would be used to describe characteristics of the participants. Second is the issue of ordering the questions in the questionnaire. In this study, items that are measuring the same dimension will be grouped together. The questions that request demographic information are put at the end of the questionnaire. The questionnaire consists of two parts. Part one is composed of a series of items pertaining to students national identity. Items represented à ¢Ã¢â€š ¬Ã‚ ¦.dimensionsà ¢Ã¢â€š ¬Ã‚ ¦.. The form of the instrument of students national identity looks like the following sample. Item Strongly disagree (1) Disagree (2) Either disagree or agree (3) Agree (4) Strongly agree (5) Item 1 1 2 3 4 5 Item 2 1 2 3 4 5 Item 3 1 2 3 4 5 à ¢Ã¢â€š ¬Ã‚ ¦ Part two of the questionnaire consists of the demographic questions and includes à ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦ as well as characteristics such as age, gender, ethnicity, geographic location. Third, the questionnaire is distributed to à ¢Ã¢â€š ¬Ã‚ ¦. For the review of clarity of directions, ease of responding and time needed to fill out the questionnaire. d. Step 4: Examining Test-Retest Reliability and Pretesting Internal Consistency Reliability After questionnaire is produced, the researcher conducts two survey studies to gather data to examine construct validity and reliability of the instrument. Step 4 is to examine test-retest reliability of the instrument, preliminarily test internal consistency reliability of the instrument, and perform item analysis. It is assumed that the construct of students national identity does not change in subjects within two to three weeks. The researcher initially mailed the questionnaire to à ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦.. In two to three weeks, a repeat survey will be mailed to the respondents from the initial mailing. The test-retest reliability of the instrument of students national identity and its à ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦..subscales are examined among the subjects responding both the initial and the repeat mailings. Cronbachs alpha of the instrument and its à ¢Ã¢â€š ¬Ã‚ ¦. Subscales is calculated using the data from the questionnaires returned from the initial mailing to pretest internal consistency reliability of the instrument and itsà ¢Ã¢â€š ¬Ã‚ ¦.subscales. In addition, an item analysis for each subscale will be performed using the data from the returned questionnnaires from the initial mailing for the purpose of refining the instrument. An item needed to meet Likerts criterion of internal consistency in order to be retained in the scale. A given item whose score significantly correlated with the relevant scale score, which is à ¢Ã¢â‚¬ °Ã‚ ¥ 0.3 is considered to meet the criterion of internal consistency and is retained in the instrument. An item whose score does not significantly related to the scale score, which is à ¢Ã¢â‚¬ °Ã‚ ¤ 0.3, is rechecked and determined for retainment, elimination or revision depending on theory, content of the item, function of the item in the instrument. e. Step 5: Testing Construct Validity and Internal Consistency Reliability In this step, the refined survey will be sent to à ¢Ã¢â€š ¬Ã‚ ¦.. to collect data in order to examine the construct validity of the instrument using factor analysis and the internal consistency reliability of the instrument using Cronbachs alpha. Factor analysis is performed using data from the returned questionnaires. Factor analysis is used to determine construct validity of the instrument of students national identity and select items for inclusion in the instrument. Factor analysis is a broad category of approaches to determine the structure of relations among measures (Nunnally Bernstein, 1994). Factor analysis can be used to determine: (1) grouping variables, (2) which variables belong to which factor and how strong their relationship, (30 how many dimensions are needed to explain the relations among the variables, and (4) a frame of reference to describe the relation among the variables more conveniently. There are two major approaches to factor analysis: exploratory factor analysis and confirmatory factor analysis. In exploratory factor analysis, one seeks to summarize data by grouping together variables that are intercorrelated. The variables themselves may or may not have been chosen with potential underlying structure in mind. Exploratory factors are defined to achieve a mathematical objective such as maximizing the variance accounted for. In confirmatory factor analysis, factors are defined directly, which incorporate the properties such as the number of factors and content (or variables) of each factor that have been hypothesized and then determine how well these fit the data (Nunnally Bernstein, 1994). According to Tabachnick Fidell (1983), exploratory factor analysis is usually performed in the early stages of research to consolidate variables and generate hypotheses about relationships in a reduced data set. Confirmatory factor analysis generally occurs later in the researc h process, when a theory about structure is to be tested or when hypothesized differences in structure between groups of research units are tested. Variables are specifically chosen to reveal underlying structural processes. Data used in confirmatory factor analysis, then, might be different from those used in exploratory factor analysis. Cronbachs alpha will be calculated in this large sample to determine the internal consistency reliability of the instrument of attitudes toward patient advocacy and its three subscales. Up to this step, the instrument of students national identity will be established. 3.5 Data Collection Schedule and Procedures There are some differences between the data collection schedule and procedures for the test-retest study and the construct validity study. For the test-retest reliability study, surveys with the cover letters will be mailed to the 200 randomly selected subjects. A stamped returning envelope will be included in the mail for return of the survey. A code number will be attached to the 200 subjects, and those respondents from the initial mailing survey will be identified and sent a repeat survey within two to three weeks after the initial mailing in order to examine test-retest reliability. Thus, anonymity of respondents is not ensured in the test-retest reliability study for the purpose of the repeat mailing to those who returned the survey. Considering budget limitation for the dissertation, the researcher will makes only one contact with the 2500 randomly selected subjects. In this contact, a mailing that included the survey, a detailed cover letter explaining the nature of the study and asking for response and a business reply envelope will be sent to the 2500 subjects. Since only one contact was made to the 2500 students, anonymity of respondents is ensured in the study. A codebook for data entry will be developed and reviewed. A data entry program will be set up to facilitate inputting data. Data will be entered as responses to the questionnaire are returned. Entered data will be verified to check for errors in data input by going over 25% of the returned and entered questionnaires and running frequencies of each item of the questionnaire. Any differences between original data entry and verification required checking the raw data and correcting the entered data. Data analysis will begins after completion of data entry and involved statistical consultation with the dissertation committee members. 3.6 Data Analysis and Interpretation Three types of data analyses will be conducted in this study, which are analysis related to reliability, analysis related to validity, and descriptive statistics. First, the content validity of the instrument will be determined by CVI. The CVI was the proportion of items given a rating of 3 (relevant and needs a little revision) or 4 (very relevant) to the objectives of a measure based on 4-point ordinal scale by at least six out of the seven experts (à ¢Ã¢â‚¬ °Ã‚ ¥86% agreement) in this study. Second, data from the respondents returned from the initial mailing in the test-retest study will be used for the analysis of Cronbachs alpha to pretest the internal consistency reliability of the instrument. Data from the questionnaires that are returned from both the initial and the repeat mailings will be used for examining the test-retest reliability. A total score for each of the subscales and the whole instrument will be obtained with the initial mailing, and a second total score for each of the subscales and the whole instrument will be obtained with the second mailing. The test-retest reliability coefficient of the instrument and its subscales will be obtained by correlating the initial mailing scores with the second mailing scores using Pearson Product Moment Correlation Coefficient. High correlation coefficient indicates high stability or test-retest reliability of the instrument. This reliability coefficient above .70 is considered satisfactory. Third, item analysis will be performed using data from the questionnaires returned from the initial mailing. Correlations among items within each subscale and between each item with the total subscale score will be examined. Items with low correlations with the relevant subscale score ( Fourth, factor analysis will be performed using data from the questionnaires to determine the construct validity of the instrument of students nationasl identity. Factor analysis is à ¢Ã¢â€š ¬Ã‚ ¦() In this study, exploratory and confirmatory factor analysis approaches will be used to examine the construct validity of the instrument. Exploratory factor analysis will be first performed to extract factors from the instrument of students national identity and determine the items to be included in the instrument. If evidence for construct validity exists, the number of factors resulting from the analysis should approximate the number of dimensions assessed by the instrument, and the items with the highest factor loadings defining each factor should correspond to the items designed to measure each of the dimensions of the instrument (Waltz et al., 1991). Exploratory factor analysis consists of two stages: extract factors and rotating extracted factors (Nunnally Bernstein, 1994). The analysis first condenses the variables (items) into the smallest number of factors that explain the most variance. Principal component analysis (PCA) and principal axis analysis (PAF) are two most popular ways to condense data. à ¢Ã¢â€š ¬Ã‚ ¦.. Then, three most common statistical rules were used to determine the number of factors. First, the Kaiser-Guttman rule retains factors with absolute eigenvalues of 1.0 or greater. However, this rule tends to suggest too many factors. The second rule is scree test that uses relative changes in these eigenvaluesà ¢Ã¢â€š ¬Ã‚ ¦.. Since the unrotated factors are usually difficult to interpret, the second stage of exploratory factor analysis is to rotate these factors to make them more meaningful or more interpretableà ¢Ã¢â€š ¬Ã‚ ¦.. Orthogonal and oblique rotations Results of the exploratory factor analysis will also be used for identification and selection of indicators (items) for the instrument. Confirmatory factor analysis was used to validateà ¢Ã¢â€š ¬Ã‚ ¦In confirmatory factor analysis, the researcher specifies which items load on each factor according to preconceived theory to test the theory. The result of this analysis indicates how well the empirical data actually conform to these specifications, that is, whether the items actually form the theorized constructs. In order to test the fit of the model to the data, multiple fit indices that reflect somewhat different facets of model fit are suggested à ¢Ã¢â€š ¬Ã‚ ¦.. Goodness-of-Fit Index (GFI) Comparative Fit Index (CFI) Adjusted Goodness-of Fit Index (AGFI) Root Mean Square Residual (RMR) Root Mean Square Error of Approximation (RMSEA) Finally, Cronbachs alpha will be calculated using the data from the returned surveys to determine the internal consistency reliability of the instrument and its subscales. High Cronbachs alpha indicates that the instrument has high internal consistency. Nunnally and Bernstein suggest that, for the newly developed instrument, the Cronbachs alpha with .70 is acceptable. Also, descriptive statistics such as frequency, mean, and standard deviation will be used to describe the characteristics of the sample in both the test-retest reliability study and the construct validity study. Test-retest reliability, item analysis, descriptive statistics, exploratory factor analysis and Cronbachs alpha will be performed using SPSS version 20.0.0 for Windows. AMOS 5.0 for