Analysis of Sample Test Statistics
Assessment Traits
Requires Lopeswrite
Assessment Description
Data from assessments can be used to determine if learners are meeting course outcomes or learning objectives. Assessments can be utilized in many ways, such as learner practice, learner self-assessment, determining readiness, determining grades, etc. The purpose of this assignment is to analyze sample test statistics to determine if learning has taken place.
To address the questions below in this essay assignment, you will need to use the information from your textbook chapter readings and the data provided in Table 11.1 ("Sample Test Statistics") in Chapter 11 of The Nurse Educator's Guide to Assessing Learning Outcomes.
In a 1,000-1,250-word essay, respond to the following questions:
Explain what reliability is and whether this test is reliable based on the information in Table 11.1 ("Sample Test Statistics"). What evidence supports your answer?
What is the range for this sample? What information does the range provide and why is it important?
What is the difference between standard deviation and standard error of measurement? How would the instructor use this information?
Explain the process of analyzing individual items once an instructor has analyzed basic concepts of measurement. Consider the three Ds (difficulty, discrimination, and distractors) in your response.
If one of the questions on the exam had a p value of 0.100, would it be a best practice to eliminate the item? Justify your answer.
If one of the questions on the exam has a negative PBI for the correct option and one or more of the distractors have a positive PBI, what information does this give the instructor? How would you recommend that the instructor adjust this item?
You are required to cite two or three sources to complete this assignment. Sources must be published within the last 5 years and appropriate for the assignment criteria and nursing content.
Prepare this assignment according to the guidelines found in the APA Style Guide, located in the Student Success Center.
This assignment uses a rubric. Please review the rubric prior to beginning the assignment to become familiar with the expectations for successful completion.
You are required to submit this assignment to LopesWrite. A link to the LopesWrite technical support articles is located in the Class Resources if you need assistance.
RUBRIC: Please follow the following RUBRIC
Rubric
collapse Reliability assessment
Reliability
15 points
Criteria Description
Explanation of reliability. Determine reliability of test from Sample Test Statistics. Evidence supporting answer.
5. Target
15 points
A well-supported and accurate explanation of what reliability is and whether the test presented in the assignment criteria is reliable is presented.
4. Acceptable
13.8 points
An accurate explanation of what reliability is and whether the test presented in the assignment criteria is reliable is presented. Some detail is needed for clarity or support.
3. Approaching
13.2 points
A general explanation of what reliability is and whether the test presented in the assignment criteria is reliable is accurately presented but additional information or evidence is needed for support.
2. Insufficient
12 points
An explanation of what reliability is and whether the test presented in the assignment criteria is reliable is presented but is incomplete or incorrect.
1. Unsatisfactory
0 points
An explanation of what reliability is and whether the test presented in the assignment criteria is reliable is omitted.
collapse Range assessment
Range
15 points
Criteria Description
Determine range for the given sample. Discuss the findings for the range and its importance.
5. Target
15 points
The range for the specified sample, what information the range provides, and why it is important is thoroughly and accurately explained.
4. Acceptable
13.8 points
The range for the specified sample, what information the range provides, and why it is important is accurately explained. Some detail is needed for clarity or support.
3. Approaching
13.2 points
The range for the specified sample, what information the range provides, and why is it important is generally explained. Overall, the explanation is accurate, but additional information or evidence is needed for support.
2. Insufficient
12 points
The explanation of the range for the specified sample, what information the range provides, and why it is important is incomplete or incorrect.
1. Unsatisfactory
0 points
The explanation of the range for the specified sample, what information the range provides, and why it is important is omitted.
collapse Standard Deviation Versus Standard Error of Measurement assessment
Standard Deviation Versus Standard Error of Measurement
18 points
Criteria Description
Differentiate between standard deviation and standard error of measurement. Explanation of how an instructor would use this information.
5. Target
18 points
The difference between standard deviation and standard error of measurement, and how an instructor would use this information is thoroughly and accurately explained.
4. Acceptable
16.56 points
The difference between standard deviation and standard error of measurement, and how an instructor would use this information, is accurately explained. Some detail is needed for clarity or support.
3. Approaching
15.84 points
The difference between standard deviation and standard error of measurement, and how an instructor would use this information, is generally explained. Overall, the explanation is accurate, but additional information or evidence is needed for support.
2. Insufficient
14.4 points
The explanation of the difference between standard deviation and standard error of measurement, and how an instructor would use this information, is incomplete or incorrect.
1. Unsatisfactory
0 points
The explanation of the difference between standard deviation and standard error of measurement, and how an instructor would use this information, is omitted.
collapse Process for Analyzing Individual Items assessment
Process for Analyzing Individual Items
18 points
Criteria Description
Explanation of process for analyzing individual items once an instructor has analyzed basic concepts of measurement. Inclusion of difficulty, discrimination, and distractors in the response.
5. Target
18 points
The process of analyzing individual items once an instructor has analyzed basic concepts of measurement is thoroughly explained. The three Ds (difficulty, discrimination, and distractors) are used in the explanation.
4. Acceptable
16.56 points
The process of analyzing individual items once an instructor has analyzed basic concepts of measurement is adequately explained. The three Ds (difficulty, discrimination, and distractors) are used in the explanation. Some detail is needed for clarity or support.
3. Approaching
15.84 points
The process of analyzing individual items once an instructor has analyzed basic concepts of measurement is summarized. The three Ds, (difficulty, discrimination, and distractors) are generally used in the explanation. Overall, the explanation is accurate, but additional information or evidence is needed for support.
2. Insufficient
14.4 points
The explanation for the process of analyzing individual items once an instructor has analyzed basic concepts of measurement is incomplete or incorrect.
1. Unsatisfactory
0 points
The explanation for the process of analyzing individual items once an instructor has analyzed basic concepts of measurement is omitted.
collapse p-Value assessment
p-Value
19.5 points
Criteria Description
Determine whether it would be best practice to eliminate a test question on the exam with p-value of 0.100. Provide justification for answer.
5. Target
19.5 points
An accurate determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is presented. Strong justification for the answer is provided.
4. Acceptable
17.94 points
An accurate determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is presented. Adequate justification for the answer is provided.
3. Approaching
17.16 points
Overall, accurate determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is presented but more information or rationale is needed to fully justify the answer.
2. Insufficient
15.6 points
A determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is presented but is inaccurate. Justification for the answer is not provided.
1. Unsatisfactory
0 points
A determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is omitted.
collapse Positive and Negative PBI assessment
Positive and Negative PBI
19.5 points
Criteria Description
Explain what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors has a positive PBI. Recommendations for adjusting item.
5. Target
19.5 points
An accurate determination of whether it would be best practice to eliminate a test question on the exam with p-value of 0.100 is presented. Strong justification for the answer is provided. An accurate and thorough explanation for what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors has a positive PBI, is presented. Clear recommendations to the instructor on how to adjust this item are presented.
4. Acceptable
17.94 points
An accurate explanation for what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors has a positive PBI, is presented. Recommendations to the instructor on how to adjust this item are presented. Some detail is needed for clarity.
3. Approaching
17.16 points
An accurate explanation for what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors have a positive PBI, is presented, but more information or rationale is needed to support the explanation. General recommendations to the instructor on how to adjust this item are presented.
2. Insufficient
15.6 points
An explanation for what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors has a positive PBI, is presented but is incomplete.
1. Unsatisfactory
0 points
An explanation for what information the instructor can gather when a question on the exam has a negative PBI for the correct option, and one or more of the distractors has a positive PBI, is presented but is incomplete.
collapse Thesis, Position, or Purpose assessment
Thesis, Position, or Purpose
10.5 points
Criteria Description
Communicates reason for writing and demonstrates awareness of audience.
5. Target
10.5 points
The thesis, position, or purpose is persuasively developed throughout and skillfully directed to a specific audience.
4. Acceptable
9.66 points
The thesis, position, or purpose is clearly communicated throughout and clearly directed to a specific audience.
3. Approaching
9.24 points
The thesis, position, or purpose is adequately developed. An awareness of the appropriate audience is demonstrated.
2. Insufficient
8.4 points
The thesis, position, or purpose is discernable in most aspects but is occasionally weak or unclear. There is limited awareness of the appropriate audience.
1. Unsatisfactory
0 points
purpose is not discernible. No awareness of the appropriate audience is evident.
collapse Development, Structure, and Conclusion assessment
Development, Structure, and Conclusion
12 points
Criteria Description
Advances position or purpose throughout writing; conclusion aligns to and evolves from development.
5. Target
12 points
The thesis, position, or purpose is coherently and cohesively advanced throughout. The progression of ideas is coherent and unified. A convincing and unambiguous conclusion aligns to the development of the purpose.
4. Acceptable
11.04 points
The thesis, position, or purpose is logically advanced throughout. The progression of ideas is coherent and unified. A clear and plausible conclusion aligns to the development of the purpose.
3. Approaching
10.56 points
The thesis, position, or purpose is advanced in most aspects. Ideas clearly build on each other. Conclusion aligns to the development of the purpose.
2. Insufficient
9.6 points
Limited advancement of thesis, position, or purpose is discernable. There are inconsistencies in organization or the relationship of ideas. Conclusion is simplistic and not fully aligned to the development of the purpose.
1. Unsatisfactory
0 points
No advancement of the thesis, position, or purpose is evident. Connections between paragraphs are missing or inappropriate. No conclusion is offered.
collapse Evidence assessment
Evidence
7.5 points
Criteria Description
Selects and integrates evidence to support and advance position/purpose; considers other perspectives.
5. Target
7.5 points
Comprehensive and compelling evidence is included. Multiple other perspectives are integrated effectively.
4. Acceptable
6.9 points
Specific and appropriate evidence is included. Other perspectives are integrated.
3. Approaching
6.6 points
Relevant evidence that includes other perspectives is used.
2. Insufficient
6 points
Evidence is used but is insufficient or of limited relevance. Simplistic explanation or integration of other perspectives is present.
1. Unsatisfactory
0 points
Evidence to support the thesis, position, or purpose is absent. The writing relies entirely on the perspective of the writer.
collapse Mechanics of Writing assessment
Mechanics of Writing
7.5 points
Criteria Description
Includes spelling, capitalization, punctuation, grammar, language use, sentence structure, etc.
5. Target
7.5 points
No mechanical errors are present. Skilled control of language choice and sentence structure are used throughout.
4. Acceptable
6.9 points
Few mechanical errors are present. Suitable language choice and sentence structure are used.
3. Approaching
6.6 points
Occasional mechanical errors are present. Language choice is generally appropriate. Varied sentence structure is attempted.
2. Insufficient
6 points
Frequent and repetitive mechanical errors are present. Inconsistencies in language choice or sentence structure are recurrent.
1. Unsatisfactory
0 points
Errors in grammar or syntax are pervasive and impede meaning. Incorrect language choice or sentence structure errors are found throughout.
collapse Format/Documentation assessment
Format/Documentation
7.5 points
Criteria Description
Uses appropriate style, such as APA, MLA, etc., for college, subject, and level; documents sources using citations, footnotes, references, bibliography, etc., appropriate to assignment and discipline.
5. Target
7.5 points
No errors in formatting or documentation are present. Selectivity in the use of direct quotations and synthesis of sources is demonstrated.
4. Acceptable
6.9 points
Appropriate format and documentation are used with only minor errors.
3. Approaching
6.6 points
Appropriate format and documentation are used, although there are some obvious errors.
2. Insufficient
6 points
Appropriate format is attempted, but some elements are missing. Frequent errors in documentation of sources are evident.
1. Unsatisfactory
0 points
Appropriate format is not used. No documentation of sources is provided.
Sample Test Statistics
Student Full Name
Institutional Affiliation
Course Full Title
Instructor Full Name
Due Date
Sample Test Statistics
The reliability of a test refers to the degree to which test scores are consistent across various occasions of testing, various versions of the test, or various raters evaluating the test taker’s responses. It is the degree to which test scores are not determined or impacted by chance factors. A reliable test is therefore not influenced by the setting in which the test was undertaken, the particular questions that were on the version of the test that the examinees took, or the specific examiners who rated the test takers’ responses (Livingston, 2018). In summary, reliability is consistency and indicates how reliably the test scores assesses something. Based on the information in Table 11.1, the test is relatively reliable given that its reliability coefficient is 0.754: although tests with more than 50 items (the sample test in question has 100 items) are supposed to have reliability coefficients greater than 0.80, the sample’s mean reflects a range of student abilities. The sample shows a normal distribution of test scores given the fact that the mean and median are almost the same.
All measures of central tendency indicate a uniform distribution of test scores thereby demonstrating that the test items were neither too easy or too hard for the group of students. The uniform distribution of test scores could also be an indication that a large number of items on broadly defined content were administered to a large heterogenous group of learners, all of which are marks of a reliable test. The range for this sample is 41 (highest score of 93 - lowest score of 52) (McDonald, 2018). This range is considerably large and indicates that some students did not perform well on the test. The range also provides a rough idea of the variability of the test scores. However, the test sample’s standard deviation indicates that despite the broad range in test scores, the average distance that scores in the distribution varied from the mean was quite low. Although the large range means that test scores were spread out, the more accurate scenario is that of a few students performing very well and another small group of students performing poorly thus the large difference in test scores.
This information is important because it shows how scores were distributed between the highest performing students and the lowest performing students. The test’s large range indicate a high variability even though a considerable number of examinees in the test sample appear to have scored close to the mean than either extremes. On the other hand, the difference between standard deviation and standard error of measurement is that the former denotes the variability within a sample while the latter reflects the variability across samples of population. In other words, the standard deviation is a measure of the degree of variability or dispersion of individual data values from the mean. Conversely, the standard error of measurement is a measure of how much evaluated test scores are distributed around a “true score”: the standard error of measurement measures the extent to which repeated measures of an entity on the same instrument are spread around the entity’s true score. The standard deviation is a useful parameter to the examiner since it indicates the average distance that scores in a distribution vary from the mean (McDonald, 2018). The larger the standard deviation, the bigger the distribution of scores from the mean while a small standard deviation indicates a great similarity in test scores to the mean.
The standard deviation is critical in making inferences about the normal curve. On the other hand, the standard error of measurement indicates the reliability of the test in that the larger the value, the lower the reliability of the test. Low standard error of measurement values indicates to the instructor th...