How the Writing Sample is Scored
EvaluationReadersScoringReported ScoresFigure 1
This document is designed to familiarize you with scoring procedures for the Medical College Admission Test (MCAT*) Writing Sample. Included is background information on the purpose of the Writing Sample, the types of skills assessed, and the scoring procedures, as well as examples of essays with explanations of their strengths and weaknesses. This guide is intended for use by examinees preparing to take the MCAT* and by medical school admissions committees, health professions advisors, and others who must interpret Writing Sample scores.
Back to Top
Development of the MCAT* Writing Sample
As early as 1973, the Medical College Admission Assessment Program task force recommended that a test of written communication skills be required for medical school applicants. Throughout the 1970s, the public and lay press emphasized the need for physicians to communicate with patients as active participants in the delivery of health care. Early in the 1980s, medical school deans joined in discussion of the need to stress writing and analytical skills in the admission process. They reported that communication and writing skills often were deficient among medical students.
Deans and faculty asked that an assessment of written communication skills be added to the information available about medical school applicants. Prior to adoption of the MCAT* Writing Sample, several medical schools independently instituted writing assessments in which applicants provided writing samples at the time of application or in conjunction with the interview. No universal instrument for assessing the writing and analytical skills of medical school applicants existed, however.
Research and development on an MCAT* essay section was conducted from 1985 through 1990, and the Writing Sample became an integral part of the revised MCAT* in 1991. This section, which requires candidates to develop and present ideas in a cohesive manner, offers medical school admission committees evidence of applicants' writing and analytical skills and provides unique information unavailable from other sections of the MCAT*.
Back to Top
Description of the MCAT* Writing Sample
The MCAT* Writing Sample consists of two 30-minute essays designed to assess skills in the following areas:
1. Developing a central idea,
2. Synthesizing concepts and ideas,
3. Presenting ideas cohesively and logically, and
4. Writing clearly, following accepted practices of grammar, syntax, and punctuation consistent with timed, first-draft composition.
Back to Top
Content of Writing Sample Items
Each Writing Sample item provides a specific topic and requires an expository response. Writing Sample topics are selected from areas of general interest such as business, politics, history, art, education, or ethics. Topics do not pertain to the content of biology, chemistry, or physics; to the medical school application process or reasons for the choice of medicine as a career; to social and cultural issues not in the general experience of college students; or to religious or other emotionally charged issues. Specific prior knowledge about the topic is not necessary to complete the Writing Sample.
Back to Top
Structure of Writing Sample Items
Each MCAT* Writing Sample item consists of a statement that expresses an opinion, discusses a philosophy, or describes a policy. The statement is followed by three writing tasks. The first is to explain or interpret the statement. Because the statement is not intended to be plainly factual or self-evident, it usually cannot be explained in a single sentence. In addressing this task, examinees should explain the meaning of the statement as thoroughly as possible.
The second task requires consideration of a circumstance in which the statement might be contradicted or judged not applicable. Examinees must present a specific example that illustrates a viewpoint opposite to the one presented in the statement and should further explore the statement's meaning.
The third task requires discussion of ways in which the conflict between the initial statement and its opposition (expressed in the second writing task) might be resolved. Here, examinees must reconcile the two viewpoints. In responding to this task, examinees should apply their understanding of the topic to more general problems of principle, choice, judgment, or evaluation raised by the conflict between the opposing viewpoints.
Back to Top
Evaluating Writing Sample Responses
Writing Samples are scored holistically. Whereas some methods for scoring essays assign several scores to a single piece of writing (e.g. separate scores for organization, development, grammar and mechanics, or fluency), holistic scoring regards an essay as a whole without separable aspects. Tbis type of scoring is based on the assumption that the various factors involved in writing are so closely interrelated that an essay should be assigned a single score based on the quality of writing as a whole.
Back to Top
Writing Sample papers are rated by a group of trained readers, many of whom are experienced teachers of writing. Using a scoring guide consisting of score point descriptions similar to those summarized below and sample papers selected to be illustrative of each score, readers are trained to score the essays reliably. Readers who are able to assign scores accurately and reliably are accepted to score actual essays. Readers' performance is closely monitored throughout the scoring process to ensure that scores are assigned fairly and accurately.
Since each Writing Sample consists of two topics, two groups of readers are used for the scorings. Each group is responsible for scoring papers for one topic only.
Back to Top
Scoring the Essays
Each essay is rated in terms of its overall effectiveness. The score is determined by the depth, cohesiveness, and clarity with which the writing tasks are addressed and by the extent to which ideas are developed. Readers are trained to score each essay on the basis of the writing skills displayed and not to judge the writer's personality, attitudes, or beliefs.
Readers are instructed to keep several guidelines in mind as they score. They are not to be unduly influenced by length (although an essay must be long enough that its ideas are developed) or by handwriting. Readers are not to be overly harsh on papers that contain minor errors in grammar, spelling, sentence structure, or punctuation. Because Writing Samples are written under timed conditions, it is assumed that some mistakes will be made. Each essay is judged on its individual merits; readers do not score with any particular score distribution in mind.
Responses that are blank, illegible, written in a language other than English, or that consist entirely of an obvious attempt to ignore the purpose of the Writing Sample (such as a drawing) cannot be scored. If either of the two responses cannot be scored, the reported Writing Sample score is an "X" for "Not Ratable."
All personally identifiable information is removed from essays prior to scoring; essays are identified only by their code numbers. Every essay is scored by two different readers. Each reader scores essays in sets or batches, and the scores are recorded on machine scannable sheets, not on the essays themselves. (Readers record only the score on these sheets; they do not make any notes of an essay's strengths or weaknesses.) When a batch of essays is ready to be circulated to a second reader, a new scannable sheet is provided. Thus, the second reader does not know the score given to an essay by the first reader.
Readers score essays using the six-point scale described below. The final score for each essay is a function of the scores assigned by two readers. If an essay receives scores that are more than one point apart (i.e., a 2 and a 4), the essay is evaluated by a supervisory third reader who determines the final score.
Score Point Descriptions
Summarized below are the typical characteristics of papers receiving each score.
|6||These papers show clarity, depth, and complexity of thought. The treatment of the writing assignment is focused and coherent. Major ideas are substantially developed. A facility with language is evident.|
|5||These essays show clarity of thought, with some depth or complexity. The treatment of the rhetorical assignment is generally focused and coherent. Major ideas are well developed. A strong control of language is evident.|
|4||These essays show clarity of thought and may show evidence of depth or complexity. The treatment of the writing assignment is coherent, with some focus. Major ideas are adequately developed. An adequate control of language is evident.|
|3||These essays may show some problems with clarity or complexity of thought. The treatment of the writing assignment may show problems with integration or coherence. Major ideas may be underdeveloped. There may be numerous errors in mechanics, usage, or sentence structure.|
|2||These essays may show some problems with clarity or complexity of thought. The treatment of the writing assignment may show problems with integration or coherence. Major ideas may be underdeveloped. There may be numerous errors in mechanics, usage, or sentence structure.|
|1||These essays may demonstrate a lack of understanding of the writing assignment. There may be serious problems with organization. Ideas may not be developed. Or, there may be so many errors in mechanics, usage, or sentence structure that the writer's ideas are difficult to follow.|
Back to Top
Reported Writing Sample Scores
Each examinee's Writing Sample is scored by a minimum of four readers. As stated above, two readers score the first essay, and two different readers score the second essay. The final scores given to each of the two responses are summed. This numeric score, which may range from 4 to 24, is then converted to an alphabetic score. Alphabetic scores range from a low of J to a high of T. Only the alphabetic score appears on the MCAT* score reports.
Because a 21-point numeric scale (the 4 to 24 scale) is converted to an 11-point letter scale, there are multiple means by which examinees may obtain a given alphabetic score. They may receive similar numeric scores on both essays or may score slightly higher on one essay than on the other. For example, a score of 20 may be obtained by receiving two fives on the first essay and two fives on the second essay, two fours on the first essay and two sixes on the second essay, etc. Although it is not uncommon for examinees to write two essays of dissimilar quality, the majority attain a given letter score by performing equivalently on both essays. It is very rare for an examinee to score exceptionally well on one essay and very poorly on the other.
Because examinees may attain a given letter score in various ways (i.e., by performing equivalently on both essays or by performing better on one essay than on the other), making inferences about the writing skills associated with a given letter scores is not straightforward. Figure 1 provides some general guidance in interpreting letter scores. This figure shows the continuum of writing skills represented by the J to T score scale and describes the typical features of essays at three places on the scale.
Back to Top
Description of Writing Skills Associated with Writing Sample Alphabetic Scores
J K L M N O P Q R S T
These essays demonstrate
These essays demonstrate
These essays respond to
*This figure portrays the continuum of writing skills represented by Writing Sample alphabetic scores. At the "Below Average" level, Writing Samples assigned scores of J or K consist of two essays which both exhibit nearly all of the characteristics included in the description. As scores move up the scale, essays possess fewer of the "Below Average" characteristics and more characteristics described under the "Average" level. For example, a Writing Sample that receives an M typically consists of two essays which exhibit some "Below Average" characteristics and some "Average" characteristics; at a score of N, even fewer "Below Average" characteristics and more "Average" characteristics would be present in the essays. Likewise, as scores increase beyond P, essays possess more of the "Above Average" characteristics and fewer "Average" characteristics. For Writing Sample scores of S and T, nearly all of the characteristics described in the "Above Average" description are present in both essays.
Back to Top
How are MCAT scores from 2013 through January 2015 different compared to previous years?
With the removal of the Writing Sample from the 2013 and 2014 exam, test takers only received scores for the three multiple-choice sections (Physical Sciences, Verbal Reasoning, and Biological Sciences). In addition, they received a total score computed by taking the sum of the three scored multiple-choice sections. For test takers who took the exam prior to 2013, their total score is a combined multiple-choice score conjoined with the Writing Sample score. For example, the writing score for the exam is converted to an alphabetic scale ranging from J (lowest) to T (highest). e.g., 42T
How are the multiple choice sections of the MCAT exam scored?
Each score that you achieve on the three scored multiple-choice sections is based on the number of questions you answer correctly. This raw score is a reflection of your correct answers only. This means that a wrong answer will be scored exactly the same as an unanswered question; there is no additional penalty for wrong answers. Therefore, even if you are unsure of the correct answer to a question, you should make your best guess. The scores from each of these three sections will be converted to a scale ranging from 1 (lowest) to 15 (high). For example, if your raw score on one of the sections is between 40 and 43, your converted score might be 11. Scores ranging from 44 to 46 might have a converted score of 12, and so forth.
How was the Writing Sample section of the MCAT exam scored (prior to 2013)?
For test takers who took the Writing Sample section prior to 2013, their raw score on the Writing Sample was determined by adding the scores they received on each of the two essays they wrote. Because two different readers rated each essay, the total raw Writing Sample score was the sum of the four scores: two for the first essay and two for the second. This sum on the Writing Sample was converted to an alphabetic scale ranging from J (lowest) to T (highest). Note that an X indicates that one or both of the essays were not able to be scored, because they were either completely off-topic, blank, unintelligible, or written in a language other than English.
The sum can result from different combinations of individual scores. (Individual scores are assigned along a 6-point scale.) For example, a student whose scores were 4 and 5 on the first item and 4 and 4 on the second—a raw score of 17—would have received the same alphabetic score point as a student who scored a 3 and 3 on the first essay and a 5 and 6 on the second.
Why are raw scores converted to scaled scores?
The conversion of raw scores to scaled scores compensates for small variations in difficulty between sets of questions. The exact conversion of raw to scaled scores is not constant; because different sets of questions are used on different exams. The 15-point scale tends to provide a more stable and accurate assessment of a student's abilities. Two students of equal ability would be expected to get the same scaled score, even though there might be a slight difference between the raw scores each student obtained on the test.
Is the exam graded on a curve?
Test takers often ask if earning a high score or higher percentile is easier or harder at different times of the testing year. They ask whether they have a better chance of earning a higher score in April or in August, for example. The question is based on an assumption that the exam is scored on a curve, and that a final score is dependent on how an individual performed in comparison to other test takers from the same test day or same time of year.
While there may be small differences in the MCAT exam you took compared to another examinee, the scoring process accounts for these differences so that an 8 earned on physical sciences on one exam means the same thing as an 8 earned on any other exam. The percentile rank provided on your score report simply indicates the percentages of test takers that received the same score or lower scores than you did.
How you score on the MCAT exam, therefore, is not reflective of the particular exam you took—including the time of day, the test date, or the time of year—since any difference in difficulty level is accounted for when calculating your scale scores (see above for information about scaling).