Events
 
   


LBPP LIA is certifying its teachers and management staff all over Indonesia with TOEIC international Certification

30-10-2009
LBPP LIA, as the chosen organization to receive the ETS TOEIC 30th anniversary grant from ETS, is distributing the gr... >>more


The 5th ETS TOEIC Technical Assistance

26-08-2009
Jakarta, August 18-19, 2009
“Improving Your English Teaching Skills”

International Test Center was hold... >>more


Penandatanganan MOU oleh LBPP LIA dan International Test Center untuk Sertifikasi 1000 Instruktur dan tenaga akademis dengan standar TOEIC

20-08-2009
Pada bulan Agustus 2009, International Test Center sebagai perwakilan resmi Educational Testing Services (ETS) di Ind... >>more


Pengumuman Pemenang Beasiswa ITC

18-08-2009
PT. International Test Center mengucapkan selamat kepada pemenang besiswa dalam International Test Center Scholarship... >>more


WINNING THE GLOBAL COMPETITION WITH COMMUNICATIVE WORKERS

10-06-2009
AN INTERNATIONAL SEMINAR organized by PT. ITC with the title of:
WINNING THE GLOBAL COMPETITION WITH COMMUNICATI... >>more


    
    

  New Products
 
   

TOEIC PREPARATION GUIDE

TOEFL PRACTICE ONLINE

 
   
 Using The Criterion Service in Teaching
 


How can the Criterion service help students?
The Criterion service gives students valuable, risk-free, self-paced writing practice and provides instant holistic scores and annotated diagnostic feedback on their essay. Students are motivated with unlimited opportunities to work either independently or collaboratively with instructors and peers — without waiting for teachers to mark their essay.

How many topics are available?
The Criterion library encompasses grade-school to graduate-level topics. Currently, there are 44 topics for grades 4-8; 29 topics for grades 9-12; 11 topics for the first two years of college; 14 topics for the GRE® test; and 22 topics for the TOEFL® test. Teachers have the flexibility to select a topic from any grade level and use it in another grade level; that topic will now receive a holistic score based on the grade-level standard. Using this Instructor Topic Scored feature, the teacher can assign any topic in the library to their students on grade level. And teachers can use the topics at the designated grade to be scored at that grade level.

Where do the topics come from?
Criterion topics come from a range of sources including ETS testing programs such as the NAEP®, PraxisTM, TOEFL® tests, and the English Placement Test designed for California State University. Grade school level teachers and ETS assessment specialists wrote many of the library topics.

Are instructors limited to the Criterion library of topics?
No. In addition to the library, the Criterion service offers a “write-your-own-topic” feature for every grade level. Teachers may select from two options:

Instructor’s Topic (diagnostic feedback only), and Instructor’s Topic Scored (BOTH diagnostic feedback and holistic score). Instructors can choose either option to create a writing assignment for any subject. The Criterion service provides topic construction guidelines to ease this process. And every level of diagnostic feedback can be reported.

California and Texas users are offered state-specific prompts developed to conform to each state’s writing standards.

Can the Criterion service provide a holistic score along with the diagnostic feedback for topics not in the Library?
Yes. When the instructors create their own topics, the Instructor Topic Scored feature generates either a 4-point or 6-point holistic score. Using a pop-up window, the Criterion service assists instructors in writing prompts for either expository or persuasive modes.The Criterion service explains the requirements and offers a link to step-by-step instructions on how to create a topic that can be scored.

How does the Criterion service handle an unusual writing style?
The Criterion service compares each essay to other essays in the database that have already been scored. It also looks for specific features in syntax, organization, and vocabulary.

If there is anything anomalous, e.g., essays are too brief, the content is repetitive, or the vocabulary is unusual, then the Criterion service posts an advisory or warning stating that it is unable to provide an accurate score. The advisory alerts the instructor to review the flagged essay(s).

Will the use of the Criterion service stifle creative writing among students?
No. The Criterion service is designed to evaluate a different style of writing from creative writing. In a test environment, students tend to write more straightforward essays and they understand that they are not being assessed on their creative writing abilities.

Will the Criterion service catch cheating or plagiarism?
No. The Criterion service simply evaluates the essay. It is the instructor’s responsibility to ascertain that students have submitted their own work.

Instructors can opt to display sample essays on the Create Assignment screen. Students can then view the samples and refer to them while they write their own essays. The sample essays are in a read-only format and cannot be copied, cut, and pasted into another document.

What information does the Criterion service report to teachers?
Teachers have easy and secure access to each student’s portfolio of essays, diagnostic reports, and scores. The student’s first-attempt essay and the most recent saved essay are included in the portfolio. In addition, teachers can view a summary of the entire class’ performance and scores.

What types of diagnostic feedback do students get?
Students receive an annotated version of their essay that points out any errors in grammar, usage, and mechanics and offers comments on style, organization, and development. They can make revisions and resubmit their essay. Each time they submit an essay they’ll receive a diagnostic analysis and a holistic evaluation of their work.

Can instructors limit what the student receives for feedback?
Yes. Instructors can select what, if any, feedback the students receive on each assignment.

What writing genres does the Criterion service cover?
The Criterion library contains assignments in nine genres:

  • Argumentative
  • Cause and Effect
  • Compare and Contrast
  • Descriptive
  • xpository
  • Issue
  • Narrative
  • Persuasive
  • Process
  • Plus, the Criterion state-specific versions include alternative formats, e.g., a response to literature, a summary, or a business letter.

What are the Criterion Writer's Handbooks?
The Criterion Writer’s Handbooks are handy online resource tools. Students can find feedback definitions, examples of correct and incorrect usage, and an explanation of how to interpret the feedback. Students can access the handbook when they are reviewing their diagnostic feedback or portfolios.

There are five version of the Writer’s Handbook:

  • Elementary – Grades 4, 5, 6
  • Middle School – Grades 6, 7, 8, 9
  • Standard version shows “prescriptive” feedback messages
  • Optional version is available for “descriptive” feedback messages
  • High School/College – Grades 9, 10, 11, 12; College 1, 2
  • English Language Learners (ELL) – Grades 7-Adult
  • Spanish/English Bilingual – Grade 6–Adult

Instructors decide which handbook to use for each class and also have the option of assigning a specific handbook to a specific student.

Additional versions of the Criterion Writer’s Handbook may be added in the future.

Is the vocabulary used in the diagnostic feedback messages also age/level appropriate as it is in the Writer's Handbook?
Yes. Feedback messages also correspond to the version of the Writer’s Handbook selected by the teacher. The feedback is displayed in a pop-up window when the student rolls the mouse over the highlighted sections in the essay.

Descriptive feedback messages are structured so that the message does not prescribe what to correct. Instead it describes the error so that the student can figure out the appropriate corrective measure.

Does the Criterion service discriminate against students who may be bright but who may not have mastered standard English, e.g., minorities and ESL students?
No. The Criterion service is blind to race, sex, national origin, and personal history.

Can instructors limit access to assignments?
Yes. Instructors have options when setting up the assignment. For example, they may set restrictions on the number of drafts and revisions that students may submit for an assignment. And instructors can make the assignment available for a certain time period.

Can instructors impose time limits on assignments?
Yes. Many of the assignments in the Criterion library have recommended time limits. Instructors may choose to use the recommendation or give the students unlimited time to complete the writing assignment.

When instructors create their own topics they may designate the amount of time to complete the task.

Can instructors limit their students' access to scores and diagnostic feedback?
Yes. Instructors can choose what, if any, of the diagnostic feedback the students receive on each assignment. They may also opt to show or hide the holistic score.

Instructors can select options by trait area when setting up a new assignment, e.g., ALL trait feedback will be reported to the student, SOME trait feedback will be reported, or NONE will be reported.

   
 Using The Criterion Service for Testing and Placement
 

How often does the computer's score agree with the score of a faculty reader?
Almost always. ETS researchers found either exact or adjacent agreement (within one point) in 98% of the comparisons between the Criterion scores and those of a trained essay reader. Both used the same scoring guidelines and scoring system.

How do institutions use the Criterion service for assessment purposes?
Two ways:
Some schools and institutions use Criterion scores for benchmark testing; Criterion writing prompts are assigned at specific times during the school year or semester.
And some institutions use Criterion scores for exit testing; the Criterion score is combined with the score from a faculty reader. If there is a variance of more than one point, then another faculty reader evaluates the essay.

How do institutions use Criterion scores for placement?
Colleges may assign students to a composition class based on a combination of the Criterion scores and other indicators.

ETS recommends combining the Criterion score with the score from a trained essay reader before placing a student in a remedial class. If there is a variance of more than one point, then another faculty reader should evaluate the essay.

How do students feel about being scored by a machine?
The majority of students are comfortable with automated testing and grades. Plus, the easy-to-use online tools, e.g., the Writers’ Handbook and the essay’s annotated diagnostic feedback messages, aid the student in understanding mistakes and making corrections and revisions.

Many students are motivated to increase their score or decrease the number of errors reported in their essay.

Can the Criterion service score essays on other topics?
Yes. Using the Instructor Topic Scored feature, teachers can create their own topics that are parallel to the Criterion library prompts and the students’ essays will receive holistic scores upon completion.

A Criterion pop-up window explains the requirements and a button link offers step-by-step instructions on how to create either a persuasive or expository topic that can be scored.

 

 Understanding the Technology
 

What kind of technology is used in the Criterion service?
The Criterion service uses a variety of automated scoring tools to provide immediate feedback. They include

  • Natural language processing (NLP), a sub field of artificial intelligence
  • Statistics
  • Computational linguistics
  • Algorithms
However, these tools only process information; they cannot read and analyze it.

How does the Criterion service come up with its scores?
The Criterion service is based on e-rater® scoring technology that was developed by ETS. In order to provide holistic scores on completed essays, e-rater technology uses features that are closely aligned with a human reader’s scoring criteria, e.g.,

  • errors in grammar, usage, and mechanics
  • comments about style
  • organization and development
  • vocabulary usage.

The E-rater scoring engine is used in conjunction with writing analysis tools to comprise the automated essay evaluation scoring engines within the CriterionSM Online Writing Evaluation.

The e-rater scoring engines provide a holistic score for an essay and real-time diagnostic feedback about grammar, usage, mechanics and style, and organization and development.

And linguistic features are identified in essays using various techniques based on Natural Language Processing (NLP), a sub field of computer science that deals with automated analysis of language.

Is the holistic score related to the diagnostic feedback?
Yes. The E-rater® scoring engine was designed to produce a holistic score on either a 4- or 6-point scale. It uses information from a set of writing features that are common to most writing texts and scoring rubrics used to evaluate writing assignments.

Since many of these features are drawn from the diagnostic analysis provided by the Criterion service, there is a clear correlation between the holistic score for a completed essay and the diagnostic feedback provided by the writing analysis tools component of the scoring engine.

What is the technology used in e-rater® scoring?
E-rater scoring is an application of Natural Language Processing (NLP), which uses computational methods to analyze characteristics of text.

E-rater scoring uses NLP to identify the features of the faculty-scored essays in its sample collection and stores them, with their associated weights, in a database. When it evaluates a new essay, e-rater scoring compares the essay’s features to those in the database and then assigns a score.

Since e-rater scoring is not doing any actual reading, the validity of its scoring depends on the scoring of the sample essays from which the e-rater database is created.

How are e-rater® scoring models created?
The E-rater software learns to score by analyzing hundreds of essays (elementary through graduate level) that have been graded by trained faculty readers; the readers use a scoring rubric that establishes the rules for awarding the individual score points. The validity of its scoring depends on the scoring of the sample essays from which the e-rater database is created.

The e-rater v2.0 program is a state-of-the-art automated essay scoring capability. When it evaluates a new essay, e-rater scoring compares the essay’s features to those in the database and then assigns a score.

E-rater scoring automatically can assign a holistic rating on a 4- or 6-point scale in two ways:

  • Prompt-Specific Method. This is a customized scoring technique that is used to score essays for some district-level and state-level tests and the state specific prompts for CA and TX in the Criterion library. The prompts are tailored to a specific standard depending on district or state requirements. Grade-Specific Method. A score can be assigned to an essay based on the grade level associated with the prompt. Most of the prompts in the Criterion library will use grade-specific models.
  • Teachers can select either an essay question from the library, or, using the Instructor Topic Scored feature, they can create their own topic and have the essays scored via the grade-specific model.
 Logistic in Using The Criterion Service
 

Can students trick the Criterion service?
Yes, to an extent. Since e-rater® can’t really read English, potentially, e-rater scoring can be fooled by an illogical, yet well-written, essay. However, if the Criterion service discerns anything anomalous, e.g., essays are too brief, the content is repetitive, or the vocabulary is unusual, then the Criterion service posts an advisory or warning stating that it is unable to provide an accurate score. The advisory alerts the instructor to review the flagged essay(s).

Teachers can thwart students’ attempts to misuse the system by announcing that independent readers will read a random sample of essays.

We encourage teachers to review essays for correctness of content and to use the Instructor Comments and Pop-Up Notes features to communicate with their students about their essay. The Criterion service allows teachers to hide the holistic score when an advisory is present.

What is an e-rater® advisory?
It’s a message to the student and/or teacher stating that e-rater scoring has had difficulty grading some or the entire writing sample.

Currently, e-rater advisories for essays flag brevity, excessive length, repetition, off-topic responses, restatement of the essay topic, or an essay with an overabundance of problems.

Must students be connected to the Internet to use the Criterion service?
Although Criterion is an online service, students can compose their essays offline using any word processing application. They then can cut and paste their work into the essay submission box after they log on to the Criterion site. Receipt of scores and diagnostic feedback for essays require an Internet connection.

What are the minimum technical requirements in order to access the Criterion site?
The Criterion service requires only an Internet connection and a web browser and is both PC and Mac compatible.

For PC users:

    Internet Explorer (IE) 5.5 and 6.0; Netscape 6.1, 7.2; Firefox 1.0.1.
    For PC Linux users: Novell Linux Desktop 9; Linspire 5.0 - browsers Firefox v1.x; Netscape v7.x

For Mac users:

    PowerMac G3/G4/G5, OS 9.2, 10.1, or 10.2, using Internet Explorer (IE) 5.2 or Netscape 7.1 or higher; Firefox 1.0.1; Safari 1.2.4 or 1.3 (compatible only with Mac OS X, not OS 9).

For a complete description of minimum and recommended standards, please ask us our Technical Specifications Sheet (PDF) to inquiry@itc-indonesia.com>

Counter Hits From Begining : 34865 This Year : 11863 This Month : 2009 This Week : 720 Yesterday : 192 Today : 91