ETS aims to better reflect student needs in tests, enhance testing experience, executive says

K-campus

Published: 27 Sep. 2023, 16:50

ETS aims to better reflect student needs in tests, enhance testing experience, executive says

Rohit Sharma, ETS's senior vice president of global higher education and workskills, speaks during an interview with the Korea JoongAng Daily. [PARK SANG-MOON]

As ETS keeps improving its tests to reflect real-life learning environments, its tests will help prepare just about any group of test takers for their studies, whether they be students, job seekers or even international students in Korea.

"One of the criticisms that higher education institutions around the world get is that they are not producing graduates that are employable," Rohit Sharma, ETS's senior vice president of global higher education and workskills, said.

"There are things known as durable skills, and many of those things will be reflected in our test types as we move into the future, so it won't be a foreign skill for students when they get admitted into academic institutions."

Working as part of a team and learning communication skills are some of the few examples of durable skills, with ETS aiming to upgrade its speaking and writing portions of tests.

One of the changes that happened for its Test of English as a Foreign Language (Toefl) in July was introducing the writing for an academic discussion section.

Test takers are given a transcript of a question presented by a professor and answers from students. The objective is to write a short response about their opinions on the posed question, learning how to communicate with others in a learning classroom.

Although the Test of Proficiency in Korean (Topik) is considered a priority for international students in Korea, Sharma says having English certifications on top of that is definitely a plus.

"Let's say you are an employer or university admissions officer, and you get two applications," Sharma said. "If everything else is the same but one person has a Toefl or Test of English for International Communication (Toeic) score besides a Topik score, it is fairly simple what they would do."

Sharma sat down with the Korea JoongAng Daily at ETS Korea's office in Yeouido, western Seoul, to talk about possible changes happening to ETS-administered tests and its goals as an institution.

This interview has been edited for length and clarity.

Q. ETS offers its tests in over 200 countries. Among them, how significant is the Korean market?

A. The short answer is that it's very significant. As we look across our product portfolio, Korea is one of the top five markets for ETS and it has been so for the last several years. It's been a top-five market for both in terms of the number of test takers and the feedback we get from higher educational institutions.

There were some changes made to Toefl in July, such as adding the writing for academic discussion section and shortening the overall test time. What was the reason behind the changes?

We listen to our main constituents, the test takers and the institutions that accept our tests. Concerns from the test takers' side was that there is a certain level of anxiousness. Taking a test is a stressful experience, and we wanted to see how we could reduce those stress levels. When we talked to our academic partners, they wanted to make sure that the tests reflect the experience the students will have in the classroom. We wanted to combine those two while also touching on other considerations from academic institutions, which was the need to ensure the test maintains the rigor, validity and reliability we stand behind at ETS through decades of research.

One way to reduce stress for the test takers was having a shorter test. But of course, we wanted to maintain the level of reliability. So we reduced the test length from three hours to just under two hours. Historically, we had some unscored items and we removed all of that.

For our academic discussion writing section, we wanted to reflect that more institutions are having their students learn in an online environment. The section isn't about one-way writing. They need to listen and actually participate in discussions that are academic in nature. It reflects what the students would experience in the classroom. So we took all of those inputs, and as a result, we came up with those upgrades.

Are there any further changes ETS plans to make to its tests?

We are really getting into what I call authentic assessments, which essentially means we want to create a testing experience that replicates real-life experiences. For that, we've been using the term "testless assessments." This interview can be an example of what such an assessment would look like. Based on what you typed, we can give you a score about how well you could listen and comprehend what I said.

Tests like Toeic and Toefl are required for university admissions and job applications in Korea, but international students in Korea are mostly required to submit Topik scores to assess Korean fluency. For them, how helpful is it to have English language certifications?

I would say it depends on the educational institution or employer. If your job or program doesn't entail anything related to the English language, then perhaps a Topik certification is good enough. However, in today's day and age, we live in an increasingly global and interconnected world. It's always good if you have additional skills that help you transcend linguistic barriers and create networks around the world. Even if you think it's not important now, you could suddenly have a job opportunity that requires you to move abroad in four or five years. If you didn't build enough English proficiency, you may think that you should have done it earlier when you had time. While it may not be required, we encourage people to think about a bit more for the long term and more broadly in terms of the skills they want to have in their back pocket.

ETS officially sets a two-year validity period for Toeic and Toefl. However, Korean government institutions adopted a 5-year validity policy starting this year and allow job seekers to apply for jobs at government institutions with scores that were from as long as five years ago. How does ETS feel about this change?

The two-year validity stems from an ethical commitment to ensure the highest standards for our tests. If you look at many other standardized tests, the validity period is the industry standard. We feel that two years is the right mark.

The integrity and reliability of a test score are higher when you are closer to when the test was taken. Several studies show that language skills decline over time if they aren't practiced and kept up to date. After two years, scores become less reliable.

While you can use the score for however long you want, it is intended to be valid for two years, and that is our ethical commitment.

ETS has been using AI to score tests and in other various areas. In what areas does ETS plan on using AI in the future?

One area in which we want to use AI is giving personalized feedback to those who take our mock tests. Using AI, when students take our mock tests, we could go more into detail rather than just telling them how they scored in different sections. For example, in the speaking section, it would be something like, "You were good with pronunciation, but you weren't good in the modulation of your voice." That becomes personalized feedback that test takers can use as a reference in the future.

Next, we want to go even deeper and provide not only personalized feedback but also contextualized feedback. Let's say we know a test taker comes from an Asian background and is of Korean ethnicity. As we collect data on Korean test takers, we can find things test takers of Korean ethnicity do more and take that into context when giving feedback.

Another example we are considering is making changes to the speaking section of the test. Right now, there's a sentence, and we ask you to read it. But if we wanted to take it to the next level, we could show a photograph of any object and ask test takers to describe what they see. But as they describe it, AI could start adding elements to the picture.

BY LEE TAE-HEE [lee.taehee2@joongang.co.kr]