At some point during your education, you were probably required to take a course or two on a foreign language. Maybe it was something you enjoyed learning, or something you took just to fulfill some graduation requirements. Or maybe you enjoyed it so much that you took up more classes on it, or even studied abroad for some time. Whatever form your foreign language education may have taken throughout the years, at some point you probably wondered: “How will I ever get really fluent in this foreign language? Am I even focusing my studies on the right thing to become fluent in this language?”

For many foreign language learners, when we start out learning a language we’re confronted with the question: “What’s the most important thing I should be focusing on when learning this language?” Is it my knowledge of vocabulary words? My accent? My speed when trying to hold a conversation? It can be hard to find this answer in the classroom when we’re bombarded with new vocabulary, grammar rules, and different types of accents that exist in your language around the world. On top of that, it’s challenging for students learning in a classroom setting to reach a high level of proficiency in a second or foreign language, most especially in regard to the development of speaking skills. Generally, the end goal of foreign language studies is to hopefully use it one day to communicate with native speakers of that language. And so when I was confronted with this question when I was preparing to study abroad in Spain in 2018, I figured: “Why not ask the native speakers themselves?” Knowing how native speakers evaluate non-native speech could provide helpful information for both language teachers and students.

Background & Research Questions

In this report, we will be exploring these questions particularly in the realm of the Spanish language. For anybody who has encountered Spanish before, perhaps you’ve noticed there are countless types of Spanish accents around the world, given that Spanish is spoken as the official language of 20 countries across the globe, not including Puerto Rico and the United States, which have garnered more Spanish speakers than Spain or Colombia (Source: telelanguage.com). In this discussion, we will be exploring the question of how native Spanish speakers perceive non-native spoken Spanish of American-born university students. In particular, we will be analyzing the perceptions of native speakers from Spain. As we explore how speakers of one distinct accent of Spanish perceives the language as it is spoken by non-native speakers from the United States, we hope that this may shed light on how we should recalibrate the focus of our foreign langugage education in terms of speaking ability.

Previous research on the topic of perceptions of non-native speech by McBride (2015) has looked at how native Mexican speakers of Spanish, native Argentinian speakers of Spanish, and foreign language instructors of Spanish who are native speakers of English have evaluated non-native speech. The study identified the features of non-native speech that are rated to be most impactful on listener evaluations, particularly in regards to comprehensibility and pleasantness of speech. The present study extends this research to a different evaluator population by collecting data in Spain, focusing on how native speakers of Spanish from Spain perceive foreign-accented Spanish. We explore the following questions:

Note: Teacher data collection is ongoing

The study consists of 2 evaluator populations: (1) native Spaniards and (2) American Spanish teachers. To date, we have completed “Phase 1” of this study in which we analyze the perceptions of the first evaluator population, native Spaniards. The study is still underway and is in its “Phase 2,” as we are beginning to collect data on the perceptions of the second evaluator population, Spanish teachers in the United States. We designed this study to collect data from these 2 different populations to offer an interesting comparison between the perceptions of 2 groups and how it may be revealing of the foreign language education system in the United States and offer points for consideration as to how we may be able to improve it with the insights of native Spanish speakers from Spain. Therefore, this discussion and report will only focus on native Spaniard data.

The Native Spaniards

During my semester abroad in Spain, I completed all data collection between September to December 2018. I collected 75 responses from native Spaniards using paper-based surveys in public spaces, such as local parks, libraries, and coffee shops. While the data was primarily collected in Salamanca, Spain, the participants came from all over the country, including the regions of Extremadura, Asturias, Galicia, Castille and Leon, Valencia, Andalusia, and both the Balearic and Canary Islands.

This offers a well-rounded sample of the perceptions of native Spaniards as our participants came from all over the country. The following visuals will provide you with an idea of the evaluator population for your consideration as we explore the results and discussion.

The evaluator population was more female than male, with 48 female and 27 male participants.

The majority of participants were university students and fell within the 18-30 years age group.

We also asked if the participants had any experience teaching, as we were curious if those who had teaching experience would be more lenient and forgiving at the time of evaluating the non-native speech. We did not find conclusive evidence to prove this, but offer this information for your consideration.

From these 2 language graphs, many of the Spaniards identified as bilinguals, and we can see that perhaps most of these participants are Spanish-English bilinguals. This means that many of the Spaniards are knowledgable of English and may perhaps be sensitive to the features of non-native Spanish from native English speakers that are most pronounced.

The speech samples

The speech samples of non-native Spanish were given by 6 students at Montclair State University with varying levels of experience and backgrounds in learning Spanish. We asked them to following questions to get an idea of their experience, and we include their answers in this report to offer context of the speech samples evaluated by the participants:

  • Do you speak Spanish at home?
  • Did you speak Spanish growing up? (By saying “no,” you confirm that you learned Spanish as a second/third language)
  • Have you ever studied study abroad? If so, for how long?
  • Have you ever lived abroad? If so, where and for how long?
  • How many years of Spanish have you taken? (throughout elementary, middle, high school, university)
speech_sample_backgrounds
##   Speaker Do.you.speak.Spanish.at.home. Did.you.speak.Spanish.growing.up.
## 1       A                            No                                No
## 2       B                            No                                No
## 3       C                           Yes                                No
## 4       D                            No                                No
## 5       E                            No                                No
## 6       F                            No                                No
##   Have.you.ever.studied.study.abroad. Have.you.ever.lived.abroad.
## 1                     Spain - a month                          No
## 2                                  No                          No
## 3                      Peru - 10 days                          No
## 4                Argentina - 4 months                          No
## 5                     Spain - a month                          No
## 6                     Spain - a month                          No
##   How.many.years.of.Spanish.have.you.taken.
## 1                                  15 years
## 2                                   7 years
## 3                                  10 years
## 4                                   8 years
## 5                                  14 years
## 6                                   8 years

The speech sample was a short, 20-30 second neutral script about the impacts of modern technologies and social media networks on teenagers and young adults taken from the Spanish newspaper El Mundo. Each speaker in the speech samples were asked to record themselves reading the script as naturally as possible in a quiet space, which lasted no longer than 30 seconds.

The task at hand

Seventy-five Spaniards were asked to evaluate these six speech samples of non-native, American-accented Spanish. The Spaniards gave each speech sample a global rating and also assessed each one in terms of several specific criteria on a scale of 1-5, where 1 referred to “Sounds non-native” and 5 referred to “Sounds native.” The data were analyzed to find overall speaker rankings, overall speaker ranking by criteria, and the criteria that made speakers rank higher or lower in the overall rankings. The criteria for evaluation were:

  1. Speed
  2. Flow
  3. Intonation
  4. Pronunciation
  5. Overall ranking

To clarify what we mean by these criteria, “speed” refers to how fast or slow the speaker spoke. “Flow” refers to how smoothly the speaker spoke, meaning the number of, or lack thereof, of pauses, stutters, and other possible trip-ups in the recording. “Intonation” refers to the rise and fall, or pitch, of the voice when speaking. Lastly, “pronunciation” referred to how native the speaker’s accent sounded. Participants were instructed to evaluate pronunciation, or accent, as the Spanish language is spoken around the world in all Spanish-speaking countries. In other words, the Spaniards were asked not to rate the accent on how “Spanish” it sounded, but rather if they may mistake the speech sample as one of a native speaker of Spanish from any Spanish-speaking country in the world.

Results & Discussion

Using the “Overall Rankings” as the baseline of performance by the non-native speakers in the speech samples, we will use this as a reference point and a type of “ground truth” to draw observations from the interplay of the criteria and its effect on ranking scores. We aim to loo for similar mappings of the criteria to the “baseline”" overall ranking scores.

How do native Spaniards in Spain and evaluate non-native spoken Spanish?

results
##   Rank Speaker Overall.Ranking.Scores Overall.Averages.by.Criteria
## 1    1       A                   4.28                         4.30
## 2    2       D                   3.74                         3.73
## 3    3       E                   3.43                         3.35
## 4    4       C                   3.18                         3.20
## 5    5       F                   2.93                         2.90
## 6    6       B                   2.83                         2.76

All 75 responses for “Overall Ranking” were averaged out for each speaker. It is interesting to draw upon the background information provided by the speakers in the speech samples and see how with longer engagement with a foreign language, most especially in terms of studying abroad, the rank of each speaker increases. Given these results from the evaluations of the native Spaniards, it seems that the evaluator population may be sensitive to how greater experience with the foreign language abroad may help a foreign language learner when it comes to sounding native in speaking abilities.

Looking at the results for “Overage Averages by Criteria,” with all 75 responses each speaker received an average score for how they did in each of the 4 main criteria (speed, flow, intonation, and pronuncation). We took those 4 numbers and averaged them out to analyze them for 2 important reasons:

  1. To show that they supported the overall rankings. This means participants were attentive and accurate in ranking both the criteria and the overall score to their own judgment throughout the experiment and did not, for instance, evaluate the speech sample based on the criteria in one way, and then rank the sample globally in a different way.
  2. When we take each sample’s average for each criteria, we can rank them against each other to see exactly which criteria makes you rank higher or lower. What does this tell us? We’re looking for the criteria that maps most closely to the overall ranking, which will be used as a type of baseline for further comparison. This ultimately reflects which criteria was most highly valued, or important, answering the overarching question: which feature makes a great speaker?

What characteristics of non-native speech do native Spaniards value most highly?

In looking at how the speakers did overall, these are the rankings:

rankings
##           Criteria A B C D E F
## 1            Speed 1 6 5 3 2 4
## 2             Flow 1 6 4 2 3 5
## 3       Intonation 1 5 3 2 4 6
## 4    Pronunciation 1 6 3 2 4 5
## 5 Overall Rankings 1 6 4 2 3 5

There are several interesting observations that we can make from the visuals above. We look primarily at Speakers C, E and F, as they show the most variation in their rankings amongst the other speakers in the speech samples by criteria. Speakers A, B and D show little variation and perform similarly across all criteria, which prevents us from drawing any observations in the interplay between the 4 main criteria and how they map most closely to the baseline “Overall Rankings” score.

  • Speaker E performed well in speed, ranking in second place among the other speakers. However, they did not perform as well in intonation and pronunciation, ranking in fourth place for both criteria. This shows that speed was not important enough to pull up their overall ranking to “first place” among the speakers, but at the same time, intonation and pronuncation were not important enough to pull down their overall ranking. In other words, their overall ranking of “third place” among all the speakers was not directly reflected in their ranking of intonation or pronunciation, in which case they came in fourth place. Rather, we see the most direct mapping of their overall ranking reflected in their performance of flow, in which they also scored in third place.

  • Speaker C performed less favorably in speed, ranking in fifth place out of the speakers for that criteria. However, they performed better in intonation and pronunciation, ranking in third place for both criteria. Again, speed was not important enough to pull down their overall ranking: in other words, while their overall ranking was fourth place, their performance in speed which earned them fifth place did not map to their overall ranking. At the same time, intonation and pronunciation were not regarded highly enough to pull up their overall ranking, as their scores of third place for both criteria are not mapped to their overall ranking of fourth place. Similarly to Speaker E, the criteria that mapped most directly to overall ranking was flow.

  • Speaker F presents similar mixed results in terms of the four main criteria. While they placed fourth for speed, this did not map directly to their overall ranking of fifth place. This means speed, again, was not the most valued characteristic of spoken speech. This indirect mapping is true, too, for intonation, while pronunication maps to their overall ranking.

Again, we are looking at the criteria that maps most closely to the overall ranking. With these observations considered this, flow is the most direct indicator of overall performance. Analyzing the criteria in this way led us to 3 conclusions.

Conclusions

The data show that for the criteria evaluated:

  1. Fluency rankings most closely reflect overall speaker rankings, followed by pronunciation rankings.
  2. The relationship between intonation and velocity rankings and overall rankings are less clear
  3. Talking too fast is better than talking too slow, unless your speed compromises the other categories.

Takeaways

For those in foreign language education, both educators and students alike, we’ve boiled down our results and conclusions into 3 important takeaways to keep in mind when you’re focusing your speaking skills.

  • “Just keep talking!” : don’t worry so much about grammatical accuracy that you neglect fluency
  • Don’t ignore pronunciation : native speakers value a native-like accent
  • Watch your speed : talking fast doesn’t make up for poor pronunciation, but talking too slow loses your listener