Subtitling preferences

Digital Television for All in PolandIn spring 2010 we carried out eyetracking tests in collaboration with Interdisciplinary Center for Applied Cognitive Studies (ICACS) at the Warsaw School of Social Sciences and Humanities (SWPS). The research was part of the larger EU-funded project known as Digital Television for All (DTV4ALL), aimed at establishing standards of subtitling for the deaf and hard of hearing.


Using eyetracking to research subtitling
Eyetracking tests enabled us to research how people read subtitles simultaneously with watching a film. All the clips we used in the tests were taken from the Polish dubbed version of Shrek with custom-made subtitles tailored to the needs of our research.

When watching subtitled films, people tend to focus on characters’ faces in the image and on content words in subtitles. The moments the eyes focus on particular elements are known as fixations. When reading subtitles, viewers tend to focus (that is fixate) not on every single word, but rather on content words, while their eyes make quick movements, known as saccades.

Below you will find a short clip showing how people watch subtitled films. The green dots are fixations, showing where a viewer looked.

If you cannot play the clip, click here.

What did we test?

We tested nine parameters, with three or two variables per parameter:

  1. Character identification
    • Colours
    • Tags
    • Speaker-dependent placement
  2. Subtitling style
    • Verbatim
    • Standard
    • Edited
  3. Position
    • Bottom
    • Mixed
    • Top
  4. Emotions
    • Description
    • Emoticons
    • No description
  5. Sounds
    • Description
    • Icon
    • No description
  6. Justification
    • Left-aligned
    • Centred
  7. Borders
    • Borders
    • No borders
  8. Box
    • Box
    • No box
  9. Shadows
    • Shadows
    • No shadows


Below you will find some of the results from the research. To fetch a full report on the study, please email us:

1. Character identification [list]

For the colour identification parameter, the following three variables were tested:

(1) Colours

Colours are frequently employed in SDH on Polish TV. The three colours in use on public TV are: yellow, green and blue; therefore, these colours were used in our tests.

(2) Tags

Tags are not usually used in SDH on Polish TV. This variable was the one that viewers were mostly unfamiliar with.

(3) Speaker-dependent placement

Speaker-dependent placement is used in SDH on Polish TV in the case of feature films and TV series. Most of the time it is combined with colours, i.e. colour subtitles are placed either to the left or to the right of the screen, depending on the speaker’s position.

Subtitles with tags earned approval of a significant number of participants, particularly those who are deaf. For them, tags are the preferred option of character identification. Hard of hearing participants, however, preferred speaker-dependent placement as their favourite option.

2. Subtitle style [list]

The following three variables were tested in this parameter:

(1) Verbatim subtitles, which included every single word from the dialogue, even words which usually do not find their way to subtitles, such as repetitions, hesitations and other elements typical of spoken language. These subtitles had shortest display times.

(2) Standard subtitles, which included most of the dialogue apart from a few minor repetitions and elements of spoken language which were not crucial to the plot.

(3) Edited subtitles, where not only many elements of oral discourse disappeared, but utterances were simplified in terms of vocabulary and syntax. These subtitles had longest display times to allow for comfortable reading.

Thanks to eyetracking tests, we managed to establish the mean time spent on reading subtitles and on watching the image in three subtitling styles: edited, standard and verbatim.

When you look at the heat maps below, you will see that the redder the area, the more time was spent looking at this spot in the clip. This means that when watching clips with verbatim subtitles, people – especially the deaf – spend a lot more time reading subtitles than watching the image.

[click here to view in high resolution (125 KB)]

Edited Standard Verbatim
Hard of hearing

4. Emotions [list]

Subtitles for the deaf and hard of hearing in Poland very rarely include information on emotions. Such information is included when it is difficult to understand an utterance based solely on the image, as is sometimes the case with some ironic remarks.

The following variables were tested in this parameter:

(1) Description – descriptions of emotions were written in capital letters and placed in brackets in order to distinguish them from dialogue.

(2) Emoticons – utterances were preceded by an emoticon denoting mood of particular characters, such as :-( for sadness and :-0) for anger.

(3) Nothing – emotions were not described in any way.

Emoticons turned out to be the most severely criticised option in the eyetracking tests. People opposed the use of emoticons in subtitles on the grounds that it hinders the reading and comprehension process. It is also difficult to interpret them – only a few of emoticons can be easily understood, while others are not very intuitive. What is more, it is impossible to have an emoticon for every shade of meaning.

5. Sounds [list]

Information on sounds in Polish television subtitles for hearing-impaired viewers is placed at the bottom of the screen in white capital letters against blue background. On DVDs, such information is usually capitalised, but no colours are used.

The variables tested in this parameter were:

(1) Description – a word or phrase explaining the sound that could be heard, for example PURRING or SHOUTING.

(2) Icons – a picture denoting a barking dog.

(3) Nothing – sound information was not described in subtitles at all.

The vast majority of people taking part in the study, including 100% deaf participants, were in favour of including information on sounds in SDH in the form of verbal description, thus choosing what they were accustomed to.

Summary of the results

For character identification, it seems that colours and speaker-dependent placement can be combined. In certain shots, for instance with off-screen voices coming from characters for whom no colours were allocated, this method can be supplemented with name tags.

Subtitling style is the most controversial parameter in the study. While most people prefer verbatim subtitles, eyetracking tests do not confirm the usefulness of this subtitling style. Therefore, perhaps the best option would be to use standard subtitling, i.e. restrict subtitle editing to the minimum and use it only with fast-paced dialogue, especially in close-ups. What is more, in our research standard captions did not differ significantly from edited captions in terms of favourable eyetracking measures. They gave viewers ample time both to read the text and look at the image (ca. 50%/50%). Viewers’ comprehension of the clip with standard captions was quite high: ca. 70%. What is more, given their linguistic similarity to verbatim captions (the text is not simplified, but only cleared from unnecessary features of spoken language which are usually not fixated on anyway), standard captions do not result in major mismatches between the dialogue and the caption text. They are thus not very likely to cause cognitive dissonance neither for the deaf (who often try to lip-read from the image and compare the result with the text in captions) nor for the hard of hearing viewers (who tend to make comparisons between the dialog and the caption text, drawing on residual hearing).

While top subtitles had best comprehension scores, they were not warmly received by the participants, who stated they prefer bottom subtitles. Since eyetracking tests did not show significant differences between top, mixed and bottom subtitles, it is advisable that bottom subtitles be selected as the most preferred position.

With lowest comprehension scores and longest time to first fixation, subtitles with emoticons – fiercely criticised by participants – are the least likely candidates to be used to describe emotions in films. The winning variable in this parameter seems to be no description of all.

When it comes to ways of describing sounds in SDH, the idea of using icons did not gain much approval from the people taking part in the study. Although the best comprehension and eyetracking scores were achieved by the clip with no description of sound whatsoever, it seems that nevertheless such verbal description needs to be added as it is often necessary for the understanding of a clip and it is also the option most preferred by viewers.

In terms of justification, although left-aligned subtitles had better comprehension scores, the preference test results and time to first fixation scores point to centred subtitles as the best option in this parameter.

Based on the data we have researched in the borders parameter, subtitles without borders drew better results when it comes to comprehension and eyetracking tests. However, it seems that some kind of visual marking needs to be added to white subtitles, especially appearing on a light background, as otherwise they could be illegible.

As for viewers’ preferences regarding the box option, it is subtitles without box that are preferred by a vast majority of people. This is also supported by results of comprehension questions, while eyetracking tests do not provide any conclusive results here.

As far as shadows are concerned, despite the fact that they took slightly longer to be noticed and had slightly longer mean reading time, subtitles with shadows had significantly better comprehension scores.

 Subtitling standards

As part of the Polish DTV4ALL Project, we conducted research on the preferences of hearing impaired Poles with regard to subtitles for the deaf and the hard of hearing. The questionnaire was based on an English template, adapted to Polish conditions.

Data collection proceeded in two ways:

(1) distribution of paper questionnaires, mainly through deaf associations, such as the Polish Association of the Deaf, and deaf schools.

(2) distribution of an online version of the questionnaire, advertised on deaf portals and through social networking.

This helped us reach both people who belong to deaf organisations and live in larger cities as well as those who live in smaller towns and villages and are not necessarily related to deaf organisations. As the two data collection methods differed, we have decided to present the results separately for those two groups of respondents.

Since not all participants provided answers to all the questions, the percentages in tables below refer to the participants who answered particular questions.

Below we present the most important research findings. If you are interested in receiving a full report, please contact us: a[dot]szarkowska[at]

Subtitling or sign language interpreting?

When asked about the preference regarding the choice between sign language interpreting and subtitling, most respondents chose subtitles. It is interesting to note a significant difference between those from the online vs. those from the paper version of the questionnaire: 91% and 65% respectively.

What do you think is
the best way to make audiovisual material accessible?
Paper version Online version
Signer 32% 2%
Subtitles 65% 91%
Other 4% 9%

What types of TV programmes do you watch?

When asked about types of TV programmes they usually watch on TV, most respondents reported to watch films and TV series (over 80% in both groups) and news (ca. 80%). This can be correlated to the availability of SDH on Polish television: priority is given to prime time programmes, which includes the major news programme (Wiadomości) and a film/TV series broadcast immediately afterwards.

Programme type Paper version Online version
News 82% 76%
Talk shows 32% 31%
Films and TV series 82% 89%
Documentaries 40% 44%
Sports 48% 15%

Verbatim or edited?

One of the most important and controversial aspects related to subtitling for the deaf and the hard of hearing is whether subtitles should be verbatim or edited. Most participants in the survey declared they want to have verbatim subtitles.

Verbatim vs. edited Paper version Online version
Literal subtitles that contain absolutely all the information 60% 72%
Not so literal but easier to read 40% 22%

 Recommendations for SDH in Poland

Below we offer recommendations on how accessibility services for the hearing impaired in Poland could progress in the light of our research.

1. Provide more subtitles for the deaf and the hard of hearing on television
Hearing impaired Poles are currently offered ca. 4% of the broadcasting time of two major public TV channels (TVP1 and TVP2), including repeats. This ratio is significantly lower than in other EU Member States. Therefore, it is not surprising that raising the percentage of subtitled programming is one of the highest priorities for hearing impaired Poles.

2. Introduce legislation obliging both public and private major television channels to provide a minimum percentage of its programme with closed subtitling
Poland does not have any road map for accessibility; there are no statutory targets or official future plans to increase the number of programmes with SDH and SLI. It seems that only the introduction of relevant accessibility legislation and public service requirements would make broadcasters, especially private ones, provide programmes with SDH.

3. Improve the system of real-time subtitling in live programmes
The lack of speech recognition software makes it difficult for live programmes (particularly news) to be made accessible. Substantial investments in speech recognition technology and/or other solutions are necessary to improve the quality and quantity of real-time subtitling of live programmes in Poland.

4. Provide verbatim subtitling
As in other countries, Polish viewers with hearing impairments demand verbatim subtitles. They are strongly against any cuts in the dialogue. Many of them stress that subtitle editing deprives them of the chance to experience the true atmosphere of the programme. They also protest against the simplification of vocabulary, stating they need subtitles for language learning. Last but not least, many demand that they should receive exactly the same product as hearing viewers.

5. Give a choice between subtitling and sign language interpreting
While most respondents in the survey stated they prefer subtitling as the best method to make programmes accessible to people with hearing impairments, many stressed it would be beneficial to have a choice between subtitles and signing.

6. Provide sign language interpreting in Polish Sign Language, not Signed Polish
Many of those who would like to see more sign language interpreting on Polish television stress that it is Polish Sign Language that should be used, not Signed Polish. Although PSL is the natural means of communication among members of the Deaf community, it has not yet been recognised as a minority language. Awarding PSL status of a minority language might pave the way for its more widespread use in sign language interpreting on television.

Comments are closed.