project Ahmed

Scandinavian Journal of Psychology, 2009, 50, 395–403 DOI: 10.1111/j.1467-9450.2009.00742.x

Background and Basic Processes

Cognition and hearing aids

THOMAS LUNNER,1,2,3,4 MARY RUDNER3,4 and JERKER RÖNNBERG3,4

1Oticon A/S, Research Centre Eriksholm, Snekkersten, Denmark 2Department of Clinical and Experimental Medicine, Linköping University, Sweden 3Linnaeus Centre HEAD, Swedish Institute for Disability Research, Linköping University, Sweden 4Department of Behavioural Sciences and Learning, Linköping University, Sweden

Lunner, T., Rudner, M. & Rönnberg, J. (2009). Cognition and hearing aids. Scandinavian Journal of Psychology, 50, 395–403.

The perceptual information transmitted from a damaged cochlea to the brain is more poorly specified than information from an intact cochlea and requires more processing in working memory before language content can be decoded. In addition to making sounds audible, current hearing aids include several technologies that are intended to facilitate language understanding for persons with hearing impairment in challenging listening situ- ations. These include directional microphones, noise reduction, and fast-acting amplitude compression systems. However, the processed signal itself may challenge listening to the extent that with specific types of technology, and in certain listening situations, individual differences in cognitive processing resources may determine listening success. Here, current and developing digital hearing aid signal processing schemes are reviewed in the light of individual working memory (WM) differences. It is argued that signal processing designed to improve speech understanding may have both positive and negative consequences, and that these may depend on individual WM capacity.

Key words: Working memory, hearing aids, signal processing, cognition, noise reduction.

Thomas Lunner, PhD, Oticon A/S Research Centre Eriksholm, Kongevejen 243, DK-3070 Snekkersten, Denmark. Tel: +45 48 29 89 18; fax: +45 49 22 36 29; e-mail: [email protected]

INTRODUCTION

Advances in hearing aid technology are of great potential

benefit to persons with hearing impairment. It is estimated that

approximately 15% of the western population have a hearing

impairment of such an extent that they would benefit from

amplified hearing by way of hearing aids. Modern hearing aids

incorporate technologies such as multiple-band wide dynamic

range compression, directional microphones, and noise

reduction. Individual settings for most of these functions are

primarily based on pure-tone thresholds. Therefore, persons

with hearing impairment with the same audiogram will receive

similar hearing aid fitting even though they may have different

supra-threshold auditory abilities relating to different patholo-

gies or individual cognitive abilities. The research community

has acknowledged that successful (re)habilitation of persons

with hearing impairment must be individualized and based on

understanding of underlying mechanisms, especially the mech-

anisms of cochlear damage and language understanding. This

paper is based on recent data suggesting that ease of language

understanding is highly dependent on the individual’s working

memory (WM) capacity in challenging speech understanding

conditions, and focuses especially on a discussion of how

different types of signal processing concepts in hearing aids

may support or challenge the storage and processing functions

of WM.

The main issues delineated in this paper concern the

trade-off between individual WM capacity – seen from an

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA

intra-individual as well as an inter-individual perspective – and

the benefits and costs involved in more advanced signal pro-

cessing, as well as factors that modulate the signal-cognition

interaction. The paper ends with a rather radical suggestion of

a concept that takes the individual storage and processing

function of WM into account to steer the function of the signal

processing in the hearing aid.

WORKING MEMORY AND INDIVIDUAL DIFFERENCES

This section is largely inspired by Pichora-Fuller (2007), and

sets up the framework of WM differences under listening con-

ditions that challenge cognitive capacity in different ways.

When listening becomes difficult, for example because of irrel-

evant sound sources interfering with the target signal or

because of a poorly specified input signal due to hearing

impairment, listening must rely more on prior knowledge and

context than would be the case when the incoming signal is

clear and undistorted. This shift from mostly bottom-up (sig-

nal-based) to mostly top-down (knowledge-based) processing

is accompanied by a sense of listening being more effortful.

In a review of different models of WM, Miyake and Shah

(1999) concluded that many fitted the following generic

description: working memory is those mechanisms or pro-

cesses that are involved in the control, regulation, and active

maintenance of task-relevant information in the service of

complex cognition, including novel as well as familiar, skilled

tasks.

logical Associations. Published by Blackwell Publishing Ltd., 9600 02148, USA. ISSN 0036-5564.

396 T. Lunner et al. Scand J Psychol 50 (2009)

The WM model for Ease of Language Understanding (ELU,

Rönnberg 2003; Rönnberg, Rudner, Foo & Lunner, 2008) pro-

poses that under favorable listening conditions, language input

can be rapidly and implicitly matched to stored phonological

representations in long-term memory, whereas under subopti-

mum conditions, it is more likely that this matching process

may fail. In such a mismatch situation, the model predicts that

explicit, or conscious, cognitive processes must be engaged to

decode the speech signal. Thus, under taxing conditions, lan-

guage understanding may be a function of explicit cognitive

capacity; whereas under less taxing conditions it may not.

WM has been proposed to consist of a number of different

components including processing buffers (Baddeley, 1986,

2000), and individual differences in WM function (e.g. Engle,

Cantor & Carullo, 1992) could relate to any of them. Indeed,

researchers have investigated a variety of properties that

contribute to individual differences in WM (e.g., resource

allocation, Just & Carpenter, 1992; buffer size, Cowan, 2001;

Wilken & Ma, 2004; processing capacity, Feldman Barrett,

Tugade & Engle, 2004; Halford, Wilson & Phillips, 1998). In

the following discussion it is assumed that, within the capacity

constraint, resources can be allocated to either processing or

storage, or both. A simple additive model is assumed;

C ¼ P þS ð1Þ

where C is the available individual WM capacity, P is the pro-

cessing component of WM, and S is the storage component of

WM. This is schematically illustrated in Fig. 1b, where the

black bars illustrate the processing (P) component, and the

grey bars illustrate the storage (S) component. For a given C,

the additive relationship defines how much of either P or S

that will be left if the other is used. If the processing and stor-

age demands of a particular task exceed available capacity this

may result in task errors, loss of information from temporary

storage (temporal decay of memories, forgetting) or slower

processing.

(a)

(b)

Fig. 1. Schematic representations of inter-individual differences in working memory capacity (a) suggesting that two individuals may differ in their working memory capacity, and (b) intra-individual differences suggesting that for a given individual the allocation of the person’s limited capacity to the processing and storage functions of working memory varies with task demands. (Adapted from Pichora- Fuller, 2007).

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

For any given individual, the greater the demands made on

the processing function of WM, the fewer resources can be

allocated to its storage function. For example, distorting the

signal or reducing the signal-to-noise ratio (SNR) or the avail-

ability of supportive contextual cues (e.g., Pichora-Fuller,

Schneider & Daneman, 1995) would all increase processing

demands with possible consequent reduction of available stor-

age capacity. Thus, recall of words or sentences is better when

target speech can be clearly heard (Rabbitt, 1968; Tun &

Wingfield 1999; Wingfield & Tun 2001; Pichora-Fuller et al.,

1995).

Complex WM tasks require simultaneous storage (maintain-

ing information in an active state for later recall) and process-

ing (manipulating information for a current computation;

Daneman & Carpenter, 1980). In the reading span task, a WM

task based on sentence processing, the participant reads a sen-

tence and completes a task that requires trying to understand

the whole sentence (by reading it aloud, repeating it, or judg-

ing it for some property such as whether the sentence make

sense or not). Following the presentation of a set of sentences,

the respondent is asked to recall the target word (such as the

first or last word in the sentence) of each sentence in the set.

The number of sentences in the recall set is increased and

recall errors noted as a function of number of sentences in the

set (WM span, WMS). The span score typically reflects the

maximum number of target words that are correctly recalled.

Span size is significantly correlated with language comprehen-

sion (Daneman & Carpenter, 1980; Daneman & Merikle,

1996). WM span measured in this way can also vary within

individuals as a function of task (Fig. 1b).

WORKING MEMORY AND HEARING LOSS

Speech recognition performance is affected for people with

hearing impairment even under relatively favorable external

SNR conditions (e.g., Larsby, Hällgren, Lyxell & Arlinger,

2005; McCoy, Tun, Cox, Colangelo, Stewart & Wingfield,

2005; Plomp, 1988; van Boxtel, van Beijsterveldt, Houx,

Anteunis, Metsemakers & Jolles, 2000). For persons with

hearing loss, perceived listening effort (as assessed by ratings

of subjective effort in different situations), may indicate the

degree to which limited WM resources are allocated to per-

ceptual processing (Rudner, Lunner, Behrens, Sundewall

Thorén & Rönnberg, 2009). Higher levels of perceived effort

may indicate fewer resources for information storage, suggest-

ing that listeners who are hard of hearing would be poorer

than listeners with normal hearing on complex auditory tasks

involving storage. Indeed, results by Rabbitt (1990) suggest

that listeners who are hard of hearing allocate more informa-

tion processing resources to the task of initially perceiving

the speech input, leaving fewer resources for subsequent

recall.

Figure 2 shows results from an experiment by Lunner

(2003) with 72 patients who had similar levels of hearing loss

as indicated by pure-tone audiograms. The participants’

logical Associations.

Correlation: r = –0.61

0 10 20 30 40 50 60 70 80 90 100 Reading Span (% correct)

–8

–6

–4

–2

0

2

4

6

8

S R

T in

n o

is e

( d

B )

Fig. 2. Scatterplot and regression line showing correlation between reading span and speech recognition in noise (n = 72). Pearson correlations with 95% confidence limits for the correlation coefficient are shown. Low (negative) SRT means high performance in noise. (Replotted from Lunner, 2003).

Scand J Psychol 50 (2009) Cognition and hearing aids 397

hearing aids were adjusted to assure audibility of the target

signal and their speech reception thresholds (SRT) in noise

were determined. SRT was defined as the level at which 50%

of words presented were correctly recalled. Individual WM

capacity, as measured by the reading span test (Andersson,

Lyxell, Rönnberg & Spens, 2001; Daneman & Carpenter,

1980; Rönnberg, 1990), accounted for 40% of the inter-indi-

vidual variance. That is, WMS was a good predictor of SRT.

These findings have been confirmed in subsequent studies

(Akeroyd, 2008; Foo, Rudner, Rönnberg & Lunner, 2007;

Rudner, Foo, Sundewall-Thorén, Lunner & Rönnberg, 2008).

HEARING AID SIGNAL PROCESSING AND INDIVIDUAL

WM DIFFERENCES

Below, we review evidence indicating that some types of hear-

ing aid signal processing may release WM resources, resulting

in better storage capacity and faster information processing in

challenging listening situations. However, hearing aid signal

processing may also challenge listening by generating

unwanted processing artifacts by distorting the auditory scene

(i.e. the distinct auditory objects builds up the listening envi-

ronment, see e.g. Shinn-Cunningham, 2008), or generating

audible artifacts and other unintended side-effects such as dis-

tortions of the target signal waveform (Stone & Moore, 2004;

2008), thereby taxing WM resources. The trade-off between

WM benefits and signal processing artifacts may depend on

the individually available cognitive resources, and therefore

individual differences in cognitive processing resources may

determine listening success with specific types of technology.

Inter-individual differences in capacity limitations constrain-

ing WM processing and storage may explain why one listening

situation may be too challenging for one individual but not for

another. Increases in WM span post hearing-aid intervention

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

(i.e. intra-individual improvements in WM storage) would sug-

gest that the intervention has resulted in listening becoming

easier with fewer WM processing resources needing to be

allocated.

Signal processing in hearing aids is designed to help users

specifically in challenging listening situations. Usually the

objective is, by some means, to remove signals that are less

important in a particular situation and/or to emphasize or

enhance signals that are more important. However, the conse-

quences for the individual in terms of communicative benefit

may depend on individual WM capacity. Several studies indi-

cate that pure tone hearing threshold elevation is the primary

determinant of speech recognition performance in quiet back-

ground conditions, e.g. in a conversation with one person or

listening to the television under otherwise undisturbed condi-

tions (see e.g., Dubno, Dirks & Morgan, 1984; Magnusson,

Karlsson & Leijon, 2001; Schum, Matthews & Lee, 1991).

Thus, in less challenging situations, individual differences in

WM are possibly of secondary importance for successful lis-

tening. The individual peripheral hearing loss is the main con-

straint on performance, and the most important objective for

the hearing aid signal processing is to make sounds audible.

This can be by means of slow-acting compression (e.g. Dillon,

1996; Lunner, Hellgren, Arlinger & Elberling, 1997). A slow-

acting compression system maintains near constant gain-fre-

quency response in a given speech/noise listening situation,

and thus preserves the differences between short-term spectra

in the speech signal. In less challenging listening situations,

greater WM capacity confers relatively little benefit and the

same is true of advanced signal processing designed to

enhance target speech and/or to reduce interfering noise. In

more challenging situations, however, signal processing

designed to enhance speech and/or to reduce noise may – or

may not – benefit the hearing aid user, depending on the

implementation.

Even though speech recognition performance may not

always be improved by the hearing aid signal processing,

reductions in subjectively rated listening effort may result (e.g.

Schulte, Vormann, Wagener, Büchler, Dillier, Dreschler,

Eneman et al., 2009). SRT in noise is typically negative (see

e.g. Fig. 2). Speech-to-noise ratios of 5dB or higher are

realistic values for real-life conversation situations, such as

conversing inside or outside urban homes (Pearsons, Bennett

& Fidell, 1977). In such listening situations, conventional SRT

tests are insensitive to signal processing improvements, and

other measures such as subjective rating of listening effort

(Rudner et al., 2009; Schulte et al., 2009) or increases in WM

span-scores post hearing-aid intervention may be a better

predictor of hearing aid signal processing effects.

Directional microphones

Modern hearing aids can usually be switched between omni-

directional and directional microphones. Directional micro-

phone systems are designed to take advantage of the spatial

logical Associations.

398 T. Lunner et al. Scand J Psychol 50 (2009)

differences between the relevant signal and noise. Directional

microphones are more sensitive to sounds coming from the

front than sounds coming from the back and the sides. The

assumption is that because people usually turn their heads to

face a conversational partner, frontal signals are most impor-

tant, while sounds from other directions are of less impor-

tance. Several algorithms have been developed to provide

maximum attenuation of moving or fixed noise source(s)

behind the listener (see e.g. van den Bogaert, Doclo, Wouters

& Moonen, 2008). Usually, switching between directional

microphone and omni-directional microphone takes place auto-

matically in situations that are determined by the SNR-estima-

tion algorithm to be beneficial for the particular type of

microphone. The directional microphone usually comes into

play when estimated SNR is below a given threshold value,

and the target signal is estimated to be coming from the frontal

position.

A review by Ricketts (2005) addressed the benefit of direc-

tional microphones compared to omni-directional, showing that

with the directional microphone, SNR improvement could be

as high as 6–7 dB, and was typically 3–4 dB, in certain noisy

environments. The noisy environments where directional bene-

fit was seen were characterized by (a) no more than moderate

reverberation, (b) the listener facing the sound source of inter-

est, and (c) the distance to this source being rather short. The

SRT in noise shows improvements in accordance with the

SNR improvements (Ricketts, 2005). Thus, at least in particular

situations, directional microphones give a clear and documented

benefit.

However, if the target is not in front or if there are multiple

targets, the attenuation of sources from directions other than

frontal by directional microphones may interfere with the audi-

tory scene (Shinn-Cunningham, 2008; Shinn-Cunningham &

Best, 2008). In natural communication, the listener often

switches attention to different locations. Therefore, omni-direc-

tional microphones may be preferred in situations requiring

frequent shifts of attention or monitoring of sounds at multiple

locations. Unexpected or unmotivated automatic switches

between directional and omni-directional microphones may

be cognitively disturbing if the switching interferes with the

listening situation (Shinn-Cunningham & Best, 2008). Van den

Bogaert et al. (2008) have shown that directional microphone

algorithms substantially interfere with localization of target and

noise sources, suggesting that directional microphones may, in

addition to attenuating lateral sources, distort natural monitor-

ing of sounds at multiple locations.

Sarampalis et al. (2009) investigated WM performance

under different SNRs, ranging from )2 dB to +2 dB, to simu- late the improvement in SNR by directional microphones com-

pared to omni-directional microphones. The WM test was a

dual-task paradigm with (a) a primary perceptual task involv-

ing repeating the last word of sentences presented over head-

phones, and (b) a secondary memory task involving recalling

these words after each set of eight sentences (Pichora-Fuller

et al., 1995). The sentences were high- and low-context

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

sentences from the Revised Speech Perception in Noise Test (Bil-

ger, Nuetzel, Rabinowitz & Rzeczkowski, 1984). Performance on

the secondary (memory) task improved significantly in the +2dB

SNR condition which simulated directional microphones. The

directional microphone intervention may have freed some

WM resources, increasing storage capacity in the (tested) noisy

situations.

Inter-individual and intra-individual differences in WM

capacity may also play a role in determining the benefit of

directional microphones for a given individual in a given

situation. Consider, for example, Fig. 2, in a situation with

0 dB SNR (dash-dotted line). If we assume that the individual

SRT in noise reflects the SNR at which WM capacity is

severely challenged, Fig. 2 indicates that the WM capacity limit

is challenged at about )5 dB for a high WM capacity person. At 0 dB SNR, the person with high WM capacity probably

possesses the WM capacity to use the omni-directional micro-

phone, while at )5 dB this person may need to sacrifice the omni-directional benefits and use the directional microphone to

release WM resources. However, for the person with low WM

capacity, even the 0 dB situation probably challenges WM

capacity limits. Therefore, this person is probably best helped

by selecting the directional microphone at 0 dB to release WM

resources, thereby sacrificing the omni-directional benefits.

Thus, it may be the case that the choice of SNR at which the

directional microphone is invoked should be a trade-off

between omni-directional and directional benefits and individ-

ual WM capacity, and that inter-individual differences in WM

performance may be used to individually set the SNR thresh-

old at which the hearing aid automatically shifts from omni-

directional to directional microphone.

Noise reduction systems

Noise reduction systems, or more specifically, single micro-

phone noise reduction systems, are designed to separate target

speech from disturbing noise by using a separation algorithm

operating on the input. Different amplification is applied to the

separated estimates of speech and noise, thereby enhancing the

speech and/or attenuating the noise (e.g. Bentler & Chiou,

2006; Chung, 2004).

There are several approaches to obtaining separate estimates

of speech and noise signals. One approach applied in current

hearing aids is to use the modulation index (or modulation

depth) as a basis for the estimation. The idea is that speech

includes more level modulations than noise (see e.g., Plomp,

1994) and thus that the higher the modulation index the greater

the likelihood that a target signal has been identified. Algo-

rithms to calculate the modulation index usually operate in

several frequency bands. If a frequency band has a high modu-

lation index, it is classified as including speech and is given

more amplification, while frequency bands with less modula-

tion are classified as noise and thus attenuated (see e.g., Hol-

ube, Hamacher & Wesselkamp, 1999). Other noise reduction

approaches include the use of the level-distribution function

logical Associations.

Scand J Psychol 50 (2009) Cognition and hearing aids 399

for speech (Ludvigsen, 1997) or voice-activity detection by

synchrony detection (Schum, 2003). However, the relative esti-

mation of speech and noise components on a short-term basis

(milliseconds) is very difficult, and misclassifications may

occur. Therefore, commercial noise reduction systems in hear-

ing aids are typically very conservative in their estimation of

speech and noise components, and only give a rather long-term

estimation (seconds) of noise or speech. Such systems do not

seem to aid speech recognition in noise (Bentler & Chiou,

2006). Nevertheless, typical commercial noise reduction

systems do give a reduction in overall loudness of the noise

compared to the target signal, which is rated as improving

comfort (Schum, 2003) and thus may reduce the annoyance

and fatigue associated with using hearing aids.

Noise reduction systems with more aggressive forms of sig-

nal processing are described in the literature, including ‘‘spec-

tral subtraction’’ or weighting algorithms where the noise is

estimated either in brief pauses of the target signal or by mod-

eling the statistical properties of speech and noise (e.g.

Ephraim & Malah, 1984; Lotter & Vary 2003; Martin, 2001;

Martin & Breithaupt, 2003; for a review see Hamacher, Chal-

upper, Eggers et al., 2005). The estimates of speech and noise

are subtracted or weighted on a short-term basis in a number

of frequency bands, which gives a less noisy signal. However,

this comes at the cost of another type of distortion usually

called ‘‘musical noise’’ (Takeshi, Takahiro, Yoshihisa & Tets-

uya, 2003). This extraneous signal may increase cognitive load

during listening since it is a competing, and probably distract-

ing signal, the suppression of which may consume WM

resources. Thus, in optimizing noise reduction systems there is

a trade-off between the amount of noise-reduction and the

amount of distortion.

Sarampalis et al. (2006, 2008, 2009) investigated the WM

capacity of listeners with normal hearing and listeners with

mild to moderate sensorineural hearing loss, using the dual-

task paradigm described earlier. Auditory stimuli were pre-

sented with or without a short-term noise reduction scheme

based on the algorithm proposed by Ephraim & Malah (1984).

For people with normal hearing there was some recall

improvement with noise reduction in low-context sentences.

The authors interpreted this as demonstrating that the algorithm

mitigated some of the deleterious effects of noise by reducing

cognitive effort. However, the results for the listeners with

hearing impairment were not easily interpreted. More research

is needed with regard to individual WM differences and short-

term noise reduction systems to determine the circumstances

under which these systems may release WM resources.

Another recent approach to the separation of speech from

speech-in-noise is the use of binary time-frequency masks (e.g.

Wang, 2005; Wang, 2008; Wang, Kjems, Pedersen, Boldt &

Lunner, 2009). The aim of this approach is to create a binary

time-frequency pattern from the speech/noise mixture. Each

local time-frequency unit is assigned to either a 1 or a 0

depending on the local SNR. If the local SNR is favorable for

the speech signal this unit is assigned a 1, otherwise it is

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

assigned a 0. This binary mask is then applied directly to the

original speech/noise mixture, thereby attenuating the noise

segments. A challenge for this approach is to find the correct

estimate of the local SNR.

Ideal binary masks (IBM) have been used to investigate the

potential of this technique for hearing impaired test subjects

(Anzalone, Calandruccio, Doherty & Carney, 2006; Wang,

2008; Wang et al., 2009). In IBM-processing, the local SNR is

known beforehand, which is not the case in a realistic situation

with non-ideal detectors of speech and noise signals. Thus,

IBM is not directly applicable in hearing aids. Wang et al.

(2009) evaluated the effects of IBM processing on speech

intelligibility for listeners with hearing impairment by assess-

ing the SRT in noise. For a cafeteria background, the authors

observed a 15.6 dB SRT reduction (improvement) for listeners

with hearing impairment, which is a very large effect. If indi-

vidual SRTs reflect the situation where the WM capacity is

severely challenged, applying IBM processing in difficult lis-

tening situations would release WM resources. However, IBM

may produce distortions that increase the cognitive load,

especially in realistic binary mask applications where the

speech and noise are not available separately, but have to be

estimated. Thus, a trade-off may have to be made between

noise reduction and distortion in a realistic noise reduction

system.

In situations where the listener’s cognitive system is unchal-

lenged, using a noise reduction system may be redundant or

even counterproductive, since distortion of the signal could

outweigh any possible gain in SNR. However, since realistic

short-term noise reduction schemes (including realistic binary

mask processing) will rely on a trade-off between amount of

noise reduction and minimization of processing distortions, the

use of such systems may be dependent on the inter-individual

WM differences, suggesting that persons with high WM capac-

ity may tolerate more distortions and thus more aggressive

noise reduction than persons with low WM capacity in a given

listening situation.

Fast acting wide dynamic range compression

A fast-acting wide dynamic range compression (WDRC) sys-

tem is usually called fast compression or syllabic compression

if it adapts rapidly enough to provide different gain-frequency

responses for adjacent speech sounds with different short-term

spectra. This contrasts with slow-acting WDRC (slow com-

pression or automatic gain control). These systems maintain

near constant gain-frequency response in a given speech/noise

listening situation, and thus preserve the differences between

short-term spectra in the speech signal. Hearing-aid compres-

sors usually have frequency-dependent compression ratios,

because hearing loss generally varies with frequency. However,

WDRC can be configured in many ways, with different goals

in mind (Dillon, 1996; Moore, 1998). In general, compression

may be applied in hearing aids for at least three different rea-

sons (e.g., Leijon & Stadler, 2008):

logical Associations.

400 T. Lunner et al. Scand J Psychol 50 (2009)

(1) To present speech at a comfortable loudness level, com-

pensating for variations in voice characteristics and speaker

distance.

(2) To protect the listener from transient sounds that would be

uncomfortably loud if amplified with the gain-frequency

response needed for conversational speech.

(3) To improve speech understanding by making also very

weak speech segments audible, while still presenting lou-

der speech segments at a comfortable level.

A fast compressor can to some extent meet all three purposes,

whereas a slow compressor alone can fulfill only the first

objective. Fast compression may have two opposing effects

with regard to speech recognition: (a) it can provide additional

amplification for weak speech components that might other-

wise be inaudible, and (b) it reduces spectral contrast between

speech sounds. It has yet to be fully investigated which of

these effects has the greatest impact on speech recognition in

noise for the individual, with regard to individual WM capac-

ity. The first studies that systematically investigated individual

differences in coping with the speed of compression were

those of Gatehouse, Naylor and Elberling (2003, 2006a,

2006b). These studies indicated that both cognitive capacity

and auditory ecology have explanatory value as regards indi-

vidual outcome of, for example, speech recognition in noise

and subjectively assessed listening comfort. In a study that rep-

licated some of the findings of the Gatehouse et al studies

(Fig. 3; Lunner & Sundewall-Thorén, 2007), the cognitive test

scores of listeners with hearing loss were significantly corre-

lated with the differential advantage of fast compression versus

slow compression in conditions of modulated noise. Other

studies have shown that cognitive performance is related to the

Correlation: r = 0.49

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 Cognitive performance score (d')

–10

–8

–6

–4

–2

0

2

4

6

8

F a s t

m in

u s S

lo w

b e n

e fi

t (d

B )

Fig. 3. Scatter plot and regression line showing the Pearson correlation between the cognitive performance score and differential benefit in speech recognition in modulated noise of fast versus slow compression. A positive value on the Fast minus Slow benefit (dB) axis means that fast compression obtained better SRT in noise compared to slow compression. (Replotted from Lunner & Sundewall- Thorén, 2007).

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

ability to cope with new compression settings (Foo et al.,

2007, Rudner et al., 2008).

Figure 3 shows a scatterplot and regression line showing the

Pearson correlation between the cognitive performance score

and differential benefit of fast versus slow compression in

speech recognition in modulated noise. The correlation in

Fig. 3 is plausibly explained as an interaction between cogni-

tive performance and fast compression as suggested by Naylor

and Johannesson (2009). These authors have shown that the

long-term SNR at the output of an amplification system that

includes amplitude compression may be higher or lower than

the long-term SNR at the input, dependent on interactions

between the actual long-term input SNR, the modulation char-

acteristics of the signal and noise being mixed, and the ampli-

tude compression characteristics of the system under test.

Specifically, fast compression in modulated noise may increase

output SNR at negative input SNRs, and decrease output SNR

at positive input SNRs. Such shifts in SNR between input and

output values may potentially affect perceptual performance

for users of compression hearing aids. The compression-related

SNR shift affects perceptual performance in the corresponding

direction (G. Naylor, R.B. Johannessen & F.M. Rønne, per-

sonal communication, December 2008); a person performing at

negative SNRs may therefore be able to understand speech

better with fast compression while the same may not be true

for a person performing at positive SNRs. Thus, the relative

SNR at which listening takes place is another factor which

determines if fast compression is beneficial or not. A person

with high WM capacity and SRT at a negative SNR would

probably benefit from fast compression in that particular situa-

tion, while a person with low WM capacity and SRT at a posi-

tive SNR might be put at a disadvantage.

COGNITION-DRIVEN SIGNAL PROCESSING

From the examples above it seems that inter-individual and

intra-individual WM differences should be taken into account

in the development of hearing-aid signal-processing algorithms

and when they are adjusted for the individual hearing-aid user.

Often it will be a case of balancing the trade-off between

opposing effects in relation to the individual’s WM capacity.

For directional microphones the trade-off is between omni-

directional and directional benefits; for realistic short-term

noise reduction schemes it is between amount of noise reduc-

tion and processing distortion and for fast-acting versus slow

compression it is between absolute performance levels in SNR

and the choice to invoke fast compression to improve output

SNR.

In less challenging situations, individual differences in WM

are possibly of secondary importance for successful listening.

The individual peripheral hearing loss is the main constraint

on performance, and the most important objective for the

design of hearing aid signal processing is to make

sounds audible (e.g. by slow acting compression, e.g. Dillon,

1996).

logical Associations.

Scand J Psychol 50 (2009) Cognition and hearing aids 401

In more challenging listening situations, hearing aid signal

processing systems such as directional microphones, noise

reduction systems, and fast compression should be activated

on an individual basis to release WM resources, taking into

account the above mentioned trade-offs between signal pro-

cessing benefits and drawbacks. Below, a new and rather radi-

cal concept is suggested where knowledge of individual WM

resources is combined with knowledge on hearing aids signal

processing concepts.

One way to conceptualize the hearing aid processing

requirements would be as a ‘‘hearing aid with cognition-driven

signal-processing’’, where the hearing aid signal processing is

designed to take individual cognitive capacity into account to

optimize speech understanding. The construction of such a

cognition-driven hearing aid requires monitoring of the indi-

vidual ‘‘cognitive workload’’ on a real-time basis, in order to

determine the level at which the listening situation starts to

challenge WM resources. WM resources are challenged differ-

ently depending on listening situation, and different individuals

may have different cognitive resources available to handle such

specific workloads. Therefore, there is a need to develop

monitoring methods for estimating cognitive workload. Two

different lines of research can be foreseen; indirect estimates

of cognitive workload and direct estimates of cognitive

workload.

Indirect estimates of cognitive workload would use some

form of cognitive model that is continuously updated with

environment detectors that monitor the listening environment

(e.g., level detectors, SNR detectors, speech activity detectors,

reverberation detectors), as well as the conversational situa-

tion; the identity, mood and behavior of the conversational

partner as well as the purpose of the communication (social,

information exchange) and feedback pattern. The cognitive

model should produce at least two states, indicating cognitive

High load or cognitive Low load. If cognitive High load is

detected, hearing aid signal processing systems, such as direc-

tional microphones, noise reduction systems and fast compres-

sion, should be invoked to release cognitive resources. The

cognitive model needs to be calibrated with the individual

cognitive capacity (e.g., WM capacity, verbal information pro-

cessing speed), and connections between listening environ-

ment monitors, hearing aid processing system, and cognitive

capacities have to be established. Inspiration might be found

in the ease of language understanding (ELU) model of Rönn-

berg et al. (2008), which has a framework for suggesting

when a listener’s WM system switches from effortless implicit

(bottom-up) processing to effortful explicit (top-down) pro-

cessing.

However, a more direct way to assess cognitive workload

would be through physically measurable correlates (e.g. Kra-

mer, 1991). Given direct estimates of cognitive load, measures

of cognitive High and Low load could be established. How-

ever, relations between environment characteristics, signal

processing features and cognitive relief would still have to be

established. A straightforward, but technically challenging

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

example of a direct estimate of cognitive High and Low load

could be obtained by electroencephalographic measurements

(EEG; Gevins, Smith, McEvoy & Yu, 1997). A wearable sys-

tem has been proposed by Lan, Erdogmus, Adami, Mathan &

Pavel (2007), which could be used to produce a cognitive state

classification system based on EEG measurements. This could

possibly be used to control the parameters of hearing aid sig-

nal processing algorithms to individually reduce cognitive

‘‘workload’’ in challenging listening situations.

In summary, the concept of cognition-driven hearing aid sig-

nal processing is at the meeting point between the audiological

and cognitive psychology disciplines, and mutual research is

of great benefit to the development of our understanding of

how hearing aid signal processing interacts with cognitive abil-

ities. In the long term, a cognition-driven hearing aid could be

beneficial not only for optimizing signal processing but also

for minimizing the negative impact of sensory impairment on

cognitive function.

We would like to thank Stig Arlinger, Kathleen Pichora-Fuller, Graham Naylor, and two anonymous reviewers for their helpful and insightful comments on earlier versions of this manuscript.

REFERENCES

Akeroyd, M. A. (2008). Are individual differences in speech reception related to individual differences in cognitive ability? A survey of twenty experimental studies with normal and hearing-impaired adults. International Journal of Audiology, 47(Suppl. 2), S125– S143.

Andersson, U., Lyxell, B., Rönnberg, J. & Spens, K.-E. (2001). Cogni- tive correlates of visual speech understanding in hearing-impaired individuals. Journal of Deaf Studies and Deaf Education, 6, 103– 116.

Anzalone, M. C., Calandruccio, L., Doherty, K. A. & Carney, L. H. (2006). Determination of the potential benefit of time-frequency gain manipulation. Ear and Hearing, 27, 480–492.

Baddeley, A. D. (1986). Working memory. Oxford: Oxford University Press.

Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Science, 4(11), 417–423.

Bentler, R. & Chiou, L.-K. (2006). Digital noise reduction: An over- view. Trends in Amplification, 10(2), 67–82.

Bilger, R. C., Nuetzel, J. M., Rabinowitz, W. M. & Rzeczkowski, C. (1984). Standardization of a test of speech perception in noise. Journal of Speech and Hearing Research, 27, 32–48.

Chung, K. (2004). Challenges and recent developments in hearing aids. Part I. Speech understanding in noise, microphone technologies and noise reduction algorithms. Trends in Amplification, 8(3), 83–124.

Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–185.

Daneman, M. & Carpenter, P. A. (1980). Individual differences in inte- grating information between and within sentences. Journal of Experi- mental Psychology: Learning, Memory and Cognition, 9, 561–584.

Daneman, M. & Merikle, P. M. (1996). Working memory and lan- guage comprehension: A meta-analysis. Psychonomic Bulletin and Review, 3(4), 422–433.

Dillon, H. (1996). Compression? Yes, but for low or high frequencies, for low or high intensities, and with what response times? Ear and Hearing, 17, 287–307.

logical Associations.

402 T. Lunner et al. Scand J Psychol 50 (2009)

Dubno, J. R., Dirks, D. D. & Morgan, D. E. (1984). Effects of age and mild hearing loss on speech recognition in noise. Journal of the Acoustical Society of America, 76(1), 87–96.

Engle, R. W., Cantor, J. & Carullo, J. J. (1992). Individual differences in working memory and comprehension: A test of four hypotheses. Journal of Experimental Psychology: Learning, Memory, and Cog- nition, 18, 972–992.

Ephraim, Y. & Malah, D. (1984). Speech enhancement using a mini- mum mean-square error short-time spectral amplitude estimator. IEEE Transactions on Acoustics, Speech and Signal Processing, 32(6), 1109–1121.

Feldman Barrett, L., Tugade, M. M. & Engle, R. W. (2004). Individual differences in working memory capacity and dual-process theories of the mind. Psychological Bulletin, 130(4), 553–573.

Foo, C., Rudner, M., Rönnberg, J. & Lunner, T. (2007). Recognition of speech in noise with new hearing instrument compression release settings requires explicit cognitive storage and processing capacity. Journal of the American Academy of Audiology, 18, 553– 566.

Gatehouse, S., Naylor, G. & Elberling, C. (2003). Benefits from hear- ing aids in relation to the interaction between the user and the envi- ronment. International Journal of Audiology, 42(Suppl. 1), S77– S85.

Gatehouse, S., Naylor, G. & Elberling, C. (2006a). inear and nonlinear hearing aid fittings – 1. Patterns of benefit. International Journal of Audiology, 45, 130–152.

Gatehouse, S., Naylor, G. & Elberling, C. (2006b). Linear and non-lin- ear hearing aid fittings – 2. Patterns of candidature. International Journal of Audiology, 45, 153–171.

Gevins, A., Smith, M. E., McEvoy, L. & Yu, D. (1997). High resolu- tion EEG mapping of cortical activation related to working mem- ory: effects of task difficulty, type of processing, and practice. Cerebral Cortex, 7(4), 374–385.

Halford, G. S., Wilson, W. H. & Phillips, S. (1998). Processing capac- ity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology. Behavioral and Brain Sciences, 21, 803–865.

Hamacher, V., Chalupper, J., Eggers, J., Fischer, E., Kornagel, U., Puder, H. & Rass, U. (2005). Signal processing in high-end hearing aids: State of the art, challenges, and future trends. EURASIP Journal on Applied Signal Processing, 18, 2915–2929.

Holube, I., Hamacher, V. & Wesselkamp, M. (1999). Hearing instru- ments: Noise reduction strategies, in Proceedings of the 18th Dan- avox Symposium: Auditory Models and Non-linear Hearing Instruments, Kolding, Denmark, September.

Just, M. A. & Carpenter, P. A. (1992). A capacity theory of compre- hension - individual differences in working memory. Psychological Review, 99, 122–149.

Kramer, A. F. (1991). Physiological metrics of mental workload: A review of recent progress. In D. L. Damos (Ed.), Multiple-task per- formance (pp. 279–328). London: Taylor & Francis.

Lan, T., Erdogmus, D., Adami, A., Mathan, S. & Pavel, M. (2007). Channel selection and feature projection for cognitive load estima- tion using ambulatory EEG. Computational Intelligence and Neuro- science, Volume 2007, Article ID 74895, 1–12.

Larsby, B., Hällgren, M., Lyxell, B. & Arlinger, S. (2005). Cognitive per- formance and perceived effort in speech processing tasks: Effects of different noise backgrounds in normal-hearing and hearing impaired subjects. International Journal of Audiology, 44(3), 131–143.

Leijon, A. & Stadler, S. (2008). Fast amplitude compression in hear- ing aids improve audibility but degrades speech information trans- mission. Internal report 2008-11; Sound and Image Processing Lab., School of Electrical Engineering, KTH, SE-10044, Stock- holm, Sweden.

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

Lotter, T. & Vary, P. (2003). Noise reduction by maximum a posteriori spectral amplitude estimation with super Gaussian speech model- ing, in Proceedings of the International Workshop on Acoustic Echo and Noise Control (IWAENC ‘03) (pp. 83–86), Kyoto, Japan, September.

Ludvigsen, C. (1997). Schaltungsanordnung für die automatische Regelung von Hörhilfsgeräten. Europäische Patentschrift, EP 0 732 036 B1.

Lunner, T. (2003). Cognitive function in relation to hearing aid use. International Journal of Audiology, 42(Suppl. 1), S49–S58.

Lunner, T., Hellgren, J., Arlinger, S. & Elberling, C. (1997). A digital filterbank hearing aid: Three DSP algorithms – user preference and performance. Ear and Hearing, 18, 373–387.

Lunner, T. & Sundewall-Thorén, E. (2007). Interactions between cog- nition, compression, and listening conditions: Effects on speech-in- noise performance in a two-channel hearing aid. Journal of the American Academy of Audiology, 18, 539–552.

McCoy, S. L., Tun, P. A., Cox, L. C., Colangelo, M., Stewart, R. A. & Wingfield, A. (2005). Hearing loss and perceptual effort: Down- stream effects on older adults’ memory for speech. Quarterly Jour- nal of Experimental Psychology A, 58, 22–33.

Magnusson, L., Karlsson, M. & Leijon, A. (2001). Predicted and mea- sured speech recognition performance in noise with linear amplifi- cation. Ear and Hearing, 22(1), 46–57.

Martin, R. (2001). Noise power spectral density estimation based on optimal smoothing and minimum statistics. IEEE Transactions on Speech and Audio Processing, 9(5), 504–512.

Martin, R. & Breithaupt, C. (2003). Speech enhancement in the DFT domain using Laplacian speech priors, in Proceedings of the Inter- national Workshop on Acoustic Echo and Noise Control (IWAENC ‘03) (pp. 87–90), Kyoto, Japan, September.

Miyake, A. & Shah, P. (1999). Models of working memory. Cam- bridge: Cambridge University Press.

Moore, B. C. J. (1998). A comparison of four methods of implement- ing automatic gain control (AGC) in hearing aids. British Journal of Audiology, 22, 93–104.

Naylor, G. & Johannesson, R. B. (2009). Long-term Signal-to- Noise Ratio (SNR) at the input and output of amplitude compres- sion systems. Journal of the American Academy of Audiology, 20(3), 161–171.

Pearsons, K. S., Bennett, R. L. & Fidell, S. (1977). Speech levels in various environments (EPA-600/1-77-025). Washington, DC: Envi- ronmental Protection Agency.

Pichora-Fuller, M. K. (2007). Audition and cognition: What audiolo- gists need to know about listening. In C. Palmer & R. Seewald (eds.) Hearing care for adults (pp. 71–85). Stäfa, Switzerland: Pho- nak.

Pichora-Fuller, M. K., Schneider, B. A. & Daneman, M. (1995). How young and old adults listen to and remember speech in noise. Jour- nal of the Acoustical Society of America, 97, 593–608.

Plomp, R. (1988). Auditory handicap of hearing impairment and the limited benefit of hearing aids. Journal of the Acoustical Society of America, 63(2), 533–549.

Plomp, R. (1994). Noise, amplification, and compression: Consider- ations for three main issues in hearing aid design. Ear and Hear- ing, 15, 2–12.

Rabbitt, P. (1968). Channel-capacity, intelligibility and immediate memory. Quarterly Journal of Experimental Psychology, 20, 241– 248.

Rabbitt, P. (1990). Mild hearing loss can cause apparent memory fail- ures which increase with age and reduce with IQ. Acta Oto-laryng- ologica, 111(Suppl. 476), 167–176.

Ricketts, T. A. (2005). Directional hearing aids: Then and now. Jour- nal of Rehabilitation Research and Development, 42 (4), 133–144.

logical Associations.

Scand J Psychol 50 (2009) Cognition and hearing aids 403

Rönnberg, J. (1990). Cognitive and communicative function: The effects of chronological age and ‘‘handicap age’’. European Jour- nal of Cognitive Psychology, 2, 253–273.

Rönnberg, J. (2003). Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: A framework and a model. International Journal of Audiology, 42, S68–S76.

Rönnberg, J., Rudner, M., Foo, C. & Lunner, T. (2008). Cognition counts: A working memory system for ease of language under- standing (ELU). International Journal of Audiology., 47(Suppl. 2), S171–S177.

Rudner, M., Foo, C., Sundewall-Thorén, E., Lunner, T. & Rönnberg, J. (2008). Phonological mismatch and explicit cognitive processing in a sample of 102 hearing aid users. International Journal of Audiol- ogy, 47(Suppl. 2), S163–S170.

Rudner, M., Lunner, T., Behrens, T., Sundewall Thorén, E. & Rönn- berg, J. (2009). Self-rated effort, cognition and aided speech recog- nition in noise. EFAS.

Sarampalis, A., Kalluri, S., Edwards, B. & Hafter, E. (2006). Cognitive effects of noise reduction strategies. International Hearing Aid Research Conference (IHCON), Lake Tahoe, CA, August.

Sarampalis, A., Kalluri, S., Edwards, B. & Hafter, E. (2008). Under- standing speech in noise with hearing loss: measures of effort. International Hearing Aid Research Conference (IHCON), Lake Tahoe, CA, August 13-17.

Sarampalis, A., Kalluri, S., Edwards, B. & Hafter, E. (2009). Objective measures of listening effort: Effects of background noise and noise reduction. Journal of Speech, Language, and Hearing Research doi: 10.1044/1092-4388(2009/08-0111).

Schulte, M., Vormann, M., Wagener, K., Büchler, M., Dillier, N., Dreschler, W., Eneman, K. et al (2009). Listening effort scaling and preference rating for hearing aid evaluation. HearCom Work- shop on Hearing Screening and new Technologies. http:// hearcom.eu/about/DisseminationandExploitation/Workshop.html

Schum, D. J. (2003). Noise-reduction circuitry in hearing aids: (2) Goals and current strategies. The Hearing Journal, 56(6), 32–40.

Schum, D. J., Matthews, L. J. & Lee, F. S. (1991). ctual and predicted word-recognition performance of elderly hearing-impaired listeners. Journal of Speech Hearing Research, 34, 636–642.

Shinn-Cunningham, B. G. (2008). Object-based auditory and visual attention. Trends in Cognitive Sciences, 12(5), 182–186.

Shinn-Cunningham, B. G. & Best, V. (2008). Selective attention in nor- mal and impaired hearing. Trends in Amplification, 12(4), 283–299.

� 2009 The Authors. Journal compilation � 2009 The Scandinavian Psycho

Stone, M. A. & Moore, B. C. J. (2004). Side effects of fast-acting dynamic range compression that affect intelligibility in a compet- ing-speech task. Journal of the Acoustical Society of America, 116(4), 2311–2323.

Stone, M. A. & Moore, B. C. J. (2008). Effects of spectro-temporal modulation changes produced by multi-channel compression on intelligibility in a competing-speech task. Journal of the Acoustical Society of America, 123(2), 1063–1076.

Takeshi, H., Takahiro, M., Yoshihisa, I. & Tetsuya, H. (2003). Musical noise reduction using an adaptive filter. Acoustical Society of Amer- ica Journal, 114(4), 2370–2370.

Tun, P. A. & Wingfield, A. (1999). One voice too many: Adult age differences in language processing with different types of distract- ing sounds. Journals of Gerontology Series B: Psychological Sci- ences and Social Sciences, 54(5), 317–327.

van Boxtel, M. P., van Beijsterveldt, C. E., Houx, P. J., Anteunis, L. J., Metsemakers, J. F. & Jolles, J. (2000). Mild hearing impair- ment can reduce verbal memory performance in a healthy adult population. Journal of Clinical Experimental Neuropsychology, 22, 147–154.

van den Bogaert, T., Doclo, S., Wouters, J. & Moonen, M. (2008). The effect of multimicrophone noise reduction systems on sound source localization by users of binaural hearing aids. Journal Acoustical Society of America, 124(1), 484–97.

Wang, D. L. (2005). On ideal binary mask as the computational goal of auditory scene analysis. In P. Divenyi (Ed.), Speech separation by humans and machines (pp. 181–197). Norwell, MA: Kluwer Academic.

Wang, D. L. (2008). Time-frequency masking for speech separation and its potential for hearing aid design. Trends in Amplification, 12, 332–353.

Wang, D. L., Kjems, U., Pedersen, M. S., Boldt, J. B. & Lunner, T. (2009). Speech intelligibility in background noise with ideal binary time-frequency masking. Journal of the Acoustical Society of Amer- ica, 125(4), 2336–2347.

Wilken, P. & Ma, W. J. (2004). A detection theory account of change detection. Journal of Vision, 4, 1120–1135.

Wingfield, A. & Tun, P. A. (2001). Spoken language comprehension in older adults: Interactions between sensory and cognitive change in normal aging. Seminars in Hearing, 22(3), 287–301.

Received 22 April 2009, accepted 30 April 2009

logical Associations.