New Binaural Strategies for Enhanced Hearing

Tech Topic | October 2014 Hearing Review

 

By Homayoun Kamkar-Parsi, PhD, Eghart Fischer, Dipl-Ing, and Marc Aubreville, Dipl-Ing

This paper describes the new binaural directional features of binax,  Siemens’ next generation of BestSound Technology. In addition to transmitting control and status data between hearing aids, the binax hearing aid system exchanges audio signals between left and right hearing aids.  Introduced in this paper are two key features of binax: 1) Narrow Directionality, an enhanced binaural beamforming algorithm designed for very difficult listening situations, and 2) Spatial SpeechFocus, a self-steering binaural beamforming algorithm designed especially for situations with talkers from side and back directions in noisy environments (eg, in cars or walking down a sidewalk).  The advantages of automatic activation and control of the new algorithms in select channels and their exceptionally low  power consumption are also reviewed.

Binaural processing can significantly improve speech understanding in background noise. At Siemens, our objective was to develop binaural algorithms that address the most challenging listening situations for those with hearing loss—hearing in difficult, noisy, listening environments, like restaurants, cocktail parties, and hearing in wind.

Two algorithms introduced in this article are Narrow Directionality and Spatial SpeechFocus. These advanced binaural beamforming algorithms use Siemens’ binaural wireless audio exchange technology (e2e wireless 3.0), which may potentially improve speech understanding in noisy and adverse acoustic environments. They may also provide a more efficient solution to the cocktail-party effect compared to bilaterally fitted conventional monaural differential microphone systems.

Narrow Directionality Processing

Narrow Directionality is designed to enhance the speech signal when noise is coming from all around the listener, and this “noise” is also made up from other people talking. It creates a narrow beam toward the front direction so the wearer can listen easily to any distinct speaker by turning his or her head toward the speaker.

Narrow Directionality improves the speech signal from the speaker in two ways simultaneously: 1) by quickly reducing other competing speech signals outside the beam angular range, and 2) by boosting the level of the speaker signal within the beam (ie, “focus” on the speaker). Additionally, the system has an automatic adaptive control that smoothly adjusts in selected channels to take directivity from a wider to a narrower beam as the SNR in the environment deteriorates. This ensures the optimal listening experience in all kinds of noisy environments, while preserving spatial queues.

As illustrated in Figure 1, Siemens’ binaural processing is comprised of three essential components: the binaural beamforming, the binaural noise reduction gain, and the head movement compensation module. These components work together to achieve the Narrow Directionality Effect.

Binaural Beamforming

This is a new kind of binaural beamforming that incorporates the head shadowing effect. It takes into account the contralateral wireless binaural signal as shown in Figure 1. For each side, the binaural beamformer is designed as follows: it takes as input the local signal, which is the monaural directional signal, and the contralateral signal, which is the monaural directional signal transmitted from the opposing hearing instrument (ie, via binaural wireless link e2e wireless 3.0). It should be noted that the monaural directional signal (from the local side or the contralateral side) is already an enhanced signal with reduced noise from the back hemisphere.

The output of the beamformer is generated by linearly adding the weighted local signal and the weighted contralateral signals. The weighting scheme, which is a crucial part of the overall design, is aimed to provide an output signal with maximum lateral interference cancellation while keeping the frontal speaker untouched.

How should the weightings be then optimized to achieve this goal? Taking the left hearing instrument as the reference, imagine the following example: a target speaker is located at 0° in front of the listener, who is fitted with binaural hearing instruments. The local and contralateral (ie, the transmitted signal from the right hearing instrument) signals will therefore arrive at the same time to both ears without any head shadowing effect. In other words, both signals will have approximately the same power and phase. However, for the case of a lateral interfering noise (eg, at 45°), the local and contralateral signals will be different due to head shadowing and interaural time difference; the signals will have power and phase (or time) differences. Therefore, given a target at 0° with the presence of an interfering lateral noise or a competing talker at 45°, the local signal power will be higher than the contralateral signal power due to the interfering noise. It also implies that the contralateral signal is the one that is less affected by the noise due to the head shadowing.

Having this example in mind, we designed an algorithm to derive the optimum weighting with the following criteria:

1. The sum of the weighted local and the weighted contralateral input signals should always produce an output signal (to each ear) with minimum power with respect to the original local and contralateral signal powers. These criteria ensure that lateral interferences are attenuated. Importantly, the weightings are also adaptive and are updated within milliseconds to closely follow fast changes in the noisy environment.
2. The additive combination of the weights should always add up to 1. This ensures that the target signal at 0° remains untouched.

Applying points 1 and 2 above, the new binaural beamformer creates a narrow beam to the front direction with the beam pattern. The resulting is a Narrow Directionality (combination of binaural beamforming and binaural noise reduction) output signal characteristic relative to monaural directionality.

Binaural Noise Reduction

To further enhance the output signal from our binaural beamformer, a new adaptive binaural noise reduction gain (fully integrated within the noise reduction unit) was developed (Figure 2). This noise reduction gain can be interpreted as a binaural Wiener-based gain computed using the local and the contralateral signals as inputs.

It was designed to have several specific properties. It attenuates lateral interfering speakers coming from outside the front target angular range (±10°). Therefore, a competing speaker, as close as 45°, is further attenuated (ie, by applying gain below 1—typical Wiener-based gain attenuation). However, if there is a front target speaker present in the front angular range, the front speaker is moderately amplified (or boosted) by applying a gain above 1 (which is not a typical Wiener-based gain property).

Narrow Directionality then gives hearing aid wearers the perception that they are focusing on the person they are facing (like a magnifying glass), as illustrated in Figure 2c. It should be noted that the adaptation of the binaural noise reduction gain is fast enough (within milliseconds) to rapidly amplify or attenuate depending on the acoustic situation—without background noise increasing.

Head Movement Compensation

Narrow Directionality also includes a head movement compensation module to avoid any distortion. This is necessary to ensure a normal, comfortable conversation without requiring the wearer to always directly face the speaker.

Head movements are a part of normal behavior during a conversation, and they usually occur very quickly. Plus-and-minus 10 degrees can be considered as a normal range of regular head movements by either the speaker or the listener. So, if the target signal is at +10°, the head compensation module modifies the originating input binaural signal at +10° and brings it back to 0° (more specifically, the phase and the level of the contralateral signal is re-adjusted to match the local signal). This allows the binaural beamforming and the binaural noise reduction to behave the same way; that is, the target signal is “seen” at 0°, which lies within the narrow frontal beam range of 0° of Narrow Directionality. As a result, the target signal is still enhanced rather than attenuated due to head movements.

Automatic Control of Narrow Directionality

The goal of this algorithm is to estimate the complexity of the listening situation, and seamlessly introduce more or less directionality when needed. For this, multiple criteria are evaluated.

The acoustic complexity of the hearing aid user’s listening environment is usually linked with background noise level. A certain level of background noise, optimized for various noisy environments, is needed to activate binaural processing. This threshold is higher than the respective noise level expected for monaural processing. If this binaural threshold is exceeded, the binaural audio transmission is enabled.

Once the audio signal from the contralateral hearing instrument is available, the hearing instrument has far more possibilities to analyze the scene. Using a combination of both ipsi- and contra-lateral metrics, the effect of the beamformer is restricted to situations and frequency bands where it is useful, and is adjusted to specific frequencies, depending on the background noise level (Figure 3).

For low noise levels, the effect is kept to a minimum, whereas for medium noise levels it is increased, and it reaches full directionality for high noise levels. This has the important advantage of keeping spatial orientation and natural characteristics of sound to a maximum in situations with medium noise, and in situations where most of the noise is contained in the lowest frequencies.

It should be noted that the classification of the acoustic situation surrounding the hearing aid wearer is also taken into account. For example, there may be some situations (eg, enjoying loud music) that should not activate a narrow beamformer in an automatic program.

Real-World Efficacy

The hearing instrument wearer only needs to be focused on audibility and listening comfort in every situation. This is why it is so important to have a seamless adaptation as the acoustic situation changes. Any control that is acting too fast is prone to misdetections, whereas any control that is acting too slowly can be noticed, and therefore irritating. If the situations change quickly, the automatic steering has to adapt quickly;  if the situation changes gradually, the automatic steering should also adapt gradually. The best possible reaction from a wearer is “I did not notice any automatic adjustments of the hearing instrument—it always seemed just right.” The maximum user benefit is reached when the directionality and noise reduction are optimized so that the overall perception is natural and unnoticeable.

The efficacy of the Narrow Directionality algorithm for speech recognition for hearing impaired individuals has been studied in clinical trials at two different independent sites.¹ These behavioral findings are very encouraging, and align with SNR advantages expected in the algorithm design.

Advanced Beamforming in 360° and Spatial SpeechFocus

The binaural audio transmission introduced with the binax platform enables not only beamforming for situations where the target speaker is in the front, but also allows for beamforming to the side for situations like walking or sitting side-by-side. Until now, the best one could do when a target speaker is located to the side was to select the omnidirectional mode. However, in these situations the interfering sources (eg, street noise) often come from other directions, and omnidirectional processing cannot suppress the undesired noise.

This is where Spatial SpeechFocus, introduced in the binax platform, comes into play. By suppressing noise from one side, and enhancing the target signal from the other side on both ears, this technology can increase listening comfort, as well as speech understanding.

The algorithm works on the same fundamental principles as human hearing. When sound is coming from one side of the head, it will have two major differences in the two ears: First, as sound propagates, it will arrive earlier at the ear closer to the sound source, described as the interaural time difference (ITD). If the sound comes exactly from the side of the person (90°), this time difference is approximately 0.7 milliseconds. This will result in a phase difference of the respective sound, which can be used by a state-of-the-art differential beamformer.² This phase difference is most evident for frequencies lower than approximately 750 Hz.

A second effect is the interaural level difference (ILD). As the sound waves travel past the head, lower frequency sound waves are diffracted, and hence, little attenuation occurs. But for higher frequencies, there is significant attenuation that can be detected and used to determine the origin of a sound. This can be accomplished even for high-frequency noises, albeit with less precision. In the Spatial SpeechFocus algorithm, a Wiener-filter-based approach is used to suppress signals coming from an undesired side.

Both ITD and ILD are utilized to create a powerful beamforming algorithm, the effect of which can be observed in the polar plots shown in Figure 4a-b. Depending on the frequency and room acoustics, the attenuation is approximately 10 dB. The major advantage over mere copying of the preferred ear to the other side is that, in this application, spatial cues are kept. That is, the user can still localize the sound and has a natural spatial impression. The interaural level differences—and thus the localization of the sources—remain untouched.

This beamformer can be controlled manually, using a remote control application (Spatial Configurator), but it also can be controlled automatically. For the latter, features that correlate with the SNR of speech are calculated independently in both hearing aids. These features represent the probability of speech from the front, the back, and one of the sides. By exchanging this information and combining the data, it is possible to determine from which direction a speech signal originates.

This is an extension of Siemens’ SpeechFocus algorithm (designed to automatically focus to the back when a talker originates from this area) to now focus to the left or right directions, depending on the direction from where the target signal originates. The Spatial SpeechFocus beamformer is now able to provide a signal focus to either side. If speech is coming from both sides, or in quiet, an omnidirectional microphone mode is chosen (Figure 4a). The switching is performed synchronously on both ears, using a smooth transition. In all situations, the correct perception of the speaker(s) will remain because the binaural cues are maintained to a large extent.

Energy Efficiency

When talking about sophisticated algorithms in tiny hearing instruments with even smaller batteries, it is critical that these new features are energy efficient. Compared to the previous micon platform (which did not have bilateral audio exchange), binax battery consumption remains the same for the normal microphone modes. More importantly, when audio streaming is active, the additional binaural features offered by binax increase the battery consumption by only 200 µA or less, from 1.3 to still below 1.5 mA, while the new features are automatically activated.

This is considerably more efficient than other products in the market using similar binaural processing. In practical terms, assuming a wearer uses his/her hearing aids 12 hours/day, and uses the binaural audio streaming for 2 hours/day, overall battery life would only decrease by 2.5%—a difference so small it is unlikely it would be noticed by the hearing aid wearer.

Summary

This paper reviewed the new binaural directional features of Siemens’ binax platform. Narrow Directionality is an enhanced binaural beamforming algorithm, designed especially for very difficult listening situations with multiple talkers in the background. It provides a narrow acoustic focus to the “looking direction” of the wearer, and thus enables the hearing aid user to understand the preferred talker, even with several other talking persons in close proximity. What also makes it unique is its automatic smooth, situation-dependent activation and deactivation, and the high resolution control of its strength of effect, which separately depends on the acoustic conditions in each frequency band.

Spatial SpeechFocus is a self-steering, binaural, beamforming algorithm especially useful for situations when speakers are talking from the left or right side in noisy environments. It is automatically activated when the presence of speech from one side is detected in the presence of background noise. Like the human ear, it uses interaural phase and level differences for maintaining a natural spatially correct sound impression.

Whether it’s Narrow Directionality for listening to a talker from the front, or Spatial SpeechFocus for understanding speech from the sides or the back, we now have effective binaural algorithms that operate automatically to assist hearing aid wearers in many different types of complex listening environments—with extremely low power consumption.

References

1. Powers TA, Froehlich M. Clinical results with a new wireless binaural directional hearing system. Hearing Review. 2014. In press.

2. Elko GW, Nguyen Pong A-T. A simple adaptive first-order differential microphone. Paper presented at: IEEE AASP Workshop on Applications of Signal Processing to Audio and Acoustics. New Paltz, NY; October 15-18, 1995.

CORRESPONDENCE can be addressed to Thomas Powers, PhD, at: [email protected]

Original citation for this article: Kamkar-Parsi H, Fischer E, Aubreville M. New binaural strategies for enhanced hearing. Hearing Review. 2014;21(10):42-45.