Musical interfaces have always tried to provide the musician with the best visual representation of the music to assist their understanding. A primary area of investigation has been the implementation of synesthetic principles in such interfaces.
Many theories have tried to explain the correlative relationship between music and color. Scientists have suggested several pairs of musical notes and colors based on their frequency ratios.
This relationship was tested in an experiment where participants assigned colors to music. The results indicated a relationship, but the exact correlation is not yet clear.
The findings were mirrored in a separate study where even non-synesthetes displayed consistent associations of high-pitched tones with bright colors, while low-pitched tones were associated with dark colors.
But, how does this apply to the design of a visual music interface?
Visual representation of audio content can be dissected into two categories: graphical sound and graphical notation. In graphical sound, visuals respond in real-time to inputs. It provides an interactive experience but may lack complex musical details.
On the other hand, graphical notation makes use of traditional music notation. It can display complex musical details but often struggles with being interpretable in real-time by musicians.
Both approaches take into account the spaces where visual and musical elements coincide. The challenge is to strip away anything that might obscure the musician's understanding of the music.
How does cognitive psychology play a role?
In visual perception, certain features naturally attract our attention. These are called preattentive features. Examples include color, shape, and size.
These features are processed in parallel before consciousness and are ideal to deploy in any type of interface, including a musical one. The ability to distinguish these features effortlessly aids the user's understanding.
Furthermore, our recognition of patterns and behaviors plays a role. We perceive things with similar traits as belonging together, known as the principle of grouping.
These principles help design an interface that supports our perception system, increasing the efficiency of information processing.
So, how are these concepts incorporated into a visual music interface?
Colors and shapes assist musicians to understand scales and musical relationships. And, it's not just about displaying the notes but also their functions within the scale.
Just as we use different colors to create a visual hierarchy, in a music interface colors and shapes can reflect the function of the musical notes.
The most important factor is that the design should support the efficiency of information delivery and facilitate the musician's comprehension.
Considering all these theories, the design of a music interface attempts to help musicians understand music theory and create music more efficiently. It's like learning a new language, where the interface is the translator.
Conclusion
By interpreting sound into visual elements, a visual music interface can aid the musician's understanding of music.
Synesthetic principles, good interface design concepts, and cognitive psychology all play a role in this process.
While the perfect implementation might still be in the future, the intent is clear: To simplify the complexities of music and make it more accessible to the performer.
Music, after all, is a language. And, like any language, it is meant to be understood, spoken, and shared.