Mikyung Lim

Professor of Music & Music Columnist

Having spent a long time as a music major in the fields of performance and education, I often encounter this question:

“Why does the same sheet music sound different depending on the performer?” This question actually touches upon the essence of music. A score only contains basic instructions such as pitch, rhythm, and indications like "loudly (forte)" or "softly (piano)." However, actual music is created in the spaces between those notes. Subtle differences—where a phrase (a musical sentence) breathes, how long a single note is held, or how much time is delayed before moving to the next chord—create the vitality of music. Traditionally, we have explained these differences using words like "interpretation" or "sensibility." However, with the recent emergence of Artificial Intelligence (AI), new changes are appearing in the way we view music.

#. When Music Needs to be Compared and Heard

When I teach piano, I often encourage my students to compare performances of the same piece by different musicians. For example, when studying Chopin's Nocturne (a lyrical piano work expressing the atmosphere of the night), I play recordings by several pianists. At first, students simply say "it feels different," but as they listen more analytically, they discover differences in musical tension and interpretation. One performer might release tension by slightly slowing down at the end of a section, while another might build the peak of tension in the middle of a phrase. In this way, performers create musical tension and flow by subtly changing the tempo—a style of expression known as rubato. This is a hallmark musical characteristic found in Romantic-era piano music, particularly in the works of Chopin.

#. How AI Grasps 'Differences in Performance'

How, then, does AI analyze these subtle performance differences? The core lies in the process of converting music into data. AI first analyzes a recording by dividing it into very short time units. For instance, a computer digitally samples sound at extremely brief intervals to identify exactly when a note begins. This process is called "onset detection"—a technology that finds the moment a sound starts. With this information, the computer can calculate the time intervals between each note in an actual performance and graph exactly where the tempo accelerates or decelerates. We can then see what kind of temporal patterns exist in a performance we perceived as "natural rubato." Another method is spectral analysis. Musical sound is not a single frequency but a mixture of various overtones. Computers analyze this frequency structure to measure the intensity and timbre of a note. This allows us to visually confirm at which points the music strengthens—that is, how the dynamics (the volume) change.

For example, analyzing recordings of Beethoven's piano sonatas by various performers clearly reveals these trends. A musical climax isn't always created just by playing loudly; often, it's formed as energy gradually accumulates through the progression of a phrase. AI shows this "energy envelope" by graphing volume changes over time. This is a way to scientifically verify the musical tension we intuitively felt through performance experience.

#. The Relationship Between Articulation and AI

This technology also helps in understanding keyboard music from the era of Bach. Students often try to play Bach's pieces by holding the keys exactly as written in the score. However, many pianists actually play by slightly separating the notes rather than connecting them smoothly. The way a performer chooses to connect or separate notes is called articulation.

AI uses real data here as well. The computer calculates how long a note is sustained and the gap until the next note. This is called the "note-length ratio." For instance, even if notes are written as the same length, in Bach's keyboard music, they are often played in a "non-legato articulation"—slightly detached. This is to respect the characteristics of the harpsichord (the primary keyboard instrument of Bach's time), which produced sound by plucking strings, resulting in a light sound and a natural separation between notes since it lacked a sustain pedal. By analyzing actual performance data, AI allows us to objectively confirm the expressive articulations found in Bach's music.

#. New Possibilities for Music History Research

These AI-based music analysis methods are opening new possibilities for music history research. AI can analyze thousands of works to find common patterns unique to each composer. For example, Bach's keyboard music is characterized by counterpoint structures where multiple melodies progress simultaneously. In Chopin's music, song-like melodies and colorful harmonic patterns appear repeatedly. AI statistically analyzes these features to display a composer's musical style as a sort of "pattern map." Through this, we can systematically understand how a composer's musical language and style were formed. As someone who teaches music and evaluates performances at a university, I view these changes with great interest. Just as the development of printing once distributed scores and recording technology preserved performances, AI is now leading us to understand music in a new way. The era is approaching where AI can analyze a student's piano performance, measure tempo flow, dynamic structure, and articulation changes, and compare them with masterful performances to examine differences in interpretation. However, since creating music that evokes deep emotion remains the domain of humans, I hope that AI technology will be used as a new tool to help us understand our music more deeply.

And through that tool, I hold the positive expectation that we will discover yet more secrets hidden within music.

Share: