It's because he has perfect pitch.
Perfect Pitch, or Absolute Pitch is "is a rare auditory phenomenon characterized by the ability of a person to identify or re-create a given musical note without the benefit of a reference tone" (Wikipedia, https://en.wikipedia.org/wiki/Absolute_pitch).
In the series, Kousei was referred to as a "piano prodigy" and a "human metronome" by others, able to play the music by ear perfectly on tempo. After his mother's death, the piano became traumatic to him, and each time he played, he said in despair that he "lost his notes" and more importantly, "couldn't hear the notes".
This confirms that Kousei DOES have Perfect Pitch. Due to the grief of losing his mother, he was not able to make out the notes and lost his musical passion.
In the scenes where Kousei was losing his "ear", the notes sounded distorted and he felt like he was drowning in darkness. At these times, he was unable to detect the pitch of the notes accurately and fell into a panic.
Another picture that proves he has perfect pitch?
THIS. If a pianist can imagine the score and know internally how it sounds, then the musician has perfect pitch.
This is my theory of why Kousei "lost his notes". Hope you enjoyed reading this!
Have any more theories about Kousei? Feel free to comment below!
b（´∇｀）
Thanks for reading!
