It doesn't explain how Terrence Mann heard the voice
If you listen to the dialog directly before Joe says "no Ray, it was you" we hear Ray say "ease his pain" when talking about his dad. At that moment Ray thinks it was his dads pain that needed to be eased. In my opinion Joe saying "no Ray, it was you" means that it was really Ray's pain that needed to be eased. He'd felt years of guilt for not making peace with his dad.