In the phrase editor, if you hit play or return to set the phrase playing, hitting the stop button does not stop the phrase’s playback. 
Windows 7 x64
Aaaaaaaaaaaaaand now it mysteriously works again.
If I can replicate it again, I’ll try to isolate what caused it.
It does it to me too, but the play button stops it.
Hitting space twice should work as well.
The phrase continues playing if triggered from a midi device.
Pressing stop or panic in Renoise itself should definitely work. You could also set the Phrase to play once by untoggling the “loop” button.
All of these things do indeed work. It’s just disconcerting that the stop button didn’t work, even if it somehow magically works again now.
As mentioned before, I’ll be keeping an eye on it. Carbonthief, if you manage to isolate when it stops working, feel free to share in my stead.