The hosts file you mention, is it the one from the homestead environment or your host environment?
Pointing your vhost directly to your homestead IP should let you use it without port, you could also try using xip.io so you don't have to modify your hosts file.
After 5 years... :) I had the same issue... Lindstrom answer is correct. You have just to run "vagrant reload" after the changes. I hope it may help someone with the same problem.