@Mathieu.l Sorry for the late reply. I tried to reproduce your problem and created the quick start tutorial in my server's home directory (/Library/WebServer/Documents) instead of my user's home directory. This is the relevant entry from my httpd.conf:
<Directory "/Library/WebServer/Documents">
Options FollowSymLinks
AllowOverride All
Require all granted
</Directory>
Also make sure the necessary module is loaded:
LoadModule rewrite_module modules/mod_rewrite.so
You don't have to add a new directory to point to the public folder.
After changing the configuration do not forget to restart Apache (apachectl restart) or the changes won't have any effect at all.
Additionally I have performed the following changes to the project's directory to make sure that the web server's process is able to access anything and write to the storage directory:
sudo chgrp -R _www /Library/WebServer/Documents/quickstart2
sudo chmod -R 775 /Library/WebServer/Documents/quickstart2/storage
_www is the group the web server/httpd is run as.
Then I have changed the .htaccess as outlined before and used the url() helper to generate links / form action urls. Works fine, the URL is http://localhost/quickstart2/public. Since you are using Ubuntu some things might be different but the above should show you what needs to be done.