Running `npm install` on server deploy using forge

I can't believe that would make it timeout, really? But you probably need to run npm as sudo.

Well, I have read that here: http://stackoverflow.com/questions/33067055/laravel-forge-kills-the-deploy-script - maybe its too old

NPM recently changed to a flat folder structure which prevents it from install the same things many times, so it has become a lot faster.

You don't include node_modules in your git repo, so somehow you have to install them on the server.

Why exactly do you need to install them?

My gulp file, sass file, JavaScript file all depends on node modules for example

But you run gulp --production to compile these. Then commit the files in your public folder (a default). You don't then need to run npm install to use those modules on the production server because your files are already compiled...

I don't run gulp production on my development, I leave that for the production server. I don't have the compiled assets in my git.

I find that to be completely unnecessary. Why add an extra step when you already have everything on your machine when developing to compile the production ready assets? Nobody would get timeouts when Forge deploys then...

@JoeDawson Because of merge conflicts with autogenerated files (like minified files, soruce maps, etc). This is very annoying when merging branches (when you work in a team).

Also these auto generated files takes up unnecessary space on github, and technically it makes no sense to store these on github (as they are generated).

???The whole purpose of the npm/gulp/bower workflow is to pull down, through npm or bower, everything you need into your development environment, then once everything is working properly, compile and push to production.

looks like some people do prefer that approach, though I don't quite get it, but I read that others using forge, just ssh'd directly into the server and handled everything there, to avoid any timeout issues.

@canfiax what auto generated files are you talking about that take up unnecessary space? How big are your minified assets (css, js etc.) for you to consider then to be taking up space on Github?

@JoeDawson Whats important is: merge conflicts when working in teams.

@canfiax just run gulp --production. Assuming you don't have conflicts in your /resources/assets folders of course. Then you just push as normal with your merged branch with the production ready assets.

If Forge has timeouts for npm install then that's a Forge bug.

Besides that if you configure your deployment procedure the right way, you don't have timeouts because of npm install you site don't even have to be down during that. Only the brief moment you run gulp --production but that can be tackled too to prevent downtime or your downtime page can be a loader that automatically redirects to the main page once it's live again.

npm install
php artisan down
git pull
php artisan migrate --force
composer install
gulp --production
php artisan up

@JoeDawson your suggestion does not help prevent merge conflicts

True @JoeDawson, but that is just a hotfix to the problem. Having to gulp every time is what we are currently doing but that takes up valuable time.

Also, using versioning on your assets, you'll often have to delete your rev-manifest.json file because it is corrupted due to merge conflicts.

@canfiax I'm not sure why Forge can't handle this, but you can put your whole script in a script on the server and start it with screen so it runs in a different thread leaving Forge.

sudo apt-get install screen
# put files in maintenance
chmod +x maintenance

screen ./maintenance

I have made some additional thoughts to this subject, and I'd really argue that having the deployment server do the minification is a plus, however, I do have some obstracles in mind, that I'd like to find a creative solution to.

Upside:

• Avoid having the developer to run gulp --production and npm run prod locally. = Higher developer efficiency
• Avoid stupid merge conflicts with autogenerated files
• Avoid having github contributions data clustered with auto generated files (currently stats are very inrelaible because it does not ignore the autogenerated files)
• Lower github data usage

Downside:

• Running gulp --production and npm run prod takes a lot of time on server, and will cause a higher downtime when upgrading server. I am thinking how we could reduce deployment time to zero, e.g. with Envoyer. Eg duplicate folder, pull down, run gulp & npm, then change symlink. Is this possible? Anyone?
• Installation of a project might be a bit more complicated (you have to run npm start, gulp, etc before project is fully working). This could also be a smaller problem if you switch a lot between very different branches. Maybe you could avoid this problem on staging and local environments only, so that the gulp/npm processes are only required on production? (however, the github contribution data should also be considered into this).

I'll let you know of my progress.

Also, feel free to chip in if you have some ideas/suggestions as to how I can fix some of above struggles.

I have found a way to do this on Envoyer (run npm install on builds).

The way envoyer works is that it makes a new directory, so therefore node_modules need to be created from scratch. If you simply added npm install to deployment hooks, it would timeout depending on the size of the packages.

To avoid this, I have found this workaround:

• copy in the latest node_modules folder into the current release
• run npm install (in case anything new has been added)
• run npm modules i need to run (e.g. npm run prod, gulp --production, etc)
• copy the node_modules folder outside the current release, so on next release, we can reuse this folder.

I'm running into this problem too.

It seems like the ideal solution would be to have an intermediate build server to do the compiling and then have forge pull from a compiled package rather than from the git repo.

Hopefully Taylor can address this, as it sounds like he's working on improving Forge.

@JoeDawson Thank you your reply. I am currently thinking about how to deploy my scripts and you explained it ina way that makes sense.

I came to this thread because I am using Forge and wanted to compile upon Deploy. This makes sense though: run npm run production before pushing up, and then the assets will be compiled when they hit the server. Thanks.

@plushyObject The general recommended process for deploying on Forge is to push the code and run npm install and npm run production. I run many Forge servers and have never had an issue with doing that. No sense in making it more complicated for the next maintainer.

My view is to never include node_modules in your version control, nor compiled build files. This should all be done by the server when deploying. This keeps your commits clean, only staging the changes in your actual written code.

Then let forge do its thing, as it does so well on all my servers

For example:

cd /home/forge/sitename.com
git pull origin master --ff-only
composer install --no-interaction --prefer-dist --optimize-autoloader
echo "" | sudo -S service php7.1-fpm reload
npm install
npm run prod

if [ -f artisan ]
then
    php artisan migrate --force
fi

php artisan cache:clear
php artisan config:clear
php artisan horizon:terminate

Hopefully someone can shed some light on this for me-- couple months into Laravel/Forge, I'm noticing that when I do commit and push up, everything is there, the latest version of everything, but its like it doesn't recognize the latest version of app.js.

If I don't do anything, in a couple hours or so (or overnight), I'll come back the next morning and magically it's all fine.

Can someone explain to me why this is happening, and what the best workflow is?

The only add to this is my original install, which was just the latest Laravel, didn't really have this problem. It's only once I did a Laravel/Spark installation, that I've been having this problem with app.js which I assume is something I'm doing wrong during the compiling phase.

I'm still running into this issue and was wondering if anyone could help me figure out what's going on here.

Now, after running npm run production, committing the results, and pushing to Forge - not only will the changes not be reflected for a couple hours, but now the app won't even open on Safari!

I can get to the home page, but /login and everything else gives me a blank screen -- but only on Safari.

Does anyone have an idea of what this might be or how to fix it? Thank you