troccoli's avatar

Dependencies caches in CI

I used to cache the node_modules and vendor directories in my CI pipelines, but I did some research recently and found out that I shouldn't.

For node dependencies I should actually cache ~/.npm (and possibly ~/cache too, not yet 100% sure of that though), which I now do.

But what about composer dependencies? There seems to be consensus on not to cache the vendor directory, but what should I cache then? Some suggested ~/.composer/cache but I'm not sure.

What would the community think about this?

0 likes
2 replies
LaryAI's avatar
Level 58

When setting up CI pipelines, caching dependencies can significantly speed up your build times. However, it's important to cache the right directories to ensure efficiency and avoid potential issues.

Node.js Dependencies

For Node.js projects, instead of caching the node_modules directory, it's recommended to cache the npm cache directory. This is because node_modules can be large and may contain platform-specific binaries that could cause issues if reused across different environments.

  • Cache Directory: ~/.npm
  • Optional: You might also consider caching ~/.cache if you use tools that store cache there, like Yarn.

PHP Composer Dependencies

For PHP projects using Composer, the consensus is similar. Instead of caching the vendor directory, which can be large and contain platform-specific binaries, you should cache the Composer cache directory.

  • Cache Directory: ~/.composer/cache

Example CI Configuration

Here's a basic example of how you might configure caching in a CI pipeline for both Node.js and PHP projects:

# Example for a CI configuration file

cache:
  paths:
    # Cache npm cache directory
    - ~/.npm
    # Cache Composer cache directory
    - ~/.composer/cache

# Example job for Node.js
jobs:
  node_install:
    script:
      - npm ci

# Example job for PHP
jobs:
  php_install:
    script:
      - composer install --prefer-dist --no-progress --no-suggest

Benefits

  • Speed: Caching the package manager's cache directory speeds up dependency installation because it avoids downloading packages that have already been fetched.
  • Consistency: By not caching node_modules or vendor, you ensure that your builds are consistent and not affected by platform-specific binaries or changes in the environment.

By following these practices, you can optimize your CI pipeline for both speed and reliability.

Please or to participate in this conversation.