fylzero's avatar
Level 67

Downsides of testing with SQLite?

I'm working on a large scale project with multiple devs and we were discussing the pros and cons of using SQLite :memory: tests.

  • We already know that SQLite does not return integers, which we're wondering if that could cause unexpected issues.
  • The main question that has been raised is, is testing using SQLite "a bad idea" because it is not actually testing what prod runs on? (which is MySQL)

What are your thoughts, opinions, insights? Have you used SQLite for phpunit testing and run into any downsides of doing so?

0 likes
7 replies
mabdullahsari's avatar
Level 16

There are definitely downsides to using SQLite for testing, but in my opinion one single advantage easily outweighs the disadvantages: speed

SQLite enables the possibility to utilize parallel testing. This is super important for projects that have hundreds or thousand of tests.

Some of the downsides are:

  • Weird quirks you have to account for in migrations (e.g. cannot drop multiple columns in a single DB transaction)
  • Functions that exist in MySQL cannot be used in SQLite (obviously, e.g. FIELD)
  • etc.

You will occasionally encounter infuriating SQLite limitations, but as I've said it's worth it if speed matters.

1 like
fylzero's avatar
Level 67

@mabdullahsari In your opinion... is it a bad move to use SQLite for testing? As in, could tests pass that otherwise might not on MySQL?

mabdullahsari's avatar

No, it'll be more like the opposite. MySQL can pretty much do everything that SQLite also can, but SQLite cannot and has limitations in certain areas. That is a good thing in a way, I guess.

You don't have to worry about such a thing, it won't be an issue.

We are actively using SQLite for tests and haven't had a major issue. (My experience) The test suite (aroun 2K tests) that normally should pass in 5-10 minutes takes less than a minute thanks to parallel testing and SQLite.

STEREOH's avatar

If I remember correctly SQLite doesn't support RIGHT JOINso that might be a problem for you.

mabdullahsari's avatar

@sinnbeck You're welcome. There are some gotchas however, but still the sheer amount of speed when testing is worth all of the effort. e.g. You won't be able to just call Storage::fake to test storage, see this blog post

Please or to participate in this conversation.