Hi all,
Browser testing is awesome, and very powerful. I love being able to script actions and save screenshots.
I'd love to be able to use these, maybe even in some basic CI like github workflows, but for some reason it is just incredibly unreliable for me.
I've worked through everything I can find with online guides and the LLMs etc. I pare down my browser test scripts and test adding one line at a time. I'll get some 20 line script working and then it will randomly start failing, and I'll try reducing it back to a few lines and it's still not working... very frustrating. Things like the screenshots being correct but the script exiting with auth errors, or timeouts etc.
How are people finding browser testing - is it as flaky as my experience, or is it generally reliable?
I would say that it's generally reliable, the thing is most likely that things sometimes takes a little longer to run/render that you would suspect. It could be a db query, or and API hit that causes the delay, and if something is missing when you try to access it, it might look like random errors.
I've been using Cypress and Playwright to run browser tests, and I had to add extra delays in multiple places to get the tests to green every time.
For each action that require some time to finish, you should always add some pause before doing anything else.
But always remember, the purpose of browser testing or unit testing is that something you would expect to happened.
Please or to participate in this conversation.