If you have:
Job1 --dispatches--> Job2 --dispatches--> Job3 --dispatches--> Job4,
you just test it from the inside out. You write a test which verifies that Job3 dispatches Job4. Then a test which verifies that Job2 dispatches Job3, and finally a test which verifies that Job1 dispatches Job2. Then, in your integration test you just verify that Job1 is being dispatched and you can be sure that the whole Job Chain works as it should.
PS: If you dont's specify a queue, there's no need to use Queue::assertPushedOn(), it's sufficient to use Queue::assertPushed().
PPS: this really has nothing to do with a specific Laravel version or even Laravel at all, so maybe you could change the Question title to "How to test job chaining", so others can find this thread even when they're looking for other Laravel versions.