Removed jQuery and started to load Stripe and PayPal SDKs async - my SEO is slaughtered

I am using https://web.dev/measure/ to help understand what improvements I can make to make my site load faster and rank higher. From my audit result I got the suggestion to:

• Eliminate render-blocking resources
• Reduce unused JavaScript

This past weekend I made an attempt to improve my site according to the suggestions. BUT my SEO has been totally ruined from the changes! 😭 Lost half of my organic clicks and impressions are down and falling...

I have asked other dev-friends to look through the PRs I made to make sure I made no unintended changes. Nothing like that has been found. The two changes I made was:

Change no.1

I am using both paypal and Stripe on my site and loads their SDK's in a <script>-tag. I read about the option of adding async to the script-tag to prevent render blocking. Like this (plz note the async): <script src="https://js.stripe.com/v3" async></script> and <script src="https://www.paypal.com/sdk/js?client-id=my_id&funding=credit,card" async>

Change no.2

Removal of jQuery. I wanted to remove jQuery. I hade some DOM-manipulating jQuery stuff. mainly some showing, color and opacity changes on hover. I translated all to vanilla js and it all works as intended. I then removed jQuery successfully.

After doing both changes my speed score on https://web.dev/measure got better! I was so happy. Then the day after (or maybe two days after) I started to notice the drop that is still on going. So my question to you now is: Do you know of any changed mentioned above could be the cause? Does google not like async loading of external scripts? Does google not like DOM-manipulation? If so I hade that before but it was jQuery doing it. Does jQuery do something that google likes? I am very confused and all help or thoughts would be appreciated. 🙏