How Can I Continuously Monitor Real-Time API Updates and Take Action Based on Changing Data?
I'm interested in understanding how to stay constantly informed about real-time updates from an API source that keeps changing. For instance, with each user-specific query, I need to provide a user parameter to the API, and it returns a value that is continuously updating in real-time. I want to learn how to build a system that allows me to monitor dynamic data as it evolves and take actions accordingly. This is essential for all users who have data stored in a specific table in my database. In other words, I need to continuously monitor and analyze real-time data for all users in that table, and trigger actions based on certain conditions. Any insights or best practices for achieving this in my application would be greatly appreciated!
@camdevius Well you need to go to the API vendor and see if this is something they support. Some services provide “webhooks” where they will ping your server with updates for you to act on accordingly, without you having to constantly hammer their API with requests, which will eventually fail as any good API will have rate-limiting in place.
Alternatively, you could build something that doesn’t rely so much on data held by a third party. If they block you from accessing their API, then it sounds like your application will be rendered pretty much useless.
@martinbean Unfortunately, the API provider does not offer webhooks, so my only option is to make requests to their server. To provide more context, a user on my platform may have an account with this provider, and these accounts can be either active or inactive. I only make requests for active accounts. These accounts are stored in my database, and I pass the '$login' as a parameter for the requests. The API returns a JSON response, and I'm primarily interested in one variable, which is constantly updating from the provider (although not always). What do you think would be the best solution?
@camdevius You need to think seriously about what is an acceptable granularity for the updates.
Assuming it is not possible to know within a second for all accounts, how about a minute? What if you did not indicate the value had changed for 5 minutes, or 15, or every hour?
This is a business decision because it impacts what you offer is to your customers, but it needs to be bounded by what is practical.
@camdevius Well again, why are you building your application around data held by a third party? What happens in that third party throttles API requests, or just flat-out blocks your entirely? Your application is then useless.
