Best way to track the data communication between applications
Hello ,
We have been developing an application with microservice architecture ,for now we are using multiple lumen application for various services,Data dependecy across the services is communicated through a event service(Subscriber and Publish), Also each service has its own beanstalkd
Below is the flow
- Every service will dispatch event job to the beanstalkd of the event service
- Event service handle the queued job
- Using the Job data event service get the subscriber list( services) and sent the job data to repective service(queue the job data to beanstalkd of the service)
- Every job data sent from service is logged to MySQL table to make sure job is completed or not,It also provide convenience to retry the failed one
For eg. If a service dispatch a Job 1 record will be inserted in log table and the event service will acknowlegde the job completion by updating log record ,Event service will insert one record for each subscribers.Similary each service will acknowlegde the job completion by updating the repective log record
In above case database is accessed in the following suitivation
- While job is dispatch(1)
- Event service acknowledgement (1)
- Subcriber Job dispatch(6)(if 6 subsciber for the job)
- Each service acknowledgement(6)
So total of 14 times database connection is made for completing the single Job data from one service to another serivce This will become a big problem if number of event dispatched to event service
Please let me know best approch to overcome the above suitivation or alternative way to track the event dispatch
Thanks in advance
Please or to participate in this conversation.