karthikeyan's avatar

Best way to track the data communication between applications

Hello ,

We have been developing an application with microservice architecture ,for now we are using multiple lumen application for various services,Data dependecy across the services is communicated through a event service(Subscriber and Publish), Also each service has its own beanstalkd

Below is the flow

  1. Every service will dispatch event job to the beanstalkd of the event service
  2. Event service handle the queued job
  3. Using the Job data event service get the subscriber list( services) and sent the job data to repective service(queue the job data to beanstalkd of the service)
  4. Every job data sent from service is logged to MySQL table to make sure job is completed or not,It also provide convenience to retry the failed one

For eg. If a service dispatch a Job 1 record will be inserted in log table and the event service will acknowlegde the job completion by updating log record ,Event service will insert one record for each subscribers.Similary each service will acknowlegde the job completion by updating the repective log record

In above case database is accessed in the following suitivation

  1. While job is dispatch(1)
  2. Event service acknowledgement (1)
  3. Subcriber Job dispatch(6)(if 6 subsciber for the job)
  4. Each service acknowledgement(6)

So total of 14 times database connection is made for completing the single Job data from one service to another serivce This will become a big problem if number of event dispatched to event service

Please let me know best approch to overcome the above suitivation or alternative way to track the event dispatch

Thanks in advance

0 likes
0 replies

Please or to participate in this conversation.