rohansinghrawat's avatar

Deadlock occured but can't get the origin of it

Hey everyone,

I've encountered a deadlock issue in my project recently and I'm seeking some insights into its origin.

The scenario is as follows:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction (SQL: update `pre_txns_2403` set `status` = SUCCESS, `response_code` = 0, `response_msg` = SUCCESS, `ipay_error` = TXN, `ipay_desc` = Transaction Successful, `sp_transid` = 407508316340, `opr_id` = 407508316340, `res_dt` = 2024-03-15 08:00:14, `optional1` = 607264, `optional2` = 9935558558, `optional3` = CENTRAL BANK OF INDIA, `optional4` = 5044.00, `optional5` = e46944ca4251477b92842311c55f5e1e, `optional6` = , `optional7` = , `optional8` = , `optional9` = , `remarks1` = 316340, `remarks2` = , `remarks3` = , `remarks4` = YWY3OTAzYzNlM2ExZTJlOdjuLnfS78RXTDsN1fnMZcU=, `remarks5` = 3991805, `remarks6` = 431501, `remarks7` = 64a43d40-642e-4fa1-84d1-3f7034aa34c2, `remarks8` = , `remarks9` = , `remarks10` = , `remarks11` = , `remarks12` = , `remarks13` = , `remarks14` = , `remarks15` = LARAVEL where `ipay_id` = P240315080011LZNVJ)

Moreover, I want to shed light on the trigger point of the error:

 try {
            DB::connection($db . '__write')->beginTransaction();

            $transaction = DB::connection($db . '__write')->table($table)->where('ipay_id', $validatedData['ipayId']);
            $transaction->update($data);

            DB::connection($db . '__write')->commit();
        } catch (\Exception $e) {
            return \Ipay::response([
                'statusCode' => 'ISE',
                'internalCode' => $e->getMessage(),
                'internalCodeAppend' => '#UTU1',
            ]);
        }

My main confusion stems from the fact that when I replicated the entire operation in my local environment and ran the same script multiple times (e.g., 10 times in 1 second), I didn't encounter any issues at all. This discrepancy between the local and production environments leaves me puzzled as to why the deadlock occurred.

Unfortunately, the error message doesn't provide much information, and I'm struggling to pinpoint the exact cause of the deadlock. I'm unsure if it was indeed caused by the query or if there might be other factors at play.

I'm reaching out to the community to gather ideas or suggestions on how to trace and identify the root cause of this issue. If anyone has insights or suggestions on how to troubleshoot deadlock scenarios effectively or how to analyze the sequence of queries leading up to the deadlock, I would greatly appreciate your input.

Additionally, if needed, I can provide a list of the queries that my script executed sequentially for further context.

Thank you all for your help and support.

0 likes
12 replies
Tray2's avatar

Deadlocks are unfortunately quite common in the database when you use transactions.

From what I can read in your code you do it in three step.

  1. Start transaction
  2. Do the update
  3. Commit the changes.

I never used transactions in Laravel, but I work with transactional databases, and we always do it in four steps.

  1. Start the transaction
  2. Select the record for update
  3. Update if previous select was successful
  4. Commit the changes.

So my guess is that someone else has taken out a lock on that particular record that you are trying to update.

1 like
rohansinghrawat's avatar

@Tray2 Thanks for your reply but as you can see my code snippet , I have done all the steps as you mentioned

Start transaction (BeginTransaction())
Do the update (Db Query)
Commit the changes. (DB::Commit())

even after this I am getting Deadlock ..

Tray2's avatar

@rohansinghrawat Since you hare handling stuff manually you should try to lock the record, which you don't do in your code.

DB::transaction(function() {
  $record = Model::where('id', 1)
      ->lockForUpdate()
       ->get();
    //You need to handle the exception thrown if the record is already locked.
   $record->field = 'New value';
   $record->save();  
});
1 like
Snapey's avatar

does that data look like what you might write in the function?

AFAIK starting a transaction does not provide any restricted access to the data therefore should never be the cause of a deadlock. In this case your transaction does nothing because either the database was updated or it wasnt, and if it wasnt then there is nothing to rollback.

Perhaps you have a lock further out, where you get the record to be updated?

rohansinghrawat's avatar

@Snapey So basically Here transaction is doing nothing , this was my concern too but question arises what could be the exact issue here , is my record is getting locked somewhere else and code tries to update it ??? because as per my case

1. this record is created first 
2. some business logic 
3. vendor's response  
4. update the record created in first step (with whatever response ,DEADLOCK appeared here)

Could you please suggest any ideas or leads or any additional information regarding this

Snapey's avatar

@rohansinghrawat is there any chance that you could be processing the same ipay_id from two different threads, eg if user double-clicks button

rohansinghrawat's avatar

@Snapey that's the best part here ...as I am aware of my whole process ,

the query I provided executes first then I also update the same record with ipay_id after the success of my first query result ..but it failed on first attempt ...

I could also provide my whole script queries .. step by step executing

rohansinghrawat's avatar

@Snapey just one more clarification is Table locked or particular row?? because numerous operations are working as usual

Tray2's avatar

@rohansinghrawat That depends on your code, if you do a SELECT * FROM table then the whole table, but if you do a SELECT * FROM table WHERE id = 1 then only that single row will be locked.

1 like
rohansinghrawat's avatar

@Tray2 We've implemented a master-slave concept in our database architecture, segregating read and write operations into separate databases. Specifically, we utilize DB__read for read operations and DB__write for write operations. The issue at hand pertains to operations conducted within the DB__write database.

Within the pre_txns_2403 table, our operations are streamlined into two distinct workflows: one for fetching transactions and another for creating and updating them. The specific scenario in question involves the creation of a record with an ipay_id of 'XXXXX', followed by the subsequent update of the same record with additional information, as outlined previously.

Your continued support is greatly appreciated. Thank you!

Tray2's avatar

@rohansinghrawat When using transactions and you are updating a record, you should always do a select for update first, otherwise you will create deadlock situations in the database.

1 like

Please or to participate in this conversation.