gitwithravish's avatar

User activity logs - Database vs text files

There are many opinions about using log files over database for saving user activity logs. But in following requirements, using log files does not solve the problem.

Example

  • An e-commerce website where the rates of the product keep changing based on the user settings or manual update. These changes need to be logged. There is an active rates table and for valid reasons, these logs cannot be stored in that active_rates table.
  • Information from the logs need to be retrieved at times. For example, when a user inquires about incorrect behaviours in the product prices, we need to perform complex queries on those logs which is not possible to perform on file based logs. (Even if we try to divide the files into multiple files and folders for search and sort ease)

So, as I have mentioned, storing logs in the database is needed.

There are dozens of user_actions for which activity logs need to be saved.

Questions

  • Should I save the logs in another database ? It is expected to have thousands of entries each day and the log table is required to contain at least a year long data. Therefore the log table would have millions of rows at pick time.
  • Is one table is okay with action_type_id or making multiple tables for each action make sense?
0 likes
2 replies
sr57's avatar

Is one table is okay with action_type_id

Yes

Should I save the logs in another database ?

I should say yes, in order to have different backup policy for your main db and log db for instance

martinbean's avatar

Should I save the logs in another database ? It is expected to have thousands of entries each day

And? Database engines like MySQL and Postgres are more than capable of storing millions of rows. That’s literally what they’re made for.

Please or to participate in this conversation.