shadkamel's avatar

Database Records

i was wondering about having a bigmount of data for example in a market database, everyday the system records become bigger and bigger, so my question is is it will causes problem in feature ? for example make the system slower than before or what will happen ? :)

0 likes
7 replies
Sinnbeck's avatar

That can of course happen yes. But don't spend all of your time on that now. Most likely it will take a looong time before it happens and most likely you can solve some of it with proper indexes, or a small server upgrade

1 like
shadkamel's avatar

@Sinnbeck so it means mostly the performance of the system depends on the performance of the servers hardware and it doesn't depends of the how much record we have ?

Sinnbeck's avatar

@shadkamel the more records the slower it is. But what I am saying is to not spend 3 weeks trying to get the database to be fast with 3 million records. Maybe you never hit 3 million records at all. And maybe things are different when you do. Or you know your data better and can find a better solution

1 like
shadkamel's avatar

@Sinnbeck actually there is a project in my mind that can have a huge amount of data, i want to know what that ways that could solve my problem. indexes and server upgrade are one of the choices, what else we have please ?

Tray2's avatar
Tray2
Best Answer
Level 73

@shadkamel Like @sinnbeck, you needn't worry about that just yet, make the app work first then when you are rolling hire a database person.

Start at one.

  1. Have a good database model
  2. Have the proper indexes (not too many, not too few), use explain to find out which columns should have them
  3. Clean out old data, either delete it or move to one or more archive tables.
  4. Use views where proper to decrease the amount of data read.
  5. Read only what you need.
  6. Paginate from the database.
  7. Never filter or sort in php, let the database handle all that.
  8. Partition your tables.
  9. Get more ram and CPU for your database host.
  10. Move to clustered databases.
2 likes
Sinnbeck's avatar

@shadkamel the key word is "can"

"i have an idea for a company that drivers delivers burgers all over the world. It could have 50000 trucks. How should I structure my company to manage that many trucks?"

The answer is you don't. You start out maybe planning for 50 trucks and build up as you go

1 like

Please or to participate in this conversation.