Level 50
Mmm.. maybe let us know what it is that's going slowly?
Nov 27, 2017
9
Level 3
60 seconds local vs 2 seconds on production
Ok I need some real expertise here.
I'm really stuck. I've spend the last year on this software and it's virtually ready for launch, except for one feature that would cement it as the best solution.
Except to test this feature, it takes 60 seconds per attempt on the local server. On production it's like 2 seconds.
This makes testing a headache to say the least.
Either I
a) code locally, then push everything up to production that I want to test (the site isn't really live yet)
This takes time, even with aliases pushing to production for every little test eventually makes you want to give up. I use forge, so I have to also, as an extra step, deploy to server.
The total time for this task is at least 90 seconds.
That's everytime I want to test something or debug.
That's annoying.
But to test it locally, it takes 60-90 seconds to run the script.
I've already put in timers to see where the bottlenecks are, I've managed to improve it significantly, but it's still 60 seconds.
So what are my options?
A) Research and find ways to improve performance on local dev? (est time: ?)
B) Just put up with pushing to production to do a test every single time. (which can add up to hundreds of times!)
C) Is there a way to use sublime to work directly on the production code? Again, I haven't really launched the site yet so no harm no foul right now to do that.
Here are my Computer specs:
OS Name: Microsoft Windows 10 Enterprise OS Version: 10.0.15063 N/A Build 15063 OS Manufacturer: Microsoft Corporation OS Configuration: Standalone Workstation OS Build Type: Multiprocessor Free Registered Owner: Windows User Registered Organization: Product ID: 00329-00000-00003-AA676 Original Install Date: 8/24/2017, 2:35:23 PM System Boot Time: 11/26/2017, 11:16:22 AM System Manufacturer: Dell Inc. System Model: Precision M6700 System Type: x64-based PC Processor(s): 1 Processor(s) Installed. [01]: Intel64 Family 6 Model 58 Stepping 9 GenuineIntel ~1200 Mhz BIOS Version: Dell Inc. A15, 8/19/2015 Windows Directory: C:\WINDOWS System Directory: C:\WINDOWS\system32 Boot Device: \Device\HarddiskVolume3 System Locale: en-us;English (United States) Input Locale: en-us;English (United States) Time Zone: (UTC-05:00) Eastern Time (US & Canada) Total Physical Memory: 16,323 MB Available Physical Memory: 8,787 MB Virtual Memory: Max Size: 20,291 MB Virtual Memory: Available: 10,930 MB Virtual Memory: In Use: 9,361 MB Page File Location(s): C:\pagefile.sys Domain: WORKGROUP Logon Server: \MONEYGETTER Hotfix(s): 5 Hotfix(s) Installed. [01]: KB4022405 [02]: KB4025376 [03]: KB4048951 [04]: KB4049011 [05]: KB4048954 Network Card(s): 2 NIC(s) Installed. [01]: Intel(R) 82579LM Gigabit Network Connection Connection Name: Ethernet Status: Media disconnected [02]: Intel(R) Centrino(R) Ultimate-N 6300 AGN Driver Connection Name: Wi-Fi DHCP Enabled: Yes DHCP Server: 192.168.2.1 IP address(es) [01]: 192.168.2.15 [02]: fe80::106c:5f10:fe3a:281f Hyper-V Requirements: VM Monitor Mode Extensions: Yes Virtualization Enabled In Firmware: Yes Second Level Address Translation: Yes Data Execution Prevention Available: Yes
Using Windows Bash. (cmder) Sublime Mysql PHP Server (artisan serve)
Production Server Spec is:
2 GB Ram, 2 Cpu Cores, 40GB SSD
I use forge and digital ocean, the most basic level, and it's just &*^( ing fast I guess? 60 seconds vs 2 seconds is no comparison.
Hope someone can help! Thanks.
Level 3
early investigation shows mysql seems to be the bottleneck... the one thing I did to bring it down from 247 seconds !! to 60 seconds, was limit how many writes I was doing to the database
but again, on production, it's 2 seconds!
Level 50
But what are you doing? Updating millions of small records, converting/importing data from a spreadsheet, creating 100's of new records in the test, or....? It's going to be hard for anyone to suggest anything without any idea what you're doing :-)
Level 3
a simple optimize table command on all tables reduced it to 20 seconds
Level 122
@ohffs its a top secret project...
What I would do is delete half of your code... and then run on something other than php server
1 like
Level 75
It is definitely a mystery.
Level 13
If the bottleneck is on your local database, maybe try connecting to the production database instead.
Level 3
so to answer my own question... the answer was A) just keep looking for bottlenecks and see if there's some fat to cut out
for the curious..
it's a scraper script that logs into hundreds websites and does stuff. fills out forms, extract profile data or status, etc. The task that was taking a long time was a simple login.
I always thought the bottleneck was maybe guzzle or proxies or something I'm super surprised to find out it was mysql all along.
I spent the day looking for bottlenecks and managed to get it down to 4 seconds vs production 2 seconds, that's good enough.
What I did: -reduced database size -ran optimize on all tables -took out unnecessary writing to the db (insert commands were especially slow)
But the best thing I did was put breakpoints in my code to identify all this in the first place.
Took a day. Not bad. I didn't think I'd find an obvious answer like this.
Thanks for the help everyone.
Level 3
@snapey local nginx is so much faster ... you're right.... duh.
I tried deleting half my code but I got an error message saying "actually, you need that code, it's not useless"
1 like
Please or to participate in this conversation.