Friday, December 30, 2016
PHP / MySql - Generate Big Reports from Big Data
Yes, that 's right indexing solved my issue. There was a table in my database which has 100K records and executing query from it was taking lots of time as there was no indexing. So I created an index on columns which are used mostly for querying that table and after creating index query was working blazing fast.
In PHP when we generate big reports it will surely take time and while working with PHP we have certain restrictions. For example, a PHP script can run for only certain time limits on any server and that limits can not be changed on most of server providers. So in case of generating big reports we surely need time as we have lots of data at backend and from it we are creating reports. So how to deal with it.
Here is the one solution I always use. You can make use of scheduled cron job. Cron jobs run in background and there is no time restrictions with it so it can run for any amount of time and it can resources as much as possible. So get the reports parameters from the user and save it in database and schedule it for certain time. For example user want to see day wise sales of products for a month. Save parameters like month in database have a scheduled cron job to read this data and start creating reports in background. Once the report is generated you can notify user about it so they can download it. Which this trick you can generated Big Reports from Big Data.