So I had to back up and reinstall website that uses PHP 5.3.X and is overall deprecated. I am using Ubuntu and I wanted to install PHP 5.3.28 so this what I did:
1) Install phpbrew:
Read carefully the requirements. In my case I had to install a bunch of stuff:
https://github.com/phpbrew/phpbrew/wiki/Requirement
2) Install your version of php:
phpbrew install 5.3.28 +apxs2
Some versions are not available anymore and I don’t know the reason, sorry. You may also find that you do not have apxs2 and you won’t find it under your repository list. So try this:
apt-get install apache2-prefork-dev
3) After your successful installation under /etc/apache2/mods-available/php5.load you should be able to see this:
LoadModule php5_module /usr/lib/apache2/modules/libphp5.3.28.so
Also if you do a phpinfo(); you should the version you are after. Remember that what php cli shows you is not the same as what you apache2 uses.