I have written previously on this, but for those out there who are not quite sure of what DOE (Design of Experiments) and Fractional Factorial Design mean or how they apply to optimization problems,
have a look at Optimost’s page on Design of Experiments For Website Testing and their case studies.

I have no connection with Optimost and while no doubt aspects of their methodology are proprietary, the general principles of experimental design are well known in the statistical community and I have been preaching this approach for some time now.