Bayesian Optimization
Suppose you have a box that generates numbers. Inside the box is some mechanism that generates the numbers when you ask for them. You give the box an X value and it spits out a y . You can't open the box and you don't have a description of the method used to generate numbers. You would like to model what is going on inside the box. In particular you would like to find the optimum (maximum or minimum) of the possible output and you would like to get an idea of the internal generating function. It's not too hard to accomplish getting the optimum and an approximation to the function if it's not expensive to generate a number from the box. You could generate 10,000 or a million X values in some range and plot the y values. Pick the maximum for the optimum and the plot gives you a picture of the generating function. What if the generating those Y values is expensive. For example if instead of a box, we had to go into a lab and do an experiment to get a value. We would...