mathematical-optimization
How to convert quadratic to linear program?
Models in this form are actually called bilinear optimization problems. The typical approach to linearizing bilinear terms is through something called the McCormick envelope. Consider variables x and y, where you want x*y in the objective of your maximization problem. If we assume x and y are bounded by xL <= x <= xU and … Read more
scipy.optimize.leastsq with bound constraints
scipy.optimize.least_squares in scipy 0.17 (January 2016) handles bounds; use that, not this hack. Bound constraints can easily be made quadratic, and minimized by leastsq along with the rest. Say you want to minimize a sum of 10 squares Σ f_i(p)^2, so your func(p) is a 10-vector [f0(p) … f9(p)], and also want 0 <= p_i … Read more
mathematical optimization library for Java — free or open source recommendations? [closed]
A good answer is dependent on what you mean by “convex” and “more general” If you are trying to solve large or challenging linear or convex-quadratic optimization problems (especially with a discrete component to them), then it’s hard to beat the main commercial solvers, gurobi, cplex and Dash unless money is a big issue for … Read more
Which is better way to calculate nCr
Both approaches will save time, but the first one is very prone to integer overflow. Approach 1: This approach will generate result in shortest time (in at most n/2 iterations), and the possibility of overflow can be reduced by doing the multiplications carefully: long long C(int n, int r) { if(r > n – r) … Read more
How to display progress of scipy.optimize function?
As mg007 suggested, some of the scipy.optimize routines allow for a callback function (unfortunately leastsq does not permit this at the moment). Below is an example using the “fmin_bfgs” routine where I use a callback function to display the current value of the arguments and the value of the objective function at each iteration. import … Read more
Fit plane to a set of points in 3D: scipy.optimize.minimize vs scipy.linalg.lstsq
Least squares (scipy.linalg.lstsq) is guaranteed to converge. In fact, there is a closed form analytical solution (given by (A^T A)^-1 A^Tb (where ^T is matrix transpose and ^-1 is matrix inversion) The standard optimization problem, however, is not generally solvable – we are not guaranteed to find a minimizing value. However, for the given equation, … Read more
How to interpret loss and accuracy for a machine learning model [closed]
The lower the loss, the better a model (unless the model has over-fitted to the training data). The loss is calculated on training and validation and its interperation is how well the model is doing for these two sets. Unlike accuracy, loss is not a percentage. It is a summation of the errors made for … Read more
Why should weights of Neural Networks be initialized to random numbers? [closed]
Breaking symmetry is essential here, and not for the reason of performance. Imagine first 2 layers of multilayer perceptron (input and hidden layers): During forward propagation each unit in hidden layer gets signal: That is, each hidden unit gets sum of inputs multiplied by the corresponding weight. Now imagine that you initialize all weights to … Read more
I am looking for a simple algorithm for fast DCT and IDCT of matrix [NxM]
Here is mine computation of 1D FDCT and IFDCT by FFT with the same length: //————————————————————————— void DFCTrr(double *dst,double *src,double *tmp,int n) { // exact normalized DCT II by N DFFT int i,j; double nn=n,a,da=(M_PI*(nn-0.5))/nn,a0,b0,a1,b1,m; for (j= 0,i=n-1;i>=0;i-=2,j++) dst[j]=src[i]; for (j=n-1,i=n-2;i>=0;i-=2,j–) dst[j]=src[i]; DFFTcr(tmp,dst,n); m=2.0*sqrt(2.0); for (a=0.0,j=0,i=0;i<n;i++,j+=2,a+=da) { a0=tmp[j+0]; a1= cos(a); b0=tmp[j+1]; b1=-sin(a); a0=(a0*a1)-(b0*b1); if (i) … Read more