Analog neural networks for constrained optimisation with non-differentiable constraints

Abstract

In using analog neural networks for solving non-linear constrained optimization problems, the general framework provided by the Lagrange programming neural network (LPNN) approach can be applied. However, the LPNN approach is limited to the differentiable circumstance, where the objective functions and the constraints are differentiable. Since many sparse approximation problems consist of nondifferentiable functions, the traditional LPNN approach is not able to handle the sparse signals. By introducing internal state vector, the local competition algorithm (LCA) approach can solve the unconstrained sparse approximation problem. Based on the concept of LCA and sub-differential, a new LPNN model is proposed in this project for the l1-norm constrained quadratic minimization. After introducing the dynamics of the new LPNN, our question is whether the LPNN can lead to the optimal solution of the optimization problem. To answer this question, we need to investigate the properties of the LPNN. The first property is the relationship between the equilibrium points of LPNN and the optimal solution of the optimization problem. In this project, I prove that under some conditions, the equilibrium points of the LPNN are the optimal solutions of the optimization problem. The second property is the stability of the equilibrium points. I show that these equilibrium points are stable. By combining these two properties, the optimal solutions of the optimization problem are achievable by the LPNN approach. Finally the simulation result shows that the performance of the LPNN is similar to that of the conventional numerical method.