Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.

/pdf/learning-deep-architectures-for-ai-2358znl654.pdf

Learning Deep Architectures for AI

Convergence of Probability Measures. By P. Billingsley. Chichester, Sussex, Wiley, 1968. xii, 253 p. 9 1/4“. 117s.

Convergence of Probability Measures

Fractional Differential Equations

From the Publisher:
Adaptive Filtering: Algorithms and Practical Implementation is a concise presentation of adaptive filtering, covering as many algorithms as possible while avoiding adapting notations and derivations related to the different algorithms. Furthermore, the book points out the algorithms which really work in a finite-precision implementation, and provides easy access to the working algorithms for the practicing engineer. Adaptive Filtering: Algorithms and Practical Implementation may be used as the principle text for courses on the subject, and serves as an excellent reference for professional engineers and researchers in the field.

Adaptive Filtering: Algorithms and Practical Implementation

We consider the problem of choosing a set of k sensor measurements, from a set of m possible or potential sensor measurements, that minimizes the error in estimating some parameters. Solving this problem by evaluating the performance for each of the (m k) possible choices of sensor measurements is not practical unless m and k are small. In this paper, we describe a heuristic, based on convex optimization, for approximately solving this problem. Our heuristic gives a subset selection as well as a bound on the best performance that can be achieved by any selection of k sensor measurements. There is no guarantee that the gap between the performance of the chosen subset and the performance bound is always small; but numerical experiments suggest that the gap is small in many cases. Our heuristic method requires on the order of m 3 operations; for m= 1000 possible sensors, we can carry out sensor selection in a few seconds on a 2-GHz personal computer.

/pdf/sensor-selection-via-convex-optimization-2nryt3q469.pdf

Sensor Selection via Convex Optimization

This paper presents an analysis of stochastic gradient-based adaptive algorithms with general cost functions. The analysis holds under mild assumptions on the inputs and the cost function. The method of analysis is based on an asymptotic analysis of fixed stepsize adaptive algorithms and gives almost sure results regarding the behavior of the parameter estimates, whereas previous stochastic analyses typically consider mean and mean square behavior. The parameter estimates are shown to enter a small neighborhood about the optimum value and remain there for a finite length of time. Furthermore, almost sure exponential bounds are given for the rate of convergence of the parameter estimates. The asymptotic distribution of the parameter estimates is shown to be Gaussian with mean equal to the optimum value and covariance matrix that depends on the input statistics. Specific adaptive algorithms that fall under the framework of this paper are signed error least mean squre (LMS), dual sign LMS, quantized state LMS, least mean fourth, dead zone algorithms, momentum algorithms, and leaky LMS.

http://ieeexplore.ieee.org/iel4/78/11401/00536676.pdf

Asymptotic Analysis of

The Octave Is Dead ... Long Live the Octave.- The Science of Sound.- Sound on Sound.- Musical Scales.- Consonance and Dissonance of Harmonic Sounds.- Related Spectra and Scales.- A Bell, A Rock, A Crystal.- Adaptive Tunings.- A Wing, An Anomaly, A Recollection.- The Gamelan.- Consonance-Based Musical Analysis.- From Tuning to Spectrum.- Spectral Mappings.- A "Music Theory" for 10-tet.- Classical Music of Thailand and 7-tet.- Speculation, Correlation, Interpretation, Conclusion.

Tuning, Timbre, Spectrum, Scale

A modified genetic algorithm is used to solve the parameter identification problem for linear and nonlinear IIR digital filters. Under suitable hypotheses, the estimation error is shown to converge in probability to zero. The scheme is also applied to feedforward and recurrent neural networks. >

/pdf/nonlinear-parameter-estimation-via-the-genetic-algorithm-1qgbxx7pt7.pdf

Nonlinear parameter estimation via the genetic algorithm

The selection and reproduction schemes of the genetic algorithm are modified, and a new operator called forced mutation is introduced. These changes are shown to improve the convergence of the algorithm and to lead to near-optimal sensor locations. Two practical examples are investigated: sensor placement for an early version of the space station and an individual space station photovoltaic array

Sensor placement for on-orbit modal identification via a genetic algorithm

This paper presents a method of detecting periodicities in data that exploits a series of projections onto "periodic subspaces". The algorithm finds its own set of nonorthogonal basis elements (based on the data), rather than assuming a fixed predetermined basis as in the Fourier, Gabor, and wavelet transforms. A major strength of the approach is that it is linear-in-period rather than linear-in-frequency or linear-in-scale. The algorithm is derived and analyzed, and its output is compared to that of the Fourier transform in a number of examples. One application is the finding and grouping of rhythms in a musical score, another is the separation of periodic waveforms with overlapping spectra, and a third is the finding of patterns in astronomical data. Examples demonstrate both the strengths and weaknesses of the method.

/pdf/periodicity-transforms-1nbpyqzuiz.pdf

William A. Sethares

Papers

Asymptotic Analysis of

Tuning, Timbre, Spectrum, Scale

Nonlinear parameter estimation via the genetic algorithm

Sensor placement for on-orbit modal identification via a genetic algorithm

Periodicity transforms