The information-theoretic learning group at CWI addresses a broad range of issues related to machine learning, both theoretically and practically, and mostly from an information-theoretic perspective. In particular, the relation between data compression, generalization properties and prediction is studied in the sense of the “minimum description length” paradigm, basically a formal version of Occam’s Razor.
Researchgroups in The Netherlands
Information Theoretic Learning, CWI, Amsterdam
Recent work includes new methods for dealing with MDL/Bayesian inference when all models are wrong. In recent work, it is shown that both (standard) MDL and Bayes can behave quite badly if the model is only slightly wrong.
More practically oriented research includes similarity analysis by data compression, which can be used to determine a compression-based "distance" between any two types of objects, be they, for example, DNA sequences, literary texts or midi files. A variation of this idea has led to the Normalized Google Distance, which uses the world-wide web to determine the similarity between objects
Prof Dr P. Grunwald
Prof Dr R. Gill