NN's typical weakness is the curse of dimensionality. In reality all? information doesn't depend on the dimensions it is mapped to. For example
- a big letter A stays a big letter A in a smaller font
- a big circular touch sensation stays a circular touch sensation even if the circle is smaller
Ways to not run into this problem is to avoid it, for example with
- early binding of raw data into higher level representations (line fitting, ...)
- other representations like in NEAT, etc.