The problem is that there is often a continuum of dialects where dialects that are near each other in this continuum are mutually intelligible, but those that are further apart are not. Where do you set the boundary between languages and dialects there.

P.S.: I'm not a linguist but because of this fact I would consider to define languages and dialects in a quite different way (the following concepts and description is my personal invention and not taken from some linguistics literature).

First we define the concept "speak-kind" as a generalization of language, dialect etc. (we use this word so that we aren't burdened by the fact that generally a dialect is defined to be some variety of a language etc.).

Now consider the graph where the vertices are the "speak-kind"s. This graph has an edge if two "speak-kind"s are mutually intelligible. Now we can define the concept of the "cont[inuum]-language": a "cont-language" is defined to be a connected component in this graph.

In this sense Bavarian and Dutch would be of the same "cont-language" (exercise: How would the connecting path in the graph probably look like?), though probably not mutually intelligible, but Hungarian and English would belong to a different "cont-language".