Because people in each region live in different growth environments, there may be great differences in all aspects, especially in language. There are many words in Cantonese that we don't know, which are difficult to learn and require a strong memory.
The second aspect I think is the difference in tone. For example, Mandarin has only four tones, which is easy to learn and remember, while Cantonese has nine tones, which is more than twice that of Mandarin, and it is difficult to learn. The difference in pronunciation makes many people flinch from learning Cantonese, and there is also a great difference between Mandarin and Cantonese. It doesn't mean that learning Mandarin well can help you learn Cantonese well.