What’s wrong with being 派遣社員？Why everybody aim for 正社員？

I'm in the middle of job hunting process, and just noticed that I never considered an option of joining 派遣会社. From what I know, cons are

• No annual bonus from company that you're working on
• Perhaps lower salary

On the other hand, pros are

• You can decide your schedule
• So no overtime work, no pressure form power-harassing ojisans
• If you happened to enter shitty team – you can leave easy and quick
• If you want to try something different – again, you can easy change company you're working for without leaving legal body of your 派遣会社

Personally, I value my time, mental health and freedom more than money, so pros look for me pretty juicy. Still, at the moment I never been 派遣社員, so I don't know to what degree what I know is related to reality.

Also at the moment the biggest thing that I worry about is visa. As 正社員 it was easy to get 5 years one. Is there any problems with it while being 派遣社員? Does immigration service treat them the same or consider being less trustworthy?

Please let me know if it's just money or there is something else behind. If anything, I'm ordinary SE dude. Thanks to everyone for any input!