Is moving to Japan realistic for my family?

So I'm a 47 year old US citizen. My wife is a 52 year old Japanese national with permanent US residency. My kids are 16 and 13.
I can't read write or speak Japanese currently. I've been around the culture for decades so I can generally understand context and meaning, if that makes sense.
My kids can speak understand and read at a rudimentary level.
I have an associate's degree in physical therapy but have been working as a workers comp claim adjuster for the past 5 years.
I'm not sure how these skills could translate in Japan as far as a job search goes.
I've been heavily involved in shingon Buddhism as well as kendo for many years, for what that's worth.
I would obviously be able to get a spouse visa and eventually permanent residency through marriage…. But I'm struggling with the idea of employment as well as the transition for my kids.
Thoughts?
Thanks in advance 🙏

by zaike3