Changed jobs mid-year in Japan — do I need to file?

Hi everyone,

I changed jobs this year — worked at one company until mid-year and then moved to another company for the rest of the year.

Both companies will issue me a gensen chōshūhyō in January. My new company said they can’t do the year-end adjustment since they won’t have my old company’s gensen in time.

If I just don’t do anything, will I still be okay? Is the only downside that I won’t get my refund, but there’s no legal trouble?

Thanks!