Will not filling in the census really be that of an issue?

Not to be that guy, but is the census really that important to fill in, as all these Reddit posts make it seem?

Last week, the census ojisan came to my house. We talked a bit and he concluded that it would be hard for me to feel in the form because lack of Japanese reading ability. He told me he would check if there was an English option, and he would come visit me again if there was. But if not, he said it would be okay and I wouldn't have to fill it in. Haven't seen him since, and I don't really mind.

Tbh I just kinda thought the census guy was just the new NHK guy (ignore it till it goes away lol) But seeing all the Reddit posts these last days, you would think the police will come arrest me if I don't fill in the census lol

I don't really have a question, and I don't think this really deserved a post, but I couldn't find the Weekly Stupid Question thread, so yeah, here we are

Edit:

The census guy just took the envelope/packet with him, so I don't have any form or QR code at my place