Ex still has my apartment key

I broke up with my boyfriend 2 weeks ago after finding out he had been cheating on me, but the main issue now is that he still has my spare apartment key.

I’ve been messaging him almost every day asking when he’s going to return it, but he either gives vague excuses or completely ignores my messages. We used to see each other almost every day for over a year, so I know he’s constantly on his phone and not “too busy” to reply.

I know the logical solution is to change the lock, but I would need to contact my landlord first, and according to my contract I technically wasn’t allowed to give my key to another person in the first place. I’m worried I could get into trouble because of that.

I also thought about asking the police for advice/help, but I’m not sure if they would do anything in a situation like this. Another thought was going to his house and demanding the key back directly, but I’m worried he could turn it around and accuse me of harassment or stalking.

Has anyone in Japan dealt with something similar before? Any advice would really help.