I’m stuck with the language. Any suggestions?

I started studying Japanese almost a year ago and I'm currently at a level between N3 and N4. I live in Japan, study at a school, and speak a little every day. However, I feel like I'm not making any progress. I learn new grammar every day and practice it in class or with my Japanese gf, but I can't use it in real-life situations; it's practically impossible for me.

When I read texts, I often don't understand what they say, even if they are N5 level and I know all the vocabulary and grammar.
Something similar happens to me when I listen to JLPT audios, since I don't understand what they are saying. It's funny because I understand real conversations that I hear on the street.

Some friends told me to try not to keep learning new grammar and just review what I've already learned, but doing that is sometimes difficult since we're always moving forward at my school.

What recommendations could you give me for my situation?

by GohunRay