I have an ICOCA card do I have to buy a limited express supplement

So I am trying to figure out if the train I am planning to take to Tokyo is a limited express train. I am not sure how to check this.

To my understanding you don't have to buy a supplement for local trains when using a IC card like ICOCA (which I have). But I have to buy a supplement after entering the gates with my IC card at a machine when the train is limited express.

Can someone help me and explain how I can distinguish between the two?

And correct me if my understanding of the supplement for the limited express train is wrong.

by zhulong444