Can we use the "EU citizen only" lane when travelling by car with one non-EU passenger inside?

I am a non-EU citizen with permanent residence in EU. My wife and my kid are EU citizens. When travelling togehter by car, are we allowed to use the "EU citizen only lane" when crossing borders, although I am not EU-citizen?