Every woman who prays or prophesies with her head uncovered dishonors her head.
The answer is best learned locally because the practice is not handled with the same visible expectation in every parish. Some communities provide scarves near the entrance. Others expect regular members to learn the practice over time.
Why the practice exists
Head covering is a bodily sign of reverence in prayer. In the Coptic tradition, bodily signs matter: standing, bowing, making the sign of the Cross, fasting, and dressing modestly all teach that prayer involves the whole person.
The practice should be taught with patience and dignity. It should lead toward reverence, not embarrassment or public correction.
What to do on a first visit
Bring a scarf if you have one and want to be prepared. If you do not have one, come anyway. Do not let uncertainty about a covering keep you from attending the service.
If you plan to keep attending, ask the priest or a trusted woman in the parish how the community practices this. That gives the custom a pastoral setting rather than making it a source of confusion.