Coach do you have any kids? If not, then it will only be worse yet. I had the honor of volunteer helping Coach Gard at Rochester for nearly 18 years at all levels until roughly 8 years ago. We would "what if" and strategize ALL the time (still do, just not as much). I did struggle with the seniors last matches, but did not have much trouble any other time. Until.... My son started wrestling. I had to leave the gym after his first match and collect myself. It hit me like a ton of bricks as I had no problem in the room or our internal wrestling events.
Long story short, you are in the extreme norm with your stress and as aoberlin said, "Once you no longer feel that way you should quit". The excitement for Saturdays is what got me through a lot of weeks and still does to this day as a spectator. It doesn't go away.