I've had it twice. First one was fine. You go to sleep, then you wake up, you feel kinda groggy for a bit, and someone else has to drive for you.
Second time was also fine, but it was a stomach surgery that left me feeling like throwing up forever. So I associate the smell of anesthesia with that sick-to-my-stomach feeling. But said feeling isn't normal for most surgeries; it just coincidentally is the surgery I had.

So you'll be fine.