Please consider reading the "https://learn.adafruit.com/trinket-gemma-servo-control/wiring" tutorial.
It says
It is suggested you use an external wall or battery supply and not
power the servo via the regulator. servos can draw up to 500mA and the
Trinket regulator can only source 150 milliamps (USB power generally
500 milliamps).
No matter what power supply you hook to the "Bat+" battery pin, if you hook the servo power to the "5V" pin on the trinket, the trinket's on-board 5V regulator will temporarily collapse under the load of the servo, which typically causes the system to reset over and over again.
Instead, please hook the servo power directly to the battery.
Alas, a typical partially-drained 9V battery has enough internal resistance that the battery's voltage will temporarily collapse under the load of the servo, which typically causes the system to reset over and over again.
Instead, use a better power source.
Many people, even when they intend to eventually run their system on batteries, start out using a wall-wart power supply, so they don't have to keep wondering "Is this glitch because of a bug in my software, or is the battery just going dead?".
The "Adafruit 16-Channel Servo Driver with Arduino" tutorial says:
Good power choices are:
You might also want to look at other Battery holders that hold lots of AA batteries in a convenient package.
Related:
"How to get high current from 9 volt batteries"
and
"What is the max current I could draw from a 9V battery?".