So I bought ATtiny85 recently, when I plug it to my PC it make sound like mouse has been plugin, but then it make sound like the device has been removed. It happen again and again until I remove the device. Is it the device problem or am I missing something? Can I fix it?
I'm assuming you've bought a Digispark (or compatible) ATTiny85 USB board.
The sound of a USB device inserting and removing is normal. When the digiSpark boots it listens to the USB bus for a short period so you can upload your sketches. Then it proceeds to boot your sketch.
If no sketch has been uploaded, it might just reboot until you upload one. The Digisparks I have, have all had a "blink" sketch preloaded but yours may not.
The "problem" will go away when you start running your own sketches. It will then only announce itself once and then remove itself from the USB bus, unless your sketch also communicates via the USB bus.