I'd like to flash the eeprom sector with content so that the eeprom already holds data when it boots the first time. How do I upload such an eeprom file like I would do with avrdude on an atmega? Is there a tool that can be installed in the arduino IDE like the data uploader?
The ESP8266 has no EEPROM.
Instead the flash memory is partitioned into different areas for different purposes. One of those areas is set aside for "EEPROM Emulation" data.
To upload to that area you first need to know where that area is in the flash memory, and that is dependent on what partitioning scheme you are using at the moment.
Once you have worked that out (look in the partition scheme files in the core for more information) you can then use
esptool to upload data to the right area of flash.