The problem is that you are opening the file with FILE_WRITE. That is a combined macro that is made up of:
#define FILE_WRITE (O_READ | O_WRITE | O_CREAT | O_APPEND)
The key one there is O_APPEND. In the source code (SdFile.cpp) t says:
O_APPEND - If set, the file offset shall be set to the end of the file prior to each write.
So the seek does ...
In the FAT filesystem the timestamps and other attributes are stored in the directory, not in the file. So you need to read the "file" that is the directory that contains the file you're interested in.
That "file" contains entries in a pre-defined structure. Reading each block of data using that structure gives you the information for each file in turn.