I’ve worked with several AVRs - the 90S2313, mega32, mega128, mega644. I have an Atmel brand serial port programmer. It costs more (like $40) and has a microprocessor inside whereas the PonyProg/Olimex is an el-cheap-o version - it depends on correct timing of the PC’s serial port control lines (rather than serial data per se). So some PCs with feeble serial ports or USB to serial won’t work. Be sure to use the COM1 port on the motherboard.
I’ve not used the tiny2313 - it’s not listed that I see among the officially supported devices for that programmer, though I’d think any MOSI/MOSO based device should work. Getting it to read device ID is step one, and has nothing to do with the C compiler or make files, assuming you use the programmer software independently.
The jumpers on the Olimex board - I think J1 should be open - no jumper, but it probably doesn’t matter. J2 - I’d leave it open. On older Olimex boards I’ve used, you must wire up jumpers on the serial TXD and RXD lines - but these come later - after you are able to program the flash memory.
With a volt meter, be sure your board’s regulator is 5V output with the programmer plugged in. Be sure the cable isn’t backwards in the ISCP connector.
That’s all I can think of. That kind of programmer can be qwirky.
Have you done a search on or post a question on the forum avrfreaks.net?