I recently bought a USBASP programmer from e-Bay, the one pictured here
It seemed to work just fine when I was programming attiny4313′s or atmega’s, but would fail on verification when programming attiny85′s (and probably other small/slow attiny’s).
Setting the fuses would work, but writing flash would fail consistently.
The trick to resolve this was to update the firmware to the 2011 version of USBASP, which can be found at http://www.fischl.de/usbasp/.
To program the USBASP itself, you need a second programmer (I used a usbtiny) and you need to jumper J2 (which are the 2 solderpoints next to the leds).
After upgrading the firmware, use avrdude with the ‘-B 10‘ option to write to slow devices. This option is not needed with faster (1.5Mhz and up) devices.