In the case of init problems it's IMO much more likely that BP was left in a wrong state than that the device is not a BusPirate. However I agree that blindly sending data to a device may not be a good idea.
On the other hand my BP appears as /dev/ttyBusPirate - the chances of something else appearing under this name are zero and I would be comfortable with OpenOCD reseting the BP for me whenever it encounters problems. How about implementing a config file directive "buspirate_auto_reset on/off" or a command line switch --buspirate-reset or -c "buspirate_reset" or something along those lines? And advise accordingly what to do in the "blah blah reset everything" message.
The "nice idea" can be borrowed from my avrdude approach to the very same problem - under normal circumstances avrdude resets from binmode upon exit, just like openocd does. If it dies unexpectedly it leaves BP in binmode, probably in SPI mode in the middle of data transfer. Again similar to OpenOCD.
When avrdude detects upon startup that BP isn't responding it calls buspirate_reset_from_binmode(), that's normally called in the exit path, and then retries initialisation. IMO OpenOCD should do the same. Perhaps first with the speed set in config file (fast or normal) and if it still wouldn't work retry once again with the other speed (ie normal or fast respectively), just in case the config has changed since the last run.
Yesterday I got my new BPv3 in the post, straight away upgraded to FW 4.2, built OpenOCD from git, wired BP with my STM32 board and got it all running on the first try! Thanks Michal for your awesome work.
One little problem though: When OpenOCD is killed with Ctrl-C it leaves BP in 1MHz binmode (?) and on subsequent run fails to run with "Error: Buspirate did not respond : ( restart everything" and BP has to be restarted. IMO in such a case OpenOCD should attempt to reset it - switch to binmode and eventualy fastmode and try a reset. Eventually use a a signal handler to catch Ctrl-C and reset BP before exiting.
[quote author="ian"] While you're in the AVRDude code, do you think it would be possible to add HVP/fuse setting to the the Bus Pirate support in AVRDude? I designed a 13volt power supply adapter to program PICs, it also has a 6pin ISP header for AVR with full control of the RESET pin (13V/H/L). [/quote]
AVRdude/BP can already set fuses. What exactly are you after here?
I think you're making it needlessly complex and slow by sending each SPI byte in halves:
0001xxxx//low 4 bits of byte + send/read byte (reply: spi byte) 0010xxxx//high 4 bits of byte (reply: 1=ok)
What if, instead, we have a command:
[font=courier:]cmd=0b0001xxxx[/font:] // start SPI transfer
where cmd&0x0F = "number of bytes - 1", i.e. 0x10 - send 1 byte, 0x1F - send 16 bytes. It doesn't make sense to send 0 bytes and it's likely that people will need to send 16 bytes more often than 15, therefore: [font=courier:]nr_bytes = cmd&0x0F + 1[/font:] That'll be followed by {nr_bytes} of binary data, reply shall be {nr_bytes} of spi bytes read in response.
For 1 SPI byte the overhead is the same as in your "split-bytes" protocol (ie 50%, 2 serial bytes for 1 SPI byte), for 16 bytes the overhead goes down to some 6% only instead of steady 50% in your protocol. That will make a real difference on the AVR programming speed.