Dangerous Prototypes

Dangerous Prototypes => Bus Pirate Support => Pirate PIC programmer => Topic started by: robots on September 15, 2011, 10:36:15 am

Title: New data file loader
Post by: robots on September 15, 2011, 10:36:15 am
Hey guys!

I was quite disappointed that this project is on hold, due to the stupid hex parser i wrote :-).

So i sat down yesterday and I have written new "memory" management. It loads memory into linked list instead of continuous memory. It is able to handle sparse HEX files (eg. bootloader with fuses)

So far i have just small testing program implemented, which does reading and writing of hex files. It will need one night work to integrate it back into PiratePICprog.

I hope this will bring back the development to this project ;-)
Title: Re: New data file loader
Post by: ian on September 22, 2011, 05:26:21 am
Thanks for taking a look at it. The non-continuous hex parsing has been a hangup.
Title: Re: New data file loader
Post by: robots on September 23, 2011, 12:05:25 am
Hi,
I just tested my creation, and seems to work without problem. :-)
I have made quite a lot changes to the piratepicprog - changes to some structures (iface), fixes of indentation, etc etc...

I am still struggling with some integration of "fuses". The SW needs to know where the fuses are. I thought that fuses are on the same place for one family, but 18f2xFxx family does have fuses on different places.

Few questions come to my mind:
- the fuses should probably be per part as opposed to per family.
- how should fuses be preserved?

on this same device the fuses are part of the last page. How do I verify that the hex file doesn't contain bogus fuses ? Or do I just check the last page and if few last bytes are missing (fuxes) replace them from device flash ?
Title: Re: New data file loader
Post by: robots on September 23, 2011, 12:07:10 am
I am trying to come up with some universal write/read routines that can be used for any pic family/chip only by replacing the page read/write functions. I will probably need more write/read functions for different memory types (eg. eeprom, flash, .. etc).
Title: Re: New data file loader
Post by: ian on September 25, 2011, 09:46:58 am
Quote
on this same device the fuses are part of the last page. How do I verify that the hex file doesn't contain bogus fuses ? Or do I just check the last page and if few last bytes are missing (fuxes) replace them from device flash ?

I think it will probably need to be info that is part of the PIC info struct. Maybe location and length of fuses? Some 24Fs have 2 or 3 words, some have up to 5. On new chips it is always the last few words of the flash space, on 18Fs and below it is typically in a special location.
Title: Re: New data file loader
Post by: tayken on October 03, 2011, 04:36:24 pm
Thanks for bringing this up again! I've tried to add 18F r/w support but gave up after a while. I wanted to get back to it again and having someone else on board (that guy being the original creator is awesome) really rocks!

[quote author="ian"]I think it will probably need to be info that is part of the PIC info struct. Maybe location and length of fuses? Some 24Fs have 2 or 3 words, some have up to 5. On new chips it is always the last few words of the flash space, on 18Fs and below it is typically in a special location.[/quote]

I agree with Ian, putting location and length in the struct works best I guess. One can easily get that info from the datasheet when adding a new PIC to the list.

[quote author="robots"]
on this same device the fuses are part of the last page. How do I verify that the hex file doesn't contain bogus fuses ? Or do I just check the last page and if few last bytes are missing (fuxes) replace them from device flash ?[/quote]

I have no idea about this, but I wonder how does PICkit2 program does it. I can do a few tests with that and maybe report you the findings? I still have to install some new programs to my new laptop + semester began today, strange times semester beginnings are.
Title: Re: New data file loader
Post by: Sjaak on October 03, 2011, 04:44:09 pm
[quote author="tayken"]
[quote author="robots"]
on this same device the fuses are part of the last page. How do I verify that the hex file doesn't contain bogus fuses ? Or do I just check the last page and if few last bytes are missing (fuxes) replace them from device flash ?[/quote]

I have no idea about this, but I wonder how does PICkit2 program does it. I can do a few tests with that and maybe report you the findings? I still have to install some new programs to my new laptop + semester began today, strange times semester beginnings are.[/quote]

I think the MPLAB IDE takes care of validating the fuses. How about adding a default one (or a fuses-mask) to the pic-struct and display a warning when the one in the .hex is different?
Title: Re: New data file loader
Post by: robots on October 05, 2011, 12:35:42 am
Hi guys,

I have just commited HUGE changes to the piratepicloader. (HUGE = svn diff was about 2000 lines) Changes are:
- new data loader (fixed bug in hex file generator)
- new memory storage (memory.[hc]), by using simple link-list, and memory cells of size of flash page
- redesigned pic/family/protocol tables to be effective and easier to maintain
- redesigned iface interface to be less generic, and to create less compile time warnings :-)
- created common functions for flash reading and writing (these should be used instead of those "protocol" specific hacked ones :-) )
- created function to preserve fuses, between full memory erase - this is going to need some option on command line.

I haven't updated the "codeblocks" project files, you will probably have to add memory.[hc] to the project.

Also there are tons of TODOs in the code. The code as such is "working". I have successfully erased and programmed my OLS, with full erase. Some debug prints are left in the code (bad me).

What is not working is the binary file generation from read flash, sorry. Only output possible is HEX so far. Will fix soon.

PIC_WriteMemory function is written too simplistic. It should really check whether the address is flash/fuse/eeprom, and call appropriate function according to that. But works for 18f24j50.

Probably last thing that comes to my mind is the coding style... could it be kept consistent ?:-) (tab as tab, not spaces, etc, etc) I am sure that there are good editors that can manage that.
Title: Re: New data file loader
Post by: tayken on October 05, 2011, 07:07:29 pm
OK, I will check this out tomorrow when I have more time. Just updated the trunk and I can see your changes. So on Linux if I go to software folder and run Make, I get the picprog executable. I will do the necessary changes for adding 18f4550 support on files in this folder, is it OK?
Title: Re: New data file loader
Post by: tayken on October 05, 2011, 07:21:24 pm
Did a quick test just now. I was not able to use -I command, it gave me an error. Will add 18F4550 support first (I use this a lot) then if I have time, I'll check out -I command error.
Title: Re: New data file loader
Post by: robots on October 05, 2011, 08:36:33 pm
-I command is a special one, there is no -I command :-)

It is definitely missing part in tables.
Title: Re: New data file loader
Post by: ian on October 06, 2011, 01:05:41 pm
Thanks for the update. I'll post this up, there are a few dozen HVP adapters out there. Since you posted this we have sold 2 of the last 3 (no previous sales for months), there must be interest.
Title: Re: New data file loader
Post by: robots on October 06, 2011, 05:53:47 pm
It seems that all are gone. Development keeps the business  going :-)
Title: Re: New data file loader
Post by: robots on October 06, 2011, 07:43:29 pm
I have just committed new changes:
- fixed bug in hex file loader
- fixed memory comparation
- finished bulk memory loading (which was missing and caused data being trashed)
- added binary writer
- added safe_malloc, to simplify memory allocation checks
- changed scope of interface functions interface scope
Title: Re: New data file loader
Post by: tayken on October 07, 2011, 08:15:09 pm
Added 18F4550 support and committed the change. It was really easy as the device is almost similar to 2550 (only id is different, realised this when checking out the datasheet)

Then I did some tests to test it out under Linux (Ubuntu 11.04). Steps are:
- Wrote a hex file (written.hex) with verification (verification failed) using BP
- Read with BP to a hex file (bp_read.hex)
- Read with PicKIT2 to a hex file (pk2_read.hex)

There are differences in all the files but I guess it works. I couldn't understand the verification error but one can see the differences between the written file and read file. Did programming with PicKIT2 and verification passed, the read file is also different from the written file but somehow the program manages to check. Just for your info, all the hex files I used are in the attached zip file.
Title: Re: New data file loader
Post by: robots on October 07, 2011, 10:52:27 pm
Ok, so ... i will try to make the verification errors bit more verbose.

your files are strange, both "read" are exactly the same (content wise). But the written is differs in one single bit  (0x80 != 0x00)

I have small program for comparing hex files using the "memory loader" i have written. So it loads both HEXes into memory and compares them. I think i will improve it and make it public ;)
Title: Re: New data file loader
Post by: tayken on October 23, 2011, 05:24:24 pm
I've added the PIC device used in v4 for programming trial. However both me and Sebastian were not able to program it, you can see his report here (http://http://dangerousprototypes.com/forum/viewtopic.php?f=28&t=2465&p=29480#p29458). Any thoughts? Write seems OK, maybe I've made a mistake in the PIC device settings?
Title: Re: New data file loader
Post by: robots on November 01, 2011, 06:34:07 pm
Sorry for replaying soo late :-)

I have noticed that the HEX contains 0xff 0xff 0xff 0x00  for any empty page, that is kinda weird. Is the flash organized in 24-bit words separated by 8bits ?
Title: Re: New data file loader
Post by: tayken on November 01, 2011, 06:51:45 pm
[quote author="robots"]Sorry for replaying soo late :-)

I have noticed that the HEX contains 0xff 0xff 0xff 0x00  for any empty page, that is kinda weird. Is the flash organized in 24-bit words separated by 8bits ?[/quote]
That one was from a 18F4550. I looked up from the datasheet: Word size is 16-bits. I'll do the same tests tomorrow. Maybe I've messed sth up during that time, not sure.

Good news is that I was able to program BPv4. Seems that Config bits were stored in bootloader and that was the problem. :)
Title: Re: New data file loader
Post by: Sjaak on November 01, 2011, 08:13:32 pm
[quote author="robots"]Sorry for replaying soo late :-)

I have noticed that the HEX contains 0xff 0xff 0xff 0x00  for any empty page, that is kinda weird. Is the flash organized in 24-bit words separated by 8bits ?[/quote]

This is the way it is stored. Only the first 24 bits are used. It took me quite a while to figure this out when I wrote the bootloader updater v3->v4. It also gives funny translations between bytes, words, long words, programwords etc. :)
Title: Re: New data file loader
Post by: robots on November 01, 2011, 10:44:23 pm
This is probably the only thing i hate about pics - their funky byte organization
Title: Re: New data file loader
Post by: robots on November 26, 2011, 12:40:37 pm
[quote author="robots"]I have small program for comparing hex files using the "memory loader" i have written. So it loads both HEXes into memory and compares them. I think i will improve it and make it public ;)[/quote]

I have committed few changes into the svn. Some bug fixes in my memory code (not too severe) and I have also added the tool for comparing 2 files (either bin or hex). Manual to this tool is inside compare.c file. Have fun.