Dangerous Prototypes

Dangerous Prototypes => Bus Pirate Support => Topic started by: broeggle on July 24, 2009, 04:22:18 pm

Title: Linux - Bootloader-Programmer
Post by: broeggle on July 24, 2009, 04:22:18 pm
Hi,
I'm currently trying to implement a linux - bootloader programmer for the bus pirate.
In this thead I want to report my progress - and if you would like to help, feel free to contact me.
Another reason I post my progress here is that you can tell me if I do something horribly wrong.

I checked out the svn repo and had a look at the sources.
-> The bootloader is based on the an851 - Application Note can be found here:
http://ww1.microchip.com/downloads/en/A ... 00851b.pdf (http://ww1.microchip.com/downloads/en/AppNotes/00851b.pdf)
More information could be found here:
http://www.microchip.com/Stellent/idcpl ... e=en012031 (http://www.microchip.com/Stellent/idcplg?IdcService=SS_GET_PAGE&nodeId=1824&appnote=en012031)

I also found a linux programmer
http://cockpit.varxec.net/software/pic.html (http://cockpit.varxec.net/software/pic.html)
 - but unfortunately it does not work that well - starting it without any params results in a segfault:
(gdb) r
Starting program: /data/an851/an851host
To read from flash Memory:

Program received signal SIGSEGV, Segmentation fault.
0x00007f994b1be480 in strlen () from /lib/libc.so.6

--> I will investigate these issues later on and provide a patch, but as I want to write a portable client I stick with python an pyserial, using the programms source-code as a reference.


Let's start:
* According to the source the bootloader operates at 9600 baud, 8N1  (however it should have an autobaud feature)
* According to AN p.3 the format is [...]
where STX = 0x0f , ETX=0x04 , CHECKSUM is the 2's complement of the lower byte of the sum of all data.
* On page 9 the read bootloader version information is 0x00 0x02

So my source looks like
Code: [Select]
import serial
ser = serial.Serial('/dev/ttyUSB0', 9600,bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=0, rtscts=0)

ser.write("x0Fx0Fx00x02xFEx04")

print "reading?"
print ser.read(4)

However I don't receive any data :/

Can somebody perhaps log what the original pic programmer sends over the serial line? - would be cool.
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 24, 2009, 05:23:37 pm
And here's the patch - but programming/erasing still not possible :/
Is there anything else I have to do than to put a jumper between PGD and PGC?
Code: [Select]
diff -uNpr an851host.c{,.orig}
--- an851host.c 2009-07-24 17:07:17.000000000 +0200
+++ an851host.c.orig    2009-07-24 17:06:23.000000000 +0200
@@ -260,7 +260,7 @@ void setline(int fd, int flags, int spee
        tcsetattr(fd, TCSANOW, &t);
 }

-void usage(char* argv0)
+void usage(argv0)
 {
        printf("To read from flash Memory: n");
        printf("tusage: %s --readflash --bytes <n> --startaddr <x>n", argv0);
@@ -280,7 +280,7 @@ void usage(char* argv0)
        printf("t  eg: %s --rows 3 --startaddr 0x01320nn", argv0);

        printf("nTo reset PIC via bootloader: n");
-       printf("tusage: %s --resetn",argv0);
+       printf("tusage: %s --resetn");
        close(fd);
        exit(-1);
 }
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 25, 2009, 09:07:06 am
You should start with the auto baud byte when you open the port (0x55). I'll try to make a dump of a transaction for you. A python programmer would be great.

Here a link to AN1157, it's based on AN851 but is specific to the PIC 24F:
http://www.microchip.com/stellent/idcpl ... e=en533906 (http://www.microchip.com/stellent/idcplg?IdcService=SS_GET_PAGE&nodeId=1824&appnote=en533906)

You you like SVN write access to backup your programming scripts?
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 25, 2009, 09:47:13 am
Ok, I used Virtual Serial Ports Emulator (http://http://www.eterlogic.com/Products.VSPE.html) to create a COM port pair between the FTDI driver COM port and a virtual COM port. I connected to the virtual COM port with Hercules (http://http://www.hw-group.com/products/hercules/index_en.html) set to display HEX output, and then connected to the virtual COM port with the Quick Programmer. Unfortunately, this doesn't tell us who said what. I can work on that if needed.

Here's the connection sequence:

Quote
{55}{55}{01}{01}{00}{00}{FF}{47}{05}{04}{00}{00}{B4}{04}{55}{55}{00}{02}{02}{01}{FB}{04}{55}{55}{01}{02}{00}{00}{00}{00}{05}{04}{05}{04}{00}{00}{00}{00}{00}{F5}{04}

I'm not fully versed on the actual protocol. I know the PIC echos a lot back to the quick programmer, but not every byte appears twice. The sequence starts with the auto baud byte (0x55), I checked without the Bus Pirate connected and the quick programmer actually sends both of the 0x55 bytes.

Here's the erase device sequence:
Quote
{55}{55}{03}{FD}{04}

This is the beginning of programming the 2.0 firmware:
Quote
{55}{55}{02}{FE}{04}{55}{55}{01}{40}{00}{00}{00}{00}{05}{04}{05}{04}{00}{00}{00}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{BE}{10}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{B3}{04}{55}{55}{02}{FE}{04}{55}{55}{01}{40}{80}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{BF}{04}{55}{55}{02}{FE}{04}{55}{55}{01}{40}{00}{01}{00}{FF}{FF}{00}{FF}{FF}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{BE}{10}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{EC}{82}{00}{00}{C0}{04}{55}{55}{02}{FE}{04}

Finally, here's the play button sequence to exit the bootloader:
Quote
{11}{0D}{0A}{48}{61}{63}{6B}{20}{61}{20}{44}{61}{79}{20}{42}{75}{73}{20}{50}{69}{72}{61}{74}{65}{20}{76}{32}{67}{6F}{0D}{0A}{68}{74}{74}{70}{3A}{2F}{2F}{77}{77}{77}{2E}{62}{75}{73}{70}{69}{72}{61}{74}{65}{2E}{63}{6F}{6D}{0D}{0A}{46}{69}{72}{6D}{77}{61}{72}{65}{20}{76}{32}{2E}{30}{0D}{0A}{48}{69}{5A}{3E}
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 26, 2009, 04:54:06 pm
Thanks ian - very nice.

Especially thanks for the link - there stx is defined in this AN as 0x55 instead of 0x02 :)
--> Now it's obvious why it didn't work :)
I'll continue hacking on the programmer and report my progress here - thanks!
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 28, 2009, 08:24:17 pm
hmm - I changed it - but it still does not work :/
Seems I have to read the AN more throughly  and perhaps study the code.

Do you perhaps know if the bootloader uses any hardware/software flowcontrol?

Thanks
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 28, 2009, 08:57:44 pm
hehe - no flowcontrol -> it works!

Code: [Select]
import serial
ser = serial.Serial('/dev/ttyUSB0', 9600,bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=0, rtscts=0)
ser.write("x55x550x02xFEx04")
#ser.write("x55x55x01x00x00xFFx47x05x04")
print "reading!"
x=ser.read(8);
for i in x:
    print hex(ord(i)),

print
Results in
[tt:]0x55 0x55 0x0 0x2 0x2 0x1 0xfb 0x4[/tt:]
-> the bootloader version is 0x1 0x2
which is defined in the config.h
Code: [Select]
//Bootloader Operation Configuration
#define MAJOR_VERSION       0x01    //Bootloader FW version
#define MINOR_VERSION       0x02

--> It works! yeah.
Now I can implement the rest of the bootloader :)
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 01:07:33 am
@ian:
hi - is there any posibility that I could have messed up the bootloader ?
I mean it still reports its version - however even after uploading the current firmware with windows I can't access the bus pirate's menu :/ - The update proceedure seems to work quite fine.

Are there any special values that don't get written via the programmer that are necessary? ( eeprom, flash, data memory config memory) ?

Would be nice if you/someone could give me any hints how I can revive my bus-pirate :)


Thanks,
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 08:08:19 am
I doubt it, the bootloader protects itself from overwriting. The worst that can probably happen is that you'll need to find a Windows PC and upload the firmware again.

Are you missing something (at the end of programming sequence) that writes the new bootloader delay value or reset vector value? At some point this is changed from 0xff (bootloader always on) to 0x00 (bootloader with pin jumper only).
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 08:11:09 am
Hey Ian,

as I said -  I already _have_ uploaded the new firmware using windows and it still does not work :/
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 09:00:03 am
Sorry, I missed that.

Hum, I'm not sure. I can't think of any situation that would stop the main program from executing but the bootloader will still work. If the bootloader still works then the configuration words are probably OK.

Just a few obvious thoughts, I'm sure you've already check this:
The bootloader is autobaud, runs at 9600bps, but the terminal is fixed at 115200bps.
How about the firmware version, are you using the version for hardware v2go (labeled [s:]v25-PIC Bootloader-v2.hex[/s:] edit: v25-Firmware-v2.hex)?

The only other thing I can think of is the bootloader delay or reset vector location being changed, but that should be reset when you use the known good Windows programmer.

I'll think about it some more, sorry I can pinpoint the issue.
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 09:08:07 am
no problem ian, don't worry - we'll find the solution:)
can you perhaps make a dump (i.e. read device in the windows programmer) and post it somewhere?
Then I can compare it to my dump and spot the differences and start guessing :)

Thanks,
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 09:28:09 am
Above I made a mistake and wrote the wrong firmware file name, it should be: v25-Firmware-v2.hex.

The program memory dump for v2 firmware on v2go hardware should be attached to this message.
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 12:21:39 pm
hmm bad news - I tried flashing again and had a look at the error report :/

[tt:]Verify Error at 0x4 should be: 0x82EC but read: 0xFFFFFF
Verify Error at 0x6 should be: 0x82EC but read: 0xFFFFFF
Verify Error at 0x8 should be: 0x82EC but read: 0xFFFFFF
.....
Verify Error at 0x82E6 should be: 0x6 but read: 0xFFFFFF
Verify Error at 0x82E8 should be: 0x0 but read: 0xFFFFFF
Verify Error at 0x82EA should be: 0x0 but read: 0xFFFFFF
Verify Error at 0x82EC should be: 0xFE0000 but read: 0xFFFFFF
Verify Error at 0xABFC should be: 0xF9DF but read: 0xFFFFFF
Verify Error at 0xABFE should be: 0x3F7F but read: 0xFFFFFF[/tt:]
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 01:02:01 pm
Is that for every address?

I've seen a few locations be wrong before when I forgot to erase the chip before reprogramming. Any chance that's the culprit?
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 03:06:35 pm
yeah -it's the same for every location - of course I've tried erasing the board, but the values still remain the same.

I experimented a bit using my api - and made the following observations
I can read flash. EEDATA and CONFIG - i.e. the values remain the same after wrting.
I can't write flash , EEDATA but I can write to config - however if I unplug the bp the values are resetted.

--> maybe I have enabled some write protection by setting (random) values to the config?
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 03:14:07 pm
That could explain it. There are usually various configuration bits to set block protection fuses. You might have also enabled code protect.

Can you make a dump of the chip using the quick programmer read function (it's probably just 0xffffff, huh)? Do you have any way to access an ICD2 and try to read out the chip? If you're at a dead end, let's make arrangements to mail it me to for repair.
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 03:47:53 pm
I asked one of my university teachers if he has one for me - maybe I'm lucky.

Isn't it possible to set the config vars using the windows-programmer? (unfortunately I don't have the menu for it?)
Title: Re: Linux - Bootloader-Programmer
Post by: ian on July 29, 2009, 03:53:19 pm
I'm pretty sure the firmware includes the configuration bits. Wouldn't it get set with the windows programmer on a firmware update?
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 29, 2009, 04:00:10 pm
If I read back the chip using the win-programmer - everythinf is FF FF FF 0 except 0x800 - 0x17BB - which is the bootloader+some code as far as I can guess :)
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 30, 2009, 01:01:15 am
Hey,
do you think a GALEP 5 from conitec would do the job?
http://www.conitec.com/english/galep5.php (http://www.conitec.com/english/galep5.php)

Thanks,
Peter
Title: Re: Linux - Bootloader-Programmer
Post by: Scorpia on July 30, 2009, 02:32:42 am
wow that programmer is serious overkill.

do you allready have a pic programmer? If so can you do a chip erase and verify that it is blank? if it is blank it should just be a matter of reloading the bootloader.

if you cant blank the chip then it becomes harder. :)

peter
Title: Re: Linux - Bootloader-Programmer
Post by: broeggle on July 30, 2009, 12:53:34 pm
oh sorry it's only a galep 4 - but this one should suffice too?
It's not mine, but I can borrow it from my university for a few weeks.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on July 31, 2009, 08:02:35 pm
Hi!

I'm also interested in writing a Linux programmer for the Bus Pirate. I've read over AN1157 and implemented most of a programmer. A few issues have come up though, let's start with the logs you posted (maybe you can provide better ones?):

Quote
{55}{55}{01}{01}{00}{00}{FF}{47}{05}{04}{00}{00}{B4}{04}
Read 1 word from offset 0xFF0000 - Result: 0x47 0x04 0x00 0x00 Checksum: B4
{55}{55}{00}{02}{02}{01}{FB}{04}
Ask for Bootloader Revision - Result: 0x02, 0x01 Checksum: FB
{55}{55}{01}{02}{00}{00}{00}{00}{05}{04}{05}{04}{00}{00}{00}{00}{00}{F5}{04}
Read 2 words from offset 0x000000 - Result: 0x00 0x04 0x04 0x00, 0x00, 0x00, 0x00, 0x00 Checksum: F5
These were fine and made sense and work in my programmer program.

Quote
{55}{55}{03}{FD}{04}
Erase Device again using the same parameters as last time
Since there are no arguments, I looked at the source code and the ini file you provided with the windows programmer. It would seem that with 0xAC00 addresses to erase that you would use 21 or 22 erase pages to clear it. This doesn't actually seem to be the case. Up to 43 "works" in that the serial doesn't time out. This seems to be the "right" number, perhaps the erase block size is wrong or I'm misunderstanding something? Programming the chip with my PICKit2 then erasing it does work though.

Quote

{55}{55}{02}{FE}{04}
Write to Flash again using the same parameters as last time

{55}{55}{01}{40}{00}{00}{00} ...
Read 0x40 words from address 0x000000 - Read back the written data for verification

{55}{55}{02}{FE}{04}
Write to Flash again using the same parameters as last time (not sure why this is at the end)
While the write appears to work (my code doesn't cause any timeouts) the chip isn't actually written to. Any ideas? I'm using the same addresses and data shown in this post (and trying the whole HEX file too). Along these lines, reading from certain memory addresses causes a timeout on the serial line. What might cause this?

Quote
{11}{0D}{0A}

{48}{61}{63}{6B}{20}{61}{20}{44}{61}{79}{20}{42}{75}{73}{20}{50}{69}{72}{61}{74}{65}{20}{76}{32}{67}{6F}{0D}{0A}
Hack a Day Bus Pirate v2go

{68}{74}{74}{70}{3A}{2F}{2F}{77}{77}{77}{2E}{62}{75}{73}{70}{69}{72}{61}{74}{65}{2E}{63}{6F}{6D}{0D}{0A}
http://www.buspirate.com (http://www.buspirate.com)

{46}{69}{72}{6D}{77}{61}{72}{65}{20}{76}{32}{2E}{30}{0D}{0A}
Firmware v2.0

{48}{69}{5A}{3E}
HiZ>

Heh, unfortunately, not very useful. The first character (0x11) I'm not sure about, but it doesn't seem to put the device back into user mode. The bootloader code suggests that the correct sequence is a data packet with the command 0x08, but that doesn't seem to work either.

Anyway! The programmer code is at http://dev.gentoo.org/~josejx/bp_prog (http://dev.gentoo.org/~josejx/bp_prog) and it's written in python. You'll need the pyserial module to make it work. It's certainly not finished, there's lots of error checking and bug fixing still to be done, but it's a start!

Things that should work:

Things that don't work:

Thanks for looking!
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 01, 2009, 09:55:19 am
Here's the work that's been done so far, just uploaded to the SVN yesterday:

http://code.google.com/p/the-bus-pirate ... tail?r=169 (http://code.google.com/p/the-bus-pirate/source/detail?r=169)
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 01, 2009, 10:10:34 am
Okay, would you like patches that add to this code? I'm not really much of a python programmer, I mostly work with C, but there's a bunch of features/options in my code that I could easily provide patches to implement.

Would it be possible to get better dumps? I don't think that what we've got is enough to really make this work, sorry!
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 01, 2009, 10:22:29 am
Thank you, a patch would be great, or you can have SVN access (send me an e-mail addy that's registered with google). Peter's actually maintaining the python QP code, I only have a passing familiarity with Python.

Looking at your break-down of my dumps, the "sequence to start the bootloader" is actually the output of the firmware after it boots. Great catch, I didn't pay any attention. I bet that's causing a problem because it's certainly not correct.

Do you want dumps that show who said what? A text file with a full, 100% log of a programming session? Do you have a favorite snooper utility? I'll be happy to provide anything I can.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 01, 2009, 10:33:25 am
It's pretty easy to pick out who said what, so don't go to that kind of trouble. The things I would like to see to fix my code would be:


The format you've used here is fine. If it's easier for you to provide a log of a whole programming session, it's pretty easy to work with.

Thanks for the help!
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 02, 2009, 03:58:29 pm
I'm doing some new dumps, they look exactly like the old ones. It looks like the serial splitter isn't echoing the data from the local application into the other local apps. I need to find a new way to dump the traffic, I think the previous dumps only show what the PIC echoes back to the application.

While I work on dumping the traffic, does the source (http://http://code.google.com/p/the-bus-pirate/source/browse/#svn/trunk/programmer/Source) to the quick programmer .dll help? The PICBOOT.h defines all the commands:

Quote
//Bootloader commands
#define COMMAND_READVER      0
#define COMMAND_READPM      1
#define COMMAND_WRITEPM      2
#define COMMAND_ERASEPM      3
#define COMMAND_READEE      4
#define COMMAND_WRITEEE      5
#define COMMAND_READCFG      6
#define COMMAND_WRITECFG   7
#define COMMAND_VERIFYOK   8

PICBOOT.c has simple C functions. Here's the erase PIC function:
Quote
/****************************************************************************

    FUNCTION:   ErasePIC

    PURPOSE:   Simple erase function.

****************************************************************************/
INT APIENTRY ErasePIC(HANDLE hComPort, DWORD PICAddr, BYTE nBlock, BYTE nRetry)
{
   BYTE InData[MAX_PACKET];      //Allocate for one packet
   INT RetStatus;
   
      
   //Build header
   InData[0] = COMMAND_ERASEPM;
   InData[1] = nBlock;
   InData[2] = (BYTE)(PICAddr & 0xFF);
   InData[3] = (BYTE)((PICAddr & 0xFF00) / 0x100);
   InData[4] = (BYTE)((PICAddr & 0xFF0000) / 0x10000);

   RetStatus = SendGetPacket(hComPort, InData, 5, MAX_PACKET, nRetry);

   if(RetStatus < 0) return RetStatus;

   return InData[1];
}

The command to make the PIC go is sent from PicComms.bas in the P24qp program. It looks like it's just a packet of two 0's:
Quote
Function GotoRunMode() As Integer
    ReDim DevID(10) As Byte
    Dim RetStat As Integer
    Dim picb As PIC

    DevID(0) = 0
    DevID(1) = 0
    RetStat = SendPacket(PicBootS.PortHandle, DevID(0), 2)
   
End Function

It's picked up by the sendpacket function in PICBOOT.c:

Quote
INT APIENTRY SendPacket(HANDLE hComPort,
                  BYTE PacketData[], 
                  WORD NumOfBytes)

I'll try to get a dump that shows both sides of the COM traffic.
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 02, 2009, 04:30:00 pm
I used the free MS Windows Portmon (http://http://technet.microsoft.com/en-us/sysinternals/bb896644.aspx) utility to get a full debug dump of the serial traffic. The virtual serial port monitor I used didn't echo back the local traffic, only the serial port traffic. I held this up by posting bad dumps, and I'm really sorry. Now we can really see what's going on. I also attached this as an archive too.

Connect:
Quote
17   0.00042659   P24QP.exe   IRP_MJ_WRITE   VCP0   SUCCESS   Length 9: 55 55 01 01 00 00 FF FF 04    
18   0.00398207   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
19   0.00000503   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
20   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 01    
21   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 01    
22   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
23   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
24   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: FF    
25   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 47    
26   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 05    
27   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    
28   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
29   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
30   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: B4    
31   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    
32   0.00028607   P24QP.exe   IRP_MJ_WRITE   VCP0   SUCCESS   Length 6: 55 55 00 02 FE 04    
33   0.01497844   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
34   0.00000503   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
35   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
36   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 02    
37   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 02    
38   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 01    
39   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: FB    
40   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    
41   0.00024780   P24QP.exe   IRP_MJ_WRITE   VCP0   SUCCESS   Length 9: 55 55 01 02 00 00 00 FD 04    
42   0.01398278   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
43   0.00000475   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
44   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 01    
45   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 02    
46   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
47   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
48   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
49   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
50   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 05    
51   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    
52   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 05    
53   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    
54   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
55   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
56   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
57   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
58   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 00    
59   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: F5    
60   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04
   

Erase:
Quote
0   0.00054476   P24QP.exe   IRP_MJ_WRITE   VCP0   SUCCESS   Length 9: 55 55 03 2A 00 00 00 D3 04    
1   0.85696953   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
2   0.00000643   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 55    
3   0.00000251   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 03    
4   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: FD    
5   0.00000223   P24QP.exe   IRP_MJ_READ   VCP0   SUCCESS   Length 1: 04    

Exit bootloader:
Quote
0   0.00102639   P24QP.exe   IRP_MJ_WRITE   VCP0   SUCCESS   Length 6: 55 55 00 00 00 04    
1   0.00000419   P24QP.exe   IRP_MJ_CLEANUP   VCP0   SUCCESS      
2   0.13068421   P24QP.exe   IRP_MJ_CLOSE   VCP0   SUCCESS   

The programming dump is only the first 26000 lines, but it includes about 2meg of text. It's in the archive attached to this message.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 03, 2009, 03:18:07 am
Thank you very much for the dumps! These really help. I have been picking at the source code to figure out what's going on, but having the exact values written to the device always makes things a bit clearer.

As far as I can tell, the "go" command should actually be two commands, a packet with 0x08 as the command (third byte), followed by the packet with 0's (which is the reset command). The reason why it wasn't working for me previously is that since the writing wasn't working properly, the bootloader timeout value wasn't being written. Because of this, when the finalize command was written, it was copying 0xFF to the timeout value then resetting back into the bootloader. Oops!

I'll take a look at the dumps tonight to figure out what I'm missing with the write command. I also noticed that it appears that the Microchip application writes values into the configuration words which isn't mentioned in the app notes, which may be related to the remaining issues with my app.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 04, 2009, 05:11:40 am
Some progress, but I haven't fixed everything yet! :)

The reads during errors are gone now, I believe that the read function works properly now, which is good! I was wondering though, what are the configuration words set to by default?

In my dump I made before upgrading the firmware to bootloader v2, I have:
Quote
DFF9FF00
7F3FFF00

Are these right? Also, I'm a little confused as to what the actual length of the memory is. With the PICKit2, reading out the entire flash results in addresses 0x000000 - 0x0157FF being read out. The flash files you provided also either pad to 0x0157FF or define data for those addresses. However, it appears that the windows configuration you provide only specifies 0x00ABFF as the upper memory address and doesn't appear to write any further (if I'm reading the code correctly). A size of 0x00ABFF would match with the datasheet better as well! To top it off, reading from addresses above this range using the application seems to cause it to error out... hence the confusion!

Thanks for the help. Once I get my version working, (which I've been updating as we go) I'll send patches for specific functionality so we can get the version in your SVN working as well. :)
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 04, 2009, 01:59:46 pm
Okay, never mind the last bit (although I'd still like to see the configuration registers to make sure I've got it right). I missed the part in the bootloader where each address addresses two bytes, not a single byte. This discrepancy is what was screwing up all of my address calculations and read/write functions. When I get out of work, I'll fix up the issues and hopefully we'll have a working Linux programmer. :)
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 04, 2009, 05:08:43 pm
Config words appear to be correct, save some endianness (from MPLAB):
@ABFC = F9DF
@ABFE = 3F7F

I also started a new topic to discuss the hardware I2C mode you PMed me about. There's already code for it, but I was never able to test it because I only had REV3 silicone.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 05, 2009, 12:50:12 pm
Success! The programmer at http://dev.gentoo.org/~josejx/bp_prog (http://dev.gentoo.org/~josejx/bp_prog) works for me! I've only tested with the v2 firmware, but it should work for others as well.

The programmer uses the same configuration file as the one used for the windows programmer. There are probably still bugs and rough edges, but I successfully used this program to read, erase, flash and restart my BP from Linux. It should also work in OSX and Windows, but I haven't tested it.

The next step is to break up the code and give it to you as patches against the work already done.
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 05, 2009, 01:08:40 pm
Congratulations, great work!

However you want to do it is fine. We can always host a second Python programmer if that's easier.
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 05, 2009, 01:53:48 pm
It's entirely up to you and Peter, whatever you think is better for the project. :)

If someone wants to use my programmer, the steps for doing so are actually pretty simple:

The "automatic" mode will erase then program using the specified file. You will most likely have to provide a serial port path since the .ini file indicates a port #, not a device. I'd also recommend using the -v command to verify the written rows to help ensure that the device was written correctly.
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 05, 2009, 02:25:13 pm
If you add an open source license to the source, I'll copy it to the SVN.

Can you please copy and past your terminal output as a usage example? I'll put a link on the project page.
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 05, 2009, 03:39:25 pm
I tried the programmer successfully with the Windows version of Python 2.x.

You ignore the bootloader area, what a nice touch, the Microchip programmer doesn't have this option.

The first two bytes gave an error, but I assume this is because the bootloader modifies the start address.

I also got verify failures at c04, c05, c06, a8f8, a8f9, a8fa, a8fc, a8fd, and a8fe.

Firmware seemed to work OK. I didn't remove the jumper, is this a problem? It went into the firmware just fine with the jumper attached. I can tweak the bootloader or firmware if this is causing an issue.

The update seems a bit slow. I used 115200bps, which works fine on my PIC, and it seemed to take longer than normal. There was much less frequent traffic, the TX LED was only on intermittently, instead of constantly as with the Microchip programmer utility.

I'm writing a guide for the Python programmer now. Is your serial port on Linux specified as /dev/ttySx? Do you know what it should look like for a Mac?

Can we change the name to P24qp.py to be consistent with the existing programmer?
Title: Re: Linux - Bootloader-Programmer
Post by: JoseJX on August 05, 2009, 04:35:53 pm
Glad to hear that it works on Windows too! On linux, at least with a modern kernel and a default udev configuration, it should show up as /dev/ttyUSB0 (assuming no other USB serial devices). 

One easy way to find the device is to look in the system dmesg for a line like:
Quote
usb 3-1: FTDI USB Serial Device converter now attached to ttyUSB0

As for OSX, I *think* it'll show up as /dev/cu.KeySerial1

The first two bytes will give an error because that's where the bootloader updates the address to jump to (either the bootloader code or the user mode code) afaik. I could tell it to ignore this location too if you want. I'm not sure why the other areas were not verifying properly, it does seem to work fine here. I'm not sure why the speed is lower, but since it does seem to work, I'm not too worried about it. :)

As for removing the jumper, you're right, it appears this isn't needed. I was concerned that it would start back up in bootloader mode again after the reset but this isn't the case. I've removed the warning.

I have renamed the file to P24qp.py and added a COPYING file, along with a GPL header. Hope that's okay. The latest version is located at http://dev.gentoo.org/~josejx/bus_pirate (http://dev.gentoo.org/~josejx/bus_pirate)
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 05, 2009, 05:02:32 pm
I uploaded the source:
http://code.google.com/p/the-bus-pirate ... tail?r=176 (http://code.google.com/p/the-bus-pirate/source/detail?r=176)

and corrected the file name in the help text:
http://code.google.com/p/the-bus-pirate ... tail?r=177 (http://code.google.com/p/the-bus-pirate/source/detail?r=177)
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 05, 2009, 05:14:49 pm
When the PIC exits the bootloader it doesn't reset, it just jumps to the modified firmware start address. That's why you don't need to remove the jumper.
Title: Re: Linux - Bootloader-Programmer
Post by: ian on August 06, 2009, 03:10:53 pm
I wrote a guide for the Python programmer:

http://dangerousprototypes.com/2009/08/ ... linux-osx/ (http://dangerousprototypes.com/2009/08/06/bus-pirate-firmware-upgrades-on-linux-osx/)

( ! ) Fatal error: Uncaught exception 'Elk_Exception' with message 'Please try again. If you come back to this error screen, report the error to an administrator.' in /var/www/dangerousprototypes/forum/sources/database/Db-mysql.class.php on line 696
( ! ) Elk_Exception: Please try again. If you come back to this error screen, report the error to an administrator. in /var/www/dangerousprototypes/forum/sources/database/Db-mysql.class.php on line 696
Call Stack
#TimeMemoryFunctionLocation
10.01782272640session_write_close ( )...(null):0
20.01812404232ElkArte\sources\subs\SessionHandler\DatabaseHandler->write( )...(null):0
30.01812405008Database_MySQL->query( ).../DatabaseHandler.php:119
40.06262543736Database_MySQL->error( ).../Db-mysql.class.php:273