Skip to main content
Topic: (Intel) .HEX reader/writer/editor (Read 8325 times) previous topic - next topic

(Intel) .HEX reader/writer/editor

[quote author="ian"]On a related note: I've never been able to find a good open source intel .HEX editor, does anyone know of one? (not a HEX editor that shows the values in any file, but one that loads intel HEX and displays it. maybe allows editing, and resaves with updated checksums)[/quote]I wrote some standard C code which loads both Intel HEX and Motorola S-Record format files and translates to a memory map. Since most of the MCU chips have limited memory, it's not too hard to simply map the entire address space.

My primary purpose was to support PIC18 and PIC16 disassemblers which decode the register names and other magic addresses, and thus are better than either of the MPLAB disassemblers.  I developed this .HEX library with callback support, so that pieces could be processed as they are read in, and thus the disassembler simply hooks in to the callback.  You could say that the disassembler is a fancy display of the .HEX data, since it would be far easier to just dump the hex values.

As for your needs, my first thought was that you could save yourself the trouble of writing a HEX editor by simply writing command-line translation tool that would turn a .HEX file into a raw .DAT file, and also the reverse.  Then you could use your favorite hex editing program on the .DAT file before translating it back to Intel .HEX.  But the real trouble is that you can have entire pieces of memory that are missing from a given .HEX file, so an editor/writer would ideally track the pieces of memory that are used and not create bloated image full of zeroes (or 0xFF).  I'm not quite sure how to support this.

One thing I want to avoid is starting with a small firmware which then expands to fill the entire Flash with zeroes.  Even though it's unlikely that I would ever exhaust 10,000 Flash writes, I still don't want to wait for empty memory to program.

My second thought was that a better tool might be simply a .HEX merge utility. It would never create output for missing sections of memory, but would merely echo the two input files into a single output file.  For this to work, it would require a precedence - perhaps the later files on the command line would overwrite earlier files on the command line, just like programming would.

To this end, I believe that a .HEX file is required to have all memory in order, even if addresses are skipped, but I am not sure about my assumption here.  If that assumption is correct, then the utility could read a line from each file, then toss any lines which are overwritten, and finally write out a single line for each address.  It might be a bit tricky, since some files on the command line might skip large sections of memory and then overwrite other sections.

Beyond simple merge, perhaps a sparse memory tracking system could be developed so you could load multiple .HEX files in at once, then write out the results while skipping addresses which were never loaded.  It would only take a 16K bitmap to track memory usage at the byte level for a PIC with 128K Flash.

One potential issue is that it would be much better to track the PIC model number, or at least specify the Flash size and block erase size, so that the resulting .HEX file would be guaranteed to fit in the PIC.  With tracking of the model number, warnings could be issued by the .HEX editor whenever an address was seen which exceeds the Flash address space or the Configuration memory addresses.

Re: (Intel) .HEX reader/writer/editor

Reply #1
[quote author="rsdio"]
[quote author="ian"]On a related note: I've never been able to find a good open source intel .HEX editor, does anyone know of one? (not a HEX editor that shows the values in any file, but one that loads intel HEX and displays it. maybe allows editing, and resaves with updated checksums)[/quote]I wrote some standard C code which loads both Intel HEX and Motorola S-Record format files and translates to a memory map. Since most of the MCU chips have limited memory, it's not too hard to simply map the entire address space.

My primary purpose was to support PIC18 and PIC16 disassemblers which decode the register names and other magic addresses, and thus are better than either of the MPLAB disassemblers.  I developed this .HEX library with callback support, so that pieces could be processed as they are read in, and thus the disassembler simply hooks in to the callback.  You could say that the disassembler is a fancy display of the .HEX data, since it would be far easier to just dump the hex values.

As for your needs, my first thought was that you could save yourself the trouble of writing a HEX editor by simply writing command-line translation tool that would turn a .HEX file into a raw .DAT file, and also the reverse.  Then you could use your favorite hex editing program on the .DAT file before translating it back to Intel .HEX.  But the real trouble is that you can have entire pieces of memory that are missing from a given .HEX file, so an editor/writer would ideally track the pieces of memory that are used and not create bloated image full of zeroes (or 0xFF).  I'm not quite sure how to support this.

One thing I want to avoid is starting with a small firmware which then expands to fill the entire Flash with zeroes.  Even though it's unlikely that I would ever exhaust 10,000 Flash writes, I still don't want to wait for empty memory to program.

My second thought was that a better tool might be simply a .HEX merge utility. It would never create output for missing sections of memory, but would merely echo the two input files into a single output file.  For this to work, it would require a precedence - perhaps the later files on the command line would overwrite earlier files on the command line, just like programming would.

To this end, I believe that a .HEX file is required to have all memory in order, even if addresses are skipped, but I am not sure about my assumption here.  If that assumption is correct, then the utility could read a line from each file, then toss any lines which are overwritten, and finally write out a single line for each address.  It might be a bit tricky, since some files on the command line might skip large sections of memory and then overwrite other sections.

Beyond simple merge, perhaps a sparse memory tracking system could be developed so you could load multiple .HEX files in at once, then write out the results while skipping addresses which were never loaded.  It would only take a 16K bitmap to track memory usage at the byte level for a PIC with 128K Flash.

One potential issue is that it would be much better to track the PIC model number, or at least specify the Flash size and block erase size, so that the resulting .HEX file would be guaranteed to fit in the PIC.  With tracking of the model number, warnings could be issued by the .HEX editor whenever an address was seen which exceeds the Flash address space or the Configuration memory addresses.
[/quote]

I believe it isn't necessary to have all memory in order, AFAIK the fileformat is developed for random access memory. Here is the specification from intel: http://microsym.com/editor/assets/intelhex.pdf  A quick search did show it up either.

Not writing the entire flash doesn't matter imo, since the same blocks will always be written and this is where the wear is going to happen (and does do the most damage :)) However if you output 0xff for empty spaces it won't harm the chip extra, since all '1's mean no program that byte (erasing flash is setting all bits to '1'). That transisions would wear the flash down.

btw I came across: http://www.x-ways.net/winhex/ but haven't looked at it at all :D

Re: (Intel) .HEX reader/writer/editor

Reply #2
[quote author="Sjaak"]I believe it isn't necessary to have all memory in order, AFAIK the fileformat is developed for random access memory. Here is the specification from intel: http://microsym.com/editor/assets/intelhex.pdf  A quick search did show it up either.[/quote]Thanks for posting that link here. I went to download it and found that I already had the file but had forgotten!  I must have used it years ago when writing my .HEX reader. The only mention of order dependence is that segment addresses affect all subsequent records until the next segment address is encountered. I agree with you that there seems to be no requirement that addresses appear in order.

Quote
Not writing the entire flash doesn't matter imo, since the same blocks will always be written and this is where the wear is going to happen (and does do the most damage :)) However if you output 0xff for empty spaces it won't harm the chip extra, since all '1's mean no program that byte (erasing flash is setting all bits to '1'). That transisions would wear the flash down.
If the size of the memory can be predicted before loading the .HEX file, then a memory image could be initialized with 0xFF before loading.  I agree that it's far better to program 0xFF than 0x00, unless there are memory types which default to 0x00 (NAND?).

Even though wear is not an issue, I still prefer the faster speed afforded by programming only the needed data and leaving the rest blank by skipping it entirely.

Re: (Intel) .HEX reader/writer/editor

Reply #3
[quote author="rsdio"]
Quote
Not writing the entire flash doesn't matter imo, since the same blocks will always be written and this is where the wear is going to happen (and does do the most damage :)) However if you output 0xff for empty spaces it won't harm the chip extra, since all '1's mean no program that byte (erasing flash is setting all bits to '1'). That transisions would wear the flash down.
If the size of the memory can be predicted before loading the .HEX file, then a memory image could be initialized with 0xFF before loading.  I agree that it's far better to program 0xFF than 0x00, unless there are memory types which default to 0x00 (NAND?).

Even though wear is not an issue, I still prefer the faster speed afforded by programming only the needed data and leaving the rest blank by skipping it entirely.
[/quote]

i don't think 0x00 is the default for any nonvolatile memory. You can safely leave 0xFF out of the final .hex to speed things up.

Re: (Intel) .HEX reader/writer/editor

Reply #4
[quote author="Sjaak"]i don't think 0x00 is the default for any nonvolatile memory. You can safely leave 0xFF out of the final .hex to speed things up.
[/quote]There's a really good idea. The only remaining challenge is a way to determine the size of the Flash being programmed before reading in one or more .HEX files for editing, merging, or maybe even disassembly.

Re: (Intel) .HEX reader/writer/editor

Reply #5
[quote author="rsdio"]
[quote author="Sjaak"]i don't think 0x00 is the default for any nonvolatile memory. You can safely leave 0xFF out of the final .hex to speed things up.
[/quote]There's a really good idea. The only remaining challenge is a way to determine the size of the Flash being programmed before reading in one or more .HEX files for editing, merging, or maybe even disassembly.
[/quote]

Use a commandline switch ;) Without reading any hex file, this is the only way.

Alternatively you could use an adaptive array, which grows. I.e. you start with 64k and double if the boundry is hit. Downside is that some processors store the config word at the end of the memory (several Mbytes)

Re: (Intel) .HEX reader/writer/editor

Reply #6
[quote author="Sjaak"]Alternatively you could use an adaptive array, which grows. I.e. you start with 64k and double if the boundry is hit. Downside is that some processors store the config word at the end of the memory (several Mbytes)[/quote]Another good idea - thanks!  It just occurred to me that since most hex files are limited to 64K before a new segment command arrives, then the .HEX editor could just allocate 64K per segment, marking each one with it's address offset, and leaving the unused memory between out of the map.  Basically, sparse array allocation.

I've mostly worked with the 18F67J50, which has 128K plus the Config.  The compiled firmware only ever sees three segment markers, so a mere 192K total.

Combined with skipping ranges that are full of 0xFF, this could work quite well.

I think there is a variation of .HEX which allows more than 64K, which would cause a trip-up, but I might be thinking of the Motorola SREC...

Re: (Intel) .HEX reader/writer/editor

Reply #7
I believe there is a record type in a intel HEX that says the next chunk is at the boundary+address.
Got a question? Please ask in the forum for the fastest answers.

Re: (Intel) .HEX reader/writer/editor

Reply #8
[quote author="rsdio"]
I think there is a variation of .HEX which allows more than 64K, which would cause a trip-up, but I might be thinking of the Motorola SREC...
[/quote]

I think you are refering to 04 and 05 records: http://en.wikipedia.org/wiki/Intel_HEX