The web server stops, Problems with ENC28J60??
Hi
Is there anyone else that has problem with that the web server interface seems to stop, it doesn’t answer to ping, don’t show any homepage but the Led 1 continues to flash so it seems that the PIC continues to go.
I have read some bad things about the ENC28J60 so it is probably that chip that freezes.
It seems to happen if I start a Ping from my pc on a local switch within 3-10minutes
â€ping 192.168.1.214 –tâ€
Or if I have the start page open in Internet Explorer for a longer time.
I using this demo http://dangerousprototypes.com/2009/12/11/prototype-web-platform/ (http://http://dangerousprototypes.com/2009/12/11/prototype-web-platform/)
Any bode worked out a workaround for problem like this?
After a reset it works fine for a while.
Maybe a self restart can fix the hookups, but that not seems reliable solution.
Hi macdanne, I'm sorry you're having a problem with the web server.
When I did the tweet_tree project I noticed that the server sometimes got stuck at the highest clock speeds, when I slowed it down a bit I never had an error again. I could try making a firmware version that operates at half clock and see if that helps at all. The dsPIC goes really fast, the SPI frequency in the demo might be too high for the ENC28J60 and it misses bits, messing up the configuration.
Could the +2%/-2% accuracy of the used internal FRC oscillator be a problem when communicationg with other devices like the ENC28J60?
The ENC28J60 is SPI, so it's driven by the clock sent by the PIC, crystal accuracy shouldn't effect the communications.
Looking at the 33fj128gp204 datasheet, SPI section, it is possible to have a 1:1 ration between the internal cycle and the SPI peripheral. That would run SPI at 40MHz, well out of the specification of the ENC28J60. I'm not 100% sure this is the case because the Microchip TCPIP stack may configure the SPI peripheral for a slower speed (the datasheet says never to use 1:1 clock), but it might still be too high.
ENC_SPICON1 = 0x17; // 1:1 primary prescale, 3:1 secondary prescale (10MHz @ 30MIPS)
Here it is. The SPICON1 controls the prescaler ratios. It's setup for 10MHz @ 30, which is probably ~13mhz at 40. I believe the ENC28J60 is rated up to 12MHz.
This is a configuration setting for the 33f pics in ENC28J60.c. Unfortunately, the Microchip TCPIP stack is gratis (free as in beer) but not libre (free as in speech), so I can fix this in a compile, but I can't give you the source.
Personally, I'm tired of working around the licensing issues on the Microchip TCPIP stack, so I used proceeds from the web platform to hire a coder to port the GPL uIP stack to the PIC (33F). Hopefully we'll have a redistributable stack soon that is a real alternative to the non-distributable Microchip stack.
Nice investigation! A free TCP/IP stack would be nice, indeed. In the meantime you could provide a patch which should be applied to the Microchip sources which adjusts the necessary settings.
If you're compiling yourself, just search for ENC_SPICON1 in enc28j60.c and increase the prescaler. It's currently 0x17 = 10111,101 (3 secondary)/ 11 (1 primary). I'd start with 100 11 (4/1) or 0x13. For a drastic, proof-of-bug test, change it 10110 (3:2) or 0x16.
Edit: this is from the register description on page 200 (202 in acrobat) of the datasheet
Very nice work ian, fantastic pickup.
lets hope this fixes the issue.
and yes a distributable tcpip stack would be great.
That looks like that can be the problem, big tanks!!
I have only run the PIC a short while now with the lower speed, but it seems to be stable, at least more stable ;-)
I'm really glad it worked! I can't wait to have a truly open stack we can improve and share. Keep us updated.
There are a lot of free ip stacks out there. One of the oldest is uIP, which is ported to a lot of platforms. There are some which use the enc28j60 chip, so it should be easy.
http://www.sics.se/~adam/uip/index.php/Main_Page (http://www.sics.se/~adam/uip/index.php/Main_Page)
Also you could take a look at http://en.wikipedia.org/wiki/LwIP (http://en.wikipedia.org/wiki/LwIP)
I dunno the project, but I believe microchip also provides a ip stack (they still do?). I do have somewhere a board from olimex with a enc28 which features the microchip stack. If you want I can look it up and see what it did use.
Microchip's stack is a licensing pain and not distributable. I've got a million ideas but limited time, so I actually hired someone to port uIP to the PIC/33F with proceeds from the web platform sales. He's supposed to start working on it in the forum and SVN soon. Paying someone to contribute to open source is the most exciting thing I've gotten to do with the blog so far.
I just read more about this platform. Also something to do with too less time :D I'm still busy with the newterm of the BP (almost finished a new big feature for it :O ) BTW a powerfull chip for just a webserver (or does it run a heavy database server? :P)
If you have some questions about IP or stacks feel free to ask (I did once a shot at writing one for the gamecube (when uIP wasn't ported/freely availabe for the GC)) I came till basic ip, icmp and udp. Then it was released, and i gave up :D
[quote author="ian"]
ENC_SPICON1 = 0x17; // 1:1 primary prescale, 3:1 secondary prescale (10MHz @ 30MIPS)
Here it is. The SPICON1 controls the prescaler ratios. It's setup for 10MHz @ 30, which is probably ~13mhz at 40. I believe the ENC28J60 is rated up to 12MHz.
[/quote]
I downloaded the Microchip TCP/IP stack yesterday and in my enc28j60.c the referenced line is inside a #else conditional that shouldn't be active because its preceded a few lines back by
#if defined(__PIC24H__) || defined(__dsPIC33F__)
ENC_SPICON1 = 0x0F; // 1:1 primary prescale, 5:1 secondary prescale (8MHz @ 40MIPS)
// ENC_SPICON1 = 0x1E; // 4:1 primary prescale, 1:1 secondary prescale (10MHz @ 40MIPS, Doesn't work. CLKRDY is incorrectly reported as being clear. Problem caused by dsPIC33/PIC24H ES silicon bug.)
#elif defined(__PIC24F__)
so it looks as if the code intends to set the SPI clock to 8MHz on dsPIC33Fs.
Isn't this in your code?
Could be. I just did a search for the ENC_SPICON1, I could have looked at the wrong one. I also have a version of the stack a few points back, could be a recent addition. Probably I was just careless in my search though. It does seem to be a speed related problem because a reduction seemed to help with the problem. It could also be too slow (??) because early ENC28J60 chips SPI had bugs at <10(?) or 12 MHz.
I got the same behavior as mcdanne82 describes in his initial posting. After a few minutes, the start page of the Microchip demonstration web-content, which shows the buttons and the blinking LED in the browser via AJAX, stops and says it has lost the connection ("Error: Connection to demo board was lost.").
wallabybob is right: The original value for ENC_SPICON1 is 0x0F (8 MHz @ 40 MIPS). I stepped through the respective code lines with the debugger. The ENC28J60 data sheet says that the SPI clock can be as high as 20 MHz. Taking into account Ians comment, that <10 MHz might be too slow, I tried 0b11011 (0x1b) which should be 20 MHz @ 40 MIPS, but this was not working at all. Next, IIRC, I used 0b10111 (0x17) which should be about 13.3 MHz @ 40 MIPS, but results did not improve.
Now I am using 0b10011 (0x13) which is 10 MHz @ 40 MIPS which gives me the most stable behavior. It is now already serving continuously for about an hour. I would be interested in results other people get.
For the adventurous, there is a partially ported version of uIP in the dangerous prototype SVN. I'd like to look at this issue closer with the new code. The bug might not be present in the new stack, and if it is we can fix it and redistribute the code.
A timeout still needs to be ported to the PIC, and there is one instance of an include that needs to be removed. It's not quire there yet, but the ENC28J60 driver portion should be done:
http://code.google.com/p/dangerous-prot ... c33_port_k (http://code.google.com/p/dangerous-prototypes-open-hardware/source/browse/#svn/trunk/uIP_Pic33_port_k)
In the past 24 hours I have been testing with the value 0b10011 (0x13) which is 10 MHz @ 40 MIPS, and although stability has improved, the web server still hangs after at most a few hours.
I read about the stability of the #tweet tree, but it was achived by running the dsPIC with only 20 MIPS.
I wonder if stability at 40 MIPS would improve, if a more stable clock source than the internal FRC could be used. I know, SPI has its own clock pin, but just wondering...
Although possible, the likeliness that a bug in the Microchip TCP/IP stack is causing this behavior is quite low.
One thing i am wondering with this , when the output locks up is the pic still operating? and can it talk to the ethernet chip?
ie is it the ethernet chip thats locking up or just the output?
When the board is in this state, the LD1 LED connected to the PIC is still blinking, so the while(1) loop is still running, and the ACT LED connected to the ENC28J60 still shows traffic on the network caused by other devices.
Mine has the same behavior as Markus Gritsch.
I have now compiled the SD Card demo http://dangerousprototypes.com/2009/12/ ... rver-demo/ (http://dangerousprototypes.com/2009/12/16/web-platform-sd-card-server-demo/) it seems that this is much more stable, at least yet. Can you try this to?
I did have a similar problem with the tweet tree, but not the SD card firmware. I've never run the EEPROM server for hours on end.
The tweet tree would run fine, but then eventually I believe the ENC28J60 got 'messed up'. From the status LED it looks like the PIC would try repeatedly to access the twitter feed and then time out.
I dropped the crystal speed because it could be done from the main.c without modifying the ENC28J60 driver code. After that it ran for 2 weeks + without indecent.
If you reduce the speed of the crystal in the SD/EEPROM firmware maybe you will see a similar improvement.
I made a few additional experiments with the EEPROM server demo.
in main.c:
PLLFBD=41; // 40 MIPS
in ENC28J60.c:
ENC_SPICON1 = 0b00011; // 8:1 / 1:1
--> no connection at all. Also no connection when using for example
ENC_SPICON1 = 0b11010; // 2:1 / 4:1
However, both settings work, when switching to
PLLFBD=20; // 20 MIPS
So what could cause this?
a) Maybe the FRC oscillator has a too high jitter at high frequencies which messes up SPI communication to the ENC28J60?
b) Maybe the dsPIC33 is fast enough at the 40 MIPS setting so that it can trigger one of the SPI related silicon bugs mentioned in the errata http://ww1.microchip.com/downloads/en/D ... 80443D.pdf (http://ww1.microchip.com/downloads/en/DeviceDoc/80443D.pdf)
I don't know what to make of this, but erratum 3, 4, and 19 are all SPI related. Maybe someone with more knowledge could read these few errata paragraphs?
Some comment in ENC28J60.c says: "4:1 primary prescale, 1:1 secondary prescale (10MHz @ 40MIPS, Doesn't work. CLKRDY is incorrectly reported as being clear. Problem caused by dsPIC33/PIC24H ES silicon bug.)" But the fix they used seems to be using 1:1 primary prescale, 5:1 secondary prescale) -- strange.
Well it might be possible to eliminate the internal osc if someone has the ability to replace the 32kHz OSc with something that culd run the PIC.
I tried the SD server demo as mentioned by mcdanne82, and the server is running without problems for over 8 hours and counting. The original ENC28J60 SPI clock frequency is used, and the PIC is running at 40 MIPS.
Ian, on which demo is the tweet tree firmware based on?
All of it is based on the same TCPIP stack code. The tweet tree is from the same base as the EEPROM server, I believe.
Yes, the same TCPIP stack, but the SD server is obviously doing something different compared to the EEPROM server, since it keeps running at full PIC speed. Maybe the EEPROM routines trigger some race condition?
The EEPROM and MPFS (file system) stuff is all stripped out of the tweet tree though. I uses the simplest stack-only configuration (as I recall). I'm pretty sure the hardware and configuration settings are the same. I'll look into it now.
PLLFBD=20; //pll multiplier (M) = +2 (41)
That's the line with the half-clock from tweet_tree.
hardwareprofile.h and the main.c look pretty much the same. The tcpipconfig too, the sd card version is actually more haphazard about constant definitions, etc.
It looks like we're closer to knowing the problem. I'm not sure what the difference between the three firmwares would be though.
Damn, the SD server just stopped responding -- back to square one :(
interestingly i have had what i think is the default demo running for like 2 days now. (microchip page that explains the tcpip stack with the led's and buttons at the top). i think its the eeprom version. unfortunatly i cant confirm which .hex is in it as im not sure.
the webpage keeps stopping but a simple F5 refresh and away it goes again. so the webboard isnt locking up. its just when my pc goes to sleep that it stopping running.
Maybe its a version which runs only at 20 MIPS? This setting seems to be quite stable, however, it is only a workaround.
well after another 24 hours of running the board finally lost connection tonight.
I have reset the board and am running it again, hopefully it will confirm it wasnt something i did to loose connection. but i suspect its as you say, it just takes longer.
Yay! I think I have found the problem!
Without the fix, stability is good when run at 40 MHz / 20 MIPS (PLLFBD=20) as used in the @tweet_tree, but degrades with increasing clock rate. At 80 MHz / 40 MIPS (PLLFBD=41) the web server frequently stops from responding after a few minutes. At an even more increased speed, the web server does not serve pages at all.
I am quite confident that I spotted the problem, because I have the web server now running for about 3 hours, at 100 MHz / 50 MIPS (PLLFBD=52) overclocked, without any stability issues :D
It seems, that (at least for the Web Platform) the serial SPI output changes on the wrong edge of the clock. After commenting out line 224 in ENC28J60.c
[tt:] //ENC_SPICON1bits.CKE = 1;[/tt:]
everything is fine.
Ian, can you imagine why CKE is explicitely set in ENC28J60.c? Could this be a bug or might it be necessary for other board designs like the Explore 16?
Great work! Is it still running?
That would make sense - at higher speeds the clock edge would matter a lot more.
Looking at the Bus Pirate configuration though, CKE=1 is the default setting for most SPI modes (output edge = active to idle). CKE=0 is idle to active edge for output.
I had it running for 4 hours at 50 MIPS overclocked, then I switched back to the nominal 40 MIPS, since the dsPIC gets a bit warm when running at 50 MIPS (but not as hot as the ENC28J60). Having the EEPROM server running since then serving 100 requests per second (caused by the AJAX request every 10 ms from the Microchip SolutionsTCPIP Demo AppWebPages2 content). Still running.
This really seems to be the correct setting, since now the SPI clock in ENC_SPICON1 can be set to almost an arbitrary value (1:1 for the primary and 1:1 for the secondary prescaler is not recommended by the data sheet). It works from 0b00000 (40 MHz / (8 * 64) = 78125 Hz) resulting in ping times of about 35 ms :) to 0b11011 (40 MHz / 2 = 20 MHz).
While it is nice that it seems to work now (of course only time will tell), it is a bit unsatisfying because I cannot yet understand why. Section 4.1 of the ENC28J60 data sheet [1] shows timing diagrams and says "[...] data being clocked in on the rising edge of SCK". The timing diagram on page 18-10 of the dsPIC33F Family Reference Manual [2] shows that SDO changes on the rising edge and is stable on the falling edge of SCK with our settings (CKE = 0, CKP = 0). This does not make sense. Any thoughts?
[1] http://ww1.microchip.com/downloads/en/D ... 39662c.pdf (http://ww1.microchip.com/downloads/en/DeviceDoc/39662c.pdf)
[2] http://ww1.microchip.com/downloads/en/D ... 70206C.pdf (http://ww1.microchip.com/downloads/en/DeviceDoc/70206C.pdf)
It doesn't make sense to me either. I've been through the datasheet and errata on both chips a bunch of times and nothing jumps out at me, but as you note, the original setting should be correct. I'll prep a firmware with both settings and try to borrow a scope to look at what's going on. Please keep us updated if this change has major long-term stability.
Looking forward to your investigation.
Maybe others can also contribute to the long term stability test. I attached the modified firmware (the only modification is the commented out CKE line) and the Microchip test website. Please try them on your Web Platform.
The modified EEPROM Server is approaching 72 hours of continuous uptime now, steadily serving 100+ requests per second without the slightest hiccup.
Because I have only one Web Platform, I am going to stop the "long term" test now, because I like to try out other things.
Taking into accout, that the fixed firmware also worked at a test run at 50 MIPS overclocked for several hours, and that the SPI clock can now be choosen freely (compared to be very picky at the clock speed without the fix), I would say the fix is correct.
The next step would be to understand why it works so much better now, despite the fact that according to the data sheets it should not. Ian, any progress on the timing analysis using an oszilloscope? Maybe we should bring it up on the Microchip forum. Is it read by Microchip engineers?
The Microchip forum is probably the best place to ask about it, I usually get good replies there. There might also be answers already, I haven't searched through it yet. I haven't gotten my hands on an oscope yet, but maybe this weekend I can borrow one for a few minutes.
I have built the demo with the said line commented out and have been serving pages for the last 4 1/2 hours. Which is much longer than I was ever able to do before without a reset. Nice work Markus!
edit... make that 9 1/2 hours!
Is it a stupid compiler trick? I'm going to check the ASM, but I've noticed other problems working with the current-version compiler and silicon (writes to PORT don't redirect to LAT). Maybe it will behave with ENC_SPICON1|= CKE; instead of the other notation?
Do you use any optimizing? in MPLAB you can also have a per source code optimizing option.
I found with the buspirate optimizing gives weird behaviour (even -O1 which doesn't do very trickery stuff)
Just checked: No optimization for the whole project and none for the individual files ENC28J60.c, main.c, and CustomHTTPApp.c.
Here is another piece to the puzzle: I can get the same stable SPI communication if I leave the original CKE = 1 in place, but add an additional SMP = 1 to the configuration, so that the data is sampled at the end of the data time. SMP must be set after MSTEN, so the modified code in ENC18J60.c reads
[tt:] ENC_SPICON1bits.CKE = 1;
ENC_SPICON1bits.MSTEN = 1;
ENC_SPICON1bits.SMP = 1; // line inserted
ENC_SPISTATbits.SPIEN = 1;[/tt:]
Does this shed any light on this strange issue for someone?
How about adding some NOPs? Perhaps it needs to settle down before it can accept a new setting?
Just thinking along (don't have experience with the used pic)..
It's not about finding a fix. Either commenting out the CKE = 1 line or adding the SMP = 1 line fixes the instability. It's about understanding *why* this fixes the instability. The original settings in the ENC28J60 file should be correct, according to the timing diagrams in the data sheets.
I started a thread at the Microchip forum:
http://www.microchip.com/forums/tm.aspx?m=486674 (http://www.microchip.com/forums/tm.aspx?m=486674)
no conclusive answer was given. Out of desperation I also opened a ticket at support.microchip.com. It has the number 1-174397. That was the answer I got:
"""
The CKE and SMP bits control the PICs interpretation of SPI clock phase and the SPI input sample timing. The idea CKE and SMP settings assuming no external delays. Once you have delay, other settings may work better. This usually happens at higher data rates, since the high and low time of the SPI signals are very quick. If you have skew between the data and clock signals due to TTL drivers or passive components the other settings may work better. Cases where the 'wrong settings' work better than the typical correct settings are uncommon.
I would recommend scoping the signals. The timing needs to meet the mode the PIC is in as well as the timing for the ENC28J60. This is the best way to verify timing.
If you have a lot of over or undershoot that may be causing general reception issues. If that is the case you may need to make circuit modifications for reliable operation.
"""
Well...
Ian, I think we will not ge any further without having someone looking at the timing with an oscilloscope.
Thanks for the investigation and update.
I'm still working on borrowing some scope time from a friend or university. I should really have one myself. I'll keep you updated.
I too had instability problem with my web platform.
I noticed that on page 82 of ENC28J60 Rev C datasheet, the minimum CS hold time, Tcsh is 210 ns, which is rather long.
I added some delay (using 9 Nop()s) in the WriteReg() function (ENC28J60.c) to ensusure this hold time is met.
[font=courier:]static void WriteReg(BYTE Address, BYTE Data)
{
...
Dummy = ENC_SSPBUF;
Nop();Nop();Nop();Nop();Nop();Nop();Nop();Nop();Nop(); //Ensure min CS hold time is satisfied
ENC_CS_IO = 1;
}[/font:]
The mod seems to work. I didn't change the original SPI clock timing which I thought is already correct.
Thanks for the update Stephen - there have been a number of ways people have gotten the ENC28J60 to stop hanging, this seems the simplest yet. Markus had it working even when overclocked by using a technically 'incorrect' SPI setting.