I have been trying to integrate an ntp client into the FreeRTOS stack and have run into memory problems when enlarging the size of the task structure. The compiler/linker pic30-gcc outputs a memory map during compilation. I've done some post-processing in OpenOffice and calculated the probable size of all variables (excluding any alignment bytes).
It looks like we disproportionally allocate uIP structures - 2.8kb of the 8kb total for holding socket state data! Given (at least my project) can get away with a maximum of 1-8 concurrent sockets, I'm going to submit some patches that cut these structures down.
RAM section
base dec sz symbol
0x0800 2048 12 ???
0x080c 2060 2 _uip_appdata
0x080e 2062 2 _uip_len
0x0810 2064 2 _uip_conn
0x0812 2066 2088 _uip_conns
0x103a 4154 4 _uip_acc32
0x103e 4158 2 _uip_udp_conn
0x1040 4160 540 _uip_udp_conns
0x125c 4700 52 _uip_stat
0x1290 4752 2 _uip_flags
0x1292 4754 4 _uip_hostaddr
0x1296 4758 4 _uip_netmask
0x129a 4762 4 _uip_draddr
0x129e 4766 2 _uip_sappdata
0x12a0 4768 2 _uip_slen
0x12a2 4770 1754 uip_listenports
0x197c 6524 552 _FatFS
0x1ba4 7076 2 _g_FreeHeapSize
0x1ba6 7078 16 __Files
0x1bb6 7094 40 __Stderr
0x1bde 7134 40 __Stdout
0x1c06 7174 870 __Stdin
0x1f6c 8044 36 _tptr
0x1f90 8080 2 _UIP_DMA_STATUS
0x1f92 8082 2 _uip_dma_rx_pending_packets
0x1f94 8084 2 _uip_dma_rx_last_packet_length
0x1f96 8086 2 _uip_dma_tx_last_packet_length
0x1f98 8088 2 _ConsoleTaskHandle
0x1f9a 8090 2 _cmdend
0x1f9c 8092 2 _cmdstart
0x1f9e 8094 22 _cmderror
0x1fb4 8116 2 _Enc28j60Bank
0x1fb6 8118 2 _NextPacketPtr
0x1fb8 8120 2 _dmaSemaphore
0x1fba 8122 6 _mac
0x1fc0 8128 2 __Size_block
0x1fc2 8130 4 __Aldata
0x1fc6 8134 4 _pxCurrentCoRoutine
0x1fca 8138 2 _pxCurrentTCB
0x1fcc 8140 4 _uxCriticalNesting
0x1fd0 8144 10 _g_resetcode
0x1fda 8154 2 ___curbrk
8156
DMA section
0x2220 8736 2 _encDMADummyStorage
0x2222 8738 1502 _uip_buf
10240
Total
0x2000 8192 7612
This is not entirely accurate as many large objects are not marked extern therefore not accounted for in the map. Enabling debug in the linker gives a more accurate dataset to process - I'm still processing this!
Sounds good. You can post patches or I can give you SVN access (if you don't have it already).
The most substantial change I found was to enable _FS_TINY = 1 in ffconf.h. This brings down the size of each TCP session (not listening socket!) by 512 bytes (we allocate 2) by removing each HTTP session of it's separate filesystem cache. The only case I think this would make a difference would be if we had to retransmit a packet whilst serving two clients - as the second client would likely have moved the FS window, causing a full directory lookup/seek to occur. Note there is still one 512byte buffer (per filesystem, so =1) in main.c which means the one-client case does not change, ie. if we have to retransmit last last sent packet it should just read the bytes from this buffer again rather than invoke another mmc read. I am still working to verify that there are no side effects of this change though.
It might be possible to change the FS model to DMA from the flash chip/microSD straight into the 512byte buffer if we have room for FATFS in the DMA range. This would require an extra 512bytes of dma memory (we're already using 1.5kb for the packet buffer) but as a benefit would allow a dma memcpy to write FATFS => uip_buf when writing the packet.
Perhaps therefore the FATFS buffer could be eliminated entirely. Since we read the file component of the URL from the HTTP header during input packet processing, this frees the entire uip_buf to be used while we wait on the filesystem for the directory search and read to complete. The filesystem could use this space as it's cluster buffer. We are therefore writing from MMC straight into the packet buffer. Implies the filesystem can only be accessed while locking uip_buf - which obviously suspends inbound packet processing. I imagine this is the usual case - we only read the FS whilst handling an inbound packet or timer expiry.
After what seems like an age, I've got the reduced memory footprint firmware working well with another co-routines task, an NTP client (http://osdir.com/ml/network.uip.user/20 ... 00020.html (http://osdir.com/ml/network.uip.user/2006-10/msg00020.html)):
Dangerous Prototypes Web Platform
Hardware v1.X, Firmware A2 v(trunk) built at 23:35:00 on Feb 7 2011.
Last system reset caused by one or more of:
Reset Button press.
SD Power on
CardType=4
0 disk_initialize(0) returns 0
f_mount(0) returns 0
Reading SD Card...
SD Power on
CardType=4
fs_type=2 csize=64
NIC DMA Init
nic init
uip init
arp init
http init
ntp init
dhcp sent discover
dhcp sent discover
dhcp sent discover
dhcp got packet type 2
dhcp sent request
dhcp sent request
dhcp got packet type 2
dhcp configured. My IP=192.168.1.3
NTP: sent ntp request
NTP: timeout
NTP: sent ntp request
NTP: received reply after = 0.00000 sec
NTP: shortest turn around time so far
NTP: turn around half diff = 0.00000 (0,0x0)
NTP: adjusted ltime = 0/0
NTP: sent ntp request
NTP: received reply after = 0.00000 sec
NTP: sent ntp request
NTP: received reply after = 0.00000 sec
NTP: time: Monday 7. Feb 2011 23:37:05 (DST)
NTP: setting hw time
The biggest problem I had was that the stack on the IDLE task was too small. A lot of the problems disappeared when I made the idle task stack 512bytes to match the uip_task. I read this is because the interrupts may freely execute on whichever task's stack happens to be running at the time of the interrupt. This does not convince me though as the interrupt functions are very small with just a handful of stack allocated variables.
I plan some more work on the webserver, then will bundle this up for testing/svn. Here is the memory map now - I have converted a lot of the previously static variables to be automatic such that they appear in the linker's debug:
0x0802 _xHeap <-- heap (from which both stacks allocated)
0x125e _FatFSA <-- single FATFS structure for SD card
0x1492 _uip_appdata
0x1494 _uip_len
0x1496 _uip_conn
0x1498 _uip_conns <-- much smaller TCP socket structs
0x1608 _uip_acc32
0x160c _uip_udp_conn
0x160e _uip_udp_conns
0x1636 _uip_flags
0x1638 _uip_hostaddr
0x163c _uip_netmask
0x1640 _uip_draddr
0x1644 _uip_sappdata
0x1646 _uip_slen
0x1648 _uip_listenports
0x1650 _rxbuff <-- eric's serial port buffers
0x1658 _txbuff
0x1758 __Files
0x1768 __Stderr
0x1790 __Stdout
0x17b8 __Stdin
0x1a3a _ntpClient <-- ntp task state
0x1a84 _dhcpCfg <-- dhcp task state
0x1ae4 _UIP_DMA_STATUS
0x1ae6 _uip_dma_rx_pending_packets
0x1ae8 _uip_dma_rx_last_packet_length
0x1aea _uip_dma_tx_last_packet_length
0x1aec _ConsoleTaskHandle
0x1aee _cmdend
0x1af0 _cmdstart
0x1af2 _cmderror
0x1b02 _Enc28j60Bank
0x1b04 _NextPacketPtr
0x1b06 _dmaSemaphore
0x1b08 _mac
0x1b0e __Size_block
0x1b10 __Aldata
0x1b14 _pxCurrentCoRoutine
0x1b18 _FatFs
0x1b1a _pxCurrentTCB
0x1b1c _uxCriticalNesting
0x1b20 _g_FreeHeapSize
0x1b22 _g_resetcode
0x1b2c ___curbrk
0x2220 _encDMADummyStorage
0x2222 _uip_buf <--- DMA TCP packet
Nice work, thank so the update. When you;re ready for SVN please let me know if you need any help.
Quick update: I've been reading more about responsible use of NTP and recommendations for embedded developers on http://www.pool.ntp.org/en/vendors.html (http://www.pool.ntp.org/en/vendors.html) . I've therefore added a DNS resolver from uIP into the FreeRTOS-eric code and changed the boot-up to wait for DNS resolution of [tt:]pool.ntp.org[/tt:] before attempting an NTP query. It seems that if the number of devices is none-trivial they (pool.ntp.org) would rather developers registered for a vendor pool, which could be worth considering for DP - see the Netgear and DLink ddos sagas!
I've modified the web server to include the demo pages from Free RTOS embedded in the application code, and the 404 page. This allows the web server can be more functional without a micro-SD card. My aim for the next couple of weeks is to have an upload facility like the
mpfsupload tool in the microchip firmware so that web pages can be uploaded over HTTP post, to either the flash or micro-sd.
Here's the bootlogs:
Dangerous Prototypes Web Platform
Hardware v1.X, Firmware A2 v(trunk) built at 18:48:05 on Feb 19 2011.
Last system reset caused by one or more of:
Reset Button press.
SD Power on
CardType=4
fs_type=2 csize=64
NIC DMA Init
nic init
uip init
arp init
dhcp init
resolv init
http init
ntp init
dhcp sent discover
NTP: waiting for IP config
dhcp sent discover
dhcp got packet type 2
dhcp sent request
dhcp got packet type 5
dhcp: IP address 192.168.1.3
dhcp: netmask 255.255.255.0
dhcp: DNS server 192.168.1.1
dhcp: default router 192.168.1.1
dhcp: lease expiry 86400 seconds
resov - dns server configured
Using resolver entry 0 to lookup 'pool.ntp.org'
NTP: waiting for DNS lookup
sending DNS request for pool.ntp.org
resolv - pool.ntp.org = 141.40.103.103
NTP client - got server IP address
NTP: sent ntp request
NTP: received reply after = 0.00000 sec
NTP: shortest turn around time so far
NTP: turn around half diff = 0.00000 (0,0x0)
NTP: adjusted ltime = 0/0
NTP: sent ntp request
NTP: received reply after = 0.00000 sec
NTP: time: Saturday 19. Feb 2011 19:12:57 (DST)
NTP: setting hw time
Nice work. A remote updater for FreeRTOS would be a powerful and interesting addition.
After a 4 week break, I'm back working on this. There's an annoying bug causing Address Exceptions that's hobbling progress. I can't easily tell if it's with FreeRTOS, UIP, or my code, seems like a pointer problem. I've got a pickkit3 on the way for debugging... burned too much time without the right equipment! I've done all the development in MPLAB X so far, have tried the built-in simulator but it's near useless without peripherals :-(
The web server + CGI code works fairly well, although 'upload' feature is still just a pipe dream! There's some code to let the web server check URLs against four sources, (1) files encoded into the PIC firmware as constants, (2) eeflash chip (3) uSD card, then (4) back to a low-priority PIC firmware list in turn. This lets me have a simple index.html in the firmware that can be overridden by uploads. I've removed the FAT16 support as it was too complex, so have a little python tool to create the file system image as a binary file. I've reserved the first 1Kbytes of the eeprom for configuration (ip address, mac address, last hwtime, etc.) so start the file system from address 1024 onwards.
Caught another bug in FreeRTOS-eric where following a reboot (intentional or otherwise) the boot-up would hang whilst in the SD card initialisation routines. This never happens from a POR, but often from a crash or reset button. I found that FreeRTOS toggles the CPU interrupt priority level (IPL) in SR<7:5> when switching to/from critical sections. If the device reboots when the interrupt priority is high, that level persists to the init code. The SD card init routines are therefore buggy - they enable Timer2 as a bootstrap (to decrement timers for SD signal timing) but leave the global interrupt level unchanged, rather than reset to match POR. So when the code hits the first tight loop waiting for the timer interrupt to count down the timer, the interrupts are masked and the code hangs.
There's also a hang in the Ethernet chip read routines - will look into the transfer speeds as I've seen others have made bug fixes to the microchip code in this area, and it looks more like a bus protocol issue. Might be worth a time out in this code to abort the packet if a stall is encountered, rather than locking up the ethernet task completely. Probably need to get Logic analyser going to snoop the buses for that one.
I've not solved the frequent Address Exceptions - debugging the address causing the fault seems to indicate 0x0157FE which was right at the very top end of the program memory memory map :-/ And there's no call stack reported by MPLAB X which is a bit annoying.
Finally, static content like .css isn't sent with headers allowing caching so they get fetched over and over.
I will upload some binaries/patches so anyone interested can try out the FreeRTOS + DNS + NTP + web/cgi combo.
I will upload some binaries/patches so anyone interested can try out the FreeRTOS + DNS + NTP + web/cgi combo.
That would be great, thanks! I'll do a post for testers.
Hello shuckc
i am interested by your upgraded Free-RTOS release and with the NTP ...
You said you will upload it but i can't see it on the SVN
Thanks for your work on this release which looks nice
I second that request. I'm trying to figure out which of the two OS ports (Contiki or FreeRTOS) is the best starting point for my experiments. They have advantages and disadvantages. I like the Contiki port because it has lower memory usage and a good network stack but that port lacks the SD Card/FatFS support. Adding that would likely chew up the ram available. On the other hand the FreeRTOS port has support for the SD Card but its network stack is missing NTP. Also I found that it has problems with stack overflows after a while and giving it more ram is likely to leave nothing for my app development.
What I'd like to do is grab adc data, do some processing on it, and send the results in udp packets, while supporting a web server for monitoring and control. So the Contiki port looks closest, but I'd really like to use the SD card if possible for another logging type application. I'm wondering if it will really be possible in 8K of ram.
Any advice (or code) appreciated.
Why not add sd support to contiki?
You can easily add a thread with dosfs or another sd fat implementation (or write your own :D)
ie. dosfs: http://www.larwe.com/zws/products/dosfs/index.html (http://www.larwe.com/zws/products/dosfs/index.html)
or fatfs: http://elm-chan.org/fsw/ff/00index_e.html (http://elm-chan.org/fsw/ff/00index_e.html)
or petit fatfs: http://elm-chan.org/fsw/ff/00index_p.html (http://elm-chan.org/fsw/ff/00index_p.html)
There are many other but thsi is a small subset of them :D
Thanks for the pointers. They look useful. I'll give it a try.
Yeah - to make it clear I gave up with FreeRTOS (kernel-enforced time switching) and concentrated on Contiki support (co-operative scheduling). Contiki has a file-system interface, although I've not used it.