Prototype: Twitter Watcher, the #twatch

in #twatch, Prototypes by Ian | 22 comments

The most recent documentation is now on the wiki.

The #twatch scrolls the latest trending topics from Twitter on an LCD screen. It’s a stand-alone network appliance that stays updated without a PC. It was awesome to watch #iranelection, Michael Jackson, and other historic events scroll by while we developed the #twatch. This article documents the #twatch hardware and design.

overview-ii.250In addition to a Twitter trend ticker, the #twatch is a generic ethernet LCD backpack. It’ll show playlists, PC stats, and other info with programs like LCD Smartie. It’s also software upgradable, so it’s never outdated.

The #twatch is open source, so you can download our designs and build your own. Seeed Studio has a few assembled #twatches for $45, including worldwide shipping. Get them while they last because we won’t make more soon. If you missed the #twatch, sign up here to be notified of any future #twatch preorders.

More videos and complete design details after the break, more discussion in the #twatch forum. We’ll send a free #twatch PCB if you’re the first to tweet the #twatch!

Concept overview

overview.ii

The #twatch grabs the latest trending topics from Twitter, then loads a few tweets for each. The trending topics and tweets scroll on a screen. The #twatch grabs fresh trends and tweets every five minutes so you always see the latest trending topics.

We also added a generic ethernet backpack mode, so the #twatch can also show PC stats from a program like LCD Smartie, more about this feature next week in part 2.

Hardware

cct.450

Click for a large image of the schematic [PNG]. We used the freeware version of Cadsoft Eagle to make the circuit and PCB. Download the latest files from the project Google Code page.

Ethernet PIC 18F67J60 microcontroller

twatch-pcb.450

The Microchip PIC 18F67J60 is perfect for this project because it combines an ethernet network interface and a 41MHz microcontroller (10MIPs) in a small package for only a few dollars. It only comes in 64pin+ TQFP packages, but we didn’t have a problem soldering it by hand to a professional PCB.

The PIC requires a 3.3volt power supply. The ethernet portion is really power hungry, so we used a giant TO-220 LD117-3.3volt regulator (VR1). We chose a big regulator because it might have to dissipate a bunch of heat depending on the input power supply. The regulator requires a small input decoupling capacitor (C15) and a large 10uF output capacitor (C3).

There’s a big catch with these chips: they can only be programmed about 100 times. That makes development difficult, so we also designed a development version of the #twatch based on a different chip. More about that design in a future article.

pic-cpu-cct

Each PIC power pin gets a 0.1uF decoupling capacitor (C17-C23). The PIC has an internal 2.5volt regulator for the microcontroller and ethernet cores, the regulator requires a 10uF tantalum capacitor (C1).

The PIC is programmed through a 5pin ICSP header. The MCLR reset pin is held high with a 10K pull-up resistor (R21), an additional resistor (R4) and capacitor (C16) recommended by the datasheet provide protection against various accidental reset conditions.

The ethernet section mandates a 25MHz external crystal (Q1). Two 33pF capacitors (C4, C5) complete the oscillator circuit.

pic-rj45-cct

We used an ethernet jack with integrated magnetics (J2). The jack is a HanRun HR911105A, provided to us by Seeed Studio – be sure to get the same jack, a compatible jack, or adjust the PCB for a jack you can find. The ethernet interface requires a termination circuit (R30-33, C10-11, L1) and 2.28Kohm 1% bias resistor (R7, not shown).

HD44780 character LCD

twatch-sideii

The #twatch supports a ‘standard’ 4line by 20character 5volt HD44780 LCD with a +5volt backlight. You can usually find them for about $10 on eBay. Be sure to verify that your LCD matches the #twatch pinout before attaching it. Most LCDs are the same, but not all of them.

lcd-highlight

Almost all character LCDs operate at 5volts, so we provide a 5volt power supply from a common 7805 regulator (VR2, C14, C2). The LCD with backlight could potentially use a bunch of current, so we used another large TO-220 regulator. C12 is a decoupling capacitor for the LCD power supply, but LCDs already have on-board decoupling. C12 need not be populated, we just included it in case of stability issues.

lcd-pullup-cct

For maximum refresh speed, the LCD is controlled through the full 8bit interface.

Most LCDs are 5volt parts that require about 4.25volts+ to register a high level on the data pins, but the PIC 18F67J60 is only a 3.3volt part. Fortunately, the PIC has a bunch of 5volt tolerant pins so we can hold the signal at 5volts with a 10K pull-up resistor (R10-R19), and then ground it by changing the PIC pin direction setting. This is usually called an open drain output.

Some newer LCDs run at 5volts, but still work at 3.3volt interface levels. The #twatch will support this mode if you leave out R10-19 so no pull-up voltage goes onto the pins, and change the firmware to switch the LAT register instead of the TRIS register in HD44780.c.

lcd-contrast-cct.iii

LCD screen contrast is controlled by a bias voltage, usually generated with a 10Kohm potentiometer. The #twatch PCB has footprints for an inexpensive 3mm SMD pot (R2), and a second space to use a larger, through-hole pot (R2A). Only one should be populated!

Just in case there’s noise in the power supply from all the ethernet stuff, we filter the bias voltage through a small ferrite bead (L2). We also included a capacitor for additional filtering (C13), but we didn’t use it since neither part is actually needed.

lcd-backlight-cct

The #twatch can control simple +5volt backlights up to 400mA or so. The PIC switches a transistor (NPN1) through a 240ohm current limiting resistor (R3, not shown). We used a transistor that can handle 800mA+ with a gain of 250hfe+, so the PIC can switch a large load with it’s 20mA maximum pin output current.

R1 is a current limiting resistor for the LCD backlight, if needed. We used a through-hole resistor so it can dissipate a lot of heat with big backlights, and because it’s the easiest size to find locally and solder yourself. If your backlight doesn’t require a resistor, just replace R1 with a piece of wire. Our LCD required a 3ohm resistor for a 240mA backlight power supply.

Some backlights use a lot of power, so we put the supply pins right next to the power supply and reinforced the ground plane with a bunch of VIAs. Some fancy LCD backlights require special drive circuits, so be sure yours uses a simple +5volt supply to avoid damage.

Power supply

vreg-side-side.470

The #twatch requires a 6-7volt power supply through a 2.1mm power supply jack (J1). 2.1mm plugs are the most common size, and should come with every universal power supply. The #twatch shouldn’t draw more than 500mA, depending on backlight use.

As the power supply voltage increases, VR1 and VR2 have to dissipated more heat. Use a supply as close to 7volts as possible. Remember that the #twatch is a prototype learning board, it’s not a complete and tested commercial product. Take appropriate safety precautions and don’t run it unattended.

PCB

Twatch-v2a-publish-parts

We used the freeware version of Cadsoft Eagle to make the schematic and PCB. Download the latest files from the project Google Code page.

The PCB is a 2-layer design with small traces and separation (10mil) around the 64pin TQFP PIC chip. We prepared gerbers and sent them to Seeed Studio’s PCB service for open source works. Extra PCBs from our order are available in the Seeed shop. If you buy our extra PCBs be sure to get the HanRun ethernet jack that fits the board.

profile.450

Since the mounting holes on 20×4 LCDs vary, we didn’t try to fit the PCB to the LCD holes. We made it as small as possible, like SparkFun’s serial LCD backpack, so it stays out of the way of the original holes. As a side effect, it’s not too terribly awkward behind some smaller screen like this 16×2 LCD at Adafruit.

Parts list

pcbiii.450

Click for a full size placement image [PNG].

Part Value Package
IC1 PIC 18F67J60 TQFP-64
C1-3 10uF tantalum capacitor, 10volts+ SMC_A
C4,5 33pF capacitor 0805
C10,11,C14-23 0.1uF capacitor 0805
ICSP 5x 0.1″ male pin header
J1 2.1mm SMD power jack SMD
J2 HR911105A ethernet jack RJ-45
L1, L2 Ferrite bead, 200ma+ 0805
NPN1 NPN transistor, 250hfe+, 800ma+ SOT-23
Q1 25MHz SMD crystal HC49UP
R2(A) 10K single turn trim resistor 3mm SMD or through hole (cheapest)
R3 240 ohms resistor 0805
R4-6 390 ohms resistor 0805
R7 2,260 ohms resistor, 1% 0805
R10-21 10,000 ohms resistor 0805
R30-33 49.9 ohms resistor, 1% 0805
VR1 LDO 3.3volt regulator (LD1117) TO-220
VR2 7805T 5volt regulator TO-220
HD44780-LCD 20×4 HD44780 character LCD

Firmware

The latest #twatch firmware download is on the project Google Code page. The code is written in C, and is compiled with the Microchip C18 demonstration compiler.

TCP/IP stack and base network functions

Microchip’s ‘free’ TCP/IP stack provides all the network functions we need to exist on a home network and grab data from Twitter. The stack is open source and free-as-in-beer, but the Microchip license prohibits distribution. Due to licensing issues, we’re only putting our public domain source code in the project Google Code SVN, learn how to get and compile the source here.

twatch-ip.250The stack has a Dynamic Host Configuration Protocol client that automatically configures the network settings using the DHCP server on your local network. The #twatch requires a DHCP server, but the vast majority of networks and routers have this enabled. The IP address, mask, gateway, and first DNS server are displayed on the LCD screen until valid Twitter data is available.

The stack also includes Microchip’s announce server. When an IP address is acquired by DHCP, the #twatch announces its IP address with a broadcast packet to all computers on the local network. Use the MCHPDetect.exe utility in the project archive to view these packets.

ping

Finally, we included a ping (IMCP) server. Use any ping client to check if the #twatch is alive on the network.

Twitter TCP client

twatch-soft.ii

lcd-update1

The Twitter trend following program is a simple TCP client, similar to a web browser, that pulls data from web servers. Twitter’s API will give us data in a variety of formats. We used the light-weight JSON format because it’s easiest for the low-power PIC chip to decode. For JSON viewing, check out JSONView if you use Firefox.

After the #twatch configures the network connection, the Twitter TCP client takes control and grabs the current trending topics. It searches through the this JSON datafeed and looks for the “name” tag. Up to 10 trending topics are copied into a 225byte buffer. A separate array stores the end pointer position of each topic so we can retrieve them in the next step.

Next, the #twatch searches Twitter for 2 tweets for each topic. It appends each topic to the end of the Twitter JSON search url, special characters like spaces and punctuation are URL encoded.

lcd-update2The TCP client parses through the search results and looks for tweets that follow the “text” tag. Tweets have multiple layers of encoding. We decode HTML reserved characters like ampersand (&) and quotes (“) because the LCD screen can display them. We remove UTF8 international characters because the HD44780 LCD doesn’t have them in its character set.

The parsed, decoded tweets are stored in a 2100byte buffer, an additional array marks the beginning and end of each tweet in the buffer. RAM space was a big problem on the 18F67J60 chip, it only has about 4000bytes total, but the 2100byte buffer seems big enough to handle 20 average-sized tweets. We took special care to protect against out of memory problems, and we tested the client under reduced RAM conditions to ensure that it fails gracefully during errors.

whaleTwitter is well known for its FAILs occasional down-time. If the #twatch can’t connect to Twitter, it displays a connection error message and retries twice. If it can’t connect after three tries, it waits five minutes before trying again. This gives Twitter a chance to fix their problems without being hammered by #twatch queries.

The #twatch grabs fresh trend and tweet feeds every five minutes. Twitter puts a limit on the number of queries a client may make, so be careful about refreshing more often. Twitter allows 150 trending topic updates per hour, and “significantly more” search queries.

Network LCD backpack mode TCP server

The #twatch can also display PC system status information from programs like LCD Smartie. The #twatch has a TCP server on port 1337 that accepts Matrix Orbital formatted commands. This also provides control over the LCD backlight. We’ll show you how to redirect LCD Smartie from a COM port to the #twatch TCP server next week, in part two of our #twatch article.

Bootloader for network firmware upgrades

bootload

The #twatch can be updated from a PC on the local network thanks to Microchip’s internet bootloader. Remember that the 18F ethernet PICs can only be programmed an average of 100 times, so upgrades are somewhat limited. We’ve yet to burn out a chip, but we only reached about 55 cycles during development.

If you’re using a brand new chip you’ll need to program the bootloader into the PIC18F67J60 through the ICSP header, then you can upload the #twatch firmware over the network. Program twatchv2-bl-vxx.HEX into the chip with a PIC programmer like an ICD2 or PicKit.

bootloader-pin

When the #twatch powers on, the bootloader runs before the main program starts. The bootloader checks for a connection between the PGD and PGC pins of the programming header, shown in the image above. If it finds a connection, the bootloader takes over and waits for new firmware to be uploaded.

There’s a very minor chance that the bootloader will accidentally start even without a jumper between the PGC and PGD pins. This won’t damage the #twatch, just disconnect the power supply and try again. Accidental bootloader entry can be prevented by moving the jumper over one position so it connects the PGD and GND pins.

bootload

The #twatch bootloader uses the IP address 192.168.1.123 and subnet mask 255.255.255.0. Your computer must also have an IP address that begins with 192.168.1.xxx to communicate with the #twatch. We chose the 192.168.1.xxx range because it’s a common default for home networks. If your computer uses another IP address range, you’ll need to temporarily adjust it before you can do the update.

How to upgrade:

  1. Ensure your PC is in the same IP range and subnet as the #twatch. Your PC should have an IP address in the range 192.168.1.xxx, and a subnet mask of 255.255.255.0. The default #twatch bootloader IP address is 192.168.1.123, make sure that no other computer connected to the same router already uses this address.
  2. Unplug the #twatch power supply.
  3. Put a jumper between the PGC and PGD pins.
  4. Plug in the network cable, if necessary, and plug in the power supply. The screen may be blank, have solid blocks, or garbage.
  5. Use a TFTP utility to send the new firmware to the #twatch IP address, we use TFTP.exe from the Windows command line.
  6. The TFTP update reports success or error.
  7. Unplug the power supply, remove the update jumper.
  8. Plug the power supply back in. The #twatch should start scrolling tweets. If the bootloader starts instead, put a jumper between the PGD and GND pins and try again.

Using it

usingit.450

The #twatch is easy to use.

  1. Connect it to a home ethernet network with Internet access. The #twatch requires automatic network configuration (DHCP), this is the default setting on almost every modern home network.
  2. Power it up. The #twatch requires a 6-7volt DC power supply. It uses a 2.1mm DC power plug, the most common type. Universal DC power supplies should include a 2.1mm plug.
  3. Adjust the contrast. LCD screens change with temperature and age, use the adjustment screw to tweak the screen contrast.
  4. The #twatch will configure the network settings and begin scrolling the latest trending topics and a few tweets from each. You may need to adjust the contrast again for the cleanest scrolling effect.

Taking it further

twatch-sideWe designed the #twatch to make full use of the resources on a single chip, an expanded design would add features but be more expensive.

The #twatch could follow your own Twitter feed. It would need a small web server to enter your Twitter login, and an external EEPROM to store the configuration information.

The #twatch could also store more tweets or extra information about each tweeter, such as name and location. Microchip doesn’t make an integrated ethernet controller with more than 4K of RAM, but we could add an external SRAM to store tweets and tweet meta-info.

Updated hardware could add an I/O header for connecting buttons with LCD Smartie.

A 20character by 4line LCD doesn’t have a lot of display space. We designed the #twatch interface around this limited amount of space. An updated firmware could handle multiple screen sizes.

The bootloader can adopt the IP address acquired by DHCP. A future #twatch firmware update will take advantage of this feature for easier network upgrades.

Next week we’ll cover the LCD Smartie compatible TCP server built into the #twatch.

Get one!

twatch-sideiii

What’s twending on your #twatch? If you want an assembled #twatch or PCB, here’s a couple options:

If you’d like to get involved, join the Dangerous Prototypes open hardware project at Google Code, or come chat in the #twatch forum.

Next week we’ll show you how to redirect LCD Smartie system stats to the #twatch TCP server.

twatch-frame-Twitter.450

This entry was posted in #twatch, Prototypes and tagged , , , .

Comments

  1. Ian says:

    Sometimes the video looks a little jerky, it’s the interaction of the the LCD refresh rate and the digital camera I used to make the video.

  2. Peter says:

    This looks good,

    would it be possible (easy?) to modify the firmware to reas an rss feed from any website? even if the website was hardcoded into firmware etc.

  3. Dave says:

    I would be very interested to hear what Ian thinks, but I assume it wouldn’t be too much of a challenge to do, JSON seems very similar to XML.

    I’d also love to hear more about the VFD screen, is there a screen that’s a pin for pin replacement for the LCD?

  4. Ian says:

    Reading XML should be a bit easier because you only have to get a simple page, rather than parsing JSON, storing it, and then using it to make additional queries.

  5. Ian says:

    @Dave – the VFD is a Futaba screen that’s HD44780 compatible. I got it on eBay, but you can find various types on the internet.

  6. Rhyvu says:

    The #twatch looks good, might there be support for identi.ca in addition to twitter in a future revision of the software?

    Also you mention that the chip used only has about 100 write cycles and therefore a development board of the #twatch based on a different chip is also designed, may I ask what chip in question is used on it? And do you imply that only the chip is changed or has there been done modifications to the board itself?

    Lastly it seems everybody has their own opinion regard this, so I would like to have yours. What are your thought in regards to the comparison between VFD and LCD since I see you have used both for developing the #twatch?

  7. Ian says:

    @rhyvu – It should be possible to adapt the firmware to work with any site, it just loops through the output and looks for a tag, when it’s located, it slurps up the text. I plan to demo changes like this in the coming weeks.

    The v1 dev board is entirely different than v2. It uses a PIC24FJ64GA002 and an ENC28J60 for ethernet access. It’s not as well done, either – it uses a 4bit LCD interface, has under-sized power supplies, and non-nonsensical routing. I’ll write more about it this week in a confessions post.

    The VFD has a better viewing angle, faster update speed, and less ‘ghosting’ problems. Of course, I have about the cheapest LCD in the world, better LCDs would compare more favorably. This VFD was on eBay for about $7 for awhile, but it’s the Futaba serial type. I had to hot-air a few resistors on the back to different positions to use it in HD44780 mode. There are other sources of HD44780 compatible VFD screens on the net though. I really like the VFD, but it’s also a bit bulky.

  8. Rhyvu says:

    Looking forward to your #twatch confession post and with regards to the VFD or LCD discussion the best option might be to get a HD44780 compatible OLED so that no compromise is made. Although I wonder what the price and availability for HD44780 compatible OLED are. I will investigate this a bit further.

  9. Dean says:

    Might consider taking a look at POE – Power Over Ethernet. Many switches these days are including that as a standard. One less cable …

  10. Ian says:

    @dean – We looked at POE, but in the end it would add a few extra bucks per board and we didn’t have any equipment to test it.

    Is there some sort of POE break out board that lets you tap the POE power supply?

  11. Synonymous says:

    That looks so awesome! There ma come a day when devices like this are as necessary in a person’s daily life as a mobile phone is today!

  12. Steph says:

    Howdy,

    Might seem silly to ask, but the DC power supply is center positive, right? Just want to make sure…

  13. Ian says:

    @steph – yes.

  14. Michael E. Califf,Sr. says:

    Was wondering if there is a way to set the info so that the backlight is alwasy on when power is applied?? As it is right now I have to redo the hexfile everytime I turn off the power.
    Is there a way to enter the hexfile so that it stays in the unit.

  15. Ian says:

    @Micheal – There’s an experimental version of the firmware here:
    http://whereisian.com/forum/index.php?topic=141.0

  16. Suraj says:

    Wonder if twatch can display my daily google calender events , without any pc

  17. Liam says:

    Great stuff. Just ordered one but has a question. Is it possible to replace the Ethernet with Wi-fi? If not with this board, would you plan to have Wi-fi soon? or if you know any alternatives?
    Thanks,

    • Ian says:

      You could use a wifi-ethernet adapter. It is not possible to replace it directly on the PCB, and I don’t plan a wifi version. There is an enclosed WIFI module from Microchip that could get the job done though.

      • Liam says:

        Thanks, I gave up with the wi-fi due to its high price. Can I ask if I can add an RF transmiter (433Mhz or 2.4Ghz) to the board to transmit the twitter texts to another remote RF receiver? Do I need to remove the LCD in this case?

      • Ian says:

        Hi Liam,

        You may be able to, it depends on the RF module you want to connect. If you can use some of the extra pins, there’s no reason to remove the LCD. This will take a bit of programming. Please give me a shout in the forum if you’d like to discuss it further.

  18. Ryu Nakamura says:

    You can purchase Noritake VF as well. I think they have the compatible module sold in their online shop. http://noritake-vfd.com/

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Notify me of followup comments via e-mail. You can also subscribe without commenting.