Michael F. McCabe (mfm8)
ECE 476: Final Project
The P-P-P-Palm:
A Microcontroller Based Handheld Device
We have designed a handheld device using a graphical LCD, a touch
screen from an old Palm m125 and an Atmel Mega32 to play the game Simon for
The
combination of a touch screen mounted on an LCD creates a highly adaptable user
interface that can be used for just about anything. Our initial concept called for the use of
both the Palm m125 touch screen and LCD as this is the ideal combination as
they have been designed to work together and are professionally packaged. However, after much research, we were unable
to determine a way to control the LCD via an Atmel microcontroller. The touch screen, however, is an analog
device with a rather simple interface that we were able to discern with minimal
effort. Separating the two, we were able
to mount the touch screen on a Crystalfontz 128x64 pixel graphical LCD which we
would actually be able to control. The
game of Simon was chosen as our application because it allows us to utilize
both the LCD and the touch screen to make a game that is impossible to play on
a television alone.
When thinking of possible projects, we considered what hardware we
had available that we might be able to use to do something interesting. At the top of this list was an old Palm m125
that would no longer power on but still had a functional LCD and touch screen,
which Mike owned. Previous ECE 476
projects have incorporated graphical LCDs, however not with a touch screen
interface. At this point we needed to
determine if it would be possible to use either the Palm LCD or touch
screen. After a good deal of research we
were not able to find any drivers or documentation for the LCD or the touch
screen used by the Palm m125. This
removed the possibility of using the Palm LCD because it would be far to
complicated to reverse engineer the protocols.
The touch screen however, only has four leads so after a few simple
tests we determined how the device functions and devised a simple way to use
it.
At the advice of
Due to the flexibility of a handheld device with an LCD and touch
screen, we were free to determine our final application of this technology
midway through the project after we had the components working. We considered using aOS to create a very
simply operating system, however that would have required converting motions on
the touch screen into characters much like Palm’s Graffiti. We determined early
on that the touch screen’s analog output did not have a high enough resolution
or linear relationship to obtain precise and intricate motions. Therefore we decided that we would implement
a game that could be played with an interactive device such as a touch
screen. The natural choice based on our
LCD and touch screen resolution was Simon.
For those not familiar with Simon, it is a game originally designed by
Hasbro in 1978 in which the player is shown four tiles flashing in a particular
order. Each round the number of tiles
flashed increases and the speed at which they flash can also increase. The player must memorize the order and press
the tiles in the proper order. There is
also a time limit on pressing the buttons that we chose not to implement in our
version, due to the need for the user to hold down and press firmly on the
touch screen. This along with the low
linear resolution inevitably forced our game design to one incorporating fairly
large blocky areas. A game like Simon,
which uses 4 such blocks lends itself well to such design requirements.
A number of tradeoffs between the hardware and software were
apparent when designing the system. The
main tradeoff was the low linearity resulting from the analog-to-digital
conversion of the x and y positions on the touch screen. While for the most part the voltages
increased or decreased from one side of the touch screen to the other, the
incremental voltages jumps between arbitrary points would vary and sometimes
not change at all, thereby creating certain areas that were very difficult to
discern through conversions. This was
the main reason that Simon was chosen as the application, because Simon uses
fairly large boxes. The only other
tradeoff was between the LCD screen and touch screen. Because the touch screen is much larger than
the LCD screen, only a portion of the touch screen is usable. Therefore we used the portion of the touch
screen with the most linearity to offset the first tradeoff as much as
possible.
We are unaware of any patent infringements in designing our handheld
device, however we are aware of the original Simon patent (see the References
section). As far as trademarks, besides
using the Atmel chip, we used the touch screen from a Palm m125 and the
graphical LCD is from a company called Crystalfontz. Finally we attempted to use Luke Delaney’s
driver code for the graphical LCD, however we had to change and rewrite a large
portion of it in order to use the LCD at all.
While our project incorporates two pieces of hardware that were
not used in the lab section of the course, the actual design required to use
these components proved to be quite minimal.
The LCD requires power and ground connections along with 14 connections
to the microcontroller ports C and D.
The only additional circuitry required is a potentiometer, which
controls the contrast. The LCD we are
using has an EL backlight, but requires a $12 chip that did not fit in our
budget to control. As such pins 19 and
20 are not connected. A full listing of
the MCU to LCD connections is below:
Mega32 CFAG12864 Name
VCC Pin 1 VCC (+5V)
GND Pin 2 GND
-- Pin 3 Contrast
PORTD.0: Pin 4 Data Bit 0
PORTD.1: Pin 5 Data Bit 1
PORTD.2: Pin 6 Data Bit 2
PORTD.3: Pin 7 Data Bit 3
PORTD.4: Pin 8 Data Bit 4
PORTD.5: Pin 9 Data Bit 5
PORTD.6: Pin 10 Data Bit 6
PORTD.7: Pin 11 Data Bit 7
PORTC.0: Pin 12 Column Select 1
PORTC.1: Pin 13 Column Select 2
PORTC.2: Pin 14 Reset
PORTC.3: Pin 15 Read / Write
PORTC.4: Pin 16 Data
/ Instruction
PORTC.5: Pin 17 Enable
-- Pin 18 -VCC (-5V output)
-- Pin 19 Backlight Anode
-- Pin 20 Backlight Cathode
The touch screen, on the other hand, did require some circuitry to
use, albeit very minimal. The touch screen
has four leads (Up, Down, Left, Right) which connect to a metal strip on each
of the sides of the screen. Since the
touch screen functions by varying resistance between the leads based on
pressure, the bare minimum circuitry required would be a voltage divider in
which the touch screen took the place of one of the resistors. For our design we chose to fix Up and Right
as inputs to the touch screen (outputs from the MCU) and Down and Left as
inputs to our filter circuitry. Pull
down resistors were also added between both Down and Left to ground to complete
the voltage divider. Our filter
circuitry consisted of a very basic RC low-pass filter (R = 30kΩ, C =
.47μF) with a cutoff frequency of 11Hz followed by an op amp follower
circuit with a gain of 3. Schematics for
this circuit along with the entire LCD and MCU circuitry can be found in the
schematic appendix. The outputs of the
gain circuits are tied to ADC channels 0 and 1.
Though done entirely done in software, the timing required to
operate the graphical LCD was probably one of the most complex low level issues
in the project. Because of the low level
timing and bit-toggling requirements, we have decided to discuss this issue in
the hardware section. Below are the read
and write timing diagrams for the CFAG12864 model graphical LCD. This diagram is taken from the CFAG12864
datasheet.
The most important timing requirements are as follows: First, both
phases of the enable signal (E) cannot have a period of less than 500ns, thus
the highest frequency that it can be toggled is 1MHz. Second, all control lines must be changed
while enable is low and remain constant until enable transitions low
again. Third, all data lines must be set
for the falling edge of enable. All
setup and hold times are substantially less than 1μs and can be found in
the datasheet. Though it is not
specifically a timing concern, we strongly recommend that anyone who attempts
to use this LCD set the Read bit high when not actively writing. This will prevent artifacts from appearing on
the screen.
The software was designed in a bottom-up approach, where we first
wrote the low level driver controls for the touch screen and the LCD and then
used them in conjunction with the Simon top-level game.
As stated in the hardware design section, the microcontroller has
four connections to the touch screen, two control lines which apply voltage
across the screen in the x and y axis and two analog inputs representing the
position on the x and y axis. The two
control bits allow the touch screen to be run in one of three useable
modes. Voltage can be applied to the y
but not the x, the y but not the x, and both the x and the y. Theoretically both control bits can be set
low, but then the touch screen’s only possible function is to protect the LCD
from being scratched. In a perfect
world, the touch screen would behavior linearly and the x and y axis would not
be coupled. In the real world, we mapped
the ADC values to a grid that we touched for all three useable control
configurations. For our low resolution
purposes leaving the control lines such that there is always 5V applied to the
y axis and 0V applied to the x axis proved to give adequate grid. While we developed a more complex scheme in
which the control states were toggled and the compilation of all 3 was used to
determine the current position (function driveXY for bias switching), we
determined that there was not any real advantage in switching between voltage
while probing and debouncing the touch screen inputs.
In order to sample the x and y positions, we simply toggle the
analog-to-digital lines from A.0 to A.1 continuously (functions sampleY, sampleX and
getPlayerSample). To allocate enough
time for x and y conversions, we implemented a flag scheme that allows the
system to sample either the x position or the y position instead of simply
sampling both one after another. This
scheme allows enough time for the ADC values to settle as well as for the ADC
MUX to change input lines.
Once sampled, the ADC values are converted to positions on the
screen. There are two different cases
for inputs from the touch screen. Either
the player is choosing the difficulty level to play Simon in, or the player is
pressing the boxes on the screen (functions convertLines and convertBoxes, respectively). Both conversion
processes are separate and compare ADC values for both x and y differently to
obtain the proper choices.
Initially the CFAG12864 model LCD was chosen because software
drivers for the Atmel Mega32 were available on the ECE 476 website courtesy of
Luke Delaney. However, we quickly
realized when attempting to use these drivers that there were serious errors
with his code. While it appears that he
had the proper structure at a high level, many of his low level bit
manipulations were blatantly wrong and as a result, we could not even use it to
turn on the LCD. After attempting to
modify his code to make it work, we simply deleted the code in most of his functions
and rewrote it based on the timing diagrams given in the datasheet for the
CFAG12864 (see the hardware section above.
The most important things to be noted from that process is that most of
the control information is actually carried on the data lines when the control
bit for instruction/data is set to instruction and that whenever you are not
writing an instruction or data to the screen, the Read/Write line must be set
to read to prevent artifacts.
The LCD uses a data and control line where the data line is
interpreted in a variety of ways as specified by the control bus. The exact values for each line is specified
in the hardware section and specific functions are used to set, reset certain
bits in the control line (setbit and resetbit), write data and instructions
to the LCD (writedata and writeinstruction), and set the cursor positions on the LCD (setcursorpos). The writedata
function allows for the actual sending of an 8-bit value to be transmitted to
the LCD and interpreted on the LCD screen as a pixel set, where a logical ‘1’
sets a pixel and a ‘0’ resets a pixel.
The cursor positions on the LCD can range from 0 to 7 in the x direction
and 0 to 127 in the y direction. As
mentioned before, proper timing issues needed to be taken into account when
writing to the LCD; therefore delays are utilized to incorporate these needed
shifts.
Once these driver functions were written, all subsequent graphical
functions could then be implemented.
Functions such as clrscreen2, blkscreen2, makeblock and makeBigBlock
are all simple functions that are used for clearing the screen, blackening the
screen and making blocks of different sizes.
The only caveat with these functions is that the orientation of the
screen is different then the actual manufacture’s design:
|
Manufacture’s
Orientation |
Our
Orientation |
|
|
|
The setcursorpos function
was written to behave as if the LCD is oriented in how the LCD manufacturer
designed the screen (so that future users of these drivers can use the CLD as
designed). Therefore in order to use the
LCD how we wanted to, we incorporated the needed X coordinate inversion, and
changes were made to the higher level functions: printWord and makeblock. All other print functions stem from these two
functions. When data is written to the
LCD, the Y position is automatically incremented; therefore we used that to our
advantage by making all blocks, letters, numbers and spaces 8 rows deep,
thereby using a very simple for-loop without the need to keep changing the
overall position of the cursor with each iteration. This allows for much faster rendering.
The function printWord is used to write a word up to 8 characters in
length. The characters within the “word”
can be any capital letters, numbers, or spaces.
The actual bit representations of these characters are set in flash in
an 8x8 bit array for each character.
These letters allow for a blank line below the character and three blank
columns to the right of the character for easier readability. This representation is similar to the
character representation used within the Lunar
Lander lab assignment where a huge array is produced with all available
characters and is executed during execution.
The character arrays are separated every 8 indexes by each row
representing a portion of the character.
All of the higher print functions are self explanatory such as displayGrid (displays the 4 boxes for
Simon), splashscreen (initial startup
screen), displayDiffScreen (displays
all difficulty choices), displayGoodGame
(displays the win condition screen), pushbuttonChoice
(clears one of the boxes on the LCD) and finally showCorrectOrder (iterates through Simon’s List and “blinks” the
boxes off and on).
The functions getRandomNumber
takes Timer 0’s two lowest significant bits and uses that as a random number
generator from 0 to 3 and assigns the generated number to the end of Simon’s
list. Finally the lcdDebouncer debounces the values from the touch screen and is
called every 20 Hz. The debouncer works
very similar to the original debouncer, however the debouncing is used to
detect an area for a box press and a non-pres” where a non-pres” acts as the terminator for the debouncer after a valid
box has been pressed. This allows for
the removal of 2 of the states that was using in our original debouncer
function when using the keypad, because in the keypad case, the terminator was
another key being pressed, and therefore we do not need to poll for another key
when using the touch screen. Instead we
just poll for the player releasing stylus from the touch screen.
Using these touch screen and LCD print functions allowed us to
implement the game Simon. The logical
structure is fairly simple and is shown as a state diagram in Figure 1. Basically the game initializes and displays
the splash screen with our names. A
difficulty select screen is then displayed in which the player can select one
of three difficulty levels. The game
commences and the computer randomly chooses one of the 4 boxes (using good old
Timer0). The program then re-iterates
the complete Simon List starting from the first random number generated to the
most recent thus reminding the player the order needed to continue the
game. Now control is given to the player
in which the player must tap the boxes in the correct order to continue
playing. Because of the debouncing,
visual cues are given to the player with each successful box registration. If the player at anytime misses the correct
order, the game ends and the player can play again by simply tapping the touch
screen. However if the player makes it
through the entire set of boxes in Simon’s list (the maximum of boxes increases
with the difficulty), the game ends on a good note, and the player can still
play again by pressing the touch screen.
Figure 1
Our handheld Simon game behaves very
well. Most of the program runs within
the main while loop while only using a timing frequency of 20 Hz for the
debouncer. While the initial tests
produced some flickering, the final result is very well polished and even encapsulated
in its own casing allowing for portability while protecting the circuitry from
the outside environment. The touch
screen’s accuracy was tweaked and analyzed for a substantial amount of time and
after many careful calibrations behaves quite well in dealing with the Simon
game. Due to lack of linearity,
calibration was needed to be done to each of the input screens, therefore any
future addendums or other handheld programs needs to be calibrated. This proved to be the only pitfall in our
project. We would of liked to obtain a
general methodology to obtain x and y positions, however due to non-linearity
on the touch screen these generalizations were not possible.
Our system does not interfere with
any other designs because it is self contained and does not use any type of
wireless communication nor does it interface with any inherently noisy devices
such as motors. Finally, we spent a
great deal of time in the interfacing of the device with the player. We made sure that the debouncer worked very
well in capturing the inputs from the user and made sure the user knew that an
input was registered by the program by displaying the current player turn. When asking the TAs to play the game, most
learned the rules and needed pressure to apply for proper playability.
The following are some screenshots
of our Simon handheld:
|
|
|
|
Circuit
Board: Front Side |
Circuit
Board: Back Side |
|
|
|
|
|
|
|
Power
Off |
Initial
Startup Screen |
|
|
|
|
|
|
|
Difficulty
Select Screen |
Pressing
a Box |
|
|
|
|
|
|
|
Win
Screen |
Lose
Screen |
|
|
|
|
|
|
|
Back
of Packaging |
|
For the most part, our final design met
our expectations quite well. The one
fault it the resolution of our touch screen.
Due to the nonlinear nature of its behavior along with the fact that
changing the amount of pressure applied will change the resistance, the degree
of resolution that we were able to achieve was substantially less than the
screen is capable of. Since Palm does
not release any documentation of this sort of component, our entire
understanding of the behavior of the touch screen had to be learned through
experimentation and reverse engineering.
It is possible that we were simply operating the screen at improper
voltage ranges or with the wrong values of resistors in the voltage
dividers. While touch screen driver
chips are available from Analog Devices, we decided that such a chip would only
have added an extra level of complexity to our design and possibly thrown us
over budget. If we were to do this
again, more research would have gone into these driver chips. It is also possible that our initial removal
of the touch screen from the Palm LCD damaged the component.
The only time we used someone else’s code
or code available in the public domain is when we attempted to use Luke
Delaney’s LCD driver code. However as it
did not work and we were forced to rewrite it, only some of the high level
structure and variable declarations are his.
As this code was part of a MEng project for
The game of SIMON is a product of Hasbro
Inc though the name Simon is not trademarked, so we are free to use the
name. The original patent for the game
is linked in our references appendix, though it has long expired. Since we are not attempting to make a profit
or sell this device, we are not violating any copyright, though we could not
find one for this game.
In order to prevent injury to either human
or the handheld, we have packaged our handheld device in protective
cardboard. However with our limited
resource we had to tape the packaging shut which requires cutting the tape when
the battery needs to be replaced. It is
possible that a person could injure himself with the blunt plastic stylus, but
such a person would likely have done massive amounts of damage to themselves
with a standard pen or pencil.
Ethical
Considerations:
2. To
avoid real or perceived conflicts of interest whenever possible, and to
disclose them to affected parties when they do exist;
When considering applications for our handheld, we considered something
that we might also be able to use for RoboCup, but decided it might be viewed
as a conflict of interest so we decided to stick to a video game.
3. To be honest and realistic in stating claims
or estimates based on available data;
When proposing our project we made sure to explain that it might not be
possible to interface with a graphical LCD and make a meaningful application
and not claim that we could do it with absolute certainty.
4. To reject bribery in all its forms;
We rejected all bribes in the making of this project.
5. To improve the understanding of technology, its
appropriate application, and potential consequences;
We feel that we have improved the understanding of technology for
future ECE 476 classes by providing a working set of drivers for an
economically feasible graphical LCD.
8. To treat fairly all persons regardless of such factors
as race, religion, gender, disability, age, or national origin;
Our handheld device will be useable by people of any race, sex or creed.
/*
ECE 476 Final Project: PPP Palm
Jeff Johnson (JAJ37)
Mike McCabe
(MFM8)
PDA device
incorporating a touch screen and graphical LCD to play the game SIMON.
Gameplay:
The game SIMON is a memory game in which
the player must tap one of
the 4 boxes on the
screen in the random order generated by the program.
The program
randomly selects one of the four boxes and the user must press
that box. The program then selects another random box
and then reiterates
the order back to
the player. This continues until the
maximum number of
iterations has occurred
(set by the difficulty level).
The user can select one of three
difficulty levels. Each difficulty level
increases the
number of boxes to memorize as well as the delay time between each
box for which the
program reiterates the order after every successful pass.
Whether the player wins or losses the
player may reset the game by simply
tapping the
screen at the win or lose screen.
Hardware
Interface:
Their are two hardware interfaces with
the MCU. The first is the touchscreen
which must be set
to PORT A due to ADC usage. The second
is the graphical LCD
For this we use
ports C and D, for control and data busses respectively.
PORT A (ADC enabled): Touchscreen
Pin 0: Y Position
Pin 1: X Position
Pin
2-7: NOT USED
PORT B: NOT USED
PORT C: Graphical LCD Control Bus,
Touchscreen Voltage Driver
Pin 0: Column Select 1
Pin 1: Column Select 2
Pin 2: Reset
Pin 3: Read / Write
Pin 4: Data / Instruction
Pin 5: Enable
Pin 6: Touchscreen X Voltage Driver
Pin 7: Touchscreen Y Voltage Driver
PORT D: Graphical
LCD Data Bus
Pin 0: Data Bit 0
Pin 1: Data Bit 1
Pin 2: Data Bit 2
Pin 3: Data Bit 3
Pin 4: Data Bit 4
Pin 5: Data Bit 5
Pin 6: Data Bit 6
Pin 7: Data Bit 7
*/
//--------------------------------------------INCLUDES
#include <Mega32.h>
#include <delay.h>
//#include
<math.h>
#include <stdio.h>
//#include
<string.h>
//--------------------------------------------DEFINES
#define t1 50 //Timer
for Debouncing (20Hz)
#define DISPLAY_ON 0x3f //0011 1111, LCD
Turn on initialization
#define DISPLAY_OFF 0x3e //0011 1110, LCD
Turn off initialization
#define easySize 5 //Simon
List size set to 5
#define medSize 10 //Simon List size
set to 10
#define hardSize 15 //Simon
List size set to 15
#define easyDelay 250 //Reiteration
delay set to 250 ms
#define medDelay 200 //Reiteration
delay set to 200 ms
#define hardDelay 150 //Reiteration
delay set to 150 ms