Do Androids Dream Of Handwriting Recognition?

Engineers and programmers have been trying for decades to teach computers and other electronics to recognize handwritten text. Only in the last few years have the world’s largest software companies made significant progress teaching smartphones and tablets to adequately recognize handwriting and translate it into typed text on the screen.

This August, Samsung launched its Galaxy Note 10.1 tablet. As one of the key competitive advantages of the device, the manufacturer cited the S Pen recognition system, created in partnership with Wacom. A similar feature exists in the Galaxy Note II device, which Samsung recently presented at the Berlin Consumer Electronics Show IFA.

The handwriting recognition function included in the device makes it just as comfortable to work on with your fingers as with the electronic S Pen. In the latest devices, the S Pen allows the transmission of up to 1,024 degrees of clicking on the screen — achieving essentially the same precise screen recognition as if writing with an actual pen and paper.

The Galaxy Note also supports the Shape Match and Formula Match features, allowing graphs, figures, charts and mathematical formulas to be drawn or recorded using either the S Pen’s recognition and subsequent conversion to text, or the graphical format.

But why, in 20 years of mobile, is this groundbreaking handwriting recognition feature just now being introduced? The answer is, it isn’t.

The first attempts to teach computers to understand handwritten text began in the Soviet Union in the 1960s, when the emergence of personal computers and smartphones was, it seemed, in an uncertain distant future. At the time, the space industry was driving the development of handwriting recognition as a way to avoid sending pencils and paper into space to cut down on costs.

Each extra gram sent into space amounted to huge additional costs. Commercial use of handwriting recognition had not been identified at the time, but by the mid-1990s, with the appearance of the first handheld computers known as PDAs, keen interest was shown by various device manufacturers.

Some form of handwriting recognition was installed on all the early devices from Palm, Apple (which entered the market with one of the first prototypes of the modern smartphone, Newton) and PDA manufacturers working under the first version of Windows Mobile. Interestingly, handwriting applications did not become the “killer apps” programmers thought they would be, simply because their development didn’t meet consumer expectations. They just didn’t work well.

Teaching’ PDAs every letter and number written by hand was too complicated and the recognition accuracy was very low. The qualitative leap in the handwriting recognition market didn’t happen until about 20 years later, due to the massive proliferation of tablets and smartphones. Entering information by hand was a natural development in the new mobile world, with increased opportunities for handwriting recognition on the sensitive touch screens that could be found in nearly every purse, pocket or briefcase.

With increased precision in the technology, handwriting recognition has gained traction. The most popular devices and platforms all have handwriting recognition services available, especially as mobile applications. In the App Store, a number of programs can be found that allow graphical information to be drawn on the screen and saved as pictures — and some programs even have the ability to recognize handwriting, including the popular note-taking app Evernote, which not only supports handwritten text notes, but also audio, speech-to-text and the ability to scan documents with cloud archiving capabilities.

The iPad tablet, because of its size and shape, naturally lends itself to being scribbled on by hand, but a special stylus for this task costs about $80. Handwriting apps aren’t just for iOS. In Google Play there are a number of applications based on handwriting recognition technology. Handwriting Dato and Handwrite Note Free, for example, are designed to store and catalog handwritten notes. More “advanced” applications, such as MyScript Calculator, are designed for writing complex mathematical calculations by hand.

A new handwriting recognition feature is also expected to be supported in the latest Microsoft Windows 8. The handwriting recognition function was already available to Windows 7 users, but only enabled by expensive electronic pens and limited to just a few Windows 7 tablets. Microsoft has made a significant step forward with Windows 8, according to one of the beta testers of the system, making writing on the screen almost as easy as writing with a pen on paper.

It is not only gadget manufacturers and their software developers that have shown interest in handwriting recognition, but also other industry players. The German automotive manufacturer Audi equipped their 2011 A8 and A6 models with on-board computers that support handwriting functions capable of entering information into the Multi Media Interface, which includes the car’s media player and navigation system. Audi’s rationale was that some users find it easier to manually enter information rather than pressing buttons.

Google, also recognizing the attraction of alternate input methods, launched its Google Handwrite feature in July to facilitate handwritten Google web searches on touchscreen smartphones and tablets. Available in 27 languages for iOS5 and higher, and Android smartphones running 2.3 and higher (4.0 and higher on tablets), Google Handwrite looks to be a natural enemy of anti-keyboard pecking input apps like Swype, Swiftkey or PenReader, and even the company’s own Google Voice Search speech recognition input method.

Not so fast. Literally.

Writing English characters by hand is still viewed as more cumbersome than typing, many believe. The path of adaptation seems to be to satisfy the ability to input information quickly — whether by typing with your fingers or through voice recognition. There has been significant progress in these areas, with pre-integrated alternate input methods in high demand. But, for whatever reason, handwriting recognition still remains a niche.

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone

Katia Shabanova is director of PR at Paragon Software Group where she handles editorial exposure for the company's mobility and disk utilities divisions. Katia studied linguistics at Moscow State Linguistic University and University of Texas at Austin; English and German philology at Santa Clara University, California; and earned Master of Arts degrees in English and German Philology at Georg-August University of Göttingen, Germany.