Over the past few months I’ve typed out 20 stories (Peppa Pig, Xiǎo Zhū Pèiqí) in Chinese language with Pīnyīn. This can act as base for some research.
First of all, the word count: 10,024 words, 53,115 characters, 11,842 vowels with diacritics. Questions arise. Answers are possible. For example:
Which tones are the most frequently used in the Chinese language? In vernacular modern Standard Chinese, that is, as Peppa Pig stories are written with that. Any Chinese teacher will say Chinese has 4 tones, and “Nǐ hǎo”, is as common as “Zàijiàn”, but is it really?
Finally, with Pinyin I can answer this question. Here is a screenshot of a software I designed to do the counting:
Clearly, the most common tone-marked tone in Chinese language is À, the 4th tone on the letter A. This is surprising to me, and relevant for Chinese Pinyin input method designs.
For example, after a lengthy calculation and reasoning, I found the following: By making à the default letter instead of a, there could be approximately 2.35% fewer keystrokes necessary to type such text in Chinese Pinyin. For the above text that would be exactly 1,356 fewer keystrokes.
While at it, the complete vowel depository looks like this:
It seems like, as if in vernacular (modern spoken) Chinese, the most common vowel is the letter I, by far, which came as a surprise to me, too.
If we look at the total word repository from the dictionary CC-CEDICT, we get a similar picture in many regards, but quite different counts for the neutral letters A, E and U. However, keep in mind that this is not spoken Chinese, but a repository, and thus more an academic exercise than a look into the soul of the Chinese language:
So far I have created two systems for Chinese Pinyin input (with two more in the making):
🍏 Method 1. Number marked Pinyin Input
Chinese Pinyin is written with diacritics. However, even modern computers notoriously lack convenient ways to input diacritics. Therefore many input systems have emerged. One of the most common ones is “number marked Pinyin input” which will automatically change the numbers 1-4 into the respective diacritics as we type.
For example typing dang4 (or da4ng) will produce dāng.
I’ve spent quite some time to write my own software to do so, and in quite a few variations, too. Here’s a screenshot of one of the Proof of Concepts I’ve created:
When typing with numbers we need one extra keystroke to produce a vowel with diacritic. This means:
- A definition: I will call vowels with diacritics, such as “ā, á, ǎ, à”, dvowels – the word vowel with a “d” in front, d for diacritic.
- For 11,842 dvowels we need double the amount of keystrokes than for vowels without diacritics, in total 23,684 keystrokes.
- This means, when typing, for example, 10,024 words with 53,115 characters (as in the Peppa Pig texts above), we will not have to type 53,115 keystrokes in total, like in a language without diacritics, such as English, but 64,957 keystrokes in total.
- The math for this is: 53,115 characters – 11,842 vowels + 23,684 dvowels = 64,957.
- How much is that in percent? 11,842 / 53,115 * 100 = 22.295
To write Pinyin Chinese with diacritics, with the number marked Pinyin input method, we have a keystroke overhead of 22.30% .
This is unavoidable and the bare minimum. It is the most direct way to input Pinyin without predictive typing methods, where a computer will suggest possible words in advance and allow the user to chose a word, rather than type it.
The qualm I have with number marked input is twofold:
Qualm 1) Reaching up for the number row is inconvenient, especially for longer typing. We humans can get used to most any inconvenience, but still it is inconvenient, or let’s say, not ideal in terms of ergonomics.
However, there’s plenty of room for optimism, I mean optimization. For example, with the stats above it became clear that a 4th tone a (à), is by far the most typed vowel with diacritic. This means, improvements can be made. For example, one idea I had yesterday:
Problem: When using number 4 to produce the 4th tone, we need to use the left hand for both keystrokes: key a and key 4, which are both on the left side on the keyboard.
Solution: It would be easier and faster if we also could use, for example, key 8 to produce the 4th tone, too. In this way we can use both hands for typing, and be faster, and more comfortable.
I’ve made a proof of concept for that, too, and it seems to indeed be quite an improvement in the typing experience. To be observed.
Qualm 2) The bigger problem I have with using numbers to type text is the thing in itself, why do I have to use numbers to write text? This doesn’t make sense to me. I understand that it is for reasons of convenience, especially when it comes to predictive text and assisted typing. But coming from a humanist background, it seems unacceptable to me to have to use numbers to write down words.
This bigger, psychological, “I stand by my principles” problem I will try to solve with inventing input methods that don’t need numbers to input text.
🍏 Method 2. Twin Tone Pinyin Input, TT-Input
If you’ve never heard of this input method before: I’ve made this one up. It’s my own invention. It’s something new.
Years of frustration, months of intense work and research, and 16 hour days of complete obsession led me to produce a robust, alternative system that reliably produces Pinyin text without the need to use numbers.
I’ve also created a proof of concept, a working software implementation. I’ve had many different attempts and angles at this problem, and ran all of them against the 102,000+ words from the CC-CEDICT dictionary. However, most of the algorithms produced an error rate of a minimum of 5%, which is unacceptable.
But this, TT-Input, this one is flawless:
Twin Tone Input (TT-Input) works by repeating a vowel twice and then typing a marker to produce the diacritic: “r” for á (2nd tone), “v” for ǎ (3rd tone), “f” for à (4th tone), or repeating the vowel one more time for the 1st tone.
For example, typing aaa gives ā, aar gives á, aav gives ǎ, aaf gives à.
aaa → ā
aar → á
aav → ǎ
aaf → à
Buufyoofng daaanxiiin → Bùyòng dānxīn
This means, just like in Telex for typing Vietnamese, we don’t need to reach up to the numbers row to produce a diacritic. Instead we use letters to type letters.
But now for the overhead. How much worse is it in terms of keystrokes?
- With TT-input, we need 3 keystrokes to type one dvowel.
- For 11,842 dvowels we need a total of 35,526 keystrokes.
- This means when typing 10,024 words with 53,115 characters (as in the Peppa Pig texts I’ve transcribed), we will need to type 76,799 keystrokes in total.
- Here’s the math: 53,115 characters – 11842 vowels + 35526 dvowels = 76,799 keystrokes in total.
- The percentage for that is: 11,842 * 2 / 53,115 * 100 = 44.59
The “TT-Pinyin Input” keystroke overhead is 44.59%. As opposed to 22.3% with number marked input.
Which means: In publishing, a standard manuscript page is often 250–300 words, which translates to 1,500–2,000 characters (including spaces). Let’s say 275 words and 1750 characters, including whitespace. To type 1 standard manuscript page, being an average typist with ~300 KPM (60 WPM):
- 5 mins 49 s (1,750 keystrokes) – English (No Diacritics)
- 7 mins 6 s (2,135 keystrokes) – Chinese (Number-Marked Pinyin)
- 8 mins 26 s (2,530 keystrokes) – Chinese (TT-Input)
To conclude, for today:
The 22.30% overhead for number-marked Pinyin input is the minimum possible for producing diacritics, if we disregard predictive text and assisted typing algorithms.
Opening keys 4–9 for number-marked input could be a game-changer. Assigning tones to the right-hand number row allows left-hand vowel input + right-hand tone selection, creating a parallel, two-handed input flow. This could speed things up significantly.
The 44.59% overhead for my latest invention, Twin Tone (TT) Pinyin Input, is double that of number-marked input. While TT-Input improves ergonomics by keeping the fingers close to the home row, the extra keystrokes could slow down typing overall, and also take a toll on the finger joints for the extra stress. On the other hand the shorter traveling distances for the fingers could improve comfort and rhythm, and might be worth investigating this input method further.
