modulus blog

When I draft my e-books, my editor often complains about my use of the abbreviation “e.g.”. She prefers me to write “for example” and, even then, there is always a struggle for me to get the commas in the right places.

Even worse, “for example” often is not all that I wish to say; often I’m trying to say something of the style “that is, for example”, so I wind up doubting whether I’m really wanting to “i.e.” or “e.g.”

I’ve decided to cut the Gordian knot; from this day hence, when appropriate I will use the newly coined T.L.A.* “i.e.g.”, meaning “that is, for example,”. I’m sure you will find i.e.g. a perfectly cromulent T.L.A.

* T.L.A. = Three Letter Abbreviation.

Google vice-president of search product and user experience, Marissa Mayer made an interesting observation recently, when speaking of on-line newspapers:

I can go to various newspapers and when you scroll down to the bottom, what do you do next? There were related stories and related videos but those were up on the top. So now the most committed user – the one that reads through the entire piece – is now looking at the bottom of the page with nothing to do.

It is well worth while looking at each of your web pages with the specific aim of ensuring that there is something engaging to reward your most committed site visitors, at the end of each article, at the bottom of each page. This is especially true for longer articles or pages.

When specifying text size in HTML markup by way of the font-size attribute in CSS, the units used can be either absolute or relative. Absolute units (such as pixels) mean that the website designer retains more control over the appearance of the page, but that requests from web browsers for larger or smaller sizes will be ignored. When relative sizes are used (such as ems or percentages) some additional testing is needed to ensure that the page does not become illegible or jumbled at large and small point sizes, but the degree of convenience offered to those whose vision is less than 100% is, in my opinion, well worth the extra five minutes of testing. Furthermore, relative sizing allows for both automatic and manual adjustment of the font size on devices such as mobile ‘phones.

I visited 40 randomly-selected websites and checked each for compliance with requests from the web browser to increase or decrease the text size (note that this is not the “zoom” function present in some browsers, it should affect text only. The results are as follows:

“Full” means that all text on the site resizes as requested by the browser, and that the end result is still legible.

“Partial” compliance means that while much of the text might resize as requested, some of it does not (or the site becomes illegible due to text overlaps); the most common symptom here is menu text which stays at the same fixed size.

“None” means that, apart from some minor and accidental spacing changes, none of the text responds to requests for size changes.

As I often do, when planning development of my text processor I consider first what capability Microsoft Word has. I have Word 2000; its “Thesaurus” function is really a list of synonyms (with the occasional antonym) rather than a proper thesaurus. When you look up “updated”, the Thesaurus form looks like this:

It’s not very pretty. There are four input boxes, four buttons, three labels and a control alignment which appears to be largely random. The substantive content is a flat list of synonyms. It occured to me that there might be better designs available.

After some trial and error, I have ended up with the following design in my text processor, Weblex:


The synoym list uses a Windows Tree-view; when you select a synonym by clicking on it, synonyms for that synonym are presented, as is the case for “modernism” displayed above.

There are two input boxes and three buttons, one label. One of the buttons is additional to the Word fuctionality (“Find on WWW”), so the net reduction in clutter is quite significant.

The moral of this story is that, despite the owerwhelming number of designers, software engineers, ergonomists, hair-dressers and spin doctors in the larger software development firms, it is still not only possible but indeed feasible realistic practical workable viable to compete with them on good design.

In 1980, Martin Porter published a stemming algorithm for English words; stemming is essentially the reduction of a word to its stem by removing its suffixes. Thus, for example, the word “stemming” can be reduced to its stem “stem” by use of Porter’s algorithm. This is immediately useful, since, when searching for the word “stemming”, we can also search for the word “stem”. Normally “stemming” will not match with “stem”, but, when reduced to its stem by suffix removal, it will.

The Porter algorithm removes about 150 common suffixes by way of an algorithm that occupies about 400 lines of Pascal or about 100 lines of Perl, courtesy of the latter’s regular expression library. For those looking for implementations of the algorithm in a variety of languages, the best starting point is Martin Porter’s own website,
tartarus.org/~martin/PorterStemmer/.

The algorithm is useful for searching (Google makes use of it) and a variety of language processing tasks. One interesting use is to find alternate bases for synonym searching. In my “weblex” text editor, when a user seeks synonyms for the word “interesting” the following happens:

1. synonyms for “interesting” are sought in the synonym dictionary and presented to the user
2. the stem “interest” is determined using the Porter algorithm
3. synonyms for the stem are sought
4. about 150 suffixes are added to the stem and, for each Frankenstinean suffixed form, synonyms are sought and presented to the user together with the suffixed form

One possible optimisation would be to check whether each suffixed form is an English word before seeking synonyms, but as the process descibed above completes in milliseconds this
does not seem to be necessary.

Software documentation and comments are highly variable in quality and usefulness. At the useless end of the spectrum are the detailed source comments which tell you what the next line of code is going to do. As the next line of code tells us exactly the same thing, this is at best pointless; at worst, when the comment does not accurately describe what the next line of code is going to do, it can be seriously misleading; here’s an example of what I’m describing:

#increase each object's value by 10
for (sort keys %objects){
$_ += 10;
}

Perl programmers will notice that the code does not increase the object values, but rather the keys of the %objects hash, so the comment is not only redundant but also plain wrong.

On the other hand, very high-level, abstract documentation which tells us what a program or function library does conceptually is only useful in a very high-level, abstract sort of way.

One type of documentation which is much more successful is the “Javadocs” style of functional documentation, which describes each function (or subroutine or procedure) principally in terms of its inputs and outputs. “Javadocs” is a documentation methodology maintained by Sun Microsystems, originally intended for documenting Java functions, but easily adapted to almost any other language. Here’s an example of the use of Javadocs on a PHP function:

/** * Returns the capability for a given capability name.
 *
 * @author Modulus Pty. Ltd. - prh
 * @version 2008 1.0
 * @param $id unique string id of the device
 * @param $name string name of the capability
 * @param $fallback boolean for considering fallback

 * @param $fallbackChain array of strings, where known, provide this to avoid unnecessary repetitive lookups
 * @return string capability

 */

function lib_getCapability($id, $name, $fallback, $fallbackChain) {
   ...
}



This documentation is immediately useful in the source code for developers maintaining or altering the source code. Furthermore, the effort required to create and maintain the documentation is quite limited in relation to the benefit derived. However, you may wish to publish an API to your functional library without publishing the source code. One way to do this is to use our javadoc.module,
which, for a modest $19.95, creates elegant, valid and conformant XHTML documentation from Perl, PHP, Javascript or Java source code. Here’s an example of the output generated:

We become accustomed to seeing system messages such as “3 file(s) deleted.” However, this smacks of laziness. One useful function that can be written in your programming language of predilection is plural(). The plural() function is fed parameters of the number and word to be inflected and returns the word approriately inflected. For example, if we call plural(3,'file') it should return ‘files’. When we call plural(1,'file') it should return ‘file’ in order that we can say “1 file deleted” and, oddly enough, plural(0,'file') should return ‘files’, in order that we can say “0 files deleted.”

Of course, English formation of plurals is not regular, so a good implementation of the plural() function would have to deal with the many exceptions in pluralisation, such as class=>classes, potato=>potatoes, data=>data, calf=calves etc. Even then, when the system message is “3 files were deleted”, we also have to deal with the declension of the verb “to be” so as not to end up spouting nonsense like “1 file(s) were deleted”; to avoid the endless complexities of declining irregular verbs, it seems best to avoid verbs in locations in system messages where number is important, i.e. “3 files deleted” rather than “3 files were deleted”.

So, how do we deal with the apparently overwhelming complexities of English grammar, to put out a simple but gramatically-correct system message when we delete a number of files? The resolution is surprisingly easy, achieved by modifying the form of the function plural() so that it takes three arguments: the number, the singular form of the word and the plural form of the word. Thus plural(3,'platypus','platypuses') will return ‘platypuses’, whereas plural(3,'radius','radii') will return ‘radii’, without the need for a long and complex function. Thus "1 "+plural(1,'file,'files')+plural(1,'was,'were')+" deleted" correctly issues
“1 file was deleted” whereas "3 "+plural(3,'file,'files')+plural(3,'was,'were')+" deleted" correctly issues “3 files were deleted”.

Here is the whole of my Perl plural() function:

#**
#* Returns a string with correct plural treatment.
#*
#* @author Modulus Pty. Ltd. - prh
#* @version 2008 1.0
#* @param $nr the number governing the result
#* @param $sstr the singular string e.g. 'match'
#* @param $pstr the plural string e.g. 'matches'
#* @return $sstr or $pstr
sub lib_plural{
   shift == 1 ? shift: $_[1];
}

This morning I took on the task of merging 200 articles which have been published over four years at Kevin Dwyer’s Change Factory website. The aim of the exercise is to create a compendium of change management and business process management advice from the articles; they will be sorted into appropriate chapters, Kevin will write some brief “glue” text and an introduction, but we want to do it without wasted effort.

Fortunately the articles are quite consistent in format, as they are written using a template. I wrote a Perl program to work through the articles directory, retrieve each article, remove its headers and footers, discard some unnecessary content, and add the article to the (large) merged HTML file. After testing its basic functionality and fine tuning for a couple of minor exceptions, the HTML file was created, weighing in at 1.1 MB, and then read in to Microsoft Word and saved as a Word document, ready for Kevin to add the glue text and introduction. The Perl program is, in itself, quite unremarkable; any programmer could write the same in a language of their choice in half an hour.

It is convenient to have each article start a new page, but as HTML does not have a paging paradigm, where to start? This turns out to be dead easy; the aforementioned Perl program creates a level 2 HTML header block for each article, e.g. <h2>This is an Article Title</h2> with the article’s title. When read into Microsoft Word, these were automatically transformed into Words “Heading 2″ elements. All that remained was the page break, and this was readily achieved by creating a style for the merged HTML document using the print-oriented part of the CSS 2 specification, viz:

h2 {
   page-break-before: always;
}

Word (which does have a paging paradigm) recognised the instruction and promptly inserted page breaks before each “Heading 2″ element, just what was wanted. It sometimes surprises me when things turn out to be so simple to achieve. Something a bit more sophisticated could be achieved using the CSS 2 “widows and orphans” controls, but as this Word document is a means to an end rather than a finished product, I heeded Voltaire’s advice that “The perfect is the enemy of the good.”

In Internet terms, a “captcha” is a “Completely Automated Public Turing test to tell Computers and Humans Apart”, i.e. one of those images which requires you to replicate the text in order to pass to some next step, such as submitting a comment. The idea of captchas is not without its problems; for the disabled it can be a complete barrier to site access. Even for people without disabilities, entering some random alpha-numeric characters which have been deliberately distorted can be difficult.

Here are five examples of captchas which I consider to be quite difficult to get right:

Over-simplification of captchas can mean that they can potentially be solved by computers. However, in the vast majority of cases the “treasure” protected by a captcha is not of sufficient worth to bother trying to break the security by OCR or artificial intelligence techniques. Indeed, for the vast majority of sites using a captcha, the enemy is a simple-minded spambot.

One approach I’m trialling at the moment is to use text that is not composed of random alpha-numeric characters, but is a valid English word, randomly selected from a largish subset of English words. We want a set of words of moderate length (say 6-9 characters) and which are reasonably familiar to most readers. We can, for example, use the names of vegetables; these names are reasonably familiar to most site visitors, tend to be about the right length and, consequently, a human site visitor has a much higher chance of typing in the right text in repsonse to a word like “spinach”, whilst very little advantage is conferred to the simple-minded spambot by the fact that the word is a familiar English word. The use of real words decreases confusion between the letters “l”, “i” and “1″, as we
know that “spinach” is spelled with an “i”, not a “1″ or “l”.

Here is what I’m trialling at the moment:

This approach does reduce security and could be broken by a sophisticated program which combined image processing capability with dictionary lookup, but that doesn’t really concern me, as I am not using captchas to protect anything more valuable than the right to leave a comment in a guestbook or send an email.

introduction

The way in which people read text has been extensively studied since the 19th century, yet when one looks for definitive information on choosing an appropriate line-length for a website for optimum legibility, a wide range of conclusions is found. Recent experiments have shown that on-screen reading speed increases with longer line lengths (testing in the range 35 – 95 characters per line). Other experiments have shown that line length had little effect on readability, but that readability was substantially affected by the size of the surrounding white-space margins, with large margins substantially enhancing readability. Another recent experiment showed that margins slow reading speed, but increase comprehension. These studies are not necessarily conflicting; taken together they indicate that line length itself is not as key as might have been thought, but providing adequate white-space margins (which, coincidentally, will somewhat decrease line length) is important to comprehension, which is even more important to a commercial website than reading speed.

My local newspaper (The Age) uses a principal layout comprising eight columns, each only 45 mm. wide, which results in 28 – 34 characters per line – so short that, in combination with the fact that the text is justified, about 15% of lines end in a hyphen.

Unlike a newspaper or magazine, which has a set size combined with a set font and font size, the website visitor has a high degree of control over properly-designed websites and can change the overall size, text size, font size and even font family. This degree of flexibility more or less rules out hyphenation of text, which, in contrast with newspapers and magazines means that very narrow columns work less well.

Justification (i.e. where inter-word padding is inserted to make all lines equal in measured length) is the rule rather than the exception in books, magazines and newspapers, but the exception rather than the rule on websites. Justification looks much better when combined with hyphenation, and is therefore less appropriate for websites. In addition, only a limited range of browsers support justification.

techniques

The simplest and cleanest way to ensure that you have adequate white-space margins around text is to set the margins directly in CSS e.g.
margin-left: 15%;
margin-right: 15%
etc.

Note that it is preferable to set these margins as percentages of the available width (rather than absolute values) so that they adjust appropriately when the page is resized. Most websites will, however, be more complex in their layout and we must also bear in mind that “white-space” is not necessarily unoccupied.

Setting CSS to justify text (text-align: justify;) or to right-justify text (text-align: right;) works with Mozilla Firefox 2.0.0.13 and its cousin Netscape Navigator 8.0.4. With Internet Explorer 6.0.28
or Opera 9.27, these justifications can be made to work, but the inheritance mechanism in their CSS implementations is not correct and so each element that is to be justified (other than the default ‘left’) has to have the justification specifically set i.e.
body {text-align: justify;}
may not be sufficient, you may need, for example,
body {text-align: justify;}
td {text-align: justify;}
to reliably achieve the justification.

This article is an extract from the “Saffron Book” (Good Practice for Commercial Website Design) available for on-line purchase at e-books.