In praise of simplicity

In the early nineties, I knew and worked with several Apple Mac users. They would enthuse about how easy and uncomplicated Macs were.  At the time I was building convention exhibits which depended on esoteric input and output devices, the kind of hardware and interfaces that really didn’t mesh well with Macs.  I railed against the “closed” nature of Macs.  That you needed special tools to get in the thing.  That they used non-standard connectors, and that annoying jack-in-a-box ResEdit program that was essential to get the machine to do anything non-standard.  But being a student of good interface design, I recognized that there was a useful logic to dictating the “correct” format of dialog boxes.  It meant that a user’s experience would be consistent across different packages.  This was why non-computer-science users liked these machines, there was a constancy about their interaction.  It made sense.  To contrast with the legendary MS-DOS message “Abort, Retry, Ignore?”, really?  Did anyone ever get something useful, by selecting “Ignore”?  Did anyone ever really code to handle an “Ignore”?  There was a design rule that Mac dialog boxes should always have no more than two options.  Simple, right?  There were exceptions.  Kai Software had a product called Bryce.  Everyone raged about how great the interface was, how innovative, how exciting.  It wasn’t.  It had a hidden menu system, options only revealed themselves when you moused over a part of the screen.  Beautiful?  Maybe.  But ridiculously hard to use.  Some users seemed to revel in its obscure nature, but it failed on a usability level.  Using it felt like you were playing Myst, randomly moving the mouse around to see if it changed icon, or revealed an option.  Fun in a game, utter crap in a software interface.

You see, people like simple stuff.  The number of people who couldn’t program their VCR (video cassette recorder, for you younger readers) is legendary, the flashing “12:00” was always a dead giveaway at your grandparent’s house.  Someone came up with a great idea in TV guides, a number next to the show you wanted to record, which you entered on your remote, and Bingo! the show was programmed.  This was even improved with a bar code, and the remote updated to include a scanner, eliminating the embarrassment of entering the code for Care Bears, and ending up with Showtime After Midnight programming.  Simple is good, it encourages people to use products more, and more use means more money.

Google’s search, when it was first released onto the world was simple.  Just a text box.  They liked it.  Critics praised its simplicity.  Users loved its simplicity.  The page loaded in a flash, you typed in your search, and chances are the thing you were looking for was on the first page of returned results.  All that “complex” search language that you’d learned in AltaVista was unnecessary.  Google just worked.  So why is it all now messed up?  There are slabs of Javascript all over the main page (about 22KB).  You can actually start typing before it’s fully loaded.  As soon as you start typing, the page re-configures itself, the text box moves.  As you move the mouse away from the text, you roll over another “hot” area and half the page explodes with content giving you a preview, only to vanish immediately as you mouse out.  “What the hell was that?  Did you see that?  Did I just have a pop-up?”, you can see the less technology savvy grandparents backing away from the keyboard already.  Oh, I’m sure it’s clever, I’m sure there’s a reason, but listen, you just made it more complicated.  You have just alienated an albeit small group of users, but you made something that didn’t need to be any more than a text box more complicated.  In the ’90s, AltaVista got a “Webby” award for placing an additional search box at the bottom of the results page.  Something so simple and obvious, was truly appreciated as useful.  Google just took their textbox at the bottom of screen away.  Why?  How could something that was so well received when it was implemented that it got an award be deemed no longer necessary?  Was it to fit in with Google’s new look?  Was it a loss of functionality to fit a visual design shift?  Was it elevating form over function?

The advantages of simple interfaces have been demonstrated time and time again.  It might even be accurate to say that Apple’s current status is a direct result of this line of design.  Although, I’d be happy if someone could explain iTunes’ chaotic user interface to me!  Simple is good.  Simple works.  Simple is efficient.  Ask anyone who’s tried to pay their AT&T wireless phone bill from a weak wi-fi connection in Mexico, how much they enjoyed waiting for that Adobe Flash intro to download and play on the opening page, and I think you’ll begin to get the picture. 

You see, everyone’s getting in on the game.  Facebook just announced “Timeline”, a new interface to your page, and what’s this?  A customizable header?  Part of the popularity of Facebook was its simple, clean interface.  Everyone’s page looked pretty much the same.  Now we have what looks like the start of an attempt to make Facebook as customizable as, say MySpace, and that’s a great looking site, right?  I don’t know what the equivalent of the flashing “12:00” on a VCR is for a website, but I do know that less is more when it comes to interface design, and if you were famous for having less and start adding more, you’re probably making a mistake.

Apple Memories

The passing of Steve Jobs reminds me of my first serious project, and by serious I mean that it was complicated and I got paid!

Back in the early eighties I was talking to the design department of a light engineering company that made precision gears and gearboxes. I was just out of high school and pretty cocky.  They showed me their method for visualizing gear tooth and gear tooth space, which was a collection of plastic templates and shapes.  You used these shapes and a ball-point pen to scribble, in the same way that you would use a Spirograph, round and round, rocking the shape around the curved template until the clean space in the middle showed you the shape of the gear teeth.

They explained to me that a gear tooth is composed of about 8 curved surfaces, some of them simple, some of them complex (involute).  I was pretty good at maths and geometry, and had been setting my self programming problems on various micros such as the TRS-80, Video Genie, PET and Sinclairs.  The design department had recently bought an Apple II and in my arrogant enthusiasm I said that if they could describe the geometry, I could write something that would draw these shapes and allow them to be printed out.  The only other option at that time was to use a very expensive early CAD system made by Racal.

It turns out that although there are many curved surfaces on a simple gear, there are only about 4 parameters used to cut them.  The gear inner and outer radius, the number of teeth, and adjustment to control the under, or over-cutting of the tooth space.  Of course, there are many complex gear shapes, but for this project we were only considering a simple gear.  No worms, or helical, or anything exotic.  Armed with an A0-size piece of paper and a drafting table I set about mapping out the various curves and how they all connected when the cutting parameters were defined.  This took much longer than I anticipated, but eventually I convinced myself that I’d got it down.

I started coding it all in Applesoft Basic, which at the time had some advantages over its Microsoft core, including some advanced trig functions and floating point arithmetic.  After a while I was generating screen images of gear teeth, the problems started when I came to figure out how to print the screen.  I remember acquiring some code from a magazine to do screen dumps, but I was already using too much memory and had reached the limits of the massive 96KB of memory.  The solution was to swap a large chunk of the program into the video memory whenever I initiated the print, and then swap it back out once printing was complete, which meant that I had to turn the screen to blank once print was started.

It all worked.  It was slow, taking about 5 or 6 minutes to plot, and was difficult to validate the output since the only other methods were to use the Spirograph or actually cut the gear.  They used it for a few years until they finally invested in a CAD/CAM system, which could also integrate with the computer-controlled lathes on the shop floor.

Drinking from the font of the Google feeds

A week or so ago, I came across a Google “code playground” website (http://code.google.com/apis/ajax/playground/#retrieve_events), which allowed you to interactively “play” with AJAX code that accessed the Google APIs.  There were all kinds of things in there.  You could access maps, blogs, calendars, images, translations services, and so on, and so forth.  It was a lot of fun, and provided a really nice way to quickly investigate the available services and features.

I’m also helping out a musician with a website to promote his services as a soundtrack composer, and one of the things I wanted to have on his site was a calendar showing the details of his upcoming performances.  Now this is work as a favor, and most of the site is going to be pretty static, so I didn’t really want to go to the trouble of making it PHP or Ruby on Rails, or to use a web content management system like WordPress.  But I did want him to be able to update the calendar, and here was a nice, simple method, which would allow him to use a public Google calendar, which in turn would be accessed by some Javascript (using AJAX) to display a nicely formatted list of performance events.

Deconstructing the sample code, there were three main areas of interest.

  1. A URL which provided access to the public calendar
  2. Interrogation of the resultset, and formatting for display
  3. Some setting of parameters for the query to the calendar service

For the purposes of testing, I created a dummy public calendar with four entries.  The  sample code listed the URL as:

var feedUri = 'http://www.google.com/calendar/feeds/developer-calendar%40google.com/public/full';

So I thought it would just be a matter of swapping in my Google ID for the string developer-calendar%40google.com.  Nope, that gave me an error indicating that the URL was wrong.  I poked around in the calendar settings for my Google account, and found that I could get the calendar address as XML, ICAL, or HTML.  In this case the calendar ID was the easy to remember! qjufu69ca69kgimjd6s7ujjjdg@group.calendar.google.com.  So I swapped that in, and bingo! No events were displayed!  I took another look and noticed that the parameters in the particular example used a date range for the query.  I altered the end date to some time in the far future and now I was displaying my dummy calendar events.
The sample code didn’t do anything more than display the “title” of the calendar event.  My application was going to need more than that.  I wanted to get everything from the date & time to the location, and notes.  For this I was going to need to look at the javadocs, or more accurately jsdocs.  There was a link to the API Developer’s Guide from the Code Playground page, but all the examples seemed to deal with adding parameters to a GET statement, and when I tried doing this, by appending the parameters to the URL, it just caused errors.  I wanted the list of methods associated with the class CalendarEventEntry, which were being returned in an array by the calendar query.  I can’t tell you exactly how I found it (probably by doing a Google search), but I found it in the end at http://code.google.com/apis/gdata/jsdoc/2.2/index.html. Selecting the “google.gdata.calendar” package, and then the “google.gdata.calendar.CalendarEventEntry” class revealed the list of available methods.  Now, my calendar entries had data in the “Description” field, but there was no getDescription() method.  There wasn’t a getDescription() method that was inherited from any of its superclasses either.  I looked up and down the list, there was a getContent() method inherited from the root atom.entry class.  I plugged that in, added the additional method getText(), since the call returned an atom.Text object, and out popped the descriptions from my calendar entry.  Great.

Now I wanted to get the date & time for each event.  Going back to the jsdocs, and realizing that I was looking for something associated with start and end times, I found the method getTimes(), this looked promising.  Unfortunately, I went round and round in circles trying to get the start time from the gdata.When object, but to no avail.  Finally, I found an example by searching, which revealed that I should be interrogating the zeroth element of the array like this event.getTimes()[0].getStartTime().  Why?  I don’t know, but it worked, and now I was able to build a pretty good display string showing the date, title, description, and start time:

html += event.getTimes()[0].getStartTime().getDate().toLocaleDateString() + " - " + event.getTitle().getText() + ' - ' + event.getContent().getText() + ' - ' + event.getTimes()[0].getStartTime().getDate().toLocaleTimeString();

I now noticed that the events weren’t being returned in chronological order.  I didn’t want to have to sort them myself, there must be a way of sorting them in the query.  Going back to the jsdocs for the CalendarEventQuery object, I saw that there were methods for setOrderBy and setSortOrder, but my inexperience, or lack of frequent practice using javadocs/jsdocs meant that I couldn’t see the one thing that was staring me in the face.  Time to phone a friend.  My friend explained to me that the section titled “Field Summary” held the equivalent of enum or constant declarations, and that this was where the secret to supplying the correct parameters to the setOrderBy and setSortOrder methods.  These static values were ORDERBY_START_TIME and SORTORDER_ASCENDING, so by popping these, using the proper syntax, I had:

query.setMinimumStartTime(startMin);
query.setMaximumStartTime(startMax);
query.setOrderBy(google.gdata.calendar.CalendarEventQuery.ORDERBY_START_TIME);
query.setSortOrder(google.gdata.calendar.CalendarEventQuery.SORTORDER_ASCENDING);

This set the date range, and the sort order.

So now I had everything in place to implement the AJAX Google calendar on a fairly static website.

Data Scraping

Time to get some automation in place.  Rather than have to enter the draw twice a week, it would be so much better to have the application grab the draw the morning after it’s made.  This is interesting stuff.  My application is going to access another server, grab the HTML, parse it and store the results in my database.  This kind of scripting has many uses, automating and creating your own data feeds is something I’ve done in the past.  I purchased a subscription weather service, and extracted key values and stored them in a database for use in a DSL, line-quality metrics application.

A little poking around in Google searches yielded a gem called “Hpricot”.  Hpricot looked pretty cool.  It allowed you to parse HTML and pull divs and spans based on HTML class and id.  Even better they had a website where you could try it all out interactively.  I quickly realized that this would suit my needs exactly.

The MegaMillions site looked like it had been put together by people who understood what they were doing.  I say this because a great deal of sites don’t.  This site had well-structured CSS and naming conventions.  Extracting the draw data was a simple as:

doc = Hpricot(open("http://megamillions.com/includes/numberData_home.asp"))
draw_dates = doc/"div.num_date"
draw_picks = doc/"div.num_num"
draw_mega = doc/"div.num_mb"

None of that old string parsing code, this method just returned exactly what I needed.  Brilliant!

I came up with a little date calculation algorithm to determine whether the application should go and get the draw.  This was based on how many days had passed since the last draw recorded in the database.  For this I needed to remember to add the “require ‘date'”.  And, just for fun I decided to add a group of nine buttons, which would allow you to review the past nine draws.

Random Rules of Programming – #2 in an occasional series

Once a process has been started, it should be possible to interrupt and terminate it before completion.

Ideally, any changes completed should be “rolled back” to the state that they were before the action was initiated.  It is recognized that this latter part can be an issue in some environments, which lack transaction-style processing.  Therefore, suitable warnings should be given when an irreversible action is about to be performed.

Bonus Picks!

Every once in a while we’d win.  Now let’s not get carried away, it would only be a few dollars every once in while.  Occasionally, as much as $10.  We had decided that any “modest” winnings would be re-invested as extra tickets for the next draw.

So I needed to provide a mechanism for adding “bonus” picks.  I’d already made allowance for this in my model, by providing a boolean “perm” column, which indicated whether or not a row of picks was part of the permanent collection.  I already had the mechanism for input from the view, using the rails <% form tag %>, all I needed to do was extend that to provide a multi-line input field. Easily done.  And to add the code in the controller to iterate through each line and add the temporary picks to the database.  Again, easy to do with text parsing and splitting on a carriage return line-feed, before reloading the page, all from within the controller.  One small thing to look out for was that the “params” were passed as an array, therefore I need to access the zero index element in the action.

Not a huge increase in functionality, but satisfyingly easy to implement and deploy.  The complement to these new functionality, was add another button to erase any temporary picks from the database, which was literally a single call to destroy_all for any non “perm” rows.

Great!  Now how about the annoying task of having to enter the twice-weekly draw.  Time to research some URI and HTML parsing.

On the subject of soda cans

I bought a can of Coke today, which is fairly rare for me.  On the way home I noticed that the top looked a little bit different than usual, and I was reminded of conversations I had in the early Nineties, when the shape of soda cans noticeably changed last.

The change that happened in the Nineties was that a crease appeared near the top of the can.  At work where industrial design was a common topic of conversation, various theories were suggested for this.  One of which was that it was to allow the contents to expand if the can should be stored below freezing.  It seemed vaguely plausible and I didn’t give it much thought until some years later, when I decided to find out once and for all what the real reason for the change was.  After some lengthy and devious boolean searches on AltaVista, I came across a research/presentation paper by the company that owned the patent on the new can design.  I turned out that the freezing theory was very much wrong.  The real reason for the switch was to reduce the weight of the can, thereby saving money on raw materials and shipping.  The new design eliminated the solid base of the can, as well as reducing the amount of aluminum required for the top, without compromising the rigidity of the container.  The savings were actually quite impressive.

I suppose that someone has improved on the design once more.  The top looks even smaller in diameter, and has a deeper groove.

The continual optimization of design always impresses me, especially when it shows up in something that most of us would consider mundane, such as a soda can.  But it’s going on all around us, all the time.  Not because of a philanthropic drive to improve our consumption of natural resources, but to maximize profit and increase efficiency of effort.  Even still, it’s nice to know that somewhere, someone is thinking, or being to told to think about ways to improve something.

Deploy, Deploy, Deploy!

So now I have a working application. It’s pretty simple, an activerecord model stores the lottery picks, the user enters the current draw, and the resulting display shows the results, as well as any winning lines. I even added a helper to take logic out of the view to handle the formatting of the SPAN elements.  All the layout and formatting is handled through CSS.

Now it’s time to try and deploy it to my hosting service.  At this time I was still with Network Solutions, which had a nifty ZIP deployment procedure for uploading Rails apps.  I create a database in the MySQL admin.  I migrate the data, such that it is, with and SQL export/import. Upload the Rails ZIP, restart the server and…. Failure, 500 error.  I check the logs, they’re empty.  I try to check versions, can’t check the version because I don’t have shell access!  I find that the versions are not the same, so I go back and recreate the app with the version running on Network Solutions, including the gems.  Still getting the 500 error, and nothing in the logs!  I change permissions on the log folder, still nothing in the logs. I determine that the 500 page is the one from my application, so something is working, just not sure where to look.  I manhandle the environment.rb configuration to force everything into development, and suddenly I have log entries.  Aha!  It was running in production all along, and coupled with a permissions issue was the reason I had no logs.  Now I see that I never updated the database.yml config for production, which I had assumed would run in develpment until I implicitly switched it over.  Wrong!  So I update the database.yml file and after a couple of iterations while I determine the correct connection details to the database, I finally get my application running from Network Solutions, accessible over the internet.  “Hello World”, indeed!

Intermission: I hate Network Solutions

I’ve been a customer of Network Solutions for some years now. My needs were simple, I wanted some domain hosting, email, and a place to store mostly HTML / Javascript websites. More recently, I’ve been hosting some Ruby on Rails (RoR) apps, as I tinker with the technology. Up until about two weeks ago, there has never been a problem.

Network Solutions got hacked. They’ll tell you that it’s a WordPress issue, but it’s their issue, since it’s their script that sets up WordPress for you. To cut a long story short, Network Solutions seems to have a massive security breach of its file servers. A few months ago, many of their hosted sites were hacked. This wasn’t someone guessing a few hundred passwords. This was someone getting close to root access, and then running a few scripts to redirect some sites. As I can figure this has happened several times in the past six months, evidenced by Network Solutions resetting everyone’s passwords, FTP and otherwise.

Two weeks ago, email started going missing, sometimes delayed for eight hours or more. This was annoying. Then all my Rails apps started breaking, I couldn’t get the logger to switch to DEBUG in production, so it was hard to analyse. Then I did manage to see the error message, and could see that the app was not able to find folders or files reliably. I submitted several tickets. I got responses fourty-eight hours later (not the twenty-four, guaranteed!), that the issue had been resolved, which is odd since we’d barely begun a detailed exchange of information, and more to the point the issue was definitely NOT resolved. I figured out a sort-of solution, which was to “Reset all file permissions”, after doing this everything worked for a few hours before it all went wrong again. I submitted another ticket explaining my solution. I got another “Everything’s resolved” response. A day later, MY solution appeared on their front page, except it wasn’t a solution, since it only worked for a few hours at a time.

I’m no longer with Network Solutions.

Hey Network Solutions! Next time you go hacking through your file system, have the good grace to tell your customers what you’re doing. Stop responding to all tickets that the issue is resolved, and if the ticket is something to do with Rails, or another add-on, don’t give a stock response saying “we don’t offer support for these add-ons”, when the problem is ALL to do with you screwing with the file systems.