The passing of Steve Jobs reminds me of my first serious project, and by serious I mean that it was complicated and I got paid!

Back in the early eighties I was talking to the design department of a light engineering company that made precision gears and gearboxes. I was just out of high school and pretty cocky.  They showed me their method for visualizing gear tooth and gear tooth space, which was a collection of plastic templates and shapes.  You used these shapes and a ball-point pen to scribble, in the same way that you would use a Spirograph, round and round, rocking the shape around the curved template until the clean space in the middle showed you the shape of the gear teeth.

They explained to me that a gear tooth is composed of about 8 curved surfaces, some of them simple, some of them complex (involute).  I was pretty good at maths and geometry, and had been setting my self programming problems on various micros such as the TRS-80, Video Genie, PET and Sinclairs.  The design department had recently bought an Apple II and in my arrogant enthusiasm I said that if they could describe the geometry, I could write something that would draw these shapes and allow them to be printed out.  The only other option at that time was to use a very expensive early CAD system made by Racal.

It turns out that although there are many curved surfaces on a simple gear, there are only about 4 parameters used to cut them.  The gear inner and outer radius, the number of teeth, and adjustment to control the under, or over-cutting of the tooth space.  Of course, there are many complex gear shapes, but for this project we were only considering a simple gear.  No worms, or helical, or anything exotic.  Armed with an A0-size piece of paper and a drafting table I set about mapping out the various curves and how they all connected when the cutting parameters were defined.  This took much longer than I anticipated, but eventually I convinced myself that I’d got it down.

I started coding it all in Applesoft Basic, which at the time had some advantages over its Microsoft core, including some advanced trig functions and floating point arithmetic.  After a while I was generating screen images of gear teeth, the problems started when I came to figure out how to print the screen.  I remember acquiring some code from a magazine to do screen dumps, but I was already using too much memory and had reached the limits of the massive 96KB of memory.  The solution was to swap a large chunk of the program into the video memory whenever I initiated the print, and then swap it back out once printing was complete, which meant that I had to turn the screen to blank once print was started.

It all worked.  It was slow, taking about 5 or 6 minutes to plot, and was difficult to validate the output since the only other methods were to use the Spirograph or actually cut the gear.  They used it for a few years until they finally invested in a CAD/CAM system, which could also integrate with the computer-controlled lathes on the shop floor.

A week or so ago, I came across a Google “code playground” website (http://code.google.com/apis/ajax/playground/#retrieve_events), which allowed you to interactively “play” with AJAX code that accessed the Google APIs.  There were all kinds of things in there.  You could access maps, blogs, calendars, images, translations services, and so on, and so forth.  It was a lot of fun, and provided a really nice way to quickly investigate the available services and features.

I’m also helping out a musician with a website to promote his services as a soundtrack composer, and one of the things I wanted to have on his site was a calendar showing the details of his upcoming performances.  Now this is work as a favor, and most of the site is going to be pretty static, so I didn’t really want to go to the trouble of making it PHP or Ruby on Rails, or to use a web content management system like WordPress.  But I did want him to be able to update the calendar, and here was a nice, simple method, which would allow him to use a public Google calendar, which in turn would be accessed by some Javascript (using AJAX) to display a nicely formatted list of performance events.

Deconstructing the sample code, there were three main areas of interest.

1. A URL which provided access to the public calendar
2. Interrogation of the resultset, and formatting for display
3. Some setting of parameters for the query to the calendar service

For the purposes of testing, I created a dummy public calendar with four entries.  The  sample code listed the URL as:

var feedUri = 'http://www.google.com/calendar/feeds/developer-calendar%40google.com/public/full';

So I thought it would just be a matter of swapping in my Google ID for the string developer-calendar%40google.com.  Nope, that gave me an error indicating that the URL was wrong.  I poked around in the calendar settings for my Google account, and found that I could get the calendar address as XML, ICAL, or HTML.  In this case the calendar ID was the easy to remember! qjufu69ca69kgimjd6s7ujjjdg@group.calendar.google.com.  So I swapped that in, and bingo! No events were displayed!  I took another look and noticed that the parameters in the particular example used a date range for the query.  I altered the end date to some time in the far future and now I was displaying my dummy calendar events.
The sample code didn’t do anything more than display the “title” of the calendar event.  My application was going to need more than that.  I wanted to get everything from the date & time to the location, and notes.  For this I was going to need to look at the javadocs, or more accurately jsdocs.  There was a link to the API Developer’s Guide from the Code Playground page, but all the examples seemed to deal with adding parameters to a GET statement, and when I tried doing this, by appending the parameters to the URL, it just caused errors.  I wanted the list of methods associated with the class CalendarEventEntry, which were being returned in an array by the calendar query.  I can’t tell you exactly how I found it (probably by doing a Google search), but I found it in the end at http://code.google.com/apis/gdata/jsdoc/2.2/index.html. Selecting the “google.gdata.calendar” package, and then the “google.gdata.calendar.CalendarEventEntry” class revealed the list of available methods.  Now, my calendar entries had data in the “Description” field, but there was no getDescription() method.  There wasn’t a getDescription() method that was inherited from any of its superclasses either.  I looked up and down the list, there was a getContent() method inherited from the root atom.entry class.  I plugged that in, added the additional method getText(), since the call returned an atom.Text object, and out popped the descriptions from my calendar entry.  Great.

Now I wanted to get the date & time for each event.  Going back to the jsdocs, and realizing that I was looking for something associated with start and end times, I found the method getTimes(), this looked promising.  Unfortunately, I went round and round in circles trying to get the start time from the gdata.When object, but to no avail.  Finally, I found an example by searching, which revealed that I should be interrogating the zeroth element of the array like this event.getTimes()[0].getStartTime().  Why?  I don’t know, but it worked, and now I was able to build a pretty good display string showing the date, title, description, and start time:

html += event.getTimes()[0].getStartTime().getDate().toLocaleDateString() + " - " + event.getTitle().getText() + ' - ' + event.getContent().getText() + ' - ' + event.getTimes()[0].getStartTime().getDate().toLocaleTimeString();

I now noticed that the events weren’t being returned in chronological order.  I didn’t want to have to sort them myself, there must be a way of sorting them in the query.  Going back to the jsdocs for the CalendarEventQuery object, I saw that there were methods for setOrderBy and setSortOrder, but my inexperience, or lack of frequent practice using javadocs/jsdocs meant that I couldn’t see the one thing that was staring me in the face.  Time to phone a friend.  My friend explained to me that the section titled “Field Summary” held the equivalent of enum or constant declarations, and that this was where the secret to supplying the correct parameters to the setOrderBy and setSortOrder methods.  These static values were ORDERBY_START_TIME and SORTORDER_ASCENDING, so by popping these, using the proper syntax, I had:

query.setMinimumStartTime(startMin);
query.setMaximumStartTime(startMax);
query.setOrderBy(google.gdata.calendar.CalendarEventQuery.ORDERBY_START_TIME);
query.setSortOrder(google.gdata.calendar.CalendarEventQuery.SORTORDER_ASCENDING);

This set the date range, and the sort order.

So now I had everything in place to implement the AJAX Google calendar on a fairly static website.

Time to get some automation in place.  Rather than have to enter the draw twice a week, it would be so much better to have the application grab the draw the morning after it’s made.  This is interesting stuff.  My application is going to access another server, grab the HTML, parse it and store the results in my database.  This kind of scripting has many uses, automating and creating your own data feeds is something I’ve done in the past.  I purchased a subscription weather service, and extracted key values and stored them in a database for use in a DSL, line-quality metrics application.

A little poking around in Google searches yielded a gem called “Hpricot”.  Hpricot looked pretty cool.  It allowed you to parse HTML and pull divs and spans based on HTML class and id.  Even better they had a website where you could try it all out interactively.  I quickly realized that this would suit my needs exactly.

The MegaMillions site looked like it had been put together by people who understood what they were doing.  I say this because a great deal of sites don’t.  This site had well-structured CSS and naming conventions.  Extracting the draw data was a simple as:

doc = Hpricot(open("http://megamillions.com/includes/numberData_home.asp"))
draw_dates = doc/"div.num_date"
draw_picks = doc/"div.num_num"
draw_mega = doc/"div.num_mb"

None of that old string parsing code, this method just returned exactly what I needed.  Brilliant!

I came up with a little date calculation algorithm to determine whether the application should go and get the draw.  This was based on how many days had passed since the last draw recorded in the database.  For this I needed to remember to add the “require ‘date'”.  And, just for fun I decided to add a group of nine buttons, which would allow you to review the past nine draws.

Once a process has been started, it should be possible to interrupt and terminate it before completion.

Ideally, any changes completed should be “rolled back” to the state that they were before the action was initiated.  It is recognized that this latter part can be an issue in some environments, which lack transaction-style processing.  Therefore, suitable warnings should be given when an irreversible action is about to be performed.

Every once in a while we’d win.  Now let’s not get carried away, it would only be a few dollars every once in while.  Occasionally, as much as $10.  We had decided that any “modest” winnings would be re-invested as extra tickets for the next draw.

So I needed to provide a mechanism for adding “bonus” picks.  I’d already made allowance for this in my model, by providing a boolean “perm” column, which indicated whether or not a row of picks was part of the permanent collection.  I already had the mechanism for input from the view, using the rails <% form tag %>, all I needed to do was extend that to provide a multi-line input field. Easily done.  And to add the code in the controller to iterate through each line and add the temporary picks to the database.  Again, easy to do with text parsing and splitting on a carriage return line-feed, before reloading the page, all from within the controller.  One small thing to look out for was that the “params” were passed as an array, therefore I need to access the zero index element in the action.

Not a huge increase in functionality, but satisfyingly easy to implement and deploy.  The complement to these new functionality, was add another button to erase any temporary picks from the database, which was literally a single call to destroy_all for any non “perm” rows.

Great!  Now how about the annoying task of having to enter the twice-weekly draw.  Time to research some URI and HTML parsing.

So now I have a working application. It’s pretty simple, an activerecord model stores the lottery picks, the user enters the current draw, and the resulting display shows the results, as well as any winning lines. I even added a helper to take logic out of the view to handle the formatting of the SPAN elements.  All the layout and formatting is handled through CSS.

Now it’s time to try and deploy it to my hosting service.  At this time I was still with Network Solutions, which had a nifty ZIP deployment procedure for uploading Rails apps.  I create a database in the MySQL admin.  I migrate the data, such that it is, with and SQL export/import. Upload the Rails ZIP, restart the server and…. Failure, 500 error.  I check the logs, they’re empty.  I try to check versions, can’t check the version because I don’t have shell access!  I find that the versions are not the same, so I go back and recreate the app with the version running on Network Solutions, including the gems.  Still getting the 500 error, and nothing in the logs!  I change permissions on the log folder, still nothing in the logs. I determine that the 500 page is the one from my application, so something is working, just not sure where to look.  I manhandle the environment.rb configuration to force everything into development, and suddenly I have log entries.  Aha!  It was running in production all along, and coupled with a permissions issue was the reason I had no logs.  Now I see that I never updated the database.yml config for production, which I had assumed would run in develpment until I implicitly switched it over.  Wrong!  So I update the database.yml file and after a couple of iterations while I determine the correct connection details to the database, I finally get my application running from Network Solutions, accessible over the internet.  “Hello World”, indeed!

All applications should produce a log file.

Even better, all applications should allow users to specify the level of events written to the log file. Log files should be truncated after a user-configurable time interval.

I’ve been a customer of Network Solutions for some years now. My needs were simple, I wanted some domain hosting, email, and a place to store mostly HTML / Javascript websites. More recently, I’ve been hosting some Ruby on Rails (RoR) apps, as I tinker with the technology. Up until about two weeks ago, there has never been a problem.

Network Solutions got hacked. They’ll tell you that it’s a WordPress issue, but it’s their issue, since it’s their script that sets up WordPress for you. To cut a long story short, Network Solutions seems to have a massive security breach of its file servers. A few months ago, many of their hosted sites were hacked. This wasn’t someone guessing a few hundred passwords. This was someone getting close to root access, and then running a few scripts to redirect some sites. As I can figure this has happened several times in the past six months, evidenced by Network Solutions resetting everyone’s passwords, FTP and otherwise.

Two weeks ago, email started going missing, sometimes delayed for eight hours or more. This was annoying. Then all my Rails apps started breaking, I couldn’t get the logger to switch to DEBUG in production, so it was hard to analyse. Then I did manage to see the error message, and could see that the app was not able to find folders or files reliably. I submitted several tickets. I got responses fourty-eight hours later (not the twenty-four, guaranteed!), that the issue had been resolved, which is odd since we’d barely begun a detailed exchange of information, and more to the point the issue was definitely NOT resolved. I figured out a sort-of solution, which was to “Reset all file permissions”, after doing this everything worked for a few hours before it all went wrong again. I submitted another ticket explaining my solution. I got another “Everything’s resolved” response. A day later, MY solution appeared on their front page, except it wasn’t a solution, since it only worked for a few hours at a time.

I’m no longer with Network Solutions.

Hey Network Solutions! Next time you go hacking through your file system, have the good grace to tell your customers what you’re doing. Stop responding to all tickets that the issue is resolved, and if the ticket is something to do with Rails, or another add-on, don’t give a stock response saying “we don’t offer support for these add-ons”, when the problem is ALL to do with you screwing with the file systems.