This is a single post from deeden.co.uk made during the period May 2002 to April 2009. During a previous grand redesign I decided to make a break with the past and consign the old entries to history. This entry is from January 2003 and lives here forevermore.

Reconstructing the site

Around 40 days ago I said that I was getting rid of the blog on this site. I fully intended to do that, for the reasons mentioned, but I’ve had a change of heart. I started out with my plan to remove the blog from the site and worked away for a couple of days to bring that about. During that phase I realised that I did want some way to mouth off about stuff if I felt like it, thus I had a rethink. I also took a step back and looked at the various things I didn’t like about the previous setup. Some of there were, in no particular order,

  1. The look of the site; I have only myself to blame!
  2. The time it took to make a change across multiple MT templates in the web interface.
  3. The fact that movable type updates pages which haven’t changed if I only choose to rebuild a particular set of pages.
  4. The short blog entries just to link to an interesting page.
  5. The problems of validating hundreds of pages, of different types.
  6. The horrible search url provided by Movable Type.

Having worked out what was bothering me I looked into fixing these problems. I put aside the look of the site as a consideration early on. That was something I could work on as I went along, and didn’t require a dedicated period of time devoted to it. Therefore I moved onto the issue of templates and the editting of MT ones all of the time.

I quickly made the decision that rather than letting MT build the site, it made more sense to build it with something I’m more used to, namely Template Toolkit. The main reason behind this was that it allowed me to move away from having to puzzle out how to get something to work in MT when I already knew how to do it in TT. This also makes it easier for me to move away from using MT in the future, if I don’t have to change the entire site whilst doing it. Moving to TT also allows me much more scope, as far as I can see, to use conditional options and other such enhancements.

Moving to TT doesn’t mean that I’ve done away with MT. I now use MT to generate some of the templates for TT to process. Thus MT generates various templates for various parts of the site, and then TT processes them and generates the real HTML and RSS files.

This is all very well but it still leaves me with the problem of MT regenerating templates which haven’t actually changed. This was reasonably simple to fix in the end. I simply told MT to write it’s templates to a certain location, and then had a small perl script compare those files to the templates that were used last time. If they haven’t changed then they’re not copied over, otherwise they are. Then, when ttree is run, only templates that have been changed are rebuilt. All of this results in only pages which have really been updated being updated.

The issue of the annoying small entries to just link to another page was solved by just creating another blog in MT. This blog is used purely for quick links (this no longer exists) to other pages. A small use of Brad Choate’s Key Values Plugin helps keep track of the various information associated with these entries.

Validating the site proved to be the most complex, and interesting issue, and it’s still not fully functional. Currently I have a script which when run validates all of the HTML and CSS it can find within a specified directory and it’s descendants. It also allows me to ignore certain subdirectories (such as The Auteurs section) which are by no means near validating. The script makes use of the WDG offline HTML validator as well as the W3C CSS validator. It will happily run and tell me how much of the site is valid and also what errors have been discovered, if any. Therefore if it comes back with a completely valid result I’m a happy man.

Rather than validating after putting the pages “live”, I’ve added another stage into the process of generating the site. Before the pages are generated for the real site, I generate them to a different directory and validate that directory. If that directory comes back as completely valid, then I generate the real site and put it live.

The one flaw in the validation system at the moment is with RSS. Currently I validate the RSS by connecting to the RSS Validator and entering the url by hand. There is an experimental SOAP interface but my limited knowledge of SOAP is proving to be an impediment to getting it to work properly. Work is continuing apace.

The final part of fixing the site to work the way I want was to change the search url. I don’t particularly like having mt-search.cgi as the name of the search script, I much prefer using /search, so I changed it. I copied the script to /search and munged the data that MT::App::Search has stored. Basically it’s just a matter of telling it which blogs to look at, as well as specifying the search template for MT. The one thing I couldn’t change easily was the use of search as the parameter name, I’d prefer to use q but what can you do? Well I could write my own module to override the behaviour I don’t like, and I might well do that, but that’s for some later burst of productivity.

After all of that effort I now have a multiple stage process for generating the site. It’s runs like this:

  1. Generate the blog templates with Movable Type.
  2. Run the perl script to copy the updated templates.
  3. Build the “validation” site.
  4. Validate the staging site. Return to the earlier stages and fix if something is wrong.
  5. Validate the RSS files manually.
  6. Build the real site.
  7. Relax.

So there you go, the site is alive again. There are still bits I’m not happy with, and other bits I’m just not sure about. Things will be tweaked and moved around as I change my mind on stuff, but at least now it’s much easier to do.