Hacking with Greg

Set the Hack Free #1 : Lazy Regression Testing for Perl.

I've got a couple of notebooks and many more loose papers with ideas for hacks that I never get around to implementing. This seems like a waste. So I've decided to start taking some of them to write up here, include any skeleton/stub code I've written and then basically purge myself (and my svn sandbox) of the burden of trying to hope that I'll get around to doing a decent implementation one day. This is the first such hack idea, so be free little hack idea, fly, fly like the wind!

I'm lazy (Larry says thats ok, although he also says that you should be dilligent – but lets ignore this) and because I'm so lazy I like the computer to do my work. Although I'd still like to get the salary/rewards - the computer would just blow it on more SLI graphics cards (bizarrely Nvidia now sell clothes) and those fancy neon case mods.

Anyway, I was thinking about regression testing one day, especially the problem of using Perl in big organisations. Imagine a programmer, lets call him Dave, now Dave has just knocked up a great new program that needs to go into production, it uses use one module – Acme::Shiny. The good news is that Acme::Shiny has already been installed on the production servers and LOTS of programs use it, the bad news is that Dave being uber-keen needs the latest shiniest version of Acme::Shiny to use the new method do_super_shiny_thing(). The problem is how do you know that the author of Acme::Shiny hasn't decided to break the interface for all the other programs that already use it.

And did i mention that Dave is the CEO's nephew and for entirely unrelated reasons the CEO is really keen to see this program go live.

Now you could go and look at the interface by hand (and at some point you should), but this becomes difficult especially if its not just Acme::Shiny that you are worried about – imagine that it also uses the bleeding edge versions of Catalyst and Plagger and their one or two dependencies.

So what is my hair brained solution for this? Whenever you install new software you keep its test suite, so somewhere you have the tarball for your installed Acme::Shiny (or you could grab it off CPAN). When it comes to the new version you run the tests as normal, but then you delete them and copy the tests from the old and installed module. If you are lucky when you run these old tests they check the old/expected interface reasonably well – and any change of interface will be spotted by the previous tests failing.

Of course this is far from fool proof, but like most tools its simply there to help you do your job a little better.

Anyway hopefully this idea is more useful here rather than in my notebook.

Popular culture meets code

It's always a bad sign when I have to start by explaining why I did something.

In this case I was trying to take a break from revision and due to caffeine levels I couldn't manage to just zone out in front of the TV.

So I caught up on some mailing lists and I saw various people looking up interesting geek phrases in Google Codesearch, such as "this is a security hole"  and "here be dragons".

While interesting, it wasn't shocking that code contained technical/geek phrases in it - what would be more suprising is if code contained "up to the minute" phrases inside it, such as the phrases in the Google Zeitgeist.

So as clkao had been boasting that he implemented Shoutwithus in a weekend (he really hadn't been boasting, but it makes for a better narrative) so I decided to implement a Google Zeitgeist to Codesearch page in a lunchtime - it actually took a little longer than an hour but as my boss knows I have a slightly elastic definition when it comes to lunch (only kidding Simon) ;-).

So I give you, Google Zeitgeist -> Google Codesearch.

Unfortunately it's quite boring, but one of these days Linux distro's will realise the value of metasyntactic variables based around Paris Hilton, and then people will be clambering for this page.

Tracking Health With Bash

So while I was in hospital they talked about "markers" as a way of monitoring how I was doing, these included my temperature and various data obtained from blood tests. Not that the consultant ever paid attention, but thats another story that ends up with the phrase "semi-self-discharge", but anyhoo.

Before I got ill, I was playing with a sad geek idea I've had before but could never be bothered with the yak shaving involved, enter Zimki. The idea was a general purpose statistics dumping area, that I could use from any computer with an internet connection to record any stat i could possibly dream up and now thanks to Zimki someone else could  worry about things like database maintenance etc.

To start testing it back then, I added a curl command to my home and work bash startup files, that would increment a daily counter every time I fired up a shell.

This morning I decided to look at my shell usage, as a way to chart my recovery. So some R, Perl, Zimki and misc other UNIX goodies later I give you my health/productivity as indicated by bash shell script startups (yes, i am that sad).

Code Archaeology

I got an email today pointing out that one of my CPAN modules didn't have licensing information in it, for once I had a chance due to some good insomnia (my head was buzzing with my latest idea) to take care of this simple request.

However boy was the code scary - totally different layout of files to the one i'd use now, no change log, wrong email address, no warnings or strict. It wasn't even under version control (this last one led to a 20 minute yak shaving expedition as I took care of some other "quick" version control maintenance jobs that had been building up). I shouldn't be surprised I wrote the module 7 years ago (while playing with Perl and OpenGL) and apart from a trivial update 4 years ago I haven't done much with it since.

Still its oddly satisfying improving something that old even if it was just a very minor improvement.

YAZR

(Yet Another Zimki Release).

Well another month/fortnight brings another Zimki release - this one has added a lot of the missing/fixed a lot of the buggy functionality that was stopping people develop applications easily. It has also been the first release Katrien (our UX guru) has started to mould and already a lot of the rough edges are dropping off.

So please have another go with it here and let me or support@zimki.com know whats good or whats bad.

Two Useful Perl Modules

I found two useful Perl modules today when I had to process every text file in a complex and deep directory tree.

First of all there was File::Slurp::Tree  by Richard Clamp. This module can take a path and return a datastructure where directories are hash references and the file contents are scalars. It can also do the inverse. Richard originally designed it for tests, not large directories due to the inherent memory usage.

The second was Data::Rmap this does a recursive map through a datastructure and works brilliantly with File::Slurp::Tree.

Putting them together, you can do something like the following ...

    print map { "$_\n" } grep { defined } rmap_scalar { /..(some regex)../; $i } slurp_tree { $path };

which will print every occurence of some regex below the given path.

So basically after a little testing I had my job done in one line, very cool.

ETech 06 Rocks (Already)

So I'm sitting on the 40th floor of the hotel looking at the great view; I'm a sucker for tall buildings.

And I must say before the conference has really started in earnest It's already been good. I've had new ideas about home automation, encouragement to hack on VOIP projects and put an old face to a name I've heard around the office recently.

I've also had some good chilli and a beer in a bar they filmed a scene from Top Gun in.

Bluetooth Advertising

Just before Christmas I experienced a Bluetooth Advert in the wild. Apollo Cinemas sent me the following animated gif by BT.  They must have quite a range as I was on the opposite side of the street in a minicab. I wonder how long it will be until this sort of thing gets really annoying.

Great Customer Service

So I've had 2 great customer service experiences in the past 2 days.

Yesterday the colo box that runs my email and website 5000 miles away in Charlotte, NC (I'm lucky enough to have a good friend and BOFH nearby who helps with hands on support) was being prodded at by some skr1pt k1dd13 so I decided it was probably a good time to upgrade all the packages. Unfortunately the upgrade and Xen didn't agree (for those of you horrified by the fact I run Xen and do upgrades 5000 miles away on a hobby box - it used to be worse, i used to run Gentoo on it ;-) ). So kaboom went the box and my mail.

Anyway I had a Bytemark VM lying around doing nothing so I decided to move email to it. So I contacted Bytemark with a view to cajoling them into replacing the filesystem image so I'd have a clean start - a) they were incredibly clueful and b) they pointed me at the admin interface - you can actually replace your file system whenever you want from it! Within about 1 hour I had a clean new box with Exim running and I just had to wait for DNS to catch up. I cannot recommend Bytemark highly enough after this and other experiences with them.

Today, I had to reset my ADSL router, and when I changed my password recently I had been diligent enough to test it first before saving and yet not diligent enough to actually save it. Anyway I didn't realise this and so called the customer support number - it was a revelation compared to BT. My ADSL supplier, Nildram, has some of the most clueful customer support people I've ever dealt with, we had figured out the problem in about 5 minutes, after about 30 seconds in queue and not having to answer noddy questions about whether or not my PC was plugged in.

So, it might not be that interesting a story but if you come across this googling for either Nildram or Bytemark - please take this as a recommendation for using either company.

delicioustag2opml

I mentioned the problem I'm currently trying to solve only briefly and not completely last night. Basically I have two copies of newsfire and I want to "sync" the blogs they are watching. But I also don't want to just have my RSS feed "bookmarks" in them and them alone as I've moved my lions share of html bookmarks up to del.icio.us recently from my browsers. So I needed (wanted) a way to be able to load newsfire from del.icio.us - hopefully this still makes sense.

So I decided to start tagging the html page of any blog I was interested in reading on an ongoing basis in del.icio.us as newsfireset. Anyway last night I started a script that I finished off today. Basically  it takes all the entries tagged with a given tag and checks if they are or have RSS feeds using Benjamin Trott's very handy Feed::Find . If they do then I put them in an OPML file which can be imported into Newsfire (although it would be even nicer if Newsfire could import a OPML data from a URL).

Anyway you can find the script here. I guess you could also decide that someday you were interested in say reading ruby blogs and hence look for all del.icio.us tags tagged with ruby and import them via OPML.