Recently, I had need to create a plugin that held a polymorphic :has_many, :through relationship. I spent a day or so trying to get this to work, only to realize that it is essentially impossible to do inside a pluggable Rails module, but possible if you templatize the classes.

PolymorphicPlugin Example Project

There’s a StackOverflow thread that describes this problem and the solution a bit; however, Based on the difficulty, and the fact that I couldn’t find any documentation specific to this problem, I figured I’d write it up as an example, and create a PolymorphicPlugin gem to illustrate it better.

The PolymorphicPlugin inside-module branch is an example of the way you’d distribute a pluggable Rails module with all classes contained in the module. The generator only creates a migration in the Rails project.

The PolymorphicPlugin outside-module branch is an example of the way you’d distribute a pluggable Rails module in such a way that the generator creates a migration in and also copies model files to the Rails project.

Problem Description

The idea is that there’s an “acts_as” like plugin that I want to add to my Rails models:

module PolymorphicPlugin
  module PolymorphicHolder

    def self.included(base)
      base.extend ClassMethods
    end

    module ClassMethods
      def holds_things
        has_many :thing_groupings, :as => :thingable
        has_many :things, :through => :thing_groupings
        include PolymorphicPlugin::PolymorphicHolder::InstanceMethods
      end
    end

    module InstanceMethods

      def has_things?
        true
      end

    end
  end
end

You can see that anything that calls holds_things should have a :has_many relationship with Thing through the ThingGrouping join table. This is a standard paradigm for a polymorphic :has_many, :through relationship.

Basically, I want to do this:

class User < ActiveRecord::Base
  holds_things
end

class Admin < ActiveRecord::Base
  holds_things
end

user = User.new
user.has_things? -> true

admin = Admin.new
admin.has_things? -> true

inside-module plugin layout

The failing case

Seems pretty simple, right?

Turns out it’s not. Take a look at the directory layout that I tried to use. This seems like a standard for a gemmable plugin. The model files are contained in the gem library, and accessed as part of the module. When you use gem 'polymorphic_plugin', the only thing that gets copied to your Rails project is the migration file. Nice and clean.

Unfortunately, that doesn’t work. If you clone this branch and run rake test you’ll get exceptions. Specifically, a NameError: uninitialized constant User::ThingGrouping because our User model, by extending the PolymorphicHolder module, doesn’t have access to this unknown model called ThingGrouping.

And no, using things like :class_name and :source were no help. I tried everything, or seemingly everything. Nothing would make this work.

outside-module plugin layout

The passing case

That is until I pulled the models outside of the plugin module.

Take a look at the directory layout now and you’ll notice that the Thing and ThingGrouping models are no longer in the plugin’s lib directory. Moving them to the templates directory is the unfortunate answer.

I say “unfortunate” because this means that the models need to be distributed by copying them to the Rails project. In other words, there’s a line in my PolymorphicPluginGenerator that reads: template "thing.rb", File.join('app/models', "thing.rb"). In other words: Copy the Thing model into this project so it can be accidentally futzed with by users and maybe, if we’re lucky, cause some interesting name collisions.

Thing and ThingGroup are no longer contained inside my PolymorphicPlugin module. They are top-level classes equivalent to anything generated with rails generate model. Not a solution that I’m fond of, but one that passes my tests. We can at least have a polymorphic :has_many, :through relationship now.

Outcome

Polymorphic :has_many, :through relationships are available in pluggable Rails modules, but it’s a bit of a kludge. You’ll need to make sure you name your polymorphic classes smartly to avoid name collisions with the calling Rails project. You’d hate to create a plugin with a model like Book only to find that Book already exists in someone’s Rails project!

I’d love to be proven wrong on this, by the way. Go ahead and grab the Github repository and play with the inside-module branch. If you can make it work, I’ll buy you a beer!

Currently, the new Rails 3.1 mountable plugin architecture will not work with RSpec. There’s a gem called Enginex that will work with RSpec, but it won’t work with Rails 3.1. So it looks like you either get RSpec or Rails 3.1. What’s the deal?

Playing around, I figured out that combining the two will work:

$ gem install enginex
…
$ enginex my_plugin --test-framework=rspec
…
$ rails -v
Rails 3.1.1
$ rails plugin new stock_plugin --full
…
$ rm -rf my_plugin/spec/dummy
$ cp -r stock_plugin/test/dummy my_plugin/spec

After that, it’s just a matter of modifying my_plugin/spec/dummy/config/application.rb to require your gem, and clean up the spec_helper.rb file. I also changed the Gem files so that my_plugin.gemspec held the dependencies, and Gemfile just called that. This is the way the stock Rails 3.1 generator works. Enginex uses Gemfile as the canonical source, and I like the new way better.

Using this I’ve currently got a Rails 3.1 gemmable plugin with RSpec, Cucumber, Capybara, Spork, Guard, and Simplecov working just fine.

The basic problem that I’ve found is that ActiveResource is a great Rails model to inherit from, except when it isn’t. And it seems that “when it isn’t” basically means “anytime you are using associations.” To be fair, associations don’t always break, it just turns out that my architecture illustrates exactly when they do break.

To lay out the situation, I have a backend server running Rails with three models, User, Service, and Groups. I have a Rails frontend server without a database hitting this (as well as other) backend servers.

Backend

class User  :members
  has_many :managed_groups, :class_name => "Group"
  accepts_nested_attributes_for :managed_groups, :services, :groups
end

class Group  :members
  belongs_to :manager, :class_name => "User", :foreign_key => :user_id
  accepts_nested_attributes_for :users  
end

class Service < ActiveRecord::Base
  belongs_to :user
end

class Member < ActiveRecord::Base
  belongs_to :group
  belongs_to :user
end

You can see it’s really basic setup. Users can have many Services, but Service can only have one User. By contrast, users can Manage many groups, and can belong to many Groups, while Groups can have many users who belong to it.

Users -> Services is a basic :has_many relationship and Users -> Groups is a :has_many, :through relationship. There is nothing complicated about this setup at all…

Frontend

…until you get to an ActiveResource frontend

class User < ActiveResource::Base
  self.site = "http://localhost:8801"
end

class Group < ActiveResource::Base
  self.site = "http://localhost:8801"
end

class Service < ActiveResource::Base
  self.site = "http://localhost:8801"
end

The frontend server sees all of these models as ActiveResources, which works fine if you’re accessing User.name, but which doesn’t work so well if you’re trying to access User.services because User.services doesn’t exist. ActiveResource simply doesn’t know anything about associations, so you have to work around a big problem.

:include => :associated_model

One easy way to make User.services available to ActiveResource is by including it in the UsersController. Here’s a sample UsersController#Show method on the backend:

def show
@user = User.find(params[:id])
  respond_to do |format|
    format.xml  { render  ml => @user.to_xml(:include => [:managed_groups, :services, :groups]) }
    format.json  { render :json => @user.to_json(:include => [:managed_groups, :services, :groups]) }
  end
end

We just include a list of whatever associated objects we want to include in the returned user package. Easy enough. Starting up the Rails console on the frontend now allows you to access User.services. However, what if you tried the following:

service = Service.create(:name => "Test Service")
user = User.first
user.services << service

Turns out that we can’t do that. We get an exception that the User model expects a Service class, but was supplied with an IndifferentHash.

With a :has_many/:belongs_to relationship, this is easy to work around:

service = Service.create(:name => "Test Service", :user_id => User.first.id)

Of course, user_id is a “primitive” type, a direct attribute of Server. So we can force it by going to the other side of the relationship and set that attribute and not have a problem.

:has_many, :through => :needles_in_your_eye

Working around the :belongs_to issue was easy, working around :has_many, :through is as fun as… well, you get the idea.

The problem is that there’s no backdoor to setting the association. You simply can’t access an array of objects as an association through ActiveResource. Look at what we see when the frontend accesses the backend server with a PUT request at /users/1.xml:

{"user"=>
  {"created_at"=>2011-06-23 02:58:15 UTC,
   "id"=>1,
   "name"=>"John Metta",
   "updated_at"=>2011-06-23 02:58:15 UTC,
   "managed_groups"=>[],
   "services"=>[
     {"created_at"=>2011-06-23 02:58:15 UTC,
      "id"=>1,
      "updated_at"=>2011-06-23 02:58:15 UTC,
      "user_id"=>1}
   ],
   "groups"=>[
     {"created_at"=>2011-06-23 02:56:37 UTC,
      "id"=>1,
      "updated_at"=>2011-06-23 02:56:37 UTC,
      "user_id"=>nil,
      "users"=>[
        {"created_at"=>2011-06-23 03:36:28 UTC,
         "id"=>6,
         "name"=>"Jefferey",
         "updated_at"=>2011-06-23 03:36:28 UTC},
        {"created_at"=>2011-06-23 02:59:36 UTC,
         "id"=>2,
         "name"=>"George",
         "updated_at"=>2011-06-23 03:05:13 UTC}
       ]
     },
     {"created_at"=>2011-06-23 02:56:37 UTC,
      "id"=>1,
      "name"=>"Site Admin",
      "updated_at"=>2011-06-23 02:56:37 UTC,
      "user_id"=>nil,
      "users"=>[
        {"created_at"=>2011-06-23 03:36:28 UTC,
         "id"=>6,
         "name"=>"Jefferey",
         "updated_at"=>2011-06-23 03:36:28 UTC},
        {"created_at"=>2011-06-23 02:59:36 UTC,
         "id"=>2,
         "name"=>"George",
         "updated_at"=>2011-06-23 03:05:13 UTC},
        {"created_at"=>2011-06-23 02:58:15 UTC,
         "id"=>1,
         "name"=>"John Metta",
         "updated_at"=>2011-06-23 02:58:15 UTC}
       ]
     }
   ]
  },
  "id"=>"1"}

Cats talking to dogs, here. They’d both be happy eating a squirrel, sure, but they just speak different languages.

Basically, the backend tries to find the “groups” and “services” properties on the User model. ActiveRecord expects groups to be, well Group objects- but it’s a Hash, not a Group. In order for this to work, those should be named “groups_attributes” and “services_attributes.”

This is basically telling me that ActiveRecord and ActiveResource basically don’t know how to talk to one another. What the hell was the Rails core team thinking?

ActiveResource should be patched to format it’s JSON string to use MODEL_attributes, right? Well, that would break a lot of stuff. ActiveResource is supposed to talk fine to my Scala backend server too, which it wouldn’t.

ActiveRecord should automatically treat MODEL: {ASSOCIATED_MODEL: …}} as something that can be converted to an ActiveRecord, right? I’m not sure of this one- it seems too possible to break things down the line that we didn’t expect.

So, you can do it from either direction without possibly horribly breaking something down the line.

Ahh, apparently the Rails core team was thinking “You know, we shouldn’t do this because we might horribly break something down the line.”

Wrestling ActiveResource to the ground

In order to fix this, I’ve added a simple hack to my UsersController on the backend. I say it’s simple because it’s pretty dumb. I say it’s a hack because I think it’s really ugly and inelegant– something I don’t expect from Rails.

Here’s the full UsersController#Update method on my backend server:

  def update
    @user = User.find(params[:id])

    respond_to do |format|
      begin
        groups = []
        params[:user][:groups].each do |g|
          groups << Group.find(g[:id])
        end
        params[:user][:groups] = groups
      rescue
      end

      begin
        services = []
        params[:user][:services].each do |s|
          services < @user.errors, :status => :unprocessable_entity }
        format.json  { render :json => @user.errors, :status => :unprocessable_entity }
      end
    end
  end

As you can see, we grab that rogue IndifferentHash groups list and, basically, walk through it and find the local groups that correspond to the Group that’s in the list. This works- it allows ActiveResource to barely communicate with ActiveRecord. It’s no longer cats and dogs talking, it’s more like dogs and wolfs.

Yes, I know, wolves bite.

One Model, One Resource

This hack works, but it’s just that- A hack. If something seems this inelegant to me, then I have to question whether it is just that inelegant.

Today a friend mentioned that he’s never used accepts_nested_attributes_for. Never. He has never used nested resources like this because his strong philosophy is “One model, one resource.” I’m going to explore this for a few reasons, not the least of which is that I don’t think I want to pass the groups around inside the user if the user is a member of 3000 groups that we don’t care about. That’s just stupid.

Trying this method would mean that the User model on the frontend would require a groups method that would search the join table based on the user, and an additional add_group method which would go through the Group model to create a new group and somehow add that group to the users list of groups.

There’s a hole in my bucket, dear Liza, dear Liza, a hole in my bucket, dear Liza, a hole…

Really not sure how that would work– at least without some inclusion of my hack above. It seems like there’s just no way to get around this problem elegantly. I hope I’m wrong about that.

Someone please tell me I’m wrong.

The problem is that Ruby and Ruby-based tools like Chef are becoming so natural that they feel like they’re just part of the BASH shell. It took me a while to figure out the following error:

% knife cookbook create somename

/Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/knife/cookbook_create.rb:90:in `initialize': No such file or directory -
 ~/Dev/Chef/chef-repo/cookbooks/sti-rewrites/recipes/default.rb (Errno::ENOENT)
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/knife/cookbook_create.rb:90:in `open'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/knife/cookbook_create.rb:90:in `create_cookbook'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/knife/cookbook_create.rb:74:in `run'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/knife.rb:127:in `run'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/lib/chef/application/knife.rb:121:in `run'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/gems/chef-0.9.12/bin/knife:25
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/bin/knife:19:in `load'
	from /Users/john/.rvm/gems/ruby-1.8.7-p302/bin/knife:19

Because, no matter how much I confirmed that the directory existed, it still failed. Why?

Because Ruby, as anyone really thinking about it would naturally realize, doesn’t expand Bash shortcuts.

To wit: The following entry in my ~/.chef/knife.rb file:

...
cookbook_path [ '~/Dev/Chef/chef-repo/cookbooks', '~/Dev/Chef/chef-repo/cookbooks' ]
...

Which I generated by running a command at the bash shell:

knife configure

As anyone would expect, changing that line to:

...
cookbook_path [ '/Users/john/Dev/Chef/chef-repo/cookbooks', '/Users/john/Dev/Chef/chef-repo/cookbooks' ]
...

fixed the problem. D’oh!

As I use more Ruby-based server configuration tools, I think I need to be a bit more conscious of when I’m with the purview of the Bash shell, and when I’m not.

I’m a consultant, and that means I work on multiple projects, and have to bill those projects. Recently, I began working on a contract that required weekly billing, using an Excel spreadsheet that can’t be modified, and a signature from a project manager in another city. Talk about tedious! My workflow for just filling out a weekly timesheet is this:

1. Use Freshbooks daily to track time
2. On Friday, open Freshbooks and pull the hours on that project
3. Open Excel (in VMWare) to update the read-only Master Timesheet file by changing the date, and adding my hours into the day slots
   1. I can’t use Open Office or anything on the Mac since the formatting changes when I save it as an Excel file- I’ve tried a bunch of ways to do this, but as I usually have VMWare open to work in Visual Studio for this project anyway, I just use MS Excel on Windows
4. Save the timesheet under another name
5. Drag the timesheet into another folder in my Dropbox (so that I have a record)
6. Open my email client and create an email to my project manager that says little more than “Timesheet, Cheers, -J”
7. Drag the timesheet into the email (I Google Mail’s ability to drag a file to attach!) and hit send
8. When he sends it back, it’s an attachment as an image- save the image to my project folder locally
9. Print out the image
10. Sign it (in blue pen, mind you)
11. Scan the signed copy
12. Move the scanned copy file to my local computer
13. Open a web browser to connect to their email system, which I’m supposed to use, but which is proxied, so I can’t use my normal email client.
14. Create another email to the timesheet recipient
15. Attach the file using the dialog boxes (Drag is not possible) and send it
16. Grab the timesheet original off my scanner and put in an envelope to mail to New York.

Now, if you think about doing all of that, it’s not like it takes half the day. Still, it takes enough time that a good bit of my morning is lost to it. There’s not only the time spent “actively doing everything” but the lost time where I could have been concentrating on something like our object model or database abstraction layer, but which I’m distracted by simultaneously dealing with the timesheet issue. Anyone working in a “mental sphere” such as programming knows that a few seconds of distraction with something will pull you out of the mindset for 20 minutes or more.

Plus, there’s the added issue of this just being boring. It’s something a robot should be doing- or at least a secretary. More importantly, it’s something that could happen automatically! We programmers tend to get annoyed when things “should happen automatically” and they don’t. It is, in fact, why many of us became programmers.

Enter Hazel

For a while, I’ve been simplifying things a wee bit using Hazel. For those who don’t know, Hazel is a Mac program that watches a folder and performs actions based on the folder contents. I first heard about it on the Mac Power Users podcast and bought the software pretty much immediately following the podcast.

Hazel rules screen

Hazel is one amazing piece of time-saving software. It has a super simple graphical dialog box to let you set things like which folder, what attributes (“if files are older than” and “if file is a:” (e.g. movie, soundfile, pdf, etc) and “if the name contains:”) and what actions (Open, move, delete, archive, change name, import into iTunes, etc.) to perform on files in that folder.

One of the ways that I use it is on my desktop. If there are any pictures or movies on my desktop that are older than 1 day, Hazel takes them off my desktop and puts them on my remote hard drive, in appropriate folders, and simultaneously imports them into iTunes (I don’t have iTunes copy files to the local drive, so they’re watched from the remote drive).

When dealing with my timesheet, I have Hazel set up so that anything that goes into the “Timesheets” folder and is an Excel file gets automatically moved to a subfolder called “archive” and gets renamed with the pattern “John_Metta_<date added>.xls.” That way, I have an easily recognizable location and file to drag into an email to my project manager.

Using Hazel to automatically move, delete and organize your files can save you a lot of time. It’s tiny little chunks of time, sure, but it’s time. In reality, it’s saving you more the headache of having to do the same thing over and over again. Hazel is powerful, check it out.

Add A Little Ruby

Because I love Ruby, I decided to automate the creation of the Excel spreadsheet a bit. The Windows version of Ruby has the win32ole module, which can open and handle spreadsheets really easily. So I created a small script that opened a read-only version of the master timesheet spreadsheet and changed the date and the hours/day based on the arguments.

Here’s the whole script:

require 'win32ole'

xl = WIN32OLE.new('Excel.Application')
xls = xl.Workbooks.open("c:\timesheets\Timesheet")
sheet = xls.Worksheets(1)

sheet.Cells(4,11).Value = ARGV[0]

#Time cell index
t = 6
ARGV.each_with_index do |a, i|
  sheet.Cells(10,t+i).Value = a
end

xls.SaveAs("Z:\john\Dropbox\Consulting Stuff\Nova\timesheets\timesheet")
xls.Close
xl.Quit

With this, the command:

 C:timesheets$ ruby timesheet.rb "10/16" 0 0 8 7 6 7 8

will create the correct timesheet for the week beginning 10/16, with the correct dates all lined up for the hours, and will save that timesheet to my Dropbox/timesheets directory- afterwhich Hazel will do the rest.¹

More with Automator

Automator: More "secretary" than I'd thought!

Earlier this week, I was still frustrated at the workflow of timesheets, however, because I still had a bunch of steps. Hazel was good at moving files and renaming them, but I still had to email them, print them, etc. So I decided to get my programming game on… then I decided that I didn’t even need to. Like all Macs, my Mac has Automator- a program that will create “workflows” and “applications” which basically amount to my own personal robotic secretary. Hazel can run both applications and Automator workflows on files, so I decided to give it a try.

Mailing to my PM

The first thing I do when I have my timesheet is move it to the timesheets folder where it gets renamed and archived so that I can send it to my PM. It would only be a single step to mail the renamed file. So I opened my Automator script and used the “create mail message,” “add attachment to open messages,” and “send mail” actions to create a script. Now, whenever a folder goes into Timesheets, it gets moved, renamed, and automatically mailed to my PM in one step!²

Printing and Signing

When my PM signs and mails it back it’s an image. So, I created another automator script to take any images with the name pattern “John_Metta_*” and the extension JPG in my “Downloads” directory and automatically move them to the “timesheets/images” directory and send them to the printer. That means all I have to do is open the email and click “download” and I’ll soon have a printed timesheet awaiting my signature. Sounds pretty secretary-like to me.

Mailing the final

My Canon all-in-one will scan automatically save files to a specific directory, so I could just set up Hazel to monitor that directory. The problem with this is that ANYTHING I scan will get processed. Rather than that, I’ve decided to have Hazel monitor that directory for a specific file name. Thus, when I scan any document, I see a dialog box on my screen prompting me to add a name. I can add a name in the rare occasions I want to scan something else. If I just hit <enter>, I’ve set Hazel up with another Automator script: Take the file, rename it with the pattern “John_Metta_Timesheet_<date addedd>”, mail it to the company as an attachment, and then throw the local copy in the trash (because Google Apps mail will store it for me).

A Robotic Secretary

The function of a secretary is to take care of everything that doesn’t absolutely need your action. A secretary should take care of everything in a “Do everything and let me just sign the paper.” That’s almost what I have now.

With this workflow, I’ve gone from 16 tedious steps to

1. Track time daily in Freshbooks
2. Friday morning, while working, run the ruby command with my hours³
3. Work until lunch/break, before stopping, check my email (have an email from my PM waiting)
4. Click “download”⁴
5. Sign, scan, then hit enter

That’s it!⁵

Coda

I highly suggest looking into combining Hazel, Automator, and any other talents you may bring to creating robotic secretaries for your automated tasks. This is just an example of a way you can take a morning’s worth of frustration and productivity vacuum and turn it into a few steps. Best of all, it took me about as long as two of these frustrating mornings to set everything up– meaning that it pays itself off in about half a month, on a year-long contract.⁶

1. There’s actually an API library for Freshbooks in Ruby that is written and maintained by Ben Curren at Outright.com, so you could pull the hours straight from the project in a Ruby script- I decided against this step
2. In full disclosure, the Automator script actually uses “Save to a variable” “create new mail message” “get from variable” “add attachments” and “send mail”– in that order. For some reason, the “add attachments” phase was throwing an error otherwise.
3. This automatically does everything through emailing my timesheet to my PM
4. This automatically files and prints the timesheet, it’s just awaiting my signature
5. Well, I still have to mail it to New York, but that final step is a lot less annoying when it’s not preceded by so much tedium!
6. I’m sure this’ll only get easier as more companies have APIs available. As I mentioned, I could probably have an automated task in Windows start the Ruby  script and use output from a Freshbooks API query to automate step 2 above, meaning that I’d do nothing until I got an email from my PM. However, I wanted the additional steps of adding my hours manually- more as a double check than anything.

Like tomato sandwiches, Celtic music, beer, and programming- Github is something that, try as I might, I just can’t make myself sick of.¹

Recently, I took the Git survey, and it contained an interesting question along the lines of “What do you use Git for?” The answers were things like “configuration files” and “large binary files.”

I use Git and Github for a lot- configuration files are one use (I love the “raw” path), but it was only a small percentage of what that list suggested, so I started wondering how else I could use it.

Today, I thought of an interesting way: Github as a CDN.²

If you know what a Content Delivery Network (CDN) is, you don’t want me to explain. If you don’t, then it’s good enough to say that it’s a way to server up pictures or other content from another server. It’s often used to, say, serve the main portion of an html page from a webserver, but serve the images, CSS files, etc. from something like Amazon to speed things up.

I recently finished work on The Great John Metta Branding Project and, like with so many projects, stored the images in Dropbox. Then, deciding I might want to version them, I stored them in Github. Then, just as I was starting to begin the rather annoying process of copying all these logos and textmarks to my various sites throughout the web and updating all the image tags, I thought to myself “Hey, if these are already in Github, and I can access the raw files, why don’t I just use that as the image source?” But I don’t want to have to remember the URL each time. What if it was somewhat dynamically generated?

A RESTful CDN system for a logo

This is exactly why I made johnmetta.com a Rails project, to play with stuff like this and then have a place to have it be live if I wanted to use it.

I’ve got a lot of different styles that my logo can take. Stacked, Icon only, textmark only, icon one-color grey, textmark one-color blue, etc. Ideally, I wouldn’t have to remember the path to each, I could type something like http://johnmetta.com/brand/icon/blue/transparent/logo.png³ and have the app return the image that is strictly the icon (no words) with blue coloring and transparency inside the logo (instead of the normal white).

Since none of these actual images is created on the fly, I created a sensible directory structure and embedded all the files in this structure, naming them all logo.png and putting the default logo in the top level. So the above image would be {base}/full/logo.png, while a version with the icon on top would be {base}/full/stacked/logo.png. A one-color version of only the icon might be {base}/icon/blue/logo.png

So I created a Brander class which would turn arguments into an appropriate path and added some new routes such as “brand/:type/:color/logo.png” and created a call to brander in the controller’s “brand” action that ends with a redirect to the returned URL.

It’s all really simple, but it allows me to have a url like http://johnmetta.com/brand/icon/logo.png and supply my logo, and http://johnmetta.com/brand/stacked/blue/logo.png for a one-color blue stacked logo. Then, if I update the colors (move the files, whatever) they update automatically. At a maximum, I just check in the changes and make minimal changes to the routes- but don’t have to copy files to servers.

A Good Illustration

Right now, the Brander class is pretty dumb- little more than a way to translate a route into the correct external URL. It was built up not as an illustration of how to accomplish “Github as a CDN,” but merely as a toy to test one way that it might be done on a small scale. It might be better done as a class where instance attributes make more sense, and I’ll clean it up a bit, but not much more than that.

Also, it’s cool to know that stuff like this can be set up so easily, and for my uses- where bandwith might reach a whopping few requests per day, it’s useful while being low impact. But for a real CDN, you’d still want to use something like AWS- or at least talk to the Github guys before swamping their webservers.

1. I’ve tried for the past two summers to make myself sick of eating tomato sandwiches. I just can’t.
2. The title is with Rails because that’s what I used, but it would take 2.4 minutes to make this a “With Django” or “With Lift” example
3. Actually having “logo.png” is unnecessary, however, sometimes the software using the URL will depend on that, so I just made it convention so that I don’t have to think about when sometimes is.

Smart and Gets Things Done

As best I can tell, this trend really took off– even if it didn’t start– with Joel Spolsky’s blog article titled “The Guerrilla Guide to Interviewing.” In that article, he stated that there was one primary requirement for working at Fog Creek Software: “Smart, and gets things done.”

His basic argument, with which I tend to agree, is that it doesn’t really matter what your past qualifications are as much as it matters that you can do to things: 1) Learn shit, and 2) Actually do shit.

His point? If you can’t learn, you’ll be stuck trying to write software in Visual Basic and someone will eventually shove you off of a roof out of frustration. Furthermore, if you don’t actually DO anything, but rather just talk about it, or think about it, or tell others why you could do it better– then you won’t even write software in Visual Basic. You won’t write any software at all, and someone will eventually shove you off of a roof out of frustration.

But if you can learn, it doesn’t matter what you come into the job knowing, because that’s merely a starting point. It’s how your knowledge and skills translate over time that’s important. You can learn <insert whatever you need to do your job here> quickly. I say quickly because you are a DOER, and doers always learn new stuff, so they can DO more stuff.

The most important thing here: The job will probably change, and when it changes, it’s the people who can LEARN, ADAPT and DO that will help the company succeed. The person who came to the job with one incredible skill but can’t learn probably has that one incredible skill in a stupid technology like Visual Basic, and someone will eventually shove–

well, you get the idea.

The problem with catchy words

So, herein lies a bit of the problem. Joel also talks about hiring the best people, treating them like “rockstars,” etc. Again, I think he has a point, but that there are a great deal of people who are stupidly picking up the hot new terms like “rockstar” and “ninja” and using the words, basically, without using their brains.

I see them a lot. Those posting that sound all hip and cool: “Are you a Python Guru?” or “We’re looking for a Ruby Rockstar for-” or “We want a PHP ninja to-” Everytime I see one of them I want to slap them in the head with the nearest O’Reilly book and then vomit.

This type of job posting proves one thing to the very people that you want hire: That they don’t want to work for you.

Why?

Because the very people you want to hire are the ones who describe themselves as “hard working” or “with a lot to learn” or even “not as good as many, but loyal, friendly and likes to learn new things.”

Do you really want to hire a ninja?

What’s a ninja?

It’s such a stupid, overused buzzword! Do you even know what it means? It’s either an assassin or a stupid 14 year old jumping out of a dumpster brandishing a medieval sword.

You don’t want either of those things!

Ninja: It's a word that's THIS overused

Yes, there’s some evidence that a well trained ninja was, if absolutely nothing else, a competent assassin (though the Samurai made fun of them). A ninja is basically a person who does one single thing really well, they kill people, and the rest of life- including the whole “getting along with people” part- they could give a rat’s ass about.

Human interaction to a ninja is “kill them!”

Conflict resolution to a ninja: “Kill them!”

For those who miss my subtly: A ninja programmer’s response to pretty much anything is going to be this: “Do it my way and no-one gets stabbed in the face with my medieval sword.”¹

Think about it. Do you really want to hire someone who does one single thing really well? To the exclusion of things like “showering” and, say “talking to other human beings?”

Ninjas are Zombies!

Here’s a neat trick: think of another stupid, overused buzzword: Zombie. What if I told you that ninjas are just zombies with black bags over their heads?

They do one thing really well: kill people (eating their brains- mostly the brains of your team if you hire them), and they could give a rat’s ass about things like “human interaction” and “conflict resolution.” What’s their solution to everything? “Kill them! (and, since we’re here, we could maybe snack on their brains too…)”

Ninjas are zombies! They’re mindless idiots going around trying to do one thing.

That’s it.

When you say “I want a ninja” you’re saying “you can be a complete asshole, refuse to learn anything new, refuse to respond to other human needs or the needs of the business, and also be a social trainwreck. Oh, and you don’t really need to know, or care, about 90% of programming or computer work, but if you can sit in your hole and <do that one thing well> and not talk to anyone, you’re the guy for us!”

You want a ninja-zombie as your lead developer, don’t you? Admit it.

Rockstars: They go to 11

Alright, I think you get my point, but let me touch upon rockstars for a moment. Here’s another place where I think Joel’s point was completely missed by a lot of people. Joel says “hire the best people you can find and treat them like rockstars” and in pure politician-like “I don’t really want to put any actual cognitive thought into this process, so I’ll just pull a buzzword”-style, job announcements start popping up for undefinable qualities such as “Ruby Rockstar.”

You want these guys in your company?

The mistake here is that all the corporate bozos think “Hey, ‘Rockstar’ is the current buzzword, so I’ll use that too!” without stopping to think about one thing: “Rockstars don’t usually make the best employee material.”

Think of the words “punctual” and “hardworking.” Okay, now, keep those words in your head, and think of the word “rockstar.”

Yeah, your neck hurts doesn’t it?

See what happens when you take things out of context? Joel’s statement was “treat them like rockstars.” In otherwords, treat them like they mean something, like they matter, like the company depends on their happiness… so they will be happy… and do really good work… and make you more money.

The line was emphatically NOT “Make them into rockstars.”

Seriously! Think about your average caricature of a rockstar.² They show up late, drunk, stoned, with a 16 year old’s bra stuck to their belt, and give you a loud, shitty performance of something they don’t feel like playing before hopping back into the bus for more sex, booze, and food.

Okay people, repeat this after me:

“I don’t want to hire a Rockstar!”

A rockstar is probably worse than a ninja, because at least a ninja- or a zombie, for that matter- can do one freakin’ thing well. The only thing the rockstar can do is pray to god that the sound team can mix together the shit they call a studio performance, and that the stage crew can hit blow the fireworks early to cover the guitarist tripping and falling because they’re too freakin stoned to remember that there’s a drumset behind them.

They think of themselves as the best thing that’s ever happened to you, and if you tell them otherwise, they’ll freak out.

Rockstars look good. Full stop.

You know who the rockstar is? It’s the young programmer I met at the Open Source Bridge conference who immediately berated me for using a different database strategy than him– without ever stopping to listen to the problem I was solving, to hear about my application’s design, or to learn why I would choose one over the other.

I was wrong, they were right, and I should really just stop being stupid and do it their way.

Yeah, that’s who I want on my team.

Fucking rockstars.

Are you an incompetent programmer with an overblown sense of self-worth?

Here’s the part that bugs me the most: The people posting these job announcements are actively selecting for people with a tendency to overstate their abilities while understating their faults.

It’s called the Dunning-Kruger effect and it’s well documented in both the scientific literature andpopular journalism.

The basic Gist is this: Incompetent people tend to overstate their abilities and think they are amazing, while highly competent people tend to downplay their skills and think that they are less than amazing.

And you know it’s true.

Go pick the first male rockstar-ninja-zombie programmer you can find and I’ll pick the first quiet, understated female programmer I can find. Then we’ll see which one can actually shut the hell up about how great they are and get something done.

The best people for the job are not the ones who are going to feel comfortable applying for the “Amazing Rockstar!!1!” position for the very reason that they are the ones that you want to hire: because they are too busy being amazed at all the stuff they don’t know to be rockstars who tell you everything that they do know.

They don’t think they know everything. In fact, with everything there is out there, they realize that they know basically nothing. Most importantly: They know that there’s a lot to learn, and they are trying to learn it.

The rockstars? The ninjas? They already know it, and they’ll tell you.

In fact, They’re more than happy to tell you how much they know.

Every single time you interact with them.

Folk Developer: Will learn and be nice for food

I don’t apply to any of these positions because, while there are a lot of things that I am and can do, there are certain things that I am emphatically not. A partial list of the things that I’m not is:

• Rockstar
• Ninja
• Zombie
• Best person in the world at <fill in whatever you want here>

Me, I’m not any of that. And that makes me think, from the majority of jobs I see lately, that I won’t be good enough to make the cut. And let’s face it, if they’re advertising for a rockstar, then I am probably not, because I do a lot of things competently, not one thing better than everyone else including you.

See here’s the thing: I’ve been programming for 25+ years, I’m competent in at least 10 different languages, and really good in at least 4.³ I’ve built everything from robotic control systems to mathematical models, from spatial applications to social web applications. I’ve taught programming, and am about to do so again, and I do other things like start a Ruby users group.

By many-if-not-all accounts, I’d be one hell of a developer to have on a team; yet many teams are seeking rockstars.

I am not a rockstar.

I’m not the guy on stage with lights and explosions and a screaming electric guitar.

I’m the guy who goes to a party and sits on the couch singing a song on a guitar. A really fun song, with really good guitar work, but not anything that needs lights and explosions. I may also pick up a banjo or back someone up on a number of other instruments, but mostly I just stay at the party and hang out with people.

No lights. No explosions.

I’m a folk developer.

I’m a seeker, I’m a learner. I’m a hard worker who will spend his free time coming up to speed on a technology for curiosity as well as success. I’m loyal, I’m friendly, I’ve got a sense of humor and would much rather laugh at myself than at anyone else. I spend my off time programming, like lots of people who love programming, but I also spend my off time playing the Irish flute and banjo, cycling, brewing cider and mead, working in community theater, and lots of other social pursuits.

Despite this (or, rather, because of it) I see myself as little more than “a decent programmer who’s probably not as good as most, but might be better than some.” In fact, I don’t have enough fingers to count the number of jobs I’ve actually turned down because I thought that I was probably not good enough– only to find that someone else was hired whom I actually know that I can outperform.

I am not going to apply for a job as a Python ninja or a Ruby Rockstar, because I’m not a ninja or a rockstar. I’m a person who knows a heck of a lot less about Ruby than many other Ruby programmers.

I’m a person with a lot to learn.

Exactly zero of my qualities describe a rockstar.

Are you an incompetent company with an overblown sense of self-worth?

And I know that’s also true of many of my developer friends and colleagues. Companies select for people who are not them.

Here’s the clincher. Most of those companies who are hiring ninjas and rockstars are probably doing so because they see themselves as ninja and rockstar companies.

They are the companies that say things like “do you want to work in our cool-ass company where everything is better than any other company you’ve ever worked for, where we have foosball all day and are all awesome and badass about everything we do?”

Sound familiar? I’ll bet it does.

It sounds like a rockstar of a company.

1. No, don’t ask your ninja programmer why they have a European weapon, you’ll get stabbed in the face.
2. which is all we’re really talking about in either case- neither Joel or you are talking about Dar Williams, here.
3. 5 if we count Ruby, which I’m learning more about every day

It’s a small thing, more of an annoyance than a real problem. Something like my tendency to forget “end” after blocks (a Python holdover) and to assume that zero equals false (a sort of “everything else” holdover).

It boils down to this:

belongs_to is singular,  you stupid dipshit! Stop being a mindless screwhead and remember that for a change, John!

Honestly, it only takes me a second once I get the ever familiar error, but it’s just better to not get the error at all, right? Efficiency and correctness.

If anyone reads this and runs into me later, I wouldn’t mind if you repeat that statement to me.

More reminders couldn’t hurt.

[gist id=450723]

1. It’s one of those things that could be in a plugin or gem, but I’m getting really sick of gems and plugins that carry all the infrastructure to support being gems or plugins when they are really simply a wee bit of code in one small file. Thus, I’m starting to just publish Gists and add them manually to my lib directory. They’re not really gems– they’re more like ungems. Maybe shards.

We’ve all got the same issues. “Use strong passwords!” which means “don’t use just words” is something we all try to take seriously, but it just ends up being something like “c00lp455″ with even the most basic cracker program would handle easily– replacing letters with numbers was maybe a good strategy in 1988, but you might as well just use “coolpass” today, because that’s not any easier than using numbers for any attack.

“Don’t use the same password often,” “Use passwords with enough entropy (randomness),” “Use passwords longer than 8 characters with upper and lower case letters, numbers AND PUNCTUATION!!”

Whatever.

It basically comes down to this: We all have one password we use for freakin’ everything, and that password is probably something like “mysillyp455.”

Sigh.

I’ve been wanting to get away from this lately, but have been generally disappointed at the options.

There are dozens of password managers out there, and hundreds or thousands of password generators. I have issues with all of them, and recently decided to address them in a bit of a crazy way: By completely re-thinking how I handle passwords.

Password generators suck

You know it’s true. They come up with the most ridiculous passwords that you’ll ever see. Who the hell is gonna remember ’5+b(*|$1e8+8@’ is their password?

Oh, right you want pronounceable passwords, because ‘m4k1tw0rk’ is going to work better than ‘makeitwork.’ Right. See above.

I mean, they do their job, sure, but password generators generating random strings are not really very useful to me, because I’ll never be able to recreate the password, which means I have to write it down somewhere or store it in a password manager…

Password managers suck

Yeah, I know, you’re a dedicated 1password user and you think I’m an idiot. It’s a decent solution¹ but it, like any manager, still suffers from some serious flaws.

One is, of course, that if you’re somewhere or somehow don’t have that manager available, it’s absolutely impossible to get your password. This has recently happened to me while trying to use an app called LastPass, which generated really nice, ridiculously hard passwords for all kinds of stuff, that I could then never remember and I couldn’t get later when I screwed up my master password.

Reminder emails galore!

Here’s another problem: My passwords are stored on somebody else’s webapp database. Oh, sure, they’re stored ‘securely,’ but how the hell do I know that’s true (read the TOS people, they limit their liability for a reason).

If the pentagon can get hacked, some 24 year old start-up geek is certainly vulnerable.

Single points of failure suck

The biggest things about all of these is the ‘single point of failure’ issue. Password managers store passwords in encrypted files, so that’s nice. But what if someone gets your all important 1password that secures all your other passwords? You are, in a word, fscked.

So, the basic problem with all of these is that they are not actually as secure as we think, and when they are, they’re not easy. So we either have secure and hard as hell, or we have easy and fundamentally flawed, right?

Maybe not.

Password generation as password management

Okay, no lie, this came to me while somewhat tipsy and pouring down beer at a local pub:

What if your password wasn’t stored anywhere, ever. What if you could simply have a way to securely recall those ridiculous passwords that generators come up with out of thin air?

Think about it. 5+b(*|$1e8+8@ is probably a pretty damn secure password– if you used that for anything, you’d be pretty safe, but you’d never remember it. But what if that was, in a way, mnemonically generated?

Enter Passfish.²

Passfish is a program (actually, it’s an idea)³ that generates passwords from the combination of words, a secret key file, and (optionally) name and passphrase parameters. Using this combination of items, Passfish will generate a high entropy password.

$ passfish 'name@email.com'
 9!b%e+70

Now, that password is not stored anywhere, not in any encrypted file, nowhere.

Furthermore, if you use the Bash shell’s ability to ignore certain commands, that won’t even be in your Bash history. No-one can get it in a file, no-one can see what you’ve typed. It’s ephemeral. You call it up, and then it goes away.

But it’s still insecure as hell, right? I mean, anyone can figure out that name@email.com turns into SOMETHING.

Generation and modification of hashes on the fly

Keyfiles solve this. When you have a private keyfile, the string name@email.com is combined with the contents of this file, a hash is generated from that combined string, and the hash is modified in a repeatable way. This means that you can have super simple “passwords” (e.g. name@email.com) and even write them down on your forehead, and no-one will be able to generate your actual password without having your same keyfile.

But wait, there’s more!

$ passfish 'email' --name 'work'
(|@+#^7b
$ passfish 'email' --name 'home'
$1e8+8@

You can specify an additional parameter, a name. This allows you to increase the security because you could have the same “password” for multiple accounts. You can also have passphrases:

$ passfish 'email' --passphrase 'I hate Dirty Dancing!'
$1e8+8@

This allows even more security!

There’s also a length parameter:

$ passfish 'email' --name 'work' --length 13
$1e8+8@$1e8+8

Simple mnemonics for complex passwords

One of the things many people do when thinking of passwords, is create devices that they use in ALL of their passwords. For instance, start with an @ and end with a $, then capitalize the first letter, so a password could be something like ‘@Email$’.

What Passfish allows you to do is to create similar mnemonics, but use them wisely to create truely complex passwords. For instance, it’s a website that I use at work called “workstuff.com”

$ passfish "workstuff.com" -n "website" -p "at work"
$1e8+8@

Thus, the password is not stored anywhere, but I have an easy way to remember it.

No single points of failure

This way, even if someone steals my computer, they can’t get my passwords unless they have ALL of the following:

1. The password identifier
2. The understanding that I’m using some stupid geeks weird Passfish program
3. Access to my private key file (which could be any file I choose)
4. Knowledge of the name, passphrase and password length

Rather than a single point of failure, there are multiple points for each password, because even if someone got your private key file figured out, they’d still have to spend considerable time getting each password– and access to one password makes it no easier to access another password! It’s not an all-or-nothing proposition.

The Status & Future

So, what’s the status? Well, it’s currently an open source application on Github, and it’s little more than a proof of concept. All it does now is combine all the parameters together into a string, hash it, and modify the hash. Pretty stupid, I know. (The plan is to do things better, like use the passphrase parameter to strengthen or change the hashing, etc.)

And as for the “but that’s stupid and hard” people, the other plan is to create apps that tie in with this underlying generator for things like Android and LaunchBar. For instance, a LaunchBar plugin that lets you type the identifier, name and passphrase, and then automatically copies the generated password to the clipboard, so that you don’t have to even see it, much less go to a terminal window to use Passfish.

So, it’s new, it’s still pretty dumb, but as a proof of concept, I throw it out there as an idea, knowing that– because it’s open source– any serious vulerabilities or blindspots will be illuminated.

Check it out, fork, modify, correct. Throw me ideas and comments or ignore it altogether.

1. or at least it would be if they’d make a freakin’ Android app!
2. Disclaimer: this is a proof of concept. A crazy scheme. It needs work, so I wouldn’t jump into using it. At least not yet.
3. actually-actually, it’s planned to be an algorithm, but right now it’s simply a quick hack as a proof of concept