Finding all ActiveRecord relationships in a database

Screen Shot 2013-11-15 at 11.51.42 AMRecently, I needed to find all of the relationships in a MySQL database. The database is not relational in the sense that the foreign key constraints are set, rather, it is managed by ActiveRecord’s has_many and belongs_to madness.1

I love Rails, and the Rails magic that makes Rails Rails, is great… except when it isn’t.

For instance, when you have a few hundred million records in a database, and need to update the IDs, but have those ID updates cascade throughout the system. You can’t, as it turns out, rely on SQL “ON UPDATE CASCADE,” because the foreign key constraints are not in place. you could, of course, rely on Rails, if you want to instantiate every object into Ruby space, thereby avoiding the efficiency of database operations.

But, we can actually ignore the reasons why someone would need actual SQL-standard table design for now, and just assume that someone like me might, in some cases, want to interact with their database directly. Given this possibility (well, given the actuality), I wrote a helper class to do something I could see no other way to do: Find all of the “This is related to that“s in my database so that later I could do something to both this and that safely.

Code

This is the stripped down version in a Github Gist. Go there, it’s too long to embed.

Given this code, and some knowledge about what’s actually in your database, it should be easier to make wide-sweeping and safe changes to IDs2.

Helper Examples

> arm = ActiveRecordMappings.new(command: "mysql -u john -p", database: "my_stuff"
> arm.all_mappings
--> { :activities=>[:user_id, :item_id, :item_type, :enterprise_id],
      :activity_items=>[:actor_id, :actor_type, :item_id, :item_type, :indirect_object_id, :indirect_object_type, :enterprise_id],      
      :applications=>[:user_id, :enterprise_id, :application_id],
      :authorizations=>[:user_id],
      :comments=>[:commentable_id, :commentable_type, :user_id, :recipient_id, :enterprise_id], :completion_records=>[:user_id, :enrollment_id]}

> arm.id_mappings
--> { :activities=>[:user_id, :enterprise_id],
      :activity_items=>[:enterprise_id],
      :applications=>[:user_id, :enterprise_id, :application_id],
      :authorizations=>[:user_id],
      :comments=>[:user_id, :recipient_id, :enterprise_id]
    }

> arm.id_mappings for: :enterprise
--> { :activities=>[:user_id, :enterprise_id],
      :activity_items=>[:enterprise_id],
      :applications=>[:user_id, :enterprise_id, :application_id],
      :comments=>[:user_id, :recipient_id, :enterprise_id]
    }

> arm.polymorphic_mappings
--> { :activities=>[:item_id],
      :activity_items=>[:actor_id, :item_id, :indirect_object_id],
      :comments=>[:commentable_id]
    }

Peanuts

I’m sure there’ll be plenty of “That’s stupid, use ActiveRecord.some_obscure_undocumented_feature to do this, idiot!” from the peanut gallery,3 but as I won’t know about that obscure, undocumented feature until some tact-challenged geek lambastes my solution, I’ll use this solution. Rails-based solutions also don’t really help if you’re outside of the context of Rails (i.e. trying to use SQL for SQLs very good efficiencies), which is the environment for which I wrote this.

  1. Note: I generally call this “goodness,” until I need to figure out actual SQL relationships []
  2. which, it turns out, is the reason I need it. Not a common scenario, but there you are []
  3. Note: This is a paraphrase of a comment to some previous code. It’s a rather sad state of cultural affairs when one feels they have to write disclaimers about everything, but, there are a hell of a lot of wannabe Aspergers coders out there. Hint, helping is good, lambasting is bad. You’re not House, you’re a dick. Grow up. []

Through the Looking #Glass

Today, an interesting post came across my Google+ stream:

We are redesigning the Twiddler (as you may have noticed in Thad’s posting in July) and would like to get your input on the use of indicator lights on the device… (via Wearable Computing – Community – Google+.)

Twiddler, a chording keyboard

For those who don’t know, the Twiddler is a one-handed chording keyboard. It’s a keyboard on which you can type any character by pressing multiple buttons (chords) at the same time. The post was interesting because it brought back memories of a time in my life when I sought out such technology with a wild fever.

Into The Matrix

In the late 1990s, I dove head first into the world of wearable computers. I had a Twiddler before upgrading to a Twiddler 2. I hooked it up to a PC/104 computer (hanging around my neck by a strap) that ran FreeBSD and combined it with with a $2500 set of almost custom bulky glasses with a projection screen. I simultaneously taught robotics to 8th graders, hacked wearable computers and early handhelds (remember the iPaq?), and designed autonomous underwater vehicles. I even published a series of articles in the then nascent journal Linux Gazette on the philosophical aspects of the Open Source Culture1

It was a heady time of seemingly endless possibilities, and an incredibly positive social outlook on what technology could do for us.

We’re connected, always connected, but who– or what– are we connected <em>to</em>?

Lately, and especially since having children, I’ve become rather sad and jaded. What seemed like a wonderful future of mobile computing and people-centered connectivity has turned into little more than a landscape of hyper-targeted advertising injected directly into the brains of a population that is ever more disconnected from the present moment. I don’t know what I wanted when I built my own computer from scratch, but it certainly wasn’t taking my kids for a walk in order to hack an Ingress portal at the local park– barely aware that my children are even with me.2

I thought it would bring us closer, but we seem to have traded actual human connection for an almost completely unconnected lifestyle. We’re connected, always connected, but who– or what– are we connected to?

We’ve gone from being present in the moment with a close friend to having a conversation with that close friend only to constantly break off to look up random, unimportant trivial facts, or text someone who isn’t even there– and doesn’t need to know right this second that you are supposedly talking with a friend.

No, no, I’m listening, really. I was just looking up something on Wikipedia while I listen…

It’s even worse when I realize that I spend all day in front of a computer, programming web applications, only to then go home to stay in front of the computer… wasting time posting on web applications. How much in the moment am I if I go on a hike in the woods and need to tweet pictures of a particularly interesting mushroom, rather than actually be present and experience the particularly interesting mushroom?

And there’s a certain abject horror to my phone pushing coffee advertisements to me as I walk past Starbucks. (“Wait, why am I suddenly seeing Amazon advertisements for a book called “Mushroom Hunting in the Pacific Northwest?”) Is it just me, or does 90% of our new tech seem to be centered around phones with better advertising? Because that’s really about all I see. It’s coached as “Location checkins” or “Friendship requests” but in reality, it’s all about better advertising. We’re just selling– rather, willfully giving away– our souls to provide advertisers better access to us.

And the more I think about all the pictures I post of my wonderful twins, the more horrifying it gets, because I’m literally donating all the data they need to identify them using facial recognition engines (For an article on how advertisers are using facial recognition to store information about children in order to market to them, here’s an article in Slate magazine that will terrify you).

Glasses and #glass

As an anthropologist, I think a lot about the cultural implications about this, and it’s rather ironic that I would turn away from one set of technology, but embrace others. Cyborg Anthropology, the study of human interaction with technology, would tell me that I’m already a cyborg. I wear glasses, which enhance my human vision with artificial technology.

Screen Shot 2013-09-11 at 12.03.25 PM
Screen Shot 2013-09-11 at 12.07.18 PM Some are better able to manage multi-tasking than I am

Part of me thinks “This is nothing more than pen and paper, which is just humans using technology.” Part of me thinks that wearing Google Glass is– in the grand scheme of things– little more than me wearing better glasses.

Amber Case, one of our3 preeminent cyborg anthropologists seems to agree. She’s commented much on how multi-tasking reduces one’s ability to focus on a single, particular task– yet as a cyborg anthropologist, she’s embraced Google Glass, moving beyond “multi-” into a hyper-tasking landscape where not just one, but endless other tasks are possible at any given moment.

The anthropologist in me thinking that this is just simple iterative advancement of human technology. Yet the human in me feels that some line has been crossed, like some vague, arbitrary– perhaps meaningful only to me– moment has been reached. I feel like I am standing at a threshold to a virtualized existence, beyond which I can’t seem to go without feeling as though I’ve lost something… real.

Maybe others are better at this than I am. Maybe I’m one of the few who can’t effectively be present in the moment with a computer literally in front of my eyes. Amber Case obviously is, I hope others are. I hope others are better than me, because if not, I’m sad for the ever increasing lack of depth every human relationship will have from this point forward.

For me, however, it’s a different story. Whatever I’m doing, I always feel like I’m thinking about something else, and that something else is always right there, accessible, at my fingertips.  I’m just about fed up with that feeling. I want to look people– look my children– in the eye and have them know that I am there, with them, in the moment, and not looking up a Wikipedia reference about something they just said.

There and Back Again, A Technologists Tale

William Gibson once spoke4 of this undocumented moment in time in New York… the week immediately before the introduction of broadcast television through the week immediately after. During the week before broadcast television, everyone was outside in the evening, sitting on their porches talking with one another. No-one knows what happened outside during the week immediately after, because everyone was inside watching television.

I feel as though we live in a similar timeframe, where something fundamental is happening to our human relationships without us consciously considering the consequences. I feel like it’s happening too fast for us to be able to consider it. I’m not anti-technology, far from it. I’ve been all the way into The Matrix– hell, I still work in it as a software developer– and yet I’m just about done with hyper-connected technology.

My preferred platform for mobile connectivity

While so many others are buying better smartphones and getting excited about Google #glass, I just dropped my data plan and got a basic phone so that I can still make a call, but not do, really, anything else. Every day I carry my silly leather satchel, filled with my fountain pens and my notebooks, and I write.

Yes, by hand.

Since January, I’ve published poetry and fiction, and written about 400 letters, often notes to people I barely know. And you know what? Every time I write something by hand– no matter how long it takes, or how trivial it is– I feel a bit less inclined to post something on the internet. Anything digital just feels less and less… real.

There’s something so vastly different between an email or Facebook post and a handwritten letter that they may as well reside in completely separate emotional landscapes. And, when I step back and look at those two landscapes, there’s really only one in which I want to live.

I’ve written before on this, most notably when I left Facebook. I’m still on Twitter and Google+, but I see myself interacting with them a great deal less as time progresses. Just like back in the 90s, when I dove headlong into the modern world of connectivity, I’m doing the same; but this time, I’m coming back in the other direction. Sometimes I feel like a lone pedestrian heading the wrong way on a one-way street full of fast moving traffic.

And just as I didn’t know exactly what I wanted from “connectivity,” I’m not entirely sure exactly what I want from a lack thereof. So far, what I’ve found is that the less “connected I am, the more connected, and the more human, I feel.

For now, that seems like a good place to start.

  1. Reading them now, it was obvious that I was just a kid. It was so long ago! I did it under my pre-marriage name: Pennington. Part 1: The Philosophy of Doubt; Part 2: Cooperation vs. Competition. Strangely, I can’t find a link to the third in the series. []
  2. You want to know crazy? I took my infants on a walk in the stroller– delaying their small but important dinner– just to walk past specific places in town at a specific time in order to play a goddamn virtual reality game on my phone. That is fucking crazy, people. []
  3. whether “our” is “The Portland Metro area” or “The Country,” I’m not sure of, but I’d bet it’s the latter []
  4. paraphrased []

Solano: Chocolaty good customer service

Today, I got back from a short vacation to find a box sitting on my desk.

Here is the story of that box, in images.

First, I wrote this

Screen Shot 2013-09-10 at 11.20.24 AM

Then, they wrote this

Screen Shot 2013-09-10 at 11.17.12 AM

So, naturally, I wrote this

Screen Shot 2013-09-10 at 11.20.13 AM

Then, an unexpected box arrived at my office…

Photo on 9-10-13 at 8.58 AM

…with a very nice note from Solano Labs tech support:

Dear John,

Thank you for being a Solano Labs customer.
Here is a refill of coffee and chocolate for you.
We hope that you can focus on your builds again!

From San Francisco with love,
Solano Labs Support.

Solano Labs, you’re too much! Thank you!

Comments Provided by Google+

Comments have been turned off on my blogs for months, and I’ve been happy with the result. Gone were both the SPAM comments and the less than useful comments. There is a full discussion of my reasoning in a Google+ post, should anyone be interested.

Recently, I learned that one could add embedded Google+ Comments on any page. Naturally, this appeals to me since I send my users to G+ to comment anyway. This provides a nice comment system in the place that I want it, embedded right into my blog posts.

It’s simple, a wee bit of code:

&lt;div id=&quot;comments&quot;&gt;&lt;/div&gt;
&lt;script type=&quot;text/javascript&quot;&gt;// &lt;![CDATA[
gapi.comments.render('comments', {
    href: window.location,
    width: '624',
    first_party_property: 'BLOGGER',
    view_type: 'FILTERED_POSTMOD'
});
// ]]&gt;&lt;/script&gt;

Added using the Post Ender plugin, and we’re in business. Now, encouraging people to comment in Google+ is automatic, and I still have no SPAM worries.

Using RSpec With Mixed Random and Ordered Tests

I prefer to have my RSpec tests ordered randomly to ensure we have condition independent code. Back in July, RSpec 2.11 was released, which added support for the –order argument on the commandline. Now you can add “–order rand” to your .rspec file and have all your tests run at random.

But what if you want some tests run in order? Easy, run your specs with ‘rspec spec/path/to/spec –order default’.

Love it.

Continuous Integration Gotchas

There’s a gotcha though. What if you are– as I am– using a continuous integration server. I want most of my tests to run in random order, but I want some of them to run top down in the file. That’s because we have some pretty specific algorithm and analytics code that depends on the system changing over time, and we use the Timecop gem to watch that change over time.

Sometimes, it pays to have specs that are dependent on the state of other specs.

We want our tests to run on the CI server in random order, but if they do, the ordered specs will fail. This is an issue– one that, it turns out, others have had.

Defining Some Specs To Run In Order

It turns out that rspec-core had a very interesting pull request about 10 months ago. One that adds support for defining, file-by-file, the order of rspec groups and examples. I haven’t seen any documentation on it, so assume it’ll come out in version 2.12, but there are some interesting specs showing how it works.

Ordered Specs In A Single File

Let’s assume you want to run all of your specs randomly except one file. Let’s call that file “ordered_specs.rb”:

require 'spec_helper'

describe Trove do
  RSpec.configure do |config|
    config.order_groups_and_examples do |list|
      list.sort_by { |item| item.description }
    end
  end

  describe "Day 2" do
    specify("PM") {}
    specify("AM") {}
  end
  describe "Day 3" do
    specify("AM") {}
    specify("PM") {}
  end
  describe "Day 1" do
    specify("AM") {}
    specify("PM") {}
  end

end

As you can see, we are calling the configure method and telling it to order the groups and examples in this spec file. Given this, the output when run is:

john@patience:~/QStream/qstream
$ rspec spec/trove/ordered_spec.rb

Trove
  Day 1
    AM
.    PM
.  Day 2
    AM
.    PM
.  Day 3
    AM
.    PM
.

Finished in 9.23 seconds
6 examples, 0 failures

Finished in 9.23 seconds
6 examples, 0 failures

Randomized with seed 9386


Randomized with seed 9386

john@patience:~/QStream/qstream
$ 

As you can see, the spec was ordered by the description, even though we’re running with “–order rand” in the .rspec file.

Now, I can have all my specs go to my continuous integration server and get run at random, but have the few tests that look at algorithm and analytics as a whole run in order!

Love. This. Stuff.

How Do I Get It?

I’m not sure this functionality is gemmed up yet, so to get it, I’ve loaded the current HEAD ref in my gemfile:

 gem 'rspec-core', git: "git://github.com/rspec/rspec-core.git", ref: "5d28318b51bf212c215ba1392fb24cc5a52dbe06"

Turning issues into pull requests

For the past several months, I’ve been using a nice feature that I’ve recently found out that many others don’t know about, so thought I’d draft it up.1

You have an issue in Github, so you do some work, then you issue a pull request to fix that issue. In Github, that means you actually have two issues in your issue list, the original issue, and the pull request. Well, did you know that you can easily turn the issue into a pull request? And it’s easy.

The secret is using the Hub gem. After that’s installed, ‘git pull-request‘ is a commandline option.

Assume I ‘cucumber’ branch, cleaning up the ‘we’re in a pickle!’ issue, which happens to be issue #39. It’s easy to turn “issue 39″ into “pull request 39″ by doing

git checkout -b fix-the-pickle-issue
git commit -m "Make some changes"
git push origin fix-the-pickle-issue
git pull-request -i 39 -b cucumber -h fix-the-pickle-issue

If you’re working on different accounts/origins, you can use fully referenced names:

git checkout -b fix-the-pickle-issue
git commit -m "Make some changes"
git push origin fix-the-pickle-issue
git pull-request -i 39 -b johnmetta:cucumber -h mettadore:fix-the-pickle-issue

The pull request is created on Github, and automatically linked to the issue, turning it from “issue 39″ to “pull request 39″. Of course, you can always close the pull request from the commandline by merging

git checkout cucumber
git merge pull-test
git commit -m "merge"
git push

The pull request is then closed on Github.

  1. realizing, of course, that this is old news to many– as usual []

Heroku, UTF-8, and DATABASE_URL

Recently, I’ve been fighting with Ruby 1.9 and encoding in a Rails 3.1 app that we’re running on Heroku. The strange thing is that encoding was working fine on any local system we would try, but would break on Heroku while rendering views with the error

ActionView::Template::Error (incompatible character encoding: ASCII-8BIT and UTF-8)

There are suggestions sprinkled all around the web on how to deal with Ruby 1.9 and character encoding.1 Do a quick Google search and you see everything from “revert to Ruby 1.8 until they2 fix Ruby” to “Add ‘encoding: utf-8′ to the top of every file you’ve ever seen.” In the beginning of this all, I thought I at least understood encoding. By the end, after trying litterally everything I could find to fix the problem, I started to think it was all magic.

Here’s the deal: We’re using an RDS database plugin on Heroku, so the actual database is the same regardless of what system you’re on. Moreover, I was using the same database for production and development environments. Despite this, the app had encoding errors on Heroku and not on any local system.

We changed the LANG environment variable, the encoding settings for Rails, for Ruby. Everything. Nothing I did changed the encoding of things on Heroku. Nothing.

The MYSQL2 Gem

Many things I saw said to use the mysql2 gem. That was actually one of the first steps we took. Unfortunately, Heroku didn’t seem to support the mysql2 gem. If you had that gem anywhere in the production environment, the deployment would fail with a gem error. So, we had the mysql gem in production and mysql2 in development:

group :production do
  gem "mysql"
  ...
end
group :development do
  gem "mysql2"
  ...
end

Despite this, in our database.yml file we were using

production:
 adapter: mysql2
 encoding: utf8

This didn’t seem to cause any problems, no errors or warnings, so I figured Heroku must be honoring it. Wrong. I’d fire up the local console, grab something from the database, and see <#Encoding: UTF-8>, then I’d fire up the Heroku console and see <#Encoding: ASCII-8BIT> for the same object.

The clue came when I changed the adapter to just ‘mysql’ and then started the app locally in production mode. Suddenly, I saw the encoding as ASCII-8BIT on my local system too. Somehow, on the local system, despite ‘mysql’ two being set in the Gemfile, ‘mysql2′ was being used because of the database.yml file. However, the reverse was true on Heroku.

WTH?

The database URI forces a gem, not a protocol?

Long story short, Heroku *does* support mysql2, but they don’t actually let you know that.

When you use the RDS plugin on Heroku, you add the URI to your account info. This is done on the Heroku site using a dialog box, which contains a very helpful bit of text describing the format of the database location as:

mysql://user:pass@rdshostname.amazonaws.com/databasename

Now, when reading this, one would naturally assume that the ‘mysql://’ part of this URI is designating that you want to use MySQL to access this database. As opposed to, say ‘postgres://’ or something.

One however, would be quite wrong.

It turns out that the format is not as described, but rather:

gemname://user:pass@rdshostname.amazonaws.com/databasename

Surprised? Yeah, me too. Suddenly, I can use the ‘mysql2′ gem in production if I use a database location of:

mysql2://user:pass@rdshostname.amazonaws.com/databasename

Doc, no doc

Now, I love Heroku, so I’ll give them the benefit of the doubt here. I’m sure someone thought it was very clever to use the <schema> portion of the URI to allow for changing of the gem. And sure, it is. But come on, folks, “The format is” when describing a URI is meaningful.

Interestingly, clicking on that little question mark brings you to a documentation page where ‘mysql://’ and ‘mysql2://’ seem to be used interchangeably, with no notes as to the consequences. If you were following this documentation, you’d be instructed to migrate your data using the ‘mysql’ gem, then you’d be instructed to use the database using the ‘mysql2′ gem. This would probably cause some confusion when suddenly your deployment fails telling you that your Gemfile is broken.

Coda

Moral of the story: If you’re on Heroku, and getting encoding errors that you can’t find with an external database, look well at the URI of your database and ensure that Heroku is actually using the gem you think it is. Unfortunate as hell that a URI set on their web app would break stuff you put in your code, but there you are.

  1. The most detailed being the post by Yehuda on the topic []
  2. whoever “they” are []

Server specific banner using RMagic gem

So, I have a complicated Rails project. I have a local server, a remote dev server, a remote staging server, and a production server. Recently, I had a situation where the dev server was shunting traffic to the production server and I didn’t catch it.

What the hell?! Why aren’t my changes showing up?!

So, I made a banner that said “Dev Server.” That was pretty useful, but then I thought it would be nice to have the banner specific for the server.

So I came up with this helper method:

  def dev_banner
    filename = "#{ENV['QSTREAM_SERVER_TYPE']}_server_banner.png"
    banner_path = "public/images/#{filename}"
    server_type = ENV['QSTREAM_SERVER_TYPE']

    unless File.exists? banner_path
      img = Magick::ImageList.new("public/images/dev_banner_background.png")
      img.background_color = "none"
      gc = Magick::Draw.new

      img.annotate(gc, 0,0,15,130, "| #{server_type} Server |") do
        self.pointsize = 13
        self.rotation = -45
        self.kerning = 1
        self.stroke = "#dfdfdf"
        self.fill = "#dfdfdf"
      end

      picture = img.flatten_images
      picture.background_color = "none"
      picture.write(banner_path)
    end
    filename
  end

This method uses the RMagick gem. It takes the dev_banner_background.png file– a banner with a transparent border (see right)– and annotates it with the string “| <name> Server |” where <name> is a string set in a local environment variable, in this case QSTREAM_SERVER_TYPE.1

I have a dev_banner.css file with the following styling:

#right-corner {
    position: fixed; /* Make sure you can align it exactly */
    cursor: pointer; /* Change the cursor on mouse over */
    top: 0px; /* Change to 100px to put it under a 100px banner */
    left: 0px; /* Change to 100px to put it left of a 100px right-side bar */
    z-index: 99999; /* make sure it is the top element always */
}

and then I have an application.rb file with the following:

%head
...
  - if @server_type.present?
    = stylesheet_link_tag 'dev_banner.css'
...
%body
  - if @server_type.present?
    %a#right-corner{:href=>"#", :target=>"_blank"}
      =image_tag dev_banner

It’s a link because I originally thought of making it able to flip over and show server-specific details on a mouse click.

Because I thought I might change the banner based on other localized information, @server_type is a variable set in the ApplicationController:

  ...
  before_filter { @server_type = ENV["QSTREAM_SERVER_TYPE"]}
  ...

This allows me to set an environment variable on any server I’ve loaded, and have a banner specific to that server auto-generated– with nothing showing on the production server.

Given this, it’s easy to see how you could add information to the banner based on the local system. Just a few simple lines of code. Unless it gets a lot more complex, it seemed unnecessary to make it a gem.

  1. It’s totally possible as well to create multiple banners of different colors, and set those in environment variables (or generate multiple banners on the fly), but I didn’t need that much complexity. []

Bundle update “killed:” Watch those log files in your gems!

Today was supposed to be a normal day. A simple pull-request merge, an automated Jenkins build, and then a database migration.

What followed instead was a search for the cause of a build failure– or, rather, the cause of a ruby gem that was causing the build failure.

The Problem

The only hint I had to go on was this message in my Jenkins console:

Installing acts_permissive (0.3.2) /tmp/hudson7424891008093103684.sh: line 5: 12423 
Killed 
bundle
Build step 'Execute shell' marked build as failure

It couldn’t install the gem? Why? To the console:

jenkins@li128-183:~/workspace/musicstand
$ bundle update acts_permissive
Fetching source index for http://rubygems.org/
Using rake (0.9.2.2) 
Using RedCloth (4.2.9) 
...

NoMemoryError: failed to allocate memory
An error occured while installing acts_permissive (0.3.2), and Bundler cannot continue.
Make sure that `gem install acts_permissive -v '0.3.2'` succeeds before bundling.

What the what? Sadly, trying it manually was no help:

jenkins@li128-183:~/workspace/musicstand
$ gem install acts_permissive -v '0.3.2'
Killed

That’s it. Nothing else.

The Investigation

Luckily1 I am the writer of the Gem in question. ActsPermissive is an instance-specific permissions system based on “circles of trust” (a bit like Google+ circles).

I went to my dev folder for the gem and explored it. Nothing seemed out of the ordinary. So I created a new gemset and ran “bundle install” forcing an installation of acts_permissive. It installed fine; however, I did notice that Bundler seemed to “pause” while installing it. On a hunch, I went to look at what the size of the .gem file was.

The Solution

Embarrassing as it is, the .gem file weighed in at a whopping 23M. That’s just ridiculous for a few text files! Further investigation showed me that my development directory was a full Gig! What the what?

Log files.

Without thinking, I was including the spec/dummy rails application, which contains a log directory, and log/test.log was nearly a gigabyte in size! Consequently, that was all getting packaged up, and then this 1G gem was trying to be compressed on the server and borking because of failed memory!

Lazy, John!

I had the normal “s.test_files = Dir[“spec/**/*”] line in my gemspec, but need to exclude some files. The “test_files” attribute is an array, thus, I added the following lines to my .gemspec:

  s.test_files = Dir["spec/**/*"]
  s.test_files.delete("spec/dummy/log")
  s.test_files.delete("spec/dummy/log/development.log")
  s.test_files.delete("spec/dummy/log/test.log")

This dropped the .gem file back to a few kilobytes, which is all it should ever be.

Coda

It took a bit of detective work, but the “bundle update ‘killed'” problem was a crisis averted. This would’ve been grueling to troubleshoot if I didn’t own the acts_permissive gem. However, in the interest of helping others, it can be pretty easy to at least rule this problem out if your ‘gem install’ or ‘bundle update’ fails mysteriously. Grab the .gem file from RubyGems and check the size. If it’s crazy huge like acts_permissive was, then you might just be filling up RAM when it’s extracted.

  1. or, unluckily, as the case may be []

Trolling, Sexism, and Dealing with Disappointment

This past week has been one is extreme disappointment for me as I survey the landscape of my beloved profession. From Portland’s own Postgres genius Selena Deckelman getting hazed on IRC to the incredibly immature sexism of Sqoot listing women as “a feature” of a hackathon I’ve just been feeling really depressed that there is this scale of stupidity toward our fellow programmers– our fellow people– in a field where, at one time, we’ve all known what it’s like to feel out of place and unaccepted.

Sqoot is just a light on what’s seemingly rampant in a field where I just assume there are awesome women doing awesome things. How much of this stupidity am I unaware of? How much do I add to it. Does it really make us feel better to treat other people like shit?

Sexism At Hydrasi

Kerridra and SI1138, Hydrasi's lovable mascots

It strikes me as somewhat odd that I am a man, in an really male dominated field, in a male dominated world, who’d write about this. I mean, what do I know? Just as with white people who don’t see things that every person of color takes for granted, I’m sure there are a great many things than I just blindly ignore in my happy placement of gender-based privilege. In fact, I know there are.

Recently, we were having some graphic design work done for Hydrasi.1 Hydrasi has a lot of inside jokes based on our name2 and our mascots, one of which is a Sasquatch and one is a cute, blue robot.3

The short story is that there are tiers of service, and for the top “enterprise” tier we were trying to make yet another cultural reference joke that could have gone bad.

For every tier, we have a different image. The free tier shows our Sasquatch, Kerridra, holding a pitchfork, the next tier– the lowest paid tier– shows Kerridra with his robotic companion SI1138. The idea, of course, is that you get more robots, more power, with higher subscription rates.

Attempt at referencing Madmen

At the top tier, the enterprise tier, I thought it would be fun to have a reference to the TV show Madmen, where rich advertising executives have martini parties and red-headed secretaries wear mini skirts. My thought here was not “focus on red-headed chics in mini-skirts” but rather on “At the enterprise level, we take care of everything for you so you can sit back and drink a martini.” Here’s the first rough cut of that design.

The final version. You get a computing rack, NOT a secretary

Again, the idea was to have something immediately identifiable as from the show Madmen, but we looked at it and felt that it came off as sexist, pointless, and sort of missing the point of what it means to be at the upper level of service. In the end we axed my Madmen idea and went with another reference, this one not to a cultural reference, but to service. We put a rack of servers behind Kerridra and decided to say “You are getting people, robots, and servers” instead of “You can get a hot red-head.”

Mindless Sexism

There’s a large part of me that’s embarrassed at the whole idea. I mean, I was just trying to go for a funny pop-culture reference in the same way as we have Star Wars references and WALL-E references. In the end, we recognized it as a misplaced, and overly sexist image, and decided to pull it before it ever went anywhere, but how many things like this don’t get pulled?

I guess the take away is that we did sit down, and make a conscious decision to axe that train of thought because if it’s overtones. We thought about it, we decided. That’s sort of what worries me. How much do we– all of us– throw out there because we don’t think about it. How much is just simply cultural privilege that we swim in like water?

Seeking The Positive

This week kind of got me down, both because of the shitty nature of the way things are, and because of my worry (as my wife and I watch her belly grow and I, at least, hope for two daughters) that I add to it as much as anyone.

I guess one thing to look toward positively is a responses to everything this week. Responses like Selena’s. She chose not to battle, but to, almost quietly, Storify the interaction and then ask for feedback (http://bit.ly/GW1XT7). This was an incredibly mature and awesome way to handle a troll (Cheers to you, Selena, for so much restraint and thoughtfulness). And there was a lot of outrage about Sqoot’s mindless objectification of women, including posts like this by Tom Morris. There’s a lot of outrage and a lot of people, not only women, saying “Why is it this way? It doesn’t have to be, and you need to stop perpetuating it!” That’s positive.

For what it’s worth, Sqoot apologized with a “we can do better” post. This made me feel worse instead of better because it just seemed way too late and seemed to radiate insincerity. The only thing I could think of while reading that was an image of Steve Carrell from The Office, spouting an apology speech which had all of the right words that he knew other people would want him to say.

Coda

Overall, it’s been a disappointing week in Tech. I’m trying to see the positive. Good, strong, well-reasoned responses to stupidity and sexism. That should make me feel good. But the fact that there’s such a culture of “brogramming” makes me feel sad. I hope we can change that, but I’d really expected it to have been changed by now, so am not sure how much faith I have in me and my fellow man.

  1. These images are secret, don’t tell anyone you saw them! []
  2. Oh no, it broke! It’s Brokedrasi! []
  3. The Star Wars connection here is obvious, but there are others we play with as well []