wuputah ~ Jonathan Dance

Stop punching the duck

2010-10-27

So there you were, minding your own business, doing some behavior driven development, thinking everything is great. You test your controllers, you spec your models, and now the final, dreadful step: actually firing up the browser. Expecting to see some terrible CSS disasters, you get a nice error screen. “That’s weird,” you say, “my cucumber tests say this should work!”

Alas, enter the duck punch. In your testing environment, rspec and cucumber are loaded. And that means so is this little duck punch. Seems like you used errors_on instead of using errors.on. This worked just fine in your tests but now that rspec is not loaded, the feature explodes.

And there’s no test you can write to check for this.

Your philosophical view of the world broken, you give up programming and move to a cave, never to be seen by civilization again.

So please, before you punch a duck, think of the caves. There’s only so many left.

Early bird dilemma

2010-10-19

One of the (many) problems I have with society today is that the early bird is rewarded, while the “late bird” is frowned upon. Meanwhile, $300 million is lost because of this every weekday.

A quick example is commuting. Traffic here in Boston starts to back up as early as 6 AM, and actually flows more regularly at 8:30 or 9am. As each person tries to get up earlier and earlier to beat traffic and get into work early, the problem has cascaded to the point where it makes more sense to go in later, not earlier.

However, people who try to take advantage of this are looked upon as lazy, as showing up to work “late” is frowned upon. 8am, 9am, or 10am meetings cause problems with this approach. If we assume the “normal” time for starting work is 9am (although more commonly, due to this phenomenon, most people arrive around 8am). It’s fine to show up to work 3 hours early (6am)—in fact, this is usually considered a sign of being hard working or responsible—but show up an hour late (10am) and you are likely to be looked down upon. You must have been “up late partying.”

Instead of encouraging everyone to show up at the same time, spreading out the work day should be encouraged. It reduces congestion, reducing traffic on the roads and mass transit systems. It encourages flexibility, allowing people to fit work into their lives how they see fit. Some people are morning people; some are not. Some people have commitments, and need to be home at a certain hour.

Even among my programmer cohorts, who are among those who have realized the folly of this system and often enjoy the flexibility of working on their schedule, some believe that this is detrimental to the work environment, productivity, team dynamics, or even personal health. Even if your organization allows a dynamic schedule, you will probably be partially judged by superiors by the time of your arrival. Debunking these myths is near impossible, as we have been risen in a society where “the early bird catches the worm.” Showing up for work on time is frequently more important than results, even if you would get better results from being well rested.

In short, the workplace should be a meritocracy. Nothing else should matter. This is never the case.

As I mentioned, this is one of many examples. Yet another early bird dilemma is the queue, which naturally favors the one who arrived first, and therefore waited the longest. Now in many cases, I have no problem with this; certainly waiting in line at the grocery store makes sense. However, when it comes to things like product releases or ticket sales, this system is fundamentally flawed. Encouraging people to wait in a queue for hours, sometimes days or even weeks, is not healthy nor productive. Certainly a lottery would be a better approach to this system.

As someone who both frequently rises early (I wrote this entry before leaving for work) and stays up all night, I fear I’ll be fighting the early bird dilemma for my entire life.

A Note to Recruiters

2010-08-07

If you’re a recruiter (that is, anyone looking to hire me), please read this before contacting me, or you risk losing a potential lead forever.

As you are probably aware, software engineers are in fairly high demand. Those with experience in Ruby on Rails are even more so. Demand is easily outstripping supply, and that means I get contacted a lot, even though I am not actively looking for a new position.

So, before sending me an email or thinking about dialing those enticing 10 digits on my home page, please note the following:

I am gainfully and happily employed. This means that your position needs to be very enticing just to consider the position. In order to do this, you must be able to provide me:
- The name of the company I would be working for.
- What I would specifically be working on.
- What it’s like to work at the company, in terms of work expectations, office environment, and the corporate culture.
- What I might expect for compensation and benefits.
- All the standard information you would include in a job posting. Ryan Bigg has a longer list on what should be in a job posting.
If you’re unable to provide this information by email, whether it’s due to contractual obligations, because your business model depends on withholding information (or you don’t trust me), or because you haven’t done your homework and don’t know, then you should not bother contacting me. You may think that phone calls are a “better way” to share information, but I can’t have phone calls every other day with recruiters when I’m not even looking for a job.
Do not contact me by email and then “follow up” with a phone call unless I have specifically requested to speak with you. Unsolicited email followed by a unsolicited phone call is likely to get you blocked. I will respond to your email if you have provided the above information and if I’m interested. If you’re thorough and polite, I will usually respond either way within a few days.
Don’t make me do your work for you. You need to sell me on the job. Know your recruit. If you want to know more about my professional experience, check out my LinkedIn profile.

All that being said, I don’t want this to be taken the wrong way. I’m happy to be contacted, but you’re not doing me—or yourself—any favors if you can’t tell me what the job is about.

Bluepill, chef, and resque

2010-07-02

Bluepill is an alternative for process monitoring, competing in the same space as heavyweight monit and hubristic god. Like god, bluepill itself and its recipes are written in pure Ruby. Bluepill’s goal is to be very lightweight, and while it weighs in at greater than 1400 LOC, it’s much smaller than both monit and god, and easily wins over god on memory usage:

At WegoWise, we began to investigate using bluepill on recommendation from EngineYard as bluepill would be an independent system apart from EY’s own use of monit. We were previously using monit to monitor and control our resque processes; bluepill would replace this role.

The Good

There are some very nice things about bluepill. Bluepill removes the need to create init.d-style scripts to control your processes. Bluepill will handle creating PID files, daemonizing, and sending a standard TERM signal to end the process (as well as number of other handy things). For our resque processes, the only truly necessary configuration is a start command.

If that’s the case, then why is our resulting chef recipe and bluepill template (which we’ve released to the public) so complicated? Well, most of the complication is queue logic, but there’s some other tricks in there that are worth explaining.

The Bad

Overall, our experience with bluepill has been very positive, but it is not without a few drawbacks. Bluepill doesn’t do graceful shutdowns, nor will it ever issue a KILL signal if the process becomes unresponsive to TERM. To solve this, we included a QUIT→TERM→KILL signal chain in our bluepill configuration.

Another problem is there’s no way to set up alerts should something go terribly wrong (we’ve yet to solve this problem). For instance, bluepill allows you to monitor flapping (rapid restarting), but bluepill will simply unmonitor the process when this occurs, and you will be none the wiser. This also occurs is if the process doesn’t start, stop, or restart within the allotted grace time. Therefore, a shutdown script like the one we used is a real necessity.

For now, we’ve solved our most critical issue directly in our pill (each bluepill configuration file is called a pill), but we might look into adding signal chains and alerts to bluepill itself. Fortunately, resque is quite stable and so alerts—even using a TERM signal—has yet to be necessary, and bluepill has been rock solid as well.

Moving attachment_fu from filesystem to S3 storage

2010-06-27

Most of this transition is fairly easy, but many users will find a small hiccup when it comes time to upload your files to S3.

Create a config/amazon_s3.yml file with your credentials.
Setup your S3 bucket, upload the files (I use S3Fox). Make sure to set the access to world-read for your uploaded files.
Add the aws-s3 gem to your environment as well as your hosting platform or server.

Ah, but there is a small problem. How do you get from attachment_fu’s default of 0010/5198/filename.png to 105198/filename.png? The solution is a small Bash script. This is designed to be run from within the top-level folder for each of your attachment_fu directories, like public/pictures, public/avatars, etc.

I’d recommend running this on a copy of your files, just in case I messed up, but the most important line is let "d=$d1$d2". This wonderful line combines the 0010 and 5198 and treats them as a number, for a result of 105198. I don’t know much about let, but it seems to work wonderfully when you need to treat strings as integers.

PS. Don’t forget step 3.

Stardates in Toto

2010-06-06

I was messing around with the date and instead of going with the 6th day of the 6th month of the year 2010; a glorious Sunday!, I decided to implement stardates. Don’t worry, you can still mouseover the stardate to get the Gregorian date, but I figured, who pays attention to the date anyway? Well, it might catch your attention now.

For the first two digits, I use a somewhat unique approach amongst trekkies: instead of using years since 1900, I drop the leading digit just like we do with our centuries. Thus, instead of today being 110430.1, it’s simply 10430.1; this fits with the 5-digit years used in the series. Of course, my stardates won’t align with those in the show.

The rest is the standard fractional year: there are 1000 stardates a year, so take the current day and divide it by the total number of days and multiply by 1000. Round to the nearest tenth. There’s 8.76 hours per star-day, and 52 minutes per 0.1 star-day. (Granted, toto doesn’t use time, just the date, so I’m using too many significant digits.)

As an aside, you might be interested to know this tidbit about the Gregorian calendar:

[The Gregorian calendar] was introduced by Pope Gregory XIII, after whom the calendar was named, by a decree signed on 24 February 1582. The need for the Gregorian reform stemmed from the fact that the Julian calendar system assumes time between vernal equinoxes is 365.25 days, when in fact it is about 11 minutes less. The accumulated error between these values was about 10 days when the reform was made, resulting in the equinox occurring on March 11 and moving steadily earlier in the calendar. Since the equinox was tied to the celebration of Easter, the reform in the calendar was undertaken by the Roman Catholic Church.^[1]

This was certainly an improvement, but now that we have standardized time, we also need leap seconds.

How to pronounce "wuputah"

2010-06-06

Here’s the problem with posting this in text: ˈwoŏpətä. You all know IPA, right? Me neither. Let me try to break it down:

The first syllable, woŏp, is like whoops, but it shouldn’t rhyme with hoops.
The second syllable, ə, is a very common vowel sound in English, and is pronounced like uhh as in cup. I also sometime pronounce this syllable as ɛ, which is closer to ehh, as in bed.
The third syllable, tä, is pronounced as in the first syllable in ta-ta!

Emphasis is on the first syllable, so it’s WHOOP-uh-ta.

You’re doing it wrong if:

You pronounce utah, as in the state.
The first syllable doesn’t include the p, and hence sounds like woo-poo-tah.

count, size, and length: associations vs named scopes

2010-05-24

I was looking into this today and it was a bit different—and much more complex—than what I expected, so I figured I’d pass on the results. On an association:

length forces the association to be loaded if it is not already, then checks the size of the array.
count always invokes a COUNT() query, regardless of the state of the association (loaded or not). It does not take unsaved records into account, and does not respect active scopes (which is why named scope does not use this). The count is never cached by Rails, but would be cached by the Rails query cache.
size’s main difference is that it includes unsaved records in its result, but it can only do so if knows about them. Associations only know about records that are queued for saving when the parent object is saved, and it can only do so if called with association.<< or association.build. (These are the same cases as when a counter_cache column is automatically updated; caveat emptor.) size can be more efficient than length in some cases – for has_many and has_many/has_one through associations, it takes advantage of counter_cache columns or invokes a COUNT() query. There is no optimization for HABTM associations.

This gets a bit trickier when you add named scopes into the mix: the results change because named scope has taken over some of the methods. To understand this, you must remember that calling a named scope creates a new Scope object which wraps the upstream object. In addition to implementing the required querying magic, it provides a separate layer of caching entirely separate from the upstream object.

length is not implemented by named scope, and so falls back to either the association’s implementation, which will work because the eventual find(:all) will be scoped, or Array#length if there is no association defined.
count is not implemented by named scope, but does not send this method to the association’s implementation, as the result would be incorrect. It always invokes an SQL count by using with_scope() and invoking the original count method on the class (defined by ActiveRecord::Calculations) with the appropriate parameters. This is probably the most confusing of the 6 cases.
size is implemented by named scopes, and either returns the length of the loaded array, or invokes an SQL count if it hasn’t been loaded in all cases. The association behavior of handling unsaved records is gone as well as any association-specific optimizations (or lack thereof).

Keep in mind that both object associations and named scopes can only cache objects internally if they’re the same instance (i.e. saved to a variable), so two otherwise identical named scopes/association calls will only be cached at the query cache level (whether by Rails or the database). self often serves as this instance for an association when writing model code, but once you use a named scope, these are much more rarely cached, so count is often the best choice if a named scope is involved. Otherwise, size or length may be faster, depending on the scenario, but it still requires a full association load before it can cache the size. However, if you’re using counter_cache, you must use size to take advantage of them (unless you read the counter attribute directly).

Lastly, while count is a good choice, you shouldn’t get in a habit of using it on everything. Array#count was added in ruby 1.8.7 (and later). String#count exists, but it takes a parameter and returns the count of the number of occurrences of characters ('confused yet?'.count('ce') returns 3).

Announcing a DNS tool without the bullshit

2010-02-20

I’ve created a simple DNS tool that allows you to easily link DNS results to share with others. Using a third-party tool is a great way to show that the DNS settings are not being affected by your local network, but most third-party tools are riddled with advertising and unclear tool names, with some even charging a membership to access some of their tools. My tool uses (and displays in the result) the command line used to generate the results instead. While not everyone will be familiar with these different commands, users can still experiment to learn and discover more about these valuable tools.

The tool is available at dns.heroku.com. The site is written using the Sinatra framework and the code is open source.

ERD diagrams from Sequel Pro

2010-01-07

If you need a diagram of your MySQL database and you’re on a Mac, generating an ERD diagram is quite easy – and completely free. Sequel Pro can export Graphviz dot files, and then all you need is a few tools to create the diagram.

Install graphviz from MacPorts via your Terminal:
sudo port install graphviz
Install Sequel Pro, run the app, connect to your MySQL server and open the database you’d like to diagram.
Go to File > Export > Graphviz Dot file, and save the file somewhere convenient.
Generate an SVG file of your diagram:
dot -Tsvg your_database.dot > your_database.svg
You can open SVG files with Opera, Safari, Illustrator, etc, but you can generate a PNG file in a number of ways. You can try installing ImageMagick or libsrvg from MacPorts, or use Illustrator or Inkscape to open and convert the file. ImageMagick: convert your_database.svg your_database.png libsrvg: cat your_database.svg | rsvg-convert -o your_database.png

The result is a basic table-based ERD, but it’s not bad for a few minutes of your time.

Thanks to the geert README for the introduction to this process.