Wednesday, October 31, 2007

How To: Manage Your Own Subversion Repository In Leopard

Mac OS X 10.5 Leopard ships with Subversion 1.4.4 pre-installed. It also ships with Apache2 pre-installed. It does not, however, ship with a pre-installed subversion repository configuration.

So let's say you want to create your own subversion repository host on your Leopard box your own source code management goodness?

You could go to the subversion homepage and download the free svn book and sort through the instructions trying to figure out how they apply to you... Or you could follow these simple directions which I've laid out for you.

Make a Repository

The first thing you need to do is to make a repository. Actually, for my needs, I had to make multiple repositories, so these instructions will set everything up to make that work. It really only changes two steps anyway, so it isn't a big deal.

Now I decided to make my repository collection root directory be in /Users/Shared/, but you can really make it be anything you want, including the ever popular /usr/local. Just be sure to replace /Users/Shared/ with your directory of choice whenever necessary.

Anyway, I opened Terminal and entered the following commands:

$ sudo mkdir /Users/Shared/svn
$ sudo mkdir /Users/Shared/svn/reposname
$ sudo svnadmin create /Users/Shared/svn/reposname
$ sudo chown -R www:www /Users/Shared/svn/reposname

Note that you can create multiple repositories by following these directions but replacing every instance of reposname with the name of the repository you want to use. Thus, if you have multiple repositories, you will have multiple directoris in /Users/Shared/svn

Make Access

Most directions do this later, but I'm going to do it now because I think you are smarter than that.

You might want to create a passowrd file, unless you want full public access to your repository. For our purposes, simple http basic authentication is fine, but remember that the password is only weakly encoded and the traffic isn't encoded at all, so a snooper could get to the information if you access your computer outside of your own computer.

So if you do want to use authentication, create the password using the following command, substituting username for a user name of your choice, and following the directions for password creation:

$ sudo htpasswd -cm /etc/apache2/svn-auth-file username

To add other users to the file, just ditch the c switch in the -cm options to htpasswd. The c stood for create, and since the file has been created you don't want it anymore.

Note that you can put the svn-auth-file anywhere you want, but this seemed like a good place for it in my mind. (Just remember where you hid it from yourself if you put it anywhere else.

Apache Configuration

Navigate to /etc/apache2/other and use your favorite command line text editor as root to make a file named anything you want (I chose svn.conf, but you could name it foobar_banana.conf and it would still work!):

$ cd /etc/apache2/other
$ sudo vim svn.conf

Now that you are editing this file as root, you want to make it contain the following bits, and save:

LoadModule dav_svn_module /usr/libexec/apache2/

<Location /svn>
    DAV svn
    SVNParentPath /Users/Shared/svn
    AuthType Basic
    AuthName "Subversion repository"
    AuthUserFile /etc/apache2/svn-auth-file
    Require valid-user

Note that you can leave off all the authentication related stuff if you didn't want authentication on your repository. Also note that you need to fix the SVNParentPath and the AuthUserFile if you varied from my directions.

Restart Apache

Now restart Apache. This can be done in the Sharing panel of the System Preferences application. Just click to turn off, and then back on, Web Sharing.

Now, if you didn't make a mistake, you should be ready! Try going to http://localhost/svn/reposname (where you need to put the repository name you chose earlier instead of reposname!) and see what happens.

If you are lucky you'll see revision 0 of your repository. But most people are human and will have made a typo that results in an error. For hints on what created the error, trying checking out /var/log/system.log and /var/log/apache2/error_log for hints as to what you did wrong. (And as a bonus, the Console application works great for monitoring thes logs as they are writen to!)

Where From Here?

And now you are ready to use your repository. At this point I figure you already know how to use SVN and don't need my help anymore. But if you need to know how to use SVN, just refer to their book, which tells you everything you need to know, including how to make your server better!

EDIT (11/6/07): Forgot to encode my character entities in the example script source. I fixed this so that the <Location> tag actually shows now.

EDIT (04/23/08): The chown line now uses the full path as it should. Thanks to all those who pointed this confusion inducing mistake out.

Sunday, October 28, 2007

Tip: Leopard and Legacy Rails

Due to the size and requirements of our enterprise healthcare application that we've written on Rails, we've been sort of stuck at rails 1.1.6 and ruby 1.8.5. Not that we don't want to upgrade, but in trying to build on functionality so we can make enough sales to sustain ourselves, we've had to forgo tracking down all the problems that keep us from going to rails 1.2.3 and ruby 1.8.6

As a good Mac geek, I upgraded my computer to Mac OS X 10.5 Leopard this weekend, deciding to use the archive & install feature because I had some issues upgrading from 10.3 to 10.4. (my compilation of ruby 1.8.4 would crash randomly). Of course, Leopard, being all cool and stuff, ships with rails 1.2.3 and ruby 1.8.6.

Fortunately, compiling ruby 1.8.5 was straight forward (just don't use the --enable-pthread option or you'll get a crash!), and otherwise the instructions to install gems, postgres, and RMagick all worked exactly the same as on Leopard.

And since Apple manages the ruby/rails installation that ships with Leopard via a Ruby framework and some symbolic links in /usr, it means that if you compile and install everything into /usr/local, you can easily revert back to Leopard's ruby/rails whenever you want.

So the net result: If you have a legacy app, just compile and install and you are good to go.

Speaking of, I really should make a set of blog articles that goes over the compile/install procedures for ruby, rails, postgres, and RMagick, on Leopard. But right now I'm having too much fun playing with Time Machine to care. :)

Edit (10/28/07): As noted in this article, rails 1.8.5 p114 has a bug in it. However it appears that p52 works just fine.

Tuesday, October 23, 2007

Video: Keep It Simple Stupid!

Being that I'm a simple-is-sexy kind of person, I've always been flabbergasted by the busy packaging so many corporate marketing teams put out.

As a developer, I've learned that the KISS principle (Keep It Simple Stupid) is very important for solving problems. And as a teacher I've experienced the confusion an onslaught of information can provide. Indeed, in most situations, whether it being a scientist, and engineer, a teacher, or even a parent, keeping it simple is important.

Which is why I loved the following video. It does such a good job of punctuating the differences between Apple's approach of simplicity, against the corporate world's attempts to attract attention while cramming the package with information.

Of course the ironic thing is that, with all these super-busy boxes on the shelves, the one that actually stands-out is the simple-sexy one.

Rant: Rails Lacks Accessor Bottlenecks

Being a longtime user of (and zealot for) Ruby, I have a made a name around the office for my unusually strong understanding of its details. Thus my employers decided that, out of our team of developers, I should be the one to write the security sub-layer for our Ruby on Rails based chiropractic applications (as it would require digging into, and understanding the internals of Rails.

Now before I proceed to beat Rails around a littl bit, I don't want you to get me wrong. I love using Ruby on Rails and would easily chooose to do this project (and future projects) in it again, but sometimes I find some of their design decisions to be, well, web-developerish, while we need a more robust enterprise-developerish solution.

A Bottle With Too Many Openings Can't Hold Water

One of the design patterns that I have seen over and over again in Object Oriented APIs is the use of bottle-necking. That is to say, even within your own class, you choose to use a set of accessors to get at instance variables instead of poking at them directly.

The big benefits to this approach are two fold: It allows you to be more agile with changes (as you only have edit the accessors to change behavior), and it allows third part developers to easily augment or override the default behavior of your code.

An simple example of this may be a simple vector class that stores a magnitude and an angle. Now this angle, in all reality, needs to be between 0 and 360 degrees. So if you have two ways to set this data (from rectangular or polar coordinates), you would have to enforce these limits, probably by using a modulus operator. (Which isn't very DRY, now is it?) But what happens when you have to change the behavior? Maybe the angle needs to be stored in radians, maybe the range needs to be changed to -90 to 90, or maybe it needs to be compass oriented instead of right-handed-axis oriented? Or maybe you just want to add a way to set the angle without changing the magnitude?

All of these requires refactoring your code, which requires duplicating and/or altering your code in several locations! But if you were to make a single accessor for getting, and a single accessor for setting the internal value, then all your other code could go through this accessor (including possible intermediate accessors) to enable fast, agile, and DRY code changes!

Playing With Steam Engines

So, now, let's imagine your employer gives you the spec that your enterprise Rails driven software must have a security system where you can restrict read and write ability to different models, on a per attribute basis. This is what I faced over a year ago.

The way I really wanted to solve this was to override the accessors to the basic hash that stores, as key/value pairs, all the data for the table row the objects represents. That way I could add my own code that would allow or deny access at this fundamental level.

Unfortunately this basic and simple idea (based on past experience with other APIs) exploded in my face. You see, the @attributes hash instance variable that ActiveRecord uses to store these key/value pairs is directly poked and prodded by many separate methods throughout the ActiveRecord class. Thus I had to override all of these methods, making them call the same friggin security check method, to decide if they could continue onto their default implementation or if I should restrict access. (I'm only lucky that the runtime generated accessors utilized standard accessors, or else I'd be re-writing the code-generation routines as well.)

Of course, all this work meant that besides sinking a lot of the company's time and money into implementing a task that should have been simple, maintaining our release against newer versions of Rails is more costly because of all the hacks I've had to do to meet the requirements.

Which is why I'm so peeved at their code. If only they had followed good design practices and bottle-necked all calls to @attributes through a read and write accessor method, it would have been easy to implement customized behavior at this point in the program flow, and thus would have made my security requirements a trivial task.

Sunday, October 21, 2007

Rant: Marching Forward; Why Should I Upgrade?

With the oncoming release of Mac OS X 10.5 Leopard, I feel compelled to rant about a post I saw on the discussion board for the open-source software AudioSlicer.

On this forum there was a post that seemed like the author was taking out his frustrations on the AudioSlicer developer that software was not backwards compatible with his old version of Mac OS X.

Now, if I picture myself as the average user, ignorant of anything to do with software development, I can see where the guy is coming from. However, as a developer, I find his attitude to be quite maddening, particularly because AudioSlicer one man's solution to a problem he was trying to solve for himself, and was nice enough to share it with anyone who wanted to download it!

Building a Home Of Code

Building software is kind of like building a home. Each operating system gives you a set of tools and building blocks to build your "home", and then you figure out how to stack them with building blocks of your own and glue them all together to make an application for people to enjoy.

In the old days, there were no power tools and, depending on the place and time you lived, you might even have to your own plaster and maybe even cut down your own trees. Thus, most of your time, energy, and money, goes just into building things for construction, much-less building a complex and interesting house.

In modern times, however, you are lucky enough to be given power tools, pre-cut lumber, drywall, and fiberglass insulation, allowing you to spend more time on building the house the way you want it, and less time preparing to build.

Now what happens if someone wants you to build that nice modern home but with old fashioned techniques and materials? You'd have to spend inordinate amounts of time re-inventing technologies such as drywall and fiberglass fabrication, and spend large amount of time overcoming the lack of power tools and even back-hoes!

No contractor would dare to take on this task because it is just not economically feasible for anyone to build a modern house using old-time building techniques.

The same is true for software developers. We are given many building blocks and tools from our operating system manufacturer that smart people have spent months or even years developing and testing, and then we take advantage of their hard work so that we can focus on building better software applications for end users.

Jaguars, Panthers, and Tigers, Oh My!

Most users were very unimpressed by OS X 10.4 Tiger's release, and I can agree with them. From a user's perspective, it was a lackluster release. It was hard for the average person to justify spending $130 to upgrade from their comfortable and productive little OS X 10.3 Panther machines.

But what these people didn't realize was the massive amount of really cool new building blocks Apple gave developers. These new building blocks (called APIs if you want to learn a new word), got developers very excited as it allowed them to make much better software for a lot less time and cost.

As a result, several months after Tiger was released, a flood of cool software came out that could only run on OS X 10.4 Tiger. This, of course, caused all the people who wanted to stick with OS X 10.3 Panther to complain that no one was releasing their cool new offerings for them.

The Amish Can't Have Cell Phones

As much as many people hate it, technology marches forward at a breakneck pace. What was sate-of-the-art in computing four years ago is slow and a useless novelty now. It may be cheaper not to upgrade, but to stay in the past is to be left behind.

If you refuse to buy pre-manufactured building materials and use power tools to build your house, you can't expect the same quality of house for the same costs and built in the same small amount of time.

So why do you expect anything different from software developers?


Development is a very long, complicated, and expensive process. Most software projects--even while taking advantage of all the tools and building blocks they have available to them--still run significantly overtime and over-budget. So how do you expect developers to find the extra time and money to spend the months (or even years) necessary to re-invent building blocks from scratch, just so that you can live in the past?

Indeed, in reality, many new pieces of software that require new building blocks were only ever made because the new building blocks exist. CoreData and CoreImage allow developers to build database-driven libraries of scanned images, and Cocoa Bindings allow developers to build whole CoreData driven GUIs without touching a hardly any code. Without these nifty new technologies, cool applications like Delicious Library would be such a chore to make that only companies with large development and marketing budgets would be able to build them.

Change Course Into The Wind

The world is always changing, and technology is no different. People are scared of change. But those who do not cope with change get left behind while those that embrace change become successful. As much as it can be a large time, energy, and monetary investment to change, it is worth it to yourself and your family to learn and adapt...

...or don't complain when you get left in the dust.

Thursday, October 18, 2007

Rant: Fixing Bugs is Like Solving a Crime

So I got a bug report the other day, from our own staff member no-less, that simply said: "Can not 'submit fixes' in finalize screen."

Now let's think about this for a minute. Fixing bugs is very much like solving a crime. You are presented with a bunch of clues about what went wrong, and now you have to follow those clues down different avenues to piece together what went wrong (which is a very time consuming process to do right). But you have to do this, because you can only fix the bug when you understand why it exists.

So, now, if you were a police detective and someone called with the brief report "I found a dead body", would you be able to catch the murderer? You don't even know where the body was found to be able to start collecting crime scene data with your trained eyes. It would, thus, be an impossible case to solve with just the information provided from your well intentioned, possibly anonymous citizen.

So, then, how does the submitter of such a brief bug report expect me to solve their bug?

The Things You Need To Provide

For a bug to be successfully fixed, the developer has a few things the he needs to know.

Although it is very important to give your computer's specifications and the software version that threw the bug, it turns out that it is actually more important for you to supply the following information:

  1. What you were trying to do,
  2. What you were doing before the bug occurred,
  3. What you expected to happen when the bug occurred,
  4. What you expected to see when the bug occurred, and
  5. What you saw when the bug occurred,

Of course, just like a murder investigation, the more information you can take the time to give, the more likely a developer can fix your problem.

Remember that a developer has to be able to re-create the bug you saw in order to fix it, so if you don't give them enough information to make the same thing happen on their own computer, they will possibly just jump to the next bug in their bug list!

And Why You Should Give It To Them

Of course, the main reason why you should give all this information to them is because you want the bug to be fixed! But you should also realize that giving all of this information also saves both you and the developer time (and money in most instances).

For one, it usually takes you about the same amount of time to describe a bug up-front or after asked for details later, so being so brief doesn't actually save you any time in the long run.

By not giving the information up front, however, the developer loses quite a bit of time due to a bunch of small reasons, such as lost time in bug list triage, inability to group and collectively solve related bugs, and loss of time in having to contact users for details.

And, I'm not going to lie; commercial companies are driven by money. So if it is going to cost them money to get the details on a minor bug, they are likely to just bump it to the bottom of the list and go on to more important things. It sucks, it may even seem wrong, but I guarantee you every software company does it (and that you would do it too).

Last Words

If there is one last thing I want to impress upon you is that, despite all my harsh words for people that don't give details, having a bad bug report is still better than no bug report. We can't fix a problem we don't know about, and appreciate any information we can get. Just remember that it can be frustrating for everyone if you don't spend the extra two minutes of your life to give us some details.

Wednesday, October 17, 2007

Article: Egos Stop Innovation; How to Have a Discussion

Any academic professional, from a software architect to a physicists, is at their peak innovative performance when they can effectively communicate, discuss, and refine their ideas with others.

Unfortunately, it seems that a large number of people are more concerned with their own egos than with innovation, as evidenced by their inability to communicate with others. It seem that these people are always irrationally attacking ideas that are not their own while taking an emotional bias towards ideas that are their own.

This is a natural thing for people to do. It is in our blood. We evolved from the genes of the top-dog alpha-males and their mating successes.

But today should be different. The human race is now capable of attaining much greater heights if we work with others instead of against them.

Take Quantum Mechanics, for example. Quantum Mechanics was not the invention of a single mind quietly working away. No. It is the hard won innovation that resulted from many great minds working together to solve a common goal.

So how can we keep from being the jerk down the hall that no one wants to work with, and help to further the innovations of yourself and your company, making your managers happy and helping you to attain popularity, love, and wealth?

I Like Friends

Let's try to learn by example.

I had two friends that, through countless discussions and debates, showed me most of what I know about a successful exchange of ideas.

One of these friends was infuriating clam and methodical in his approach, but his goal was always to lead to a common understanding of the truths behind the material we discussed.

The other of these friends was irrational and stubborn, always hanging onto his idea no matter how well it could be proved false, and then would stomp off in a hissy fit whenever he was defeated.

What I learned from all this is contained in the ten rules below; but before I go there I wanted to follow my own rules and define two terms. These definitions aren't the dictionary definitions of these terms, but as long as you can understand my definition then you can follow what it is I'm trying to say.

Defining Talking

The way I see it, there are essentially two ways to exchange and refine ideas with others: discussion and debate.

I define "discussion" to be the friendly and logical open exchange of ideas, where the goal of everyone involved is to reach a new, common understanding of the material, knowing that this will most likely be different than any of the ideas brought to the table by anyone there.

On the otherhand, I define "debate" to be what happens when a discussion breaks down into egos and arguing, caused by even just one person to not want to budge from their flawed arguments, leading to an overall breakdown of the process of innovation.

The Ten Commandments

These rules take practice and hard work to follow, but following them is important not just to others, but to yourself as well.

One last thing before I start, though: I should note that rules 1-3 are mostly concerned with how to hold yourself, rules 4-7 are about communication, rules 8 and 9 are about arriving at a conclusion, and rule 10 is stating an obvious fact that people seem to forget about in the heat of a debate.

So without further ado, on with the show!

1.Be civil; always treat other people with respect and dignity.

People will only take you seriously if you treat them like an intelligent human being. If you let your frustration take over, you run the risk of insulting another person, causing them to close themselves from your point of view, destroying the whole process.

2. Place your ego aside; readily admit when you are wrong.

I doubt the knowledge you bring to a discussion is without flaws, inaccuracies, and other mistakes. Therefore you need to know and admit the limits of your knowledge. Admitting when you are wrong is probably the biggest and hardest step for people, but being ready to admit when your idea just isn't right is an important part of innovation. Put another way: don't let yourself look like an idiot by defending a lame-duck idea to the bitter end. People just won't ask for your input anymore because no one likes a self-centered, stubborn donkey!

3. Be open to new and different ideas; put yourself in the shoes of others.

Great thinkers are able to view things from many points of view other than their own. You do want to be like a great thinker, right? It is important, then, that you put yourself into the shoes of people presenting alternative (and usually contradictory) ideas and try hard to understand why they support that idea. This can help you either rebut their idea, accept their idea, or realize that there is no way to agree.

4. Make sure everyone agrees as to what the question really is.

As stupid as it seems, I have seen (and been in) many discussions or debates where each person thought a different question was trying to be answered! This, of course, causes much frustration. If it seems like the other person isn't understanding, try rephrasing what it is you are trying to find out, and see if they agree that that is the question at hand.

5. Define terms; be vigilant of disagreements caused by different definitions.

One of the funny things that often happens is that communication break downs can be the cause of many long discussions where everyone actually agreed the whole time. For example, I was in a debate with someone once where, after three hours, we found out that we were using slightly different definitions of the word "money". Once we hammered out a common definition, we suddenly found that we never disagreed on the real question at hand! This happens more often than one would think! So be vigilant of disagreements stemming from different definitions of terms and try to nip them in the bud.

6. Listen patiently and carefully to what others say.

This is really a two fold problem. One is that people get in such a hurry to say what is in their mind that they stop listening to what everyone else is saying and just want to blurt out their own thoughts. But listening turns out to be one of the most important skills in innovation. So don't be a jerk, listen up! The other part of this is that people naturally interpret, filter, and infer the words of others. It is important to pay attention to detail and make sure you understand what they mean and to ask questions when you don't understand.

7. Say what you mean.

There seems to be some sort of mangle-o-matic filter between the brain and the mouth. Be careful to say what you mean, try to make statements that don't leave anything to inference, and be willing to re-explain yourself in different terms if someone is confused as to what you meant. (Seems simple? It is harder than you'd think!)

8. Strive to reach the crux of any disagreement.

In order to reach resolution on a disagreement, it is important to find the crux of what it is, exactly, that you disagree on. It is no fun spending three hours hammering over a topic just to find that the crux of the disagreement lay in a misunderstanding of a word definition. Pealing away the layers to reveal the point of disagreement quickly will save everyone a lot of time, energy, frustration, and headache.

9. Discussions hinging on personal values are doomed to become debates.

Some discussions have no agreeable resolution. This is especially true of many socio-political discussions. When the crux of a disagreement hinges on a personal value or opinion, there is no way to agree. Whether it be a disagreement over something as stupid as the best flavor of ice-cream or the best band ever, or it be over deep issues such as abortion, gay marriage, and the validity of your own religion, there just isn't an answer that everyone can agree on. This doesn't mean that you can't understand and respect what the other person believes, but it means that you'll probably never agree, and so should agree to disagree.

10. Use logic, facts, and reasoning.

This should go without saying, but it doesn't seem to be the case. People cloud their reasoning with emotion. This is, once again, part of being human. But if you want to convince someone of the validity of a viewpoint, you must always support that with facts and logical reasoning, while being careful to avoid such traps as logical fallacy, inaccurate facts, and mis-representation of your knowledge limits. (But if you follow the other 9 rules, none of this should happen to you, right?)

Every Article Needs A Conclusion

So there they are, in all their glory. Some simple rules that take a lot of hard work to follow; but will quickly make you that innovative, team-playing, cool-guy, that everyone wants to have on their team and at their parties.

Monday, October 15, 2007

Code: Passphrase Generator

In a couple previous articles I talk about the benefits of randomly generated passphrases. But how do I generate random passphrases you may ask?

The answer is that I use the following Ruby script that looks at the words list that is provided on every Mac OS X 10.4 box:


words = []'/usr/share/dict/words','r').each { |line| words << line.chomp }

selected_words = []
while( selected_words.length != 4 )
  w = words[rand(words.length)].capitalize
  selected_words << w if (3..6).include?(w.length)

puts selected_words.join

In the event that you don't have access to the same words list, you can use your own by changing the filepath in the second line to that of your own text file. Just put one word per line and save!

EDIT: In case you are wondering, I did have another 2-line version of the script that harnesses the power of ruby's API and syntax, but it actually runs slower, so I went with this one.

Saturday, October 13, 2007

Rant: Passphrases Continued -- Why They Haven't Caught On?

In a previous article I talked about passwords vs. passphrases, and why, in general, randomly generated passphrases are a better idea than passwords. But while discussing my results with others the question came up "if they are so cool, why haven't they caught on?"

I think there are several reasons for that. One of which is that, historically speaking, space was limited, so passwords couldn't be longer than 8 characters. Thus password culture has been built around an eight-character password, even though this limitation has disappeared from pretty much every system.

But I think the main reason is that they just don't feel secure.

Think about it. Which one feels more secure to you? Opjk8J2Q or PiecesStudySmoothCatch? With all the security experts always telling you that you need a password with symbols and numbers in it, while avoiding dictionary words, passphrases seem to go against everything we've been told!

But as my article showed, a random four-word passphrase that is generated from a dictionary of only about 4000 words is just as difficult to crack as a random 8-character password made of alphanumeric characters! Indeed, if the exact dictionary of words you are using for password generation is unknown by an attacker, it becomes even more secure, as they have to try passphrases with words that have no relevance to your system.

A Little Extra

While I'm here I do want to mention two extra thoughts that came up while discussing my previous article with others.

The first thing is that passphrases are really no good if people can choose their own, as people are likely to choose certain words more over others. This uneven distribution could be figured out through simple studies and then exploited by an attacker.

On the other hand a random passphrase is so much easier for someone to remember that with a secure situation, issuing passphrases that people cannot change will work out better than issuing random passwords (since people usually have to write down random password, creating a place to breach security by a spying party; e.g. a janitor) while being more secure than a password that someone chooses for themselves.

Remember, depending on the system, it only takes one weak password for an attacker to hack into a system and get access to password information to start doing brute-force attacks on administrator passwords!

Video: iPhone in a Blender

Last night I was checking my e-mail after a long evening of playing The Sims 2, and found that my mother-in-law sent me this video.

As someone that loves his two Macs (and rarely boots his PC because he has little use for it), I should be abhorred by the wanton destruction of an iPhone. But as the curious scientist in me wanted to see what happened, I found myself being quite entertained by the absurd and humorous presentation.

Friday, October 12, 2007

Article: Security Words

Ever since I've been a kid, I've been fascinated with codes and cryptography. Of course, this topic has some great overlap with security, such as solving problems with storing, retrieving, and comparing passwords. So I was the obvious choice to implement the security engines for the software at the small start-up company that I work for.

Now this article is NOT about implementing security in Rails. This is a big and complicated topic that I do plan on writing about because implementing enterprise level security in Rails is not easy at all.

No, this is about another common topic. We often have to reset passwords for users. (We, of course, only store the hashed form of the password with salt so that no one get them!)

Now most of the time, assigned passwords are generated out of random sets of characters. This is great. A randomly generated password is very secure. But it is really hard to remember.

So I got to thinking. What if instead of passwords, we used a passphrase? That is to say, instead of joining together a series of individual characters, what if we joined together a fewer number of words? For me, at least, it is easier to remember a series of words than random characters, as I can come up with some visual or some rhyme to help me remember it.

For example, most sys-admins might give you a random password that looks something like 2qLzj94k. But what if, instead, they gave you a password like GreenRunDallasOrchard? I'd be willing to bet that you'd be much more likely to remember the 4 words better than the 8 random letters.

But this gives rise to the question: How many words would I need to string together, and from a dictionary how large, in order to match the security of a random string of characters?

Now For The Math

If you hate math, you may want to skim this section. Though I am pretty sure that if you are still actually reading this, then either you are a technical individual and like math, or you are a manager who was forced to read this by your IT staff because you just don't get it.

Normal passwords can be generated using all the letters (uppercase, and lower), digits, and a plethora of symbols. Looking at an ASCII table, it looks like there are, 94 eligible characters that one could use in a password.

Strangely enough, though, most random passwords are generated just with a subset of all letters and numbers, giving only 62 possible symbols. This reduction of 32 symbols leads to a drop of 5,877,349,279,825,920 passwords from all the possible 8 character passwords, which is a reduction in passwords of about 1/28th the fully possible 6,095,689,385,410,816!

Now to calculate the number of random passwords possible with just letters and numbers given password of a given length, we just raise 62 to the power of that length. Thus the following table hilights the number of possible passwords that exist with lengths of 4, 6, 8, and 16.

Possible Passwords Composed of Letters and Numbers

So the real question then becomes: If we had a pass-phrase of 4 words, how many words would have to be in our dictionary of random words to match the security of a random string of letters and numbers of a given length?

To calculate this, it is just a matter of the reverse problem from above. We know how many passwords we want there to be, and we know the length needs to be 4, so we use some n-th roots to produce the following table:

Dictionary Size Needed For 4 Word Passphrases to Match Passwords of a Given Length

The Answer For The Math Weenies

So what do all those numbers mean? They mean that to reach the security of a randomly generated 8-character password generated just of letters and numbers, we only need to pull four random words from a dictionary of 3844 words, which is a completely reasonable feat.

Indeed, doing a quick
grep -Ec "^[a-z]{3,6}$" /usr/share/dict/words
on my OS X box seems to indicate that there are 29,041 words, none of which are proper nouns, that are from 3 to 6 letters long that could be used. And expanding the list to contain proper nouns results in a dictionary of 33,925 possible words.

Thus with a dictionary of 30,000 words, it would be possible to match the security of a random 10 letter password made of letters and numbers, with only a four word passphrase! And if we increased the number of words in the passphrase to five, it would add 24,299,190,000,000,000,000,000 passphrases, which is the same security level as a 15 character alphanumeric password!

Tuesday, October 2, 2007

Tip: Time Scroller

With my co-worker working remotely the Philippines for the next 6 weeks, I have been finding that I'm often interested in two things:

  1. What time is it there right now?
  2. When is a good time to talk to him?

The first of these issues is handled easily enough with Apple's World Clock widget, which everyone has with their shiny copy of Tiger.

But to solve the second one of these issues, I was finding myself wanting a Little Orphan Annie decoder ring style thing--with color coded section--to let me quickly scan over times and see how things line up.

TimeScroller to the rescue!

After a quick search in Apple's downloads area, and a run around the block looking at the various widgets available, I found TimeScroller to be the best for what I wanted.

With a quick press of F12, I'm whisked to a place where I can quickly see the time in as many (or as few) cities as I please, and with a flick of a scroll-bug, I can cycle through the times, checking to see how things line-up without doing any mental arithmetic. And the best part: the customizable colored orbs that allow me to quickly tell when business hours align.

Indeed, the only thing I would like to have out of this is a way to define different hours for the different cities (as my co-worker currently works from 4am to noon Manila time while I work 10am to 6pm Spokane Time).

To download TimeScroller, visit the TimeScroller webpage at