Almost 10 years ago I stumbled onto Joel Spolsky’s site “Joel on Software” and found a bunch of really great insight into the development process from an ex Microsoft developer with a great sense of humor. One article in particular “The Joel Test: 12 steps to better code ” I adopted immediately and added to my virtual toolbox of great ideals for the development process.

Since that time there have been a few shifts in the details of how software is built (or how I like to have it built), for instance Joel speaks about daily builds whereas I prefer the continuous build process, Joel mixes in the details of having developers write code during the interview (which I also do, but don’t think it belongs on this list per-say), and Joel speaks of Specs and I have included the Backlog, etc…

So, 9 years after Joel posted his original “Joel Test”, I thought I would share my variation of this “classic” list I have toted around with me and strived to implement in all of my development endeavors to this day. I use this list whenever I’m sizing up a client for consulting work, not necessarily to force them into “my” way of thinking, but to use as a guide for them to be sure, and to give me a clear indication of exactly how big the mess is that I’m walking into.

The Rob Test

1. Do you use source control?
2. Do you run automated builds?
3. Do you have a bug database?
4. Do you fix bugs then write new code?
5. Do you have an up to date schedule?
6. Do you have a spec?
7. Do you have a backlog?
8. Do developers have a good environment?
9. Do you use the best tools for the job?
10. Do developers write unit tests first?
11. Do you have testers?
12. Do you do hallway usability testing?

Not that far off from the original “Joel” test, but enough of a variation that I thought it would be worth mentioning and breaking down for todays developers, project managers and general geeks of IT.

As Joel points out 12 is perfect, 11 is manageable, but 10 or less and your heading into uh … interesting territory. As Joel also points out this is not the only hard and fast list to use, but I can tell you from personal experience and almost a decade of use, this list of 12 things will provide you a very good indication of what to get fixed first in your process to make the development iterations manageable and on time.

1. Do you use source control?
CVS has all but been replaced with SVN (Subversion) now, but CVS is still in use and there are plenty of alternatives to SVN from Microsoft Source Safe (not recommended to anyone), Git for the mainstream Rails community (although still not as accepted as SVN), and Mecurial (may actually become the new SVN one day). The bottom line is still the same though, if you don’t use source control (even for a single developer), you will loose working code, waste hours re-writing things, and not have a clear log of what you did and when. Everyone should use source control.

2.  Do you run automated builds?
Originally this was “Can you make a build in one step?” but I prefer the idea of continuous builds. Anytime a developer commits their code to the repository there is a “build box” that sees the checkin and checks out the code and runs through the whole build process in a single step, automatically. Note that for ridiculously large or complex builds that take an excess of 20 minutes to get from end to end, you may want to step down the process to individual modules of code (only replace what was committed), or take a look at the architecture of your build process. or confine yourself to daily builds as Joel originally recommended as his #3. In 2002 Joel and I had a conversation about Final Builder as a Windows tool for automating daily builds, I was at Rogers Cable at the time and welcomed the advice, it worked great!!! (Thanks for the tip Joel, it was a great recommendation) .

3. Do you have a bug database?
There are plenty of clients I have been involved with, that do not track their issues formally as Joel recommends. This is usually a good indication that your walking into a shit storm, my personal preference today is Redmine for defect tracking, although when I was @ v.1 we did use Fog Creek’s FogBugz and it is a great product that I totally endorse, Redmine is “today’s” favorite for me just because it’s written in Ruby, and Free. I have also used Mantis extensively (PHP based but isn’t all that pretty), and have tried to tinker with bug genii. I would have to say for commercial trackers, go with FogBugz, for Rails development Redmine is a winner, and if your confined to a php environment and need free, look at Mantis. The bottom line, is get all of your issues into a tracker if you want to release the best code possible. There is no other way to ensure that your issues aren’t falling through the cracks, and to think there is, or to think that your code will be bug free is just plain insanity.

4. Do you fix bugs then write new code?
Hank Rainwater pointed out the managing developers is like Herding Cats, and he is absolutely right. Making sure that developers fix the bugs assigned to them before they write that cool widget can be a fun exercise in developer management. It’s best managed (IMHO) by ensuring that anytime the bug count for any developer is greater then 10, they are not allowed to commit anything except bug fixes for a day. Joel prefers a zero count and I couldn’t agree more, but the honest truth of the matter is if you have a feature that has to be implemented and 3 open but low priority issues that are obscure and near impossible to replicate, I say let the developer add the new functionality to appease the stakeholders in the project. BUT if the bug count for any developer hits 10, thats an instant “your working on bugs today” conversation during the daily scrum.

5. Do you have an up to date schedule?
Joel has a great article for Painless Software Schedules and I highly recommend it as reading to see just how simple scheduling can be (despite the don’t read this message), read it, and then read the Evidence Based Scheduling article too… That said I prefer the Agile approach to development and the use of daily scrums while running through short sprints of software development. This moves the actual management of the daily schedule out to the scrum master and only the items in the sprint are managed. The scrum meeting should never take more then 10 minutes and only needs to answer 3 questions for each developer. 1) What have you done since yesterday? (last meeting). 2) what will you have done by tomorrow? (next meeting). 3) what (if anything) is standing in your way? This puts the onus on the developer for managing their time and helps them get better at time estimates. I will have to write out a future article on my methodologies for software scheduling but I think you get the just of it.

6.Do you have a spec?
The software specification for what your developing (in it’s current iteration) is usually the one thing that most clients I deal with are missing, sure they have a spec from what they originally wanted, but tis document is almost always out of date. Joel has a 4 part series on writing software spec’s that I can’t endorse enough, and while writing a spec is a necessary step in ensuring that your developing something that has had thought put into it’s design, keeping this document up to date is the whole key to making it a successful tool. There are different types of specifications that go hand in hand with development, and they all need to kept in sync with the actual development cycles, but specifically I’m referring to the 100,000 foot view of the project. The features that are in or out, and the general design ideas that make up the project as a whole. After all, we have automated tools that generate the technical documentation of your code based on your comments, you do comment your code (or ensure your developers do) right?

7. Do you have a backlog?
There are really 2 backlogs (not covered by Joel’s methodologies) and the concept is taken from the Agile approach to software development, Put simply, all of the high level features that will go into the project make up the project backlog. Each item is spec’d out for priority and time, and then specific items from the project backlog make it into the current sprint backlog, and sprint schedule. As new features creep in (scope creep) they should only ever be added to the project backlog and then reviewed at the beginning of each sprint iteration. This ensures that developers can focus on the tasks at hand in the sprint backlog and don’t get sidetracked by management or executives for “must have” features that really don’t make the grade. Each week (or iteration) the project backlog is reviewed and a new sprint backlog is generated.

8. Do developers have a good environment?
This is updated from the “quiet working conditions” Joel suggested, while technically the same thing, no two environments are created equally, and some developers (like myself) prefer to have background noise like my coding playlist playing, some studies show that different types of music can increase or decrease productivity. So simply stating that all developer environments should be quiet doesn’t really fit with my personal views. How can you determine if the developers have a “good” environment? Ask them. If they have phones on their desk that the company uses to page everyone else in the company, which causes a distraction they should disable this on the phone and place it in do not disturb mode (Feature #85 on most of the interoffice systems I’m in). If the developers desk is in the hallway, expect them to be distracted constantly, and in turn, expect things to take longer to finish. Once a developer has lost their focus it is a very real problem to get back into the “zone” and you should work directly with your developers to get them into an environment that makes them as productive as possible.

9. Do you use the best tools for the job?
While the “best that money can buy” (Joel’s #9) sits along the same line, I know of a few places that were using expensive tools the just weren’t right for the job, but the developers were practically forced to use them anyway because the company had paid so much money for them. Management had made the case for a tool and after it had been purchased the developers discovered that it didn’t provide any value add what so ever, and using it caused more slowdowns then anything, but because management had gone out and made the case for the tool there it was. This is incredibly daft (IMHO) and developers should always be on the lookout for the best tools for any job. Having developers that are on the pulse of the development community will usually yield the right kinds of decisions about tools, and as methodologies or processes change they can update or implement new tools that will help in the process without arguments from management if the tools are free. sometimes the best tool is the most expensive, and if thats the case, cost shouldn’t be an object. Just make sure that the tool is a requirement for the task at hand and not a nice to have, or something that the developers want for the sake of having it.

10. Do developers write unit tests first?
While not 100% accurate, the point here is really do the developers write unit tests at all? I won’t go into the debate over test driven development vs. development that has unit tests added regularly, I believe that the former is possible only when all of the developers have seen it work, and/or believe it will work. But if you have a code base that has no unit test, and no integration tests, I bet your re-creating the same issues over and over again in your development cycles. Developers should always be responsible for writing unit tests to ensure that what they just coded works as expected, but here in lies part of the testing problem. Developers test differently on their own code. They usually will only code to test that the code does specifically what they intended it to do, wherever possible have a co-developer or alternate developer test the bounds of the code (this will yield best results), the same thing applies to integration testing, the developer integrating the code will only test it is limited ways. Never have the developer writing the code responsible for the end user testing, this leads me into point 11.

11. Do you have testers?
My personal rule of thumb is that for every 2 developers you need 1 dedicated tester, Joel is right on the money with this, the tester may be able to handle up to 3 developers, but once you hit 4 developers, get another tester. The best testers are usually jr. developers that want to get better at coding, they enjoy breaking things and learning how to fix things, and you can usually give them the ability to hunt for the root cause in code as long as you have a more senior developer reviewing any changes a tester or jr. developer makes. Make sure that your testers are adding to the issue tracker (bug database) and make sure that at a minimum every bug found (regardless of how simple the fix is) contains the following 3 things. 1) A clear summary of the bug. 2) A more detailed description of the bug including a) exactly what happened, and b) what was expected to happen (i.e. why is this a bug). and 3) most importantly, what are the steps to reproduce the bug. If a developer can’t re-produce the bug then it won’t get fixed. Then make sure that it gets assigned, but I prefer to let the developers the bug gets assigned to set the priority. All to often a beta tester will enter everything as a show stopper and it takes time away from fixing things to have to re-prioritize issues in the tracker.

12. Do you do hallway usability testing?
I love this one which is why I have made sure it has stayed on my list over the years. So many times I have seen someone nabbed out of the hallway and told here, use this for a second to see them totally break it and wreak havoc on the system that the developer just told me minutes earlier was “bullet proof”. How does this happen, simple. Developers only test that things work the way they do them, if the developer always uses the keyboard shortcut for a button event and the nabbed hallway tester user only uses a mouse guess what … that onRelease code that never got tested in development just got some exercise. While Joel points out you also get invaluable user interface design issues from this testing, the number of times I have seen people break something is seconds makes this a great tool .

There you have it, my updated “craftsman” version of the Joel test… A day or two ago I was thinking that this material would make a great crash corse for managers and developers alike, not to mention a chance to play with my moodle setup. I would want to get permission and blessing from Joel of corse to refer people to his material, so Joel, if your reading this give me a thumbs up .

Comments are welcome ladies and gents.

Today the 400 hwy was shut down so I couldn’t commute the 100+km into the office, can you hear the disappointment in my tone? Luckily, I’m good at keeping focus when i’m not in the office and actually I think I was more productive today then I have been all week (and its still early), Not just with the work I’m doing on this project, but in managing a few calls I had to make but have been putting off … I can feel the cold shroud of fog lifting off of my brain finally, its been months since I have had this clarity, I did notice that my posts for almost the past week haven’t gone live .. I really need to keep on top of this … The earlier part of the week was simply me slacking in my writing, but I caught up … they just never made it live…

Bare with me good people, the time is soon that my blog makes it consecutively up on time.

Now I need to go back to battling with a TinyMCE plugin that doesn’t want to cooperate with this Rails app … so it’s a shortened post today.

Cheers!

This week should be a short workweek because of the holiday yesterday … Family Day! … But somehow this week has managed to turn into a longer week then normal, I have a scheduling conflict that requires me to be in Oshawa Today and I usually only work Wed, Thurs, Fri in an office environment (stability, regular pay, etc..) this leaves me Sat, Sun, Mon, Tues to do the freelance stuff on my plate which you would think is more then enough time, but it never is…

The original plan was to work 3 days in an office / part time gig and 3 days on freelance leaving me one day to NOT work. It hasn’t panned out this way yet. But I’m still trying. I know that I’m pretty blessed in the sense that I can get away with working a nine to five grind only three days a week, but I did have to sacrifice a fair bit to accomplish it. (about 50%) of the income I could normally make in a week doing freelance. In the end though the stability is more important then the money because with just freelance you can end up going weeks or months with no work which translates to no pay.

Unfortunately though the flip side of that is the ability for scheduling in appointment in just two of the normal weekdays, I don’t have myself set up yet so that things move fluidly. Which is the criteria that I have marked to call this method of working successful. When I manage to have a few months of continuos work for the freelance and the regular working side and I don;t have to work until 1am or Saturdays, then I will be able to say that this is a good way to work. Until then I’ll hold my judgments.

Worked late (4:30-ish AM) finishing up that lagging project (finally!!!) … Overslept, had a meeting with Mr. Bradfield (Jeff) at noon, but I didn’t even drag my but out of bed until 12:30… (but look its 8 full hours of sleep!!!!) Sent him a text to let him know we were still ‘on’ and that I was just behind, and drove down to Scarborough. I was going to nab Mr. Lodin (Kal) along the way, but he was already downtown so it was just Jeff and I.

We had a good time, did some work on his site (http://www.jeffbradfield.ca) and discussed a project we are collaboratively working on that moving really slow on account of us not getting together and keeping on top of things… I’m going to try again for the monday morning meetings (mastermind group). Jeff and I reviewed some layouts for the site we want for the new project and I’ll spend a few cycles this week (during my morning commute) seeing what I can get hammered out in terms of a loose ‘spec’.

We want to keep really agile on this and because it’s a collaborative thing between some great colleagues that I’m proud to call friends… its pretty relaxed.

Hopefully this will be one of those projects that turns into a revenue stream for us and will send us cheques in the mail … I really like those projects … the ones you do once but keep getting paid on. (You can thank Timothy Ferriss for my new found love of the 4 hour work week ideals).

I set Jeff up with some GTD love too … I’m sure that if has a peek he will get hooked.

Oh, and YES! .. I did get it done, and I feel great because of it

So as I was ranting about changes yesterday, what do you think came out of my meeting? (aside from feelings of absolute hunger and/or starvation because the meeting went through lunch and stretched out to four and a half hours)… Thats right, changes … 12 new feature requests and a slew of changes actually. Some of the changes are simple enough and fall just outside of the 2 minute rule to make, so they will be tasked. The application has been built to manage at least some degree of change that way, but some of the change requests will require a fair bit of thought to design and implement. This is definitely a great example of a ‘typical’ client and contract.

So, being all GTD stoked now … Craig and I get to exercise some of what we have been learning, so this isn’t really a rant parsee, its more of an observation of what to be grateful for … yea … yea … thats what i’ll call it But seriously, I am grateful for such a ‘typical’ opportunity to exercise the GTD principals I have been learning, and I’m sure that as long as Craig gets some sleep he will feel the same way too.

The collection process went well, Craig was on top of making sure that nothing got overlooked, I was sure to make relevant notes to the way the underlying system needs to be changed to support some of the new-ish functionality and although I will be coding a bit over the weekend, the majority of the notes can wait until Monday to get processed … I had to leave and get to another meeting immediately, so processing will have to wait (unless of coarse Craig updates the project site over the weekend). There is more then enough to get cleaned up before Monday that was already in the queue.

The sprint for this next iteration has been set to twice the length (two week sprint this time), that should provide more then enough time to get things back on track and done. I’m hoping that the organization strategies of the GTD principals will be the solution I’m looking for, time will tell the tale, and i’ll be sure to update as I go through it.

As always, comments, thoughts, feedback and flames are always welcome in the comments.

I’ve been listening to David Allen a lot this past week, that guy has it totally figured out. Truly inspiring. The inspiration naturally translates into working until midnight on a project that keeps lagging, but the end is in sight which is good because I have to meet with the client today (who undoubtedly has more changes). It’s a well known fact in software development that in order to make a product ship on schedule you have to nail down the features and functionality. Sadly, thats not what happens 90% of the time because people change their minds (all the time).

It never ceases to amaze and astound me how clients can fail to understand that if you change the scope, the timeline is also going to change (even when you look the client in the eye and tell them, this is going to add time). But thats how the cookie crumbles, and the developers inevitably end up burning the midnight oil to get something functional for a client who somehow has managed to miss that things take time, and time is worth money.

In the Freelance world, how much you let a project change is ultimately up to you, I think that one of the bigger things I have to take away from this project is that once the requirements start to change from the initial requirements, you need to nail it down (fast). In a development shop, you have less control over this but there has to be a way to manage it (fast). Otherwise you end up dealing with ‘add feature x please’ one day, and ‘oh I don’t want that anymore’ two days later which just totally zaps the energy out of a project.

If you can sense the slight irritation in my post today, its because I’m slightly irritated. Not with the client though, with myself. This is the second project this year that I’ve ended up, essentially working for free, (read outside of alloted hours for no additional pay). I’m not totally sure what the solution is, but I know I’m looking for one, and I also know that this is one of those problems that can work itself into just about any environment.

I wonder what I’m trying to tell myself?

Please post suggestions in the comments.