The Sam Evaluation

This came from this blog post about the Joel Test. I was looking for a way to evaluate potential clients. I needed a way to determine where they are at so that I can figure out how best to help them. I also wanted to use it to stimulate their thinking about what’s possible and help them uncover any gaps.

These are not designed to be yes or no questions, but rather thought provoking exercises. As you are going through this with your team, it should raise as many questions as it does answers. The value is in the questions, not the specific answers. I want you to question the way you do things and investigate if there is a better way.

• Describe your team’s strategy for managing changes using Source Code Control. How do you clearly communicate that to the team? How do you ensure they have the skills to actually implement that strategy?
• Describe what happens from the time a developer makes a change until they can verify that all the tests pass and the application builds? How do you ensure that process is fast and repeatable?
• Describe how you take a built application and get it into the hands of your users. How do you ensure that process is fast and repeatable?
• Describe how you manage issues and prioritize your work. How are your customers and developers involved in that process? How often do you meet? What happens when you are under pressure? What gets cut and why?
• Describe your interview process. How do you verify people’s technical skills? How do you verify their ability to function as part of a team?
• Describe your bug-fixing process. How do users report bugs? How are they tracked? How do you prioritize them? How do developers fix them and release the fixed code? How do you ensure that process is fast and repeatable?
• Describe your learning and continuous improvement plan for your developers. What drives that process?
• Describe your process for retrospectives? How often do you have them? How do ensure everyone feels safe enough to give honest feedback? What do you with the results?
• Describe your process for on-boarding developers to new projects. What strategies do you use to get people up to speed and contributing quickly?
• Describe your process for on-boarding customers. What do you do to set expectations?
• Describe your process for distributing knowledge across your team. How do you ensure that multiple eyes get on your code before it is shipped?