Demonstrated technical competenceIn what technologies does the partner claim expertise? Regardless of whether you have a technical stack in mind or are deferring to the partner, you want confidence the partner has skill and competence.
The strongest evidence is successful completion of projects using the proposed technologies. Your partner ought to provide examples of other projects, including how the technologies were used, the extent to which they were used, and the results. You want to know the projects were roughly comparable in terms of complexity (scope and team size). If not, then you want a clear picture of how the partner can scale their experience to meet the demands of your needs.
Second, ask for corporate credentials or relationships that indicate expertise with the given technologies. For example, if the partner claims extensive .NET experience, ask them to elaborate on their relationship with Microsoft. I'm not talking about getting a certification for buying a bunch of Microsoft stuff. I'm talking about actually engaging the Microsoft developer tools and technology program teams. Have they participated in any Microsoft ISV programs? Have they been in a TAP? While not perfect, this can give you a sense that they have invested additional effort in mastering one or more technologies.
As a concrete example, a partner I work with has the largest absolute (and as a percentage of staff) number of HL7 certified developers on staff of any company I know on the planet. This is a strong signal of their deep organizational commitment to healthcare interoperability.
Reference customersYour partner should have a set of customers willing to talk to you. I like to ask for 5-6 customers. Some in related fields and some not. Then I pick some (or all) and call them. Here are some of the questions I typically explore:
- How does the partner handle the contracting process? What about changes? I want to know if the contractor views this as a transaction or a relationship.
- Are the people on the team the same as the people that were introduced during the sales process? I want to know if I can trust the partner, or is this going to be bait-and-switch.
- How is the quality of the deliverables (not just the software)? How are quality problems addressed? I want to know if the partner takes pride in what they produce.
- How is the interaction with the partner's leadership team? How would you describe their character? Organizations take on the characteristics of their leaders.
- If you have visited their location, describe the physical environment (including hardware and equipment they have) and work conditions. Is it a sweat shop or do the employees appear well cared for? This impacts turnover, and consequently, the overall productivity of the team.
- Have you awarded the partner additional business since you begin the relationship with them? Yes or no, why? You can finesse the other answers, but where you put your money says a lot.
- If possible, describe briefly your project. Ask the reference if they would recommend the partner based on your description. Arguably, it might be little to go on, but you would be surprised. You may describe a highly technical research project and find that the partner is better suited at a well defined line of business application.
The teamAsk for resumes or biographies of the key team leaders that will be assigned to your project. Don't accept "representative" overviews. Get the details of the specific people that will be assigned. And be sure to interview them to verify their credentials and background. You should treat these key leaders no differently than if you were going to hire them onto your team directly - because that is what you are doing.
In addition to the team itself, ask about their training program. Elements you should look for include:
- New hire orientation programs
- Domain specific training (e.g, Healthcare, Banking, etc.)
- Function specific ladders with associated expected skills (e.g., Junior Developer through to Developer Team Lead)
- If appropriate, language and communication skills training
Again, this is not an exhaustive list. But it should get you thinking about more than just cost rates. Leave some of your questions or other items as comments. I'd love to hear what you think.