The right team is key to crafting software systems capable of supporting innovation. Software delivery talent, however, is extremely difficult to find for a multitude of reasons.
High-Complexity Activity
Computer science is fractal in nature. At each layer of a system’s stack, and at each level of abstraction, there is a dizzying amount of complexity that is easy to get lost in. Your team will need to understand complexity at each of these layers well enough to understand the tradeoffs of different design decisions and how they might impact larger decisions such as scope and flexibility as your system matures.
Trying to manage your labor within this ecosystem is equally challenging. It’s often difficult to know how long a task will take until the developer has begun building the feature and you can see how the codebase responds. Your leadership team will need to account for this ambiguity by growing comfortable with more uncertainty than you’re used to when building budgets, forecasts, and project plans.
Generalist and Specialist Skill Sets Requirement
It’s unrealistic to think any one developer can understand all the complexity described above. You need to build a team of complementary skill sets and give that staff the time and space to build expertise to handle the unique problems and constraints across your system.
Your team will need to feature two different kinds of expertise: breadth and depth. You will need developers with a detailed understanding of the holistic system: where are the important interaction points, what are the key dependencies, and how is the system shaped?
You will also need team members with expertise in some of the specialties in the system; for example, operations, front-end development, or data engineering. The specialist expertise required may change depending on the nature of the problems you need to solve, but it is critical to build the team with the right skill sets and to ensure they are fully utilized.
Both types of experts are essential to your team and to your ability to understand and plan for the complexities within your system in a responsible way. But they cannot exist in a silo. These two types of experts are most valuable when they interact with each other. This adds a layer of complexity to team management decisions, as there cannot be a full division of labor across your system. The different team members focusing on different problems need to constantly learn from each other and collaborate if they are to achieve a high-quality solution optimized for your system.
What makes this arrangement even more challenging is that neither of these types of expertise is static. Every six months, thought leaders introduce new architectural concepts that can improve your system, and new tools will show up to support new languages. Becoming an effective expert in even one specific component requires actively looking outside of your company’s code for new ways to constantly improve the system.
No Formal Path to Trained Expert
With so much fluctuation within an expertise, it should not be a surprise that there is no standard path to becoming an expert. There is no straight line from novice to expert. Every developer’s journey to mastery goes through fits and starts, leaps and setbacks, and will only result in true mastery if every step is built on a solid foundation. The immaturity of the industry means there is no consistent institution or curriculum to ensure the foundation exists.
While everyone walks his or her own path, the industry has failed to establish an agreed-upon way to measure progress, and there is no standard way to gauge competency in a software discipline. Many in the industry have adapted by placing an emphasis on length of experience. If you scroll through any of the thousands of job postings looking for software developers, you will see teams looking for developers with some specified amount of experience — 5, 10, even 15 years’ professional experience using technologies that have barely been around that long.
This metric, however, is woefully inadequate for judging someone’s actual capabilities. Simply delivering software is an insufficient metric, and it becomes misleading in predictable ways. Software development 10 years ago looked a lot different than developing today’s modern software systems. Every year we discover new ways of architecting our code, gain new tools to build on new platforms, and discover new best practices to apply to these. As a result, the experience of developing a particular product 10 years ago is often less relevant than the lessons you would have gained from developing a similar product just 10 months ago.
Throughout my career, I have seen dozens of software crafters with only a few years’ experience mentor others who — though they have been industry professionals for decades — had never been taught to follow the more foundational principles and practices that lead to higher-quality software. While the experienced developer might have a better recall of idiomatic nuances, the inexperienced one may be the developer who understands how to apply best practices and will drive your team to success.
Another popular approach to training developers for success has been the influx of coding bootcamps and similar programs. These programs are good crash courses for developers to quickly get up to speed on the skills they need to contribute to a particular software stack, but the expediency of this training is often chasing after trailing indicators of what the industry wants.
One lesson we can take away from these programs is that there’s no shortcut to mastery. It requires an ongoing investment in continual learning at every stage of a developer’s career.
Fast-Changing Skill Sets
These problems are endemic to software because the industry itself has immature and changing best practices. Software is still a relatively new discipline, and the frequency of change in tools and technologies has made it difficult to cement established best practices the way other disciplines have, much less have those practices become commonplace around the world.
Without strong guardrails around the ways we work, teams are often tempted to take shortcuts to deliver more quickly and please their stakeholders. This short-sighted strategy has led to enormous waste that riddles the industry.
Every corner cut at the beginning of a project has the potential to add exponential complexity later. Several months down the road, as you’re collecting live user feedback and fine-tuning your business model, what might seem like trivial software changes will end up requiring massive refactorings or rewrites because so many layers of complexity will have been built on top of a decision your team never considered might need to change.
[For more from the author on this topic, see “Transformation Starts with the Team.”]
