Mine the gap

Mine the gap

March 2, 2021

The opportunities and challenges for social change are often greater between than within. 

Western societies are sufficiently advanced that few simple problems remain unsolved. The bad news is that the vast majority and most serious social ills like poverty, homelessness, failed education systems and the like are deeply complex in the way that their root causes are embedded in multiple failed systems. They are moreover organic and dynamic; morphing in response to corrective action in much the same way that a virus will transform itself in response to a potential cure. Push on the problem in one place and it surfaces another.

How is it then that we persist in rifle shot approaches to solutions? Foundations, non-profits and businesses that limit their engagement to flogging these problems independent of each other will continue to be disappointed with the results. And leaders wishing to make a difference who don’t equip themselves with requisite experience and tools will be challenged to succeed.

My purpose here is to shed light on the tools and talents necessary to make a difference in complex problem solving. In order to do so, let me first illustrate the point about complexity, for we are still at heart a blame-placing society more inclined to locate and castigate ‘perps’ than identify root-causes and corrective action. The story of quality improvement in manufacturing businesses is illustrative.

You can peel the roof off a manufacturing plant and witness virtually every interaction that can take place between systems, people or process. No matter the type of problem, you can find it in a factory. I had a plant once that was failing to deliver quality product on time and at a competitive cost. After flogging each of these elements of the equation, it turned out a drill press critical to components needed for assembly was incapable of meeting the specs, largely because the operators had not been trained in proper tool changing. Problem solved? Hardly. Training as a process was not working well and union work rules were an additional constraint on putting the right people on a difficult job. Plant management was not trusted by either the union or the employees and no amount of sweet talk by the plant manager or by me was going to change that overnight. It would take years to build a more transparent and trusting culture in the plant and a succession of trustworthy leaders to sustain it. And it would take the reconciliation of short term shareholder value interests with long term continuous improvement to make it stick.

 The history of quality improvement in American business is still largely unwritten and incomplete because this kind of reconciliation is rare. Advancement to the executive suite is more often based upon short term impact on profitability. Successful problem prevention is sufficiently boring and heroism in fighting fires so well regarded that we condition our business school graduates through case study to fight fires if not ( financial services?) to start them.

 Harsh perhaps, but not far off the mark.

Complex social problems have traditionally been left to civil servants and non-profit workers. These men and women are for the most part drawn to this work because of their passion for the mission of the organization and, in my experience more prone to be trusting of their colleagues and their leadership than those in other sectors. In non-profits, trust is for management and boards to lose through their actions.1 Given the right tools and resources, the prospects for continuous and sustained quality within a non-profit have always seemed to be greater than in government agencies or the private sector.

However, even if every non-profit in this country became high-performing in its chosen mission, chances are good that graduation rates, societal inclusion, and economic well-being for the underserved would show little in the way of progress.

How is it then that the problems of, say, a failed education system in this country, of poverty and homelessness, of social injustice and inequality are to be solved? And what are the skills and experience required to make progress in this important work?

1. Getting to the edges and depths…

 It is widely understood and agreed that our most intractable social problems result from the failures of not just one but several systems feeding off each other. Poverty, recidivism, school dropout rates stem from economic inequality, the breakdown of family structures, social injustice and a host of other societal failures. Government bureaucracies, either under resourced or poorly managed, are matched with family structures that are broken or dispirited. They are supported by a proliferation of non-profits and foundations that are themselves understaffed and poorly governed. Bureaucrats don’t talk to each other and little enough to those whom they are in place to serve (the teachers, students, the underserved they are there to help). Non-profits don’t communicate since they are competing with each other for funds. And foundations generally don’t speak with anyone other than through grant applications and written reports.

These networks in place to solve problems are systems (social enterprise, grant-making, public policy) and prone to failure just as surely as are education or public welfare. They are processes joined together to produce a product or result and like all systems, they work ‘perfectly’. A process for, say, combatting homelessness that is poorly designed will fail on a consistent and predictable basis. And if more than one system (e.g. welfare, education, the family) is involved, the incapability of any one of them will result in the failure of the whole.

At one time, I was involved with diesel engine assembly for a living and I can recall speaking to the large group of suppliers who provide the components for assembly on the subject of product delivery. I knew, as all assemblers do that if each of those suppliers shipped quality products 98% on time (which each of them would take pride in accomplishing), we might never build a complete engine

Cancer and the search for a cure have many of these same attributes. The disease itself moreover has chameleon-like qualities and the multiplicity of medical disciplines and research organizations under- communicating in the search for a cure compounds the problem of finding one.


Thus, when a number of processes and systems are involved in both creating and in solving social problems it doesn’t take much failure to impede progress.

 The first tool then in the complex problem solving tool box is to treat holistically both the systems and participants that engage the problem (foundations, non-profits, business and government) as well those at its source…the clients, or victims of these failed systems as well as those on the front lines (social workers, teachers, etc.)

2. Cross Walking…

Engaging all of those involved in social problem creation and solution-seeking does not involve convening a ‘senate’ of all parties in the search. In my experience, mobilizing such widely diverse and unwieldy groups to work together compounds the problem and if attempted at all, should occur only when solutions are in sight. Rather, each of these groups need to be brought together separately by their leadership to first see and understand both the scope of these several systems and their role within them.

 Businesses, for example may view themselves as merely ‘customers’ of the education system. With proper facilitation, business leaders can come to see how they are contributors to and enablers of the persistent failure of these systems through their own selection, training and corporate philanthropic practices, and that their contributions to a solution should extend, for example to modifying and applying business problem solving tools within the social sector.

 Each of these groups, then needs to be approached separately. The leadership for these separate discussions can come from those who through their size, resource capability, political clout or thought leadership have the ability to bring these groups together. As individuals, they should be distinguished by their own cross-sector experience, facilitation skills and trustworthiness among those they seek to convene.

 These men and women need to convene beforehand in order to align their own understanding of these systems and to set expectations as to what results of these discussions are intended to be, what comes next and when. Whether they take the form of a steering committee or board, or serve as more of a collaborative network would depend on whether, when, and to what extent this approach is to be formalized. Regardless, this group has a critical role to play once these separate groups have begun work.

3. Weaponry…

The tools for problem solving come to us from centuries of use in widely diverse settings. Given that all of them share a requirement of being fact- and data-based, the ease and effectiveness of root cause analysis, Six Sigma and the like has been greatly facilitated by the advent of web-based technologies..

It is not my purpose here to elaborate on these tools; they are widely used in the business community and with minor modifications (mostly having to do with language) can be applied in a social setting.

Rather, my focus is on important aspects of this group work which while they are generally understood as important by those who regularly use these tools, are especially critical here.

Scope: the problem-solving work of a group should encompass only those systems over which it has some control.

If for example, the non-profits associated with homelessness are brought together to conduct root cause analysis, their focus should be primarily on the contributions of this sector to the problem or solution (e.g. communication among non-profits, donor competition, etc.), with causes attributable to others noted but not analyzed. Narrowing the scope in this way does not preclude seeking input from others outside the sector (e.g. business, government, advocacy groups etc.) in identifying opportunities for improvement.

Scheduling: tight agendas with tools training and application should be the focus, with full participation and follow-up expected. 

The men and women who work on social problems have less time and tolerance than most for time- wasting and process-heavy engagements, even if they are prone to excessive process themselves. The best way to snuff out collective work in the early going is to bore the participants with irrelevance.

Non-blaming: an essential premise to this work is that no one individual, organization or system is at fault

The underlying premise in collaborative problem solving is that no one organization is at fault, but rather that the solutions are to be found across and between participating systems and organizations. Even where individuals or organizations are underperforming, the root cause is more likely to be a system not adequately ‘fail-safed’ against human fallibility.

Contextual: societal problems are addressed within a framework of government law and regulation which needs to be understood in order for problem-solving to be effective.

Federal, State and local laws, regulations, ordinances, etc. no doubt have a part to play in both enabling and constraining solutions. These groups need to be aware of these requirements but they should remain outside the scope of all but government groups convened for this purpose. The politics associated with addressing these constraints come later and are best left to political leadership

Results-Oriented: these groups should have beginnings and ends, with clarity as to what result is expected and by when

The results expected should associate directly with the scope of the assignment. Using the example above, the purpose of a non-profit group in this context is not to solve homelessness but to identify the root causes that prevent non-profits from being more effective individually and collectively.

1.  Walk Talking…

 While there may not be a senate overseeing this work, communication and transparency are essential requirements for its long term success, both among and across all those individuals and organizations effected by it.

 There is first the need to understand the relationships of each group member within a group to their constituents. Businesses have shareholders, customers, employees, trade unions and boards of directors. Non-profits may have all of the above plus their donors; government agencies have political leadership, and so forth. The convening and the work of each of these groups needs to be expressed to constituents with a consistent message but in terms appropriate to their interests. Thus messaging will be different and once the message is agreed, left to each participant to tweak.

 Communication across groups is no less important and in doing so, transparency is essential. It is likely that one group may perceive another as biased, ill-informed or adversarial based on prior experience. The attributes of effective problem-solving within a group (fact-based, non-blaming) apply here and maintaining an open and candid flow of communication across groups will be essential to future work.

2. Gap-Mining

 While there are certainly instances where the root cause of a problem seems traceable to a single source (a drill press), these circumstances are increasingly rare. When social goals are widely shared, like economic well-being, the elimination of poverty, or no-child-left- behind and it is the means not the ends that are in dispute, solutions are more likely to fail because of the ‘disconnects’ between and among processes and systems.

 Disconnects, or “gaps” occur where two processes within a social system or two systems meet for a hand-off. A student passes from a tutor to a teacher or from one grade to the next; a parolee passes from incarceration to rehabilitation, an unemployed person passes from welfare to job training, and so forth. In a perfect world one process or system delivers to another with clarity as to what has been accomplished so far and what is needed next – clarity that comes from both processes first being clear on what is expected and then variances, if any, that have occurred from was expected.

 Gap problems are generally information-based and born of poor or incomplete communication. In my experience most occur despite the best intentions of either side of it. To illustrate:

 –  Non-profits working on the same problem from different perspectives don’t talk to each other


1 Separately I’ve written about the failures in non-profit governance and the insufficient and selective application of process improvement tools as at the source non-profit underperformances. See “Are Non-Profits Different”

Subscribe error, please review your email address.


You are now subscribed, thank you!


There was a problem with your submission. Please check the field(s) with red label below.


Your message has been sent. We will get back to you soon!