Preliminary brain dump
A little while ago I finally responded to a PM from a friend of mine regarding
DevOps and really getting at the work I have been doing in developing a
presentation about DevOps (a currently popular concept in the world of Cloud
Software development). My presentation is really about my experiences doing
DevOps-like-things in areas of technology that seem very different and
incompatible with some DevOps practices, but in fact have very interesting and
significant relationships to one another. It is sort of an exercise in zooming
out the biggest big-picture I can manage and asking what does DevOps mean in
this context, or simply “How to apply DevOps to everything.”
I have discovered that I have been doing “DevOps” or something akin to it for
most of my widely varied professional career in high tech. Much of this was
before DevOps was even really a thing. If you want to learn more about the other
ideas that are similar and actually influence the development of DevOps,
Kaizen , and
W. Edwards Deming are good
places to start. For a lite and more story like introduction to the concept This
American Life has an interesting show on the NUMMI plant.
https://www.thisamericanlife.org/561/nummi-2015
In my mode of thinking about DevOps, the following things come to mind (no
particular order here, but rather in a sort of symbiotic everything contributes
to everything else sort of way):
- Shared Responsibility ( we all help / take responsibility where we can to
solve whatever problems we are facing )
- Continuous Improvement ( always working to improve not just product quality
our work output, but also improving ourselves, our work environment, and
customer outcomes, etc. ).
- Common vision ( we are all working towards a common goal. There may be many
facets to that goal, but we are all headed in the same direction ).
- Empowerment ( enabling every person to contribute to their full potential.
Also empowerment could apply to the customer in terms of value delivered and
ability to influence development and production )
- Toil reduction ( reducing painful, boring and repetitive tasks and situations
through quality and efficiency gains, automation, and also potentially in
delivery of value to the customer )
- Observation ( building the ability to see and effectively and reliably
measure, and hopefully correlate, what is happening in regards to productivity,
process, and product performance, and customer value )
- Agility ( being active and efficiently and effectively responsive to changes
in technology, the marketplace, in customers, and in the workforce )
The above tend to then be achieved through the following:
- Tooling and Automation
- Collaboration (including cross functional collaboration)
- Problem solving
- Development of process and standards (including exceptions)
- Identification and optimization of informational feedback loops
- Design (UX), Documentation, and effective communication and education
- Iterative improvements
Key and important concepts:
- F.I.R.E. (I tend to use Elegance as encapsulating most of this and because I
see risk in emphasis of the others, especially Fast and Inexpensive in some
contexts).
- Associate failure with opportunity to learn and improve (not to be taken or
made personal as in shame/blame)
- Encourage experimental innovation (both exploring new ideas, but also solving
chronic or unsolved problems by trying new approaches)
- Identifying and recognizing positive value in people and harnessing that
through empowerment as opposed to exploitation.
- Harnessing the value of metrics without falling into the trap of
“metric-mania” Metrics should not become the goals and thus should be seen as
tools to help point us towards and measure progress to our objectives. They
should be open to review, interpreted with context, and subject to critical
review.
- Root Cause Analysis. We are looking to understand or discover the underlying
causal structure that leads to problems in order to prevent future occurrences
and uncover deeper systemic needs or problems to address. We are not just
explaining the basic cause, assigning blame, or simply making the current
problem go away. Ask why, and when you have an answer ask why to the answer.
Repeat several times.
- Maybe don’t automate everything, at least not all right now. I really liked
this talk and it covers this well and give four “ironies of automation” at the
end. https://www.youtube.com/watch?v=JY4HPhXuWFg Matches my sentiment about some
of the challenges and problems that automation can lead to. In other words
automate to reduce toil, and beware that many times this is an economic trade
off of one form or another (more toil elsewhere, more expense, fragility, or
inflexibility).
- Slow down to speed up. Need to realize that efficiency is not the same thing
as speed. Speed can be messy, expensive, problematic, etc. But taking the time
and doing the hard work to develop quality and efficiency can result in at least
some gains in speed without the negative side effects. Also not all problems can
best solved (or solved at all) through decomposition into small increments. See
Hammock Driven Development - Rick Hickey
- Make room and time for growth. Getting to the place where things are done well
is more a process than a prescription, and the results are usually better,
longer lasting and more sustainable when it involves a healthy growth process.
This means starting where you are and applying iterative improvements to
yourself, the team, the process, etc. It is like what we do with the product.
My personal emphasis concepts:
- Economics in terms of relationship. Relationship here can and should be
understood to encompass both formal relationships (i.e. mathematical functions)
and personal relationships (i.e. subjective value). Good economics is about
optimizing the production or value (both objective and subjective) withing a
relationship. Both entities in a good economic relationship should be seeking
and ideally realize optimal gains in overall value in the process of exchange.
- Expansion of the notion of DevOps by looking at the most general notion of
development (product development, self development, etc.) and operation (doing
some task). DevOps then can be understood as a strategy or set of principals to
optimize the economic relationship between product and producer (employee and/or
company) with the objective of optimizing the economic relationship between the
prior pair and the customer.
- Many forms of optimization are really a form of contextual balance. Use the
analogy of riding a horse. You can fall off one side or the other. Ideally you
remain balanced atop the horse. Principals then tend to be expressible in pairs
as “guard-rails” against excess leading to imbalance.
- There is long term and sustainable value to be found in encouraging healthy
and natural diversity. Ecosystems build in strength and find balance through a
cycle of feedback loops involving natural life and death processes. The cycles
and feedback loops can be subtle and adaptable but also very powerful. I
differentiate between the death that is part of the natural cycles feeding new
life and the more unnatural death associated with sterility. In complex dynamic
systems we can either seek to understand and work with the natural forces within
the system to drive desirable behavior which take more effort to learn but less
effort to execute or we can use overwhelming force to drive towards the results
we desire. The former encourages life, but will also exhibit death (or
negatives) as the natural system adjusts. The latter may produce results more
immediately, but typically also with diminishing returns and ultimately a
sterile form of death where the natural cycles themselves are broken down and
lost. I’ve been pointing people to a movie called “The Biggest Little Farm”
which is a documentary of sorts about this ecological dynamic. I see this as a
good analogy to many complex and dynamic systems, and find it to be illustrative
as such of DevOps implementations and their long-term and large scale economic
results.
That’s sort of a brain dump of a big chunk of ideas that make up the structure
of how I think about DevOps. I may have missed or forgotten some ideas Also I’ve
mostly just tabulated mental indices to ideas, and not necessarily fleshed them
out.
DC - 20191007(PV) - 20240727