Jonathan's Blog
name

Jonathan's Blog

Mindful Leadership and Technology


TagSoftware Development
Featured

Software Development Technical Debt Technology

Technical Debt as Metaphor, Where it Works and Where it Doesn't

Posted on .

I am across this article doing research for another post, I really enjoyed and think it is extremely valuable way to think about technical debt.

https://martinfowler.com/bliki/TechnicalDebtQuadrant.html

I'd go so far as to say, I'm going to expect developers I work with to read it and tell me which kind of technical debt they're talking about.

You should take the time to read the whole article. It's great. I'm not going to summarize it, only talk about a couple of things that I think are valuable and interesting.

Here's the quadrants for technical debt as Fowler defines them:

Technical Debt Quadrants

The first part of the article discusses the concept of technical debt and how it works as a metaphor to financial debt. Largely this distinguishes the concept of prudent from reckless debt, which is a useful distinction to have on hand for discussing projects with developers and clients.

In the financial realm you have prudent debts that you take on carefully, after doing your research, in order to further your goals. Buying a home, buying a car, purchasing rental property.

And then you have reckless debt that is taken on without proper research or out of alignment with your goals like buying a luxury items you can't afford.

There are two parts of this that I want to talk about in a bit more depth.


The first piece has to do with the concept of prudent, deliberate debt and where we begin taking it on in a project. I've had many conversations with developers over the years who may struggle to shift their mindset as you reach the end stages of a project.

Developers like to do things the right way - the scalable way, the smart way, the most effective way. I certainly did when I was a developer, and all the good developers I've known share this characteristic.

But there are times when you just need to finish off a feature or two and launch a product. Delays are dangerous in this situation. This is the moment to switch from avoiding all debt to taking on prudent, deliberate debt. It's time to say, "Yes, we could design it that way, but we need these features and we need to go to QA next week so please get it done."

Business realities drive us to these decisions, and most importantly, it is not wrong to do this. If you were a very savvy PM or product owner you might even put a date on the calendar when you think the decision making may change.

This doesn't negate the value of the all the design and hard work that went before it, nor does it mean you can't go back and revisit these items in the future. It just means, right now, in this phase of the project, we are making pragmatic decisions to ship and that is 100% OK and in line with best practices.


The second is the concept of prudent, inadvertent debt. Every project has it. No good developers are ever really satisfied as a result of it. But we can't ever truly get rid of it.

Why is that?

Just for clarification: prudent, inadvertent technical debt is the idea that some parts of system design will only really be apparent after the project is complete or once a particular phase is too far along to alter them.

This concept has no analog in the financial debt metaphor. There isn't a financial debt you can take on inadvertently and still be prudent. Being prudent implies that you don't have inadvertent financial debts.

But software is not finance - every software project has things that the development team would like to refactor toward the end.

If every project has this type of technical debt, why can't we eliminate it?

We can't eliminate it for two reasons:

  1. Technology changes too quickly. New tools come along and those tools have real business value so they can't be ignored. We bring them in to projects knowing we don't have every design ready for them. We do this because we push for ever greater scale, effectiveness, and feature delivery. We do it knowing it results in imperfect designs.
  2. Business Spaces are too diverse and changing too rapidly. Just when we think we understand an audience or a business, it changes. You can't be complacent, you can't rest thinking you know everything, you don't. Neither can you be paralyzed by such knowledge.

So, when you mix 1 and 2 together you get a situation where you can't design everything in advance. You look at the past, look at what you've done wrong and right, look at what you know about the tools you're using. And then you take the leap. You know you will taken on prudent, inadvertent debt by doing so.

The best bosses and clients understand this. It's why we like working for them and why work hard to eliminate everything else and build the great products we're capable of building.

Featured

Leadership Agile Software Development

Software Requirements are Business Requirements

Posted on .

Software and technology are bringing a lot of changes to our lives - just think of the things in the news set to make waves in the near future - robots, AI, blockchain, and self-driving vehicles.

If only part of the revolutionary claims for this tech comes true, the world will look very different.

It's exciting.

This cocktail of technologies brings a lot of opportunity as well a great deal of upheaval.

One thing I've noticed about this digitally transformed world: every technical requirement of a project is also a business requirement.

I've often heard these words spoken: "That's just technical detail."

But I agree with less and less as time goes on.

It sounds wrong now. Just a technical detail? What product are we building here?

Yet it's true, in a way. It is a technical detail. But that doesn't mean that it isn't important or you can just ignore it or you can pretend to ignore it until it becomes a problem.

So how do we deal with this?

Over time we'll all benefit from current changes in eduction - more focus on reasoning, outcomes, and learning code will help. More people who know what code is and who have written a line or two will help demystify the whole thing a bit.

Business people: You need technical awareness and patience and you need to ask the good questions. Again, eduction will help.

Technical people: You also need patience and improved listening and communication skills. I've written before about the pain of the unasked question. Ignoring hard questions because you don't like the answer isn't OK. Ask the question you know is there.

We all need to get better at explaining trade-offs in non-technical ways. This really shouldn't be that hard.

But what is the quickest fix? What can happen now? Where do I see people stumble?

The answer is more agile adoption and commitment to great product owners.

I'm currently reading Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland and J.J. Sutherland. Here's their quote, that I like the best on this. The Product Owner:

Needed to spend half their time talking to people buying the product (getting their thoughts on the latest incremental release and how it delivered value) and half their time with the team creating the Backlog (showing them what the customers valued and what they didn't).

That would definitely help a team move fast and understand the right technical/business requirements very quickly.

It's why I believe so strongly that Agile is the right methodology and that following as closely as possible to the recommendations makes sense. I've seen a lot of product owners who did a lot of the first or a lot of the second, but only a few who did a great job of balancing both (and who were given the organizational bandwidth and charter to do it).

Today everything - every part of an application - is a business requirement. Waiting around for business experts to answer questions, when they have many other responsibilities, is a delay that almost no one can afford any more.

We all have to go fast.

And we all need the business expertise and customer awareness built into our teams.

Featured

Rebuild Application Rebuild Software Development

The Many Guises of Hazy Rebuild Requirements

Posted on .

Clients frequently request that we use existing applications as the basis for work when we are rebuilding or recreating a system with them.

This is very difficult. It can work, but it is much less effective for developers and results in a lot of iterations as the dev team goes back and forth figuring out what an application needs to do. It leads to a lot of bugs that are only discovered in UAT.

In the beginning it is much more efficient for your business stakeholders, which is why they ask for it. Why go through the existing application and document it? That's a lot of work for users and business people, not to mention the BAs you have to pay to do it.

In the end, the same people who didn't want to go through the process of documenting will be mad because it is taking so long. It is your responsibility as a developer, lead, or PM to set the right expectations in the beginning.

If your stakeholder (whether client, boss, CEO, etc.) requires you to work this way you need to expose the risk and set appropriate expectations. The very best case scenario is that you have to set aside time for late project iterations when you run into these challenges. The worst case scenario is that the project will drag on so long that will be cancelled. It happens all the time. Also possible: you fixed-bid the project and put your company out of business trying to complete it.

Setting the right expectations can be difficult, and that is a topic for another post some time. My goal today is just to describe a few of the ways this has come up over the years, so you know what to look out for.

  1. Just make the new app do what the old one does. Why do you need requirements? An all-time classic. Why not? I mean there's already code, how hard can be to just: read it, understand it, understand all the subcomponents and UI, assess whether or not it is still necessary, talk to users, and figure out how to test it. It definitely wouldn't be easier to have that done and approved before you start development. Danger level: Red Flag
  2. Preserve the business logic, everything else you can get rid of. This assumes that preserving the business logic is easy, which it usually isn't. Even when someone has done a good job of separating business logic from UI and data access (very rare) often the technical and business requirements of the rebuild make reusing the code in its original form impossible. Danger level: Yellow Flag
  3. We'll figure it out as we go along. This one can seem reassuring in that your stakeholder has seemingly granted you permission to iterate. But be careful here: set expectations, ask follow-up questions, establish what 'figuring it out' will really look like. Danger level: Yellow Flag
  4. There's a lot you can re-use. Just tell me what it will cost to rewrite the things that you can't re-use when it comes up. You should assume that you are rewriting everything. If you get to re-use something that's a win. It will most likely save you a little time in testing, but not anywhere else. Danger level: Yellow Flag
  5. You don't need to talk to users. Bob knows everything about this application and Bob is your main point of contact. Unless Bob is the only user (and even then) he almost certainly doesn't know everything. Even if Bob isn't an ego-maniac (and he might be) you are still facing delays in understanding the system because Bob has to go ask someone instead of you asking them. Danger level: Red Flag
  6. This spreadsheet will tell you everything that you need to know about the system. AKA Our old system was a spreadsheet, just look at that. Usually these are worse than looking at code because in addition to code (like VBA) you also get things like embedded charts, formulas in cells, and obsure data access thrown into the mix, requiring even more detailed reading to understand. Danger level: Red Flag
  7. Jennifer developed most of the original system and she'll be working closely with you on this project. Usually Jennifer is retiring, and that is a hard deadline. Also, what does most mean? Danger level: Red Flag
  8. We have all the requirements from 20 years of work we did on the old system. You can read through that. To paraphrase Sartre, Hell is other people's requirements documents There are a few reasons why this is true, but probably the most profound is that writing the requirements document immerses the team members in the system. Without having written it, asked the questions, and fully digested the material then you or your BA will always be at a deep disadvantage. I wish this weren't true, but it is. At least 50% of the reason for documenting stuff is to make sure the person responsible really understands it. Danger level: Red Flag
Featured

Leadership Software Development Work

Three Types of Fun, Applied to Work

Posted on .

This came up in a meeting I attended. I described something a development team was working on as fun, and I was informed that it (creating peer-to-peer connections between Android devices) was Type 2 Fun, at best.

https://www.rei.com/blog/climb/fun-scale

I had never seen this before.

A lot of our work (hopefully) falls squarely into the Type 2 Fun category. Not that fun at the time, but fun to remember later, usually because I learned a lot and became better because of the work.

This is worth aspiring for in a career. If you find yourself having a lot of Type 1 Fun (regular, fun-while-it-is-happening fun) you probably either work at a skating rink and really enjoy the YMCA, are a little delusional, or a very self-actualized person.

Should you have some Type 1 Fun on a regular basis? Yes, you should. Can some of it be at work? Absolutely.

But a better achievement for our society is if a lot of us can sit around and say, "You know I learned a lot from that and it helped me excel in my career." If we can say that a lot of the time about a lot of the work we did, we've come a long way from where we were even 100 years ago.

Type 1 Fun is helpful and helps make people more productive. Type 2 Fun is necessary for the success of our companies and careers. And I do mean necessary - things that are a slog (Type 3 Fun) are unsustainable, in the long run.

I've certainly come acrosss some Type 3 Fun, also called, "Let's never do that again." But then, they are still jobs, even when it is your career.

Here's a quick reference, updated for the workplace:

Type 1 Fun: Remember that project where you learned a lot, everything was on time and on budget, and no one got frustrated ever? Me either. Remember that time you won the Fantasy Football League or had a great time at the Christmas party? Type 1 Fun all the way, and work related.

Type 2 Fun: Remember the difficult project where you started behind the 8 ball, worked long hours, but pushed yourself and came out the other side with a ton of knowledge? And you shipped a great product? Remember that time when you got fired and worked hard in your time between jobs so that once you landed a new job you were a changed person with different goals and a new outlook on life? That's Type 2 Fun.

Type 3 Fun: Remember having to do arbitrary work that didn't matter because it fit into someone else's system? Remember the client who used personal attacks on people because they were very unprofessional? That's Type 3 Fun, and it isn't migrating to Type 2 any time soon.

Featured

Bots Conversational UI Software Development

Bots Rising Continued - Creating a Bot with Dialogflow for Google Assistant and Google Home

Posted on .

I wrote some blog posts way back in the good old days of 2016 about a bot we developed for an intenral work tool. It ran inside of Slack. When we did the development we were basically writing code to handle all the states of the conversation. It was clunky and had some real limitiations.

Fast forward to more recent times - we bought a Google Home at the end of last year. Because I usually want to make something and not just consume information, I started to play around with Google Assistant and how to create an Assistant App for Google Home with tools on Google Cloud.

So right away I found myself working in the Dialogflow ecosystem. This was a big improvement over how we built our Slackbot in 2016 where we were hand-coding our state and writing a million if/else statements to handle the user interaction.

The Dialogflow designer interface is intuitive for creating your conversation. My biggest sticking points were working through the integration between it (Dialogflow) and my existing webservices. The answers were there in the documentation but it took some time to work through my SSL issues (doesn't allow self-signed certificates) and formatting of request. This was a little frustrating at times, but nothing out of the ordinary for using a tool that you haven't worked with before.

I also came across Let's Encrypt for solving my SSL issues. Let's encrypt allows you to get a real-ish certificate for development and test purposes. It is fast and automates several parts of the SSL cert process I've never seen automated before - very handy!

My Google Assistant App is submitted now and waiting for approval. I believe I adhere to all their guidelines, so hopefully they will approve me! Fingers crossed!

Featured

Mindfulness Leadership Software Development

Journeys and Guides and Maps

Posted on .

It's hard to draw a map of a place if you've only ever been through it once, going in a straight line in a fast moving vehicle.

For a guide, you may want someone who can draw a map from memory. You may want someone who has been lost where you're going. You may want someone who has, at least, been there a lot of times, walking in more of a zig-zag pattern and seeing what's around.

OR you're accepting a guide who knows a little more than you but can't draw the map.

Both can work, but understand what you're getting and accept the advantages of the latter if you choose it. That is, you will be getting lost along the way, but you will also be a guide at the end of the journey.

Sometimes there is no map and never will be - for the country is forever new. In these cases we can only compare it to country we have been in before and offer help to one another.