Jonathan's Blog
name

Jonathan's Blog

Mindful Leadership and Technology

TagSoftware Development
Featured

Firebase Software Development Positive Behavioral Metrics

Firebase Errors for Humans - aka Best Error Ever

Posted by Jonathan FriesJonathan Fries on .

I'm working on a system for positive behavioral recognition and metrics using Firebase and React. This is a hobby/side project that I work on usually on nights and weekends.

This weekend I had an issue with Firestore (the newer of the two data solutions inside the Firebase product). I was querying on multiple data attributes in a collection. This is not allowed in Firestore, unless you have created an index that includes those two attributes.

I had no such index, so I got this error:

Does that say, "You can create it here:" ?

Yes, the short answer is that that link says, "You can create it here." And you can click on it and it takes you to the page where you can build the index.

AND...

And it has pre-populated the index for you with the two fields that you need so that you can click, "OK". And the index builds, just like you need it:

OK, thanks, that was easy.

This is really pretty amazing and I have never seen an error like this.

Often, errors are very generic and it takes time to search through forums trying to find someone with the same error who is actually having the same problem as you.

I have seen a number of errors in React that offer suggestions as to where your bug is and they are often correct. This is very helpful and a huge improvement.

The Firestore error takes that level of customer service to a new level: here's your problem, here's how to fix it, and here's a link that will basically fix it for you.

I would very much like to see more of this.

What has happened here is that someone at Google (Firebase is a Google product) has applied the ideas of user experience to the error handling in their product. Developers are the users of the error system, so thank you for that!

Firebase is a web-based product, so there is no reason why it can't know enough to point you in the right direction.

If you are in software development, though, you can appreciate that this is a high bar to jump over:

  1. Someone had to think of this early on, so that it could be woven into the product.
  2. Leadership had to include time and budget to implement it.
  3. Their UX had to be consistent enough to allow it.
  4. The error system handling system had to be built smart enough to make it all work.

All of this to say - a lot had to happen technically, organizationally, and culturally to make this happen.

When you think about the amount of time that often goes into error handling and management, it makes this all the more amazing.

Developers ARE the users of your error system, your API, your toolkits, and your documentation. Doing things like this makes your tool attractive to developers, managers, and business decision makers.

Why managers and business decision makers? Because it makes good developers better and faster, and it makes new developers far more productive than they would be if you didn't have it.

New developers spend a lot of time searching for answers online and asking more senior team members for help. Imagine a world where the platform itself can answer your questions and guide you through solving the problem.

Kudos to Firebase/Google on an amazing innovation.

Featured

Software Software Development I hate the phrase 'spaghetti code'.

This Code is Fragile on Purpose

Posted by Jonathan FriesJonathan Fries on .

This post is a follow up to my earlier posts on so called fragile or spaghetti code where I made a point about reading code - not the most fun part of anyone's day, but it generally makes code a lot less 'fragile' if you do it.

This idea is the equivalent of taking the time to read the instructions - you're a lot less likely to break something if you know how you're supposed to operate it.

This post is still about code fragility, but it is about code that actually is fragile, only it is fragile on purpose.

There are a number of reasons why this can happen, I will illustrate them from some recent work I've been doing, as well as some client projects. But first I am going to focus on what people mean by 'fragile', so it is clear what this means.

You may hear the phrase 'this code is fragile' and think that that means that it doesn't work, or that there are purposes to which it is not suited. As in, 'This vase is fragile so I probably shouldn't fill it with chunks of broken concrete.

But this is really not what 'fragile' code means. Usually it means:

  1. This code is hard to work on and it is likely to break if I do something to change it.
  2. This code may break in the future if the conditions in which it runs change.
  3. This code may break if any of our integration partners alter their API.
  4. This code does not fail gracefully.
  5. This code is not commented well and that makes it hard for me to read and understand what it doing.

As it pertains to number 1, this is just the scenario from the other article where you aren't working hard enough to understand it. Said differently, you are really saying, "My understanding of this code is fragile."

I put 1 and 5 at opposite ends of this list for a reason. Because 5 should not be an obstacle to reading code. It is a better excuse, than 1 for your lack of progress, but it still an excuse. Poorly commented code simply means you have to spend more time reading, testing, debugging the code to understand it. No code is ever commented well enough in my opinion, so you can't let it stop you from working on it. Code can be readable or unreadable, it can be commented or not, but that should not stop you from understanding it if you know what you are doing and that is your job. You can get a different job, but chances are you will be in the same situation there.

5 is only worth noting separately, because you should plan for and avoid that situation. Try to avoid committing this sin if you can.

For both 2 and 3, a significant portion of what will happen in the future is unknowable, and so how much time you spend dealing with 2 and 3 should be carefully callibrated. You can know that a partner is going to publish an API update in 6 months if they tell you, but if you need to go live now you may simply have to deal with that change when they publish it or make it the new API available for testing.

Of course with failures and crashes (number 4) you should avoid these things and use defensive coding practices to avoid exposing your users to failures, even if they originate outside of your systems. A few examples:

  • A good NoScript section of your website.
  • A graceful handling handling of offline operation.
  • Handling nulls, empty strings, and other straightforward data situations.
  • Use exception handling properly to deal with unexpected situations

Etc, etc.

But be careful of swallowing problems within the system - you may protect users and also hide them from yourself, only to have them come back and bite you later on.

It's always surprising to me how many people present an utterly, blank empty page to the user if JavaScript is not enabled or working.

OK, so here we are. We need to do some development work. We have coding practices to deal with number 4 and 5. We're working on a new system, so hopefully number 1 is a non-issue. What, if anything should we do about 2 and 3?

My answer is: it depends.

Truthfully, you can't predict the future or what your integration partner will do to update their API until they publish a specification.

We were recently working through updating a Dialogflow implementation for v2, but we couldn't have changed anything or done anything until they published their v2 specs. It would have been guesswork.

As I've been working on my side project Pricekite.io, I am dealing with the billing APIs for AWS, Azure, and Google Cloud, each of which is in varying stages of development. I am faced with 2 challenges:

  1. What are they going to do in the future to update these APIs?
  2. What additional features am I going to want to add in the future?

My decision for both is to do nothing. If/when I decide to add a feature that requires me to improve code, I'll improve it. I had to add some data storage in order to have data on hand for 1 provider (cough Azure cough) because their billing API is not the most efficient thing in the world.

I had not intended to add data storage until phase 2, but my hand was forced by the 6 seconds it took to pull and process the data. So, I call the 6 second method every 30 minutes and store the values, which can then be pulled at any time. It introduces latency to the data, but that does not matter at this juncture at all. If it turns out to matter, I simply can crank up the frequency of the polling.

So, the code is fragile. But it works, and it was able to be written by 1 person quickly, and it will be very easy to read as, for the most part, you have absolute line of sight readability in the code.

Featured

Leadership Software Development Agile

Software Product Owner - Do I Need One?

Posted by Jonathan FriesJonathan Fries on .

The most effective Product Owner I have ever worked with did not have that title. I mention this at the outset of the post merely to say that titles don't really matter that much. What matters is the role itself and organizational empowerment to do the job right.

You have a Product Owner because you want to go fast and develop great software. If you don't have one you are making the choice to go slow, which is always a bad idea. You may develop great software anyway, but it will take longer than it needs to.

The product owner is a critical role in modern software development. The Product Owner’s role is to supply the following to the development team:

  • Product Vision
  • Release Vision
  • Feature Backlog
  • Prioritization
  • Decision Making

Effective Product Owners work with stakeholders (usually customers, executives, business decision makers, and visionaries) to supply an effective workstream for their projects. Very often good Product Owners are neither the visionary nor an executive, but are someone with the ability to manage those people and the guts to make decisions about the product.

Ineffective product owners are too busy doing other things to be bothered with the day-to-day decision making of the team and leave questions unanswered for long periods of time, delaying the work.

Effective product owners manage and participate effectively in two work cycles:

  • The Development Sprint Cycle (usually 2–3 weeks) A good Product Owner makes time to attend important meetings and answer questions for the team doing the work.
  • The Product Development Backlog Cycle – A good product owner is constantly working with business stakeholders and customers (end-user customers) to understand what is working and what isn’t.

Ineffective Product Owners don’t spend the time necessary to get their product backlog prepared. This leaves their teams unable to plan or estimate work – delaying progress, de-motivating their teams, and creating blind spots in cost and timeline for their projects.

Effective product owners work hard to understand their competitors and customers, they know their place in the industry and where they are trying to go. Being an effective Product Owner is typically a full-time position, I seldom sees a product owner that can handle owning more than 1 product, and I have never seen one able to effectively handle more than 3.

Featured

Software Development Technical Debt Technology

Technical Debt as Metaphor, Where it Works and Where it Doesn't

Posted by Jonathan FriesJonathan Fries on .

I am across this article doing research for another post, I really enjoyed and think it is extremely valuable way to think about technical debt.

https://martinfowler.com/bliki/TechnicalDebtQuadrant.html

I'd go so far as to say, I'm going to expect developers I work with to read it and tell me which kind of technical debt they're talking about.

You should take the time to read the whole article. It's great. I'm not going to summarize it, only talk about a couple of things that I think are valuable and interesting.

Here's the quadrants for technical debt as Fowler defines them:

Technical Debt Quadrants

The first part of the article discusses the concept of technical debt and how it works as a metaphor to financial debt. Largely this distinguishes the concept of prudent from reckless debt, which is a useful distinction to have on hand for discussing projects with developers and clients.

In the financial realm you have prudent debts that you take on carefully, after doing your research, in order to further your goals. Buying a home, buying a car, purchasing rental property.

And then you have reckless debt that is taken on without proper research or out of alignment with your goals like buying a luxury items you can't afford.

There are two parts of this that I want to talk about in a bit more depth.

The first piece has to do with the concept of prudent, deliberate debt and where we begin taking it on in a project. I've had many conversations with developers over the years who may struggle to shift their mindset as you reach the end stages of a project.

Developers like to do things the right way - the scalable way, the smart way, the most effective way. I certainly did when I was a developer, and all the good developers I've known share this characteristic.

But there are times when you just need to finish off a feature or two and launch a product. Delays are dangerous in this situation. This is the moment to switch from avoiding all debt to taking on prudent, deliberate debt. It's time to say, "Yes, we could design it that way, but we need these features and we need to go to QA next week so please get it done."

Business realities drive us to these decisions, and most importantly, it is not wrong to do this. If you were a very savvy PM or product owner you might even put a date on the calendar when you think the decision making may change.

This doesn't negate the value of the all the design and hard work that went before it, nor does it mean you can't go back and revisit these items in the future. It just means, right now, in this phase of the project, we are making pragmatic decisions to ship and that is 100% OK and in line with best practices.

The second is the concept of prudent, inadvertent debt. Every project has it. No good developers are ever really satisfied as a result of it. But we can't ever truly get rid of it.

Why is that?

Just for clarification: prudent, inadvertent technical debt is the idea that some parts of system design will only really be apparent after the project is complete or once a particular phase is too far along to alter them.

This concept has no analog in the financial debt metaphor. There isn't a financial debt you can take on inadvertently and still be prudent. Being prudent implies that you don't have inadvertent financial debts.

But software is not finance - every software project has things that the development team would like to refactor toward the end.

If every project has this type of technical debt, why can't we eliminate it?

We can't eliminate it for two reasons:

  1. Technology changes too quickly. New tools come along and those tools have real business value so they can't be ignored. We bring them in to projects knowing we don't have every design ready for them. We do this because we push for ever greater scale, effectiveness, and feature delivery. We do it knowing it results in imperfect designs.
  2. Business Spaces are too diverse and changing too rapidly. Just when we think we understand an audience or a business, it changes. You can't be complacent, you can't rest thinking you know everything, you don't. Neither can you be paralyzed by such knowledge.

So, when you mix 1 and 2 together you get a situation where you can't design everything in advance. You look at the past, look at what you've done wrong and right, look at what you know about the tools you're using. And then you take the leap. You know you will taken on prudent, inadvertent debt by doing so.

The best bosses and clients understand this. It's why we like working for them and why work hard to eliminate everything else and build the great products we're capable of building.

Featured

Leadership Agile Software Development

Software Requirements are Business Requirements

Posted by Jonathan FriesJonathan Fries on .

Software and technology are bringing a lot of changes to our lives - just think of the things in the news set to make waves in the near future - robots, AI, blockchain, and self-driving vehicles.

If only part of the revolutionary claims for this tech comes true, the world will look very different.

It's exciting.

This cocktail of technologies brings a lot of opportunity as well a great deal of upheaval.

One thing I've noticed about this digitally transformed world: every technical requirement of a project is also a business requirement.

I've often heard these words spoken: "That's just technical detail."

But I agree with less and less as time goes on.

It sounds wrong now. Just a technical detail? What product are we building here?

Yet it's true, in a way. It is a technical detail. But that doesn't mean that it isn't important or you can just ignore it or you can pretend to ignore it until it becomes a problem.

So how do we deal with this?

Over time we'll all benefit from current changes in eduction - more focus on reasoning, outcomes, and learning code will help. More people who know what code is and who have written a line or two will help demystify the whole thing a bit.

Business people: You need technical awareness and patience and you need to ask the good questions. Again, eduction will help.

Technical people: You also need patience and improved listening and communication skills. I've written before about the pain of the unasked question. Ignoring hard questions because you don't like the answer isn't OK. Ask the question you know is there.

We all need to get better at explaining trade-offs in non-technical ways. This really shouldn't be that hard.

But what is the quickest fix? What can happen now? Where do I see people stumble?

The answer is more agile adoption and commitment to great product owners.

I'm currently reading Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland and J.J. Sutherland. Here's their quote, that I like the best on this. The Product Owner:

Needed to spend half their time talking to people buying the product (getting their thoughts on the latest incremental release and how it delivered value) and half their time with the team creating the Backlog (showing them what the customers valued and what they didn't).

That would definitely help a team move fast and understand the right technical/business requirements very quickly.

It's why I believe so strongly that Agile is the right methodology and that following as closely as possible to the recommendations makes sense. I've seen a lot of product owners who did a lot of the first or a lot of the second, but only a few who did a great job of balancing both (and who were given the organizational bandwidth and charter to do it).

Today everything - every part of an application - is a business requirement. Waiting around for business experts to answer questions, when they have many other responsibilities, is a delay that almost no one can afford any more.

We all have to go fast.

And we all need the business expertise and customer awareness built into our teams.

Featured

Rebuild Application Rebuild Software Development

The Many Guises of Hazy Rebuild Requirements

Posted by Jonathan FriesJonathan Fries on .

Clients frequently request that we use existing applications as the basis for work when we are rebuilding or recreating a system with them.

This is very difficult. It can work, but it is much less effective for developers and results in a lot of iterations as the dev team goes back and forth figuring out what an application needs to do. It leads to a lot of bugs that are only discovered in UAT.

In the beginning it is much more efficient for your business stakeholders, which is why they ask for it. Why go through the existing application and document it? That's a lot of work for users and business people, not to mention the BAs you have to pay to do it.

In the end, the same people who didn't want to go through the process of documenting will be mad because it is taking so long. It is your responsibility as a developer, lead, or PM to set the right expectations in the beginning.

If your stakeholder (whether client, boss, CEO, etc.) requires you to work this way you need to expose the risk and set appropriate expectations. The very best case scenario is that you have to set aside time for late project iterations when you run into these challenges. The worst case scenario is that the project will drag on so long that will be cancelled. It happens all the time. Also possible: you fixed-bid the project and put your company out of business trying to complete it.

Setting the right expectations can be difficult, and that is a topic for another post some time. My goal today is just to describe a few of the ways this has come up over the years, so you know what to look out for.

  1. Just make the new app do what the old one does. Why do you need requirements? An all-time classic. Why not? I mean there's already code, how hard can be to just: read it, understand it, understand all the subcomponents and UI, assess whether or not it is still necessary, talk to users, and figure out how to test it. It definitely wouldn't be easier to have that done and approved before you start development. Danger level: Red Flag
  2. Preserve the business logic, everything else you can get rid of. This assumes that preserving the business logic is easy, which it usually isn't. Even when someone has done a good job of separating business logic from UI and data access (very rare) often the technical and business requirements of the rebuild make reusing the code in its original form impossible. Danger level: Yellow Flag
  3. We'll figure it out as we go along. This one can seem reassuring in that your stakeholder has seemingly granted you permission to iterate. But be careful here: set expectations, ask follow-up questions, establish what 'figuring it out' will really look like. Danger level: Yellow Flag
  4. There's a lot you can re-use. Just tell me what it will cost to rewrite the things that you can't re-use when it comes up. You should assume that you are rewriting everything. If you get to re-use something that's a win. It will most likely save you a little time in testing, but not anywhere else. Danger level: Yellow Flag
  5. You don't need to talk to users. Bob knows everything about this application and Bob is your main point of contact. Unless Bob is the only user (and even then) he almost certainly doesn't know everything. Even if Bob isn't an ego-maniac (and he might be) you are still facing delays in understanding the system because Bob has to go ask someone instead of you asking them. Danger level: Red Flag
  6. This spreadsheet will tell you everything that you need to know about the system. AKA Our old system was a spreadsheet, just look at that. Usually these are worse than looking at code because in addition to code (like VBA) you also get things like embedded charts, formulas in cells, and obsure data access thrown into the mix, requiring even more detailed reading to understand. Danger level: Red Flag
  7. Jennifer developed most of the original system and she'll be working closely with you on this project. Usually Jennifer is retiring, and that is a hard deadline. Also, what does most mean? Danger level: Red Flag
  8. We have all the requirements from 20 years of work we did on the old system. You can read through that. To paraphrase Sartre, Hell is other people's requirements documents There are a few reasons why this is true, but probably the most profound is that writing the requirements document immerses the team members in the system. Without having written it, asked the questions, and fully digested the material then you or your BA will always be at a deep disadvantage. I wish this weren't true, but it is. At least 50% of the reason for documenting stuff is to make sure the person responsible really understands it. Danger level: Red Flag