At my current company we have a fair amount of data about the activities that constitute our work - developing software, planning projects, testing, selling, communicating with clients, etc.

While some of this data has some volume - the work management certainly runs into the millions of records - some of it is at a more modest scale. For instance, the number of projects that we've run in the last 5 years is in the thousands, not tens of thousands or millions.

Yet there is still plenty of insight in this data if you know the right questions to ask. Or if you are willing to ask questions on a journey to finding the right ones. On this journey you must understand that the first questions you ask may affect what questions you decide to ask later. If you enter into the process assuming you know the answer or even all the right questions, you are at risk of missing the point.

Some of our information about successful and failed projects comes from a deeper look at client types, project types, and technology types. We didn't always ask the right questions first, but we kept asking questions and looking for insight in the data.

In the end the answer to a couple of hard problems revealed themselves in the data with relatively simple SQL queries (no advanced degree or BI tool required). You just need to keep collecting the data and keep asking questions.