A year ago I wrote a blog post about Kart, the static site generator that I have written. At the time the program was perfectly usable (I used it to create this site) and so I thought that only little time was needed before reaching a final state. How far from the truth!
Pareto principle
The Pareto principle states that 80% of the outcome comes from 20% of the work, and vice versa. One year ago I was exactly at 80% of the code, but I had spent only 20% of the time. To finish the project I didn't need one fourth of the time already spent, but four times as much! Why is that? At the beginning of our project we spend most of the time programming, creating a prototype. The rest of the time is spent improving little by little the code, so that it can accept a wider range of inputs, or something similar. This task takes a very long time, even if it doesn't take a lot of lines of code.
In my particular case, I wanted Kart to be customizable in every aspect, so that every user could tailor the result exactly to his needs. At first I thought that Kart was already pretty customizable, but each time I looked at the code I found something I had written thinking about my particular case, and that couldn't be modified by the user. Removing each of this parts took a very long time, and the process isn't finished yet! If I look at the predictions of the Pareto principle I calculated that I will need another year, at the current speed, before I am satisfied with the result! Ouch!
This has helped me thinking about estimating a project time. We constantly underestimate the time necessary for a task, from doing errands to studying to programming. It is strange that even though we always underestimate time, we seem to never learn. This is true for our personal time, but also for big projects. We constantly hear about big project's deadlines being pushed down the line again and again! To stop underestimating we could use a simple rule of thumb: each time we make a prediction we multiply the time by a factor of 2 to 4, depending on the difficulty of the task. Probably the prediction will be more accurate, and in the worst case we have more time for other projects!
The problem of designing an architecture
Designing a good architecture is difficult. Very difficult. More difficult if it is the first time. Before Kart I only wrote little scripts, or built site using an architecture that had been already designed from someone else. This time however I was the architect, and I did some errors.
Before implementing a project, we should first draw a very precise architecture. And I though I did. I was wrong. As I didn't know what a detailed architecture looked like I only though about a very loose architecture. When I started writing the code I realized some problems in the architecture, so I modified it. After sometime the architecture was more of a patchwork!
Maybe this is not a very big problem. If we are not prepared to change the design while trying to implement it, we will end up with a very bad result. But modifying the design everytime we think we have a better idea is just as bad! If a program is constantly being rewritten, we will spend lot of time in this process instead of refining the code, and if many people are using it they will feel a lot of pain! We need to find a middle ground between a very fast project and a somewhat stable one!
The importance of a documentation
The documentation! I think most of programmers, and I certainly put myself in this category, hate writing a documentation. It is very time consuming and not at all rewarding. However for an open source project it is absolutely important as it will determine if people will be able to use it. A proof of this is that you can determine with enough certainty the popularity of a project by how good its documentation is: the most popular open source projects have pretty good documentation while lost of little projects don't have a documentation at all. I am not saying that writing a good documentation will turn your project into a successful one, but the inverse is probably true, a terrible documentation will prevent the project from succeeding.
I've said all of this because I have realized how important the documentation is. However writing a documentation takes a very long time, and is probably the last reason I didn't expect this project to be this long. In fact as I am writing this post I have only finished the basic tutorial part of the documentation, and I still have to do the more advanced part and the API documentation. It will definitely be draining, but now I know that it will be worth the cost!
Conclusion
This was rough! In hindsight I see that I did a lot of errors underestimating the task I was putting myself in. But I think that understanding our errors makes us stronger, because we will be less likely (let's hope) to repeat them again. Anyway this blog post will remain here for me, and for everyone who read this, as a useful remainder before starting a new project!