Eight Pro Tips on Writing Better Computer Programs

I have been writing computer programs for more than 20 years now. I thought it’s about time to gather some of my experiences and write some advice on building better computer programs.

1. Solve the right problem

It is all too often that one tries solving something  before understanding the problem at hand. We have a certain understanding of what the issue is, write something to solve it, but it turns out that we wrote the wrong solution because we misunderstood the problem. Do some research, read some articles, talk to people and read some books before attempting to solve a problem.

2. Don’t rewrite the wheel

You need a web server? Use an off-the-shelf one. It’ll be supported, it will be stable and it will implement the whole specification. Developers all too often think they can just write an XML parser overnight. Maybe for a simple one, but have you considered all the edge cases? How about UTF-16 character encoding in big endian mode? Are you prepared to support this code for the next 10 years?

3. Think about who, where, and where will use the system

Who is likely to use this system? How will their understanding of the system shape their usage? With what intent will they approach the system? Will they be forced to use it? Will they be trained? How can you train/help them? Will it be used for short periods of time only? You may need to make the interface very simple then — you can’t have deep menus and 20 configuration parameters. Will it be used by professionals? They might easily accommodate (and need) hundreds of settings.

4. Create simple prototype(s) and keep iterating

Try creating something simple that can do the job. If it doesn’t fully address the issue yet, you can now gain a better understanding of how the product works when it’s plugged into the processes of the people it’s supposed to help. Observe and fix what seems to be missing. Remember, if it aint’t broke, don’t fix it.

5. Focus on the customer experience

The classic example is that people will be happier if you can accurately tell them that there are 10 minutes left to finish a task rather than not showing them anything but finishing in only 5 minutes. Never forget what purpose the software is supposed to fulfil. Customer experience is paramount.

6. Design the system for change

There is only one thing that is constant about any project I’ve ever worked on: change. Requirements change, our understanding of the processes change, the underlying hardware and software environment changes, the legal and social framework the system is operated in changes. If you want to build for success, you have to make sure the system can accommodate these changes over time. Building rigid systems essentially sets you up for failure.

6. Don’t do everything that every customer wants

Tactics without strategy is the noise before defeat. Have a long-term strategy in mind where you want your system to go. Without such a strategy in place, it will be hard to know which customer demands should be fulfilled.

7. Don’t just fix bugs. Fix the specification.

The specification should be part of your test suite. If there is a bug it means that your specification was incomplete or wrong. The hardest bugs are rarely technological. They are almost always a bug in the specification. Fix the specification by adding the missing tests and make sure the specification is just as agile as your software development. Your specification must change, so your test suites, tools, and environments must be easy to maintain and change.

8. Enjoy doing what you do

Enjoying what you do, and making sure everyone on the team enjoys it too, makes not only for a fun time, but will also make people on the project committed to its success. Every project experiences ups and downs. In order to wither these storms and keep people around with deep knowledge and understanding of both the product and its environment (human, technological, process, legal, social), you have to make sure that everyone in the team is enjoying working on the project. Remember: it’s the people who build, maintain, and sell the system.

Memory layout of clauses in MiniSat

I have been trying to debug why some MiniSat-based solvers perform better at unit propagation than CryptoMiniSat. It took me exactly 3 full days to find out and I’d like to share this tidbit of information with everyone, as I think it might be of interest and I hardly believe many understand it.

The mystery of the faster unit propagation was that I tried my best to make my solver behave exactly as MiniSat to debug the issue, but even though it was behaving almost identically, it was still slower. It made no sense and I had to dig deeper. You have to understand that both solvers use their own memory managers. In fact, CryptoMiniSat had a memory manager before MiniSat. Memory managers are used so that the clauses are put close to one another so there is a chance that they are in the same memory page, or even better, close enough for them not to waste memory in the cache. This means that a contiguous memory space is reserved where the clauses are placed.

Continue reading

Benchmark randomisation for SAT Comp’16

Things are heating up for the SAT competition 2016. I will of course compete. However, I would publicly like to ask the organisers to please for the love of whatever you believe in, please randomise the benchmarks. Just a tiny, little bit. It’s ridiculous that people are tuning their solvers so they can solve some randomly solveable instance like the vmpc* series. It’s laughable and it’s making the whole community look bad. Really, it’s time to stop this madness. I wrote that article with a bunch of ideas in 2013. It’s time. Not even the largest of organisations move this slowly, and this is a research group of about 50 people max.

Continue reading

On Testing and Security Engineering

I have been working in a large organization for quite a while now where I have seen a lot of testing going on. As an information security engineer, I naturally aligned more with testing and indeed, information security assurance does align well with testing: It’s done on a continuous basis, its results usually mean work for developers, operations people, system architects, etc. and not caring about it is equivalent to accepting unknown risks. Since I have been working in an environment where testing was paramount, I have been digging more and more into the testing literature.

Continue reading

Testing CryptoMiniSat using GoogleTest

Lately, I have been working quite hard on writing module tests for CryptoMinisat using GoogleTest. I’d like to share what I’ve learnt and what surprised me most about this exercise.

An example

First of all, let me show how a typical test looks like:

Here we are checking that intree probing finds that the set of three binary clauses cause a failure and it enqueues “-2” at top level. If one looks at it, it’s a fairly trivial test. It turns out that most are in fact, fairly trivial if the system is set up well. This test’s setup is the following test fixture:

Continue reading