Wednesday, February 15, 2012

jrm bloviates

I recently received an email asking me, “ Do you know of any good books about developing software or thinking in abstraction? I have books, but how did you train your mind?”

I'm not sure I even have a clue. I think that some of the things I've read and worked on were influential, but it isn't clear to me whether anyone else would agree. I've met people that have read the same books and their thought processes are completely alien to me. I know people that are very good at what they do, and reach solid, well-engineered solutions, but I cannot talk with them because they seem to have an insane basic approach to understanding. I've also been on the other end of this where I cannot seem to get the other person to understand a very simple idea on which everything hinges. But since this is a blog, I hardly need to be informed to express an opinion.

College was a huge influence on my thinking. I was fortunate enough to learn from some of the luminaries not only in Computer Science, but also in Math (Gian-Carlo Rota, Alar Toomre), Material Science (August Witt), Physics (I forget), Film (Richard Leacock), etc. What I noticed was that these really smart people spent an awful lot of their time and energy thinking about and talking about the basics and fundamentals. Over and over again they would stress going back to first principles. They invariably would lecture about “creating a model” to help in understanding. In the introductory courses, we'd focus on the commonly used models, but the professors would usually show us alternative models that had interesting properties. In the advanced courses, we would often use several different models. Sometimes the models were apparently contradictory, but were nonetheless useful.

6.001, Structure and Interpretation of Computer Programs, was heavily oriented towards controlling complexity with abstraction. To a large extent, it wasn't a programming course, but a survey course that explored a huge number of abstractions that have been found useful. The fact that it used Scheme as an expository language is irrelevant. The real point of the course was to look at the many different techniques of abstraction and to illustrate the commonalities and differences between them. However, the difficulty of the course tended to obscure this for many students. When you're trying to learn the syntax and semantics of a language, you can easily get bogged down in the minutiae and miss the big picture. For example, the “Triangle Language” problem set made heavy use of higher-order procedures.

The pedagogic goal was twofold: to become comfortable manipulating procedures as first-class objects, but also to be able to create an abstract model of picture composition through triangle-oriented primitives. At the abstract level, we had triangular `pictures' that could be decomposed by cutting larger triangles into sections, or composed by pasting smaller triangles together with scaling and rotation.

The Triangle Language is not a very complex language. It is not very general or useful. It is simply an example of a trivial domain-specific language. It is possible for a student to do the entire problem set, learn a huge amount about higher-order procedures, ace the quiz, and completely miss the point about solving a problem through creating a domain-specific toolkit. Many did. It was ok because every problem set contained some more examples of this. Eventually it will sink in.

6.004 also spent a lot of time talking about different abstractions, but it proceeded differently. We started with the linear model of a transistor, then developed the digital abstraction, from there we had switches. With a couple of more electronic elements, we could implement combinational logic. The combinational logic could be used to implement flip-flops and finite state machines. The state machines, combined with some memory, could implement simple push-down automata and stack machines. A stack machine is a good target for a higher level language, so we could write a compiler. The really interesting part of this course was the lab work where we'd wire these things up. Each week we'd augment the circuits built the previous week and we had built a small computer by the time we got to the end of the course. What was good about this was that you could see all the levels of abstraction at once. The little black transistors over on that side of the board were implementing a gate in a state machine that drove the clock for the microcode unit. You could look at the transistor as switching bits, computing part of a trivial function, or throttling electrons, whichever abstraction was the most relevant at the time.

I guess that the old maxim “practice makes perfect” is true. I trained my mind — They trained my mind by constantly teaching about various abstractions.

As for books, I obviously recommend Structure and Interpretation of Computer Programs, but there aren't many books that just talk about abstractions. It is well worth learning alternative abstractions to things you already know. Hamiltonian and Lagrangian physics has a different view than Newtonian physics, for example. (Unfortunately, it is hard to just get a simple introduction to Lagrangian and Hamiltonian physics.) It unfortunate that many useful alternative abstractions are very poorly taught. Try to find alternative abstractions that make sense to you. Try recasting familiar ideas in different abstractions (this can be fun and bizarre if the abstraction is from an unrelated subject). Rewrite a program based on a completely different abstract model. Try to simplify a complex program to the minimal abstraction possible. Just keep on practicing!


John Cowan said...

I second this, and point to N. David Mermin's article "From Cbits to Qbits: Teaching Computer Scientists Quantum Mechanics", which gives a simple view of those quantum-mechanical abstractions needed by people doing quantum software. All the mathematics is discrete, and indeed Planck's constant is only mentioned in a section called "Where is h-bar?!", which explains to physics professors why computer scientists don't need to know about it.

Like most Mermin articles, it is well and wittily written. Here's an excerpt:

[T]o understand how to build a quantum computer, or to study what physical systems are promising candidates for realizing such a device, you must indeed have many years of experience in quantum mechanics and its applications under your belt. But if you only want to know what such a device is capable of doing in principle, then there is no reason to get involved in the really difficult physics of the subject. The same holds for ordinary (“classical”) computers: one can be a masterful practitioner of computer science without having the foggiest notion of what a transistor is, not to mention how it works.

Anonymous said...

As you say, “practice makes perfect” is surely a component. But I think it also helps to read a lot of code. If you want to be a writer, you have to read lots of books before. Same with arts, you should first understand and learn to admire the works of the great masters.

Programming is much more than just writing syntactically correct code. Michael Sperber of the University Tübingen, maintainer of Scheme48, said once: "It's nice that the students that come to us already know Java. We just have to teach them how to program!"

So guys, read more code. Good code. And bad code too. Can you feel the difference? Can you tell *what* makes the difference? Most of the people I work with don't have a feeling for beauty, for beautiful code. This is, as I see, the greatest contribution of Free Software to the world: beautiful code that anyone can read and learn from.

Unknown said...

In order to be good at abstraction, you need to identify what is essential and nonessential because, at first, everything seems essential. The end result should be the relevant stuff. Practice makes perfect and there is no way around it. I guess, if there were books on it, everyone would be an expert.

But, your comment was really good.

grant rettke said...

Read enough books and practice until you could have written the books yourself. Eventually you will run out of books.

Unknown said...

I understand abstraction. I don't know how to write about it? A complex problem can be reduced to sub-problems. Hopefully, the sub-problems are manageable. In the process, hopefully, we can separate the essential from the non-essential stuff, which, in my opinion, is a skill. Is abstraction the essence, which allows us to develop elegant code? My question about abstraction stemmed from a *rant*, which is a long story.

Unknown said...

I look at abstraction the other way around... when you need to build UP to a complex system, every so often it'll get to where it's more complicated than you can mentally account for all at once. Below that level, you add an abstraction layer, so that you're dealing with "chunks" of the previous layer. In this way you can add arbitrary amounts of complexity without exceeding the VERY limited amount of state a human brain can normally hold.