How do you approach a problem?

Douglas Gregor wrote me:

I'm trying to hone my problem solving skills. Do you know any good books written on how to solve complex problems? Did school, i.e., teachers, books, labs, peers, help define the process? How do you approach a problem? How do you break it apart — the fluff from the stuff? How do you formulate a plan?

Wow. These are not easy questions.

Years ago I had a book called How to Solve It by George Pólya. I remember not giving it the attention that I think it deserves. I recall that some of the advice seemed rather obvious. In retrospect, I wish I had payed more attention to it. When someone like Pólya suggests something obvious, it probably isn't as obvious as you think and it is probably worth pondering.

College was a huge influence. The whole focus of the undergraduate curriculum was to “teach you how to think”. The detail of the coursework was almost secondary. It was there in order to be stuff to think about. It had the benefit of being topical. You got to know the jargon of the field and get familiar with the kind of day-to-day problems you would encounter, but that was secondary.

Graduate students would teach the recitations, but the professor was usually a tenured researcher. That made a difference. I found that the professors were amazingly sharp people. They also enjoyed tremendously what they were doing and their enthusiasm showed.

Nearly every course would start with a lecture where the professor would say “we're here to teach you how to think&rdquo. Then they would start in on the “first principles”. For example, in electrical engineering we started with Kirchoff's Laws. Now Kirchoff's Laws are really obvious: all the current flowing into and out of a node has to add up to zero (otherwise it would pile up somewhere or come out of nowhere), and all the voltages around a loop had to add up to zero (or you'd have made power out of thin air). But the professor would always start from “first principles&rdquo.

The next thing the professor would do is introduce a model. Invariably the professor would take a pretty simple and straightforward problem and point out that it is essentially impossible to understand and solve. Again, to use the example from electrical engineering, the action of any electrical circuit whatsoever follows Maxwell's equations (it has no choice), but no one is going to solve those (what's the B field around an irregular lump of solder?) The professor would show us the basic model and how to use it to approximate solutions to real problems.

Finally we'd go over the building blocks. There were a selection of primitive elements and a handful of ways to combine them.

What was interesting is that virtually every course took this approach. It didn't matter whether it was chemistry, electronics, physics, or computer science. It came down to “first principles&rdquo, models, and the basic elements.

Another thing that I noticed was the almost absurdly naive way that the professors would approach things. They always wanted to mention their research, so we'd hear about some of the cutting edge of science. The problems that the professors were working on could usually be summed up in a sentence or two: “Why do things fall down instead of up?” “How about if I made a computer chip with really tiny mirrors on it and computed with photons?” “If I run my computer backwards, can I get back the energy I put in?”

Of course someone would object and say “That wouldn't work.” or “You can't do that.” and the professor would immediately counter with “Why not?” and start showing that his ideas were actually plausible if you went back to first principles and used some imagination.

I found that the best professors seemed to have what seemed to be the most naive questions. I also found that they usually had a long, long record of posing interesting questions and developing astounding solutions. And they were fun to talk to and hang around with.

I really enjoyed the way that certain professors thought about things and I tried to emulate that. I keep telling myself to start with first principles and make a model. (I have to keep reminding myself of this, too. It's too easy to get wrapped up in the detail.)

There's one more thing these professors did: teach. Some of them taught undergrad courses that weren't their specialty because they wanted to learn the material themselves. If you can teach a non-expert enough to understand what it is you are doing, then you probably have a good grasp of it. Otherwise, you might not even understand what problem you are trying to solve. (That's one reason I want to write more blog entries. It helps me organize my thoughts by trying to describe what I'm doing to other people.)

How do you approach a problem? How do you formulate a plan?

This question gave me a lot to think about. I kept trying to think how I formulate a plan and I realized just this moment that I don't. I think maybe I disagree with Pólya on this point. He gives these steps:

1. First, you have to understand the problem.
2. After understanding, then make a plan.
3. Carry out the plan.
4. Look back on your work. How could it be better?

The interesting problems are the ones where I have no idea at all how to go about solving. Then there is a good opportunity to learn something (and I love learning things).

I usually don't understand the problem, and that's the biggest hurdle. Of course I often think that I do understand it and get off on the wrong path. I have found myself with a ‘solution’ to something I didn't really want to solve, or the answer to a question that wasn't worth asking. I have to keep reminding myself of what question I want the answer to.

I either don't have good plans, or I can't recognize the good plans amongst the bad ones. I look at a lot of different approaches and try one that looks promising. If that doesn't work, I'll try another. I'll keep refining the plan on the way.

When I get really into a problem, I'm constantly explaining it to other people to try to get some insight. Often I find I don't have a clear enough concept to be able to explain it to someone. Usually the person I'm explaining it to gives me some irrelevant suggestions, but it helps me understand how they are perceiving or understanding what I am asking. Sometimes they say words that trigger an idea.

I also like the oddball approaches. For a lot of problems there seem to be one or two people on the ‘fringe’. They eschew the standard approach to something and advocate something that seems to come out of left field. Sometimes, however, a non-standard approach gives you some insight into the problem that you wouldn't have gotten otherwise. A good example is using Clifford Algebras for computer graphics.

That's about all I can think of right now. I hope I answered your questions.