Thursday, July 30, 2009

How do you approach a problem?

Douglas Gregor wrote me:

I'm trying to hone my problem solving skills. Do you know any good books written on how to solve complex problems? Did school, i.e., teachers, books, labs, peers, help define the process? How do you approach a problem? How do you break it apart — the fluff from the stuff? How do you formulate a plan?

Wow. These are not easy questions.

Years ago I had a book called How to Solve It by George Pólya. I remember not giving it the attention that I think it deserves. I recall that some of the advice seemed rather obvious. In retrospect, I wish I had payed more attention to it. When someone like Pólya suggests something obvious, it probably isn't as obvious as you think and it is probably worth pondering.

College was a huge influence. The whole focus of the undergraduate curriculum was to “teach you how to think”. The detail of the coursework was almost secondary. It was there in order to be stuff to think about. It had the benefit of being topical. You got to know the jargon of the field and get familiar with the kind of day-to-day problems you would encounter, but that was secondary.

Graduate students would teach the recitations, but the professor was usually a tenured researcher. That made a difference. I found that the professors were amazingly sharp people. They also enjoyed tremendously what they were doing and their enthusiasm showed.

Nearly every course would start with a lecture where the professor would say “we're here to teach you how to think&rdquo. Then they would start in on the “first principles”. For example, in electrical engineering we started with Kirchoff's Laws. Now Kirchoff's Laws are really obvious: all the current flowing into and out of a node has to add up to zero (otherwise it would pile up somewhere or come out of nowhere), and all the voltages around a loop had to add up to zero (or you'd have made power out of thin air). But the professor would always start from “first principles&rdquo.

The next thing the professor would do is introduce a model. Invariably the professor would take a pretty simple and straightforward problem and point out that it is essentially impossible to understand and solve. Again, to use the example from electrical engineering, the action of any electrical circuit whatsoever follows Maxwell's equations (it has no choice), but no one is going to solve those (what's the B field around an irregular lump of solder?) The professor would show us the basic model and how to use it to approximate solutions to real problems.

Finally we'd go over the building blocks. There were a selection of primitive elements and a handful of ways to combine them.

What was interesting is that virtually every course took this approach. It didn't matter whether it was chemistry, electronics, physics, or computer science. It came down to “first principles&rdquo, models, and the basic elements.

Another thing that I noticed was the almost absurdly naive way that the professors would approach things. They always wanted to mention their research, so we'd hear about some of the cutting edge of science. The problems that the professors were working on could usually be summed up in a sentence or two: “Why do things fall down instead of up?” “How about if I made a computer chip with really tiny mirrors on it and computed with photons?” “If I run my computer backwards, can I get back the energy I put in?”

Of course someone would object and say “That wouldn't work.” or “You can't do that.” and the professor would immediately counter with “Why not?” and start showing that his ideas were actually plausible if you went back to first principles and used some imagination.

I found that the best professors seemed to have what seemed to be the most naive questions. I also found that they usually had a long, long record of posing interesting questions and developing astounding solutions. And they were fun to talk to and hang around with.

I really enjoyed the way that certain professors thought about things and I tried to emulate that. I keep telling myself to start with first principles and make a model. (I have to keep reminding myself of this, too. It's too easy to get wrapped up in the detail.)

There's one more thing these professors did: teach. Some of them taught undergrad courses that weren't their specialty because they wanted to learn the material themselves. If you can teach a non-expert enough to understand what it is you are doing, then you probably have a good grasp of it. Otherwise, you might not even understand what problem you are trying to solve. (That's one reason I want to write more blog entries. It helps me organize my thoughts by trying to describe what I'm doing to other people.)

How do you approach a problem? How do you formulate a plan?

This question gave me a lot to think about. I kept trying to think how I formulate a plan and I realized just this moment that I don't. I think maybe I disagree with Pólya on this point. He gives these steps:
  1. First, you have to understand the problem.
  2. After understanding, then make a plan.
  3. Carry out the plan.
  4. Look back on your work. How could it be better?
The interesting problems are the ones where I have no idea at all how to go about solving. Then there is a good opportunity to learn something (and I love learning things).

I usually don't understand the problem, and that's the biggest hurdle. Of course I often think that I do understand it and get off on the wrong path. I have found myself with a ‘solution’ to something I didn't really want to solve, or the answer to a question that wasn't worth asking. I have to keep reminding myself of what question I want the answer to.

I either don't have good plans, or I can't recognize the good plans amongst the bad ones. I look at a lot of different approaches and try one that looks promising. If that doesn't work, I'll try another. I'll keep refining the plan on the way.

When I get really into a problem, I'm constantly explaining it to other people to try to get some insight. Often I find I don't have a clear enough concept to be able to explain it to someone. Usually the person I'm explaining it to gives me some irrelevant suggestions, but it helps me understand how they are perceiving or understanding what I am asking. Sometimes they say words that trigger an idea.

I also like the oddball approaches. For a lot of problems there seem to be one or two people on the ‘fringe’. They eschew the standard approach to something and advocate something that seems to come out of left field. Sometimes, however, a non-standard approach gives you some insight into the problem that you wouldn't have gotten otherwise. A good example is using Clifford Algebras for computer graphics.

That's about all I can think of right now. I hope I answered your questions.


  1. What a wonderful post, thanks!

    It's the second time that I hear a reference to that book. I guess there's no excuse not to read it. :)

  2. This was a really great post! I recognized some of my teachers in your description.

    The most useful thing I found when solving problems is developing an intuitive understanding of the model. Going one step further from Kirchoff's laws, I began to think about capacitors and inductors as things that constrict the flow of current depending on frequency, thus creating a voltage drop across them. Unfortunately, the guy who taught us Circuit Theory was not only extremely smart, but also extremely obscure in his explanations and I had to figure out this stuff on my own.

    Having this intuitive understanding allows me to do two things. One is to to solve the problem intuitively, to get an idea about how the final result should look. For instance, if I'm looking at an oscillator, I can usually get an idea about the shape of the generated signal without actually solving the circuit. The other thing I can do is run a first quick check of my solution: if my intuition is wrong, the math will quickly prove it to be so. If it's not, and the circuit is not acting like I thought it should, I immediately have a few leads to follow and see if my analysis is correct.

    If I have to build something rather than analyze what others have built, I usually have a similar approach. I try to think about how I would do that without plugging any numbers. I try to use abstractions liberally and doodle a lot. In the end, if the result is very large, I tend to write down the final version so that I can have a reference.

    I also find it very useful to explain my problem to someone else. Ironically, I find it even more useful to explain my problem to someone who doesn't have a clue about my field, because it forces me to simplify things, explain them intuitively and focus on the big picture.

  3. Mr. The Donkey wrote: I find it even more useful to explain my problem to someone who doesn't have a clue about my field, because it forces me to simplify things, explain them intuitively and focus on the big picture.

    I love being able to do this. It's a real blast if you can give the other person some insight into what you do.

  4. I appreciate your response. It was helpful to me. I'm in good company to say, we're not satisfied with just solutions, we're satisfied with elegant solutions. Many people will say it's obvious, but, if you give them a problem to solve their approach will be to throw it against the wall, and when it sticks that's the solution. Now, I have to write down my approach to solving complex problems. Is there an elegant answer to my question? Please don't say it's obvious.

  5. I read "How to Solve It" by G. Polya and "Euclid's Elements Book I." These two books enhanced my ability to solve complex problems. After college, my ability to solve complex problems was amazing, but, unbeknownst to me, over time, my ability was getting dull. At work, I have to "crank" out the results in an unrealistic amount of time. I can only speak for myself, but, due to the pressure, I just solve them cranking out the results meeting the unrealistic deadline. But I digress. I never lost my problem solving abilities. These two books helped me wipe the dust off them. I'm sure a lot of people laughed at my question, but ask yourself, be honest, do you know how to solve complex problems? Now, go solve "The amusing problem" ( This was the blog post that got me thinking about my abilities to solve complex problems. Good luck.