The various filters I tried on my data set were really disappointing. To try to figure out what was going on, I made a fake data set that had spikes every 15.625 milliseconds, but was zero everywhere else. If you perform a Fourier transform on this, you'll see the problem immediately. If my signal was simply a 64 Hz sine wave, I'd have a single point on the Fourier transform. If it were a square wave, I'd have the primary frequency and the odd harmonics. With a pulse, there are harmonics

*everywhere*.

But since I know that the spikes come every 15.625 milliseconds, and I know the phase, it occurs to me that I can simply erase those samples. No filters, no transforms, just a little rounding. In fact, if I walk through the data in 15.625 millisecond chunks, but make my first step exactly half that size, every chunk will have exactly one spike (although it might contribute to more than one sample within the chunk).

If I call this a “learning opportunity” maybe I'll feel better about this.

This has been an entertaining -- and, maybe -- educational series of posts! My math is pretty weak, but I can still get a feel for what you're up to.

ReplyDeleteThis comment has been removed by a blog administrator.

ReplyDeleteThis makes sense. If you have a signal that approaches a delta function, you'll have energy in frequencies from DC to daylight.

ReplyDelete