Kannan:
But let me assume you want something very very very accurate. I don’t think any IMU you could place on a person or camera will deliver that accuracy. Certainly not for longer than a few seconds as you film a scene.
Can you explain more here please.
An IMU doesn't output a position, it outputs (primarily) acceleration along 3 orthogonal axes. These can be numerically integrated to get a change in position (from the initial position). Basically the acceleration outputs are fed into that equation you learned in grade school;
distance traveled at time=t secs = 1/2 * acceleration (assumed to be constant) * t^2 + initial velocity * t = initial distance
In general and in your case acceleration is not constant but changes as time passes and so you estimate a solution to the above equation using a sample data system where the acceleration is measured every “dt” secs, where dt might range from 1/10secs to 1/500secs. In a perfect world this would work for you but there are (at least) 2 big problems using hobby grade IMU components. First is that these accelerometers have an offset, they output a non-zero acceleration even when not accelerating. And this output is not constant, it varies with time, temperature and movement in a not wholly predictable way. So as time passes a small error in acceleration leads to an ever increasing error in position estimation. Ever increasing means proportional to the accel error and proportional to the square of time.
Additionally these accelerometers measure the 1G force due to the Earth’s gravity. That means with an otherwise perfect accelerometer, whose orientation wrt the Earth is known, you’ll measure 1G of “acceleration” just as the IMU sits there, unmoving. Now since this is a constant, you can subtract 1G from the measured acceleration and be good. Alas in order to do that you need to know, EXACTLY, the orientation of the accelerometer wrt to the Earth. Any error in that angle means some small part of that 1G will be treated as subject acceleration and end up, just as the offset above did, as an error in the estimate of the subject’s position. And 1G is generally a large value compared to a person’s movements. A hobby grade IMU might give you an angle error of +/- 0.1 deg, if you’re lucky. That’s 0.0017G added (erroneously) to your subjects acceleration. That’s about 2/3 inch after 1 sec, 5.5 feet after 10 secs and about 100 feet after 60 secs. While I don’t know what “sub-pixel” accuracy is exactly, I figure 100’ of error certainly ain’t it. Nor is 5.5’ or perhaps even 2/3". Nor is 0.1 deg of pointing error, all by itself.
And these are just 2 of the error sources. There’s also limited bits of resolution, noise, gain error, non-linearity, non-orthogonality of axes, temperature sensitivity, etc, etc for accelerometers … with some of the aforementioned being reducible via calibration, some not. And then you have to estimate the change in angular orientation of the IMU. That’s done by integrating gyro outputs, which is similar to getting position from accelerometers and subject to a different though similar set of error sources.
This is why IMU’s used for inertial navigation in planes cost 10’s of K$s, those used in ICBM’s even more. And you’re not going to fit one of those to an actor.