Fitting a model may be sometimes easy, sometimes difficult or even frustrated. Especially the one that I am using, the full Bayesian models. Yes I know it is extremely flexible, powerful, literally can do anything. But the problem is it is not so convenient and slow (not surprise for its simulation method). The funny part is it often gives you similar results for complex models (the very reason to use Bayesian methods). So the question is, is it worth the effort?
For example, you want to model count data. You may use Poisson, NB, Poisson lognormal with CAR; the 1st order neighbour structure, the 2nd order structure; fixed or random effect for time effects, or even using autocorrelation… Maybe NB is good enough. There are lots of researchers around the world playing around and showing the rest “complex” ones, though I guess they initially used NB for testing different parameter combinations before finalise the models. Then they “found” the results are similar for complex and less complex models. This is expected and the reason they use less complex, fast models to test isn’t it? I am not saying this is purely “showing off”, but as in this TRB 2009 meeting, one presenter stated, the full Bayes is only worth if the data has small units that are correlated, and if you have a good PhD student (like me:)) who is willing to do the analysis…
Still, there are numerous problems about the models. Sometimes the problem seems so simple and basic. Let’s look at the following example that my supervisor gave me the other day:
The Simpson’s Paradox:
Let’s say there are two persons: Sleve and Mark. They are working to produce some “products”:
|Year 1||Year 2|
where 500, 320 etc may be the number of products; and they are divided by, say months, to get the “outcome per month” so they can be compared with each other.
So Sleve claims that he is more productive than Mark, because clearly, indeed, the resulting values (50, 80) are higher than Mark’s (45, 70), each year.
Then Mark says, hold on, I am more productive than you (Sleve), why? Let’s take a look at the whole process: for the whole two years, Sleve got (500+320)/(10+4)=58.6; and Mark got (270+700)/(6+10)=60.6. Clearly Mark is more productive than Sleve.
So whose claim is correct? It should be also noted that this has nothing to do with sample size as we are comparing ratios.
That said, model and data are not easy to understand. Sometimes the question is as simple as, say, you got “more police more crime” relationship from your model, so is it the case that police encourage more crime; or crime decreases but crime record increases because of more police in place?
Maybe we should always keep this in mind when playing with the models: “All models are wrong, but some are useful.” – George E. P. Box.