The Bureaus are going to a common system here shortly, replacing the various Beacon and FICO metrics (there are several different kinds or score, each focuses on different factors and is used by different clients for different purposes). With the new single metric the only difference in scores should be caused by discrepencies in what is reported to each Bureau.
As far as it being "secret", they don't publish the exact weight of each factor but if you pay your bills on time, maintain the same 3-4 open credit lines for several years with no lates and don't carry a balance over 20% of each card's limit, you will have a score above 700, which is golden. It isn't rocket science.
Since you can get free reports showing all content, blaming the Bureaus because you haven't checked for fraud is a little disingenuous. I assume everyone checks their bank statements for errors every once in a while.
And, like it or not, just like actuarial science on insurance has improved since its beginnings in London 500 years ago, credit scoring has in fact been demonstrated to have statistically significant correlation to other personal risk. It isn't just "convenience" that makes insurance companies use it, it's proven past performance.
In any event, if you want to have a live underwriter calculate your loan, be my guest. here's what happens, they take every scrap of paper from your past 2 or 3 year history , chase down any irregularities and in the end apply the strictest standards of whatever investor is actually supplying the money, with very little compensating factors allowed. The reason? If they get it wrong and you default the mortgage company has to buy back the loan.
Conversely, if you have a credit score, I plug your info, including credit history, into the investor's Automated Underwriting System, it looks at all the info I provide it and makes the decision.
If it comes back "refer" we can go to hand underwriting. If it comes back "approve" we're done, no matter how weird the loan looks on the surface. in some cases, instead of 3 years of taxes and bank statements I get your most recent pay stub, your most recent account statements and we are ready to close.
Astonishingly, both ways have almost identical failure rates, but one is just much faster and efficient.