Stock Research for the Global Settlement: Qualitative or Quantitative Approaches?

Executive Summary

In order to help satisfy terms of the Global Settlement, quantitative stock research, primarily driven by computerized statistical analysis, is now actively competing with Wall Street’s traditional, qualitative research, largely driven by the analyst’s experience and insights. This raises some pressing questions:

• Which approach is preferred by individual investors?

• Which has the better track record historically?

• Which is better equipped to provide investors protection from conflicts and biases, a primary goal of the global settlement?

In a quest for answers, this paper relies on three studies:

Study #1. Investor survey of qualitative and quantitative approaches. We surveyed 1,203 individual investors and found that, although they leaned modestly toward quantitative approaches, they did not have a strong preference for a particular methodology or reporting format. Instead, the responses seem to indicate that their overriding desire is for a good track record to help them make money and protect their capital.

Study #2. Performance of qualitative and quantitative insurance company ratings. Although in a different field, the competitive battle among various insurance company rating firms in the early 1990s sheds light on today’s debate. Tracking their ability to warn of future insurance company failures, we find that:

• qualitative rating systems were accurate 13% of the time;

• quantitative rating systems were accurate 92.3% of the time; and

• one rating agency maintained both kinds of systems in parallel. Its quantitative ratings were among the most accurate in the industry, while its qualitative ratings were among the least accurate.

Study #3. Qualitative vs. quantitative stock ratings. Based on a similar study of stock research firms in 2002, we find that:

• qualitative methods were accurate in 9.1% of cases studied, and

• quantitative methods were accurate in 45% of the cases.

Discussion and conclusions: In both historical studies, quantitative methodologies greatly outperformed qualitative methodologies in protecting investors. However, the primary reason may not be intrinsic to the methodologies. Rather, these results may be due primarily to the fact that the greater subjectivity of qualitative methods makes the process more vulnerable to biases and conflicts from a wide variety of sources. Among others, these can include direct or indirect income from rated companies, other business ties with rated companies, personal securities transactions in rated companies, and bearish or bullish orientations in general.

In contrast, due to the greater discipline imposed by quantitative models, they are better equipped to act as a protective shield against such conflicts. In addition, quantitative models can:

• serve as a central clearing house for a wide range of inputs by both qualitative and quantitative analysts;

• help ensure consistent output among different companies, across industries, and over time; plus

• replicate the thought processes of a qualitative analyst with a series of decision-making trees.

Recommendations:

1. Qualitative researchers should be more firmly grounded in quantitative analysis.

2. Quantitative researchers could benefit by adding a layer of qualitative analysis.

3. Buyers of research should seriously consider not only a firm’s independence but also what procedures and guidelines are in place to guard against other biases that may seep into the ratings process.

4. Regulators should shed any prejudice they may still have regarding quantitative research, fostering an environment in which investors can gain access to ratings generated by a broad diversity of methodologies.

5. All producers and users of research must never ignore the strategic advantage of quantitative approaches in the role of capital preservation, without which long-term investing success is unlikely.

Introduction

Thanks to the Global Settlement, many issues with respect to stock research and ratings have been largely laid to rest:

We know that conflicts of interest on Wall Street create bias.

We know that bias can harm investors.

And we know that removing conflicts and bias from the research process is a paramount goal.

But there’s one big issue remaining: To achieve that goal, what kind of research do we really want?

• Do we want qualitative research that’s largely driven by an individual analyst’s years of experience and judgment?

• Or, do we want quantitative research, primarily driven by computerized statistical analysis?

The goal of this paper is to make progress toward some answers by highlighting the results of three separate studies that pertain directly to these issues:

• The first study is our survey of investor preferences;

• the second study is a performance review of qualitative vs. quantitative approaches in another industry that is directly relevant to this discussion; and

• the third is a performance review of qualitative vs. quantitative equity research in terms of its ability to protect investors from downside risk.

Each study represents merely a first step toward throwing new light on this vital issue.

Study #1. Investor Survey Regarding Qualitative vs. Quantitative Equity Research Methods and Reports

Which approach is better — qualitative or quantitative?

Naturally, if you ask most Wall Street research analysts, they’ll probably tell you they prefer the qualitative approach. No one wants to be replaced by a computer.

If you ask decision-makers at major Wall Street firms, you’ll likely get a similar answer. Qualitative research has been the standard for many years, and it is the research that many still seem to favor.

But what about individual investors, the ultimate users of the research? Has anyone asked them what they want? Actually, we have. Two weeks ago, we e-mailed a survey on this issue to over 24,000 retail investors.

The sampling techniques were not scientific. However, we were able to reach investors with a broad range of incomes and interests. We received 1,203 completed surveys, representing a 5% response rate.

In the survey, we asked each respondent to rank the importance of various quantitative and qualitative factors or features relating to two aspects of the research:

1. The researcher’s methodology for coming up with a recommendation, and

2. The research reports for communicating and explaining the recommendation.

For each of these two categories, we provided an even balance of six clearly qualitative approaches and six clearly quantitative approaches, listed in alphabetic order. Plus we asked two general questions, including one regarding the analyst’s track record. The text of the survey is in the appendix, and the average scores are in Table 1 below.

1. Investor Preferences Regarding Equity Research

(average scores of all respondents)

	Qualitative	Quantitative	General
Methodology	3.59	3.72
Reports	3.62	3.81
Track Record			4.07
Scale: 1 = viewed as least important; 5 = viewed as most important Data: Weiss Ratings, Inc. survey, February, 2004

 

Each respondent was asked to score qualitative and quantitative approaches on a scale of 1 to 5, with 1 representing “least important” and 5 representing “most important.”

In response to our first question — regarding the analyst’s methodology — the average score for qualitative factors was 3.59 compared to 3.72 for quantitative factors. Likewise, in response to the second question — on the content of research reports — the average score for qualitative information was 3.62 compared to 3.81 for quantitative.

Thus, the results show that, although there was a modest leaning toward quantitative approaches, respondents did not have a strong preference regarding either the methodology or the reports. Respondents did have a relatively stronger opinion, however, on another level: The highest average score of all — 4.07 — was recorded for track record, regardless of methodology used.

In effect, investors seem to be telling us: We don’t have a strong preference regarding how you reach an opinion on a stock or how you explain it. What we do care most about is whether or not your opinions are right or wrong.

In other words, the inference we derive from this survey is that individual investors do not strongly favor any particular methodology or any particular reporting scheme. Their overriding concern is to make money and protect their capital.

With that in mind, let’s narrow the debate down to two issues:

Issue #1. Which approach — qualitative or quantitative — is less prone to bias and conflicts?

Issue #2. Which approach is more likely to provide better performance overall for investors, helping them not only to grow their wealth but also to avoid serious losses.

To help address these issues, we conducted two historical studies. In each study, we compared qualitative and quantitative ratings systems. And in each, we asked the question: Which rating approach had the better track record in protecting consumers or investors from downside risk?

We recognize that downside risk is only one side of the equation. But we feel it is an extremely important one. Almost any research can get passing grades in stable times or in a rising market. We believe that the tougher test is how research holds up in unstable times or declining markets. Needless to say, investors can generally be successful even if they make less money. But few can recover if they suffer devastating losses.

Our next study is in a different field but with direct relevance to this topic.

Study #2. Performance Review of Qualitative vs. Quantitative Approaches To Insurance Company Ratings

Much like today, the field of insurance company safety ratings was in flux in the early 1990s, with two competing approaches: A qualitative approach and a quantitative approach. And much like today, there was heated debate regarding which would provide better overall performance. Five major U.S. rating agencies were in the field, divided as follows:

• Three of the agencies used predominantly a qualitative approach. These agencies argued that analysts must talk to management, must review a series of unquantifiable factors, and must allow each analyst’s intuitive feelings about a company’s prospects to play a significant, often overriding, role in forming an opinion.

• One of the agencies used primarily a quantitative approach, with qualitative aspects considered mostly in exceptional situations. This agency argued that too much leeway for judgment by the individual analyst could potentially prejudice the results, even in the absence of conflicts of interest.

• Plus, there was also one other well-established agency that had two separate rating systems — one predominantly qualitative and one purely quantitative. We will refer to this agency as “Agency D.”

In 1994, the U.S. General Accounting Office (GAO) published a landmark study comparing the ratings performance of all five of the rating agencies1. However, at the time the GAO initiated its study, it did not have adequate data on the quantitative ratings of Agency D. Therefore it covered strictly its qualitative ratings.

Subsequently, Weiss updated the GAO study, using the same methodology as the GAO, but, this time, including Agency D’s quantitative ratings as a separate item. Thus, the Weiss study covered the performance of six ratings systems overall.

Focusing on large failed companies, the GAO’s basic performance metrics revolved around two questions:

• Did the agency provide adequate warning to consumers of subsequent insurance company failures?

• With how much lead time?

The results of the GAO and Weiss studies are summarized in Tables 2 and 3. Between the two studies, there were 11 large companies that failed, covered by the six ratings systems.

Qualitative insurance company ratings

Table 2 covers the four qualitative systems. The figures shown are the number of days between the date of failure and the date each agency issued its first warning or “vulnerable” rating, as defined by the GAO. Negative numbers indicate the number of days before the failure. Positive numbers indicate the number of days after the failure.

As an illustration, Agency A issued a warning six days before the failure of Company 1, one day after the failure of Company 2, two days after the failure of Company 3, etc.

2. Qualitative Ratings Assigned to Large Failed Insurers

Number of days between issuance of first warning and failure.

Minus sign = warning issued before failure. Plus sign = warning issued after failure

Large Failed Companies	Agency A	Agency B	Agency C	Agency D
Company 1	-6	-422	-41	-190
Company 2	+1
Company 3	+2			-6
Company 4	+5			-3
Company 5	Never			+351
Company 6	+3	+2		Never
Company 7	0		+2	+10
Company 8	0		+2	+10
Company 9	+5
Company 10	+4
Company 11	+3
Adequate Advance Warning	0%	50%	33%	14%
Data: U.S. General Accounting Office, rating agencies, state insurance commissioners

 

We define “adequate advance warning” as a warning issued at least one week prior to failure. It is assumed that less than one week is inadequate time for the warning to be published in major venues, disseminated to the public and acted upon.

Based on this definition, Agency A did not issue an adequate warning for any of the failed companies. Agency B issued a warning in one out of two of the companies it covered, Agency C issued a warning in one out of three, and Agency D’s qualitative system issued a warning in only one out of seven cases.

Please note that, even if we define advance warning as any time before the date of failure, the results do not improve significantly. Also note that on two occasions, an agency dropped coverage of the companies around the time of the failure and never issued a warning.

Quantitative insurance rating systems

Table 3 summarizes the results of the two quantitative rating systems. Agency D’s quantitative system issued an adequate warning in three out of three of the cases and Agency E, in nine out of ten of the cases.

3. Quantitative Ratings Assigned to Large Failed Insurers

Number of days between issuance of first warning and failure

Minus sign = warning issued before failure

Plus sign = warning issued after failure

Large Failed Companies	Agency D*	Agency E
Company 1		-379
Company 2		-372
Company 3		-308
Company 4		-617
Company 5		-162
Company 6		-40
Company 7
Company 8		+6
Company 9	-74	-134
Company 10	-994	-1152
Company 11	-228	-621
Adequate Advance Warning	100%	90%
* Agency D ceased publication of its quantitative ratings once qualitative ratings were published on the same company. Data: U.S. General Accounting Office, rating agencies, state insurance commissioners.

 

4. Qualitative vs. Quantitative Safety Ratings

Assigned to Large Failed Insurers

Methodology	# of ratings	# correct	% accuracy
Qualitative	23	3	13.0%
Quantitative	13	12	92.3%

 

The overall results of this study are summarized in Table 4. Counting the total number of accurate ratings in comparison to the total number of ratings issued, we find that:

• The qualitative rating systems were accurate in three out of 23 cases, or 13% of the time, while

• The quantitative rating systems were accurate in 12 out of 13 cases, or 92.3% of the time.

Two additional factors support this pattern:

Larger sample: Not included in the tables above is a larger sample of failed companies discussed by the GAO. Using a sample of 30 failed companies which were exact cohorts (rated in common and simultaneously by both agencies), the GAO found that the quantitative agency was first to warn investors of pending troubles in 23 cases and the qualitative agency was first in 7 cases. Thus the quantitative approach beat the qualitative by a three-to-one ratio.

In response, the qualitative firm requested that its ratings scale be redefined so that its “B” and “B-” ratings (labeled “good” in its literature) could be construed as “vulnerable” and considered an adequate warning to investors. The GAO declined to accept this redefinition, but stated that if it did, the quantitative agency would still have eclipsed the qualitative agency by a factor of two to one.

Agency D: The agency that had both qualitative and quantitative ratings systems is an especially interesting case. Both of its ratings systems were running under one roof. We can generally assume, therefore, that its analysts were potentially subject to the same or similar conflicts of interest at the firm level. And we can also assume that they had access to, or were hampered by, essentially the same strategic advantages or disadvantages the firm might have had in terms of its resources or knowledge base. Agency D had a policy of ceasing publication of its quantitative ratings as soon as a qualitative rating was published on the same company. Therefore, based on publicly available information, it is impossible to compare identical universes of companies rated in common by both of the agency’s systems. However, the record shows that:

• The agency’s quantitative ratings were largely accurate. They provided ample advance warning of future difficulties for all three of the large companies that subsequently failed. Additional data not included in this paper further support that conclusion.

• In contrast, the agency’s qualitative ratings were largely inaccurate. They provided advance warnings for only one out of seven of the large companies that subsequently failed.

• Among the six ratings systems, Agency D’s qualitative ratings appear to have one of the worst track records, while its quantitative ratings appear to have one of the best track records.

Study #3. Performance review of qualitative vs. quantitative equity research with respect to downside risk

In reviewing the historic performance of various stock ratings systems, there’s one issue that seems to be in danger of falling through the cracks: Survivorship. The concern is that independent research providers or data aggregators may be purging the ratings history for those companies that have filed for bankruptcy and ceased to be traded on major exchanges. This is unfortunate for three reasons:

• First, there were a large number of corporate failures in the first three years of the new century.

• Second, since corporate failures typically wipe out 100% of the shareholder’s capital, they can have a significant impact on overall portfolio performance.

• Third, from an academic point of view, bankruptcy is a definitive event which provides a solid testing ground for the accuracy of the ratings in general, and for sell signals in particular. Although this is conceptually different from the metrics currently under discussion, we believe it sheds a new perspective on the debate.

Qualitative Stock Ratings

Table 5 is based on a study of companies that failed in the first four months of 2002 and that were covered by at least one qualitative firm2.

5. Qualitative Stock Ratings on Companies Filing for Chapter 11 Bankruptcy

(includes all covered companies filing for bankruptcy in first four months of 2002)

	On Date of Bankruptcy Filing		6 Months Before Bankruptcy Filing
Ratings Issued	# of Ratings	%	# of Ratings	%
“Buy” or equivalent	38	31.4%	67	55.4%
“Hold” or equivalent	72	59.5%	49	40.5%
“Sell” or equivalent	11	9.1%	5	4.1%
Total	121	100.0%	121	100.0%
Source: Crisis of Confidence on Wall Street, Weiss Ratings, Inc., July 11, 2003, http://www.weissgroupinc.com/research/crisis_of_confidence.html

 

Of the 121 ratings available from major public sources six months prior to failure, a total of 95.9% were “buy,” “hold” or equivalent, while only 4.1% were “sell” or equivalent.

By the date of failure, the percentage of “buy” and “hold” ratings available from major public sources had declined to 90.9%, while the percentage of “sell” ratings had risen to 9.1%.

In short, as we saw in the previous study, the qualitative ratings failed to issue warnings for approximately nine out of every ten companies that subsequently failed, even up to the very day of the failure.

Quantitative Stock Ratings

Table 6 is based on a study of companies that failed in the same period and that were covered by at least one quantitative firm for which data are available.

6. Quantitative Stock Ratings on Companies Filing for Chapter 11 Bankruptcy

(includes all covered companies filing in first four months of 2002)

	On Date of Bankruptcy Filing		6 Months Before Bankruptcy Filing
Ratings Issued	# of Ratings	%	# of Ratings	%
“Buy” or equivalent	1	5%	1	5%
“Hold” or equivalent	10	50%	9	45%
“Sell” or equivalent	9	45%	10	50%
Total	20	100%	20	100%
Data: Bank of New York Jaywalk

 

Of the 20 ratings available six months prior to failure, half were “buy,” “hold” or equivalent, while half were “sell” or equivalent.

By the date of failure, the percentage of “buy” and “hold” ratings available rose slightly to 55%, while the percentage of “sell” ratings declined slightly to 45%.

7. Qualitative vs. Quantitative Stock Ratings

Assigned to Failed Corporations on Date of Failure

Methodology	# of ratings	# of sells	% accuracy
Qualitative	121	11	9.1%
Quantitative	20	9	45.0%

 

The total number of available ratings meeting the criteria for this study was relatively small. However, the performance differences between qualitative and quantitative ratings systems, summarized in Table 7, are large enough to be statistically significant.

We find that:

• The qualitative rating systems were accurate in 11 out of 121 cases, or 9.1% of the time, while

• The quantitative rating systems were accurate in 9 out of 20 cases, or 45% of the time.

Discussion and Conclusions

This gives us two studies, each in a different industry and a different time, leading to a very similar conclusion: Historically, quantitative ratings have been far better at warning of future downside risk than qualitative ratings. This is strikingly clear in Table 8, which summarizes the results of the two studies.

8. Accuracy of Qualitative vs. Quantitative Ratings In Two Different Industries

(warnings or sell ratings issued as a % of ratings issued on all failed companies)

Methodology	Insurance Ratings	Stock Ratings
Qualitative	13.0%	9.1%
Quantitative	92.3%	45.0%

 

The data indicate that …

• in the insurance ratings area, quantitative methodologies were 14 times more accurate than qualitative ratings in warning of downside risk, while

• in the stock ratings business, quantitative methods were nearly five times more accurate.

These results, by themselves, do not necessarily prove that quantitative methodologies are intrinsically superior to qualitative ratings. This is because, historically, in both the insurance and stock research fields, qualitative methodologies have been more closely associated with serious conflicts of interest, while the quantitative methodologies have been largely the domain of independent firms.

Therefore, one might argue that there’s nothing particularly wrong with qualitative methodologies — and they may even be superior — as long as one can remove the conflicts and biases.

But therein lies the great dilemma. Qualitative approaches, by relying too heavily on personal judgment and experience, can open a Pandora’s box of potential conflicts and biases. Let’s review just a few forms these can take:

1. Bias from direct payments. The income of the firm or the research analyst is tied directly to the results of the rated company.

2. Business bias. The firm has other types of business ties with the rated company, such as consulting, credit ratings contracts, shared interests in other ventures, cross ownership of stock, directors in common, etc.

3. Bias stemming from security transactions. The firm or the analysts buy, sell and hold securities in the rated companies.

4. Prestige bias. The prestige of the firm or the analysts is directly or indirectly tied to the results of the rated company.

5. Preference bias. The analysts or the firm have special likes or dislikes for particular companies, based on past relationships and experiences, or based on current interactions in the procurement of information.

6. Bullish/bearish bias. The analysts or the firm are influenced by their bullish or bearish views of the future. These, in turn, can be driven by their own commitments to personal finances or business strategies.

7. Intellectual bias. The analyst develops an intellectual commitment to previously published opinions and is reluctant to change it, typically out of concern that changes may be interpreted as an admission of error or a lack of conviction.

8. Emotional bias. The analyst is emotionally driven by a particular fear or hope, such as fear of reprisal or hope for some future favors.

Clearly, independence alone does not guarantee the removal of conflicts and bias. Nor would it seem practical to establish firewalls, rules and guidelines that could be efficient enough to prevent these biases from seeping into the ratings process.

This is the overriding advantage of quantitative approaches. Although bias is certainly still possible in the construction of a quantitative model, the model can serve as a powerful shield that protects the ratings process from the push and pull of conflicts and bias.

Looking back at the historic examples we have presented here, we must recognize that there was a chicken-and-egg reason for the link between conflicted processes and qualitative methods. Likewise, there was a reason for the link between independence and quantitative methods. Specifically:

• Conflicted and biased firms or analysts may prefer qualitative approaches precisely because it gives them greater latitude for influence and power over the ratings process. And over time, qualitative approaches are more apt to open the door to bias and conflicts as people naturally seek to use the power available. In contrast,

• Many independent and objective firms or analysts generally prefer quantitative approaches for the simple reason that they do not want that power. And over time, the quantitative discipline provides little or no leeway for the firm to evolve conflicted relationships with rated companies.

The discipline typically imposed by quantitative models also provides other advantages:

Advantage 1. A quantitative model helps ensure consistency in the application of various metrics among many analysts, across many different companies, and over time. In nature and as in economics, nearly all situations and events, no matter how rare, are generally recurring situations and events. Quantitative models are far better equipped at evaluating them consistently and even-handedly, whether they’re used by one analyst on this company today or by another analyst on that company tomorrow.

Advantage 2. A quantitative model can include not only hard financial data but also “soft” information normally associated with qualitative approaches. By using various methods of scoring these qualities, the analyst or model builder can convert these into numerical values and compare them quantitatively.

Advantage 3. A quantitative model can act as a central clearing house for the collective input and wisdom provided by a wide variety of analysts and specialists. For example, analysts can be assigned to contribute and maintain industry-specific modules which are dedicated to the special circumstances of its companies, products and marketplace.

Advantage 4. A quantitative model need not be a black box containing a melting pot of formulas. It can and should be built with decision-making trees that seek to replicate the rational thought processes of live analysts.

Is there a role for a qualitative layer?

Provided a quantitative model serves the role of providing adequate discipline to protect the ratings process from conflicts and bias, qualitative research can add value to the results in the following circumstances:

1. Model building and maintenance. The task of building an accurate model is not strictly a quantitative one. As in any scientific endeavor, the model builder must develop and test hypotheses by combining intuitive as well as statistical skills. In this sense, qualitative analysis plays a significant role in the building of a quantitative model.

2. Exceptional situations. Quantitative models should contain a mechanism for recognizing exceptional situations that are outside defined ranges. Specifically, the model should generate an exceptions report, which the analyst can then analyze manually using both quantitative and qualitative approaches. Once the manual analysis is completed for these companies, however, the analyst should return to the model to establish parameters for handling similar exceptions that may appear in the future.

3. Recent events and changes. If there is a broad, fundamental change — say, a major regulatory event in a particular industry, analysts may not have adequate time to modify the quantitative model before the next round of ratings is due for publication. Therefore, they may have to manually adjust the output for impacted companies until such time as the model can be updated.

4. Large, complex companies. The unique circumstances of large, multi-faceted, conglomerates are sometimes not easily captured by quantitative models whose primary goal is to provide the broadest possible coverage of companies of all sizes. Although it may be possible to construct single-company quantitative modules within a broad-coverage model, we are unaware of any research organization that has done so at this time. However, for the most part, the needed information about these companies is embedded in the consensus estimates for the company and in the current stock trends, both of which can easily be included in current quantitative models.

Recommendations

All research organizations, whether independent or not, are subject to pressures that can lead to bias. Therefore, one of the key goals of all parties must be to establish procedures that can best protect the research from those pressures.

Most quantitative researchers could benefit by adding a qualitative layer to their research, e.g., through the use of an exceptions report. In addition, we believe they should

• be careful to remove bias from the model-building process; and

• make every effort to evaluate and score nonfinancial information, converting it into data that can be measured in a statistical model.

Most qualitative researchers could probably benefit from better grounding in quantitative analysis. To the degree that they wish to retain direct personal control over ratings, they need to

• take great care to remove bias from the ratings process, with a state of high awareness and sensitivity to bias at every stage of the process;

• establish narrow and strict statistical parameters for the individual analyst. To the degree that these are tightly and uniformly enforced, significant progress can be made to reduce bias while still allowing each analyst freedom to form an independent, individual opinion.

• recognize that bias can enter the ratings process from many directions and at many stages. In the absence of a quantitative model, each of these must be continually and consciously monitored.

All research firms should leverage the unique strengths of both qualitative and quantitative approaches; both computers and live analysts. In this regard, we believe that

• the best use of computer power is to scan a wide range of data to evaluate the overwhelming majority of companies most of the time. Conversely,

• the best use of the analyst’s time and intellect lies within the process of recommending or making evolutionary improvements to model, while continuing to evaluate companies and situations that show up on an exceptions report.

Brokerage firms and other buyers of stock research should continue to pay close attention to factors that may reduce or increase bias. Since a primary goal of the global settlement is to avoid repeating the mistakes of the past, one key factor buyers of research should consider is the steps that research providers have taken — not only to maintain their independence, but also to guard against other biases that may creep into their research, whether they have a qualitative or a quantitative orientation.

Regulators, for their part, should:

• avoid favoring traditional methodologies or reports. As our survey has shown, it may not necessarily be what the investor favors. Plus, as is evident from our two historical studies, traditional methodologies may be more closely linked to past problems than is commonly realized;

• encourage investor access to the widest possible diversity of methodologies and approaches, without prejudice; and

• recognize that quantitative methods can play a critical role in removing or reducing conflicts and biases, a fundamental goal of the Global Settlement.

Finally, all producers and users of research, including individual investors, must not ignore the strategic advantage of quantitative approaches in the role of capital preservation, without which long-term investing success is unlikely.

Appendix: Weiss Survey

Step 1. Stock analysts use various methods to come up with a “buy,” “sell,” or “hold” recommendation on a stock. Plus, each analyst may have special strengths or weaknesses in their approaches. In order to give you the confidence you feel you need to act on their recommendations, some of their methods and strengths may be more important to you, and some may be less important. We’ve listed a few below (in alphabetical order). Please rank them according to your view of their importance, with “1” being the least important and “5” being the most important.

	Not Important			Very Important
Analysis of data on capital, debt, and other balance sheet items	1	2	3		4	5
Analysis of data on cash flow	1	2	3		4	5
Analysis of data on sales, profits and losses	1	2	3		4	5
Communication with company’s management	1	2	3		4	5
Computer analysis of industry data	1	2	3		4	5
Opinions and forecasts of future sales and earnings	1	2	3		4	5
Ideas and insights about the company	1	2	3		4	5
Ideas and insights from non-public sources	1	2	3		4	5
Knowledge and experience in the industry	1	2	3		4	5
Knowledge of the company’s unique characteristics	1	2	3		4	5
Recency of data from public sources	1	2	3		4	5
Statistical screens of many companies	1	2	3		4	5
Track record of analyst, regardless of methods used	1	2	3		4	5

 

Step 2. The analysts typically write up their conclusions in a research report on each stock. However, these reports can vary widely in terms of the actual information and features they contain. Below, please rank the information and features you would like to see in a stock research report, according to their importance, with 1 being the least important and 5 being the most important.

	Not Important			Very Important
Analyst’s insights based on personal knowledge and experience	1	2	3		4	5
Analyst’s opinion and forecasts of future sales and earnings	1	2	3		4	5
Data about sales, profits and losses	1	2	3		4	5
Data about the company’s cash flow	1	2	3		4	5
Data from the balance sheet such as capital and debt	1	2	3		4	5
Information about (or derived from) the company’s management	1	2	3		4	5
Numerical comparisons of various companies	1	2	3		4	5
Recommendation to buy, sell or hold	1	2	3		4	5
Review of ideas and insights from non-public sources	1	2	3		4	5
Statistics about the industry	1	2	3		4	5
The analyst’s insights of the company’s unique characteristics	1	2	3		4	5
The analyst’s view of the company’s strengths or weaknesses	1	2	3		4	5
Up-to-date or recent data from public sources	1	2	3		4	5

 

[1] Comparison of Private Agency Ratings for Life/Health Insurers, U.S. General Accounting Office, GAO/GGD-94-204BR, September 1994. Also available at http://www.weissratings.com/gao_study.asp

[2] For a larger sample of companies, see, Why Reforms Don’t Adequately Protect Investors, Weiss Ratings, Inc., May 2, 2003. http://www.weissratings.com/settlement.asp