Investing: Man vs. AI Machine

GPT-4 takes on Human Equity Analysts - Which emerged triumphant?

Researchers at the University of Chicago’s Booth School of Business demonstrated that GPT-4, OpenAI's advanced language model, outperforms professional financial analysts in equity research tasks.

The results were a surprise because the researchers started with the premise that “Financial statement analysis is a broad task that is more of an art than science, whereas machines typically excel in narrow, well-defined tasks”. They knew that human beings account for broader context including knowledge of the industry, regulatory, political, and macroeconomic factors - something that they assumed a machine cannot do.

They were proven wrong!

This research may have significant implications for the future of financial analysis and decision-making, as it demonstrates the potential for large language models to augment and streamline the work of financial professionals.

But will it enhance or replace human expertise in complex financial domains?

The Research

The researchers provided GPT-4 with anonymized financial statements from Compustat, comprising 150,678 observations from 15,401 companies. They used two years of balance sheet and three years of income statement data. To evaluate GPT-4's predictive capabilities, they used two methods: a 'simple prompt' and a Chain of Thought (CoT) prompt. The simple prompt instructed GPT-4 to determine future earnings direction, while the CoT prompt involved identifying financial changes, computing ratios, and making predictions with rationale and confidence levels.

ChatGPT's predictions were compared to human analyst forecasts from the Institutional Brokers’ Estimate System (IBES) database. The evaluations were based on accuracy (percentage of correct predictions) and F1-score (harmonic mean of precision and recall).

Incredibly, GPT-4 achieved an accuracy of 60.4%, surpassing human analysts by 7 percentage points one month after earnings releases. The model also outperformed analysts in terms of F1-score, which balances precision and recall, with GPT-4 scoring 60.9% compared to 54.5% for human analysts.

One of GPT-4's significant advantages lies in its ability to handle and synthesize large datasets quickly. Unlike human analysts, who may be limited by time and cognitive load, GPT-4 can parse through extensive financial documents, news articles, and market reports to extract relevant insights. More particularly, the study found that GPT-4's algorithmic precision and consistency in evaluating financial metrics and trends contributed to its higher accuracy. This precision is partly due to the model's training on a diverse and comprehensive dataset, enabling it to recognize subtle patterns and correlations that might elude human analysts.

GPT-4's ability to perform real-time analysis may be a game-changer in the fast-paced world of finance. While human analysts require time to compile and interpret data, GPT-4 can deliver near-instantaneous insights, allowing for quicker pricing and decision-making.

Financial institutions can leverage GPT-4 to enhance decision-making processes, using AI to complement human judgment and provide a second layer of analysis. It is certainly far more efficient and can significantly reduce costs associated with human labour, potentially allowing firms to reallocate resources to other strategic areas.

The benefits may extend beyond the realms of the large financial firms. Smaller firms or individual investors would have immediate access high-quality research and analysis that had hitherto only been available to large financial institutions with large teams of analysts.

Conclusion

The study demonstrates that GPT-4 is not just a theoretical tool but a practical asset that can outperform human analysts in specific financial tasks. As AI continues to evolve, its role in finance and other industries will likely expand. Is this likely to take us on a path to achieving the efficient market hypothesis which, until now, has only been illusive text book theory? Is this good or bad? Investors tend to make money by exploiting market inefficiencies. I welcome thoughts of readers.

Source Reference: Financial Statement Analysis with Large Language Models by Alex Kim, Maximilian Muhn, Valeri V. Nikolaev