Can You Use ChatGPT to Create a Portfolio of Stocks?

by Charles Rotblut | August 17, 2023

Featured Tickers: AMZN
JNJ
MSFT

A new study looks at using ChatGPT to build a portfolio of stocks. While the authors claim some success, the portfolio-creation process was the result of giving the artificial intelligence (AI) platform the same prompts over and over. Plus, the sample period and the universe of stocks analyzed were small.

I’m going to delve more into the study in a moment but before I do, I want to share some of my experiences with ChatGPT.

First, the dataset in ChatGPT-4 (currently the most advanced version) only goes up to September 2021. Any information it can give you about a company will be dated. Second, I’ve found that it no longer reads and summarizes third-party websites. This is a shame because doing so could provide a quick summary of U.S. Securities and Exchange Commission (SEC) filings and earnings call transcripts.

Universe of stocks picked by ChatGPT

The third thing is the most important: I’ve seen it “hallucinate.” This is the terminology used to describe how ChatGPT makes things up. For example, when I gave it a list of stocks from our VMQ Stocks website and asked the AI platform to rank the list of stocks from best to worst based on their assigned Value, Momentum and Quality Scores, ChatGPT created both fictional stocks and fictional scores for them. “Well, ChatGPT is a generative language model,” responded a friend of mine after I shared my experience with her.

For the aforementioned study, researchers used this prompt in ChatGPT: “Using a range of investing principles taken from leading funds, create a theoretical fund comprising of at least X stocks (mention their tickers) from the S&P500 with the goal to outperform the S&P500 index.” The X in this prompt referred to 15, 30 or 45 stocks. ChatGPT gave 15 to 20 stocks, 30 to 35 stocks and 45 to 50 stocks.

The study’s authors “sent each request 30 times to capture a broad range of responses.” They then updated the list of stocks every time and measured the frequency with which stocks were mentioned. Only Microsoft Corp. (MSFT), Amazon.com Inc. (AMZN) and Johnson & Johnson (JNJ) were included in all 30 of the 15-stock portfolios generated by ChatGPT, even though the prompt did not change. (It should go without saying that humans don’t need the help of AI to be indecisive.)

After constructing portfolios based on the frequency at which stocks were named, the study’s authors asked ChatGPT to weight those stocks in a portfolio. It created portfolios that were diversified across various sectors. When asked for the rationale it used to create those weighted portfolios, the AI model responded by saying it considered diversification, market capitalization, growth potential and stability, among other factors.

The ChatGPT-weighted portfolios went on to outperform the S&P 500 index over the period of September 1, 2021, through July 31, 2023. The portfolios’ outperformance was accompanied by greater volatility and a larger drawdown relative to the large-cap index.

There are a few things to note. The period analyzed was very short—too short to draw any conclusions. The portfolio itself was created after ChatGPT was given many prompts and researchers selected the stocks named with the greatest frequency. It’s unclear how the AI-generated portfolio would have performed if the prompt was only run once or 100 times. An element of luck is also involved.

Based on this and my personal experiences with the AI model, ChatGPT should not be relied on as a tool for making investment decisions in its current state of development. Its propensity to make things up is a big problem. The study itself looked at a narrow range of stocks with no attribution to how market-cap size may have affected the results. Most importantly, the 23-month time horizon is too short to make an assessment about any strategy.

More on AAII.com
Participate

Members are looking for your input. Can you help with this question from the Mutual Funds & ETFs Community?


“In a recent article for the AAII Journal, we spotlighted the technology sector’s recent rebound. Considering the rapid rebound in the first half of 2023 after the significant drop in 2022, how, if at all, are you adjusting your tech exposure?”


Answer This Question in the AAII Community »


Tap the button and then choose the Join the Community button on the right to answer this question in the AAII Community




AAII Sentiment Survey

Bullish sentiment decreased, ending its 10-week streak of above-average readings in the latest AAII Sentiment Survey. Both neutral and bearish sentiment increased.

Bullish sentiment, expectations that stock prices will rise over the next six months, decreased 8.8 percentage points to 35.9%. After 10 consecutive weeks of above-average readings, optimism is below its historical average of 37.5%.

Neutral sentiment, expectations that stock prices will stay essentially unchanged over the next six months, increased 4.2 percentage points to 34.0%. Neutral sentiment is above its historical average of 31.5% for the third time in 11 weeks.

Bearish sentiment, expectations that stock prices will fall over the next six months, increased 4.6 percentage points to 30.1%. At 11 consecutive weeks, this is the longest that pessimism has been below its historical average of 31.0% since a 23-week streak from February to July 2021.

The bull-bear spread (bullish minus bearish sentiment) decreased 13.4 percentage points to 5.8%. This is the first time in 11 weeks that the bull-bear spread is below its historical average of 6.5%.

This week’s special questions asked AAII members how they would describe the current economy. Here are the responses:

  • Strong: 13.8%
  • Mixed with areas of strength and weakness: 69.2%
  • Weak: 15.9%
  • Not sure/no opinion: 1.2%

This week’s Sentiment Survey results:

Bullish: 35.9%, down 8.8 points
Neutral: 34.0%, up 4.2 points
Bearish: 30.1%, up 4.6 points

Historical averages:

Bullish: 37.5%
Neutral: 31.5%
Bearish: 31.0%

See more Sentiment Survey results.



Discussion

Rob from NC posted over 2 years ago:

AI has a very long way to go to become truly "intelligent."


Dennis from Washington posted over 2 years ago:

Something I learned programming software 50 years ago "Garbage in, garbage out" is still true today. The output of a program, even a generative AI program is no better than the quality of the input. Expecting a program to scrape unauthenticated and un-curated data randomly from the internet and getting something useful is a fool's errand. I have played with ChatGPT a bit and could only get meaningful results if I used very narrowly defined queries that would only reference authenticated sources. And even then, I was usually disappointed at the limited results. The version of ChatGPT that we are allowed to play with for free is just not sophisticated or "smart" enough to know useful data from garbage or fiction. So, I learned I cannot throw my stock analysis Excel spreadsheets away just yet, but maybe I can use them to train the software...


John L from NJ posted over 2 years ago:

I am most impressed by the smart people who come up with the nice sounding excuses for generative language models. Hallucinate is so much better sounding than "pathological liar". But it doesn't matter. Would you hire someone who suffers from hallucinations or is a pathological liar to invest your funds. I think AI is over hyped. Both as a technology and as an investment theme.


gregg w from california posted over 2 years ago:

SORRY - the comment system removes all line breaks! If you want to read a longer explanation of why ChatGPT is more like a parrot than an intelligent Excel spreadsheet, read https://pastebin.com/5ZE3JqgM The biggest danger to "AI" in general, and ChatGPT specifically, is how it's marketed and perceived. it's basically a deep fake of human intelligence as defined by the contents of electronic documents, but people don't realize how fake it is.


Barry from TX posted over 2 years ago:

ChatGPT is to "intelligence" as ... Crypto is to banking security ... and The Internet is to privacy. All their promises are GIGO FUBAR. Every great "invention" has been implemented as BOTH creative/progressive and destructive/repressive forces - fire, civilization, commerce, religion, printing, markets, political parties, nuclear fission/fusion, the internet, etc. There are no reasons to expect that AI will be different. A lot of people are going to get rich and even more will lose their fortunes by investing in AI. That is the nature of investment markets.


Barry from TX posted over 2 years ago:

Maybe I was too hard on ChatGPT. I should have redirected my trash-talking to the known deficiencies in current investing industry practices. In their Abstract, the authors of the background article say, "TRUSTING investment advice from Generative Pre-Trained Transformer (GPT) models is a challenge due to model "hallucinations", necessitating careful verification and validation of the output." That quibble makes it fair game to ask, what is the difference between this disclaimer of their experimental results and any pro forma investment industry disclaimers like (1) "Past performance is no guarantee of future results" (a sample from a broker I use) and (2) "Deeper analysis should always be completed before making investing decisions" (a sample from an AAII offering). Relative to claim #1, ChatGPT "hallucinations" sound very similar to the pro forma "results" backtesting and model portfolios produce -they are presented without any guarantee of a linkage to real-world results you should expect. Relative to claim #2, the ChatGPT data in the Figure in the AAII review from the study "advises" all investors to rely on their own "due diligence" process rather than implement a list of the outputs from any bot ... or ... from a fund portfolio manager. #2 sounds like a distinction without a difference. Trying to tell if you are interacting with a computer or a “sentient” person goes back 75 years to the birth of computing. 'The father of modern computing', Alan Turing begins his 1950 paper, "Computing Machinery and Intelligence," with the question, 'Can machines think?' Turing sidestepped the challenge of defining "thinking" and created a very practical, albeit subjective, test called the “Imitation Game,” now known as the Turing Test: A human interrogator – who communicates with a computer only by exchanging messages -- decides if they are interacting with a computer OR a human subject based on the replies the interrogator poses. By means of a series of Turing tests, a computer’s success at “thinking” can be measured by its probability of misidentifying the computer as a human subject. The advent of ChatGPT reignited the conversation about the likelihood that the components of the Turing Test had been met. The referenced article does not resolve Turing’s 75-year-old question. AAII member comments above, including mine, suggest that ChatGPT is closer to “a blind chimpanzee throwing darts in the dark” than it is to the typical active fund manager who cannot beat the market index his fund benchmarks.


Barry from TX posted over 2 years ago:

Charles, your prescient topic started an avalanche of data searches for me. So far I have placed the whole "AI is the next big thing" that will change the world in the same trash-talking bin as Blockchain and crypto. Unlike those "hallucinations" that have been reigned in somewhat by the SEC, Generative AI promises to be truly disruptive on a much larger human scale. It may be far off (?), but it is a mile-long freight train approaching an RR crossing in many neighborhoods. Here is a friendly link to an article in today's WSJ that has two charts that project the impact of Generative AI on (1) educational attainment and (2) employment (by industry). The article has additional data points from research by Goldman Sachs, Morgan Stanley, McKinsey, and MIT. https://www.wsj.com/articles/generative-ai-promises-an-economic-revolution-managing-the-disruption-will-be-crucial-b1c0f054?st=m2298ou73kmh5zc&reflink=desktopwebshare_permalink Net: An average of 40%-70% of various educational degrees will be impacted (McKinsey) and 60%-64% of jobs in most industries (Goldman Sachs). The projections are extrapolations and speculations, but they are data points from reputable sources outside the AI industry which is prone to its own human "hallucinations." If these data points are anywhere near accurate, the big questions are (1) What /where will other jobs for these millions of dislocated people come from? and (2) How can colleges justify the exorbitant fees they charge to educate (there is a separate WSJ article on the topic of rising costs of student debt) people for careers/ jobs that will not exist? And my favorite, (3) how can I as an investor be a willing "yet to be named co-conspirator" getaway driver in this "bank heist"? As prior comments point out, we should expect a lot of hype from the AI crowd. They tend to lean over the tips of their skis. And they are a kiddie slope crowd on a Triple Black Diamond trail.


You need to log in as a registered AAII user before commenting.
Create an account

Log In