Sign up here to receive our free Work Tech newsletter in your inbox.
Charter Work Tech reviews—including our picks and the products we choose to review—are editorially independent. If you purchase something through a link on our site or in our newsletter, Charter may earn an affiliate commission.
Advanced Data Analysis, a ChatGPT plugin developed by OpenAI, performs tasks like data analysis by running computer code in response to prompts given in plain language. Organizations can use it for a variety of purposes, such as generating graphs for marketing reports or analyzing financial data.
As a reporter covering artificial intelligence in the workplace, I’m always on the lookout for new ways to use the technology for my own work. So my interest was piqued when I learned in a conversation with Rebecca Hinds, head of The Work Innovation Lab at Asana, that her team used Advanced Data Analysis, previously called Code Interpreter, to help them analyze data for their new report on AI and work.
To see how it might help me interpret data for an article I was writing, I uploaded a dataset with information about job postings and started asking it some questions. I quickly saw how much the tool lowers the barrier to entry for data science: It let me make tables and graphs, edit files and convert them to different formats, and run statistical tests. Rather than memorize specific commands, I could just tell it what I wanted.
Genuinely impressed with what I saw, I tested Advanced Data Analysis on a handful of additional datasets to see how it performed on tasks that would be useful in a business setting. Here are some of the experiments I ran and the lessons I learned.
Background and privacy
When you ask Advanced Data Analysis to do something, it runs the command in Python, a programming language. The tool is currently a beta feature available to users of ChatGPT Plus, which costs $20 per month—you can turn it on in settings under “Beta features.” You can also access it through OpenAI’s new ChatGPT Enterprise plan (inquire for pricing).
If you access Advanced Data Analysis through ChatGPT Plus rather than ChatGPT Enterprise, you should refrain from sharing sensitive data with it, because the chatbot improves by training on the conversations it has with people. You can’t access Advanced Data Analysis while your chat history is disabled.
According to OpenAI, the ChatGPT Enterprise plan, which also offers the Advanced Data Analysis plugin, does not train on conversations.
I used a handful of different datasets to test out Advanced Data Analysis:
- A smaller dataset about jobs postings, shared by workforce intelligence company Revelio Labs.
- A smaller dataset from a recent survey by Charter about workers’ views toward AI.
- A few larger datasets from this list of public datasets.
- The summary table from WeWork’s S-1.
- The summary table from Google’s S-1.
Something you’ll notice when using Advanced Data Analysis is that the plugin seems to have a pretty good ‘understanding’ of what it’s looking at. When I gave it the job postings file, for example, it knew what it was about without me providing any additional context, and it was able to infer what abbreviations in the dataset stood for (e.g., it figured out that “postingcount_ai” stood for the number of job postings related to AI for the specific job role in the given month.)
One of the most useful features of Advanced Data Analysis is its ability to quickly find facts in a sea of data, essentially making it a very powerful Command F tool (though it’s capable of much more). Using the plugin and the Revelio Labs job postings dataset, I was able to ask questions like, ‘What are the five job categories whose job postings most often mentioned AI in July?’ and it would generate the answer in about 10 seconds. You could find this answer with traditional programs like R or Excel, but you would have to know what command to use. With Advanced Data Analysis, you can ask in plain language.
Beyond simple descriptive statistics, Advanced Data Analysis is capable of running statistical tests. For example, in our Charter survey about workers’ views toward AI, we have data on whether or not people are using generative AI in their jobs, broken down by gender. Something we noticed in the dataset is that a higher share of male workers are using generative AI in their job than female workers. But is that difference statistically significant (in other words, are we sure it isn’t due to chance)? With the Advanced Data Analysis plugin, I was able to run a chi-square test to find that the difference is, in fact, statistically significant.
Another feature that impressed me about Advanced Data Analysis is its ability to make edits to the files you give it. For the job postings file, for example, I wanted to rearrange the file by month—so all January entries next to each other, all February entries next to each other—and then in descending order for one of the variables. Here’s what I asked it to do:
Rearrange this file by putting all of the entries in month order, and then putting them in descending order for the variable "postingcount_ai_share.” So like this:
- All January entries in descending order by "postingcount_ai_share"
- All February entries in descending order by "postingcount_ai_share.”
- And so on.
Based on that prompt, Advanced Data Analysis rearranged the file and gave me a downloadable link to a new, organized spreadsheet. Users can also use Advanced Data Analysis to convert files from one format to another.
Advanced Data Analysis can easily make graphs and tables for you (from scratch or using the datasets you give it), and then make edits based on your feedback. You can ask it to change the axes, the colors, the thickness of the line, etc. It can also generate links for you to download those images.
A useful feature was the tool’s ability to use graphics to explain concepts. For example, I asked it to visually explain the difference between a t-test and a chi-square test. It first explained the difference in a couple of sentences and then generated two graphs: one with categorical data (for the chi-square test) and one with continuous data (for the t-test).
If you want to get really creative with using Advanced Data Analysis to visually explain things, read this article by Wharton professor Ethan Mollick, who asked it to “prove to a doubter that the earth is round with code in the most beautifully visual way possible.”
A practical business use case for Advanced Data Analysis is using it on financial statements to make projections for the future—though based on my testing, this use case, in particular, may require careful oversight.
I uploaded WeWork’s and Google’s S-1s from 2019 and 2004, respectively. To get a better sense of some of the simple questions a financial professional might want to ask about these documents, I worked with Sarah Janowsky, Charter’s head of business operations.
Interestingly, Advanced Data Analysis wasn’t able to read pdfs of the S-1s, possibly because of the way they’re formatted. Instead, we took screenshots of the summary table with the companies’ financial data, which the plugin was able to read. From there, we started asking it questions like:
“Find the growth rate in total expenses for the first half of 2018 to the first half of 2019. Use that growth rate on total expenses for 2018 to find what the projected total expenses are for 2019. Then tell me what the revenue growth rate must be in 2019 in order for this company to become profitable by the end of the year, given the expense that you calculated for all of 2019. Show your work.”
Advanced Data Analysis started by laying out each step and the formulas it would use, then performed the calculations. This experiment required much more oversight than the others. The expense growth rate it gave us was unrealistic. When we asked it what numbers it used to calculate the growth rate, it showed us, revealing that it pulled data from the wrong row of the table. When we pointed out its mistake, it accepted the new numbers we gave it and calculated the answer correctly. The mistakes could have been due to the fact that it was pulling data from an image (specifically, a PNG file)—anecdotally, I noticed fewer mistakes when working with CSVs.
For our final test, we wanted to see how well Advanced Data Analysis could handle less concrete instructions, so we asked it:
“Consider commonly used financial ratios. Tell me if there's anything concerning to you about this business.”
The program rattled off a series of important ratios that it could calculate based on the tables we gave it, including:
- Profit margin
- Operating margin
- Expenses as a percentage of revenue (e.g., R&D spending as a percentage of revenue)
- Year-over-year revenue growth
- Stock-based compensation to net revenue ratio
Advanced Data Analysis then calculated values for all of those ratios and gave us its opinion about each based on the data that was contained in their S-1 summary tables. For example, it told us that while Google was in a very healthy financial position when it filed its S-1 in 2004, the company’s high stock-based compensation ratio “can have implications for shareholder dilution and indicates a significant non-cash expense,” adding: “The impact and sustainability of this practice should be further evaluated, especially in the context of the company's industry, growth stage, and competitive landscape.”
Tips for using Advanced Data Analysis
Check in before you start. Start with something like, “Tell me what I’m looking at” or “What does this file contain?” to make sure you and the tool are on the same page before you start asking questions. If it doesn’t understand the file or it can’t read parts of it for formatting reasons, it’s better to know that early on.
Check its outputs. Advanced Data Analysis can get statistics wrong or imagine seeing things in the dataset that weren’t there, though it seems to do this less frequently than ChatGPT. For example, when working with the job postings dataset, I asked it to tell me the job categories whose postings most often mentioned AI. One of its answers—machine learning engineer—wasn’t in the dataset. When I pointed that out, it apologized and regenerated the list correctly. Like any generative AI product, Advanced Data Analysis still requires human oversight.
Ask it to show its work. Micromanaging is a positive here. I found it useful to ask Advanced Data Analysis to show its steps and explain its reasoning when performing a given task. That way, you can interject if you disagree with its approach. (This is how we identified that it used the wrong data points to calculate the expense growth rate.)
Ask lots of questions. You want to make sure you’re still in the driver’s seat while using Advanced Data Analysis. If you don’t understand why it did something in some way, ask it. Can you write out the formula for the calculation you just did? How did you get that number? Why was a t-test the right test to run? Keep asking questions until you’re on the same page.
Don’t worry about finding the perfect prompt. Unlike normal ChatGPT, Advanced Data Analysis isn’t picky about the words you use. As Wharton's Mollick has written, it’s “much less about prompt crafting than about having a conversation with the AI.”
Focus on clear communication. Though the specific words in your prompt seem to matter less with Advanced Data Analysis, it’s still important to clearly explain what you’re looking for. If you’re worried it won’t understand a command, add more details. For example, when I wanted it to rearrange my file by month and then put everything in descending order, I gave it examples:
- All January entries in descending order by "postingcount_ai_share
- All February entries in descending order by "postingcount_ai_share.
- And so on
Use Advanced Data Analysis as a thought partner. When you start using the Advanced Data Analysis plugin, you might be tempted to just think about it as a very advanced calculator that can also run code. But because it’s part of GPT-4, it can also reason and interpret the numbers it's giving you. For example, when we gave it S-1 summary tables for WeWork and Google, we asked it to compare the two and tell us which company’s financials were more concerning. It gave us the high-level takeaways for each, and then gave us a reasonable answer.
Don’t share sensitive data. You access Advanced Data Analysis through ChatGPT Plus, which improves by training on the conversations it has with people. For that reason, you should refrain from sharing any sensitive data with it. This limits your ability to use it on important business documents, like private financial statements. ChatGPT Enterprise, however, offers stronger data protections and also has the Advanced Data Analysis plugin.
Sign up here to receive our free Work Tech newsletter in your inbox.