What economic tasks is artificial intelligence performing? – An empirical analysis based on millions of conversations

Key Chapter Title List

1. Introduction
2. Background and Related Research
3. Methods and Analysis
4. Task-Level Analysis of AI Usage
5. Presentation of Professional Skills
6. AI Usage by Salary and Entry Barrier
7. Comparative Analysis of Automation vs. Augmentation
8. Usage Patterns of Different Model Types
9. Discussion
10. Conclusion

Document Introduction

While the potential impact of artificial intelligence on the labor market has sparked widespread discussion, systematic empirical evidence on the application of AI systems in real-world economic tasks remains relatively scarce. Existing methods such as predictive models, controlled experiments, and periodic surveys struggle to dynamically track the relationship between the evolution of AI capabilities and their practical application. This research gap highlights the necessity for large-scale empirical analysis.

This study proposes a novel empirical framework. Through privacy-preserving analysis of millions of real conversations on the Claude.ai platform, combined with the occupational classification system of the U.S. Department of Labor's O*NET database, it achieves for the first time a large-scale quantitative study of AI application patterns in economic tasks. This framework can not only identify current characteristics of AI usage but also provide early indicators for anticipating the potential impact of technological evolution on the economic sphere.

The research data originates from conversations collected from the free and professional versions of Claude.ai between December 2024 and January 2025. Utilizing the Clio privacy-preserving analysis tool, the conversation content was mapped to dimensions such as occupational tasks, skill requirements, and interaction patterns within the O*NET database. The analysis process constructed a task-level classification system, ensuring precise matching of nearly 20,000 unique tasks, and guaranteed data compliance through strict privacy control measures.

Core findings reveal that AI usage is primarily concentrated in software development and writing tasks, together accounting for nearly half of total usage; in about 36% of occupations, at least a quarter of tasks involve AI application, while only 4% of occupations have AI covering over 75% of tasks. Cognitive skills (such as reading comprehension, writing, and critical thinking) account for the highest proportion in human-machine dialogues, whereas physical skills and management skills are represented at very low levels. In terms of salary, AI usage peaks in the upper salary quartile, with relatively lower usage rates in both extremely high-salary and low-salary occupations. Regarding entry barriers, occupations requiring a bachelor's degree or higher (Job Zone 4) exhibit the highest AI usage rates.

In terms of application patterns, 57% of human-machine interactions manifest as augmentation of human capabilities (e.g., task iteration, knowledge learning), while 43% exhibit characteristics of automation (e.g., direct task execution, feedback loops). Different AI models show differentiated application scenarios: Claude 3.5 Sonnet is more commonly used for coding and technical tasks, whereas Claude 3 Opus accounts for a higher proportion in creative writing and educational content development.

The methodology of this study provides an automated and granular empirical foundation for dynamically tracking the evolution of AI applications in the economic domain. Its findings offer crucial reference points for policymakers, businesses, and researchers to understand the actual impact of AI on work scenarios, while also providing a basis for decision-making in addressing labor market adjustments brought about by technological transformation.