I hit the 5hr limit for Codex with the pro $200/m plan

Reaching the 5-Hour Limit with Codex: A Deep Dive into Advanced Data Processing and AI Integration

Recently, I encountered the 5-hour session limit when utilizing the Codex API under the Pro plan ($200/month). This milestone prompted me to reflect on the extensive work I’ve been pursuing using AI-driven data analysis and software development. In this article, I’d like to share some of the projects I’ve worked on, the insights gained, and the larger vision behind my efforts.

Harnessing Massive Data Sets for Geographic Analysis

One of my key projects involves gathering and analyzing colossal datasets related to road networks. Using multiple simultaneous AI chat sessions, I compiled approximately 330 million images of roads—covering both paved and unpaved surfaces—and aggregated around 18 billion data points. This comprehensive dataset enabled me to generate an in-depth classification of roads across regions, primarily focusing on paved versus unpaved categories through inference from large AI ensembles.

The scale of this data collection is noteworthy. For example, in North America alone, I expanded my dataset to include detailed road information, with plans to extend this coverage across Europe, which features twice as many roads within a smaller geographic area. Currently, I’ve processed nearly a billion road images for the EU, a task that underscores the importance of robust data infrastructure; I recommend deploying Linux with filesystem options like ZFS or XFS for handling such large datasets efficiently.

Innovative Approaches to Geographic Visualization

Beyond static data, I’ve developed visualizations to interpret and showcase these insights. One example is an interactive map overlay (linked here) that enables users to explore surface types and land cover in high detail. This tool provides a visual foundation for more complex analyses concerning accessibility and terrain.

Advanced Routing and Traffic Modeling

Building upon this data, I crafted an ultra-fast routing engine capable of generating billions of shortest-path routes—specifically, over 10 billion in North America alone. This system analyzes usage patterns and traffic flow predictions, providing valuable insights into how different roads are utilized. It’s designed to simulate scenarios such as traffic congestion and route optimization for various transportation modes.

Developing the World’s First “Prompt-to-Route” System

Perhaps the most ambitious aspect of my work is the development of the world’s first “prompt-to-route” platform. This software, a culmination of two years of dedicated effort, allows users to input natural language prompts to generate customized routes based on specific criteria. For example, requesting “a motorcycle route to Sturgis that avoids gravel roads and storms” produces tailored pathways that align with user preferences and real-time conditions.

While I have prototypes available, I’ve been cautious about sharing them publicly until the system reaches a more refined stage.

Reflections and Future Directions

While my recent work leverages incredible AI and data processing capabilities, it also highlights the importance of reliable infrastructure and efficient system design. For those tackling similarly massive datasets or computational endeavors, I recommend using Linux with filesystems like ZFS or XFS, which excel at managing large-scale data workloads.

In summary, my journey has been one of relentless exploration—transforming raw data into actionable insights and user-centric tools. The achievements detailed here represent just a fraction of what’s possible with AI, big data, and innovative software engineering.

Thank you for taking the time to read about this project. I look forward to sharing further developments as I continue pushing the boundaries of geographic data analysis and personalized route planning.