My First AWS re:Invent
Intro
It seems counterintuitive that after more than a decade in data, 2024 marked my first time attending AWS re:Invent in Las Vegas, Nevada. I attribute this to years of being an open-source zealot and maintaining a healthy skepticism toward large corporations like AWS. But I have to admit, my cold facade melted away. Over the course of the week, I found myself becoming an unabashed fan of what AWS offers, particularly in the realm of Generative AI (GenAI). This blog captures my journey from skeptic to believer as I reflect, digest, and share my insights from the event.
Day 1
Practical Tips
Vegas is, well, Vegas. I arrived on Sunday and stayed through Friday, which was probably a rookie mistake. Anything more than 2–3 days in this neon fever dream is a test of endurance, even for seasoned travelers. My advice? Plan non-casino activities early, because wandering around aimlessly while losing money gets old fast. Thankfully, Vegas has evolved. You can explore world-class restaurants or marvel at the hotels’ over-the-top architecture. Pro tip: strolling through holiday-themed hotel lobbies counts as cardio.
AWS re:Invent Specific
NFL AI with Amazon Bedrock: Playbook Pro
The first session I attended showcased the NFL Media team’s “Playbook Pro,” a generative AI application built using Amazon Bedrock. This tool allows users to query game footage and stats using natural language prompts. For example, a request like, “Show all Patrick Mahomes completions with probability less than 50 percent,” retrieves relevant clips and statistics.
What makes this application unique is its use of agentic AI workflows, which go beyond generating static content. Playbook Pro leverages multiple services in tandem: interpreting queries, accessing APIs, fetching additional context, and orchestrating responses in real time. For example, the tool parses ambiguous queries into structured API calls, retrieves relevant video clips, and integrates metadata to refine outputs. Seeing this level of orchestration was eye-opening — it’s not just about AI generating content but also about seamlessly integrating it into workflows that add real value.
The session also highlighted technical challenges, such as handling ambiguous queries, clarifying user intent, and ensuring robust API mappings. AWS tools like Lambda and MemoryDB play critical roles in achieving this fluidity. For anyone interested in AI-enhanced workflows, this was an inspiring showcase of Bedrock’s potential.
Session Slides: Playbook Pro
Amazon Q: The Duke
The second session introduced “The Duke,” another NFL project leveraging Amazon Q. This self-service conversational AI platform simplifies building lightweight AI applications without requiring extensive technical expertise. The NFL used Amazon Q to replace manual Slack workflows with an AI-driven system that streamlined communication and improved efficiency. By the end of this session, I was hooked and scheduled multiple hands-on Amazon Q workshops for later in the week.
Session Slides: Amazon Q & The Duke
Day 2
Practical Tips
Let me paint a picture: I averaged 25,000 steps a day, my legs felt like rubber by noon, and the desert air turned my skin into sandpaper. Vegas has a way of breaking even the most disciplined among us. My advice? Learn from my mistakes. First, bring comfortable shoes — yes, even if they make you look like a tourist. Second, use the monorail or AWS shuttles. Walking from venue to venue might seem virtuous, but trust me, you’ll miss half the sessions while nursing blisters at a Starbucks. Lastly, stock up on moisturizer unless you want to resemble a dried-out cactus by Day 3.
AWS re:Invent Specific
Entity Resolution with United Airlines
This session was a highlight, showcasing how United Airlines transformed its traveler experience using AWS Entity Resolution. The service achieved an impressive 90 percent match rate by combining deterministic and probabilistic ID matching, which reconciled duplicate records across multiple systems. United streamlined its operations and enhanced customer satisfaction through these unified customer profiles.
The session delved into the underlying data architecture, which included integrating loyalty and reservation platforms through ETL pipelines. AWS tools like Amazon Connect Customer Profiles and Clean Rooms enabled United to securely unify and analyze data. For someone who has built similar systems, seeing AWS abstract these capabilities into ready-made services was both validating and inspiring.
Session Slides: Unified Customer Profiles
Workshop: Move Data to Amazon S3 & Kickstart GenAI Workflows with Amazon Q
This hands-on workshop demonstrated the power of Amazon Q in action. We used RAG (Retrieval-Augmented Generation) techniques to migrate legacy data into S3 and build a travel agent application. By indexing and crawling the S3 data, the app allowed users to query itineraries using natural language prompts. It was an excellent example of how GenAI can be seamlessly integrated into workflows to enhance productivity and deliver personalized experiences.
Workshop Slides: GenAI Workflows
Day 3
Practical Tips
Here’s the thing about AWS re:Invent: unless you’re an ultramarathoner or a wizard, you can’t be everywhere at once. My secret advice? Stay in the middle of the Strip if you can afford it. Places like the Cosmopolitan or Bellagio are prime real estate for minimizing time spent schlepping between venues like the Venetian and Mandalay Bay, which are on opposite ends of the Strip. Another lifesaver is the Content Hubs. These simulcast sessions, so you can sit in one place and still catch content without sprinting across Vegas like you’re in a bad action movie.
AWS re:Invent Specific
Workshop: Amazon Bedrock Python SDK
In this workshop, we used the Amazon Bedrock Python SDK to create generative text and image applications. Through Jupyter Notebooks, we explored how foundational models like Titan can be integrated with AWS services such as S3 and Rekognition. For example, we generated creative prompts like “a cat dressed as Spider-Man” and validated the outputs with Rekognition’s object detection capabilities. This hands-on experience provided a tangible way to experiment with GenAI.
Workshop: Implement Security Fixes to Codebase using Amazon Q Developer
This session showcased Amazon Q’s potential as a coding companion. Using Visual Studio Code, we scanned for vulnerabilities, optimized CloudFormation templates, and implemented inline code suggestions. Amazon Q’s ability to simplify complex development workflows made this one of the most impactful workshops of the week.
Day 4
Practical Tips
Vendor Village is like Black Friday meets Comic-Con: loud, chaotic, and full of shiny distractions. It’s also overwhelming enough to trigger an existential crisis if you’re not prepared. My advice? Don’t wander in aimlessly. Plan your visits during off-peak times, like between sessions, so you can actually talk to vendors without being trampled by swag-hungry mobs.
AWS re:Invent Specific
Designing Generative AI Workloads for Resilience
This session offered invaluable insights into building resilient GenAI workloads. Key takeaways included the importance of fault isolation, redundancy, and capacity planning. AWS emphasized tools like semantic caching and layered observability to ensure robust and efficient AI pipelines. As someone who focuses on modern data stacks, this felt like a natural extension of those principles into the AI domain.
Session Slides: Designing GenAI Workloads
Advanced Prompt Engineering with Bedrock
Prompt engineering is a critical skill for maximizing the utility of GenAI. This session covered techniques such as Chain-of-Thought reasoning and Few-Shot prompting, demonstrating how small adjustments to prompts can lead to significantly improved outputs. Seeing these techniques applied provided actionable insights for future projects.
Session Slides: Prompt Engineering
AWS re:Play
On Thursday night, AWS hosted re:Play, an event that can only be described as an adrenaline-filled tech carnival. Held at the Las Vegas Festival Grounds, the evening featured a sprawling food tent with options that would make even the pickiest eater happy, a game tent filled with every imaginable activity — including a GIANT Connect Four board — and a free concert from one of my favorite bands — Weezer! If AWS re:Play isn’t on your agenda for future re:Invent trips, you’re doing it wrong.
Day 5
By Friday, the schedule had mostly tapered off, with only a handful of sessions running in the morning. I used this time to tie up loose ends, revisit my notes, and decompress before heading to the airport for my red-eye flight. While not as action-packed as earlier in the week, this slower pace was the perfect way to wrap up an incredible experience at AWS re:Invent.
Closing
AWS re:Invent exceeded my expectations. I arrived as a skeptic and left as a fan, inspired by AWS’s GenAI capabilities. The tools AWS offers make developing GenAI applications accessible, much like how Databricks democratized data engineering. Whether this trend sustains long-term remains to be seen, but it’s clear that AWS is paving the way for innovation.
Looking ahead, I’ll be sharing much more content on GenAI and its intersection with data engineering. Stay tuned for previews from my upcoming book with Manning Publishing, as well as new Medium posts diving deeper into these topics.
