Thursday
Room 2 - Level 3
16:20 - 17:20
(UTC±00)
Talk (60 min)
Using GPT Visual Capabilities to Solve a Wordle Puzzle
The visual capabilities of GPT-4 open up new scenarios of possibility with a multimodal model.
In this session, we will explore what this model can do, and rather than just showing a perfect polished final demo, I will walk you through my entire journey of trying to use the model to solve Wordle puzzles, starting with "Hello World". Along the way, you will gain a good understanding of the model's capabilities, along with learning some prompt engineering techniques that drove progress in this journey (along with what didn't work!). We'll close with a live demo to attempt to solve today's Wordle! This session will tackle a fun problem, but the underlying prompt engineering techniques for image understanding that you will learn are applicable to a wide variety of business problems.
Jennifer Marsman
Jennifer Marsman is a Principal Engineer in the Office of the CTO at Microsoft, focused on generative AI with OpenAI models. She has worked for Microsoft for over 20 years; previous teams include Azure Cognitive Search, AI for Earth, CSE, Developer Evangelism, the Assistance Platform, and the Natural User Interface Platform (reorged from the Natural Language Group). Jennifer is a frequent speaker at software development conferences around the world, including the United Nations AI for Good Summit.
Since 2016, Jennifer was recognized as one of the “top 100 most influential individuals in artificial intelligence and machine learning” by Onalytica, reaching the #2 slot in 2018. In Dec 2017, Jennifer was named one of the “10 Influential Female Developers to Know and Love on StackOverflow” by Traackr. She has been featured in Bloomberg for her work using EEG and machine learning to perform lie detection. In 2009, Jennifer was chosen as "Techie whose innovation will have the biggest impact" by X-OLOGY for her work with GiveCamps, a weekend-long event where developers code for charity. She has also received many honors from Microsoft, including the “Best in Role” award for Technical Evangelism, Central Region Top Contributor Award, Heartland District Top Contributor Award, DPE Community Evangelist Award, CPE Champion Award, MSUS Diversity & Inclusion Award, Gold Club, and Platinum Club.
Previously at Microsoft, Jennifer was the Principal Software Engineer Lead of the Grok AI group on the Azure Cognitive Search team, where she focused on applying machine learning to unstructured data. Prior to that, Jennifer was the Principal Engineer of the AI for Earth team, where she used data science, machine learning, and artificial intelligence to aid with clean water, agriculture, biodiversity, and climate change. Jennifer was also a software developer in Microsoft’s Natural Interactive Services division, where she earned two patents for her work in search and data mining algorithms. Jennifer has also held positions with Ford Motor Company, National Instruments, and Soar Technology. Jennifer holds a Bachelor’s Degree in Computer Engineering and Master’s Degree in Computer Science and Engineering from the University of Michigan in Ann Arbor. Her graduate work specialized in artificial intelligence and computational theory. http://twitter.com/jennifermarsman