How can a team move from experimenting with AI to using it in work they can measure and trust? That question shapes ODSC AI NYC 2026. The in-person AI for Work Summit runs December 2–3 at Jay Conference Center Bryant Park in Midtown Manhattan. Its preliminary program brings engineers, business practitioners, and leaders together for working sessions on agents, evaluation, everyday workflows, and the decisions that make AI adoption last.
The event has four tracks: Build, Business Workshops, Lead, and AI Engineering Fundamentals. You can follow one track or compare approaches across them. The confirmed instructor list includes practitioners from Google, Meta, Mistral, Fireworks AI, and Canva, alongside independent engineers and researchers. More session details are still being added, so treat the current agenda as a planning tool and check it again before you travel.
Start with a question your team already has
Jon Krohn, Chief Data Scientist at Nebula.io and host of the SuperDataScience podcast, opens the Build track with “Riding the Exponential: How to Thrive as Agents Double Task Horizons Every 7 Months.” His keynote examines the evidence behind reported gains in what agents can complete and asks what that progress means for engineers and organizations.
Bring a real use case to the summit, even if it is still a rough idea. What task would you hand to an agent today? How would you check its work, and where would a person need to approve the result? Those questions give you a way to compare the workshops that follow.
If you are new to building AI applications, the AI Engineering Fundamentals track provides a more structured starting point. Its five-session sequence covers API calls, context and retrieval, simple agents, evaluation, and production basics. The program is designed to take attendees from a first technical workflow to a running agent, with the limitations and checks visible along the way.
Build agents you can investigate
An agent that produces a plausible answer can still take the wrong path to get there. Michael Levan, AI Architect and Forward Deployed Engineer at solo.io, addresses that problem in “Instrumenting an Agent for Observability.” Participants work through traces for reasoning steps, tool calls, and retrieval, then follow a failure backward to its cause. The workshop also covers context propagation to model providers and MCP servers, production signals such as token use and cost, and how to turn traces into a reusable evaluation set.
When a user reports a bad answer, a team needs to know whether retrieval, a tool call, or the agent’s decision caused it. Instrumentation provides evidence for a fix and a test to catch the same failure again.
The preliminary program also lists “From Pilot to Production: Building Reliable Domain Agents for Complex Work,” a talk on scoping an agent’s responsibilities and preparing for failures that appear under real workloads. Gerry Wolfe, an AI consultant and engineer at Lambda Engine, is among the confirmed instructors.
Make evaluation part of the workflow
Bruno Gonçalves, a consultant at Data For Science and international science fellow at the ISI Foundation, leads “You Can’t Ship What You Can’t Measure — Building a Production-Grade LLM Eval Harness.” The hands-on workshop uses MMLU questions to build an evaluation suite from the ground up. Participants compare answers with deterministic checks, assess explanations against LLM-judge rubrics, and use human spot-checks where automated scoring needs scrutiny. They also use confidence intervals and paired tests before comparing prompt or model configurations with a command-line tool.
Mike Taylor, Head of Evals at Every Inc., approaches the same problem from a team’s daily work. In “Building an Eval Suite Your Team Will Actually Run,” participants collect representative cases with domain experts, translate feedback into explicit pass/fail checks, and connect those checks to an active workflow. The session also examines false passes, inconsistent judging, and how to build a trusted set of examples that catches regressions.
Gonçalves focuses on rigorous system comparisons; Taylor shows how to capture the judgment of people who know the task. Together, the workshops give teams a clearer way to decide whether an AI application has improved.
Give everyday AI work clear boundaries
The Business Workshops track looks at the tasks people already repeat in sales, marketing, finance, operations, and HR. Greg Rosner, founder of PitchKitchen and host of the AI Workforce Clinic, presents “Your AI Agent Needs a Job Description.” The session asks attendees to define an agent’s role, operating context, review records, and success criteria before handing it work. It is a practical exercise in turning a vague request into an assignment someone can assess.
Mark Kashef, founder of Prompt Advisers, takes that idea into a tool people can use directly. His workshop, “Claude Cowork for Everyone: Turn Everyday Work into Repeatable AI Workflows,” walks through defining a role, adding durable context, connecting documents and tools, and building review points into a sequence of steps. The goal is a reusable approach for research, analysis, content, and operations work.
The summit’s full-day AI for Work training broadens the view from one delegated task to a team process. Its four parts cover task specifications, access to documents and business systems, multi-step workflows, and review controls. Attendees bring work they own, test how a workflow handles missing inputs or unclear instructions, and define who approves what.
Decide what should last beyond the current model
Technical choices have a longer life than the model version behind them. The preliminary Build agenda includes “The AI Decisions That Outlast the Model,” a session on architecture, evaluation, data access, and review practices that should hold up as models change.
Leaders face a related measurement problem. Michael Schrage, Research Fellow at MIT’s Initiative on the Digital Economy, is listed for the panel “Proving ROI When the Baseline Was Never Measured.” It addresses a familiar situation: a team adopts AI before recording how long the original work took or how well it performed. The discussion asks how leaders can judge value and decide where continued investment makes sense when a clean before-and-after comparison is unavailable.
Other confirmed instructors include Sid Dhulipalla of Fireworks AI, Paige Bailey of Google, Christine Long of Meta, Ben Morss of Mistral, Sheamus McGovern of ODSC AI, and Chris Hadges of Canva. Check the program for their final session assignments.
Plan your two days around a real decision
Start with a project or workflow you can describe in one sentence. If you build AI systems, compare the observability and evaluation workshops against a failure your team has already seen. If you manage a business process, look for a session that helps you define the task, its review points, and a useful measure of success. Colleagues attending together can split across tracks and compare what each learned about the same problem.
The ODSC AI NYC program and passes include a free Open Pass for keynotes, the open talk track, and networking; workshops and function labs require another pass. Review the live agenda before choosing. Then arrive with a question you need answered and leave with a concrete next step for your team.
