ODSC AI NYC— AI for Work Summit —

Riding the Exponential: How to Thrive as Agents Double Task Horizons Every 7 Months

Frontier agents' task horizons have been doubling roughly every seven months — a trend visible across coding, AI engineering, and knowledge work. This keynote defines what an agent is (and isn't), examines the empirical evidence behind the exponential, and gives concrete strategies for positioning yourself and your organization to benefit from agentic AI rather than be disrupted by it.

Instrumenting an Agent for Observability

Agents break the assumptions observability stacks were built on: non-deterministic control flow, per-request paths, and failure modes that look like success. Instrument an agent from the ground up — span design for reasoning steps, tool calls, and retrieval; OpenTelemetry GenAI conventions; context propagation to MCP servers; eval types that matter in production — then read a real failure trace backward to root cause and mine production traces into a golden eval set.

You Can't Ship What You Can't Measure — Building a Production-Grade LLM Eval Harness

Design, build, and operate a real automated LLM evaluation harness end to end — from first principles to a regression suite you can drop into CI. Using MMLU as the system under test, layer deterministic answer-matching, LLM-as-judge rubrics, and human spot-checks, add statistical rigor with bootstrap CIs and paired significance tests, and build a CLI that compares two prompt/model configurations and flags significant regressions. Leave with working, reusable code.

Panel: Proving ROI When the Baseline Was Never Measured

Most organizations adopted AI before capturing a baseline for the work it was supposed to improve. This panel tackles the hardest measurement question leaders face: how do you prove value — or decide where to stop investing — when you can't compare against a 'before' that was never recorded?

AI for Work: Full-Day Training

Your AI Agent Needs a Job Description

Vague requests produce vague output. Learn to treat your AI agent like a new hire: write it a job description with a clear role, operating context, review records, and success criteria — the difference between a chat window and a repeatable, accountable workflow you can hand work to.

Claude Cowork for Everyone: Turn Everyday Work into Repeatable AI Workflows

Build a practical Claude Cowork workflow from scratch: define the AI's role, give it durable context, break work into reliable steps, connect tools and documents, and add review points that prevent confident mistakes. Leave with a reusable cowork framework for research, analysis, content, and operations.

Fundamentals 1: LLM & Agent Foundations

Start with the basic building blocks behind AI applications: API calls, structured outputs, tool use, and simple agent loops — the difference between prompting a model and building a repeatable AI workflow.

Fundamentals 2: Context Engineering & RAG

Learn how AI systems use external information: documents, chunking, embeddings, retrieval, context assembly, and the practical limits of RAG with real internal data.

Fundamentals 3: Building a Simple Agent

Move from single prompts to multi-step tasks: planning, tool use, memory, failure handling, and where an agent is useful — and where a simpler workflow is the better answer.

Fundamentals 4: Evals & Observability

A demo that works once is not enough. Test AI outputs, create evaluation examples, trace agent behavior, monitor failures, and decide whether a system is improving or getting worse.

Fundamentals 5: Production Basics: Cost, Routing & Guardrails

Model selection, routing, caching, latency, cost, guardrails, and basic risk controls — what changes when an AI workflow moves closer to production.