Member of Technical Staff (Applied AI Engineer, Agent Capabilities)

LIKELY QUESTIONS

- How would you evaluate whether a frontier model is meaningfully better for a real-world agent task, rather than just better on a benchmark?
- Design an agent that can complete a long-running browser-based task with tool calls, intermittent failures, and changing user requirements.
- Tell me about an LLM or agent system you built. What did you own, what were the hardest reliability problems, and what were the results?
- How would you decide when an agent should continue autonomously, ask the user for clarification, retry a tool call, or stop?
- What evaluation suite would you build for Perplexity Computer, and which metrics would you use to connect agent behavior to user value?
- How would you structure context management for an agent working across a long task with web pages, documents, tool outputs, and prior actions?
- A new model appears substantially better at tool use in demos. How would you rapidly assess whether it is ready for production integration?
- Describe how you would design the backend for a scalable agent-execution platform using Python, Go, Rust, AWS, PostgreSQL, DynamoDB, and TypeScript.

BEHAVIOURAL QUESTIONS

- Tell me about a time you took an ambiguous technical or product problem and turned it into a shipped system.
Model approach: Situation: Describe an unclear user problem with incomplete requirements. Task: Explain your responsibility for defining scope and delivering an outcome. Action: Show how you gathered evidence, wrote a problem statement, built a prototype, aligned product and engineering partners, and defined success metrics. Result: Quantify launch impact, adoption, latency, reliability, or reduced manual effort, and explain what you changed after launch.

- Tell me about a time an AI, ML, or automation system failed in production or produced poor-quality outputs.
Model approach: Situation: Describe the failure mode clearly, such as hallucination, tool misuse, low task completion, unsafe output, or high latency. Task: State your ownership of diagnosis and mitigation. Action: Explain your debugging process, including logging, error taxonomy, evaluation cases, root-cause analysis, guardrails, retries, fallbacks, or UX changes. Result: Give before-and-after quality metrics and describe the monitoring put in place to prevent recurrence.

- Tell me about a time you disagreed with a research, product, or engineering partner on the right technical approach.
Model approach: Situation: Set up a concrete disagreement, such as optimizing model quality versus launch speed, or building a custom system versus using an existing model API. Task: Explain the decision you needed to influence or make. Action: Emphasize listening, clarifying assumptions, creating a lightweight experiment or decision framework, and using user impact and operational cost as criteria. Result: State the decision, measurable outcome, and how the collaboration improved.

- Tell me about a time you moved quickly on a prototype but still maintained a high bar for production quality.
Model approach: Situation: Describe a time-sensitive opportunity involving a new capability, customer need, or model release. Task: Clarify the need to validate value quickly without creating uncontrolled technical debt. Action: Explain how you separated prototype and production paths, instrumented the system, scoped an initial launch, defined reliability thresholds, and created a follow-up hardening plan. Result: Share what the prototype proved, how rapidly it shipped, and the production improvements that enabled broader rollout.

SMART QUESTIONS TO ASK

- What are the highest-value user workflows Perplexity Computer is trying to make reliably actionable over the next 6 to 12 months?
- How does the team currently measure agent success beyond model-level benchmarks, especially for partial completion, recovery quality, and user trust?
- Where do agents most often fail today: planning, browser or tool execution, context management, long-horizon task state, or model judgment?
- What is the expected balance in this role between rapid experimentation with new models and building durable platform capabilities used by multiple product teams?
- Can you describe a recent agent capability that moved from prototype to production, including what evidence was required before launch and what changed after real-user feedback?

RED FLAGS TO WATCH FOR

- The team cannot articulate concrete user outcomes, evaluation methods, or quality thresholds, and relies mainly on impressive demos or generic benchmark scores.
- Ownership is described as broad but decision-making authority, staffing, infrastructure support, and product priorities are unclear or highly centralized.
- Leaders minimize reliability concerns such as tool failures, hallucinations, privacy, observability, cost, or safe recovery in favor of shipping autonomous features quickly.