AI Specialist | MCP, Plugins & Connectors | $190/hr

About the role

We are seeking experienced LLM power users, AI productivity experts, MCP users, plugin/connector specialists, and advanced AI users to evaluate how next-generation AI systems handle complex, personalized real-world tasks. This project is focused on people who already use AI extensively in their personal lives for planning, research, organization, decision-making, and multi-step workflows involving connected apps and services.

Responsibilities

• Create realistic prompts based on complex personal-life workflows
• Complete multi-step tasks while recording your screen
• Use MCP servers, plugins, connectors, and connected applications during task execution
• Evaluate whether AI responses are useful, personalized, accurate, realistic, and actionable
• Test how well AI systems retrieve and use information from connected tools
• Identify failures involving missing context, poor planning, incorrect tool use, unrealistic recommendations, or unsafe assumptions
• Compare different approaches and determine what an excellent AI response should look like
• Write detailed feedback explaining AI strengths, weaknesses, and mistakes
• Create and apply evaluation rubrics for complex AI workflows

Example Workflows

Tasks may involve:

• Travel planning and itinerary creation (flights, hotels, restaurants, activities, and reservations)
• Personal health research and organization (fitness, wellness, and lifestyle planning)
• Food and dining decisions
• Home maintenance and repair services
• Career searches and job planning
• Personal productivity and organization (documents and file management, calendar and scheduling workflows)
• Personal research and information gathering
• Shopping and purchase research
• Event and activity planning
• Personal finance organization
• Email and communication workflows
• Multi-app research and decision-making
• Long-running projects that require AI to remember personal context

Who Can Apply

Relevant backgrounds include:

• AI Power Users, LLM Power Users, Chatbot Power Users, Generative AI Power Users, AI Productivity Experts, AI Workflow Specialists, AI Automation Specialists, MCP Users, Model Context Protocol Users, Plugin Experts, Connector Experts, AI Tool Specialists, AI Enthusiasts, AI Researchers, AI Evaluators, AI Trainers
• Prompt Engineers, Prompt Designers, AI Workflow Designers, AI Operations Specialists, AI Product Testers, AI Quality Analysts, Model Evaluators, LLM Evaluators, AI Benchmark Evaluators, Human Feedback Specialists, AI Data Specialists, AI Annotation Specialists, AI QA Specialists, AI Red Teamers, AI Safety Evaluators with strong real-world LLM usage
• Software Engineers, Developers, Automation Engineers, No-Code Developers, Low-Code Developers, Integration Specialists, API Users, Developer Relations Professionals, Technical Product Managers, Solutions Engineers, and Technical Consultants who regularly use AI tools and connectors in their own lives
• Productivity Experts, Knowledge Management Specialists, Notion Power Users, Google Workspace Power Users, Personal Knowledge Management Experts, Digital Organization Specialists, Research Professionals, Virtual Assistants, Executive Assistants, Operations Professionals, Project Managers, and Remote Work Specialists with extensive AI usage

Relevant AI Experience

Strong candidates may regularly use AI for:

• Complex research
• Trip and vacation planning
• Hotel and flight comparisons
• Restaurant and activity discovery
• Career searches and application planning
• Personal scheduling and calendar management
• Email management
• Document search and summarization
• Personal knowledge management
• Health and wellness research
• Home improvement research
• Shopping comparisons
• Local service discovery
• Financial organization
• Decision-making across multiple sources
• Long, multi-step conversations requiring personal context
• Tasks that require AI to take information from one application and use it in another

MCP, Plugins & Connectors

LLM & AI Platforms Experience may include heavy use of:

• ChatGPT, Claude, Gemini, Perplexity, Microsoft Copilot, Cursor, Notion AI, or other advanced LLM-based products and AI assistants

We are especially interested in people who go beyond simple question-and-answer usage and regularly use AI for complex, multi-step workflows requiring context, planning, judgment, and tool usage.

Requirements

• Heavy personal use of LLM products
• Approximately 6+ months of active LLM usage history
• Regular use of AI tools multiple times per week
• Strong hands-on experience with MCP and/or AI plugins and connectors
• Experience connecting LLMs to external applications, files, services, or personal data
• Ability to complete complex multi-step tasks using AI
• Strong written judgment and attention to detail
• Ability to clearly explain why an AI response succeeds or fails
• Comfortable recording your screen while completing project workflows
• Comfortable using your personal plugins/connectors during project tasks

Highly Preferred

• Extensive experience evaluating AI or LLM outputs
• Previous AI training, benchmarking, human-feedback, or model-evaluation work
• Experience designing detailed evaluation criteria
• Strong rubric-writing and rubric-evaluation experience
• 100+ hours of previous rubric-based project work
• Experience identifying subtle errors in AI reasoning or tool usage
• Experience testing AI agents across multi-step workflows
• Familiarity with MCP server setup, configuration, and usage
• Experience using multiple connectors together in a single workflow
• Rich personal usage history across several AI products
• Experience testing AI personalization or memory capabilities
• Ability to distinguish technically possible AI behavior from unrealistic or impractical outputs

The strongest candidates are people who already treat AI as a personal operating system rather than a simple chatbot. You may regularly use AI to plan trips, research purchases, organize documents, manage schedules, compare restaurants, search for jobs, research health topics, coordinate activities, manage personal projects, or make complex decisions using information spread across multiple applications.