Financial AI Evaluator (Remote, Hourly Contractor)
CNTXT AI · Manhattan, NY · 1 wk ago
RemoteRemoteInformation TechnologyFull-time
Position Summary
In this remote, hourly contractor role, you will evaluate AI-generated financial content and develop cases that test analytical reasoning accuracy. Your work directly improves how leading AI models handle financial information, making them more accurate, reliable, and clearly explained.
- Evaluating AI-generated responses across banking, insurance, accounting, and risk management for factual accuracy, analytical soundness, and correct application of financial principles
- Identifying errors in financial methodology, flawed assumptions, misapplied regulations, and unsupported conclusions, fact-checking against reliable sources, and explaining corrections clearly in writing
- Assessing AI reasoning across multi-document financial scenarios, including cross-referencing data across balance sheets, reports, and operational records
- Developing prompts and test cases that test AI performance on practical financial operations and data analysis, including Excel-based evaluation tasks
- Rating and comparing AI responses based on correctness, internal consistency, and adherence to the prompt
Profile Requirements
- Bachelor's degree or higher in Finance, Accounting, Economics, Business, or a closely related field
- Minimum 1 year of full-time professional experience in banking, insurance, accounting, or risk management
- Ability to explain complex financial concepts clearly and accurately in writing
- Full professional English proficiency
- Reliable and self-directed, with consistent output quality in a remote, asynchronous workflow
Preferred Experience
- Prior experience with AI data training, annotation, or model evaluation
- Familiarity with multimodal AI tools or LLM evaluation workflows