Archives: Jobs

About the Role

We are looking for a data-driven professional to join our analytics and quality team.

In this role, you will analyze the performance of AI agents across finance-sector tasks to identify root causes of failure and drive improvements in task design, evaluation, and benchmarking frameworks.

Key Responsibilities

• Conduct statistical failure analysis to identify recurring patterns in AI agent performance across prompts, quality rubrics, templates, and file types
• Investigate root causes behind performance issues related to task execution, framework design, or data complexity
• Perform multi-dimensional analysis across finance sub-domains, task categories, and input types
• Develop dashboards and reports highlighting performance trends, failure clusters, and improvement opportunities
• Recommend enhancements to task design, quality rubrics, and evaluation metrics based on data-driven insights

Required Qualifications

• Strong understanding of statistical analysis, hypothesis testing, and pattern recognition
• Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analytics
• Hands-on experience in exploratory data analysis (EDA) and transforming data into actionable insights
• Familiarity with AI/ML evaluation techniques and quality assessment metrics
• Proficiency in Excel, SQL, and visualization tools such as Tableau or Power BI

Preferred Qualifications

• Prior experience in AI/ML model evaluation, data labeling quality, or automation analysis
• Knowledge of finance-sector concepts or willingness to learn finance domain structures
• Experience with multidimensional failure analysis or benchmarking studies
• Understanding of large-scale evaluation frameworks or test dataset design