Job Description
What this opportunity involves
- Create structured test cases that simulate complex human workflows
- Define gold-standard behavior and scoring logic to evaluate agent actions
- Analyze agent logs, failure modes, and decision paths
- Work with code repositories and test frameworks to validate your scenarios
- Iterate on prompts, instructions, and test cases to improve clarity and difficulty
- Ensure that scenarios are production-ready, easy to run, and reusable
What we look for
- 3+ of software development experience with strong Python focus
- Experience with Git and code repositories
- Comfortable with structured formats like JSON/YAML for scenario description
- Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
- Familiarity with Docker
- English proficiency - B2
How it works
- Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Project time expectations
Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Payment
Paid contributions, with rates up to $40/hour*
Fixed project rate or individual rates, depending on the project
Some projects include incentive payments
*Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
Get notified of similar jobs
We'll send you an email when jobs similar to "Freelance Agent Evaluation Engineer" are posted.
Related Jobs You Might Like
View all jobs →Assistant Operations Manger - testing and inspection in industrial services
SGS
Company Description We are SGS – the world's leading testing, inspection and certification company. We are recognized as the global benchmark for sustainability, quality and integrity. Our 99,600 employees operate a network of 2,600 offices and laboratories, working together to enable a better, safer and more interconnected world. Job Description We are seeking an experienced Operations Manager to oversee QA/QC and TPI service contracts, ensuring efficient operations, compliance, and high-quality service delivery. Lead and manage day-to-day operations of QA/QC & TPI contracts Ensure compliance with HSE standards and company policies Plan and coordinate logistics, resources, and operational activities Monitor performance, drive efficiency, and manage departmental P&L Develop and implement operational strategies, procedures, and quality controls Coordinate with clients, internal teams, and external stakeholders Manage budgets, procurement, and reporting processes Lead, mentor, and develop team members Support client relations, quotations, and issue resolution Qualifications Bachelor’s or Diploma in Business Administration, Operations Management, or related field Minimum 8 years of experience in operations management or similar role Strong knowledge of operations, supply chain, and quality control processes Proven experience in budget and resource management Excellent leadership, communication, and problem-solving skills Ability to work in a fast-paced environment and manage multiple priorities Knowledge of HSE regulations and compliance requirements Proficiency in operations management tools/software
Store Associate -Shoexpress_Nizwa
LANDMARK GROUP
Job Summary Responsible for maximizing sales through exceptional customer service. Responsible for maintaining retail operations, customer service, replenishment, and recovery standards in the store. About Landmark Group Started in 1973 with a single store in Bahrain. Grown into a global retail and hospitality group. Creator of 25+ value-led brands across retail, hospitality, food, and leisure. Over 2200 retail stores, leisure, and hospitality outlets across GCC, Middle East, India, Southeast Asia, and Africa. About Shoemart Largest retailer of footwear and accessories in the Middle East. Over 160 stores. 34-year history. Shoexpress: Leading value footwear brand (50 stores). Pablosky: Premium kids' footwear brand (9 stores). Preferred footwear business for 150+ vendors. Extensive range of in-house and international brands.
Major Acct Manager - BFSI
Fortinet
Job Description Major Accounts Manager - BFSI From the start, the Fortinet vision has been to deliver broad, truly integrated, high-performance security across the IT infrastructure. Today, we secure the largest enterprises, service providers and government organizations around the world. As a Major Account Manager, you will play an integral role in new business pitches, hold responsibility for the effective on-boarding of new clients and focus on growing and developing existing customers. This exciting role will allow you to use your extensive network to run and grow opportunities, write business and account plans for all current and new business tender opportunities and act as the key interface between the customer and all relevant divisions. Responsibilities Develop strong relationships with customer contacts at executive level to gain insight into the business imperatives and IT drivers enabling development of a strategy to sell Fortinet solutions. Sell the organization’s products and services through a consultative selling approach and maintains good relationships with key individuals within the accounts. Develop strategies and orchestrates company resources to maximize sales volume within assigned accounts. Effectively engage and build cooperative relationships with sales resources: System Engineers, Channel, Specialist sales team and executives as required. Drive quarterly business reviews with customer to ensure value from Fortinet investment. Ensures customer is kept up to date with Fortinet product roadmaps enabling the customer to plan and mitigate risks to their business. Requirements Have proven track record of enterprise sales experience in hi tech industry (preferably Cyber Security and Networking) developing strategic customer relationships. Are a highly motivated self-starter with a competitive personality, strong attention to detail and a hunger to win. Have exceptional communication skills, both oral and written, coupled with excellent listening skills. Benefits Excellent training and development opportunities, providing you with all the tools you need to be successful. An open working environment, sharing knowledge and information collaboratively and transparently, with respect to everyone’s thoughts and opinions. The opportunity to be part of an innovative, collaborative and winning team. A market competitive salary package, including stock awards, and opportunity to over-achieve Why Join Us At Fortinet, we embrace diversity and inclusivity. We encourage applications from diverse backgrounds and identities. Explore our welcoming work environment designed for a rewarding career journey with an attractive Total Rewards package to support you with your overall health and financial well-being. Join us in bringing solutions that make a meaningful and lasting impact to our 660,000+ customers around the globe. We will only notify shortlisted candidates.