Job Description
What this opportunity involves
- Create structured test cases that simulate complex human workflows
- Define gold-standard behavior and scoring logic to evaluate agent actions
- Analyze agent logs, failure modes, and decision paths
- Work with code repositories and test frameworks to validate your scenarios
- Iterate on prompts, instructions, and test cases to improve clarity and difficulty
- Ensure that scenarios are production-ready, easy to run, and reusable
What we look for
- 3+ of software development experience with strong Python focus
- Experience with Git and code repositories
- Comfortable with structured formats like JSON/YAML for scenario description
- Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design
- Familiarity with Docker
- English proficiency - B2
How it works
- Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Project time expectations
Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Payment
Paid contributions, with rates up to $40/hour*
Fixed project rate or individual rates, depending on the project
Some projects include incentive payments
*Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
Get notified of similar jobs
We'll send you an email when jobs similar to "Freelance Agent Evaluation Engineer" are posted.
Related Jobs You Might Like
View all jobs →Sr. Planning Engineer
ACWA Power
JOB PURPOSE:To ensure the effective implementation of efficient Work and Outage management process /execution in each cluster business. Drive the continuous improvement and optimization of the respective business processes together with cost-driven value creation (inventory, maintenance digitalization,.) to position the fleet within the top quartile globally over the project lifecycle.KEY ACCOUNTABILITIES:Plant performance:Monitor and report the cluster KPIs on a monthly, quarterly, and yearly basis, highlighting deviation and record reasons for variance. Prepare an action plan to close the gaps, in close coordination with each site, to ensure timely corrective action with all concerned Cluster stakeholders.Ensure Cluster business monthly MIS report is prepared, circulated to all stakeholders highlighting the achievements of KPI targets.Ensure Audits observations related to work Planning and process within a cluster are properly addressed and closure in time to improve plant performance and customer satisfactionMaintenance Planning Outage Management:Ensure an effective implementation of the Maintenance Strategy including Critical assets PM alignment with EPRI, SCD management and SPOF Mitigation Planning corporate guideline.Ensure the corporate Maintenance Planning and Scheduling procedure, computerized maintenance management processes are effectively implemented to a prudent maintenance planning regime.Drive the continuous improvement process of Maintenance Procedures/ programs to ensure high level of reliability and resource efficiency within the business.To ensure an effective Planned Maintenance Optimization program for each of the assets within the cluster, to ensure resources utilization based on Criticality and improve wrench time.Carryout analysis of similar assets PM efforts within a cluster and calibrate with other clusters to ensure standardization across the fleets aligning with corporate guideline.Carryout periodic Cluster assessment to ensure an effective maintenance program implementation, execution, and closure with quality records.Ensure SPOF & Critical equipment Spares, BOM availability and support cluster business in Spares replenishment process coordinating with business and SCM.Ensure that Corporate Outage Management Procedure is effectively implemented across the cluster complying KPI at each outage phase.Ensure close coordination with NMES to align the outage schedule and optimize resources loading within a cluster (as applicable).Ensure availability of critical spares and right sufficient resources in place and on time as per outage eventCoordinate with the cluster engineering team to ensure Outage TILS, insurance recommendation, ROS enhancement outage related jobs are planned adequately.Ensure post outage lesson learnt captured with closure action plan KPIs are achieved.Conduct a session with cluster Finance before and after outage for outage cost variances analyses, improvement action plan and implementation for the next outage (as applicable)New Build /Project Mobilization:Ensure effective implementation of Project Mobilization and takeover procedure in the cluster business as applicable for new projects.Ensure updated mobilization plans within the cluster with its regular follow up, reporting of major deviations from agreed plan and mitigation.Work with Cluster management and new project site teams in the development of lifecycle maintenance planning (in accordance with OEM recommendation and EPRI PMBD).Support respective cluster plant in preparation of master mobilization plan achieving O&M readiness before COD/PCOD.Support business in the preparation of CMMS master data aligning with the standard including criticality ranking, SCD, SPOF identification as per corporate procedure.Ensure Mobilization actions are compiled in a given time frame.Coordinate with Engineering team to ensure Technical Risk Register review and action plans are developed and implemented.
Assistant Purchasing Manager
Hilton
Exceptional Hospitality Starts with YouPicture yourself brightening someone’s day. When you join our Hotels team, that’s exactly what you’ll do every time you come to work! As a Purchasing/Receiving Assistant Manager, you’re not just overseeing the purchasing process – you’re spreading the light and warmth of hospitality by delivering memorable experiences that make the stay for every guest.Join an Award-Winning Workplace CultureAt Hilton, we don’t just deliver exceptional experiences for our guests—we build an exceptional workplace for the Team Members who make it all possible. As a global leader in hospitality, we’ve welcomed more than 3 billion guests worldwide, all while staying true to our founding vision: to fill the earth with the light and warmth of hospitality.Our award-winning culture has earned us repeated recognition on the World’s Best Workplaces list by Great Place to Work and Fortune. With our suite of world-class brands https://www.hilton.com/en/brands/ and a company-wide commitment to providing the best stay for every guest, we’re setting new standards for the future of travel.Whether you’re starting your career or exploring something new, Hilton supports your journey every step of the way. Come for the job, stay for the career—and help us make every stay a little more magical.Curious about life at Hilton? Explore our Careers Blog to see why we’re more than a great place to stay—we’re a great place to work.
HSE Engineer - Omanization
Hill International
Job SummaryThe HSE Engineer is responsible for implementing and monitoring health, safety, and environmental policies and procedures, ensuring compliance with regulations and project requirements, conducting inspections and risk assessments, and promoting a safe working environment across the project.About Hill InternationalHill International, with more than 4,300+ professionals in over 100 offices worldwide, provides program management, project management, construction management, project management oversight, advisory, facilities management, and other consulting services to clients in a variety of market sectors. Engineering News-Record magazine recently ranked Hill as one of the largest program management firms in the world. For more information on Hill, please visit our website at www.hillintl.com.Hill is exclusively a program, project, and construction management provider, meaning we have one focus as a company: protecting client interests. Free of any potential conflicts, our entire business is geared towards helping clients achieve their desired outcomes. We are dedicated to exceeding expectations throughout the entire construction project lifecycle and adapt to the needs of each assignment to develop tailored approaches and solutions to meet those needs.Hill is a GISI Consulting Group company, part of the Global Infrastructure Solutions, Inc. (GISI) family of companies. Learn more about GISI Consulting Group at www.gisiconsulting.com.Equal Opportunity EmployerHill International is an Equal Opportunity Employer/Veteran/DisabledNote: This job description is intended to provide an indication of the nature and levels of work, knowledge, skills, abilities and other essential functions to be carried out as part of fulfilling the role mentioned in the job title. Various other duties as required....