Senior Python Engineer - AI Coding Agent Evaluation (Freelance)
MindriftJob Description
About Mindrift
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
What This Opportunity Involves
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. You'll create challenging tasks and evaluation criteria within realistic simulated environments.
- Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
- Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
- Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
- Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust
What This Is NOT
- Not data labeling
- Not prompt engineering
- Not writing code from scratch - the agent writes most of the code; you guide and evaluate
What We Look For
- 8+ years in software development
- Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
- Experience writing tests (functional, integration)
- English proficiency - B2+
Why This Is Hard
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.
How It Works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Effort Estimate
Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Compensation
Up to $200/hr equivalent, depending on level and pace. Tasks are estimated at ~30 hours each; you set your own schedule.
How to Apply
Please submit your CV in English and indicate your level of English proficiency.
Get notified of similar jobs
We'll send you an email when jobs similar to "Senior Python Engineer - AI Coding Agent Evaluation (Freelance)" are posted.
Related Jobs You Might Like
View all jobs →
License Owner / Operator, Bahrain
Stranger Soccer
Build the Future of Football in BahrainWhat if your passion for football could become something you lead, grow, and make your own? Stranger Soccer is expanding into new cities around the world and is looking for a License Owner / Operator to bring the Stranger Soccer experience to Bahrain.This isn't a traditional management position. It's an opportunity to build, own, and grow a local football operation while becoming part of an established international brand. We're looking for entrepreneurial individuals who are passionate about football and driven to create a meaningful impact in their local community.Imagine Being the Person Who...Brings football players together through a seamless mobile platformBuilds a vibrant football community in BahrainCreates memorable playing experiences week after weekLeads a growing local operation backed by proven technology and global expertiseWhat You'll DoAs the License Owner / Operator, you'll oversee every aspect of Stranger Soccer, including:Growing the local player communitySecuring football venues and partnershipsRecruiting and managing Official HostsLeading marketing and community-building initiativesDelivering consistently outstanding game experiencesYou'll have the independence to grow your local operation while receiving ongoing support, technology, and operational guidance from our Singapore headquarters.Why Stranger Soccer?Stranger Soccer makes playing football as simple as booking a gym session. Through our mobile app, players can discover games, reserve a spot, and simply turn up to play.Founded in Singapore, Stranger Soccer now operates in 10+ cities worldwide and has delivered more than 100,000 games. Our proven model is built for growth—and we're ready to bring it to Bahrain.Who We're Looking ForA football enthusiast with strong local insightAn entrepreneurial self-starter who enjoys taking ownershipA leader who can build relationships and grow a local communitySomeone looking for more than a job—someone who wants to create lasting impactReady to Take the Lead?Visit www.strangersoccer.com and explore "Bring Stranger Soccer to Your City" to learn more and express your interest.Help shape the future of football in Bahrain—and build something you'll be proud of....
Sous Chef
Marriott
JOB SUMMARYAccountable for overall success of the daily kitchen operations. Exhibits culinary talents by personally performing tasks while leading the staff and managing all food related functions. Works to continually improve guest and employee satisfaction while maintaining the operating budget. Supervises all kitchen areas to ensure a consistent, high quality product is produced. Responsible for guiding and developing staff including direct reports. Must ensure sanitation and food standards are achieved.CANDIDATE PROFILEEducation and ExperienceHigh school diploma or GED; 4 years experience in the culinary, food and beverage, or related professional area.OR2-year degree from an accredited university in Culinary Arts, Hotel and Restaurant Management, or related major; 2 years experience in the culinary, food and beverage, or related professional area.CORE WORK ACTIVITIESEnsuring Culinary Standards and Responsibilities are MetManages kitchen shift operations and ensures compliance with all Food & Beverage policies, standards and procedures.Estimates daily production needs on a weekly basis and communicates production needs to kitchen personnel daily.Assists Executive Chef with all kitchen operations and preparation.Prepares and cooks foods of all types, either on a regular basis or for special guests or functions.Develops, designs, or creates new applications, ideas, relationships, systems, or products, including artistic contributions.Assists in determining how food should be presented and creates decorative food displays.Maintains purchasing, receiving and food storage standards.Ensures compliance with food handling and sanitation standards.Performs all duties of kitchen managers and employees as necessary.Recognizes superior quality products, presentations and flavor.Ensures compliance with all applicable laws and regulations.Follows proper handling and right temperature of all food products.Operates and maintains all department equipment and reports malfunctions.Checks the quality of raw and cooked food products to ensure that standards are met.Leading Kitchen OperationsSupervises and coordinates activities of cooks and workers engaged in food preparation.Leads shifts while personally preparing food items and executing requests based on required specifications.Utilizes interpersonal and communication skills to lead, influence, and encourage others; advocates sound financial/business decision making; demonstrates honesty/integrity; leads by example.Encourages and builds mutual trust, respect, and cooperation among team members.Serves as a role model to demonstrate appropriate behaviors.Maintains the productivity level of employees.Ensures employees understand expectations and parameters.Establishes and maintains open, collaborative relationships with employees and ensures employees do the same within the team.Ensures property policies are administered fairly and consistently.Communicates performance expectations in accordance with job descriptions for each position.Recognizes success performance and produces desired results.Ensuring Exceptional Customer ServiceProvides services that are above and beyond for customer satisfaction and retention.Manages day-to-day operations, ensuring the quality, standards and meeting the expectations of the customers on a daily basis.Sets a positive example for guest relations.Empowers employees to provide excellent customer service.Interacts with guests to obtain feedback on product quality and service levels.Handles guest problems and complaints.Maintaining Culinary GoalsAchieves and exceeds goals including performance goals, budget goals, team goals, etc.Develops specific goals and plans to prioritize, organize, and accomplish your work.Utilizes the Labor Management System to effectively schedule to business needs.
Sales & Groups Coordinator
Marriott
POSITION SUMMARYThis position reports to the Director of Sales and Distribution. He/she performs daily quality checks, ensures that all tasks are performed to The Ritz-Carlton standards. He/she is involved in scheduling and small projects to enhance Sales & Marketing services. Manage mini hotel inventory and ensure rate and market code accuracy in MARSHA and OPERA / Power of M. Submit reports outlining group performance for past, present, and future groups, focusing on groups past and approaching their cut-off dates. Process billing, room blocking, and other reservation tasks for groups. Ensure optimum service to bookers, guests, meeting planners, and other departments.RESPONSIBILITIESThe primary responsibilities of the role include:Sales & MarketingProvide administrative support for the Sales Executives, Sales Managers, Director of Business Travel, Director of (Segment) Sales, and Director of Sales and Distribution.Filing system must be accessible and accurate at all times.To have a tracing system for follow-up work by the sales personnel you are assigned to.Department’s policies and procedures, training and development manuals, and relevant sales and marketing reports.Minute meetings, transcribe and distribute to all relevant parties.Retrieve and deliver departmental mail.Highest level of phone responsiveness within the department. Phone coverage must be available from 09:00 to 17:00 hours.Assist in the preparation of the hotel’s month – end report and annual budget.Participate in daily Sales Line-up, weekly Sales Meeting, and other business-related meetings and functions.Availability and storage of sales – related collateral.ReservationsEnsures that all reservations are handled according to Ritz Carlton Brand Standards.Handles reservation requests received in an accurate, complete and timely manner.Has full knowledge of FIT reservations and able to handle ERS and Wholesale reservations and assists in these areas when neededEnsures that guest and blocks profiles are linked to the reservation.Ensures 100% accuracy of Market Segmentation in groups.Ensures that all amendments and changes are updated in the system.Notes and follows up on guest preferences and special requests.Monitors group activity with Reservation Manager, Revenue Manager, Director of Sales and Dir of CCS.CommunicationEnsures that all pertinent information is provided to bookers, guests and sales managers and all special requests are communicated to the CCS manager and Front office.Ensures all group requirements are entered completely and accurately.Maintains confidentiality of all guest information.Maintains complete knowledge of upcoming and in-house groups and communicates any necessary information.Communicates status of groups – groups approaching cut-offs and full blocks to Revenue manager & meeting planers.Attends daily line-ups and communicate all challenges and successes with the rest of the team.Promptly, accurately and consistently reports all guest incidents and how they were solved.Attends designated meetings prepared with information related to group reservations.Prepares and hosts weekly group communication meetings with Revenue/Sales/CCS.Delegates groups and tasks among the Group Reservations team.Attends Resume and Internal meetings in relation to events with rooms in the hotel.Coaching and Developing OthersEnsures ongoing training and development of Group Reservation Agents (wherever applicable).OtherPerforms all tasks as detailed in daily task list.Processes visa applications for groups.Produces weekly group tracker in the absence of the Group Reservations Supervisor.Maintains the cleanliness and tidiness of the work area at all times.Participates in related project ...