Senior Python Engineer - AI Coding Agent Evaluation (Freelance)
MindriftJob Description
About Mindrift
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.
What This Opportunity Involves
We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. You'll create challenging tasks and evaluation criteria within realistic simulated environments.
- Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
- Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent
- Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
- Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust
What This Is NOT
- Not data labeling
- Not prompt engineering
- Not writing code from scratch - the agent writes most of the code; you guide and evaluate
What We Look For
- 8+ years in software development
- Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
- Experience writing tests (functional, integration)
- English proficiency - B2+
Why This Is Hard
Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.
How It Works
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid
Effort Estimate
Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
Compensation
Up to $200/hr equivalent, depending on level and pace. Tasks are estimated at ~30 hours each; you set your own schedule.
How to Apply
Please submit your CV in English and indicate your level of English proficiency.
Get notified of similar jobs
We'll send you an email when jobs similar to "Senior Python Engineer - AI Coding Agent Evaluation (Freelance)" are posted.
Related Jobs You Might Like
View all jobs →
Sous Chef
Marriott
JOB SUMMARYAccountable for overall success of the daily kitchen operations. Exhibits culinary talents by personally performing tasks while leading the staff and managing all food related functions. Works to continually improve guest and employee satisfaction while maintaining the operating budget. Supervises all kitchen areas to ensure a consistent, high quality product is produced. Responsible for guiding and developing staff including direct reports. Must ensure sanitation and food standards are achieved.CANDIDATE PROFILEEducation and ExperienceHigh school diploma or GED; 4 years experience in the culinary, food and beverage, or related professional area.OR2-year degree from an accredited university in Culinary Arts, Hotel and Restaurant Management, or related major; 2 years experience in the culinary, food and beverage, or related professional area.CORE WORK ACTIVITIESEnsuring Culinary Standards and Responsibilities are MetManages kitchen shift operations and ensures compliance with all Food & Beverage policies, standards and procedures.Estimates daily production needs on a weekly basis and communicates production needs to kitchen personnel daily.Assists Executive Chef with all kitchen operations and preparation.Prepares and cooks foods of all types, either on a regular basis or for special guests or functions.Develops, designs, or creates new applications, ideas, relationships, systems, or products, including artistic contributions.Assists in determining how food should be presented and creates decorative food displays.Maintains purchasing, receiving and food storage standards.Ensures compliance with food handling and sanitation standards.Performs all duties of kitchen managers and employees as necessary.Recognizes superior quality products, presentations and flavor.Ensures compliance with all applicable laws and regulations.Follows proper handling and right temperature of all food products.Operates and maintains all department equipment and reports malfunctions.Checks the quality of raw and cooked food products to ensure that standards are met.Leading Kitchen OperationsSupervises and coordinates activities of cooks and workers engaged in food preparation.Leads shifts while personally preparing food items and executing requests based on required specifications.Utilizes interpersonal and communication skills to lead, influence, and encourage others; advocates sound financial/business decision making; demonstrates honesty/integrity; leads by example.Encourages and builds mutual trust, respect, and cooperation among team members.Serves as a role model to demonstrate appropriate behaviors.Maintains the productivity level of employees.Ensures employees understand expectations and parameters.Establishes and maintains open, collaborative relationships with employees and ensures employees do the same within the team.Ensures property policies are administered fairly and consistently.Communicates performance expectations in accordance with job descriptions for each position.Recognizes success performance and produces desired results.Ensuring Exceptional Customer ServiceProvides services that are above and beyond for customer satisfaction and retention.Manages day-to-day operations, ensuring the quality, standards and meeting the expectations of the customers on a daily basis.Sets a positive example for guest relations.Empowers employees to provide excellent customer service.Interacts with guests to obtain feedback on product quality and service levels.Handles guest problems and complaints.Maintaining Culinary GoalsAchieves and exceeds goals including performance goals, budget goals, team goals, etc.Develops specific goals and plans to prioritize, organize, and accomplish your work.Utilizes the Labor Management System to effectively schedule to business needs.
Sales & Groups Coordinator
Marriott
POSITION SUMMARYThis position reports to the Director of Sales and Distribution. He/she performs daily quality checks, ensures that all tasks are performed to The Ritz-Carlton standards. He/she is involved in scheduling and small projects to enhance Sales & Marketing services. Manage mini hotel inventory and ensure rate and market code accuracy in MARSHA and OPERA / Power of M. Submit reports outlining group performance for past, present, and future groups, focusing on groups past and approaching their cut-off dates. Process billing, room blocking, and other reservation tasks for groups. Ensure optimum service to bookers, guests, meeting planners, and other departments.RESPONSIBILITIESThe primary responsibilities of the role include:Sales & MarketingProvide administrative support for the Sales Executives, Sales Managers, Director of Business Travel, Director of (Segment) Sales, and Director of Sales and Distribution.Filing system must be accessible and accurate at all times.To have a tracing system for follow-up work by the sales personnel you are assigned to.Department’s policies and procedures, training and development manuals, and relevant sales and marketing reports.Minute meetings, transcribe and distribute to all relevant parties.Retrieve and deliver departmental mail.Highest level of phone responsiveness within the department. Phone coverage must be available from 09:00 to 17:00 hours.Assist in the preparation of the hotel’s month – end report and annual budget.Participate in daily Sales Line-up, weekly Sales Meeting, and other business-related meetings and functions.Availability and storage of sales – related collateral.ReservationsEnsures that all reservations are handled according to Ritz Carlton Brand Standards.Handles reservation requests received in an accurate, complete and timely manner.Has full knowledge of FIT reservations and able to handle ERS and Wholesale reservations and assists in these areas when neededEnsures that guest and blocks profiles are linked to the reservation.Ensures 100% accuracy of Market Segmentation in groups.Ensures that all amendments and changes are updated in the system.Notes and follows up on guest preferences and special requests.Monitors group activity with Reservation Manager, Revenue Manager, Director of Sales and Dir of CCS.CommunicationEnsures that all pertinent information is provided to bookers, guests and sales managers and all special requests are communicated to the CCS manager and Front office.Ensures all group requirements are entered completely and accurately.Maintains confidentiality of all guest information.Maintains complete knowledge of upcoming and in-house groups and communicates any necessary information.Communicates status of groups – groups approaching cut-offs and full blocks to Revenue manager & meeting planers.Attends daily line-ups and communicate all challenges and successes with the rest of the team.Promptly, accurately and consistently reports all guest incidents and how they were solved.Attends designated meetings prepared with information related to group reservations.Prepares and hosts weekly group communication meetings with Revenue/Sales/CCS.Delegates groups and tasks among the Group Reservations team.Attends Resume and Internal meetings in relation to events with rooms in the hotel.Coaching and Developing OthersEnsures ongoing training and development of Group Reservation Agents (wherever applicable).OtherPerforms all tasks as detailed in daily task list.Processes visa applications for groups.Produces weekly group tracker in the absence of the Group Reservations Supervisor.Maintains the cleanliness and tidiness of the work area at all times.Participates in related project ...
Diagnostic Technician I Motor City Service Center I Mameer
Ebrahim K. Kanoo
Job SummaryWe, at Ebrahim K. Kanoo are looking for a Diagnostic Technician for our Motor City Service Center. You will be responsible for accurately diagnosing electrical, electronic, mechanical, and vehicle system-related faults using advanced diagnostic equipment and technical knowledge.Key ResponsibilitiesPerform comprehensive diagnostics on vehicles to identify electrical, electronic, mechanical, and system-related faults.Use manufacturer-specific and advanced diagnostic tools and equipment to retrieve and interpret fault codes and system data.Conduct systematic troubleshooting to identify the root cause of reported vehicle issues.Inspect vehicles and carry out diagnostic tests based on customer complaints and technician/service advisor reports.Diagnose issues relating to engine management, transmission, ABS, SRS/airbags, air conditioning, body control systems, electrical systems, and other vehicle electronic systems.Interpret wiring diagrams, technical information, fault codes, and diagnostic data.Carry out electrical testing using appropriate diagnostic and testing equipment.Determine whether a fault is related to a component, wiring, sensor, control module, or other vehicle system.Provide clear technical recommendations regarding the required repairs or further investigations.Work closely with service advisors and workshop team leader to communicate diagnostic findings and repair requirements.Support technicians during complex repairs and provide technical guidance where required.Verify completed repairs by conducting appropriate diagnostic checks and vehicle testing.Ensure that diagnosed faults are accurately documented in the relevant workshop/job card system.Escalate complex or unresolved technical issues to the workshop manager or relevant technical support team.Keep up to date with new vehicle technologies, diagnostic systems, technical bulletins, and manufacturer repair procedures.Ensure all diagnostic activities are carried out in accordance with manufacturer standards and workshop procedures.Maintain diagnostic equipment and ensure tools are handled correctly and kept in good working condition.Follow all company health, safety, environmental, and workshop procedures.Required Skills & ExperienceStrong experience in vehicle electrical and electronic diagnostics.Hands-on automotive background with the ability to interpret diagnostic results and identify the root cause of vehicle issues.Ability to recommend appropriate repair solutions and work closely with technicians to ensure repairs are completed accurately and efficiently.Good understanding of modern vehicle electronic systems and control modules.Ability to read and interpret automotive wiring diagrams and technical information.Strong troubleshooting skills.Attention to detail.Good communication skills.Ability to work in a fast-paced workshop environment.Minimum 3-4 years of automotive diagnostic experience, preferably within a dealership or professional automotive workshop environment.QualificationsDiploma, certificate, or equivalent technical qualification in Automotive Engineering, Automotive Technology, Vehicle Mechanics, or a related field.