Simon Roy
I work on robot learning: world models, language-conditioned planning and vision-language reward models.
I am a PhD student at Polytechnique Montréal and Mila, advised by Giovanni Beltrame. I am also co-founder and CTO of Aquilix, where we put computer vision to work on construction and industrial sites.
News
- New first-author preprint on searching satellite image archives with text queries.
- Two preprints with collaborators: LAGO, on latent goal prediction for model-based planning, and VISTA, on scale-aware visual navigation.
- Started my PhD in robot learning at Polytechnique Montréal and Mila.
- First-author preprint revisiting the learning objectives of vision-language reward models.
- Teaching assistant for INF3995, the third-year robotics design project at Polytechnique.
- Co-founded Aquilix, where I am CTO.
Company
Aquilix builds self-contained AI cameras that detect risks and critical events in real time on construction sites, rail yards and remote sites. A unit installs in a few hours and runs over LTE, with no site internet needed.
- Lead the technology end to end: camera systems, on-device vision models and the analytics platform.
- Turn research in computer vision and vision-language models into systems that run unattended in the field.
Research
-
Finding Change in Satellite Archives from Text: How to Combine Before-and-After Images Efficiently
-
Latent Goal Prediction from Language for Model-Based Planning
-
VISTA: Scale-Aware Visual Navigation via Action History Conditioning
-
To Select or not to Select, that is the Question: Distilling Robot Skill Prediction into a Small Ensemble
-
Revisiting the Learning Objectives of Vision-Language Reward Models
Projects
-
Goat Brain
A speech-to-action agent for a mobile robot: a LoRA-tuned LLM with retrieval turns spoken instructions into ROS 2 behavior trees, with a visual interface to follow what the robot is doing.
-
Shepherd
Open-vocabulary semantic mapping for a mobile robot. Segments and captions objects with FastSAM, CLIP and BLIP, then answers natural-language queries over the map.
-
Search-and-rescue robot autonomy
Club Capra, ÉTS
Autonomy stack for a tracked search-and-rescue robot: ROS 2 architecture in Docker, 3D SLAM, exploration, object tracking, arm inverse kinematics and Gazebo simulation. Best in class for search and rescue at EnRicH 2023; competed at ELROB 2024.
-
Entangled Explorer
QuantumÉTS
A 2D puzzle platformer that teaches quantum computing concepts to a general audience.
Experience
-
Computer vision and machine learning engineer
NIO Robotics
Developed computer vision and machine learning methods to identify the paintable areas of a wall for an autonomous wall-painting robot.
-
Quantum machine learning research
Developed fermionic machine learning algorithms and data pipelines in the quantum machine learning group.
-
Autonomy software lead
Led the autonomy stack of a search-and-rescue robot: containerized ROS 2 architecture, SLAM and EKF-based navigation, simulation, and a vision-language pipeline for 3D object mapping. Competed at EnRicH 2023 and ELROB 2024.
-
Robotics software engineer
Built mission-planning tools for the Canadian lunar rover (Django, React), worked on path planning (RRT*, genetic algorithms) and operated the lunar map server and its CI/CD.
-
Co-founder and president
Founded the ÉTS quantum computing club: about 20 active members, workshops for more than 200 students, and hackathon teams ranked in the top 5% at QHack two years in a row.
-
Software developer
Signal processing and generation features for the Skydel GNSS simulator, in C++, Python and MATLAB.
-
Hardware engineering
Teaching
-
Teaching assistant, INF3995
Third-year team design project in robotics. Taught ROS 2, Docker and Linux, introduced course rules on responsible use of AI tools, and ran a rate-limited inference server so every team had fair access to shared models.
-
Teaching assistant, MTI882B
École de technologie supérieure
Algorithmic aspects of quantum computing. Wrote the course notebooks and taught PennyLane.
Education
-
PhD, Robot learning
Polytechnique Montréal and Mila GPA 4.0/4.0
Advised by Giovanni Beltrame. World models, language-conditioned planning and vision-language reward models.
-
M.Sc., Artificial intelligence
Polytechnique Montréal and Mila GPA 4.0/4.0
-
B.Eng., Software engineering
École de technologie supérieure (ÉTS) Graduated with honours
Distinctions
| 2024 | Honorable mention from NVIDIA, ETH Zurich Quantum Hackathon |
| 2024 | Competed with Club Capra, ELROB, European Land Robot Trial |
| 2023 | Best in class, search and rescue, EnRicH, European Robotics Hackathon |
| 2023–24 | Top 5% of teams, two years running, Xanadu QHack |
| 2022 | 3rd place, BIG Quantum Hackathon |
| 2020 | 1st place, Halifax, NASA Space Apps Challenge |
Contact
Email is the best way to reach me about research, teaching or Aquilix.